The business world has changed dramatically in the past few years: leaner and restructured organizations, new technology and business relationships, increased performance and reliability expectations of customers and the investment community all demand greater resiliency of your organization. Continuity of operations is integral to all that you do and the challenges to maintaining that continuity are growing on a daily basis. Planning and ability to perform are paramount, but is your business continuity plan (BCP) sufficient, and can you respond to a disaster in a manner that protects the organization?
Each year, disasters such as fires, hurricanes and earthquakes present business continuity challenges that have the potential to cripple companies in all industries. It is likely that at some point, your business will suffer a crisis situation that will bring your computer systems and important operations to a stop. What is your plan to minimize your downtime and help ensure that your operations are back up and running in a timely fashion?
Regulations and requirements for business resilience continue to evolve, but the world around you is changing even faster. Information is increasingly becoming a valued commodity, and if its flow is interrupted, your company is losing revenue. Lead times have shrunk, technical recovery capabilities have improved and competition is fiercer than ever; therefore, continuity expectations have increased substantially. If you encounter a disaster, your customers and suppliers will demand that your operations and services continue, or they will be forced to take their business elsewhere.
Generally speaking, regulations do not specify how quickly a business must recover. However, companies are now required to not only maintain a recovery plan, but to test it regularly to demonstrate that it works. Testing results are being increasingly scrutinized by regulators, customers and even investors, and this trend will likely continue in the future. Organizations often have difficulty balancing what is demanded from a regulatory aspect and the level of efficiency that customers and stakeholders have come to expect.
Requirements and expectations
In the event of a business disruption, it is imperative that you are prepared to be up and running more quickly than in the past because the impact of potential downtime is greater. You have regulatory requirements you must abide by, but the expectations of your customers, stakeholders and employees hold you to an even higher standard. Adhering strictly to the continuity standards of your industry is likely not enough.
Business is moving at a quicker pace than ever, and chances are you face greater competition than you did just a few years ago. In a dynamic and evolving marketplace, your competitors are likely to court your customers, and your customers are more likely to look for other options. Even though customers and vendors may prefer to work with an established business partner, even the most loyal relationship can be jeopardized if one party cannot deliver on its commitments. The same holds true for employees—if you can’t meet your obligations to them, they will seek other options.
Your customers have expectations of your business and often have tight timelines that can easily become disrupted due to any amount of downtime. Just-in-time inventory is an example that demonstrates that if you are even a day late with a customer shipment, the client may be required to shut down its manufacturing line and not meet its requirements. There are many instances in which operating windows have become tighter, expectations have risen and companies don’t have the timing flexibility they used to have.
Large-scale disasters such as Hurricane Katrina and the blackout in the Northeast have presented challenges toeven the strongest BCPs, and have served as a reminder to businesses that were not directly affected that these types of events can affect anyone. Situations have arisen that have been almost impossible to plan for, but lessons can be learned and controls can be implemented to prepare your business in case it is ever involved in a similar crisis situation. Sometimes the unthinkable does happen, so your business must be proactive to plan for such a scenario. More often than not, there are several common issues that deter an organization’s recovery and your plan must allow you to overcome these difficulties before they pose a problem.
Common BCP issues and concerns
There are several prevalent obstacles that can have a negative impact on your BCP following a disaster. Many of these aspects can be overlooked or forgotten in the planning process, but each could have an adverse effect on the efficiency of your organization’s recovery efforts. Fortunately, each of these hindrances can be mitigated with proper plan design, foresight and communication, allowing you to continue operations and tend to the needs of your customers.
Outdated plans
Often, a business has built a continuity plan around strategies to meet goals that were established a number of years ago. Those same strategies may not meet today’s needs. That plan may have included a contracted hot site, and the contract has been renewed every year or two without reevaluating whether this strategy still utilizes the technologies, the capacity or even the recovery window you need. Your system requirements have likely changed over the years while your window for recovery has gotten smaller. You may need more workspace as your physical location may no longer be appropriate. If you are just renewing contracts and conducting the same tests on a yearly basis, you may not realize that the plan has become obsolete.
Uncertain roles within the plan
Another issue that businesses encounter is defining who is involved in the BCP process as well as assigning specific roles and responsibilities. It is not unusual for a business to designate someone in IT to be responsible for the entire BCP process. In this scenario, an IT professional designs a sufficient plan from a technology perspective, but they may have limited knowledge of how the business operates. Management needs to be involved in the process, but since every organization and culture is different, there is no universal right answer regarding who should own the BCP process. Some companies designate the process under insurance risk management, some under IT and some under the CFO. While the reporting relationship can vary depending on the goals and strategy of the company, it is always important to ensure the BCP program is positioned to command attention and have visibility throughout the enterprise.
Having an executive sponsor and building a steering committee are critical to defining and achieving the vision of your BCP program. The steering committee will provide strategic direction, define priorities and establish an awareness program that helps ensure the company’s employees are knowledgeable of the BCP, and prepared to respond at the time of a disaster.
Testing
When it comes to BCP testing, there are several different types of exercises that you can perform to validate and improve your BCP program. The key is your testing program must evolve to meet the changing objectives and requirements of your BCP.
The most common type of test is a tabletop or a walk-through test, where an organization already has a plan in place and each department periodically comes together to talk through a simulated disaster, such as a tornado or earthquake. Contingency plans such as communication strategies and an evolving timeline of planned action steps should be shared. Tabletop or walk-through tests should be performed against a variety of scenarios, with varying impacts and outage durations, to exercise and validate various aspects of your BCP.
Another type of exercise is a simulation test, which involves a more hands-on and realistic situation. In this test, operations are migrated to a hot site or other alternate location where recovery activities are performed by members of different departments. Participants will physically travel to the site to make sure that they can get their job done and perform necessary processes effectively.
A checklist test is a non-invasive test that involves confirming that required documentation, supplies and other resources are in place and available as designated in the BCP. This type of testing is fairly simple, but it can be effective in ensuring that designated resources are where they should be, and employees are aware of how and where to obtain these items.
If you are not bringing new variables into your testing processes, or if you are using invalid methods and scenarios, you may overlook red flags that could indicate trouble. However, performing valid testing is only one part of the equation. Your business must evaluate what information the testing yields and incorporate the results into your BCP to provide proper protection in the event of a disaster.
In some cases, organizations simply restore their backup tapes and believe they have completed an effective disaster recovery test. However, this may fail to simulate a realistic scenario and often does not achieve particular objectives or goals. For instance, a valid test should consider how quickly the alternate equipment can be mobilized, which backups would be available to be used for the restoration and how data would be brought current and synchronized. These factors should be considered as your organization plans its testing and defines its BCP goals.
These various types of tests should be considered by your organization, and you should be sure to challenge your employees and your plan by varying the types of testing performed each year. There should be many components of your testing, as there are many variations in the way that a disaster can unfold in terms of timing and what could be affected. If you are repeatedly testing the same processes or components of your plan, you could leave yourself vulnerable.
Supply chain continuity
Even if you don’t experience a disaster directly, your business can be severely affected by disruptions to your supply chain. Similar to how your customers depend on you, you likely have little slack time where you can wait to receive critical products or services. Although it’s critical to confirm that your vendors are prepared for disasters, you also need to identify contingencies that would allow you to continue operating even if one or more third parties experience a disruption. In some cases, such contingencies may simply mean having secondary and tertiary suppliers for a given product. However, in many cases, such solutions are not so simple. To ensure that you can continue your operations despite a disruption at one or more of your vendors, your BCP program should include an analysis of your external dependencies, and you should define strategies that minimize your exposure to failures in your supply chain.
Differences between high availability and disaster recovery
Often, businesses rely on their high availability architecture as the foundation for their BCP. This generally means that if a server is disabled, another server immediately and automatically assumes the role of the disabled device. While this strategy can greatly increase the availability of the related systems and data, it may not adequately protect you from viruses, data corruption or even natural disasters that affect both systems simultaneously.
For this reason, it is important that you consider your needs for both high availability and true disaster recovery, and implement solutions that allow you to achieve both objectives. Beinghighly available involves planning for glitches and having a level of redundancy built in. However, the dollars you invest in high availability solutions do not always help you recover from a disaster. It will not always be possible to implement one solution that meets both needs to the degree you may prefer, but there are options that can provide an acceptable level of both high availability and recoverability.
Service-provider recovery capabilities
Since many businesses outsource major computer systems, they may not address these systems within their BCP. However, since these systems may be vital to your business operations, you could be severely affected if your service provider experiences a disaster. Although companies typically verify that their service providers have disaster recovery plans, many organizations fail to confirm these plans meet their own disaster recovery objectives and windows. You need to know the details of your vendors’ recovery plans, and determine how long it will take them to recover if they experience a disaster. You also need to understand how current your data will be following a recovery, and you should be aware of any restrictions that you can expect if a vendor does activate its disaster recovery plan. If a vendor would be using backup tapes from a previous day (or beyond), or if the restored systems may offer incomplete functionality or capacity, their recovery capabilities may not be sufficient to meet your needs. If you outsource even a portion of your computer systems or your business operations, it is critical to understand your vendors’ recovery plans and capabilities and to confirm that they properly align with your recovery strategies and meet your business continuity needs.
BCP capability improvements
Now for some good news: while requirements have become more demanding, advances in technology and better testing methodologies have made it easier for businesses to implement a robust BCP that meets even the smallest recovery windows. Solutions that may not have been an option in the past are now more affordable, available and reliable. While these enhanced recovery capabilities only further increase BCP expectations and requirements, they also provide the tools and options you need to meet such demands.
BCP software
If your business has not implemented a BCP software package, this may be an investment you should look into. Specialized BCP software allows you to maximize the effciency of your BCP efforts by more effectively managing and maintaining your documentation. Today’s software packages can streamline the BCP development, maintenance and distribution functions, allowing you to spend less time tending to administrative matters, and more time improving your BCP.
Affordable hardware and communication
Businesses now have much more affordable hardware options to establish their own internal recovery capabilities, even by simply upgrading existing hardware and moving old hardware to a new location to use as a backup. Communication has also become much more affordable, with many more options and increased availability. In the past, you had a primary data center, and you could often only afford to connect your employees to that site. If you had an alternate data center, you needed twice as many data connections, which became costly. Now, dynamic options are available, which allow all sites to “talk” with each other in a cohesive, affordable manner. You no longer need the separate pipes that were previously required to connect with each individual data center, and each location has the ability to communicate within a singular communication cloud. Without a prohibitive investment, these technologies give you the ability to seamlessly transfer your data center from one site to another with little or no telecommunications changes.
Virtualization
The latest virtualization technologies greatly reduce the complexity of recovering large collections of IT servers. In the past, a business that had 100 servers literally had to maintain 100 physical boxes. Restoring those servers was both costly and time-consuming, with steps ranging from installing and maintaining operating systems to restoring data, all having to be repeated for each individual machine. Now, with virtualization, you can still use 100 independent servers, but they can be housed on one box that can be restored as a single component. Virtualization technologies provide substantial cost savings while greatly simplifying and expediting a recovery effort.
Remote computing
Any organization’s BCP must address relocating staff from a disabled facility to one or more alternate work locations. Such arrangements can be expensive and complicated, particularly for companies that have many employees working under one roof. With remote computing technologies, companies can now greatly decrease their alternate worksite arrangements, and instead plan to have large portions of their staff utilize their company laptop, personal computer or other machine to perform their critical duties from home or another remote location. In fact, remote computing is now so reliable and cost-effective that many companies allow (and even promote) this practice as part of their normal day-to-day operations. Thin-client and virtual desktop technologies have reduced the importance of the local storage and processing capabilities, thereby allowing PCs to become practically interchangeable. Even if an office PC has been disabled or is inaccessible, the company can quickly distribute basic laptops that allow employees to access the same systems and data, and perform their normal duties, just like they would at their standard workstation.
Cloud services
The emergence of cloud computing may allow you to push software or services to the cloud that used to be a recovery planning nightmare. This may allow your organization to adequately address system and file availability requirements, making one less issue that you have to worry about in your BCP. However, there are risks associated with cloud services, so your organization should appropriately assess and consider these before adopting this strategy.
Conclusion
A comprehensive BCP that helps to ensure a timely recovery in the midst of a disaster can be difficult to develop and implement, and it is tough to do on your own. Since you work with your BCP processes every day and every year, you might be too close to identify weaknesses or areas for improvement. This white paper may give you ideas for improving your BCP, but to identify process improvements and develop an actionable plan, you may want to consider bringing in an outside third party to either assess your BCP or design a testing plan. A third party can design a scenario that includes different variables that have not been addressed by your recent tests. An outside partner can also help your business understand, either through testing or assessments, where your strengths or opportunities for improvement exist to allow you to optimize your BCP efforts and maximize your recovery capabilities.
