When we think about disaster, we tend to think of the dramatic. In Australia, that’s not unreasonable – we do seem to get more than our share of bushfires and floods. Crises of human origin are common enough too – the recent South Australian power shortages will have made many business leaders in other states nervous.
Beyond the headlines, many smaller problems can have a catastrophic effect, and for the unprepared, even a burst water main or a local power cut can wreak havoc on businesses. Whatever the disaster, the first priority is, of course, the safety of your people. That aside, faced with disruption to your usual operations, the question is how quickly, and how effectively, you can mobilise your workforce and continue to serve your customers.
We commonly encounter an assumption that maintaining operations is simpler in the mobile era. Now that employees are used to working remotely, it is in some respects true. If that cultural shift has been successful, your people will in the most part be comfortable performing their duties from home. That comfort, though, depends on whether they can access the assets, systems and data they need to continue earning revenue – and whether they can do it securely.
Security is one of the big tests. You need to be sure that your security can match current levels if you have to switch over to your Disaster Recovery (DR) environment. Just because you’re fighting fires, it doesn’t mean cyber-criminals will take a break – in fact, given their opportunistic nature, they may feel emboldened by the distraction.
The first, and most important piece of advice, is not new. You’ll have heard us say that your DR plan must be tested – and tested well. The DR plan must be a living document with a schedule of review, because things change, people leave, and new technologies emerge. It is very important to get a fresh set of eyes.
Hint: Put a calendar entry for every 4 months to ‘Review DR Documentation’. This is as important as ‘Meeting with CIO’.
Have you ever had a visitor who saw something about your home that you never noticed? Or maybe a friend learned something about your child that you never realised? It is a matter of familiarity – when we are close to something or someone every day, we often miss things that others notice. It is a part of human programming.
Even though we work on DR plans all the time, we still get someone else to check our own plan. It is simply good practice to get someone external, or from another part of the business, to take a look. They may just see some important details that have been glossed over, or ask questions that nobody else voiced.
When we talk about testing, we mean for a variety of contingencies. We test reaction to component failure, and systemic failure, where certain parts of the infrastructure become unavailable. Added to that, we test how our plan works against environmental failure, where part or all of our premises become unavailable.
Testing, above all, needs to be realistic, which means engaging your people in the process. Some of our customers treat it as a bonding exercise combined with an out-of-hours education session – they get in pizza, talk through how different personnel should react, and try replicating the different types of failure.
Even though people can be resilient, and able to improvise in a crisis, for many that is much easier to do if there is a plan. As with fire drills, if they have practiced, they have a much better chance of surviving the real thing. When people in all departments are involved in DR testing, the chances of success increase considerably.
No DR discussion would be complete without mentioning the as-a-service options that now exist. Team partners with the best Data centre providers who have an ever-increasing number of service options and experience, and take away a lot of the resource and security concerns of creating a fully-prepared DR environment. You no longer have to do everything yourself. We are here to assist you, from the development of your DR plan to being that extra pair of hands when a “downtime incident” occurs.
Having everything managed for you doesn’t take away the need for testing and planning, of course, but it certainly does make life easier – and they can be another tremendous resource when it comes to fine-tuning your plans.
No matter how good your plan is, there will almost certainly be things that have been missed, so be prepared to deal with a few surprises. We are always happy to find you somewhere to put data, a machine to put that backup tape into, someone to help with that legacy application. Our people have skills in a wide variety of systems, capability and capacity available for contingencies.
Much as we could promise to take it off your hands as a massive project, there really is no substitute for in-house commitment. DR is one area where you can’t abdicate responsibility to a contracted partner – we can help, and act as the fresh eyes we mentioned, but your role is vital.
And even more important is that the DR planning and commitment must come from the very top of your organisation. DR must be a ‘culture’ within the organisation. E.g. you have just provisioned a new SQL Server – your very next thought must be ‘how do I provision DR for this new server’.
We’re always happy to chat through your DR challenges, or act as your fresh set of eyes. To learn more, contact our DR specialists at Team Computing.