COVID-19 – Is Disaster Recover now Disaster Response?
As business leaders, ask yourself if you’ve done your system back-ups recently? If not, do you really believe that a “normal” disaster is less likely now than at any other time? If you have, do you currently have a team available, with the right skills, to be able to respond to that disaster?
We have to recognise that the COVID-19 outbreak is a disaster to many businesses. I think we also have to acknowledge that it’s not the sort of disaster that was on many of our “likely” scenarios. Also, as it’s happened once, then we must assume that it can happen again and consequently we need to think about how we should prepare ourselves to respond in the future.
Traditional DR responses
Ever since computers started being an essential part of our business processes we have been considering “what if” scenarios. What if it breaks? how do I keep it running? etc. Our industry has matured throughout its lifetime to deal with the changing threats.
We all know the importance of backing up our data. Everyone has experienced that dreaded moment when a Word or Excel document is lost; a few of us have experienced a time when a complete system goes down without back-up. Both events are frustrating and embarrassing and both can have severe consequences. We’ve learnt from this, both personally and professionally, and even then we still make mistakes.
Having a back-up is great but what if the equipment itself breaks? Again, we’ve learnt to cope with this; RAID (redundant drives) are prevalent in all servers and we acknowledge the need to have spare equipment available (if not onsite then from our DR providers). We might even test this provision on a regular basis. Have you?
High Availability Equipment and Data Centres
The next step to having back-up equipment was considered to be segregated data centres. That way if a disaster hit one of the datacentres, the other could still run and still allow your business to continue. The most advanced version of this was the High Availabilty Data Centre, a data centre that so efficiently replicated your operations, users didn’t know whether they were using the Primary or Secondary resource. We used to consider this to be the ultimate in DR protection; after all how could something affect two data centres tens of miles apart?
Segregated Data Centres
Then came ransomware…
We realised that if a high availability system was to be infected with an uncontrolled encryption, then so would the back-up system. The disaster recovery and security industry had some serious thinking to do and some very clever firewalls and AI detection technologies were introduced. HA systems became less “reliable” and some had a deliberate replication delay introduced, to make sure that both databases couldn’t be impacted by the same event.
DR Office Locations
Some of the larger and more risk averse companies had taken steps to allow their teams to access different office locations should the office itself be inaccessible, and had arrangements to move critical teams (e.g. call centres) to different locations. Companies like this also had policies to issue laptops rather than desktops to teams to allow them to work from home, but this was really seen to either be a perk, or to allow a lot of working travel, or – in some circumstances – to enable staff to work longer or more flexible hours than would otherwise be possible within the office.
All of these scenarios were based on reacting to a physical and timebound event; to be able to continue operating if a piece of equipment or a site was inaccessible or unavailable. Today’s disaster has changed that ‘disaster’ to focus on people. Now people simply aren’t allowed to get together, going to the office is actively discouraged and people need to be able to work productively from home.
Over the last few weeks there has been a massive rush to build a homeworking infrastructure; Amazon has been reporting shortages of monitors and webcams. IT infrastructure companies have been building and licencing VPN’s and Disney/Netflix have been downgrading their video streams. This has increased the load on the IT teams, increased the dependency on them and forced them to come up with innovative ways to deliver a reasonable service to their users. Many businesses are in a situation that they never expected and a situation where their infrastructure simply cannot reasonably cope.
So, what next?
I believe that in these circumstances we need to fully embrace cloud computing. Let’s get rid of our dependency on internal infrastructure. Why do we need access to a fileserver when we can use Sharepoint online? Why do we need to have our own email (exchange) server? Do we really need high powered workstations when we can remotely control cloud “PC’s” “on demand”? Do you really need to install applications on servers based “on premise”?
Under these circumstances we reduce the dependency on our IT teams if we move most of these services into cloud applications. If we do this, we can remove the need for physical back-ups (outsourcing that role to the cloud infrastructure provider). We can obtain a better service with SAAS products, as they are available whenever we want them, without worrying about back-up windows. We can access our documents and work from wherever we are. We can collaborate.
Our IT teams no longer need to get to a single location, the administration of servers can be reduced, capital expenditure significantly reduced and the dependency of critical resources to “keep the business running” is reduced. Simply put then, if the team get ill then its Microsoft’s (or other providers) responsibility to service the computers, do the back-ups, ensure email is available, protect your documents, etc, etc.
Clearly this isn’t possible for everything; some applications control and respond to physical devices and therefore must be on site, however, if we can reduce the number of these critical applications, then the support arrangements are much reduced and our critical IT teams can focus on the infrastructure that really drives the business.
From a user viewpoint, what can be automated? Can we make our decision-making more exception based, enabling only the most important decisions to be actioned by people? Can we put escalation processes in place so that (in the event of unexpected absence) escalations can occur quickly?
How can we help?
At HSO we have already embraced this technology. We don’t need a VPN for the majority of our users, email is served from the cloud, our documents are available from SharePoint online and our ERP is delivered from the cloud. Our users can work from anywhere. We can help you achieve the same.
Our Solution Architects can help you make the right choices to get your business to that same place. Let us help you understand your infrastructure and come up with a plan to make your business resilient to the challenges in this new world. Let us help you develop the ability to flexibly respond to the next disaster. We use Microsoft technologies to deliver a global service for our commodity technology (email, file servers, collaboration); we then leverage our skills to package this up in a way that provides you with business specific expertise and help.
We look forward to working with you to build a more flexible, resilient business.
Enterprise Architect, HSO