Disaster strikes when you least expect. Oftentimes it is a matter of when, not if. The recent (March 9, 2021) fire in the Strasbourg, France OVH data center group is one of the latest, and more visible, examples that bad things really do happen.
Don’t get me wrong - this could have just as easily been AWS, Azure or Google Cloud. Yes, cloud services are highly reliable. Until they aren’t...
In this case, all four (4!) separate data centers went offline - one was completely destroyed, another partially, with the remaining two down for an extended period due to a complete power outage in the region as a safety precaution. OVH was forced to send a “We recommend that you activate your Disaster Recovery Plan” message to all affected customers.
This begs the question: Do you have a solid Disaster Recovery (DR) plan in place to activate when the time inevitably comes?
One of our customers was in the middle of this disaster. The good news is, should there be any of these types of cases, the fact is that they had deployed a multi-site, geo-distributed Tungsten Cluster that kept their business-critical operations going.
Even with the Tungsten DR solution in place -- which provided the continuous MySQL operations -- the result of this mayhem was countless additional hours of recovery efforts. They had to deal with the resulting impacts to their operations before they were fully recovered, and without Tungsten Cluster, they could have been out of commission for a long time.
If you are interested in how long it actually took for OVH to recover, please check out this Twitter thread: https://twitter.com/olesovhcom/status/1369478732247932929. Very educational. (As of today, over six weeks later, it is still not fully up and running.)
If you feel you have done everything to be ready for a possible disaster, we salute you!
If not, we are more than happy to review your current deployment, along with any disaster plans you may already have in place, and then recommend possible changes that might be made. We naturally do this Disaster Recovery planning consulting effort free of charge.
For Continuent, that goes without saying, right? At your service, 24/7 within a moment's notice.
Smooth sailing, steady as she goes!
CEO & Founder, Continuent