We recently came across a very unlucky DBA stuck with the SeveralNines ClusterControl solution. His predecessors chose ClusterControl because it was cheaper, supposedly. And closer to open source, they thought.
This DBA's honest and rather brutal comments about his ClusterControl experience so far when we asked about the good, the bad and the ugly were (in his own words):
The Good - None so far, really.
The Bad - Documentation, undocumented things, un-configurable things, support.
The Ugly - SeveralNines sells ClusterControl packaged with open-source products (such as ProxySQL), yet they tell customers that the performance issues on these included open source components are out of scope of support.
Welcome to the wonderful world of open source! You really are on your own, even if you think you bought a solution that works.
Many of the companies offering solutions based on open source components wrapped with expensive consulting engagements really can’t help you as they are not the original developers of some of the important core components, and they do not have the necessary skills to fix them either.
Complete, fully-tested, proven, and 24/7 supported. At the lowest TCO.
We like to say that Continuent has a complete, fully-tested, proven, and 24/7-supported solution for MySQL high-availability, disaster recovery and geo-scale deployments at the lowest TCO.
But what does that actually mean?
Complete — This means that we offer one single integrated solution with proxy, orchestration and advanced replication in a complete, combined package. 100% developed, tested and supported by Continuent.
We call these components Tungsten Proxy (aka Tungsten Connector), Tungsten Manager and Tungsten Replicator. While these are best-of-breed solutions for the respective operations on their own merits, the real differentiation comes from how these components talk to each other.
Tungsten Proxy, Manager and Replicator work closely together. During a Primary database failure the Tungsten Manager will:
- recognize the failure,
- command the Tungsten Proxy to pause the traffic,
- decide upon and promotes the new Primary database,
- coordinate with the Tungsten Replicator so that the new Primary catches up,
- connect the remaining active Replicas to the new Primary,
- send information to the Proxy about the new cluster configuration (which node is the Primary and which Replicas), and finally
- instruct the Proxy to resume the Application traffic.
All this happens automatically. And often in just seconds, at times sub-second.
With tightly integrated components we get far more reliable solutions. Something that behaves predictably, both for the single local HA cluster, and also for our geo-distributed cluster of clusters (or a composite cluster as we call it). One and the same software for HA, DR and Geo-scale clusters.
This does not even touch the performance boost one can gain from intelligent load-balancing using the Proxy. Or the built-in monitoring and integration with various alert systems to notify database administrators about any changes in the cluster status. Or the fact that one can run these clusters practically anywhere, on-prem, on virtual instances or in any cloud, and in any combination of the hybrid environments.
And all this is capped with excellent, very thorough documentation and extensive online training courses.
What do you think? Complete? Check!
Fully-tested — To ensure that the various customer-valued features and benefits Tungsten Cluster offers are stable, Continuent has implemented a comprehensive Tungsten Test Framework, developed over many years. When we make changes in our software, either major or minor, each change goes through an extensive automated test routine.
Our current Tungsten Test Framework includes hundreds of thousands of different test scenarios which we use to validate each Tungsten release.
In the Test Framework there are 124 individual Test Suites, and each Test Suite contains hundreds of tests. Test Suite is a group of tests with common focus, such as specific topology or security, etc. We also test all operations, like switch, failover, recover, backup… each containing tens of steps (small tests). All these Test Suites are run against all possible combinations of different Linux variants (9 versions) and MySQL versions (6 MySQL versions, 6 MariaDB versions).
So, do the math! A lot of testing!
With Tungsten Manager alone, we have over 78 different 'business rules’ (matching different failure scenarios) covering a large number of possible failure corner cases and how to deal with each of those. When planning for High-Availability and Disaster Recovery, the devil truly is in the details. While everyone loves fast failover operations, equally important is to resist executing a failover during a false positive event. Understanding these scenarios and continuously testing is the only way to perfect the solution.
Fully-tested, plus some? Check!
Proven — Tungsten Cluster solution has powered some of the largest MySQL applications well over decade. It is truly and thoroughly battle-tested.
Our customers run a combined billions of dollars of business, on thousands of Tungsten Clusters worldwide. The marquee customers of ours include Adobe (Adobe Sign and Marketo), Carfax, Garmin, Modernizing Medicine, RiotGames, VMware and Vonage, along with a host of others. On average these customers have stayed with us 8+ years, and we expect them to remain on board for the foreseeable future.
The proof is in the pudding, right? (I never really understood what this means. It becomes more obvious when you hear the original phrase: “The proof of the pudding is in the eating.”)
You don’t need to trust our word. We offer a POC, along with any needed services, free of charge for anyone interested in experiencing first-hand the power of our software and the amazing 24/7 support services that come along with it.
Proven? If not convinced, ask for a POC and see it for yourself. Check!
(Very fast) 24/7 Support Included - When you build an ad-hoc DYI MySQL HA solution from open source components, the only throat you can choke is your own. And that is not a very nice feeling, is it?
Even the more complete clustering solutions currently available on the marketplace tend to have some external open source component, such as the proxy, that they must rely on. Who you gonna call (when you need help with that)? Ghostbusters? Good luck with that!
Continuent offers a world-class, truly the best-of-breed and amazingly fast, 24/7 support services for our solutions.
When you contact Continuent, you will reach us within minutes -- the average response time for an urgent ticket during the last several years across all of our customers has been less than 3 minutes. You will be connected with highly technical DBA or SRE, each with over 20 years of experience running business-critical MySQL backends.
None of the support requests go unanswered. Not only do we know the internals of the Tungsten solution, but we also have an excellent knowledge of MySQL as a whole. And if needed, we can produce a patch to alleviate problems in a very short time, often on a same-day or next-day basis.
24/7 Support with Minutes Notice? Check!
Total Cost of Ownership (TCO) - The cost of open source software is a curious thing. It is free, right? Wrong!
Yes, when compared to commercial license, the initial cost of the open source solution is lower. The license cost, that is. But definitely not the Total Cost.
Continuent has spent literally thousands of person-months to reach our current level of HA/DR/Geo-Scale solution maturity. What would you expect your development time and cost to be, even if you start with some existing open source components, to reach even remotely close to the functionality and reliability of a Tungsten Cluster. A lot.
Then you need to consider how much you need to keep paying to those resources, internal or external, who developed the solutions in order to address the ever changing environment in which you run your databases. Not cheap either.
And these individuals are not typically willing and able to be available 24/7 to provide the coverage needed for the business-critical and mission critical applications. Thus you would need to pay for additional for the 24/7 coverage, by the people who really do not understand your DIY solution in the first place. Paying for a baseless sense of security, effectively money for nothing.
The last but not least is the real possibility for the lost revenue. Especially with ad-hoc, patched up, not fully tested solutions. The lost revenue could be, and often is, the largest amount of money on the table.
Continuent offers reasonable, predictable cost for highly functional, very reliable and amazingly supported solutions. This practically guarantees that there won’t be surprises. No surprises related to the operations nor the cost.
Lowest Total Cost? Yeah, by a wide margin. Check!
What about the famous lock-in?
Oh yeah, I forgot to address the ‘lock-in’. With open source you are free like a bird. There is no cost and you can do whatever, and whenever you want. Sounds great, right?
With commercial solutions you are stuck. Locked in a jail, with no escape.
But what if it is not a jail? What if it feels like a 5-star luxury hotel? With wonderful amenities and impeccable service. A place that is a dream come true.
How much would you be willing to pay to stay at our luxurious “Swan Hotel”? With Continuent it is less than you expect!
Smooth sailing, steady as you go,
PS. BTW, what happened with the unfortunate DBA dealing with challenges of the less than stellar ClusterControl solution by SeveralNines? A good question!
Continuent being such nice guys and gals, we naturally took care of him by offering our software and services free of charge through the remaining duration of his current agreement with SeveralNines. Thus no extra expenses and no worry about the ‘sunk-cost syndrome’.
And we will keep the cost reasonable also going forward. We are on this venture on a long-haul. We will never try to maximize the short term revenue.
If you find yourself in a similar position with ClusterControl, Galera, InnoDB Cluster, Percona XtraDB Cluster or even with your own DIY solution, please reach out to us. We will help you out!