Ever since MongoDB emerged on the open source database scene in 2009, comparisons with MySQL have kept the open source database community debating the pros and cons, some swearing by MySQL while others firmly rooting for MongoDB. These user communities are pretty passionate about the technologies they use, which makes for great discussions and debates … and which ensures that these database technologies keep evolving for the better. In reality though, it transpired over the years that hybrid environments were inevitably created, as no one database by itself will ever be able to answer every need users have.
Which is where today’s topic really comes in: replication from MySQL to MongoDB.
As most of our readers will know, MongoDB is currently the most popular (open source) document store - according to DB-Engines’ document story popularity listing - available both for deployment on self-managed infrastructure and as a global, fully-managed cloud database service in the form of MongoDB Atlas (deploy across AWS, Azure, or GCP).
While MySQL continues to be not only the most popular (open source) RDBMS, but also the most popular open source database overall. https://db-engines.com/en/ranking
How MySQL & MongoDB Compare
An obvious difference between MongoDB and MySQL is that MongoDB is a document store that stores data in JSON formatted text. Users can generate their own identifiers or have MongoDB generate them. The document can then be stored alongside as a blob, in a similar way to most key-value stores.
- Databases are called … databases
- Tables, however, are called collections
- No strict structure needed as in MySQL
- Schema can be changed at any time
- Note: applications have to be designed in a way that data may either be missing or it will actually receive unexpected data.
Despite the differences, MongoDB is still a database that needs to perform a certain workload on a limited set of CPU cores, a limited amount of RAM and plenty of fast storage.
It needs to be treated as one would MySQL:
- Track memory usage
- Ensure there is sufficient disk space
- Benchmark systems: how much IO can be done with MongoDB?
Note: while MongoDB features a schemaless design it still needs indexes to find the data. Similar to MySQL, it features primary and secondary indexes and it can apply or remove them at runtime.
For a comprehensive comparative checklist go to: https://db-engines.com/en/system/MongoDB%3BMySQL
Things to Know About MySQL to MongoDB Replication
During the replication process, data is converted from the MySQL database/table/row structure into corresponding MongoDB structures. In general, it is easier to understand that a row within the MySQL table is converted into a single document on the MongoDB side, and automatically added to a collection matching the table name.
When preparing the hosts you must be aware of this translation of the different structures, as it will have an effect on the way the information is replicated from MySQL to MongoDB.
For more details on how to prepare for MongoDB Replication: https://docs.continuent.com/tungsten-replicator-5.4/deployment-mongodb-preparation.html
Why Replicate From MySQL to MongoDB
There could be any number of reasons as to why someone would want or need to replicate from MySQL to MongoDB over and above some of these practical ones:
- A user who might have scientific data stored in a MySQL database and wants to use MongoDB to take advantage of its map/reduce functionality to power some web charts. They’ll be looking for a way to have new writes to MySQL replicated into MongoDB. They may be looking for a solution where they can inspect the binary MySQL log and update accordingly, just like standard MySQL replication.
- Another user may have a MongoDB server and MySQL server, and they may want to use MySQL for "write" and MongoDB for "read". They’ll typically be looking for tools or processes by which they could transfer the MySQL writes to the MongoDB server on regular intervals.
- And so forth…
Let us know about your own MySQL to MongoDB replication needs or use cases… and check out the details below in order to experience the Tungsten Replicator (AMI) yourselves!
How MongoDB Replication Works
Option 1: Local Install
The Extractor reads directly from the logs, even when the DBMS service is down. This is the default.
Option 2: Remote Install
The Extractor gets log data via MySQL Replication protocols (which requires the DBMS service to be online) This is how we handle Amazon Aurora extraction tasks.
How to Get Started With the Tungsten Replicator AMI for MongoDB
MongoDB Atlas Compatibility
Please note that Tungsten Replicator’s MongoDB applier includes MongoDB Atlas as a target and is updated to use the latest MongoDB JDBC Driver.
Getting started with the 14-Day free trial
Users can try one instance of each type of the AMI for free for 14 days to get them started (AWS infrastructure charges still apply).
Please note that free trials will automatically convert to a paid hourly subscription upon expiration and the following then applies in the case of MongoDB targets.
Replicate from AWS Aurora, AWS RDS MySQL, MySQL, MariaDB & Percona Server to MongoDB / MongoDB Atlas from as little as $0.40/hour
With Tungsten Replicator (AMI) on AWS, users can replicate GB's of data from as little as $40c/hour:
- Go to the AWS Marketplace, and search for Tungsten, or click here.
- Choose and Subscribe to the Tungsten Replicator for MySQL Source Extraction.
- Choose and Subscribe to the target Tungsten Replicator AMI of your choice.
- Pay by the hour.
- When launched, the host will have all the prerequisites in place and a simple "wizard" runs on first launch and asks the user questions about the source and/or target and then configures it all for them.
Watch the Getting Started Walkthrough
If you need help getting started, our colleague Chris Parker recorded this handy walk-through on how to get started with the Tungsten Replicator AMI with plenty of tips & tricks.
Extraction and Appliers
Below you’ll find the full listing of extractors and appliers that are available with Tungsten Replicator.
- MySQL (all versions, on-premises and in the cloud)
- MariaDB (all versions, on-premises and in the cloud)
- Percona Server (all versions, on-premises and in the cloud)
- AWS Aurora
- AWS RDS MySQL
- Azure MySQL
- Google Cloud SQL
Available Replication Target Databases
Do give Tungsten Replicator AMI a try and let us know how you get on or if you have any questions by commenting below!
Check out our related replication blogs if you’re (also) working with these databases / platforms:
- Vertica: Real-Time MySQL Analytics — How to Replicate Data in Real-Time from MySQL, MariaDB or Percona Server to Vertica
- PostgreSQL: How Replication to PostgreSQL Works: Replicating Data in Real-Time from MySQL, MariaDB or Percona Server to PostgreSQL
- Kafka: https://www.continuent.com/resources/blog/how-replication-between-mysql-kafka-works-replicating-data-real-time-mysql-mariadb-or