Flex Your Tungsten Replicator: Two New Roles

Rock and Role!

In this blog post, we will cover how two new Tungsten Replicator roles allow for THL transfer when not actively extracting or applying events. This new behavior provides for better control and performance under a variety of conditions.

Existing Tungsten Replicator Roles: Primary & Replica

As we know, the core of operations for Tungsten Replication are the extract and apply functions. The other key operation is the transfer of THL from extractor to applier via the network.

master

  • Summary: a database node in the master role performs the extraction and thl-server operations when online.
    (Note: Continuent made the decision to convert all “master/slave” references to “primary/replica.” While this is an important initiative for us, the process will take some time as we do not want these changes to destabilize the software, cause unexpected consequences or lead to downtime. To guarantee continuous operations always comes first.)
  • A Tungsten Replicator in the master role (aliases: primary or extractor) extracts data from a source database.
  • The extractor reads the binary log from a MySQL server and generates THL on disk.
  • The extractor uses two stages, binlog-to-q and q-to-thl.
  • The Primary replicator also serves THL to other replicators over a network connection (thl-server).
  • During normal operations, a master role Replicator will have both an extractor and a thl-server running.

slave

  • Summary: the slave role when online performs the thl-client and applier operations.
  • A Tungsten Replicator in the slave role (aliases: replica, secondary or applier) pulls MySQL events in the form of THL data from an upstream THL server (thl-client) and writes them to disk.
  • The applier then reads those MySQL events from the local, on-disk copy of THL and applies them to a target database using JDBC.
  • The applier uses three stages, remote-to-thl, thl-to-q and q-to-dbms.
  • During normal operations, a slave role Replicator will have both a thl-client and an applier running.

For more information about Replicator stages, please see this blog post:
Mastering Tungsten Replicator Series: Understanding Pipelines and Stages.

New Tungsten Replicator Roles: thl-server & thl-client

Two new roles have been added to the Tungsten Replicator to make the system more flexible.

thl-server

  • Summary: the thl-server role when online performs only the thl-client operations.
  • Normally, when a Primary is taken offline, both the extractor and the thl-server operations are stopped. There may be times when we wish to stop extracting, but continue to serve THL to downstream clients.
  • The thl-server Replicator role is used on Primaries to temporarily enable the THL server when a Primary is taken offline.
  • This will allow downstream Secondary replicators to download and apply any TH that hasn't yet been processed by that Applier even though extraction is stopped.

thl-client

  • Summary: the thl-server role when online performs only the thl-client operations.
  • Normally, when a Replica is taken offline, both the thl-client and applier operations are stopped. There may be times when we wish to continue downloading THL without applying it to the target database.
  • A thl-client replicator role is used on Secondaries to temporarily enable the THL client when the replicator is taken offline.
  • This will allow the Secondary replicator to download any THL available from the upstream THL server, but does NOT apply the THL to the target database.

Using the New Roles: thl-server

Used on a Primary node, the thl-server role allows downstream clients access to the THL when we have stopped extracting binary logs from the MySQL server.

To enable the thl-server role, issue the following statements in MAINTENANCE mode:

shell> trepctl offline
shell> trepctl setrole -role thl-server
shell> trepctl online

To disable the thl-server role and revert to full extractor operations, issue the following statements in MAINTENANCE mode:

shell> trepctl offline
shell> trepctl setrole -role master
shell> trepctl online

Using the New Roles: thl-client

Used on a Secondary node, the thl-client role allows the Replicator to continue downloading THL even when it has stopped applying events to the local MySQL server.

To enable the thl-client role, issue the following statements in MAINTENANCE mode:

shell> trepctl offline
shell> trepctl setrole -role thl-client
shell> trepctl online

IMPORTANT: One key aspect of thl-client operation is that the `trepctl status` output will show the seqno of the last THL record written, NOT the seqno of the last event applied to the database! When the Replicator role is changed back to slave, the seqno will revert back to normal and track the event last applied into the database.

shell> trepctl status -name tasks
Processing status command (tasks)...
NAME                    VALUE
----                    -----
appliedLastEventId    : mysql-bin.000343:0000000096496363;-1
appliedLastSeqno      : 1215911
appliedLatency        : 0.124
applyTime             : 4.098
averageBlockSize      : 1.000     
cancelled             : false
commits               : 58236
currentBlockSize      : 0
currentLastEventId    : mysql-bin.000343:0000000096496363;-1
currentLastFragno     : 0
currentLastSeqno      : 1215911
eventCount            : 58235
extractTime           : 688.201
filterTime            : 0.091
lastCommittedBlockSize: 1
lastCommittedBlockTime: 0.001
otherTime             : 0.493
stage                 : remote-to-thl
state                 : extract
taskId                : 0
timeInCurrentEvent    : 0.018
Finished status command (tasks)...

To disable the thl-client role and revert to full applier operations, issue the following statements in MAINTENANCE mode:

shell> trepctl offline
shell> trepctl setrole -role slave
shell> trepctl online

New Tungsten Replicator Roles: Wrap-Up

In this blog post, we explored how the two new Tungsten Replicator roles allow for THL transfer when not actively extracting or applying events. This new behavior provides for better control and performance under a variety of conditions.

Online Documentation:

https://docs.continuent.com/tungsten-replicator-6.1/operations-status-repl-roles.html

https://docs.continuent.com/tungsten-replicator-6.1/cmdline-tools-trepctl-command-setrole.html

About the Author

Eric M. Stone
COO

Eric is a veteran of fast-paced, large-scale enterprise environments with 35 years of Information Technology experience. With a focus on HA/DR, from building data centers and trading floors to world-wide deployments, Eric has architected, coded, deployed and administered systems for a wide variety of disparate customers, from Fortune 500 financial institutions to SMB’s.

Add new comment