Best Practices

A Quick-Reference Guide for Tungsten Operations

Check back often for updates to this page

Getting Support and Training

Deployment Best Practices

How Do I Get Started?

  1. You will work with Continuent Support who will guide you through the process below

  2. https://docs.continuent.com/tungsten-clustering-7.0/deployment-best-practices.html#deployment-best-practices-deployment

  3. Verify that all Prerequisites have been satisfied

    1. https://docs.continuent.com/tungsten-clustering-7.0/deployment-requirements.html

  4. Sign up on our download portal and wait for an email confirming your activation

    1. https://www.continuent.com/user/customer-register

  5. Download the software package(s) you want

  6. Decide the type of install you want

    1. INI-based: maintain a config file on each node and perform install and update on every node, so ssh used

    2. Staging-based: maintain a configuration on a single node and perform install and update once on this selected “staging server”, password-less ssh access required from the “staging server” host to every node in every cluster you want to manage from the staging server host

  7. Decide the level of security you want

    1. Non-Secure - do not use SSL to encrypt the cluster communications, set disable-security-controls=true

    2. Secure - use SSL to encrypt the cluster communications, set disable-security-controls=false

    3. https://docs.continuent.com/tungsten-clustering-7.0/deployment-security.html

  8. Create the desired configuration, usually aided by Continuent support

    1. Ensure install=true

    2. Ensure that profile-script= has been specified for the selected Tungsten OS user path, normally “tungsten” to ensure that the env is loaded properly

  9. Create the database nodes with MySQL Server running on all nodes

  10. Optionally seed the full dataset to all nodes

  11. Become the Tungsten OS user via sudo, for example: sudo su - tungsten

  12. Copy the software package to /opt/continuent/software (or some other place recorded in your config)

  13. Extract the tarball

  14. cd to the new dir

  15. Optionally execute the validation step with tools/tpm validate -i

  16. Execute the installation with tools/tpm install -i

  17. Once the install is done on all nodes, refresh the env by logging out of the tungsten OS user shell, then log back in to load the new env

  18. For INI-based installs, pick one node and copy the keys from /opt/continuent/share/ to all other nodes.

    1. If you have password-less SSH enabled between the nodes, use the tpm copy-keys command, as of v7.0.3

    2. For v7.0.0 and onwards, do this step even if you have a non-secure deployment and have set disable-security-controls=true

    3. https://docs.continuent.com/tungsten-clustering-7.0/deployment-security-enabling-ini.html

  19. Start the cluster by calling startall on every node

  20. Check the cluster status using echo ls | cctrl command on one or more nodes, do at least once per cluster for Composite topologies

  21. Validate the cluster with cctrl> cluster validate

  22. Validate the cluster topology with cctrl> cluster topology validate from the global datasource

  23. Use cctrl> cluster heartbeat to verify that replication is working on all nodes

    1. this will insert a new seqno on the Primary which will then be replicated to all downstream replicas.

    2. Check the Progress= number in cctrl> ls to confirm

  24. Once your cluster is up and running, please strongly consider using our browser-based GUI, the Tungsten Dashboard

    1. Docs for the Tungsten Dashboard are here: https://docs.continuent.com/tungsten-dashboard-1.0/index.html

    2. If you would like help with the Dashboard installation, please open a support case at https://continuent.zendesk.com/requests/new

Configuration

  • You always need an odd number of nodes in a cluster. Connector-only nodes do not count, only nodes with a Manager.

  • The standard install path for Tungsten Clustering is /opt/continuent and /opt/replicator for the standalone Tungsten Replicator

  • The standard location for storing the Tungsten configuration file is /etc/tungsten/tungsten.ini

  • For OS’s with systemd, you must include install=true and keep that setting so that updates preserve the system state for Tungsten processes

  • Ensure that the binlog-format is defined in your my.cnf

    • The default setting is binlog-format=MIXED

    • Use binlog-format=ROW for Composite Active/Active (CAA) clusters, also for any cluster that will be feeding a non-MySQL target like Vertica

  • It is imperative that the open files limit for both the Tungsten and MySQL OS users be increased

  • Use /etc/hosts, NOT DNS - if DNS fails, so will the entire cluster!

  • Make sure that the required ports are open in all firewalls and iptables rules, check with iptables -L

  • Active witness nodes that run Manager only can provide the odd node when needed

    • it is important that enable-active-witnesses=true goes into each individual cluster section, NOT the default section

    • the witness node must be listed under the members= property

  • The use of ORM/Hibernate will prevent automatic read/write splitting by the Connectors as Hibernate will execute all reads within transactions. The Connectors are unable to inspect statements within transactions and therefore all transactions will always be directed to a Primary node.

  • The use of Triggers needs to be handled very carefully because Tungsten does not automatically disable triggers on the Replicas when using Row-based replication, which is possible when using MIXED, the recommended default for Clustering (except for CAA, which should use ROW - see the section below “CAA Facts and Best Practices”).
 Please see the online documentation for more details on how to address this situation:
http://docs.continuent.com/tungsten-clustering-6.1/troubleshooting-known-issues-triggers.html 


Updates/Upgrades

  • When doing INI-based upgrades, just place the cluster in maintenance mode and issue the upgrade on all nodes at the same time

    • https://docs.continuent.com/tungsten-clustering-7.0/cmdline-tools-tpm-ini-upgrades.html

    • As long as the cluster is in maintenance mode then nothing needs to be shunned to perform the software upgrade

    • The only potential impact would be a momentary pause in replication as the replicators are restarted

    • The database is not restarted or touched

    • Performing the upgrade without a switch is actually less impactful as you do not need to worry about connections being terminated and re-routed to the new primary

    • If the connectors are on the same host then simply add --no-connectors to the tpm update command, and then issue tpm promote-connector at an appropriate time to restart the connector. At that point any in-flight connections will be terminated, but since this is under your control you can mitigate the impact by scheduling the restart accordingly.

  • Always use tools/tpm update --replace-release from the extracted software directory when upgrading version

Security

There is excellent documentation about implementing security at https://docs.continuent.com/tungsten-clustering-7.0/deployment-security-enabling.html

  • Setting the “umbrella” tpm option disable-security-controls=false sets the following other tpm options for you: --rmi-ssl=true, --thl-ssl=true, --rmi-authentication=true, --jgroups-ssl=true, --datasource-enable-ssl=true, --connector-ssl=true, --connector-rest-api-ssl=true, --manager-rest-api-ssl=true, --replicator-rest-api-ssl=true

  • As of v7.1.0 the tpm cert command handles all security operations for Tungsten

  • Use one of the below openssl commands to view multiple certificates stored in the same file:

  • Some useful AWS CLI commands:

    • Add inbound rule(s) for a security group ID:
      shell> aws ec2 authorize-security-group-ingress --group-id sg-NNNNNNNN --protocol tcp --port 80 --cidr '0.0.0.0/0'

    • Delete inbound rule(s) for a security group ID:
      shell> aws ec2 revoke-security-group-ingress --group-id sg-NNNNNNNN --protocol tcp --port 80 --cidr '0.0.0.0/0'

    • List security groups by security group ID:
      shell> aws ec2 describe-security-groups --output json | jq -r '.SecurityGroups[]|.GroupId+" "+.GroupName'

    • List inbound rules for a specific security group ID:
      shell> aws ec2 describe-security-groups --group-ids sg-NNNNNNNN --output json | jq -r '.SecurityGroups[].IpPermissions[]|. as $parent|(.IpRanges[].CidrIp+" "+($parent.ToPort|tostring))'

Topologies

Topologies, Defined

STD

Standard 3-node cluster

One single cluster, consisting of three or more nodes.

All clusters must have an odd number of nodes to establish voting quorum for failovers.

https://docs.continuent.com/tungsten-clustering-7.0/deployment-primaryreplica.html

CAP

Composite Active/Passive

2 or more 3-node clusters, one site active, all other sites passive, manual site-level failover

https://docs.continuent.com/tungsten-clustering-7.0/deployment-composite.html

CAA

Composite Active/Active

2 or more 3-node clusters, all sites active

https://docs.continuent.com/tungsten-clustering-7.0/deployment-activeactive-clustering.html

DAA

Dynamic Active/Active

2 or more 3-node clusters, all sites active, but the Connector is configured to route all writes to a single cluster at a time, auto-failover to another site if an entire cluster becomes unavailable

https://docs.continuent.com/tungsten-clustering-7.0/deployment-dynamicactiveactive.html

DDG

Distributed Datasource Groups

One single cluster, typically consisting of five or more nodes, always an odd number, possibly spread across more than one datacenter or region

DDG provides site-level automatic failover for the highest level of availability, and a single write Primary for data simplicity. This topology also supports MIXED-based replication for lower overhead and better performance.

https://docs.continuent.com/tungsten-clustering-7.1/deployment-advanced-ddg.html

https://www.continuent.com/resources/blog/tungsten-distributed-datasource-groups-automatic-mysql-server-database-failover

Topologies Blog Posts

We have a number of great blog posts on this subject:

CAA Best Practices and Key Facts

  • What are the requirements for an active/active cluster (network infrastructure and replication latency)?

    • The requirements are similar, but there will need to be an additional port opened for a second replication stream, and some MySQL requirements:

      • Row based replication - this helps avoid problems due to non-deterministic SQL statements and prevent data drift between locations.
        binlog-format = row

      • Auto-increment needs to be configured properly based on the number of clusters
        auto-increment-offset = 1
        auto-increment-increment = 4

      • The topology needs to be set explicitly in the INI:
        [global]
        topology=composite-multi-master
        composite-datasources=east,west

      • Primary keys on all tables (strongly recommended.

    • There really isn't any latency requirement for asynchronous replication

  • The documentation page shows the commands to use to view the status: https://docs.continuent.com/tungsten-clustering-7.1/deployment-activeactive-clustering-bestpractices.html

  • What level of effort is required to go from active/passive to active/active?

    • There is some effort involved to do it with no downtime.   If you can take a maintenance window, then we could just uninstall/reinstall

  • How does Tungsten handle latency between the two clusters? i.e. What happens if a record is written to region A and read from region B before the DB replication happens?

  • Tungsten Cluster does not handle conflict resolution – that is entirely up to the application.

  • When using active/active, the risk of requiring re-provision of nodes is greater, with large amounts of data this may introduce significant latency to re-provision Replicas, especially if all other sites need to be rebuilt due to a data drift issue. Create a procedure or become familiar with tprovision to re-provision nodes, and test your procedure ahead of time.

  • Use the InnoDB table type on all MySQL servers because without InnoDB storage engines we cannot guarantee that Replica replicators are crash safe, i.e., able to recover after either MySQL or the Tungsten Replicator fails.

  • Are there additional cost for an active/active cluster?

    • Yes, the license fee is a bit higher

  • Any other suggestion/considering before migrating to an active/active cluster?

Monitoring and Alerting

Operations Best Practices

General

  • Before performing any operation, use cctrl> ls to see and UNDERSTAND the state of the cluster

  • All OS clocks MUST be in sync using NTP and all MUST have the same timezone.

  • Never shun a Primary

  • Never have two online Primaries at the same time - that causes a cluster panic and FAILSAFE-SHUN condition

  • A cluster datasource will not come online automatically via the Manager until the replicator on that node has been online for the first time

  • The Replicator’s current position information is stored in the MySQL server in a table called trep_commit_seqno . Use the dsctl get command to see the values.

  • To get access to Tungsten command-line tab-completion and the Tungsten commands in your PATH, please source the environment file /opt/continuent/share/env.sh in your shell init script (i.e. ~/.bash_profile)

  • To perform maintenance on just one node, the best practice is to remain in Automatic mode and simply SHUN that specific node. This procedure keeps automatic failover and recovery active, keeping the risk position lower.

    • cctrl> datasource NODE shun

  • To perform maintenance on the current Primary, the best practice is to remain in Automatic mode and execute a manual switch command inside the cctrl cli tool. This will move the Primary role to another node, allowing the old node to be shunned as it is now a Replica after the switch.

    • cctrl> switch

    • cctrl> datasource OLD_PRIMARY shun

  • To perform maintenance on more than one node in a local cluster then

    • cctrl> set policy maintenance

Cluster Logging

  • The default location to look for Tungsten logs is in /opt/continuent/service_logs

  • The key log files to look for are:

    • Connector: connector.log

    • Manager: tmsvc.log

    • Replicator: trepsvc.log

  • Most of the files in /opt/continuent/service_logs are symbolic links to the real log files in various sub-directories, example link targets shown below:

/opt/continuent/tungsten/tungsten-manager/log/cctrl.log
/opt/continuent/tungsten/tungsten-connector/log/connector-api.log
/opt/continuent/tungsten/tungsten-connector/log/connector.log
/opt/continuent/tungsten/tungsten-manager/log/manager-api.log
/opt/continuent/tungsten/tungsten-replicator/log/mysqldump.log
/opt/continuent/tungsten/tungsten-replicator/log/replicator-api.log
/opt/continuent/tungsten/tungsten-manager/log/tmsvc.log
/opt/continuent/tungsten/tungsten-replicator/log/trepsvc.log
/opt/continuent/tungsten/tungsten-replicator/log/xtrabackup.log

Failover Facts

Cluster Policy Modes

  • The Tungsten Manager is constantly executing a set of health checks and reacts to the results of those tests based on rules.

  • The cluster policy dictates how the cluster reacts when certain conditions are met. The policy controls if the rules are fired or not.

    • In Automatic policy mode, or “automatic mode” for short, any rules that are triggered are acted upon. When the cluster is in Automatic mode, failures of either the Primary or any Secondary are handled without human intervention.

    • In Maintenance policy mode, or “maintenance mode” for short, any rules that are triggered are ignored. When the cluster is in Maintenance mode, failures of either the Primary or any Secondary are ignored and require human intervention.

  • Maintenance mode should be used when administration or maintenance is required on the entire cluster, or you want automatic failover and recovery to be disabled.

  • Advanced: When a cluster is in Maintenance mode, a Connector will NOT pause all incoming requests if connectivity is lost with the selected Manager. This allows for safe Manager restarts and updates/upgrades without the calling application/clients being impacted.

  • cctrl> set policy maintenance

  • cctrl> set policy automatic

  • https://www.continuent.com/resources/blog/tungsten-cluster-policies-be-automatic

  • https://docs.continuent.com/tungsten-clustering-7.0/operations-policymodes.html

  • https://docs.continuent.com/tungsten-clustering-7.0/cmdline-tools-check_tungsten_policy.html

Node State: SHUNNED

Recovery

  • When in doubt during a Production outage or problem:

  • When healing a node, ensure each layer is healthy before proceeding to the next one, in order from first to last:

    • MySQL Server must be up and running and accepting connections, test using:

      • service mysqld status

      • tpm mysql

    • Tungsten Replicator must be running and online, test using:

      • replicator status

      • trepctl services

      • trepctl -service {svcName} qs

      • trepctl -service {svcName} perf

      • trepctl -service {svcName} status

      • NOTE: if there is only one Replicator service, there is no need to specify -service {svcName}

    • The Tungsten manager must be running and the datasource (host) must be online, check with:

      • manager status

      • echo ls | cctrl

    • The Tungsten Connector proxy must be running and able to reach the database server, along with at least one available Primary node, and for Composite clusters, the cluster(s) must be online too. Test using:

      • connector status

      • tpm connector

  • When healing a failed-over cluster node, use the datasource NODENAME recover command NOT datasource NODENAME online

  • If you cannot get connected through the connector, are all services online at the Composite level? - check with cctrl -multi

  • For Manager Java Heap Alerts and slow Manager response times:

    • cctrl> set policy maintenance

    • On all nodes: shell> manager stop

    • When all Managers have been stopped on all nodes, then on all nodes: shell> manager start

    • cctrl> set policy automatic

  • To re-provision (rebuild) a replica node, use the tprovision command

Command-Line Tools

Command: tpm

The tpm command is used to install, configure, upgrade and maintain Tungsten clusters. Stands for “Tungsten Package Manager”.

Command: cctrl

The cctrl command is used to control the cluster. It connects to the local Manager process, so the Manager must be started and fully initialized before cctrl will be able to connect.

  • https://docs.continuent.com/tungsten-clustering-7.0/cmdline-tools-cctrl.html

  • Use TAB to get a list of available sub-commands and to auto-complete hostnames

  • Use cctrl -multi in composite clusters to be able to see remote clusters

  • cctrl> help

  • cctrl> help {TOPIC}

  • cctrl> use <TAB>

  • cctrl> ls

  • cctrl> ls -l

  • cctrl> cluster heartbeat

  • cctrl> ls resources

  • cctrl> cluster validate

  • cctrl> cluster topology validate

  • cctrl> set policy maintenance

  • cctrl> set policy automatic

  • cctrl> ping {HOST}

  • ADVANCED: You can get cctrl to execute commands via a pipe: echo 'ls' | cctrl

  • ADVANCED: shell> cctrl -output debug -multi -expert

Command: trepctl

Use trepctl commands to see the current status of the Replicator, along with the optional -r X for repeat every X number of seconds. The Replicator process must be running and fully initialized since the trepctl command connects to the local Replicator process by default.

Command: thl

The thl command provides access and control of the extracted events stored on disk in THL format

  • https://docs.continuent.com/tungsten-clustering-7.0/cmdline-tools-thl.html

  • Use the thl -service {serviceName} info command to see a summary of all available THL on that node

  • Use the thl -service {serviceName} index command to see a list of all available THL files and the sequence numbers in each file

  • Use thl -service {serviceName} list -seqno {seqno} -sql to see the contents of an event

Command: dsctl

Command: tmonitor

Command: tapi

The tapi command is used to interface with the Tungsten REST APIv2 from the command line.

Tungsten Connector Proxy

Tungsten Manager

  • https://docs.continuent.com/tungsten-clustering-7.0/manager.html

  • Use the cctrl command to control the cluster

  • Use the cctrl -multi command to control a Composite cluster

  • Use the cctrl> set policy maintenance to set the cluster to Maintenance mode where the manager will do nothing to heal the cluster

  • Use the cctrl> set policy automatic to set the cluster to Automatic mode where the Manager will take actions to heal the cluster

  • Validate the cluster with cctrl> cluster validate

  • Validate the cluster topology with cctrl> cluster topology validate from the top-level datasource

  • To change the role of a datasource, specify the role using the cctrl command

    • To make a node the Primary: cctrl> datasource NODENAME master

    • To make a node a Replica: cctrl> datasource NODENAME slave

    • To make a node a Relay: cctrl> datasource NODENAME relay

  • In MAINTENANCE mode, stopping/restarting any or all of the Managers will have zero impact on the cluster other than preventing automatic failover in the event of a MySQL Server failure; client traffic will continue to flow because the Connectors have been told to freeze state.

Tungsten Replicator

I Want To…

MySQL 8 Upgrade Guide

Performance Tuning

tungsten.ini Examples

  • Example INI [default] Section

#### Basic defaults
home-directory=/opt/continuent
profile-script=~/.bash_profile
start-and-report=true
user=tungsten
mysql-allow-intensive-checks=true

#### DDG is disabled by default - update individual tungsten.ini files with value > 0 to enable
# https://www.continuent.com/resources/docs/latest/tc/deployment-advanced-ddg.html#deployment-advanced-ddg-configure
datasource-group-id=0

#### Security is enabled by default
disable-security-controls=false
# https://www.continuent.com/resources/docs/latest/tc/cmdline-tools-tpm-configoptions-c.html#cmdline-tools-tpm-configoptions-connector-ssl-capable
connector-ssl-capable=true

#### Define the application layer settings
application-port=3306
application-readonly-port=3307
application-user=app_user
application-password=goodpassword

#### Define the database layer settings
replication-user=tungsten
replication-password=goodpassword
replication-port=13306

# https://www.continuent.com/resources/docs/latest/tc/how-to-enable-database-ssl.html
datasource-enable-ssl=true
datasource-mysql-ssl-ca=/etc/mysql/certs/ca.pem
datasource-mysql-ssl-cert=/etc/mysql/certs/client-cert.pem
datasource-mysql-ssl-key=/etc/mysql/certs/client-key.pem

#### Connector Settings of Interest
connector-bridge-mode=true
connector-readonly=false

# https://www.continuent.com/resources/docs/latest/tc/connector-routing-smartscale.html
connector-smartscale=false
connector-smartscale-sessionid=DATABASE

#### Replicator Settings of Interest
#backup-method=xtrabackup
#repl-backup-online=false
#log-directory=/var/lib/mysql
#thl-directory=/opt/continuent/thl
#repl-thl-log-retention=7d
#thl-log-file-size=100000

#### Define the Prometheus exporter settings
property=manager.prometheus.exporter.enabled=true
property=replicator.prometheus.exporter.enabled=true
property=connector.prometheus.exporter.enabled=true
property=replicator.prometheus.exporter.port=8091
property=manager.prometheus.exporter.port=8092
property=connector.prometheus.exporter.port=8093

#### Define the REST API settings

## REST API user & pass
rest-api-admin-user=tungsten
rest-api-admin-pass=goodpassword

## Connector REST API Settings
connector-rest-api=true
connector-rest-api-address=0.0.0.0
connector-rest-api-authentication=true
connector-rest-api-port=8096
connector-rest-api-ssl=true

## Manager REST API Settings
manager-rest-api-port=8090
manager-rest-api=true
manager-rest-api-address=0.0.0.0
manager-rest-api-full-access=true
manager-rest-api-ssl=true
manager-rest-api-authentication=true

## Replicator REST API Settings
replicator-rest-api-port=8097
replicator-rest-api=true
replicator-rest-api-address=0.0.0.0
replicator-rest-api-ssl=true
replicator-rest-api-authentication=true

  • Example INI for Standalone Clusters (STA)

[alpha]
topology=clustered
master=db11
members=db11,db12,db13
connectors=db11,db12,db13,app11,app12

  • Example INI for Composite Active/Passive (CAP)

[alpha]
topology=clustered
master=db11
members=db11,db12,db13
connectors=db11,db12,db13,app11,app12

[beta]
topology=clustered
relay-source=alpha
relay=db21
members=db21,db22,db23
connectors=db21,db22,db23,app21,app22

[global]
composite-datasources=alpha,beta

  • Example INI for Composite Active/Active (CAA)

[alpha]
topology=clustered
master=db11
members=db11,db12,db13
connectors=db11,db12,db13,app11,app12

[beta]
topology=clustered
master=db21
members=db21,db22,db23
connectors=db21,db22,db23,app21,app22

[global]
composite-datasources=alpha,beta
topology=composite-multi-master

Useful Shell Aliases

These are the shell aliases I regularly use on our demo nodes. As always, Your Mileage May Vary ;-}

alias api='tapi -j '
alias atat='echo '\''select concat("R/O=",@@hostname); select concat("R/W=",@@hostname) for update;'\'' | tpm connector 2>/dev/null | grep -v '\''@@hostname'\'''
alias atatro='echo '\''select @@hostname;'\'' | tpm connector 2>/dev/null | grep -v '\''@@hostname'\'''
alias atatrw='echo '\''select concat("R/W=",@@hostname) for update;'\'' | tpm connector 2>/dev/null | grep -v '\''@@hostname'\'''
alias catdp='cat /opt/continuent/tungsten/cluster-home/conf/dataservices.properties'
alias ccls='echo ls | cctrl'
alias cclsm='echo ls | cctrl -multi'
alias cclsp='echo ls | cctrl | grep progress'
alias ccm='cctrl -multi'
alias cdb='cd /home/tungsten/bin'
alias cdd='cd /home/tungsten/diags'
alias cdg='cd /opt/continuent/generated'
alias cdh='cd /opt/continuent/share'
alias cdini='cd /etc/tungsten/'
alias cdl='cd /opt/continuent/service_logs'
alias cdr='cd /opt/continuent/releases'
alias cds='cd /opt/continuent/software'
alias cdsl='cd /opt/continuent/software/latest'
alias cdt='cd /opt/continuent/tungsten'
alias chb='echo '\''cluster heartbeat'\'' | cctrl'
alias cini='cat /etc/tungsten/tungsten.ini'
alias ini='echo /etc/tungsten/tungsten.ini'
alias isonline='check_tungsten_online'
alias loadwatch='while true; do uptime| awk -F'\''load average:'\'' '\''{print $2}'\'' ; sleep 5; done'
alias rso='replicator start offline'
alias se='source /opt/continuent/share/env.sh'
alias stamp='/usr/bin/date '\''+%Y%m%d%H%M%S'\'''
alias tca='tpm cert aliases'
alias tcg='tpm cert gen'
alias tci='tpm cert info'
alias tcl='tpm cert list'
alias tfo='tungsten_find_orphaned'
alias tin='tools/tpm install'
alias toff='trepctl -all-services offline'
alias ton='trepctl -all-services online'
alias tq='tpm query '
alias tqs='tpm query staging'
alias tqv='tpm query version'
alias treset='trepctl -all-services offline; trepctl -all-services reset -all -y; trepctl -all-services online'
alias ts='trepctl status -all-services'
alias tsa='trepctl services'
alias tsg='trepctl status | egrep stat\|appl\|role'
alias tsl='trepctl services | grep serviceName | awk -F: '\''{print $2}'\'''
alias tssg='trepctl services | egrep stat\|appl\|serviceName\|role'
alias tui='tpm update -i'
alias tun='tpm uninstall --i-am-sure'
alias tuns='tools/tpm uninstall --i-am-sure'
alias tup='tools/tpm update -i --replace-release'
alias tv='tools/tpm validate'
alias tvu='tools/tpm validate-update'
alias tvup='tools/tpm validate-update'
alias vini='vim /etc/tungsten/tungsten.ini'
alias vlisteners='vi /opt/continuent/tungsten/tungsten-connector/conf/listeners.json'
alias vumap='vi /opt/continuent/tungsten/tungsten-connector/conf/user.map'

Key Release Notes

v7.1.4 - October 1, 2024

Minor feature and bugfix release

v7.1.3 - June 25, 2024

4 new features, two new commands and 8 bugfixes.

v7.1.2 - April 3, 2024

Two new features, one new command and 9 bugfixes.

  • New Feature: The systemd startup scripts will now have a dependency on the time-sync service (if available) being started before tungsten components.

  • New Feature: Filter binarystringconversionfilter - provides a way to convert varchar data from one charset into another

  • New Command: tungsten_mysql_ssl_setup - creates all needed security files for the MySQL database server and acts as a direct replacement for the mysql_ssl_rsa_setup command which is not included with either Percona Server or MariaDB. This command will now be called by tpm cert gen mysqlcerts instead of mysql_ssl_rsa_setup, see https://docs.continuent.com/tungsten-clustering-7.1/cmdline-tools-tungsten_mysql_ssl_setup.html

v7.1.1 - December 15, 2023

Two new features, and some key bugfixes.

  • BugFix: Release 7.1.1 contains a major bug fix that prevents switches from working correctly after upgrading from v6.

  • Added Support: Ruby 3.x - Installations will now succeed on hosts running Ruby 3.x

v7.1.0 - August 16, 2023

6 new features, 2 new commands, and a bunch of improvements along with 18 bugfixes.

v7.0.3 - April 4, 2023

  • A new Connector advanced listeners feature has been added to the connector for even great control of client connectivity. For more details, see https://docs.continuent.com//tungsten-clustering-7.0/connector-advanced-listeners.html

  • Added Connector user-based authentication to existing host-based authentication to restrict connections per host and per user. For more details, see https://docs.continuent.com//tungsten-clustering-7.0/connector-authentication-host.html

  • A new thl tail command has been added, allowing you to view the live THL changes as they are generated.

  • The tpm command now prints a warning when running as the root OS user during operations that make changes.

  • A new command, tpm copy-keys, has been added to assist with the distribution of security-related files after an INI-based installation. This tool requires password-less ssh access between the cluster nodes for the copy to work.

  • The tpm ask summary command has two new keys, isSmartScale and isDirect, which are also available individually on the command line.

  • The tungsten_skip_all command (along with alias tungsten_skip_seqno) now shows the full pendingExceptionMessage instead of just pendingError, and the More choice shows the pendingErrorEventId and the pendingError.

  • Increased the default manager liveness heartbeat interval from 1s to 3s

v7.0.2 - December 9, 2022

  • The various user-xxxx.log files are no longer generated

  • check_tungsten.sh has been removed (it was deprecated in v6.1.18)

  • In CAA clusters, no longer sending write traffic to the remote site unless the local site is fully offline. In case of local failover, the connector will now pause connections until a new primary is elected. This will avoid risks of out-of-order apply after local failover

  • A failsafe shunned cluster (Caused by a network split) will be auto recovered after the network connection is re-established.

  • A new standalone status script has been added called tungsten_get_status that shows the datasources and replicators for all nodes in all services along with seqno and latency.

  • A new command thl dsctl has been added to show the dsctl set command for a given seqno

  • A new command set trepctl pause and trepctl resume has been added

  • New logs files have been added for the REST API:

    • service_logs/connector-api.log

    • service_logs/manager-api.log

    • service_logs/replicator-api.log

v7.0.1 - June 13, 2022

  • The default value for the tpm property repl-svc-fail-on-zero-row-update has been changed from warn to stop

  • Added the ability to turn auto recovery on or off dynamically, removing the need to run tpm update. This is done by running the following command:

    • trepctl -service servicename setdynamic -property replicator.autoRecoveryMaxAttempts -value <number>

    • Note: The service must be offline before changing the property

  • A new tpm report sub-command has been added to display all communications channels and associated states.

  • The thl list command now displays an approximative field size in bytes for row-based replication.

  • Added a connector mode command to print which mode the connector is running in, either "bridge", "proxy" or “smartScale”

v7.0.0 - March 29, 2022

  • The tpm diag command now uses tar czf instead of the zip command to compress the gathered files. The zip command is no longer a pre-requisite for tpm diag.

  • tungsten_set_position has been deprecated and no longer available in this release. dsctl should be used instead.

  • tungsten_provision_slave has now been renamed to tprovision

v6.1.14 - August 17, 2021

Last updated: March 27th, 2025 at 10:20am EDT