Failover configuration

Every application integrating with the toolkit can configure how it achieves failover.

Configuration is defined by a set of key names and their values in the key=value format and can be provided to a Streams application by a text-based config file, the Application Configuration method, or both. If both methods are specified, then text-based config file takes precedence.

Text file configuration

Every application can have its own cross DC configuration specified via its own text based file (e.g. my-dc1-crossdc-config.txt).

The following configuration key=value pairs must exist for every application. A typical text file based cross DC configuration for a given application will look as shown below.


# Specify a name for this local data center.
localDataCenterName=dc1
# Specify the operation mode for this local data center: 0 for passive, 1 for active
crossDCOperationMode=1
# Specify the HTTP port number you want to use for the Cross DC Http Receiver.
crossDCHttpPort=25091

# In the property below, you can either give a single or multiple Streams application machine name(s) or
# the IP addresses of those machines that are used in the remote Data Center dc2. 
# If you have multiple machines, separate them by a comma: Machine1,Machine2,Machine3
remoteDataCenterApplicationMachineNames=d0702.acme.com,b0517.acme.com

# Specify the data snapshot storage directory.
# (Leave it commented out when not using a file system based storage.)
dataSnapshotStorageDirectory=/storage/cross-dc-snapshot/CrossDataCenterFailoverSample/dc2

# Specify the Relational Database access details for the data snapshot storage.
# (Leave them all commented out when not using an RDBMS based storage.)
#dataSnapshotJdbcUrl=jdbc:db2://h0319b14.acme.com:50000/boadb
#dataSnapshotJdbcUser=dragon
#dataSnapshotJdbcPassword=fire
##### You must create an opt sub-directory at the top-level of your 
##### application directory and then copy the required JDBC driver file there.
#dataSnapshotJdbcDriverLib=opt/db2jcc4.jar
#dataSnapshotJdbcClassName=com.ibm.db2.jcc.DB2Driver
##### You can name your table and its columns as you like.
##### But, keep the order and type of the columns as shown below:
##### id varchar(256) NOT NULL, replicationTime varchar(256), snapshot BLOB(32M), PRIMARY KEY (id)
#dataSnapshotTableName=dc2_cdc_rep
#dataSnapshotPrimaryKeyColumnName=id

# For all the config values that appear below, you can leave them as it is unless you
# really have a need to change them. In most cases, the default value below is sufficient.

# One time initial delay at the start of the application before the CrossDC toolkit goes to its real work.
crossDCInitDelay=40.0

# Heartbeat gets exchanged across the data centers for every 30 seconds.
######################################################################
# To deactivate the RemoteDC failover completely, set this 
# time interval to a very large value so that the heartbeat exchange will not 
# trigger anytime soon.  Set it to 444444444.00 (This means once in 14 years).
######################################################################
heartbeatExchangeInterval=30.0

# Data center failover will happen after four consecutive heartbeat misses i.e. after 120 seconds.
consecutiveHeartbeatMissesAllowed=4

# Periodic in-memory state data snapshot gets exchanged across 
# the data centers to do the data replication for every 180 seconds.
######################################################################
# To deactivate the RemoteDC snapshot/replication completely, set this 
# time interval to a very large value so that the data snapshot exchange will not 
# trigger anytime soon.  Set it to 888888888.00 (This means once in 28 years).
######################################################################
dataSnapshotExchangeInterval=180.0

# Specify whether you want to send the cross DC heartbeat and data snapshot messages to
# all the machines you configured above.
sendToAllRemoteMachines=false
# Specify whether you want to log the HTTP errors all the time.
alwaysLogHttpErrors=false
# Specify the HTTP connection timeout in seconds.
httpConnectionTimeout=25
# Specify the HTTP read timeout in seconds.
httpReadTimeout=100
# Specify the need to retain the pre-existing data snapshot files during the data center startup. 
retainOlderDataSnapshotsAtStartup=false
# Specify the need to send the replicated data snapshots to their
# origin DC in a rare case when both DCs go down simultaneously and
# get started at the same time after that event.
sendDataSnapshotsToOriginDCAtStartup=false

For the remoteDataCenterApplicationMachineNames field, it is required to give all the application machine names in the remote DC that can potentially run the operators/PEs belonging to a given Streams application.

A copy of the config file can be found in the toolkit under etc/crossdc-config.txt.

App config method

If a configuration is specified by app config, an app config named CrossDCFailover must be created in the Streams instance and the app config must have its configuration in this format:


namespace::MainCompositeName_key=value

For example:


com.acme.test::MyTest1_localDataCenterName=dc1
...

To assist in creating this app config, see the etc/make-crossdc-appconfig.sh script in the toolkit.