Teradata Replication Solutions Overview - …dbmanagement.info/Books/MIX/1152_TeraData.pdf ·...

Teradata Replication SolutionsOverview

Release 12.0B035-1152-067A

October 2007

The product or products described in this book are licensed products of Teradata Corporation or its affiliates.

Teradata, BYNET, DBC/1012, DecisionCast, DecisionFlow, DecisionPoint, Eye logo design, InfoWise, Meta Warehouse, MyCommerce, SeeChain, SeeCommerce, SeeRisk, Teradata Decision Experts, Teradata Source Experts, WebAnalyst, and You’ve Never Seen Your Business Like This Before are trademarks or registered trademarks of Teradata Corporation or its affiliates.

Adaptec and SCSISelect are trademarks or registered trademarks of Adaptec, Inc.

AMD Opteron and Opteron are trademarks of Advanced Micro Devices, Inc.

BakBone and NetVault are trademarks or registered trademarks of BakBone Software, Inc.

EMC, PowerPath, SRDF, and Symmetrix are registered trademarks of EMC Corporation.

GoldenGate is a trademark of GoldenGate Software, Inc.

Hewlett-Packard and HP are registered trademarks of Hewlett-Packard Company.

Intel, Pentium, and XEON are registered trademarks of Intel Corporation.

IBM, CICS, DB2, MVS, RACF, Tivoli, and VM are registered trademarks of International Business Machines Corporation.

Linux is a registered trademark of Linus Torvalds.

LSI and Engenio are registered trademarks of LSI Corporation.

Microsoft, Active Directory, Windows, Windows NT, and Windows Server are registered trademarks of Microsoft Corporation in the United States and other countries.

Novell and SUSE are registered trademarks of Novell, Inc., in the United States and other countries.

QLogic and SANbox trademarks or registered trademarks of QLogic Corporation.

SAS and SAS/C are trademarks or registered trademarks of SAS Institute Inc.

SPARC is a registered trademarks of SPARC International, Inc.

Sun Microsystems, Solaris, Sun, and Sun Java are trademarks or registered trademarks of Sun Microsystems, Inc., in the United States and other countries.

Symantec, NetBackup, and VERITAS are trademarks or registered trademarks of Symantec Corporation or its affiliates in the United States and other countries.

Unicode is a collective membership mark and a service mark of Unicode, Inc.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Other product and company names mentioned herein may be the trademarks of their respective owners.

THE INFORMATION CONTAINED IN THIS DOCUMENT IS PROVIDED ON AN “AS-IS” BASIS, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES, SO THE ABOVE EXCLUSION MAY NOT APPLY TO YOU. IN NO EVENT WILL TERADATA CORPORATION BE LIABLE FOR ANY INDIRECT, DIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS OR LOST SAVINGS, EVEN IF EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

The information contained in this document may contain references or cross-references to features, functions, products, or services that are not announced or available in your country. Such references do not imply that Teradata Corporation intends to announce such features, functions, products, or services in your country. Please consult your local Teradata Corporation representative for those features, functions, products, or services available in your country.

Information contained in this document may contain technical inaccuracies or typographical errors. Information may be changed or updated without notice. Teradata Corporation may also make improvements or changes in the products or services described in this information at any time without notice.

To maintain the quality of our products and services, we would like your comments on the accuracy, clarity, organization, and value of this document. Please e-mail: [email protected]

Any comments or materials (collectively referred to as “Feedback”) sent to Teradata Corporation will be deemed non-confidential. Teradata Corporation will have no obligation of any kind with respect to Feedback and will be free to use, reproduce, disclose, exhibit, display, transform, create derivative works of, and distribute the Feedback and derivative works thereof without limitation on a royalty-free basis. Further, Teradata Corporation will be free to use any ideas, concepts, know-how, or techniques contained in such Feedback for any purpose whatsoever, including developing, manufacturing, or marketing products or services incorporating Feedback.

Copyright © 2005-2007 by Teradata Corporation. All Rights Reserved.

mailto:[email protected]

Teradata Replication Solutions Overview 3

Preface

Purpose

This book provides an overview of Teradata Replication Solutions–GoldenGate Replication Products (abbreviated hereafter as “Teradata Replication Solutions”), a suite of products that supports near-real-time movement and synchronization of data between Teradata Database and other source or subscriber databases. Teradata Replication Solutions are made up of Teradata Database and GoldenGate Replication Products. This overview:

• Introduces features and benefits of using Teradata Replication Solutions

• Discusses product components, architecture, and interoperability with other Teradata Database features

• Summarizes the major tasks performed to implement Teradata Replication Solutions with GoldenGate Replication Products

• Directs readers to sources of more detailed information about implementing and maintaining Teradata Replication Solutions in an enterprise

Audience

This book is intended for:

• System and application programmers responsible for writing programs to access data from the Teradata Database

• System administrators

• Database administrators and relational database developers

• Users and business decision makers

• Other implementers

• Teradata service professionals

Supported Releases

This book supports the following releases:

• Teradata® Database 12.0

• GoldenGate Replication Products version 9.0.3

• Teradata Tools and Utilities 12.0

PrefacePrerequisites

4 Teradata Replication Solutions Overview

Prerequisites

This book is an overview document, so no prerequisite reading is required; however, the reader should be familiar with basic computer technology, database management systems, and the utilities that load and retrieve data.

Read the current Release Definition for any additional information that may not have been available at the time this document was completed.

Changes to This Book

Date Description

October 2007 Chapter 2: Described how to resolve a mapping error in bidirectional replication when the values of primary key columns do not match on source and target tables.

September 2007 • Standardized terminology throughout the book to use source, subscriber, and replication server consistently.

• Clarified meaning and removed irrelevant details on internal processing in Chapters 2 and 3.

• Chapter 1: Clarified that GoldenGate can only deliver data to MySQL and generic ODBC-compatible database management systems.

• Chapter 1: Showed that the replication server is currently certified only on 32-bit platforms.

• Chapter 1: Added information about bidirectional replication between Teradata Database 12.0 and a prior version of Teradata Database.

• Chapter 2: Reflected replication scalability and down node handling.

• Chapter 2: Showed that the GoldenGate COMPRESSUPDATES feature now supports Teradata Database.

• Chapter 2: Revised figures to add ODBC, TCP/IP, and TAM.

• Chapter 3: Clarified when the RSG spill file is used and the importance of allowing disk space for it.

• Chapter 3: Clarified the role of Teradata Dynamic Workload Management in handling disconnections from the replication server.

• Chapter 3: Removed a restriction regarding replication of UNICODE and KANJISJIS character sets. TIME, TIMESTAMP, and INTERVAL data types are now supported.

PrefaceAdditional Information


Additional Information

For detailed information supporting GoldenGate for Teradata, see Chapter 4: “Sources of Further Information.”

November 2006 • Added GoldenGate Director, which replaces the GoldenGate Activity Console.

• Added temporary restriction: Change Data Capture does not work if a table in a replication group contains a LARGE DECIMAL data type.

• Added information on stopping table copy.

• Added the new default temp directory for Linux.

• Explained what to do after the connection to GoldenGate is lost in maximum protection mode and Teradata Dynamic Workload Manager is busy during reconnection.

• Added information on when to use Teradata Replication Solutions.

• Rewrote section on character sets and GoldenGate for Teradata to improve accuracy and clarity

November 2005 • Added information regarding enabling encryption of replication messages.

• Replication now supports these new features:

• Mixed operating system Teradata server support, with any combination of MP-RAS, Windows Server 2003, and SUSE Linux

• Any number of source servers replicating to any number of subscriber servers

• Bidirectional replication of data in same table on source and subscriber Teradata servers

• GoldenGate for Teradata intermediary server now supports Windows 2000 Server and Red Hat Linux in addition to Windows Server 2003

June 2005 Initial release.

Date Description

PrefaceAdditional Information


Additional information that supports this product and Teradata Database is available at the following Web sites.

Type of Information Description Source

Overview of the release

Information too late for the manuals

The Release Definition provides the following information:

• Overview of all the products in the release

• Information received too late to be included in the manuals

• Operating systems and Teradata Database versions that are certified to work with each product

• Version numbers of each product and the documentation for each product

• Information about available training and support center

http://www.info.teradata.com/

Click General Search. In the Publication Product ID field, enter 1725 and click Search to bring up the following Release Definition:

• Base System Release DefinitionB035-1725-067K.

Additional information related to this product

Use the Teradata Information Products web site to view or download the most recent versions of all manuals.

Specific manuals that supply related or additional information to this manual are listed.


Click General Search, and do one of the following:

• In the Product Line field, select Software - Teradata Database for a list of all of the publications for this release.

CD-ROM images This site contains a link to a downloadable CD-ROM image of all customer documentation for this release. Customers are authorized to create CD-ROMs for their use from this image.


Click General Search. In the Title or Keyword field, enter CD-ROM, and click Search.

Ordering information for manuals

Use the Teradata Information Products web site to order printed versions of manuals.


Click How to Order under Print & CD Publications.

General information about Teradata

The Teradata home page provides links to numerous sources of information about Teradata. Links include:

• Executive reports, case studies of customer experiences with Teradata, and thought leadership

• Technical information, solutions, and expert advice

• Press releases, mentions and media resources

Teradata.com

http://www.info.teradata.com




http://www.teradata.com

PrefaceReferences to Microsoft Windows and Linux


References to Microsoft Windows and Linux

This book refers to “Microsoft Windows” and “Linux.” For Teradata Database 12.0, these references mean the following:

• “Windows” is Microsoft Windows Server 2003 32-bit and Microsoft Windows Server 2003 64-bit.

• “Linux” is SUSE Linux Enterprise Server 9 and SUSE Linux Enterprise Server 10.

Teradata plans to release Teradata Database support for SUSE Linux Enterprise Server 10 before the next major or minor release of the database. Therefore, information about this SUSE release is included in this document. The announcement regarding availability of SUSE Linux Enterprise Server 10 will be made after Teradata Database 12.0 GCA. Please check with your account representative regarding SUSE Linux Enterprise Server 10 availability in your location.

PrefaceReferences to Microsoft Windows and Linux



Table of Contents

Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3

Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3

Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3

Supported Releases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3

Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4

Changes to This Book. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4

Additional Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5

References to Microsoft Windows and Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7

Chapter 1: Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

What is Teradata Replication Solutions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Definition of Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Benefits of Teradata Replication Solutions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Teradata Dual Active Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Active Data Warehousing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

When To Use Teradata Replication Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Supported Source and Subscriber Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Hardware and Software Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Compatibility Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Chapter 2: Teradata Replication Solutions Architecture . . . . 21

Change Data Capture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

GoldenGate for Teradata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29Create a Replication Group and Start Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Table of Contents


Copy Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30Replicate Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30Execute Management Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30General Error Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31Down Node Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31

GoldenGate Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31

For More Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

Chapter 3: System Administration of Teradata Replication Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33

Task Overview: Setting Up Teradata Replication Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . .33

Setting Up Teradata Access Module. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33

Creating the RSG Spill File Directory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34

Defining RSG vprocs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34

Setting Replication Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34

Creating a Replication Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35

Teradata Transaction Commit Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35

Handling Disconnections from the Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35Under Maximum Protection Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35Under Maximum Performance Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36Replication Server Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37

Handling Backlog Change Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37

Determining the Commit Mode for a Replication Group . . . . . . . . . . . . . . . . . . . . . . . . . .37

Data Flow Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39

Transaction Sequencing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39

Character Sets and Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40

LARGE DECIMAL Restriction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40

Disk Capacity Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40

Enabling Encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41

Chapter 4: Sources of Further Information . . . . . . . . . . . . . . . . . . . . . .43

GoldenGate Documentation and Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43

Teradata Replication Documentation by Task. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44

Orange Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45


CHAPTER 1 Introduction

This chapter introduces concepts important to understanding the features, benefits, and basic operation of Teradata Replication Solutions. In this chapter, you will find information about Teradata Replication Solutions, including:

• What is Teradata Replication Solutions?

• Definition of Terms

• Benefits of Teradata Replication Solutions

• Supported Source and Subscriber Systems

• Hardware and Software Requirements

What is Teradata Replication Solutions?

In general terms, replication is the process of copying data from one database system to another. Typically, replication also includes a step for synchronizing data changes to ensure the two databases are consistent. Teradata Replication Solutions provides replication capabilities between instances of Teradata Database, and between Teradata Database and other supported databases.

Purpose

Teradata Replication Solutions allows users to capture changes made to a specific set of tables in one database and apply those changes to corresponding tables in another database in near-real time. Replication can serve several purposes in database information management:

• Replication can provide a backup of specified table data in the event of problems with your source database.

• If your site has Teradata Dual Active Solutions, and one system becomes unavailable, the remaining system can automatically take over database operations. The data on the systems will have been automatically synchronized via replication. There are various database replication tools and methodologies available for use with Teradata Dual Active Solutions. (See “Teradata Dual Active Solutions” on page 16.) GoldenGate Replications Products is one such tool.

• When implemented between Teradata Database and databases from other vendors, you can migrate data from one system to the other, making data accessible across the different environments. This capability can support data acquisition, described in “When To Use Teradata Replication Solutions” on page 17, and Active Data Warehousing, described in “Active Data Warehousing” on page 17.

Chapter 1: IntroductionWhat is Teradata Replication Solutions?


Components

Teradata Replication Solutions include components furnished by both Teradata and GoldenGate Software, Inc., a leading developer of data replication products. These include:

• Teradata Database 12.0

• GoldenGate Replication Products version 9.0.3

This suite of GoldenGate software products supports replication between instances of Teradata Database and between Teradata Database and other databases. GoldenGate for Teradata is required. For a detailed description of the GoldenGate Replication Products and their operation in Teradata Replication Solutions, see Chapter 2: “Teradata Replication Solutions Architecture.”

The typical configuration using Teradata Replication Solutions includes:

• A source database, running on a source server, that captures data for replication

• A subscriber database, running on a subscriber server, to which captured data is applied

One of these must be Teradata Database. For information about supported database systems, see “Supported Source and Subscriber Systems” on page 18.

• A server running GoldenGate for Teradata (called the replication server); GoldenGate for Teradata transports data captured from Teradata Database and applies it to the subscriber database.

• The ODBC driver for Teradata installed on the replication server; this supports data transfer between the GoldenGate software and the database systems.

• Teradata Access Module installed on the replication server; this supports data transfer between a Teradata Database and the replication server.

Figure 1 illustrates the major components of Teradata Replication Solutions for replication between servers running Teradata Database. For more detailed information about the components and subcomponents of Teradata Replication Solutions and their operation, see Chapter 2: “Teradata Replication Solutions Architecture.” Information about hardware and software requirements of systems running Teradata Replication Solutions components appears in “Hardware and Software Requirements” on page 19.

Chapter 1: IntroductionDefinition of Terms


Figure 1: Major Components of Teradata Replication Solutions

For an illustration of the replication between Teradata and other vendors’ databases, see the “Process” topic in Chapter 2: “Teradata Replication Solutions Architecture.”

Definition of Terms

This section lists the terms used to describe concepts and components related to Teradata Replication Solutions.

Teradata Server A(Source)

Change DataCapture

GoldenGate for Teradata

ODBC

Teradata Server B(Subscriber)

Replication Server

1152A017

TeradataAccess Module

RSG vprocs

SourceDatabase

ReplicationGroups

SubscriberDatabase

SubscriberTables

Term Definition

apply The process by which the subscriber database receives replicated data; equivalent to “delivery” in the context of GoldenGate for Teradata.

asynchronous replication

A type of replication in which a transaction that updates a replicated table may be committed on the source server without waiting for the transaction to be propagated to the subscriber.

capture In the context of GoldenGate for Teradata, the process of retrieving transaction information for inserts, updates, and deletes from the database change source.

change data capture The process of capturing changes made to a production data source. Change data capture is typically performed by a change data capture component in the database or by reading the source database log. It consolidates units of work, ensures data is synchronized with the original source, and reduces the nightly batch, high data volume in a data warehousing environment. Teradata Database includes a Change Data Capture (CDC) feature that is used by Teradata Replication Solutions.

data synchronization The process of automatically updating remote copies of a table in response to changes in the original table.

delivery In the context of GoldenGate for Teradata, the process of applying replicated data to the subscriber database.

Chapter 1: IntroductionDefinition of Terms


dual active solution Two or more active database systems that operate in tandem and provide a highly available production environment that also offers rapid disaster recovery. A dual active solution virtually eliminates all down time and provides seamless disaster recovery protection for critical users and applications.

GoldenGate Director A multi-tiered Java client-server application that enables the configuration and management of GoldenGate instances from a remote client.

heterogeneous replication

Replication between disparate databases (Teradata Database to/from another database system).

homogeneous replication

Replication between like databases (Teradata Database to Teradata Database).

Relay Services Gateway (RSG)

A virtual processor (vproc) that provides a network interface through which the source server exports replication data. Each node may have one RSG.

replication A process for copying data from one database to another, while maintaining data integrity between the source and subscriber databases through synchronization.

Teradata Replication Solutions provides asynchronous and synchronous replication.

This release supports bidirectional replication between databases (where each database serves as both a source and subscriber of replication). Bidirectional replication between different tables is common. Bidirectional replication between the same table on multiple systems requires careful planning to ensure that potential conflicts can be appropriately resolved.

replication group A set of tables for which data changes are being captured on a source server.

replication server The server that hosts the GoldenGate software and Teradata Access Module.

rollback A process for reversing selected data changes.

source database The database from which data will be extracted or copied into the subscriber database.

source server The server that owns the data to be replicated. On the source server, client applications execute transactions using Teradata SQL or utilities, such as MultiLoad, and update the tables of one or more replication groups. The changes are captured by Teradata Replication Solutions and given to the replication server.

subscriber database The database in which replicated data will be loaded or inserted.

subscriber server A server to which changes captured from a source server by the replication server are applied. Changes are applied to tables that duplicate those of the source. Teradata Replication Solutions executing on the servers provides the capture and delivery functions.

synchronous replication

A type of replication in which a transaction that updates a replicated table is not committed on the source server until the transaction has been propagated to the subscriber.

Term Definition

Chapter 1: IntroductionBenefits of Teradata Replication Solutions


Benefits of Teradata Replication Solutions

Teradata Replication Solutions supports the following important customer needs:

• Recoverability

• Availability

• Performance continuity

• Resource optimization

Recoverability, availability, and performance continuity are key features of Teradata Dual Active Solutions, a Teradata technology offering designed to maximize business continuity. For a discussion of Teradata Dual Active Solutions and the benefits it offers to Teradata customers, see “Teradata Dual Active Solutions” on page 16.

To help you make optimal use of systems resources, Teradata Replication Solutions supports acquisition and transfer of data from systems running in diverse environments, whether they are running Teradata Database or databases from other vendors.

Teradata Access Module (TAM)

A software package that runs on a server running GoldenGate for Teradata that allows the GoldenGate software to communicate with the RSG.

trail In the context of GoldenGate for Teradata, a data queue that stores changes captured from the source database.

Term Definition

Chapter 1: IntroductionBenefits of Teradata Replication Solutions


Figure 2 illustrates the implementation of Teradata Replication Solutions on computers performing various roles.

Figure 2: Roles for Teradata Replication Solutions

Teradata Dual Active Solutions

A dual active solution is a set of components or technologies that allow a secondary instance of Teradata Database to act as a backup for a primary instance of Teradata Database. Such a solution addresses a number of business continuity needs:

• Allows business operations to continue if a planned or unplanned outage occurs.

• Provides quick restoration of access to system resources.

• Enables businesses to enjoy consistent levels of service from their systems.

Replication support is a key capability of a dual active solution. When implemented as part of Teradata Dual Active Solutions, Teradata Replication Solutions captures and synchronizes data between two instances of a database, where one is a production system and the other is a backup. As a result, the backup system can take over if the production system fails and maintain data availability and integrity.

Figure 3 provides an overview of Teradata Dual Active Solutions. Teradata Replication Solutions offers a way to implement the synchronization shown between two Teradata Databases. The figure also depicts Teradata Query Director, which handles load distribution across the computers running Teradata Database.

Non-TeradataSystem

TeradataProduction

System

TeradataNon-Production

System

TeradataAlternateSystem

Non-TeradataSystem

Active Data Warehousing Data Acquisition

Teradata Dual Active Solutions

3020A006

Chapter 1: IntroductionWhen To Use Teradata Replication Solutions


Figure 3: Overview of Teradata Dual Active Solutions

For more information about Teradata Dual Active Solutions, search for “dual active” on the Teradata Web site: http://www.teradata.com/.

Active Data Warehousing

Because Teradata Replication Solutions supports bidirectional data synchronization, it can be used for Active Data Warehousing (ADW). An ADW environment can include the need to capture data changes in real time from operational systems, and apply them to Teradata Database, where real-time decision making can occur. The results of those decisions can be populated back to the operational systems, where appropriate actions can take place. This bidirectional data movement can take place in seconds or less.

Note that Teradata Replication Solutions itself is not intended as a replacement for EAI and message bus tools, where true two-phase commit is required at the application layer between Teradata and operational system applications.

When To Use Teradata Replication Solutions

Teradata Replication Solutions can capture data from other types of databases and apply it to a Teradata Database. This data acquisition capability is not a replacement for a high-volume ETL (extract, transform, load) capability that requires significant transformation capabilities, but is designed to be used where:

• Real-time capture and apply is required

• There is a need to eliminate batch windows

• The volume of data that changes regularly is relatively moderate

• The customer wants to pull data from a non-Teradata database, minimizing the overhead to the source system

TeradataSystem A

DataSynchronization

MonitoringAdministration

Operations Control

TeradataSystem B

3020A007

Users/Applications

Users/Applications

Users/Applications

TeradataQuery

Director


Chapter 1: IntroductionSupported Source and Subscriber Systems


Note that Teradata Replication Solutions does not replace bulk data loading utilities, such as FastLoad and MultiLoad.

Supported Source and Subscriber Systems

GoldenGate provides support for the following database management systems:

DB2 SQL/MP

Enscribe SQL/MX

Microsoft SQL Server Sybase

MySQL(delivery only)

Teradata Database

Oracle All ODBC-compatible database management systems (delivery only)

Chapter 1: IntroductionHardware and Software Requirements


Hardware and Software Requirements

Separate hardware and software requirements apply for Teradata Database and GoldenGate Replication Products, as shown in the following table.

Requirement Teradata Database GoldenGate

Hardware platform SMP and MPP systems Intel-based 32-bit hardware platforms supported

Note: Make sure that the storage capacity is sufficient to store unapplied data during periods when network outages or other problems prevent transmission of extracted data from the source system to the subscriber system. For information about storage on the replication server, see “Disk Capacity Considerations” on page 40.

Operating system MP-RAS®, Microsoft® Windows Server™ 2003, Microsoft® Windows 64-bit, andSUSE Linux®

Microsoft® Windows Server™ 2003 or 32-bit Red Hat® Enterprise Linux

Database, database driver Source: V2R6.0 or laterSubscriber: V2R5.1 or later

Version 3.04 or later ODBC driver for Teradata resident on the server running GoldenGate for Teradata:

Teradata Access Module Not applicable • For V2R6.x: TAM 1.x

• For Teradata Database 12.0: TAM 12.0

GoldenGate Director

(client-server application for configuring and managing GoldenGate from a remote client)

None required • Server–Any platform that supports Java 1.4 or later (Windows, Solaris, AIX, Red Hat Linux). The server requires:

• 1 GB RAM

• 100 MB disk storage

• Java Software Development Kit (JSDK) version 1.4 or later

• Either SQL Server 2000 or MySQL 4 (free from the MySQL web site). Either is a small (200 MB minimum) database in which Director Server will install and maintain a repository of tables.

• Client–Any Windows or UNIX platform that supports Java Runtime Environment (JRE) version 1.4 or later. The client requires a monitor with at least 1024 x 768 pixel resolution, preferably 1280 x 1024.

• Web–Any of the following web browsers are required:

• Microsoft Internet Explorer version 5.0 or later

• Netscape Navigator version 7.0 or later

• Mozilla Firefox version 1.3 or later

• Apple Safari version 1.2 or later

Chapter 1: IntroductionCompatibility Considerations


Compatibility Considerations

It is possible to do bidirectional replication between Teradata Database 12.0 and Teradata Database V2R6.x only if both TAM 1.x and TAM 12.0 reside on the replication server and the extract parameter files point to the compatible TAM version.

Figure 4: Bidirectional Replication between Teradata Database 12.0 and a Prior Version of Teradata Database

1152A019

Teradata

V2R6.2 or

prior

Teradata

12.0

TAM

1.0.0.1

TAM 12.0

GoldenGate

Replication

Products


CHAPTER 2 Teradata Replication SolutionsArchitecture

The Teradata Replication Solutions architecture relies on the Change Data Capture feature of Teradata Database and on functionality provided by GoldenGate Replication Products. This chapter describes the operation of these components in detail.

• Change Data Capture

• GoldenGate for Teradata

Change Data Capture

Teradata Replication Solutions uses the Change Data Capture facility of Teradata Database to collect change data from a source server for application to a subscriber server. Using this method, enterprises can replicate data from running systems without interruption and can limit the impact on system resources because only changes are copied.

Components

The Teradata Database features that support replication include:

• Replication group management

Replication groups are collections of tables on the source server that are to be replicated. Teradata Database supports multiple data connections for each replication group (one per RSG).

Teradata SQL includes a set of extensions for defining and managing replication groups. The CREATE REPLICATION GROUP and COMMENT ON GROUP functions are examples of these extensions.

• Replication security

A set of extensions to the Teradata SQL GRANT, SET SESSION, and BEGIN LOGGING/ END LOGGING statements that controls access to the replication management functions and lets you temporarily modify the default behavior of replicated tables.

You can use the BEGIN LOGGING and END LOGGING statements to suspend replication activity temporarily when replicated tables are the target for large-volume bulk data loads, such as those using TTU bulk loading utilities like MultiLoad.

For more information about these extensions, see SQL Reference: Data Definition Statements.

Chapter 2: Teradata Replication Solutions ArchitectureChange Data Capture


• Relay Services Gateway (RSG) vprocs

Replication tasks run on RSG vprocs on the source server for connections with the replication server. The connection implements the TCP/IP protocol. Each system node can have one RSG.

Process

As implemented in Teradata Replication Solutions, replication involves interaction between:

• A source database server, typically running Teradata Database, which contains the data to be captured.

• An replication server running GoldenGate for Teradata, which captures the changed data and transmits it to the subscriber server over a TCP/IP connection.

• A subscriber server running Teradata Database or another database system that receives the captured change data.

In bidirectional replication, each server is both a source and subscriber server. If both systems are running Teradata Database, both will perform change data capture operations.

The source database server supplies change data for capture based on transactions executed by client applications. The changes captured and replicated can include inserts, updates, and deletes resulting from the execution of SQL DML statements or utilities, such as MultiLoad and TPump.

Note: FastLoad cannot be run on tables that are part of replication groups.

In addition to change data, the source server can supply data for a table-copy operation. In this case, a complete table is captured and the replication server applies the captured table copy to the subscriber database in the same manner as it does change data.

When Teradata runs on the source database server, it provides functions that:

• Define a set of replicated tables for which change data should be captured.

• Enable the source server to accept a connection from the replication server for extracting changes made to the set of tables.

• Allow the replication server to connect to a subscriber server and deliver changed data to a set of replicated tables.

When a transaction begins, the Teradata Dispatcher assigns the transaction to an RSG vproc. AMPs send all changes to the assigned RSG vproc. The RSG vproc collects transaction change data in memory until the end transaction record is received. Then the Teradata server transmits change data to the server running GoldenGate for Teradata via Teradata Access Module. The replication server applies transaction data to the subscriber server.

During a one-way change data capture process between a source and a subscriber server, interaction between the source server and the replication server consists of:

1 Establishing data connections to the RSG vprocs.

2 Sending a data connection sign-on message to the RSG vprocs to define the replication groups for which data is to be captured.

3 Establishing a command connection to one RSG vproc.



4 Sending a connect command to define the replication group for which data is to be captured and the RSG vprocs used for transmitting transaction data to the replication server.

5 Sending an initiate command to start receiving transaction data.

6 Receiving transaction data through all of the data connections.

The process operates in both directions for bidirectional replication.

Data flows from the server to the replication server as long as active transactions are updating tables in the replication group. For information about how to manage the data flow or respond to interruptions, see Chapter 3: “System Administration of Teradata Replication Solutions.”

Figures 4 through 6 show examples of unidirectional replication of homogeneous and heterogeneous systems using local and remote connections; Figures 7 through 10 show examples of these types of systems handling bidirectional replication.

Bidirectional replication between different tables of two databases involves the basic replication process occurring in both directions, and requires no special considerations. Bidirectional replication between the same table on two systems, however, requires careful planning to ensure that potential conflicts can be appropriately resolved.

A mapping error can occur in bidirectional replication if the before values of the primary key columns in any row do not match on the source and target tables. To resolve the conflict, do one of the following:

• Use stored procedures to handle the exception. The stored procedure can be used to update the target row directly. Specify a FILTER option in the Replicat parameter MAP statement so the conflicted row is not processed. The syntax is as follows:

MAP ...., TARGET ...., EXCEPTIONSONLYSQLEXEC(SPNAME ...., ....., PARAMS(....), BEFOREFILTER),FILTER (0);

• Specify a KEYCOLS parameter on a column that is identical in the source and target tables.

Figure 5: Example of Unidirectional Replication Between Teradata Systems in a Local Data Center

1152B004

Replication Server

Teradata Database Teradata DatabaseCapture

Delivery

DirectorClient

TCP/IP ODBC

DirectorDatabase

GoldenGateDirector

Application

TAM



Figure 6: Example of Unidirectional Replication From Oracle/DB2 to Teradata Database in Remote Data Centers

Figure 7: Example of Unidirectional Replication Between Teradata Systems in Remote Data Centers

Oracle or DB2Delivery

1152B006

Teradata DatabaseReplication

Server

Capture

DirectorConsole Data Center #2Data Center #1

DirectorWeb

DirectorDatabase

GoldenGateDirector

Application

ReplicationServer

Teradata Database Teradata DatabaseDelivery

1152B008

ReplicationServer

Data Center #2Data Center #1

DirectorWeb

DirectorDatabase

GoldenGateDirector

Application

TCP/IP ODBC

Capture

TAM



Figure 8: Example of Bidirectional Replication Between Teradata Systems in a Local Data Center

Figure 9: Example of Bidirectional Replication Between Teradata Systems in Remote Data Centers1152B01

Replication ServerTeradata Database Teradata Database

Delivery

Delivery

DirectorClient

DirectorDatabase

GoldenGateDirector

Application

Capture

TAM

Capture

TAMTCP/IP

TCP/IPODBC

ODBC

ReplicationServer

Teradata Database

Delivery

Delivery

1152B0


Server


DirectorWeb

DirectorDatabase

GoldenGateDirector

Application

Capture

TAM

Capture

TAMTCP/IP

TCP/IPODBC

ODBC



Figure 10: Example of Bidirectional Replication Between Oracle/DB2 and Teradata Database in a Local Data Center

Figure 11: Example of Bidirectional Replication Between Oracle/DB2 and Teradata Database in Remote Data Centers

Oracle or DB2

Delivery

1152B014


Server

Capture

Delivery

DirectorClient

DirectorDatabase

GoldenGateDirector

Application

Capture

TAMTCP/IP

ODBC

Oracle or DB2

Capture

Delivery

ReplicationServer

Delivery

1152B016


Server


DirectorWeb

DirectorDatabase

GoldenGateDirector

Application

Capture

TAMTCP/IP

ODBC

Chapter 2: Teradata Replication Solutions ArchitectureGoldenGate for Teradata



GoldenGate for Teradata is one of the GoldenGate Replication Products delivered with Teradata Replication Solutions. GoldenGate for Teradata supports replication between instances of Teradata Database, and between Teradata Database and the other database management systems described in “Supported Source and Subscriber Systems” on page 18. GoldenGate software includes:

• GoldenGate Director. See “GoldenGate Director” on page 31.

• GoldenGate Rollback, a facility for fast recovery from downtime. Rollback provides tools for performing selective backout processing on enterprise databases and eliminates the need for full restore operations that usually require several hours or more.

• The COMPRESSUPDATES feature, which processes only changed columns and the primary key for updates. The default is to process all columns.

The following sections describe GoldenGate for Teradata and how it works. For information about other GoldenGate products, see the appropriate GoldenGate documentation.

Components

GoldenGate for Teradata is made up of software modules that capture, deliver, and manage data. The Capture module retrieves and writes selected data to a trail (a data queue) on the server running GoldenGate for Teradata. GoldenGate can apply the captured change data to different types of subscriber databases using SQL appropriate to the subscriber system.

The Capture module of GoldenGate for Teradata obtains change data from the database on the source server. In the following example, the RSG vprocs residing on nodes of a Teradata server communicate with GoldenGate for Teradata via Teradata Access Module, which also resides on the replication server.

Figure 12: Interaction of Database Servers with the Replication Server


Change DataCapture


ODBC


Replication Server

1152A017


RSG vprocs

SourceDatabase

ReplicationGroups

SubscriberDatabase

SubscriberTables



The Delivery module retrieves changes from the trail created by the Capture module and posts updates to the subscriber database using SQL via ODBC. Delivery uses checkpoints in the trail to track its position and ensure posting of all records to the subscriber database.

To ensure data and referential integrity, by default the delivery module mimics the source database by:

• Applying changes in the same order as they were committed in the source database

• Grouping operations into transactions the same as in the source database

Conflicts can be detected using before images to determine whether update data has changed. To resolve conflicts, users must develop custom logic appropriate for their operating environment and their specific needs.

The Manager module constantly monitors the operations and status of the EXTRACT and REPLICAT programs of the Capture and Delivery modules and executes a number of data management tasks. This module interfaces with GoldenGate Director, which is used to start and stop the EXTRACT program (described in the next section) and obtain the status of replication.

The following components are installed on the replication server:

• ODBC

• Teradata Access Module

• EXTRACT

• REPLICAT

Figure 13: Components on the Replication Server

Teradata Access Module handles the interaction between the replication server and Teradata Database related to the capture process. A GoldenGate program called EXTRACT uses Teradata Access Module to connect to the change data capture functions on the Teradata server (System A in Figure 14) to acquire data from the source system. The EXTRACT program reads database change records from data sources and outputs the data to one or more extract files.

ODBC

Replication Server

1152A018


GoldenGate Capture, Delivery, Manager

EXTRACT REPLICAT

Supported Operating System



Note: For databases other than Teradata Database, the EXTRACT program must reside on the source database to capture changes.

Another GoldenGate program called REPLICAT retrieves data from the appropriate extract files, then replicates the data to the subscriber database. The REPLICAT program retrieves the data, connects to the subscriber system via ODBC, and applies the data.

During bidirectional replication, the same process occurs simultaneously in the opposite direction, originating with System B as the source for data replication to System A, as shown in Figure 14. The EXTRACT program connects using Teradata Access Module, and the REPLICAT program applies data to the subscriber system (System A).

Figure 14: Replication Process Sequence

Note: This example shows replication in maximum performance commit mode; an illustration depicting maximum protection commit mode would be more detailed. For more information about transaction commit modes, see “Teradata Transaction Commit Protocols” on page 35.

Operation

Typically, GoldenGate for Teradata runs on a separate server that interfaces between two Teradata Database systems and allows transactional data to be replicated from one system to another. However, GoldenGate for Teradata is able to route data from any number of sources to any number of subscribers, and can also replicate data between different types of databases. For a list of supported databases, see “Supported Source and Subscriber Systems” on page 18.

The replication server can be local or remote to the source server, however a remote connection can increase transaction latency.


Change DataCapture

TeradataAccessModule

Extract

Queuedchange data

on disk (trails)

GoldenGate Director


2.

1.

4.

3. 7.

5.6.

Replication Server

1. Control connection2. Data connection3. Dictionary queries

4. Dictionary and setup queries5. Write to disk6. Retrieve transactions

7. Apply data

DeliveryComponent

CaptureComponent

1152B001

ODBC

REPLICAT



The replication process can include the following activities:

• Create a Replication Group and Start Replication

• Copy Data

• Replicate Data

• Execute Management Commands

• General Error Handling

• “Down Node Handling”

Note: Unless otherwise noted, at the beginning of each activity, the GoldenGate capture process sends an Initialize call and, in response, GoldenGate for Teradata reads the TAM parameter file specified in the arguments of the Initialize call.

Create a Replication Group and Start Replication

If the TAM parameter file contains a CREATE REPLICATION GROUP statement, TAM executes the statement via ODBC. If the statement succeeds and returns a replication group security token and group ID, TAM updates the parameter file to insert the security token and group ID, and begins replication.

Copy Data

The copy operation synchronizes the source and subscriber tables. If the TAM parameter file is configured for table copy, GoldenGate for Teradata stores the parameters and returns control to the GoldenGate capture process. Upon receiving a Control:Copy command, GoldenGate for Teradata establishes control and data connections to Teradata Database for Copy and initiates the copy operation with Teradata Database. GoldenGate for Teradata starts receiving multiple data streams from the Teradata Database and queuing the data.

Replicate Data

If the TAM parameter file is configured to start replication for an existing replication group, GoldenGate for Teradata establishes control and data connections to Teradata Database. GoldenGate for Teradata then begins receiving and processing change data from the source database system. It does so continually until terminated or suspended by a user command. For more information about the transaction commit protocol, see Chapter 3: “System Administration of Teradata Replication Solutions.”

Execute Management Commands

In response to one of the following commands—Control:Suspend, Control:Resume, or Control:Terminate—GoldenGate for Teradata sends the appropriate Suspend, Resume, or Terminate command over the control connection to the Teradata Database.

Note: To stop table copy and terminate the replication group, be sure to stop table copy first. Use the following procedure:

1. Stop the table copy by issuing Control-Terminate on the table copy extract process (the one started with Control:Copy). The replication group continues to be in the INITIATED state.

2. Terminate the replication group by issuing Control:Terminate on the regular extract process (the one started with replication).

Chapter 2: Teradata Replication Solutions ArchitectureGoldenGate Director


If you terminate the replication group before stopping table copy, the replication group will be in SUSPEND state. To recover from this, restart the table copy extract and perform steps 1 and 2, in that order.

General Error Handling

If there is an error or unexpected result of a function within GoldenGate for Teradata, GoldenGate for Teradata provides information about the error to either the GoldenGate logging facility or GoldenGate Director, as appropriate for a given error condition.

Down Node Handling

If an RSG is unavailable for any reason, such as a node failure, the DBS will restart and replication will continue to function via the remaining RSGs.

GoldenGate Director

GoldenGate Director is a graphical enterprise application that allows administrators to define, configure, manage, and report on all GoldenGate transactional data synchronization processes. It can manage multiple GoldenGate instances from a remote client or the web.

Director components include:

• Server application: Provides services that control security, host information services, object modeling, diagramming, consolidated event logging, and alert services.

• Client: Provides a GUI for managing GoldenGate instances.

• Web: Allows remote, browser-based monitoring and control of GoldenGate instances without the need to install software on the client system.

• Administrator: Configures the Director server. You can use it to add or remove GoldenGate instances and users and to manage the overall Director configuration.

Chapter 2: Teradata Replication Solutions ArchitectureFor More Information


Figure 15: GoldenGate Director

For more information on the requirements for running GoldenGate Director, see “Hardware and Software Requirements” on page 19. For more information on the installation, components, and use of GoldenGate Director, see the GoldenGate Director Administration Guide.

For More Information

A discussion of change data capture on the Teradata server appears in the SQL Reference: Data Definition Statements.

A discussion of how to use the GoldenGate EXTRACT, REPLICAT, and GoldenGate Director programs appears in documentation available from GoldenGate software:

• GoldenGate Administrator Guide for Windows and UNIX

• GoldenGate Reference Guide for Windows and UNIX

• GoldenGate Director Administration Guide

• Practice Labs and a tutorial for GoldenGate for Teradata are provided during GoldenGate training

For detailed information about accessing GoldenGate documentation, see Chapter 4: “Sources of Further Information.”

Director Database

Director ServerClients GoldenGate Instances

1152A002

DirectorWeb

DirectorClient

DirectorAdministrator


CHAPTER 3 System Administration of TeradataReplication Solutions

This chapter summarizes the steps involved in implementing and maintaining an installation of Teradata Replication Solutions. These steps cover setup, configuration, and security.

• Task Overview: Setting Up Teradata Replication Solutions

• Teradata Transaction Commit Protocols

• Data Flow Commands

• Transaction Sequencing

• Character Sets and Replication

• LARGE DECIMAL Restriction

• Disk Capacity Considerations

Task Overview: Setting Up Teradata Replication Solutions

Before using Teradata Replication Solutions, administrators must perform the following setup procedures:

• Setting Up Teradata Access Module

• Creating the RSG Spill File Directory

• Defining RSG vprocs

• Setting Replication Security

• Creating a Replication Group

For information about each of these tasks, see the following sections.

Setting Up Teradata Access Module

To set up Teradata Access Module:

• Make sure that any running replication operations have terminated.

• Obtain the Teradata Access Module from the Downloads Center on the Teradata website (http://www.teradata.com/). Select Resources, Downloads, Drivers, and TAM (for Teradata Access Module).

• Verify that the GoldenGate software is installed on the replication server and locate the installation directory.


Chapter 3: System Administration of Teradata Replication SolutionsTask Overview: Setting Up Teradata Replication Solutions


• Copy the TAM library file into the installation directory created by the GoldenGate software installation. If Teradata Access Module is provided in compressed format (for example, as a .zip file), extract the file into this directory.

Creating the RSG Spill File Directory

If a transaction generates unusually large volumes of change data, Teradata Database uses a spill file on disk to store the transaction data. The RSG creates a directory for the spill file automatically if none exists. You should ensure that there is sufficient disk space for the directory as part of preparing the source system. The default paths are as follows:

The threshold defining when a spill file will be used is a tunable parameter in DBSControl. For more information, see Utilities.

For more information about this replication requirement, see Database Administration.

Defining RSG vprocs

Because the process of capturing change data from a source Teradata Database relies on RSG vprocs, system administrators must define RSG vprocs on the source system. The Parallel Upgrade Tool (PUT) provides the Configure Teradata user interface for adding one RSG to every node. For information about using this interface, see the appropriate document for your Teradata Database platform:

• Parallel Upgrade Tool (PUT) for Microsoft Windows User Guide

• Parallel Upgrade Tool (PUT) for UNIX MP-RAS and Linux User Guide

Setting Replication Security

The REPLCONTROL privilege provides security for Teradata Replication Solutions. The privilege is a total system right in the same manner that ROLE and PROFILE rights are total system rights; that is, the privilege is not granted and revoked on specific tables, databases, or users. The super-user DBC can grant the REPLCONTROL privilege to any other user along with the grant option.

The REPLCONTROL access privilege controls:

• Defining and managing replication groups

• Executing SQL statements that cause data values to be changed for a table when that table is in a state that would not normally allow changes to be made

Operating System Spill File Directory

MP-RAS /var/tdrsg

Windows TDAT\tdTemp\tdrsg

where TDAT is typically C:\Program Files\NCR\TDAT

Linux /var/opt/teradata/tdtemp/tdrsg

Chapter 3: System Administration of Teradata Replication SolutionsTeradata Transaction Commit Protocols


For information about defining the REPLCONTROL privilege, see SQL Reference: Data Definition Statements and Database Administration.

Creating a Replication Group

To replicate changes, first define the group of tables that you want to capture changes for as a replication group using the CREATE REPLICATION GROUP statement. Tables specified for inclusion in a replication group must be base tables. In addition, the definition for any table specified for inclusion in a replication group must already exist before you can submit a CREATE REPLICATION GROUP request containing it. For more information about the SQL syntax for defining, changing, and using replication groups, see SQL Reference: Data Definition Statements.

Teradata Transaction Commit Protocols

If tables defined as part of a replication group have constant update activity, Teradata Database continually sends the update information to GoldenGate for Teradata. This information is sent in the form of messages that contain:

• Begin and end transaction records, which delimit each set of change records for an SQL transaction.

• A count of change records in the transaction.

There are two modes of transaction commit available for a replication group.

Handling Disconnections from the Server

If the servers or GoldenGate for Teradata become disconnected, Teradata Database handles the uncommitted data depending on the transaction commit mode chosen.

Under Maximum Protection Mode

If GoldenGate for Teradata disconnects from the server, under maximum protection mode, the Teradata Database server still processes transactions, but they are not committed back to the client. Processing of transactions continues up to the time specified by the “Client Reset Timeout” field of DBSControl. If GoldenGate for Teradata:

Commit Mode Description

Maximum Protection Offers the highest level of data protection for replication. Transactions are not committed on the Teradata Database server until GoldenGate for Teradata has acknowledged receipt of all data for the transaction.

Maximum Performance Offers the capture of change data with the minimum impact to normal Teradata Database performance. Transactions are committed on the Teradata Database server and back to the client concurrently with the transmission of the transaction data to GoldenGate for Teradata.



• Reconnects before the time limit expires, then the system sends GoldenGate for Teradata the changed data for all transactions held by the server. The transactions are committed to the client.

• Does not reconnect before the time period expires, Teradata Dynamic Workload Management blocks all incoming transactions to tables in replication groups. The DBS also aborts all transactions not acknowledged by GoldenGate for Teradata. This may include some transactions actually sent to GoldenGate for Teradata before the disconnection occurred. The system records each such transaction as a row in the DBC.InDoubtResLog table. When GoldenGate reconnects after the time out period, TAM reads the rows in DBC.InDoubtResLog to find the in-doubt transactions that were aborted by the server. TAM notifies GoldenGate to abort those transactions.

If the server resets before GoldenGate for Teradata reconnects, any outstanding transactions remain in the in-doubt state, and the timer is started again (regardless of whether GoldenGate for Teradata was connected at the time of the server reset). If the timer subsequently expires before GoldenGate for Teradata reconnects, the outstanding transactions are aborted. When GoldenGate for Teradata reconnects (after an outage either of GoldenGate for Teradata or of the server) before the time period expires, then GoldenGate for Teradata and Teradata Database coordinate to determine which in-doubt transactions were received by GoldenGate for Teradata; the received transactions are then committed, while the remaining in-doubt transactions are rolled back.

Note: When the connection to the replication server is re-established, Teradata Dynamic Workload Management should permit incoming transactions to tables in replication groups. However, if Teradata Dynamic Workload Management is busy when the connection is re-established, the re-connection will fail and a 3310 error will be returned. The user must retry the connection to the replication server. The DBS will be unaffected.

Note: Maximum Protection mode employs a two-phase commit protocol between the source server and the replication server to ensure that no transactions are lost or applied more than once in the event of failure of either the source or subscriber server, TAM or GoldenGate software, or communication links. Use of this protocol results in increased transaction latency and reduced throughput. If minimal latency and maximum throughput are required, use Maximum Performance mode.

Under Maximum Performance Mode

If GoldenGate for Teradata is ever unavailable to receive change data, updates to replicated tables are allowed to continue. The system captures and maintains the change data on the server up to the time limit specified by Client Reset Timeout field of DBSControl.

If GoldenGate for Teradata reconnects before the time limit expires, the saved transaction change data is sent to GoldenGate for Teradata. When the time limit is reached, maintenance of change data is discontinued, but updates to the tables of the replication group are allowed to continue. Change data already captured by the server is discarded.

If the server resets while GoldenGate for Teradata is:

• Disconnected, the change data for those transactions committed during that period is lost.



• Connected, transactions committed back to the client but not yet fully received by GoldenGate for Teradata are lost. Transactions not yet committed to the client are rolled back, and an error is returned to the client process.

Caution: If, when operating in Maximum Performance mode, the server running GoldenGate for Teradata disconnects while Teradata is attempting to transmit transaction change data, the data for that transaction will be lost. For recovery of transactions that might be lost in the event of failure, use of checkpoints and other resynchronization techniques is recommended.

Replication Server Failure

RSGs on the source server must be able to detect and respond to the loss of connections to the replication server. A direct disconnect is reported as an error condition from the socket I/O function, so it is detected immediately, allowing the RSGs to gracefully clean up their connections. An example of a direct disconnect is termination of GoldenGate’s EXTRACT program.

If the replication server drops off of the network, however, no error condition is reported. If the connection to GoldenGate is idle, this disconnection will not be detected. When an RSG attempts to send change data to the replication server, TCP can detect a disconnection, and will signal the RSGs to take appropriate action. Following an replication server failure, either restart Teradata Database to force a complete recovery, or generate replication activity, which will cause the RSGs to immediately detect the failure and drop their connections.

Handling Backlog Change Data

Transaction change data is collected in memory on the RSG vprocs until the end transaction record is received. Teradata Database then transmits the change data to GoldenGate for Teradata if it is connected. If a transaction generates unusually large volumes of change data, Teradata Replication Solutions spills the in-memory transaction data to disk on the nodes on which RSG vprocs are configured. The system writes to a separate disk file for each transaction so that transactions can be transmitted to GoldenGate for Teradata and deleted from disk as they are committed. Teradata Replication Solutions generates a unique file name for the data of each transaction. The system writes all files to the spill file directory, located as specified in “Creating the RSG Spill File Directory” on page 34.

If the disk space becomes exhausted, the effect is the same as if the timeout interval had expired. A single error log entry is created when this event occurs.

Determining the Commit Mode for a Replication Group

Submit a HELP REPLICATION GROUP statement to get the group name, group identifier, table states, and transaction commit mode for a specific replication group.

IF the Protection Mode column is… THEN Teradata Replication Solutions is using…

Y maximum protection mode.

N maximum performance mode.



For example:

In addition to the commit mode reported by the MaxProtect column, you can also obtain the status of a replication group and its tables. The status also indicates what SQL statements are permitted on the tables in that group. The codes reported are as follows:

HELP REPLICATION GROUP receivables_group;

*** Help information returned. One row.

*** Total elapsed time was 1 second.

GroupName---------------------

Identifier----------

Operation---------

MaxProtect----------

Status------

receivables_group 1176 C Y I

***Help information returned. 2 rows.

DatabaseName---------------------

TableName------------------

Identifier-------------

Operation---------

Status------

payables invoices 000090070000 C I

payables transactions 000091070000 C I

Code Status Description

NULL Defined The table has been defined as a member of a replication group. Any type of SQL statement is accepted for the table.

C Connected GoldenGate for Teradata has connected to the server to start a data capture operation. DDL and DML statements are no longer accepted for the table. SQL SELECT statements are accepted.

F Failed GoldenGate for Teradata has connected for change data capture, initiated the activity, but is no longer responding to any TCP/IP communications from the server. No DDL statements are accepted for the table.

If the replication group was connected for maximum protection mode, then transactions in process containing DML statements are aborted. New requests containing DML statements for the tables of the replication group are held by the server until GoldenGate for Teradata has reconnected.

If the replication group was connected for maximum performance mode, then DML statements are accepted for the table.

I Initiated GoldenGate for Teradata has sent a control command to the server to initiate data capture. DDL is no longer accepted. DML statements are accepted.

Chapter 3: System Administration of Teradata Replication SolutionsData Flow Commands


Tables that are members of a replication group cannot be restored from archive. To restore them, drop them from the replication group first.

For more information on HELP REPLICATION GROUP syntax or restrictions on replication group tables, see “SQL Help and Database Object Definition Tools: HELP and SHOW” in SQL Reference: Data Definition Statements.

Data Flow Commands

Commands that influence the flow of data from the Teradata Database server to the replication server include:

• Suspend, which causes the Teradata Database to stop accepting DML change data statements for one or more tables of the replication group. The system rejects DML statements with an error response. The system completes SQL change statements in the process of execution and then sends them as change records to the replication server.

• Terminate, which causes Teradata Database to stop collecting and sending changed data to the replication server. A Terminate command from the replication server first sends a Suspend command followed by the actual Terminate command.

• Resume, which causes the server to start accepting and executing SQL statements that change data for one of the suspended tables of the replication group. Resume also causes the state of the table to be set to initiated on Teradata Database server. For more information about replication group table states, see “SQL Data Definition Language Statement Syntax” in SQL Reference: Data Definition Statements.

Transaction Sequencing

Teradata Replication Solutions ensures that change records on the subscriber are ordered in the same sequence that the source transactions were executed.

S Suspended GoldenGate for Teradata has sent a suspend command to the server for the table. No DDL or DML statements are accepted for the table. SQL SELECT statements are accepted.

T Terminated GoldenGate for Teradata has issued a terminate command for a change data capture operation for the replication group. Any type of SQL statement is accepted for the table.

Code Status Description

Chapter 3: System Administration of Teradata Replication SolutionsCharacter Sets and Replication


Character Sets and Replication

GoldenGate for Teradata supports replication of the following character sets between Teradata servers:

• LATIN (single byte character set)

• UNICODE (double-byte character set)

• KANJISJIS (multi-byte character set)

GoldenGate for Teradata also supports replication of the following character sets between different system types:

• LATIN

• UNICODE

Wherever UNICODE and KANJISJIS are used, the following limitations exist:

• Source and subscriber character sets must be the same type.

• Data must be sent in “Pass thru” mode. No filtering, transformations, or changing of character sets is allowed.

• GoldenGate for Teradata supports only ASCII-based table and column names. UNICODE and KANJISJIS table names and column names are not supported.

• Replication is supported via a Windows-based replication server only.

LARGE DECIMAL Restriction

Change Data Capture does not work in the current release if a table in a replication group contains a LARGE DECIMAL data type. Therefore, do not include any tables with LARGE DECIMAL data types in a replication group. This issue will be fixed as soon as possible.

Disk Capacity Considerations

When estimating disk capacity needs for replication, you need to account for occasions when the subscriber server might be down for a period of time and unapplied data accumulates. To determine your disk-space requirements in these situations, consider the following:

• Longest expected downtime

• Average rate of inserts, updates, and deletes for each table being replicated

• Row size per table for each of insert, update, and delete, as stored in the Golden Gate trail

Caution: Avoid allowing replication to proceed for an extended period with capture rates exceeding apply rates.

To ensure GoldenGate for Teradata frees disk space of data that is no longer needed after extracts are complete, set the PURGEOLDEXTRACTS parameter in the Manager parameter

Chapter 3: System Administration of Teradata Replication SolutionsEnabling Encryption


file. For more information about this option, see the GoldenGate Reference Guide for Windows and UNIX.

Enabling Encryption

By configuring the TAM parameter file, you can choose whether or not to encrypt replication messages. If you choose encryption, you can include control messages, data messages, or both.

For more information about replication encryption, see these documents:

• Security Administration

• GoldenGate Administrator Guide for Windows and UNIX

Chapter 3: System Administration of Teradata Replication SolutionsEnabling Encryption



CHAPTER 4 Sources of Further Information

This section provides information on how to find additional documentation available to support Teradata Replication Solutions.

GoldenGate Documentation and Training

GoldenGate Software provides the following documentation and training to support users of Teradata Replication Solutions:

• The GoldenGate Administrator Guide for Windows and UNIX introduces GoldenGate components and explains how to plan for, configure, and implement GoldenGate on the Windows, UNIX, and Linux platforms.

• The GoldenGate Reference Guide for Windows and UNIX provides detailed information about GoldenGate parameters, commands, and functions.

• The GoldenGate Director Administration Guide explains how to use GoldenGate Director to define, configure, manage, and report on all GoldenGate transactional data synchronization processes.

• GoldenGate requires training for customers purchasing GoldenGate Replication Products. As part of this training, customers receive the GoldenGate Tutorial and Practice Labs documents as learning aids. These materials cover how to set up GoldenGate for Teradata and run replication in various server configurations.

The GoldenGate documentation is available from the GoldenGate support web site (http://support.goldengate.com/). Access to this site requires the user name and password you received during GoldenGate training.

http://support.goldengate.com

http://support.goldengate.com

Chapter 4: Sources of Further InformationTeradata Replication Documentation by Task


Teradata Replication Documentation by Task

This section provides a quick reference to replication tasks and concepts and the documentation that describes them.

Orange Book

For an excellent discussion of best practices, limitations, troubleshooting, and performance considerations, see the orange book for Teradata Replication Solutions. (For access, register through Teradata @ Your Service, available from the “Training/Support” link on the Teradata web site.)

For information about... See this document...

How to use replication as a data-protection strategy

Database Administration (B035-1093-067A)

How to set up Teradata Database for replication

SQL Reference: Data Definition Statements (B035-1144-067A)

How to grant users the REPLCONTROL privilege for creating, altering, and dropping replication groups.


Statements for adding and managing replication groups such as ALTER REPLICATION GROUP


Names of tables and views that contain replication group names and associated information

Data Dictionary (B035-1092-067A)

How Teradata Database utilities handle insert, update, and delete functions that generate replication source data

Teradata Parallel Data Pump Reference (B035-3021-067A)

Teradata MultiLoad Reference (B035-2409-067A)


Index

AActive Data Warehousing 17

BBacklog change data 37BEGIN LOGGING 21Bidirectional replication

conflict resolution 23with different releases of Teradata Database 20

CCapture module 27Change Data Capture 21Character set usage 40COMMENT ON GROUP 21Components

GoldenGate for Teradata 27Teradata Replication Solutions 12

COMPRESSUPDATES feature 27Conflict resolution in bidirectional replication 23CREATE REPLICATION GROUP 21, 35

DData acquisition 17Data flow commands 39Data integrity 28Database systems supported 18Definition of terms 13Delivery module 28Director, GoldenGate 31Disconnection from server

error during reconnection 36handling 35

Disk capacity considerations 40Down node handling 31Dual Active Solutions 16

EEncryption 41END LOGGING 21Error 3310 36EXTRACT program 28

FFailure

reconnection to replication server 36recovery 40replication server 37

FastLoad 22

GGoldenGate Director 31

hardware and software requirements 19GoldenGate documentation and training 43GoldenGate for Teradata 27

character sets 40components 27operation 29support information 5update compression 27

Graphical user interface 31

HHardware requirements 19

IInformation Products Publishing Library 6

KKANJISJIS 40KEYCOLS parameter

use in resolving bidirectional replication conflicts 23

LLARGE DECIMAL restriction 40

MMaximum Performance mode 35Maximum Protection mode 35Modes, transaction commit 35MultiLoad 22

NNode failure 31

Index


OOrange book 44Ordering publications 6

PParallel Upgrade Tool. See PUTPublications 6PUT 34

RReferential integrity 28Relay Services Gateway. See RSGReleases supported 3REPLCONTROL 34REPLICAT program 28Replication

benefits 15components 12defined 11documentation by task 44server failure 37terms defined 13typical configuration 12

Requirements, hardware and software 19Restriction

LARGE DECIMAL data type 40Resume (data flow command) 39RSG 22

defining 34spill file 34unavailability 31

SServer

disconnection 35error during reconnection 36

SET SESSION 21Software requirements 19SQL GRANT 21Stored Procedures

use in resolving bidirectional replication conflicts 23Support 5, 6Supported database systems 18Supported releases 3Suspend (data flow command) 39System administration 33

TTable copy, stopping 30Teradata Access Module 22

compatibility with Teradata Database versions 20

Teradata Database support information 6Teradata Dual Active Solutions 16Teradata Dynamic Workload Management and server resets

36Teradata Query Director 16Teradata Replication Solutions

benefits 15components 12introduction 11overview 11set-up 33

Terminate (data flow command) 39Terminology 13TPump 22Training 43Transaction commit modes 35Transaction sequencing 39

UUNICODE 40Updates, compressing 27

Teradata Replication Solutions Overview - …dbmanagement.info/Books/MIX/1152_TeraData.pdf ·...

Documents

Transcript of Teradata Replication Solutions Overview - …dbmanagement.info/Books/MIX/1152_TeraData.pdf ·...