Data Replication With Oracle GoldenGate - doag.org · PDF fileOracle GoldenGate Architecture:...

BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA

HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART VIENNA ZURICH

Data Replication With Oracle GoldenGate – Looking Behind The Scenes –

Robert Bialek Principal Consultant Partner

Who Am I

Principal Consultant, Partner and Trainer at Trivadis GmbH in Munich

– [email protected]

Focus:

– Oracle Database High Availability

– Database Architecture/Internals

– Backup/Recovery

– Troubleshooting/Performance Tuning

– Linux Administration

Trainer for the following Trivadis courses

– Oracle Grid Infrastructure (O-GRINF)

– Oracle Real Application Cluster (O-RAC)

– Oracle Data Guard (O-DG)

2 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes

Agenda

3

1. Introduction

2. Oracle GoldenGate Architecture

3. Oracle GoldenGate Topologies

4. Demo – Integrated Extract/Replicat

5. Oracle GoldenGate Licensing

6. Core Messages

21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes

Introduction


What is Oracle GoldenGate?

Oracle strategic Middleware software product used for data extraction and replication

– Use-cases: data/service high-availabillity, “zero-downtime” upgrades/migrations,

easy fallback (minimize risk)

Moves only committed transactions across TCP/IP network without Oracle Net

Services

– In contrast to Oracle Streams…

Supports

– asynchronous data transfer (DML, DDL)

– heterogeneous sources and targets (databases, operating systems)

– different extraction, propagation and apply methods

– different topologies, mappings, transformations, filtering, error handling, …


Why Oracle GoldenGate?

Oracle Streams is deprecated in the Oracle 12c version (but still supported!)

– https://docs.oracle.com/database/121/UPGRD/deprecated.htm

According to the Oracle GoldenGate Statement of Direction

– http://www.oracle.com/technetwork/middleware/goldengate/overview/statement-of-

direction-gg-132120.pdf

Beginning with the version 12.1.0.2, the non-CDB architecture is also deprecated

– As of now, Streams is not supported with the new CDB architecture

Oracle Streams continues to be supported but will not be enhanced to

support any new database features in future releases. This means that

Streams will continue to support exactly what it supports today (11.2)

in future releases. It also means that any new database features, data

types, or capabilities will not be supported by Streams.


Oracle GoldenGate Architecture


Oracle GoldenGate Architecture: Big Picture


Source DB Target DB

Primary

(Regular)

Extract

Primary

(Passive)

Extract

Secondary

Extract

Data Pump

Collector

Replicat

Local Trail Remote Trail

Primary

(Alias)

Extract

Op

tio

nal

Oracle GoldenGate Instance Oracle GoldenGate Instance

Static

Configuration

Runtime

Configuration

Runtime

Configuration

Static

Configuration

Optional Optional

Manager

(VIP) Manager

(VIP)

MONITOR MONITOR

Primary Extract Group: Capture

Primary Extract can run in either

– Classic Capture mode (all database versions)

– Integrated Capture mode (full support: 11.2.0.3+)

Extracts and optionally filters/transforms the transactions from the source database

– Only for configured objects

Many Extracts per source database can operate at the same time

Only committed transactions (CSN) will be propagated to the target system

– Optionally written to a local trail (recommended)

– Without a local trail, the primary Extract send the data to the target system

– Not committed transactions will be discarded from memory or dirtmp directory

9

Primary

(Regular)

Extract

Primary

(Passive)

Extract

21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes

Source DB

Primary Extract Group: Classic Capture

Classic Capture mode

– Traditional mode, reads directly Oracle database redo logs

– Able to mine redo logs in ASM (two methods)

– All database versions supported

With some restrictions. For example no support for:

– BASIC, OLTP, Exadata Hybrid Columnar

compression

– PDML on RAC database

– CDB architecture!

DDL, TDE support – no out-of-the box support

Not transparent to RAC configuration changes (#threads)


Source DB

Primary

(Regular)

Extract

Secondary

Extract

Data Pump Local Trail

Redo Logs

OS

System Calls

Primary Extract Group: Integrated Capture

Integrated Capture mode

– Interacts with the database logmining server

– Required to capture data from a CDB architecture

– Fully transparent with RAC, TDE, ASM

– Full support for databases 11.2.0.3+

– Source database 11.2.0.4+ does not require

supplementary objects for DDL support

– Supports more complex data types than classic

capture

– Integrated capture deployments

• Local or a downstream deployment

The best choice for most deployments


Source DB

Primary

(Regular)

Extract

Secondary

Extract


Redo Logs Logmaining

Server

LCR LCR

Secondary Extract Group (Data Pump): Propagation

Optional but recommended component

– Reads the local trail and propagates the data to the

remote trail (via target collector process)

– Data can be transferred “as is” (PASSTHROUGH)

– Optionally filters/transforms the transactions from the source database

(NOPASSTHROUGH)

– Isolates the primary extract from TCP/IP timeouts

The network transfer can be optionally encrypted

– Not necessary, in case the trail is already encrypted


Encryption

Secondary

Extract


Replicat Process: Apply

The Replicat process can run in either

– Classic Replicat mode (nonintegrated) - all database versions

– Coordinated Replicat mode (nonintegrated) - all database versions

– Integrated Replicat mode – database versions 11.2.0.4+

The Replicat process reads the remote trail, reconstructs

the SQL and applies them to the target database

– Optionally filters/transforms the data

The application of the data can be optionally delayed (DEFERAPPLYINTERVAL

parameter)


Target DB

Replicat

Remote Trail

Replicat Group: Classic/Coordinated Mode

Classic Replicat

– Constructs SQL statements from the trails

and applies the SQL through OCI

– Supported for all database versions

– Transactions are applied sequentially

(single-threaded)

Coordinated Replicat

– Transactions are applied in parallel (multi-threaded)

– There is 1 coordinator plus

many apply threads

– Supported for all database

versions

– GoldenGate 12c required


OCI

SQL

Target DB

Replicat

Remote Trail

OCI

SQL

Target DB

Remote Trail

Coordinator

Thread 1

Thread 2

Thread n

Replicat Group: Integrated Mode

Integrated Replicat

– Supported for target database

versions 11.2.0.4+

– Constructs LCRs and transmits

DMLs to the database inbound

server

Some changes (e.g. DDL) are not supported directly by the inbound server

– In those cases the Replicat process disconnects from the inbound server and

executes the transaction in direct apply mode through OCI

With many transactions applied in direct apply mode it might be necessary to switch

to nonintegrated Replicat


LCR

Target DB

Replicat

Remote Trail

Database

Inbound

Server

SQL

Capture/Apply: CDB & Pluggable Databases

Extract must operate in integrated mode

– Extract must connect to the CDB$ROOT container as a common user (c##)

– One Extract can capture from one or many PDBs to a single trail

Replicat can connect and apply only to one PDB (one Apply/PDB, connect using

service)

Source DB – CDB

CDB$ROOT PDB1 PDB2

Target DB – CDB

CDB$ROOT PDB1 PDB2

Primary Extracts

INTEGRATED Data Pumps Collectors

Replicats


common_user_prefix

(12.1.0.2)

Capture & Apply: Different Combinations Possible

Different combinations are possible

– Classic Extract with integrated or classic/coordinated Replicat

– Integrated Extract with integrated or classic/coordinated Replicat

You can switch between the modes with some configuration changes


Source DB Redo Logs

Logmaining

Server

LCR LCR

LCR

Target DB

Replicat(s)

Database

Inbound

Server OS

System Calls

Primary

Extract(s)

OCI

SQL

Classic Integr

ated

Integr

ated

Classic Coordi

nated

Oracle GoldenGate Topologies



Active-Passive

– Data replication only in one direction

– Only one set of primary Extract, Data Pump and Replicat groups

Active/Standby

– Additional inactive primary Extract, Data Pump and Replicat groups

– Switchover: Stop one set of process Start the inactive one



Active-Active

– Data replication in both directions

– Two active sets of primary Extract, Data Pump and Replicat groups

– With some restrictions: e.g.: DDL-support restricted to two databases with identical

metadata

Additionally broadcasting, cascading, consolidation is possible


Demo – Integrated Extract/Replicat


Oracle GoldenGate: Setup Flow


Architecture

Design

Software Installation/

Patches Database Preparation

Manager Configuration

Primary Extract, Data

Pump, Replicat

Configuration Files

Target Database

Instantiation

Register/Start Primary

Extract, Data Pump

Register/Start Replicat Cluster Integration /

INIT Scripts

Tuning/Customizing/

Maintenance/…

1 2 3

6 5 4

7 8 9

n

Oracle GoldenGate: Demo Environment


Source DB: 2-nodes RAC 11.2.0.4

Schema: APP1

GoldenGate version: 12.1.2.1.2

Active/Passive data replication

CDB$ROOT APP1 APP2

Primary Extract

INTEGRATED

Target DB: CDB 2-nodes RAC 12.1.0.2

PDB: APP1, Schema: APP1

GoldenGate version: 12.1.2.1.2

Active/Passive data replication

APP3

Schema: APP1

Local Trail

Secondary Extract

Data Pump

RemoteTrail

Collector

Replicat

INTEGRATED

Licensing


GoldenGate Licensing

Please contact your Oracle sales representative for additional/more detailed

information!

Licensed customers of Oracle Database Enterprise Edition who purchase Oracle

GoldenGate are entitled to a full use of

– Oracle Active Data Guard

– Oracle Global Data Services (12.1.0.1+)

Oracle Store (https://shop.oracle.com)

– 304.00€ / Named User Plus (min. 25 NUPs per physical CPU)

– 15,194.00€ / Processor

You need to fully license the source and the target environment

– Independent of the topology

In a cluster environment all nodes need to be licensed

– Soft-Partitioning cannot be used to limit the amount of licenses!


GoldenGate Licensing – Example


DB1 DB2

APP1

APP2

APP3

APP1

APP4

APP5

GoldenGate replication for the schema APP1

DB1

Standby database

GG replication will be activated

after loss of the primary system only

GG license

3 nodes

GG license

3 nodes

Oracle 10-day

failover rule

applies

Core Messages


Core Messages

GoldenGate is a very powerful replication software framework

For most deployments the integrated Extract/Replicat is the best choice

Supplemental logging generates more undo and redo

– In some cases you need even to supplementally log all columns!

Script all the configuration steps

– Very useful if you need to re-setup the whole environment

You don’t need to replace your current Streams installation ASAP by Oracle

GoldenGate , but

– Streams is deprecated in the version 12c and will not be enhanced in future

releases

– There is a framework to ease the migration from Streams to GoldenGate

(streams2ogg)

– As of now Streams is not supported with CDB architecture

– For new projects it might be a better choice (licensing)!


Questions and answers…

Robert Bialek

Principal Consultant

Partner

Tel. +49 89 99275930

[email protected]


Data Replication With Oracle GoldenGate - doag.org · PDF fileOracle GoldenGate Architecture:...

Documents

Transcript of Data Replication With Oracle GoldenGate - doag.org · PDF fileOracle GoldenGate Architecture:...