Doag data replication with oracle golden gate: Looking behind the scenes
-
Upload
trivadis -
Category
Presentations & Public Speaking
-
view
493 -
download
1
Transcript of Doag data replication with oracle golden gate: Looking behind the scenes
Data Replication With Oracle GoldenGate – Looking Behind The Scenes –
Robert BialekPrincipal ConsultantPartner
Who Am I
Principal Consultant, Partner and Trainer at Trivadis GmbH in Munich
Focus:
– Oracle Database High Availability
– Database Architecture/Internals
– Backup/Recovery
– Troubleshooting/Performance Tuning
– Linux Administration
Trainer for the following Trivadis courses
– Oracle Grid Infrastructure (O-GRINF)
– Oracle Real Application Cluster (O-RAC)
– Oracle Data Guard (O-DG)
2 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Agenda
3
1. Introduction
2. Oracle GoldenGate Architecture
3. Oracle GoldenGate Topologies
4. Demo – Integrated Extract/Replicat
5. Oracle GoldenGate Licensing
6. Core Messages
21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Introduction
4 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
What is Oracle GoldenGate?
Oracle strategic Middleware software product used for data extraction and replication
– Use-cases: data/service high-availabillity, “zero-downtime” upgrades/migrations, easy fallback (minimize risk)
Moves only committed transactions across TCP/IP network without Oracle Net Services
– In contrast to Oracle Streams…
Supports
– asynchronous data transfer (DML, DDL)
– heterogeneous sources and targets (databases, operating systems)
– different extraction, propagation and apply methods
– different topologies, mappings, transformations, filtering, error handling, …
5 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Why Oracle GoldenGate?
Oracle Streams is deprecated in the Oracle 12c version (but still supported!)
– https://docs.oracle.com/database/121/UPGRD/deprecated.htm
According to the Oracle GoldenGate Statement of Direction
– http://www.oracle.com/technetwork/middleware/goldengate/overview/statement-of-direction-gg-132120.pdf
Beginning with the version 12.1.0.2, the non-CDB architecture is also deprecated
– As of now, Streams is not supported with the new CDB architecture
Oracle Streams continues to be supported but will not be enhanced to support any new database features in future releases. This means that Streams will continue to support exactly what it supports today (11.2) in future releases. It also means that any new database features, data types, or capabilities will not be supported by Streams.
6 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Oracle GoldenGate Architecture
7 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Oracle GoldenGate Architecture: Big Picture
8 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Source DB Target DB
Primary (Regular)Extract
Primary (Passive)Extract
Secondary Extract
Data Pump
Collector
Replicat
Local Trail Remote Trail
Primary (Alias)Extract
Optio
nal
Oracle GoldenGate Instance Oracle GoldenGate Instance
Static Configuration
RuntimeConfigurati
on
RuntimeConfigurati
onStatic
Configuration
OptionalOptional
Manager (VIP)
Manager
(VIP)MONITORMONITOR
Primary Extract Group: Capture
Primary Extract can run in either
– Classic Capture mode (all database versions)
– Integrated Capture mode (full support: 11.2.0.3+)
Extracts and optionally filters/transforms the transactions from the source database
– Only for configured objects
Many Extracts per source database can operate at the same time
Only committed transactions (CSN) will be propagated to the target system
– Optionally written to a local trail (recommended)
– Without a local trail, the primary Extract send the data to the target system
– Not committed transactions will be discarded from memory or dirtmp directory
9
Primary (Regular)Extract
Primary (Passive)Extract
21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Source DB
Primary Extract Group: Classic Capture
Classic Capture mode
– Traditional mode, reads directly Oracle database redo logs
– Able to mine redo logs in ASM (two methods)
– All database versions supported
With some restrictions. For example no support for:
– BASIC, OLTP, Exadata Hybrid Columnar compression
– PDML on RAC database
– CDB architecture!
DDL, TDE support – no out-of-the box support
Not transparent to RAC configuration changes (#threads)
10 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Source DB
Primary (Regular)Extract
Secondary Extract
Data PumpLocal Trail
Redo Logs
OS System Calls
Primary Extract Group: Integrated Capture
Integrated Capture mode
– Interacts with the database logmining server
– Required to capture data from a CDB architecture
– Fully transparent with RAC, TDE, ASM
– Full support for databases 11.2.0.3+
– Source database 11.2.0.4+ does not require supplementary objects for DDL support
– Supports more complex data types than classic capture
– Integrated capture deployments
• Local or a downstream deployment
The best choice for most deployments
11 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Source DB
Primary (Regular)Extract
Secondary Extract
Data PumpLocal Trail
Redo LogsLogmaining Server
LCR
LCR
Secondary Extract Group (Data Pump): Propagation
Optional but recommended component
– Reads the local trail and propagates the data to the remote trail (via target collector process)
– Data can be transferred “as is” (PASSTHROUGH)
– Optionally filters/transforms the transactions from the source database (NOPASSTHROUGH)
– Isolates the primary extract from TCP/IP timeouts
The network transfer can be optionally encrypted
– Not necessary, in case the trail is already encrypted
12 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Encryption
Secondary Extract
Data PumpLocal Trail
Replicat Process: Apply
The Replicat process can run in either
– Classic Replicat mode (nonintegrated) - all database versions
– Coordinated Replicat mode (nonintegrated) - all database versions
– Integrated Replicat mode – database versions 11.2.0.4+
The Replicat process reads the remote trail, reconstructs the SQL and applies them to the target database
– Optionally filters/transforms the data
The application of the data can be optionally delayed (DEFERAPPLYINTERVAL parameter)
13 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Target DB
Replicat
Remote Trail
Replicat Group: Classic/Coordinated Mode
Classic Replicat
– Constructs SQL statements from the trails and applies the SQL through OCI
– Supported for all database versions
– Transactions are applied sequentially (single-threaded)
Coordinated Replicat
– Transactions are applied in parallel (multi-threaded)
– There is 1 coordinator plus many apply threads
– Supported for all database versions
– GoldenGate 12c required
14 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
OCI
SQL
Target DB
Replicat
Remote Trail
OCI
SQL
Target DB
Remote TrailCoordinat
or
Thread 1
Thread 2
Thread n
Replicat Group: Integrated Mode
Integrated Replicat
– Supported for target database versions 11.2.0.4+
– Constructs LCRs and transmits DMLs to the database inbound server
Some changes (e.g. DDL) are not supported directly by the inbound server
– In those cases the Replicat process disconnects from the inbound server and executes the transaction in direct apply mode through OCI
With many transactions applied in direct apply mode it might be necessary to switch to nonintegrated Replicat
15 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
LCR
Target DBReplicat
Remote Trail
Database
Inbound Server
SQL
Capture/Apply: CDB & Pluggable Databases
Extract must operate in integrated mode
– Extract must connect to the CDB$ROOT container as a common user (c##)
– One Extract can capture from one or many PDBs to a single trail
Replicat can connect and apply only to one PDB (one Apply/PDB, connect using service)
Source DB – CDBCDB$ROOT PDB1 PDB2
Target DB – CDB
CDB$ROOT PDB1 PDB2
Primary ExtractsINTEGRATED
Data Pumps CollectorsReplicats
16 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
common_user_prefix (12.1.0.2)
Capture & Apply: Different Combinations Possible
Different combinations are possible
– Classic Extract with integrated or classic/coordinated Replicat
– Integrated Extract with integrated or classic/coordinated Replicat
You can switch between the modes with some configuration changes17 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Source DBRedo Logs
Logmaining Server
LCR
LCR
LCR
Target DB
Replicat(s)
Database
Inbound
Server
OS System Calls
Primary Extract(
s)
OCI
SQL
Classic
Integrated
Integrated
Classic
Coordinate
d
Oracle GoldenGate Topologies
18 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Oracle GoldenGate Topologies
Active-Passive
– Data replication only in one direction
– Only one set of primary Extract, Data Pump and Replicat groups
Active/Standby
– Additional inactive primary Extract, Data Pump and Replicat groups
– Switchover: Stop one set of process Start the inactive one
19 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Oracle GoldenGate Topologies
Active-Active
– Data replication in both directions
– Two active sets of primary Extract, Data Pump and Replicat groups
– With some restrictions: e.g.: DDL-support restricted to two databases with identical metadata
Additionally broadcasting, cascading, consolidation is possible
20 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Demo – Integrated Extract/Replicat
21 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Oracle GoldenGate: Setup Flow
22 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
ArchitectureDesign
Software Installation/Patches Database Preparation
Manager Configuration
Primary Extract, Data Pump, Replicat
Configuration Files
Target Database Instantiation
Register/Start Primary Extract, Data
Pump
Register/Start Replicat
Cluster Integration / INIT Scripts
Tuning/Customizing/ Maintenance/…
1 2 3
6 5 4
7 8 9
n
Oracle GoldenGate: Demo Environment
23 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Source DB: 2-nodes RAC 11.2.0.4Schema: APP1GoldenGate version: 12.1.2.1.2Active/Passive data replication
CDB$ROOT APP1 APP2
Primary ExtractINTEGRATED
Target DB: CDB 2-nodes RAC 12.1.0.2PDB: APP1, Schema: APP1GoldenGate version: 12.1.2.1.2Active/Passive data replication
APP3
Schema: APP1
Local Trail
Secondary ExtractData Pump
RemoteTrail
Collector
ReplicatINTEGRATED
Licensing
24 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
GoldenGate Licensing
Please contact your Oracle sales representative for additional/more detailed information!
Licensed customers of Oracle Database Enterprise Edition who purchase Oracle GoldenGate are entitled to a full use of
– Oracle Active Data Guard
– Oracle Global Data Services (12.1.0.1+)
Oracle Store (https://shop.oracle.com)
– 304.00€ / Named User Plus (min. 25 NUPs per physical CPU)
– 15,194.00€ / Processor
You need to fully license the source and the target environment
– Independent of the topology
In a cluster environment all nodes need to be licensed
– Soft-Partitioning cannot be used to limit the amount of licenses!
25 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
GoldenGate Licensing – Example
26 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
DB1 DB2
APP1
APP2
APP3
APP1
APP4
APP5
GoldenGate replication for the schema APP1
DB1
Standby database
GG replication will be activatedafter loss of the primary system only
GG license3 nodes
GG license3 nodes
Oracle 10-day failover rule
applies
Core Messages
27 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Core Messages
GoldenGate is a very powerful replication software framework
For most deployments the integrated Extract/Replicat is the best choice
Supplemental logging generates more undo and redo
– In some cases you need even to supplementally log all columns!
Script all the configuration steps
– Very useful if you need to re-setup the whole environment
You don’t need to replace your current Streams installation ASAP by Oracle GoldenGate , but
– Streams is deprecated in the version 12c and will not be enhanced in future releases
– There is a framework to ease the migration from Streams to GoldenGate (streams2ogg)
– As of now Streams is not supported with CDB architecture
– For new projects it might be a better choice (licensing)!
28 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Questions and answers…Robert BialekPrincipal ConsultantPartner
Tel. +49 89 [email protected]
29 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes