1

Oracle GoldenGate 12c Solutions, Uni-Directional Configuration and IOUG Data Integration SIG

Bobby Curtis, Accenture Enkitec Group Joseph deBuzna, Oracle

2

Agenda

•  Oracle GoldenGate 12c Solutions •  Oracle GoldenGate 12c Uni-Directional

Configuration •  IOUG Data Integration Special Interest Group

(SIG)

3

Legacy/Traditional Replication and ELT Tools

Tools: 1.  Export/Import (Datapump/Traditional) 2.  Advanced Replication 3.  Streams 4.  Warehouse Builder

Replication/Transforming data has been around in some form for a really long time.

Other Venders Tools: 1.  Dell Shareplex 2.  SQL Server Transactional Replication/Sync Framework

4

Present/Future trends of Replication and ELT Tools

Question:

What product can help meet these trends?

•  The need to always move/transform data will continue to increase in the future.

•  Requirements for real-time data/metrics will continue to increase.

•  More data sources will be required regardless of vendor (i.e. Oracle, IBM, Microsoft, etc…)

5

Present/Future trends of Replication and ELT Tools

Question:

What product can help meet these trends?

0

10

20

30

40

50

60

70

80

1 2 3 4 5 6 7 8 9 10

Data (GB)

Years

6

Oracle GoldenGate Solutions

Unidirectional Query Off-loading/Reporting Application integration

Active-Active Bidirectional with CDR Complex Active Meshes

Consolidation Data Warehouse Single Source of Truth Mergers & Acquisitions

Distribution Data dissemination Cascade replication

Hub & Spoke Centralized sharing Data distribution

Zero downtime migrations High Availability

7

Scenario ACME company wants to build an online reporting system that will provide guarantee access to data for a critical business system.

Requirements: 1.  Provide real-time data for reporting 2.  Map data between different schemas 3.  Maintain availability of data at remote site

8

Uni-Directional Solution

One way replication between databases

Used for: 1.  Query offloading/Reporting 2.  Near-Zero Downtime Migrations 3.  Application Integration 4.  High-Availability Support

ACME.* ACME_RPT.*

9

What is required? Processes to ensure replication:

1.  Manager 2.  Extract (capture) 3.  Data Pump (extract – shipping) 4.  Replicat (apply)

ACME.* ACME_RPT.*

10

Manager Process

Performs: 1.  Starts/Restarts processes 2.  Manage port numbers (static/dynamic) 3.  Trail File Management 4.  Create reports for events, errors and

thresholds

The manager process is the controller process within the Oracle GoldenGate environment.

11

Extract Process

Performs: 1.  Reads online redo log/archive logs

(transactional logs) 2.  Writes extracted transactional data to trail files

The extract process is used to perform change data capture from the source database.

12

Data Pump Process

Performs: 1.  Protects against network failures 2.  Consolidate data from multiple sources 3.  Distribute data to multiple targets 4.  Used for file adaptors (Flat File / Java)

The data pump process is a secondary extract that is used to help send data over a network.

13

Replicat Process

Performs: 1.  Applies transactions in chronological order as trail files

are processed 2.  Maps and transforms data 3.  Performs Conflict Detection and Resolution

The replicat process is the apply process within the Oracle GoldenGate environment.

14

Trail Files

Performs: 1.  Stores transactions in chronological order 2.  Created by extract 3.  Can be read by extract/replicat

Oracle GoldenGate binary files used to store transactions in chronological order for shipping and applying to the target side.

15

GLOBALS File (Optional) •  File used store global parameters that relate to the Oracle GoldenGate

as a whole.

•  All parameters in file apply to all processes for the instance

•  Parameters in file can be overridden by process specific parameters

•  Does not need to exist before starting processes rather is read by each process as they are started

Example: GGSCI> edit params ./GLOBALS

16

Uni-Directional Breakdown

Capture Trail Files Pump Delivery

Trail Files

Capture (extract): committed transactions are captured (and can be filtered) as they occur by reading the transaction logs. As of V.11.2.1, GoldenGate offers two options for capture for Oracle; Classic & Integrated Capture

Trail: stages and queues data for routing.

Pump: distributes data for routing to target(s).

Route: data is compressed, encrypted for routing to target(s).

Delivery: applies data with transaction integrity. New with GoldenGate 12c, Integrated Delivery (replicat).

MGR MGR

17

Parameter Files Parameter files is where the Oracle GoldenGate magic is performed.

Editing parameter files can be done two ways:

1.  From GoldenGate Service Command Interface (GGSCI) 2.  Edited using a text editor (vi, textpad, etc..)

Example: GGSCI> edit params <process name>

18

Manager Process Parameter File --Parameter files will vary per environment-- PORT 15000 DYNAMICPORTLIST 15010-15035 PURGEOLDEXTRACTS ./dirdat/*, USECHECKPOINTS, MINKEEPDAYS 2 AUTORESTART ER *, RETRIES 6, WAITMINUTES 2, RESETMINUTES 30 LAGCRITICALSECONDS 30 LAGREPORTMINUTES 60 LAGINFOMINUTES 0

19

Extract Process Parameter File --Parameter files will vary per environment-- -- CHECKPARAMS EXTRACT EXTI SETENV (ORACLE_HOME="/u01/app/oracle/product/11.2.0/db_3") SETENV (ORACLE_SID="orcl") USERID <ggate user>, PASSWORD <pwd> TRANLOGOPTIONS DBLOGREADER WARNLONGTRANS 1h, CHECKINTERVAL 30m EXTTRAIL ./dirdat/lt TABLE ACME.*;

20

Data Pump Process Parameter File --Parameter files will vary per environment-- --CHECKPARAMS EXTRACT PMPI PASSTHRU RMTHOST 10.16.10.10, MGRPORT 15000, COMPRESS RMTTRAIL ./dirdat/rt TABLE ACME.*;

21

Replicat Process Parameter File --Parameter files will vary per environment-- -- CHECKPARAMS REPLICAT REPC SETENV (ORACLE_HOME="/u01/app/oracle/product/11.2.0/db_3") SETENV (ORACLE_SID="orcl") USERID <ggate user>, PASSWORD <pwd> ASSUMETARGETDEFS DISCARDFILE ./dirrpt/REP.dsc, append, megabytes 500 MAP ACME.*, target ACME_RPT.*;

22

GLOBALS File (Optional) --Parameter files will vary per environment— CHECKPOINTTABLE <ggate user>.checkpointtable GGSCHEMA <ggate user> ENABLEMONITORING –enables the Jagent for monitoring

23

Adding processes Add Extract

GGSCI> add extract exti, tranlog, begin now GGSCI> add exttrail ./dirdat/lt, extract exti, megabytes 500

Add Data Pump GGSCI> add extract pmpi, exttrailsource ./dirdat/lt GGSCI> add rmttrail ./dirdat/rt, extract pmpi, megabytes 500

Add Replicat GGSCI> dblogin userid <gguser> password <ggpwd> GGSCI> add replicat repc, exttrail ./dirdat/rt

*Manager is added by default with installation

24

Successful Setup

Capture Trail Files Pump Delivery

Trail Files

Source: Target:

MGR MGR

25

IOUG Data Integration SIG

26

IOUG Data Integration SIG

•  Initially discussed at Oracle Open World 2014 •  Focus/Products •  Resources •  Board of Directors

27

IOUG Data Integration SIG

•  Initially discussed at Oracle Open World 2014 •  Popularity of Oracle GoldenGate and Oracle Data

Integrator is rising •  Launch of Oracle Metadata Management

28

IOUG Data Integration SIG

Focus/Products:

IOUG DI SIG will focus on Oracle’s data integration tools

•  Oracle GoldenGate •  Oracle Data Integrator •  Oracle Metadata Management •  Many others…

29

IOUG Data Integration SIG Resources:

•  Community Site (ioug.org or linkedin.com) •  Regular webcasts (quarterly) •  SIG meetings at Collaborate and Oracle

Open World •  Activities promoted on social media sites •  Sharing IOUG content with community

30

IOUG Data Integration SIG

Current Board of Directors:

Bobby Curtis, AEG, @dbasolved, http://dbasolved.com

Seth Miller, Collier IT, @seth_m_miller, http://sethmiller.org

Danny Bryant, AEG, @dbcapoeira, http://dbaontap.org *There are spots opened if you would like to participate

31