Oracle GoldenGate Presentation from OTN Virtual Technology Summit - 7/9/14 (PDF)
-
Upload
bobby-curtis -
Category
Software
-
view
326 -
download
2
description
Transcript of Oracle GoldenGate Presentation from OTN Virtual Technology Summit - 7/9/14 (PDF)
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Understanding Oracle GoldenGate Architecture
Bobby CurCs Senior Technical Consultant Accenture Enkitec Group July 9, 2014
Oracle ConfidenCal – Internal/Restricted/Highly Restricted 1
Performance Tuning to the OS Level
Speaker Info • Douglasville, Georgia • Senior Technical Consultant • Expert OEM12c (2013)/PracCcal ODA
(2014) • TwiYer: @dbasolved • Blog: hYp://dbasolved.com • Email: [email protected] [email protected]
Accenture Enkitec Group
Enkitec has joined Accenture’s Infrastructure Services as Accenture Enkitec Group
• 17,000 Infrastructure Services professionals • 52,000 Oracle professionals
Focus on Oracle Engineered Systems SoluCons • Database MigraCons & TransformaCons • Database-‐as-‐a-‐Service • Oracle ApplicaCons on Engineered Systems • Cloud-‐based soluCons leveraging Engineered
Systems
Agenda
• General InformaCon • What is GoldenGate • Benefits • Basic Architecture • Use Cases
• Tuning InformaCon • GoldenGate Tuning • OperaCng System Tuning
• Processes • Memory • Disk
General InformaCon
What is Oracle GoldenGate • Comprehensive soeware for real-‐Cme data
integraCon in heterogeneous environments • Benefits
• High Availability SoluCons • Real-‐Time Data IntegraCon
• TransformaCon of Data • TransacConal Change Data Capture
• Data ReplicaCon • VerificaCon of Data (Veridata)
• Primary ReplicaCon Tool in Oracle Database 12c (12.1.0.1) • Oracle Upgrade Guide, SecCons 8.1.6 & 8.1.7
• MulCple ways to successfully monitor Oracle GoldenGate
Benefits
• Real-‐Time Data Capture • Near-‐Zero DownCme migraCons
• Quickly upgrade databases • Open Architecture (Heterogeneous) • Disaster Recovery • Data DistribuCon
• Distributed Offices • Query Off-‐loading
• ReporCng Instances
Basic Architecture
Capture: commiYed transacCons are captured (and can be filtered) as they occur by reading the transacCon logs.
Trail: stages and queues data for rouCng.
Pump: distributes data for rouCng to target(s).
Route: data is compressed, encrypted for rouCng to target(s).
Delivery: applies data with transacCon integrity, transforming the data as required.
Source Oracle / Non-Oracle Database
Target Oracle / Non-Oracle
Database
Capture
Delivery
Trail Files
Pump
Trail Files
Pump
Delivery
Capture
Bi-directional
LAN / WAN / Internet Over TCP/IP
Trail Files
Trail Files
Modular Architecture
Use Cases Unidirec8onal Query Offloading Zero-‐DownCme MigraCon Data IntegraCon Cloud or On-‐Premise
Bi-‐Direc8onal AcCve-‐AcCve for MulC-‐Master/HA Cloud or On-‐Premises
Data Distribu8on via Messaging
Cloud Apps Integra8on
Big Data Delivery
Real/Time and Batch Delivery Structured Data to Data Reservoir
Tuning InformaCon
Performance Tuning
• Per Oracle • Baselining
• GoldenGate Tuning • Lag Times • AWR/ASH Reports
• CPU • Memory • I/O
• OperaCng System Tuning
Per Oracle
• MulCple Process Groups • Data Integrity • Number of Groups
• Dependent on amount of memory • Use Network Efficiently
• Benchmark Extract/Replicat (checkpoints)
• Eliminate disk I/O • Writes data sequenCally, RAID 0+1
• Manage VM and Paging • Cachemgr
Baselining
• Should be first thing you gather • What should be gathered?
• GoldenGate • Lag • Checkpoint Info • General AWR InformaCon (DB)
• OperaCng System • CPU performance (mpstat) • Memory performance (vmstat) • I/O performance (iostat)
GoldenGate Lag Info
• ReporCng Lag • Gives an idea of how long it takes data to transfer on network
• GGSCI commands • lag extract <group name> • lag replicat <group name>
GoldenGate Checkpoint Info
• Checkpoint • Helps detect network boYlenecks • GGSCI commands
• info extract <group name>, showch <number>
Database Tuning
• AWR Reports • Wait Event: Streams Miscellaneous Event
CPU InformaCon
• Use mpstat • mpstat –P ALL [count [interval]]
• sar can be used as well
Memory Performance
• Use vmstat • Reports on virtual memory staCsCcs • vmstat –S M [count [interval]]
Disk I/O Tuning
• Use iostat • Reports staCsCcs transfered between devices • iostat –m [count [interval]]
Baseline Completed
Metric Baseline Running
Lag 5 sec (ext)/12 sec (rep) 1 sec (ext)/4 sec (rep)
CPU 4.64% (iowait (all)) 7.73% (iowait(all))
Memory 2214M (free) 2109M (free)
Disk IO 29.16 tps 29.06 tps
• ObservaCon: • Serial processing (no parallel) • Baseline was taken with higher transacCon count • Disk I/O was slightly lower with high CPU waits • More memory was used during the “running” test
Tune GoldenGate
Tune GoldenGate • MulCple Process Groups (Classic)
• Parallel Replicat Groups • 3 to 1 raCo (replicat to trail (classic))
• Reduced with Integrated Replicat • DB (11.2.0.4) and OGG 12c (12.1.2)
Source Oracle & Non-‐Oracle Database(s)
Target Oracle & Non-‐Oracle
Database(s)
Capture Trail Files
Pump Delivery Trail Files
Delivery
Delivery LAN / WAN / Internet Over TCP/IP
Tune GoldenGate • MulCple Process Groups (Classic)
• Parallel Extract Groups • 1 to 1 raCo (extract to trail to replicat)
• Reduced with Integrated processes
Source Oracle & Non-‐Oracle Database(s)
Target Oracle & Non-‐Oracle
Database(s)
Trail Files
Pump Trail Files
Delivery
Delivery Delivery
LAN / WAN / Internet Over TCP/IP
Capture
Capture
Capture
Tune GoldenGate
• Use Network Efficiently • Lag Cmes increase
• Check target database performance • Consult Network Team for any performance issue
• Checkpoint Extract/Replicat • send replicat rep, status
• Indicates any delay with replicat
Tune GoldenGate
• Eliminate disk I/O • Try to eliminate this first • System ConfiguraCons
• Most shops, disk are controlled by SAN Admin or internal devices
• Choose RAID 0+1 over RAID 5 • GoldenGate ConfiguraCon
• Increase values of parameters: • CHECKPOINTSECS • GROUPTRANSOPS • EOFDELAY or EOFDELAYSECS
Tune GoldenGate
• Manage VM and Paging • Default: use as much memory allowed • Default: ./dirtmp and as much space as allowed
• Can be controlled with cachemgr parameter • Valid for Extract/Replicat • Current seyngs can be viewed in report file
Tune GoldenGate (cachemgr con’t)
• EsCmate swap space needed • Extract_Swap_Needed =(PROCESS_VM *
num_extracts) + (swap_other_processes) • Replicat_Swap_Needed =(PROCESS_VM *
num_replicats) + (swap_other_processes) • Review in report file
• send [extract/replicat] <group name>,
cachemanager
Profiling GoldenGate
Profiling the GoldenGate Processes
• perf • Used to profile OS processes and hardware performance
• strace • Used to see what processes are calling
• backtrace file
Profiling the GoldenGate Extract
• backtrace file output • InteresCng find with extract process (6 Threads = 1 extract)
Profiling the GoldenGate Extract
• Threads IdenCfied for extract • Number of threads increase/add extracts • 1 extract = 6 threads • 2 extracts = 12 threads, etc..
• Each thread takes/uses memory • Average Memory 22-‐50 MB
• 3.6-‐8.3 MB per thread
Conclusion
• Reviewed Oracle GoldenGate Architecture
• Discussed tuning aspects of Oracle GoldenGate for applicaCon and operaCng system
• Quickly looked at trace of an extract process
Speaker Info • Douglasville, Georgia • Senior Technical Consultant • Expert OEM12c (2013)/PracCcal ODA
(2014) • TwiYer: @dbasolved • Blog: hYp://dbasolved.com • Email: [email protected] [email protected]
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Understanding Oracle GoldenGate Architecture
Bobby CurCs Senior Technical Consultant Accenture Enkitec Group July 9, 2014
Oracle ConfidenCal – Internal/Restricted/Highly Restricted 35
Performance Tuning to the OS Level