Dbvisit replicate: logical replication made easy

of 32 /32
Infrastructure at your Service. Dbvisit replicate: logical replication made easy

Embed Size (px)

Transcript of Dbvisit replicate: logical replication made easy

1. Dbvisit replicate: logical replication made easyInfrastructure at your Service.Dbvisit replicate:logical replication made easy 2. Dbvisit replicate: logical replication made easy 07.11.2014Infrastructure at your Service.About mePage 2Franck PachotSenior consultantOracle Technology LeaderMobile +41 79 963 27 [email protected]s, Articles, blogs, forums 3. Dbvisit replicate: logical replication made easyExperts At Your Service>35 specialists in IT infrastructure>Certified, experienced, passionateBased In Switzerland>100% self-financed Swiss company>Over CHF 4 mio. turnoverLeading In Infrastructure Services>More than 100 customers in CH, D, & F>Over 30 SLAs dbi FlexServicecontracted07.11.2014dbiservicesWho we arePage 3 4. Dbvisit replicate: logical replication made easy1.Why logical replication?2.Demo (setup)3.Configuration alternatives4.Demo (run)5.Why Dbvisit?07.11.2014AgendaPage 4 5. Dbvisit replicate: logical replication made easyLogical vs. Physical07.11.2014Why Logical replicationPage 5REDODBSQLstatementsREDO LOGrecordsREDO LOGrecordsApplied on tablesApplied on filesApplied on filesAPPLYPhysical replication>Redo based>Oracle EE or Dbvisitstandbyredoredo 6. Dbvisit replicate: logical replication made easyLogical vs. Physical07.11.2014Why Logical replicationPage 6REDODBSQLstatementsREDO LOGrecordsREDO LOGrecordsApplied on tablesApplied on filesApplied on filesSQLstatementsApplied on tablesAPPLYtriggermviewlogsqlLogical replication>Application based>Oracle Streams 7. Dbvisit replicate: logical replication made easyLogical vs. Physical07.11.2014Why Logical replicationPage 7REDODBREDO LOGrecordsApplied on tablesApplied on filesApplied on filesSQLstatementsApplied on tablessqlLogical replication>Redo based>Golden Gate or DbvisitreplicateSQLstatementsMINEredoREDO LOGrecordsAPPLY 8. Dbvisit replicate: logical replication made easyPhysical replication>High performance, exact copy (HA)>Same version (platform, database, application)>No filter or transformation (same tables, columns, indexes, partitions)>Oracle Data Guard (EE), Dbvisitstandby, Manual standby and scriptsLogical replication based on redo>More complex (supported datatypes, conflict resolution)>Destination can differ (version, data model, data)>We choose what we replicate (filter, change deletes to inserts)>Destination is Read-Write, can be 2-ways>Oracle Golden Gate, Dbvisitreplicate, StreamsLogical vs. Physical07.11.2014Why Logical replicationPage 8 9. Dbvisit replicate: logical replication made easyOffloaded query environments>Reporting, Data QualityDistributed databases>local proxies replicated to masterData warehousing and Audit>Extract data for ETL, can be real timeAudit>Stores all modifications (old an new values)Migration projects>Oracle Edition downgrade (go to Standard Edition)>Oracle upgrades, OS platform, to the Cloud, etc.>11g to PDB (without non_cdb_to_pdb.sql)What are the use cases?07.11.2014Why logical replicationPage 9 10. Dbvisit replicate: logical replication made easy1.Why logical replication2.Demo (setup)3.Configuration alternatives4.Demo (run)5.Why Dbvisit?07.11.2014AgendaPage 10 11. Dbvisit replicate: logical replication made easyhttp://www.dbvisit.com/repattack/07.11.2014Demo (setup)Page 11 12. Dbvisit replicate: logical replication made easyTwo VirtualBoxVM:>source>Linux Centos 6.5, Oracle XE 11g with XE database>Swingbenchschema, data and application>target>Linux Centos 6.5, Oracle XE 11g with XE database (empty)Downloaded and installed dbvrepon both machines>As root, on source and target:>Run wizard: http://screencast-o-matic.com/u/nTb6/dbvisit-repattack-setupDemo environment07.11.2014Demo (setup)Page [email protected]# rpm [email protected]# dbvrepDbvrep> setup wizard 13. Dbvisit replicate: logical replication made easySetup wizard and initialization07.11.2014Demo (setup)Page 13Setup WizardDescribedatabases(ttorcl_srcand ttorcl_trg)Definereplicationpairs (initializationwithimpdpflashback_scn)Definereplicatedtables (all REPOE schema)Configure processes(MINE on source, APPLY on target)InitializationRunall.sh (createsdbvrepschema)Copy scripts (seeNextsteps.txt)CreateDBLINKrunapply.sh (impdp)Start replicationStart MINE processStart APPLY processRunstart-console.sh 14. Dbvisit replicate: logical replication made easy1.Why logical replication2.Demo (setup)3.Configuration alternatives4.Demo (run)5.Why Dbvisit?07.11.2014AgendaPage 14 15. Dbvisit replicate: logical replication made easyEditions and Features07.11.2014Configuration alternativesPage 15LTDReplicatedata for The simplest: impdpnetwork_link=from targetdp_exp>Create a dump file with expdpon sourceexp>Obsolete exp/imp (old versions)ddl_run>Get metadata as DDLOther alternatives:>RMAN duplicate, Storage snapshots, etc.Initialization alternatives to import data07.11.2014Configuration alternativesPage 21sql 22. Dbvisit replicate: logical replication made easy>Which changes are replicated by import?>Which changes are replicated by MINE?>We need to MIN complete transactionsWe need to lock in Share mode to stop transactions when getting the SCNInitialization alternatives to get SCN07.11.2014Configuration alternativesPage 22Transaction 1Transaction 3Transaction 5Transaction 2Trans. 4Trans. 6?Transaction 1Transaction 3Trans. 5Transaction 2Trans. 4Trans. 6SCNlockData transferredby initializationData transferredby replicationData transferredby initializationData transferredby replication 23. Dbvisit replicate: logical replication made easyWe must MINE complete transactions: cannot initialize when we have current transaction -> Lock in Share mode.one-by-one>Locks table one by one and get SCN for each table ->import one by onesingle-scn>Locks all tables to get one SCN ->easier importddl-only>no lock, no data ->starts with empty tables (for audit)resetlogs>Using standby or duplicate instead of importno-lock (since 2.7)>Get SCN for the oldest transaction, no lock>When supplemental logging must be already setInitialization alternatives to get SCN07.11.2014Configuration alternativesPage 23 24. Dbvisit replicate: logical replication made easy1.Why logical replication2.Demo (setup)3.Configuration alternatives4.Demo (run)5.Why Dbvisit?07.11.2014AgendaPage 24 25. Dbvisit replicate: logical replication made easySetup wizard and initialization07.11.2014Demo (setup)Page 25Setup WizardDescribedatabases(ttorcl_srcand ttorcl_trg)Definereplicationpairs (initializationwithimpdpflashback_scn)Definereplicatedtables (all REPOE schema)Configure processes(MINE on source, APPLY on target)InitializationRunall.sh (createsdbvrepschema)Copy scripts (seeNextsteps.txt)CreateDBLINK and runapply.sh (impdp)Start replicationStart MINE processStart APPLY processRunstart-console.sh 26. Dbvisit replicate: logical replication made easyIssue>Table has no primary key and has duplicates (all columns have same values)>By default, a redo record replication must touch one and only one row>When we delete on source, the sqlgenerated affect 2 rowsConflict too many rows>The first redo record received touches 2 rows>We force do delete those 2 rows>Then the second redo record touches no rows, we ignore itManual conflict resolution07.11.2014Additional demosPage 26dbvrep> list conflictdbvrep> resolve conflict resolve as FORCEdbvrep> list conflictdbvrep> resolve conflict resolve as IGNORE 27. Dbvisit replicate: logical replication made easyConflict handlers>For automatic resolution, we choose to add a where rownum=1 in the generated sql.>Syntax is available from online helpConflict happen in logical replication, especially>With delete cascade constraints>With triggers doing DML>With 2-way replicationSolution: either disable on target, or set conflict handlersAutomatic conflict resolution07.11.2014Additional demosPage 27dbvrep> set_conflict_handlersfor table REPOE.DEMODUP for DELETE on TOO_MANY to sqls/$/ and rownum= 1/dbvrep> helpdbvrep> help set_conflict_handlers 28. Dbvisit replicate: logical replication made easyReasons:>Reorganization on source>Bad conflict resolution, easier to restart then resolve1. Stop replication>The prepare gives the SCN for the initialization2. Initialize the target table3. Restart replicationResynchronize a table07.11.2014Additional demosPage 28dbvrep> unpreparetable REPOE.DEMODUPdbvrep> pause applydbvrep> prepare table REPOE.DEMODUPSQL> truncate table REPOE.DEMODUPSQL> insert /*+ append */ into REPOE.DEMODUP select * from [email protected]_srcas of scndbvrep> resume 29. Dbvisit replicate: logical replication made easy1.Why logical replication2.Demo (setup)3.Configuration alternatives4.Demo (run)5.Why Dbvisit?07.11.2014AgendaPage 29 30. Dbvisit replicate: logical replication made easyDbvisitReplicate vs. GoldenGatevs. Streams07.11.2014Why Dbvisit?Page 30What DbvisitsaysMain differences we see:>GoldenGatecurrently has more features but>Dbvisitis per socket (CHF 2500 to 6000) Monthly rental possible>Dbvisitoptimistic apply: mine and apply dont wait for commit>Dbvisitis easy to start (setup wizard) 31. Dbvisit replicate: logical replication made easyKnow if you need logical replication>Its not the magic solution. Its a project.Testing (POC) is easy>Install a Dbvisitreplicate configuration in one hour>test it on your data (datatypes, operations,etc.)Logical replication is a DevOpsproject>Operations is involvedsupplemental logging, redo/archived, filesystemstorage, network>Application is involveddefine conflict rules, trigger behavior, supportddatatypes,Core message07.11.2014Why Dbvisit?Page 31 32. Dbvisit replicate: logical replication made easy 07/11/2014Page 32Any questions? Please do ask.Infrastructure at your Service.We look forward to working with you!dbiservices is the Dbvisitpartner in Switzerland>We offer fixed prices for Dbvisitstandby>A POC for Dbvisitreplicate is few days>We have direct access to supportRepattackon 20thNov. in Nuremberg (DOAG)>Dbvisitreplicate on your laptop