Epsilon – Oracle Sharding Use Case in Public Cloud · 2018-12-06 · • Epsilon is...

14
©2014 Epsilon. Private & Confidential Epsilon – Oracle Sharding Use Case in Public Cloud Gairik Chakraborty Senior Director, Database Administration, Epsilon 1

Transcript of Epsilon – Oracle Sharding Use Case in Public Cloud · 2018-12-06 · • Epsilon is...

Page 1: Epsilon – Oracle Sharding Use Case in Public Cloud · 2018-12-06 · • Epsilon is all-encompassing global marketing company, we are global leader in turning data -driven marketing

©2014 Epsilon. Private & Confidential

Epsilon– OracleShardingUseCaseinPublicCloud

GairikChakrabortySeniorDirector,DatabaseAdministration,Epsilon

1

Page 2: Epsilon – Oracle Sharding Use Case in Public Cloud · 2018-12-06 · • Epsilon is all-encompassing global marketing company, we are global leader in turning data -driven marketing

Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|

• SnapshotofEpsilon• OracleShardingUseCaseinPublicCloud

Agenda

Page 3: Epsilon – Oracle Sharding Use Case in Public Cloud · 2018-12-06 · • Epsilon is all-encompassing global marketing company, we are global leader in turning data -driven marketing

©2014 Epsilon D

ata Managem

ent, LLC. Private & C

onfidential

3

• Epsilonisall-encompassingglobalmarketingcompany,wearegloballeaderinturningdata-drivenmarketingintopersonalizedcustomerexperienceandlastingrelationships

• Morethan9000associatesand70officesworldwide

• Largestpermission-basede-mailerintheworld,deliveringover75billionemailsannually

• World’sleadingsourceofdatawithinformationcoveringover1.5Bindividualrecordsand278Mdevices

• Morethan2,000globalclients,including26oftheFortune10049outof10TopBanks48outof10TopRetailers49outofTop10PharmaceuticalCompanies

EpsilonataGlance

Page 4: Epsilon – Oracle Sharding Use Case in Public Cloud · 2018-12-06 · • Epsilon is all-encompassing global marketing company, we are global leader in turning data -driven marketing

Copyright ©

Epsilon 2017 E

psilon Data M

anagement, LLC

. All rights reserved.

4

Wedeliverpersonalizedconnections,buildloyaltyanddrivebusinessforbrandsaroundtheworld

Page 5: Epsilon – Oracle Sharding Use Case in Public Cloud · 2018-12-06 · • Epsilon is all-encompassing global marketing company, we are global leader in turning data -driven marketing

©2014 Epsilon D

ata Managem

ent, LLC. Private & C

onfidential

5

OracleShardingUseCaseinPublicCloud

5

Page 6: Epsilon – Oracle Sharding Use Case in Public Cloud · 2018-12-06 · • Epsilon is all-encompassing global marketing company, we are global leader in turning data -driven marketing

©2014 Epsilon D

ata Managem

ent, LLC. Private & C

onfidential

6

• Loyaltyrealtimemarketingsolutionwithextremeavailability,performanceandscalabilityrequirement

• Deploymentshouldsupportanypubliccloudaswellasonpremisesifneeded–productofferingiscloudfirst

• PlatformsupportsrealtimePOSintegrationwithaveragecalltimefrom200-500msaverageandeachAPIcallcanhavemultiple(20–100)SQLsinternally

• Scaleoutsolutionwhichcangrowflexiblybasedonworkloaddemand

• Scalable,reliableandhighlyavailableinfrastructurealongwithindustrystandardsecurityandauditing

HighLevelOLTPApplicationBusinessRequirements

Page 7: Epsilon – Oracle Sharding Use Case in Public Cloud · 2018-12-06 · • Epsilon is all-encompassing global marketing company, we are global leader in turning data -driven marketing

©2014 Epsilon D

ata Managem

ent, LLC. Private & C

onfidential

7

WhywelikeOracleSharding

• Loyaltymarketingsolutionincludesprocessingofhighvolumeofcomplexfinancialtransactionwhichrequiresstrongmultiversionconcurrencycontrol,dataprotection,security,etc.§ NoSQLsolutionsarenotverygoodfitforthoseusecases

• Applicationisusingmanycomplexjoinsandasolutionneedstosupporttheexistingcodewithminimalchange

• Useallotherexistingoraclefeaturelikedataconsistency,security,availability,robustperformanceoptimizer,backupandrecoverywhicharekeyforbusinesscriticalapplicationdeploymentandarealreadypartofOracleShardingframework

• UsingOracleSharding,OLTPdatabasecanscaleupandscaleoutandweexpecttouselessernumberofshardsbasedonworkloaddemand

Page 8: Epsilon – Oracle Sharding Use Case in Public Cloud · 2018-12-06 · • Epsilon is all-encompassing global marketing company, we are global leader in turning data -driven marketing

©2014 Epsilon D

ata Managem

ent, LLC. Private & C

onfidential

8

ConsiderationbeforeusingOracleSharding

• Applicationdatamodelneedstochangetosupportsharding

• Identifytransactiontableswhichcanbesharded,referencetableneedstobeduplicated

• Selectappropriateshardingkeyandshardingmethod

• NondefaultglobaldatabaseservicecreatedusingGSDCTLneedstobeusedforconnection

• DataaccessforOLTPshoulduseshardingkeyasmuchaspossibletoavoidcrossshardoperationwhichisexpensiveintermsofexecutiontime

• Applicationneedtousedriverwhichhasshardingsupportinbuilt.ExampleincludesOracleJDBC,OCI,ODP.NET(unmanageddriver),OracleUCP,etc.

• Shardcatalogsetupisprotectedwithactivedataguardwithmaximumavailabilityprotectionandfaststartfailoverenabled

Page 9: Epsilon – Oracle Sharding Use Case in Public Cloud · 2018-12-06 · • Epsilon is all-encompassing global marketing company, we are global leader in turning data -driven marketing

©2014 Epsilon D

ata Managem

ent, LLC. Private & C

onfidential

9

OLTPSystemDeploymentArchitectureatPublicCloud

Page 10: Epsilon – Oracle Sharding Use Case in Public Cloud · 2018-12-06 · • Epsilon is all-encompassing global marketing company, we are global leader in turning data -driven marketing

©2014 Epsilon D

ata Managem

ent, LLC. Private & C

onfidential

10

Application Load Balancer

Web Servers – ODP.NET Connection Pool

Web Service Call

OLTP Batch Reports

OLTP Service

Batch Service

Site 1

Primary DB

Application Load Balancer

Web Servers – ODP.NET Connection Pool

OLTP Batch Reports

Report Service

Site 2

Standby DB

ApplicationServicePlacement:CurrentState

FAN/FCF

ActiveDataGuard

Web Service Call

FAN/FCF

Page 11: Epsilon – Oracle Sharding Use Case in Public Cloud · 2018-12-06 · • Epsilon is all-encompassing global marketing company, we are global leader in turning data -driven marketing

©2014 Epsilon D

ata Managem

ent, LLC. Private & C

onfidential

11

Application Load Balancer

Web Servers– ODP.NET Connection Pool

Web Service Call

OLTP Batch Reports

OLTP Service

Batch Service

Site 1

Application Load Balancer

Web Servers – ODP.NET Connection Pool

OLTP Batch Reports

Report Service

Site 2

Standby DB

ApplicationServicePlacement:TargetStatewithSharding

FAN/FCF

ActiveDataGuard

Web Service Call

FAN/FCF

Shard Directors

Shard Catalog

Shard Catalog

Shard Director

Primary Shards Standby Shards

Page 12: Epsilon – Oracle Sharding Use Case in Public Cloud · 2018-12-06 · • Epsilon is all-encompassing global marketing company, we are global leader in turning data -driven marketing

©2014 Epsilon D

ata Managem

ent, LLC. Private & C

onfidential

12

POCResult– HorizontalScalability

Page 13: Epsilon – Oracle Sharding Use Case in Public Cloud · 2018-12-06 · • Epsilon is all-encompassing global marketing company, we are global leader in turning data -driven marketing

©2014 Epsilon D

ata Managem

ent, LLC. Private & C

onfidential

13

Best Practices

• GlobalserviceusedforODP.NETneedstohave–notificationsettoTRUEusingGDSCTLwhileconfiguringserviceforFAN

• ClientsideTNSshouldpointtomultiplesharddirectorsforhighavailabilityinsteadofshardnodes.

• Backupsshouldnotrunduringchunkmovement(eitherduringre-shardoperationormanualchunkmovement)asthatwillnothavecorrectdatalayout.

• Duringrecovery,validateorrecovershardoptionmaybeneededtosyncitwithshardcatalog

Page 14: Epsilon – Oracle Sharding Use Case in Public Cloud · 2018-12-06 · • Epsilon is all-encompassing global marketing company, we are global leader in turning data -driven marketing

©2014 Epsilon D

ata Managem

ent, LLC. Private & C

onfidential

14

Future Plans

• CollaboratewithoracleengineeringteamtoconvertEpsilonloyaltysolutiontobeshardingcapable,configureapplicationforzerodowntimeusingfeatureslikeapplicationcontinuityduringshardfailureorre-shardoperation,etc.

• PlantodeployOracleshardingformanycustomersasnextgenerationtruehorizontallyscalablemultiplecloudglobalsolutionacrossanyregionaspartofEpsilon’sgloballoyaltyplatformdeploymentplan


MySQL HA Sharding-Fabric

MySQL HA Sharding-Fabric

Using Oracle ShardingContents 1 Oracle Sharding OverviewWhat is Sharding 1-2 About Oracle Sharding 1-3 Benefits of Oracle Sharding 1-4 Example Applications using Database Sharding

Using Oracle ShardingContents 1 Oracle Sharding OverviewWhat is Sharding 1-2 About Oracle Sharding 1-3 Benefits of Oracle Sharding 1-4 Example Applications using Database Sharding

Introduction to MongoDB sharding

Introduction to MongoDB sharding

Proxysql sharding

Proxysql sharding

Five data models for sharding - PGConf ASIA · • Not sharding is always easier than sharding • Identify your sharding approach/key early, denormalize it even when you’re small

Five data models for sharding - PGConf ASIA · • Not sharding is always easier than sharding • Identify your sharding approach/key early, denormalize it even when you’re small

The Future of Postgres Sharding - Bruce Momjian · The Future of Postgres Sharding BRUCE MOMJIAN This presentation will cover the advantages of sharding and future Postgres sharding

The Future of Postgres Sharding - Bruce Momjian · The Future of Postgres Sharding BRUCE MOMJIAN This presentation will cover the advantages of sharding and future Postgres sharding

Mongo db sharding guide

Mongo db sharding guide

MongoDB: Replication,Sharding,MapReduce

MongoDB: Replication,Sharding,MapReduce

Sharding Architectures

Sharding Architectures

Sharding Overview

Sharding Overview

Data sharding

Data sharding

Sharding morning session

Sharding morning session

Lightning Talk: MongoDB Sharding

Lightning Talk: MongoDB Sharding

Data sharding and replication - Vargas-Solarvargas-solar.com/cloud-data-management/wp-content/...Sharding and replication n Sharding with no replication: unique copy, distributed data

Data sharding and replication - Vargas-Solarvargas-solar.com/cloud-data-management/wp-content/...Sharding and replication n Sharding with no replication: unique copy, distributed data

Oracle sharding : Installation & Configuration

Oracle sharding : Installation & Configuration

Cignex mongodb-sharding-mongodbdays

Cignex mongodb-sharding-mongodbdays

Epsilon Email Marketing Spring 2012 Gallery

Epsilon Email Marketing Spring 2012 Gallery

Sharding Using Spockproxy_ A Sharding-only Version of MySQL Proxy Presentation

Sharding Using Spockproxy_ A Sharding-only Version of MySQL Proxy Presentation

Michał Gruchała - Data sharding

Michał Gruchała - Data sharding

Introduction to sharding

Introduction to sharding