ProxySQLUseCaseScenarios

Percona Live Amsterdam Oct 3-5 2016

Alkin Tezuysal René Cannaò

2

Who we are

•  Alkin Tezuysal Sr. Technical Manager, Percona @ask_dba

•  René Cannaò MySQL SRE, Dropbox / ProxySQL @proxysql

3

Top 5 reasons to use ProxySQL

•  Improvedatabaseopera:ons.

•  Understandandsolveperformanceissues.

•  Createaproxylayertoshieldthedatabase.

•  AddHigh-Availabilitytodatabasetopology.

•  EmpowertheDBAswithgreattool.

4

ProxySQL Highlights

Scalability HighAvailability

AdvancedQuerySupport Manageability

5

Use case overview - Scalability

Connec:onPoolingandMul:plexing

Addconnec:onpoolinglayer

Reduceconnec:onthreadoverhead

Read/WriteSplitMorescalabilityforanyMySQLtopology

Mustforanyheavyworkloads

Read/WriteSharding

Effortlessshardingimplementa:on

WithoutDev.supportandcostsavings

6

Use case overview - High Availability

SeamlessFailover

Gracefulfailoversupport

Queryrerou:ng

LoadBalancingAllowseasyscaling

ForbePeru:liza:onofservers

ClusterAwareSupportsPXC/Galeraimplementa:ons

Smallfootprintforbiggain

7

Use case overview - Advanced Queries

Querycaching Cachefrequentlyuseddata

Addsanothercachelayer

Queryrewrite AddSQLflexibility

Fasterproblemsolving

Queryblocking Adddatabaseawarefirewall

WithoutApp.support

8

Use case overview - Advanced Queries

QuerymirroringAudit,Analyze,Review,CacheWarming

Allowbenchmarkingonlivedata

QuerythroPlingSetpriori:za:onforimportantqueries

Allowsmanualinterven:ontoproblemqueries

Query:meout Addanother:meoutlayer

Flexiblequerylevel:meout

9

Use case overview - Manageability

AdminU:lity Authen:ca:onsupport

Limituseraccesstodatabasepool

Run:mereconfigura:on

Ontheflymodifica:ons

Configuredatabasewithoutrestartordown:me

Monitoring Statscollec:onandrepor:ng

Inves:gatedatabaseworkload

10

Scalability with ProxySQL - Connection Pooling

•  Any application without persistent connection to database

•  PHP applications to be specific without built in connection pool

Application MySQL ProxySQL

•  Reduces number of new connections to the database

11

Scalability with ProxySQL – Connection Multiplexing

•  Any application with persistent connection to database

•  Java applications to be specific with built in connection pools

Application Pool

MySQL

•  Reduce connections similar to Aurora

•  Testing being performed for 300K database connections

Application Pool Application

Pool

ProxySQL

12

Scalability with ProxySQL - Read/Write Split

•  On the fly Read / Write implementation

•  Use read_only flag to switch traffic

Application

MySQL Master

•  Load balancing made easy

MySQL Slave

Write hostgroups

Read hostgroups MySQL

Slave

13

Scalability with ProxySQL – User and schema level sharding

•  Granular sharding per username and schema

•  Backend pooling based on user activity to specific schema

Application User A

ProxySQL

•  Use advanced sharding to parallelize queries

•  Scale beyond the sharding per host limitations.

Application User B

MySQL Master

MySQL Slave

MySQL Master

MySQL Slave

MySQL Master

MySQL Slave

14

MySQL Slave MySQL Master

High Availability with ProxySQL – Seamless failover

•  Neither VIP setup nor service discovery needed

•  Use read_only flag to switch traffic

Application Pool MySQL Master

•  Integration with other HA Managers

Application Pool Application

Pool

ProxySQL

15

High Availability with ProxySQL – Improve Multi DC implementation

•  No connection pools latency on SSL connections.

•  Costly delays across the water.

MySQL Master ProxySQL

MySQL Master

ProxySQL

Application

DC1

DC2

16

High Availability with ProxySQL – Adoption to Clustering

•  PXC / Galera / Group Replication

MySQL

ProxySQL

• 

ProxySQL

Application

Node 1

Application

Application

MySQL

Node 2

MySQL

Node 3

17

Advanced Queries with ProxySQL – Caching

•  Improve performance on read intensive workloads

•  Reduce slave provisioning

Application A

ProxySQL

•  Improved query cache over default implementation

•  Query aware caching over general caching

Application B

MySQL Master

MySQL Slave

MySQL Slave

MySQL Slave

Cache

MySQL Slave

MySQL Slave

18

Advanced Queries with ProxySQL – Query Timeout

•  Built in query killer a.k.a query sniper

•  Adjustable threshold based on query rule

Application A

ProxySQL

•  No idle or long running queries

Application B

MySQL Master

MySQL Slave

MySQL Slave

MySQL Slave

Query Timeouts

MySQL Slave

MySQL Slave

19

Advanced Queries with ProxySQL – Firewall

•  Protect database from unwanted traffic

•  Stop unwanted user, account , application (new code)

Application A

ProxySQL

•  Fast modification to database behavior

•  Added protection for DDOS and other attacks.

Application B

Query Blocking

MySQL Master

MySQL Slave

MySQL Master

MySQL Slave

MySQL Master

MySQL Slave

20

Advanced Queries with ProxySQL – Query rewrite engine

•  Most wanted feature by DBAs

•  Rewrite queries overloading the database on the fly.

Application A

ProxySQL

•  Simply buy time until application can be modified

Application B

MySQL Master

MySQL Slave

MySQL Slave

MySQL Slave

Query Rewriting

MySQL Slave

MySQL Slave

21

Advanced Queries with ProxySQL – Query retry

•  Server failures or maintenance do not loose a query.

•  Fails over and resubmits the query to another host.

Application A

ProxySQL

•  Improved operation and new way of handling slaves.

Application B

MySQL Master

MySQL Slave

MySQL Slave

MySQL Slave

Query Retry

MySQL Slave

MySQL Slave

22

Advanced Queries with ProxySQL – Query routing

•  Selective routing based on importance.

•  Use metrics based on stats_mysql_query_digest and decide

Application A

ProxySQL

•  Go beyond read/write split.

Application B

MySQL Master

MySQL Slave

MySQL Slave

MySQL Slave

Query Routing

MySQL Slave

MySQL Slave

23

Advanced Queries with ProxySQL – Query mirroring

•  Mirror incoming queries to different back ends

•  Use it for audit, analytics, cache warming, benchmarking.

Application A

ProxySQL

Application B

Query Mirroring

MySQL Master

Host A

Host B

MySQL Slave

MySQL Master

MySQL Slave

24

Advanced Queries with ProxySQL

Querycaching QueryTimeout

QueryRetry QueryBlocking

QueryRou:ngandMirroring

25

Clustered ProxySQL at scale

Tested with:

●  8 app servers with 3k clients’ connections each (24k total) ●  4 middle layer proxysqls processing 4k connections each from local

proxysqls (16k total) ●  256 backends/shard (meaning 256 routing rules) processing 600

connections each (150k total) Single ProxySQL was tested with up to 150k connections At today, ProxySQL is able to process up to 750k QPS

26

Questions

3UR[\64/ LV DQHZ 3UR[\ ZLWK DQDGYDQFHG PXOWL�FRUH DUFKLWHFWXUH�,WV EXLOW IURP WKHJURXQG XS WR

VXSSRUW KXQGUHGVRI WKRXVDQGV RIFRQQHFWLRQVFXUUHQWO\�

PXOWLSOH[HG WRSRWHQWLDOO\KXQGUHGV RI

EDFNHQG VHUYHUV�

+,*+3(5)250$1&(

� 7KH GDWD JDWHZD\ �

6+$5',1*

3UR[\64/ LV DYHU\ SRZHUIXOSODWIRUP DEOH WRFRYHU PXOWLSOHVKDUGLQJ

VFHQDULRV ZLWK DQDGYDQFHGFRQILJXUDWLRQEDVHG RQ WKHHQRUPRXVIOH[LELOLW\ RIFRQFDWHQDWLQJVLPSOH SURFHVVLQJ

UXOHV�

)25 '%$V %< '%$V

7DNH EDFN FRQWURORI \RXU 0\64/FOXVWHU WRGD\�

6FDOH DQG RSHUDWHLW VPRRWKO\�8QORFN DQ

XQSUHFHGHQWHGOHYHO RI IOH[LELOLW\DQG SHUIRUPDQFH�$OO WKH ZKLOHJHWWLQJ WKH

XQOLPLWHG IUHHGRPWKDW FRPHV ZLWK D*3/ /LFHQVH�

+LJK 3HUIRUPDQFH 0\64/ 3UR[\ ZKHQ VFDODELOLW\ PDWWHUV

4XHU\ FDFKLQJ

6XSSRUWV IDLORYHU

5HDO�WLPH VWDWLVWLFV

$GYDQFHG FRQILJXUDWLRQ

7UDIILF PLUURULQJ $GYDQFHG WRSRORJ\ VXSSRUW

ZZZ�SUR[\VTO�FRP �SUR[\VTO

Proof 3UR[\64/ LV D UHYROXWLRQDU\ SUR[\ WKDW KHOSV

\RX VTXHH]H WKH ODVW GURS RI SHUIRUPDQFHRXW RI \RXU 0\64/ FOXVWHU� IHDWXULQJ +LJK$YDLODELOLW\� VHDPOHVV IDLORYHU ZLWK ]HURGRZQWLPH� VKDUGLQJ� DGYDQFHG URXWLQJ�FDFKLQJ DQG PXFK PRUH ZLWKRXW FRQWUROOLQJWKH DSSOLFDWLRQV WKDW JHQHUDWH WKH TXHULHV�

+LJK 3HUIRUPDQFH0\64/ 3UR[\ ZLWK D*3/ /LFHQVH

V

4XHU\ &DFKLQJ)RUJHW DSSOLFDWLRQV WKDW JHQHUDWHORDG E\ LQHIILFLHQWO\ DFFHVVLQJ WKHVDPH UHVXOWV RYHU DQG RYHU DJDLQ�3UR[\64/ TXLFNO\ MXPSV LQ ZLWK LWVDGYDQFHG UXOH HQJLQH� 5HVXOWV FDQDOVR EH FDFKHG IRU D FRQILJXUDEOHWLPHVSDQ� LQ QDWLYH 0\64/ SDFNHWVIRUPDW�

$SSOLFDWLRQ OD\HU SUR[\3UR[\64/ GRHV QRW IRUZDUG WUDIILFEOLQGO\� ,W XQGHUVWDQGV WKH 0\64/SURWRFRO DQG DFWV DFFRUGLQJO\�7KDWV ZK\ LW FDQ HDVLO\ VHUYHDGYDQFHG XVH�FDVHV VXFK DV VWLFN\WUDQVDFWLRQV RU UHDO�WLPH� LQ�GHSWKVWDWLVWLFV JHQHUDWLRQ DERXW WKHZRUNORDG�

4XHU\ 5RXWLQJ7DNH DQ DGYDQFHG FDVH� ZKHUH GLIIHUHQWFODVVHV RI TXHULHV QHHG WR EH URXWHG WRGLIIHUHQW 0\64/ FOXVWHUV� ZLWK GLIIHUHQWFRQILJXUDWLRQV� 6ROYH LW ZLWK 3UR[\64/VKRVWJURXS FRQFHSW� %DVHG RQ DQDGYDQFHG PDWFKLQJ HQJLQH� LW LV DEOH WRURXWH TXHULHV WUDQVSDUHQWO\ WRZDUGV WKHGHVWLQDWLRQ FOXVWHU WKDW FDQ H[HFXWH WKHPPRVW HIILFLHQWO\�

)LUHZDOO,Q FDVH RI RIIHQGLQJ TXHULHV WKDWFDXVH SUREOHPV WR WKH '% �64/LQMHFWLRQ RU LQHIILFLHQW UHWULHYDO RILQIRUPDWLRQ YLD 6(/(&7 ZLWKRXW:+(5(� IRU H[DPSOH�� 3UR[\64/DFWV DV D JDWHNHHSHU EHWZHHQ WKHDSSOLFDWLRQ DQG WKH '%� DOORZLQJ'%$V WR UHDFW TXLFNO\�

Proof

27

Rate Our Session!

DATABASE PERFORMANCE MATTERS