ProxySQL Use Case Scenarios - Percona · ProxySQL Use Case Scenarios Percona Live Amsterdam Oct 3-5...
Transcript of ProxySQL Use Case Scenarios - Percona · ProxySQL Use Case Scenarios Percona Live Amsterdam Oct 3-5...
ProxySQLUseCaseScenarios
Percona Live Amsterdam Oct 3-5 2016
Alkin Tezuysal René Cannaò
2
Who we are
• Alkin Tezuysal Sr. Technical Manager, Percona @ask_dba
• René Cannaò MySQL SRE, Dropbox / ProxySQL @proxysql
3
Top 5 reasons to use ProxySQL
• Improvedatabaseopera:ons.
• Understandandsolveperformanceissues.
• Createaproxylayertoshieldthedatabase.
• AddHigh-Availabilitytodatabasetopology.
• EmpowertheDBAswithgreattool.
4
ProxySQL Highlights
Scalability HighAvailability
AdvancedQuerySupport Manageability
5
Use case overview - Scalability
Connec:onPoolingandMul:plexing
Addconnec:onpoolinglayer
Reduceconnec:onthreadoverhead
Read/WriteSplitMorescalabilityforanyMySQLtopology
Mustforanyheavyworkloads
Read/WriteSharding
Effortlessshardingimplementa:on
WithoutDev.supportandcostsavings
6
Use case overview - High Availability
SeamlessFailover
Gracefulfailoversupport
Queryrerou:ng
LoadBalancingAllowseasyscaling
ForbePeru:liza:onofservers
ClusterAwareSupportsPXC/Galeraimplementa:ons
Smallfootprintforbiggain
7
Use case overview - Advanced Queries
Querycaching Cachefrequentlyuseddata
Addsanothercachelayer
Queryrewrite AddSQLflexibility
Fasterproblemsolving
Queryblocking Adddatabaseawarefirewall
WithoutApp.support
8
Use case overview - Advanced Queries
QuerymirroringAudit,Analyze,Review,CacheWarming
Allowbenchmarkingonlivedata
QuerythroPlingSetpriori:za:onforimportantqueries
Allowsmanualinterven:ontoproblemqueries
Query:meout Addanother:meoutlayer
Flexiblequerylevel:meout
9
Use case overview - Manageability
AdminU:lity Authen:ca:onsupport
Limituseraccesstodatabasepool
Run:mereconfigura:on
Ontheflymodifica:ons
Configuredatabasewithoutrestartordown:me
Monitoring Statscollec:onandrepor:ng
Inves:gatedatabaseworkload
10
Scalability with ProxySQL - Connection Pooling
• Any application without persistent connection to database
• PHP applications to be specific without built in connection pool
Application MySQL ProxySQL
• Reduces number of new connections to the database
11
Scalability with ProxySQL – Connection Multiplexing
• Any application with persistent connection to database
• Java applications to be specific with built in connection pools
Application Pool
MySQL
• Reduce connections similar to Aurora
• Testing being performed for 300K database connections
Application Pool Application
Pool
ProxySQL
12
Scalability with ProxySQL - Read/Write Split
• On the fly Read / Write implementation
• Use read_only flag to switch traffic
Application
MySQL Master
• Load balancing made easy
MySQL Slave
Write hostgroups
Read hostgroups MySQL
Slave
13
Scalability with ProxySQL – User and schema level sharding
• Granular sharding per username and schema
• Backend pooling based on user activity to specific schema
Application User A
ProxySQL
• Use advanced sharding to parallelize queries
• Scale beyond the sharding per host limitations.
Application User B
MySQL Master
MySQL Slave
MySQL Master
MySQL Slave
MySQL Master
MySQL Slave
14
MySQL Slave MySQL Master
High Availability with ProxySQL – Seamless failover
• Neither VIP setup nor service discovery needed
• Use read_only flag to switch traffic
Application Pool MySQL Master
• Integration with other HA Managers
Application Pool Application
Pool
ProxySQL
15
High Availability with ProxySQL – Improve Multi DC implementation
• No connection pools latency on SSL connections.
• Costly delays across the water.
MySQL Master ProxySQL
MySQL Master
ProxySQL
Application
DC1
DC2
16
High Availability with ProxySQL – Adoption to Clustering
• PXC / Galera / Group Replication
MySQL
ProxySQL
•
ProxySQL
Application
Node 1
Application
Application
MySQL
Node 2
MySQL
Node 3
17
Advanced Queries with ProxySQL – Caching
• Improve performance on read intensive workloads
• Reduce slave provisioning
Application A
ProxySQL
• Improved query cache over default implementation
• Query aware caching over general caching
Application B
MySQL Master
MySQL Slave
MySQL Slave
MySQL Slave
Cache
MySQL Slave
MySQL Slave
18
Advanced Queries with ProxySQL – Query Timeout
• Built in query killer a.k.a query sniper
• Adjustable threshold based on query rule
Application A
ProxySQL
• No idle or long running queries
Application B
MySQL Master
MySQL Slave
MySQL Slave
MySQL Slave
Query Timeouts
MySQL Slave
MySQL Slave
19
Advanced Queries with ProxySQL – Firewall
• Protect database from unwanted traffic
• Stop unwanted user, account , application (new code)
Application A
ProxySQL
• Fast modification to database behavior
• Added protection for DDOS and other attacks.
Application B
Query Blocking
MySQL Master
MySQL Slave
MySQL Master
MySQL Slave
MySQL Master
MySQL Slave
20
Advanced Queries with ProxySQL – Query rewrite engine
• Most wanted feature by DBAs
• Rewrite queries overloading the database on the fly.
Application A
ProxySQL
• Simply buy time until application can be modified
Application B
MySQL Master
MySQL Slave
MySQL Slave
MySQL Slave
Query Rewriting
MySQL Slave
MySQL Slave
21
Advanced Queries with ProxySQL – Query retry
• Server failures or maintenance do not loose a query.
• Fails over and resubmits the query to another host.
Application A
ProxySQL
• Improved operation and new way of handling slaves.
Application B
MySQL Master
MySQL Slave
MySQL Slave
MySQL Slave
Query Retry
MySQL Slave
MySQL Slave
22
Advanced Queries with ProxySQL – Query routing
• Selective routing based on importance.
• Use metrics based on stats_mysql_query_digest and decide
Application A
ProxySQL
• Go beyond read/write split.
Application B
MySQL Master
MySQL Slave
MySQL Slave
MySQL Slave
Query Routing
MySQL Slave
MySQL Slave
23
Advanced Queries with ProxySQL – Query mirroring
• Mirror incoming queries to different back ends
• Use it for audit, analytics, cache warming, benchmarking.
Application A
ProxySQL
Application B
Query Mirroring
MySQL Master
Host A
Host B
MySQL Slave
MySQL Master
MySQL Slave
24
Advanced Queries with ProxySQL
Querycaching QueryTimeout
QueryRetry QueryBlocking
QueryRou:ngandMirroring
25
Clustered ProxySQL at scale
Tested with:
● 8 app servers with 3k clients’ connections each (24k total) ● 4 middle layer proxysqls processing 4k connections each from local
proxysqls (16k total) ● 256 backends/shard (meaning 256 routing rules) processing 600
connections each (150k total) Single ProxySQL was tested with up to 150k connections At today, ProxySQL is able to process up to 750k QPS
26
Questions
3UR[\64/ LV DQHZ 3UR[\ ZLWK DQDGYDQFHG PXOWL�FRUH DUFKLWHFWXUH�,WV EXLOW IURP WKHJURXQG XS WR
VXSSRUW KXQGUHGVRI WKRXVDQGV RIFRQQHFWLRQVFXUUHQWO\�
PXOWLSOH[HG WRSRWHQWLDOO\KXQGUHGV RI
EDFNHQG VHUYHUV�
+,*+3(5)250$1&(
� 7KH GDWD JDWHZD\ �
6+$5',1*
3UR[\64/ LV DYHU\ SRZHUIXOSODWIRUP DEOH WRFRYHU PXOWLSOHVKDUGLQJ
VFHQDULRV ZLWK DQDGYDQFHGFRQILJXUDWLRQEDVHG RQ WKHHQRUPRXVIOH[LELOLW\ RIFRQFDWHQDWLQJVLPSOH SURFHVVLQJ
UXOHV�
)25 '%$V %< '%$V
7DNH EDFN FRQWURORI \RXU 0\64/FOXVWHU WRGD\�
6FDOH DQG RSHUDWHLW VPRRWKO\�8QORFN DQ
XQSUHFHGHQWHGOHYHO RI IOH[LELOLW\DQG SHUIRUPDQFH�$OO WKH ZKLOHJHWWLQJ WKH
XQOLPLWHG IUHHGRPWKDW FRPHV ZLWK D*3/ /LFHQVH�
+LJK 3HUIRUPDQFH 0\64/ 3UR[\ ZKHQ VFDODELOLW\ PDWWHUV
4XHU\ FDFKLQJ
6XSSRUWV IDLORYHU
5HDO�WLPH VWDWLVWLFV
$GYDQFHG FRQILJXUDWLRQ
7UDIILF PLUURULQJ $GYDQFHG WRSRORJ\ VXSSRUW
ZZZ�SUR[\VTO�FRP �SUR[\VTO
Proof 3UR[\64/ LV D UHYROXWLRQDU\ SUR[\ WKDW KHOSV
\RX VTXHH]H WKH ODVW GURS RI SHUIRUPDQFHRXW RI \RXU 0\64/ FOXVWHU� IHDWXULQJ +LJK$YDLODELOLW\� VHDPOHVV IDLORYHU ZLWK ]HURGRZQWLPH� VKDUGLQJ� DGYDQFHG URXWLQJ�FDFKLQJ DQG PXFK PRUH ZLWKRXW FRQWUROOLQJWKH DSSOLFDWLRQV WKDW JHQHUDWH WKH TXHULHV�
+LJK 3HUIRUPDQFH0\64/ 3UR[\ ZLWK D*3/ /LFHQVH
V
4XHU\ &DFKLQJ)RUJHW DSSOLFDWLRQV WKDW JHQHUDWHORDG E\ LQHIILFLHQWO\ DFFHVVLQJ WKHVDPH UHVXOWV RYHU DQG RYHU DJDLQ�3UR[\64/ TXLFNO\ MXPSV LQ ZLWK LWVDGYDQFHG UXOH HQJLQH� 5HVXOWV FDQDOVR EH FDFKHG IRU D FRQILJXUDEOHWLPHVSDQ� LQ QDWLYH 0\64/ SDFNHWVIRUPDW�
$SSOLFDWLRQ OD\HU SUR[\3UR[\64/ GRHV QRW IRUZDUG WUDIILFEOLQGO\� ,W XQGHUVWDQGV WKH 0\64/SURWRFRO DQG DFWV DFFRUGLQJO\�7KDWV ZK\ LW FDQ HDVLO\ VHUYHDGYDQFHG XVH�FDVHV VXFK DV VWLFN\WUDQVDFWLRQV RU UHDO�WLPH� LQ�GHSWKVWDWLVWLFV JHQHUDWLRQ DERXW WKHZRUNORDG�
4XHU\ 5RXWLQJ7DNH DQ DGYDQFHG FDVH� ZKHUH GLIIHUHQWFODVVHV RI TXHULHV QHHG WR EH URXWHG WRGLIIHUHQW 0\64/ FOXVWHUV� ZLWK GLIIHUHQWFRQILJXUDWLRQV� 6ROYH LW ZLWK 3UR[\64/VKRVWJURXS FRQFHSW� %DVHG RQ DQDGYDQFHG PDWFKLQJ HQJLQH� LW LV DEOH WRURXWH TXHULHV WUDQVSDUHQWO\ WRZDUGV WKHGHVWLQDWLRQ FOXVWHU WKDW FDQ H[HFXWH WKHPPRVW HIILFLHQWO\�
)LUHZDOO,Q FDVH RI RIIHQGLQJ TXHULHV WKDWFDXVH SUREOHPV WR WKH '% �64/LQMHFWLRQ RU LQHIILFLHQW UHWULHYDO RILQIRUPDWLRQ YLD 6(/(&7 ZLWKRXW:+(5(� IRU H[DPSOH�� 3UR[\64/DFWV DV D JDWHNHHSHU EHWZHHQ WKHDSSOLFDWLRQ DQG WKH '%� DOORZLQJ'%$V WR UHDFW TXLFNO\�
Proof
27
Rate Our Session!
DATABASE PERFORMANCE MATTERS