Maximizing MySQL Availability, Data Integrity, Performance, and Scalability · Maximizing MySQL...

. © 2011 Schooner Information Technology, p1

Maximizing MySQL Availability, Data Integrity, Performance, and Scalability

Schooner Information Technology. October 25, 2011

Dr. John R. Busch Founder and CTO Schooner Information Technology [email protected]


Maximizing MySQL Availability, Data Integrity, Performance, and Scalability

Abstract: First, we discuss the use of synchronous replication of InnoDB to eliminate data loss and to provide cluster-wide data consistency. We then discuss instant automatic fail-over providing five 9s to maximize service availability. Next we discuss the use of high parallelism and granular concurrency control with synchronous replication to fully exploit commodity servers and storage to maximize performance. Next we discuss extending base synchronous clusters with tightly-coupled, parallel asynchronous replication to achieve auto-failover and high consistency across geographically dispersed data centers. Bio: Dr. John R. Busch is Founder and CTO of Schooner Information technology, focusing on high performance and highly available databases for high volume web facing, cloud, and enterprise services. Prior to Schooner, John was director of computer system research at Sun Microsystems Laboratories, VP of Engineering with Diba, founder of Clarity Software, and R&D director of computer systems at Hewlett Packard. John holds a Ph.D. in Computer Science from UCLA, M.S. in Mathematics from UCLA, M.S. in Computer Science from Stanford University, and attended the Stanford Sloan program.


Data

§ Most important and valuable component of modern applications and websites § Driving revolutionary changes in computing and the internet

Ø New opportunities for generating revenue Ø More efficient use of current business processes and infrastructure

§ Data access downtime or poor performance has a major cost to a business’ bottom line

Business’ Most Valuable Asset: Its Data


“Let me tell you the difference between Facebook and everyone else, we don't crash EVER! If our service is down for even a minute, our entire reputation is irreversibly destroyed Facebook and Google invest hundreds of millions of dollars every year on custom software and hardware infrastructure to optimize availability, performance, administration, and cost

The Mission-Critical Imperative


§ Providing high data availability, excellent response time is critical for key classes of businesses

Ø  Web 2.0 Ø  eCommerce Ø High-volume websites Ø Telecommunications

§ They require a mission critical database

The Mission-Critical Imperative


Mission-Critical Database Requirements

High Availability

High Performance

and Scalability

Simple and Powerful

Administra<on Data Integrity Cost Effec<ve

Standards and

Compa<bility


Mission-Critical Database Goals and Metrics

§  Service unavailability (minutes/year) from failures, disaster recovery, or during planned administration

§  Probability of data loss or corruption; data consistency levels

§  Transaction throughput, response time;

performance scalability; performance stability §  Ease of cluster administration; fail-over

automation; monitoring and optimization tools

§  Total cost of ownership (TCO); return on investment (ROI)

§  Level of standards compliance and certification

§  High Availability

§  High Data Integrity

§  High Performance and Scalability

§  Simple and powerful administration §  Cost effective

§  Standards and Compatibility

Goals Metrics


Loosely-Coupled Asynchronous and Semi-Synchronous Replication

Master mysqld Slave mysqld

Relay log

InnoDB Tx log DB MySQL

Bin log InnoDB Tx log

DB

Last tx=100 Last tx=100 Last tx=70 (asynch Last tx=100 (synch)

Last tx=50

Log events pulled by Slave

Tx.Commit(101) Repl.apply(51)

tx=101 tx=101 tx=51 tx=51

Read version based on tx=50

Example Products : MySQL Enterprise 5.1 Asynchronous and 5.5/5.6 Semi-Synchronous Replication



Master mysqld Slave mysqld

Relay log

InnoDB Tx log DB MySQL

Bin log InnoDB Tx log

DB

Last tx=100 Last tx=100 Last tx=70 (asynch Last tx=100 (synch)

Last tx=50

Log events pulled by Slave


tx=101 tx=101 tx=51 tx=51

Read version based on tx=50

Inconsistent (Stale) Data

Slow Recovery Complex Management

Slow ExecuCon

PotenCal Data Loss

Example Products : MySQL Enterprise 5.1 Asynchronous and 5.5/5.6 Semi-Synchronous Replication



Limited Service Availability • Master fail-over, re-synch of slaves

Limited Data Integrity • Lost data; inconsistent Data

Limited Performance and Utilization

• Low throughput and low utilization Complex Administration

• Manual processes, slave re-synch

High Cost of Ownership • High capital expense from server sprawl • Increased operating expense from power, space, admin • Reduced revenue and customer satisfaction from service downtime


Tight Coupling and Fully Synchronous Replication

•  Slaves in lock-step with Master •  At master transaction commit, all Slaves guaranteed to have received and committed the changes

SchoonerSQL (Master)

Cluster Admin

MySQL

Op<mized Parallel Execu<on Threads in

SchoonerSQL

SchoonerSQL (Read Master)

Parallel Synchronous Replica<on During Transac<on Execu<on

MySQL clients

MySQL clients

Concurrently Execu<ng

Transac<ons Parallel

SchonerSQL Replica<on Threads

InnoDB

Concurrently Execu<ng

Transac<ons

Red Hat, CentOS Linux

Standard X86 Server

Parallel SchoonerSQLReplica<on Threads

InnoDB

Red Hat, CentOS Linux

Standard X86 Server

Op<mized Parallel Execu<on Threads in

SchoonerSQL


Tight-Coupling and Synchronous Replication

SchoonerSQL Master

SchoonerSQL ReadMaster

InnoDB Tx log DB InnoDB

Tx log DB

Last tx=100 Last tx=100 Last tx=100 Last tx=100

Log for tx=100 pushed to Slave


tx=101 tx=101

Slave ACK for tx=100

Eliminates Service InterrupCons § Fast , Transparent Fail-‐Over (VIPs) § Fast incremental recovery

Easy Management High Performance High UClizaCon

No Data Loss Cluster-‐Wide Consistent Reads


Tightly-coupled MySQL synchronous replication with fast, automated application transparent fail-over and can provide much higher service availability than that achievable with asynchronous or semi-synchronous replication

Availability Improvement from Synchronous ReplicaCon (% CumulaCve Down Time ReducCon)

Tight Coupling and Synchronous Replication with Transparent Automatic Fail-over Improves Service Availability by 90%

0

10

20

30

40

50

60

70

80

90

100

MySQL 5.5 SchoonerSQL Auto-‐Failover

Schooner SQL On-‐Line Upgrades


Tight Coupling and Synchronous Replication Provides Much Higher Performance Throughput per Server

DBT2 open-source OLTP version of TPC-C 1000 warehouses, 32 connections 0 think-time Result metric: TPM (new order)

Measurement Configuration 2 node Master-Slave configuration 2 socket Westmere 72GB DRAM

Synchronous TransacCon Throughput per Server can be Much greater Than Asynchronous or Semi-‐Synchronous

(with hard disc drives (HDDs))

0 2 4 6 8 10 12 14

MySQL 5.5 asynch

MySQL 5.5 semi-‐synch

SchoonerSQL

Transac<on Throughput with Hard Drives (kTPM)


Tight Coupling and Synchronous Replication Enables Vertical Scaling with Commodity Flash Memory, Cores

DBT2 open-source OLTP version of TPC-C 1000 warehouses, 32 connections 0 think-time Result metric: TPM (new order)

Measurement Configuration 2 node Master-Slave configuration 2 socket Westmere 72GB DRAM

TransacCon Throughput with Hard Disc Drives

0 10 20

MySQL 5.5 asynch

MySQL 5.5 semi-‐synch

SchoonerSQL

0

20

40

60

80

100

120

MySQL 5.5 asynch MySQL 5.5 semi-‐synch

SchoonerSQL

TransacCon Throughput with Flash Drives


Response Time (ms)

Tight Coupling and Synchronous Replication Lowers Response Times

0

1

2

3

4

5

6

7

MySQL 5.5 asynch MySQL 5.5 semi-‐synch

SchoonerSQL

Performance : Transac<on Response Time


Tight Coupling and Synchronous Replication Provides High Performance Stability

SchoonerSQL MySQL 5.5

Asynchronous MySQL 5.5

Semi-synchronous


Tight Coupling and Synchronous Replication Lowers Total Cost of Ownership

Lower Cost § Reduced capital and operating costs through reduction in servers, power, space, admin § Savings from increased service availability and associated revenue and customer retention

§ TCO and ROI models are customer and workload specific § Function (throughput/server; server, rack, and network costs, software license and support costs, admin costs; space and power costs; cost of downtime)

0

1

2

3

4

MySQL 5.5 SchoonerSQL

Total Cost of Ownership (rela<ve)


Tight Coupling and Synchronous Replication with Transpaent Automated Failover and Admin GUI Greatly Simplifies Administration

§ Fail-over can be completely automatic and instant § requiring no administrator intervention or service interruption

§  Cluster Administrator GUI and CLI can provide a single point for cluster-wide management

§  single click slave creation and database migration


Powerful Administration : Monitoring, Trouble-shooting, Tuning, Alerts


Extending Synchronous Replication with Parallel Asynch and Auto-Failover in WAN Geographic Distribution/DR

•  WAN/geographically dispersed data centers – Requires Asynchronous replication

–  Can’t add additional ~100ms with high potential variance to query response time for synchronous replication

•  Data Integrity Requirement : Remote consistency lag and recovery time should be ~ WAN latency

–  Maximize WAN data consistency –  Minimize disaster recovery time

•  Requires high performance asynchronous replication –  Must have multi-threaded asynchronous parallelizing updates


Extending Synchronous Replication with Parallel Asynch and Auto-Failover in WAN Geographic Distribution/DR •  HA Requirement: Must automatically fail-over when synchronous

master fails-over or asynch slave fail-over occurs, requires WAN asynchronous replication tight coupling with synch replication group


Unlimited Query Scaling

0 50000

100000 150000 200000 250000 300000

1 2 4 8

Requ

ests/s

Nodes in Cluster

SchoonerSQL Throughput (sysbench OLTP)

Reads/s Writes/s

Query Scaling in a Synchronous Replication Group

§ Fully replicated Master/Slave cluster §  No cluster overhead for adding queries to a slave § Can add synchronous query read master nodes linearly § Unlimited query scaling extension with parallel asynchronous replication (no data loss, auto-failover, high data integrity)

§ With shared nothing architectures, there is potentially severe cross node query degradation


Scaling Updates

•  Database Update Scalability – Vertically scale with commodity : flash memory, more cores,

higher frequency

•  Compelling option exploiting low cost, high performance commodity technology

0 50

100 150

Schooner Synch Cluster: Hard Drives

Schooner Synch Cluster : Flash


Unlimited Update Scaling : Optimized Transparent Sharding

•  Database Update Scalability

…After Optimal Vertical Scaling: Unlimited Query Scaling : Horizontally Scale Through Transparent Workload Aware Database Partitioning and Replication

•  Database workload aware

–  Administrator analysis and configuration tools –  Allows optimal layout, including replicated structures, for query data access

optimization

•  Application Transparent –  High performance, highly available dynamic query execution across shards


MySQL 5.5

MySQL NDB Cluster

Clustrix

Linux DRBD

ConCnuent Tungsten

Golden Gate

Schooner SQL

Fail-‐Over Down<me Minutes-‐hours

seconds seconds minutes seconds Minutes-‐hours

seconds

Automated Fail-‐over No Yes Yes No Yes No Yes

Data Loss Yes No yes No Yes Yes No

Data Consistency No Yes Yes No No No Yes

Performance Med Med/Low Med/Low Med Low Low High

Scalability Low Med/Low Med/Low Low Low Low High

Ease of Management

Low High High Low Med Med High

WAN perf and auto-‐ fail-‐over

No No No No No No Yes

InnoDB Compa<ble High Med/Low Med/Low High High High High

Custom Hardware No No Yes No No No No

Cost (TCO) High Med High High High High Low

Heterogeneous Databases

No No No No Yes Yes No

MySQL Database Alternatives for Mission-Critical Deployments


SchoonerSQL Advantage

•  4-‐20x more throughput/server vs. MySQL 5.5 • High performance synchronous and asynchronous replica<on

Highest Performance and Scalability

•  No service interrup<on for planned or unplanned database down<me • Instant automa<c fail-‐over •  On-‐line upgrade and migra<on •  90% less down<me vs. MySQL 5.5 • Full WAN support with master auto-‐failover

Highest Availability

Visibility and Control •  Easy cluster administra<on •  No error-‐prone manual processes • Monitoring and Op<miza<on

•  No lost data •  Cluster-‐wide data consistency

Highest Data Integrity

•  Full MySQL + InnoDB: not a toolkit •  Free your staff to build your business, not a custom database

•  Cut server capex (consolida<on) •  Cut opex (power, pipe, DBA <me) • Increase revenue (eliminate service interrup<ons) •  TCO 70% cheaper than MySQL 5.5

Compelling Economics

Out-‐of-‐the-‐box Product

100% MySQL Enterprise InnoDB CompaCble

Broad Industry Deployment •  eCommerce, Social Media, Telco,

Financial Services, Education •  High volume web sites •  Geographically distributed websites

!


Try SchoonerSQL! We guarantee SchoonerSQL will increase your Availability, Performance, and Scalability Contact us: Schooner Information Technology, Inc. 501 Macara Avenue, Suite 101 Sunnyvale, CA 94085 USA Tel: +1 408-773-7500 www.schoonerinfotech.com Email: [email protected]

Evaluating the Options and Trade-offs for Your Data Center? Let Schooner Help!


Thank You!

Maximizing MySQL Availability, Data Integrity, Performance, and Scalability · Maximizing MySQL...

Documents

Transcript of Maximizing MySQL Availability, Data Integrity, Performance, and Scalability · Maximizing MySQL...