http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Building Scalable Enterprise SolutionsArchitecture, Performance, Scalability, Migration, Knowledge Transfer
1
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Ronald BradfordPrincipal
42SQL
[email protected] 1.0 3.Aug..2008
Best Design Practices
For MySQL Architects, DBA’s and
Developers
2
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Introduction
❖ Lots of information to discuss❖ Basic principles❖ Rules
3
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Agenda
❖ Architecture❖ Availability❖ Scalability❖ Accountability
4
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Architecture
❖ Toolset❖ The performance equation❖ Data integrity
ARCHITECTURE
5
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Your Toolset
❖ Generics are inefficient❖ different RDBMS is not enough ❖ You have chosen MySQL
ARCHITECTURE
6
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Your MySQL Toolset
❖ Maximize MySQL strengths❖ Scale out / HA Options❖ Different Storage Engines❖ Query Cache - Both good and bad
ARCHITECTURE
7
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Your MySQL Toolset
❖ Minimize MySQL weaknesses❖ No Online Alter❖ Backup Strategies❖ Instrumentation
ARCHITECTURE
8
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Your MySQL Toolset
❖ Connector/J❖ Supports read/write splitting❖ Supports read balancing across slaves❖ Supports failover
❖ Connection.setReadOnly(true | false);
http://dev.mysql.com/doc/refman/5.0/en/connector-j-reference-replication-connection.html
Example
ARCHITECTURE
9
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Your Toolset
RTFMARCHITECTUREManual: http://dev.mysql.com/doc
Book: High Performance MySQL - 2nd Edition
10
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
The performance equation
Disk = Memory = Performance
❖ Every single byte counts❖ Average saving of 25% - 30%❖ Better 60% ❖ Best 78%
ARCHITECTURE
11
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
The performance equation
❖ Favorite signs of poor design in data types❖ INT(1)❖ BIGINT AUTO_INCREMENT❖ no UNSIGNED used❖ DECIMAL(31,0)
ARCHITECTURE
12
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Data integrity
❖ MySQL historically has been very lax❖ Warnings (e.g. Truncations) are rarely every caught
❖ sql_mode=strict_all_tables
ARCHITECTURE
http://ronaldbradford.com/blog/why-sql_mode-is-important-part-i-2008-07-17/13
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Data integrity
❖ With Schema❖ NOT NULL❖ ENUM❖ UNSIGNED
ARCHITECTURE
http://ronaldbradford.com/blog/why-sql_mode-is-important-part-i-2008-07-17/14
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Data integrity
❖ Storage Engines❖ Always design for transactions❖ Always use transactions❖ Use a transactional storage engine
ARCHITECTURE
15
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
From Top 20 Design Tips
1. Know your technology tools
2. Know your disk footprint3. Choose your numeric data type
4. Other data type efficiencies5. Application data type efficiencies
6. NOT NULL
7. Know about character sets8. When VARCHAR is bad
9. Be Wary of TEXT/BLOB10. Know every SQL Statement
11. Monitor Every SQL Statement
12. The impact of Indexes13. Index types for design
14. Minimizing internal MySQL Processing15. Transactions
16. Data Integrity is key
17. Leveraging the Query Cache18. Create Object appropriately
19. Naming Standards20. Testing, Testing, Testing
http://ronaldbradford.com/presentations.htm#200804
ARCHITECTURE
16
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Further Discussion (Part B)
❖ How to design for caching at multiple levels❖ Cloud computing considerations (MySQL/ Non MySQL)
❖ Supporting 3rd party applications (REST/SOAP/XMMP)
❖ CAP Theory (Consistency, Availability, Partition)
❖ Balancing design with agile approaches❖ Hardware requirements (Disk, Memory, Network, CPU)
❖ Managing different Read/Write environments
ARCHITECTURE
17
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Site presence
❖ What happens when your site is:❖ Unavailable❖ Under load❖ Something is broken❖ Undergoing maintenance❖ Somebody elseʼs mistake (e.g. Data Center)
AVAILABILITY
18
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
19
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
20
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Firefox Download day - Example 1 21
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Firefox Download day - Example 222
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
23
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
24
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Minimum Ideals
❖ Static version of site❖ Support for Read Only & Write Enabled❖ Support for components being unavailable
AVAILABILITY
25
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Further Discussion (Part B)
❖ What is no downtime and how to achieve it?❖ Tracking referrer through site navigation❖ What is the minimum MySQL configuration?❖ The strengths and weaknesses of MySQL Topologies
AVAILABILITY
26
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scalability
❖ Successful scaling is?
SCALABILITY
27
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scalability
❖ Successful scaling is?
Scale Out not Scale Up
SCALABILITY
28
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scalability
❖ To scale out is to?
SCALABILITY
29
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scalability
❖ To scale out is to?
Shard
SCALABILITY
30
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale out
❖ The art of Sharding is?❖ A one day seminar (ask me for more)
SCALABILITY
31
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale out
❖ A database table consists of❖ Columns❖ Rows
SCALABILITY
32
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
TABLE col1 col2 col3 col4 col5 col6 col7row1row2row3row4row5row6row7row8row9
row10row11row12row13row14
Columns
33
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
TABLE col1 col2 col3 col4 col5 col6 col7row1row2row3row4row5row6row7row8row9
row10row11row12row13row14
Columns
34
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
TABLE col1 col2 col3 col4 col5 col6 col7row1row2row3row4row5row6row7row8row9
row10row11row12row13row14
Columns
35
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
TABLE col1 col2 col3 col4 col5 col6 col7row1row2row3row4row5row6row7row8row9
row10row11row12row13row14
Rows
36
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
TABLE col1 col2 col3 col4 col5 col6 col7row1row2row3row4row5row6row7row8row9
row10row11row12row13row14
Rows
37
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
TABLE col1 col2 col3 col4 col5 col6 col7row1row2row3row4row5row6row7row8row9
row10row11row12row13row14
Rows
38
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale out
❖ A database consists of❖ Tables❖ Table Rows
SCALABILITY
39
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
table1 table2 table3 table4 table5
Tables
40
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
table1 table2 table3 table4 table5
Tables
41
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
table1 table2 table3 table4 table5
Tables
42
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
table1 table2 table3 table4 table5row1row2row3row4row5row6row7row8row9
row10row11row12row13row14
Table Rows
43
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
table1 table2 table3 table4 table5row1row2row3row4row5row6row7row8row9
row10row11row12row13row14
Table Rows
44
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
table1 table2 table3 table4 table5row1row2row3row4row5row6row7row8row9
row10row11row12row13row14
Table Rows
45
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale out
❖ Partitioning is?❖ Grouping like rows in a table together❖ e.g.
❖ By Date❖ By Local❖ By parent grouping
SCALABILITY
46
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
TABLE col1 col2 col3 col4 col5 col6 col7row1row2row3row4row5row6row7row8row9
row10row11row12row13row14
Partition
47
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
TABLE col1 col2 col3 col4 col5 col6 col7row1row2row3row4row5row6row7row8row9
row10row11row12row13row14
Partition
48
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
TABLE col1 col2 col3 col4 col5 col6 col7row1row2row3row4row5row6row7row8row9
row10row11row12row13row14
Partition
49
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale out
❖ Sharding is?❖ Grouping like tables together❖ optionally partition subsets of rows
SCALABILITY
50
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
table1 table2 table3 table4 table5
Table Group
51
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
table1 table2 table3 table4 table5
Table Group
52
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
table1 table2 table3 table4 table5
Table Group
53
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
table1 table2 table3 table4 table5
Table Group + Rows
54
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
table1 table2 table3 table4 table5
Table Group + Rows
55
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
table1 table2 table3 table4 table5
Table Group + Rows
56
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale outSCALABILITY
table1 table2 table3 table4 table5
Table Group + Rows
57
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale out
❖ How do you group tables?
SCALABILITY
58
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Scale out
❖ How do you group tables?
Thatʼs the secret
SCALABILITY
59
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Further Discussion (Part B)
❖ The partition key is the key❖ Poor examples of sharding❖ The theory of ideal partition key❖ Balancing and re balancing
SCALABILITY
60
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Further Discussion (Part B)
❖ Master/Master does not solve scalability❖ Store data in a database, donʼt always retrieve
from the database❖ Cache at multiple layers
SCALABILITY
61
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Accountability
❖ The rule of everything ❖ You should look at your logs
ACCOUNTABILITY
62
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
The Rule of Everything
❖ Everythingʼs are ❖ Monitor❖ Measure❖ Graph❖ Automate
ACCOUNTABILITY
63
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Easy Information to know
❖ Know your error logs❖ System, Database, Application, RAID
❖ Seen degraded RAID multiple times ***❖ donʼt ignore errors (such as Innodb)
ACCOUNTABILITY
64
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Further Discussion (Part B)
❖ Testability❖ Donʼt think of testing as to know your software works,
testing should be to break your software ❖ Know every single SQL Statement❖ Your application should have detailed metrics
ACCOUNTABILITY
65
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Executive Summary
Architecture
❖ Learn and know MySQL specifics❖ Disk = Memory = Performance❖ Integrity is important, but how important is
consistency?
ARCHITECTURE
66
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Executive Summary
Availability
❖ Test your site response under error cases❖ Disaster is inevitable. Donʼt wait for it❖ Try testing unavailability first ❖ Write failing test cases for your website first
AVAILABILITY
67
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Executive Summary
Scalability
❖ Think low cost commodity hardware❖ like general cloud computing architecture
❖ Understanding sharding from day 1❖ XMPP
SCALABILITY
68
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Executive Summary
Accountability
❖ Donʼt let users tell you when things donʼt work❖ You need numbers to predicate failure❖ You have to have history for trending❖ Log, Review & Monitor all SQL
ACCOUNTABILITY
69
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Whatʼs next
❖ Any key point could be an hour discussion
❖ What 3 points are best topics for audience?
70
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Looking forward
A word on cloud computing
❖ Donʼt put all your eggs in one basket❖ Keep your important data close
71
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Professional Help is Available
❖ 2 decades Expertise & Experience ❖ Architecture, Design, Performance & Scalability
❖ 9 years in MySQL
ronaldbradford.com42sql.com
72
http://42sql.com(c) Copyright 2008
Best Design Practices - Part A
Building Scalable Enterprise SolutionsArchitecture, Performance, Scalability, Migration, Knowledge Transfer
73
Top Related