Scaling to 200K Transactions per Second with Open Source - MySQL, Java, curl, PHP
Scaling PHP to 40 Million Uniques
-
Upload
jonathan-klein -
Category
Technology
-
view
5.495 -
download
3
description
Transcript of Scaling PHP to 40 Million Uniques
![Page 1: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/1.jpg)
Scaling PHP to 40 Million Uniques
Northeast PHP 2013Jonathan Klein@jonathanklein
Saturday, August 17, 13
![Page 2: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/2.jpg)
Scaling PHP to 40 Million Uniques
Northeast PHP 2013Jonathan Klein@jonathanklein
Saturday, August 17, 13
![Page 3: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/3.jpg)
Scaling PHP to 60 Million Uniques
Northeast PHP 2013Jonathan Klein@jonathanklein
Saturday, August 17, 13
![Page 4: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/4.jpg)
Who Am I?
Saturday, August 17, 13
![Page 5: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/5.jpg)
Who Am I?
• Performance Engineer at Etsy
Saturday, August 17, 13
![Page 6: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/6.jpg)
Who Am I?
• Performance Engineer at Etsy• Boston Web Perf Meetup Organizer
Saturday, August 17, 13
![Page 7: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/7.jpg)
Who Am I?
• Performance Engineer at Etsy• Boston Web Perf Meetup Organizer• Previously at Wayfair
Saturday, August 17, 13
![Page 8: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/8.jpg)
Who Am I?
• Performance Engineer at Etsy• Boston Web Perf Meetup Organizer• Previously at Wayfair• Led team that converted to PHP
Saturday, August 17, 13
![Page 9: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/9.jpg)
Slides, Links:jkle.in/nephp
Saturday, August 17, 13
![Page 10: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/10.jpg)
codeascraft.com
Saturday, August 17, 13
![Page 11: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/11.jpg)
Some Etsy Stats
Saturday, August 17, 13
![Page 12: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/12.jpg)
Some Etsy Stats• 1.5 billion page views/month
Saturday, August 17, 13
![Page 13: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/13.jpg)
Some Etsy Stats• 1.5 billion page views/month• Almost $1B in sales last year
Saturday, August 17, 13
![Page 14: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/14.jpg)
Some Etsy Stats• 1.5 billion page views/month• Almost $1B in sales last year• Over 1M lines of PHP
Saturday, August 17, 13
![Page 15: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/15.jpg)
Some Etsy Stats• 1.5 billion page views/month• Almost $1B in sales last year• Over 1M lines of PHP• 60M+ unique visitors/month
Saturday, August 17, 13
![Page 16: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/16.jpg)
2012 Traffic
Saturday, August 17, 13
![Page 17: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/17.jpg)
August
2012 Traffic
Saturday, August 17, 13
![Page 18: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/18.jpg)
August
2012 Traffic
50% Higher
Saturday, August 17, 13
![Page 19: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/19.jpg)
Holidays:~90M Uniques
Saturday, August 17, 13
![Page 20: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/20.jpg)
Saturday, August 17, 13
![Page 21: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/21.jpg)
Saturday, August 17, 13
![Page 22: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/22.jpg)
A few others...
Saturday, August 17, 13
![Page 23: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/23.jpg)
A few others...
• Search (Solr)
Saturday, August 17, 13
![Page 24: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/24.jpg)
A few others...
• Search (Solr)• Gearman
Saturday, August 17, 13
![Page 25: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/25.jpg)
A few others...
• Search (Solr)• Gearman• Redis
Saturday, August 17, 13
![Page 26: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/26.jpg)
A few others...
• Search (Solr)• Gearman• Redis• Postgres (legacy)
Saturday, August 17, 13
![Page 27: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/27.jpg)
Saturday, August 17, 13
![Page 28: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/28.jpg)
Hardware (Supermicro)
Saturday, August 17, 13
![Page 29: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/29.jpg)
Hardware (Supermicro)2x 8-core Intel E5-2960 CPUs
Saturday, August 17, 13
![Page 30: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/30.jpg)
Hardware (Supermicro)2x 8-core Intel E5-2960 CPUs
24GB of RAM
Saturday, August 17, 13
![Page 31: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/31.jpg)
Hardware (Supermicro)2x 8-core Intel E5-2960 CPUs
24GB of RAM
160GB SSD
Saturday, August 17, 13
![Page 32: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/32.jpg)
Apache
Saturday, August 17, 13
![Page 33: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/33.jpg)
Apache
• Apache 2.2
Saturday, August 17, 13
![Page 34: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/34.jpg)
Apache
• Apache 2.2• Prefork MPM
Saturday, August 17, 13
![Page 35: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/35.jpg)
Apache
• Apache 2.2• Prefork MPM• mod_php5
Saturday, August 17, 13
![Page 36: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/36.jpg)
Apache
• Apache 2.2• Prefork MPM• mod_php5
StartServers 30MinSpareServers 30MaxSpareServers 60ServerLimit 60MaxClients 60MaxRequestsPerChild 0
Saturday, August 17, 13
![Page 37: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/37.jpg)
PHP
Saturday, August 17, 13
![Page 38: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/38.jpg)
PHP
• PHP 5.4
Saturday, August 17, 13
![Page 39: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/39.jpg)
PHP
• PHP 5.4• Zend OPCache (3GB Memory Segment)
Saturday, August 17, 13
![Page 40: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/40.jpg)
PHP
• PHP 5.4• Zend OPCache (3GB Memory Segment)• memory_limit: 128M
Saturday, August 17, 13
![Page 41: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/41.jpg)
PHP
• PHP 5.4• Zend OPCache (3GB Memory Segment)• memory_limit: 128M• max_execution_time: 30
Saturday, August 17, 13
![Page 42: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/42.jpg)
Optimizing PHP
http://www.slideshare.net/jnklein/northeast-php-high-performance-php
Saturday, August 17, 13
![Page 43: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/43.jpg)
Optimizing PHP
• See last year’s talk
http://www.slideshare.net/jnklein/northeast-php-high-performance-php
Saturday, August 17, 13
![Page 44: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/44.jpg)
Optimizing PHP
• See last year’s talk• Use an Opcode Cache
http://www.slideshare.net/jnklein/northeast-php-high-performance-php
Saturday, August 17, 13
![Page 45: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/45.jpg)
Optimizing PHP
• See last year’s talk• Use an Opcode Cache
• xhprof
http://www.slideshare.net/jnklein/northeast-php-high-performance-php
Saturday, August 17, 13
![Page 46: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/46.jpg)
Optimizing PHP
• See last year’s talk• Use an Opcode Cache
• xhprof
• StatsD/Graphite
http://www.slideshare.net/jnklein/northeast-php-high-performance-php
Saturday, August 17, 13
![Page 47: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/47.jpg)
Optimizing PHP
• See last year’s talk• Use an Opcode Cache
• xhprof
• StatsD/Graphite
• Find hotspots
http://www.slideshare.net/jnklein/northeast-php-high-performance-php
Saturday, August 17, 13
![Page 48: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/48.jpg)
Optimizing PHP
• See last year’s talk• Use an Opcode Cache
• xhprof
• StatsD/Graphite
• Find hotspots
• Upgrade it
http://www.slideshare.net/jnklein/northeast-php-high-performance-php
Saturday, August 17, 13
![Page 49: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/49.jpg)
Static Arrays
Saturday, August 17, 13
![Page 50: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/50.jpg)
Translationsclass language_de {
static $translations = array( "<hash>" => array("content" => 'Accessoires', "file" => "TAXONOMY"), "<hash>" => array("content" => 'Keramik', "file" => "CATEGORY"), "<hash>" => array("content" => 'Dekorieren', "file" => "CATEGORY"), "<hash>" => array("content" => 'Getaggt %s', "file" => "Foo.php"), "<hash>" => array("content" => 'Badezusatz', "file" => "CATEGORY"), "<hash>" => array("content" => 'Datum:{% $date %}', "file" => "bar.tpl")
... snip ...
);}
Saturday, August 17, 13
![Page 51: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/51.jpg)
Saturday, August 17, 13
![Page 52: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/52.jpg)
Saturday, August 17, 13
![Page 53: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/53.jpg)
Atomic Deploys
Saturday, August 17, 13
![Page 54: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/54.jpg)
Deploying PHP
http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/
Saturday, August 17, 13
![Page 55: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/55.jpg)
Deploying PHP
• mod_realdoc - Apache module
http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/
Saturday, August 17, 13
![Page 56: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/56.jpg)
Deploying PHP
• mod_realdoc - Apache module• incpath - PHP Extension
http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/
Saturday, August 17, 13
![Page 57: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/57.jpg)
Deploying PHP
• mod_realdoc - Apache module• incpath - PHP Extension• A/B symlink swap
http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/
Saturday, August 17, 13
![Page 58: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/58.jpg)
Deploying PHP
• mod_realdoc - Apache module• incpath - PHP Extension• A/B symlink swap• Avoid recompilation to opcodes
http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/
Saturday, August 17, 13
![Page 59: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/59.jpg)
Upgrading PHP
Saturday, August 17, 13
![Page 60: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/60.jpg)
PHP 5.4 vs. 5.3 - CPU
PHP 5.3
PHP 5.4
Saturday, August 17, 13
![Page 61: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/61.jpg)
PHP 5.4 vs. 5.3 - Memory
PHP 5.3
PHP 5.4
Saturday, August 17, 13
![Page 62: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/62.jpg)
Saturday, August 17, 13
![Page 63: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/63.jpg)
Understand Framework
Overhead
Saturday, August 17, 13
![Page 64: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/64.jpg)
http://systemsarchitect.net/performance-benchmark-of-popular-php-frameworks/
Saturday, August 17, 13
![Page 65: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/65.jpg)
Saturday, August 17, 13
![Page 66: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/66.jpg)
Saturday, August 17, 13
![Page 67: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/67.jpg)
Saturday, August 17, 13
![Page 68: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/68.jpg)
Frameworks are Harder to Scale
Saturday, August 17, 13
![Page 69: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/69.jpg)
Saturday, August 17, 13
![Page 70: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/70.jpg)
Memcached
Saturday, August 17, 13
![Page 71: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/71.jpg)
Memcached
• Roughly same hardware as webs
Saturday, August 17, 13
![Page 72: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/72.jpg)
Memcached
• Roughly same hardware as webs• More memory (48GB)
Saturday, August 17, 13
![Page 73: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/73.jpg)
Memcached
• Roughly same hardware as webs• More memory (48GB)• Shard keys across servers
Saturday, August 17, 13
![Page 74: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/74.jpg)
Memcached
• Roughly same hardware as webs• More memory (48GB)• Shard keys across servers• mctop - https://github.com/etsy/mctop
Saturday, August 17, 13
![Page 75: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/75.jpg)
Saturday, August 17, 13
![Page 76: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/76.jpg)
MySQL
Saturday, August 17, 13
![Page 77: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/77.jpg)
MySQL
• MySQL 5.5 (http://jkle.in/nephp)
Saturday, August 17, 13
![Page 78: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/78.jpg)
MySQL
• MySQL 5.5 (http://jkle.in/nephp)• ~20 master/master pairs
Saturday, August 17, 13
![Page 79: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/79.jpg)
MySQL
• MySQL 5.5 (http://jkle.in/nephp)• ~20 master/master pairs• innodb
Saturday, August 17, 13
![Page 80: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/80.jpg)
MySQL
• MySQL 5.5 (http://jkle.in/nephp)• ~20 master/master pairs• innodb• thread_cache=800
Saturday, August 17, 13
![Page 81: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/81.jpg)
MySQL
• MySQL 5.5 (http://jkle.in/nephp)• ~20 master/master pairs• innodb• thread_cache=800• max_connections=2500
Saturday, August 17, 13
![Page 82: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/82.jpg)
MySQL
• MySQL 5.5 (http://jkle.in/nephp)• ~20 master/master pairs• innodb• thread_cache=800• max_connections=2500• [Almost] no joins
Saturday, August 17, 13
![Page 83: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/83.jpg)
Theoretically we can serve ~9000 req/sec
Saturday, August 17, 13
![Page 84: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/84.jpg)
Saturday, August 17, 13
![Page 85: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/85.jpg)
Static Content
Saturday, August 17, 13
![Page 86: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/86.jpg)
Saturday, August 17, 13
![Page 87: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/87.jpg)
~500 Million Images
Saturday, August 17, 13
![Page 88: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/88.jpg)
Offloading Requests
Saturday, August 17, 13
![Page 89: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/89.jpg)
Offloading Requests
• Multiple CDNs
Saturday, August 17, 13
![Page 90: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/90.jpg)
Offloading Requests
• Multiple CDNs• Img hit rate > 90%
Saturday, August 17, 13
![Page 91: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/91.jpg)
Offloading Requests
• Multiple CDNs• Img hit rate > 90%• CSS/JS hit rate ~100%
Saturday, August 17, 13
![Page 92: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/92.jpg)
Offloading Requests
• Multiple CDNs• Img hit rate > 90%• CSS/JS hit rate ~100%• Edge serves ~7000 reqs/sec
Saturday, August 17, 13
![Page 93: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/93.jpg)
What about misses?
Saturday, August 17, 13
![Page 94: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/94.jpg)
Saturday, August 17, 13
![Page 95: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/95.jpg)
Change Management
Saturday, August 17, 13
![Page 96: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/96.jpg)
Feature Flags
Saturday, August 17, 13
![Page 97: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/97.jpg)
Saturday, August 17, 13
![Page 98: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/98.jpg)
// Owner: Username$server_config['foo']['bar'] = array( 'users' => array('jsmith', 'jdoe'),);
// Owner: Username$server_config['baz'] = [ 'enabled' => [ 'variant1' => 0, 'variant2' => 0, 'variant3' => 50 ],];
// Owner: Username$server_config['qux'] = array( 'enabled' => 100, 'data' => array( 'somedata' ));
Saturday, August 17, 13
![Page 99: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/99.jpg)
if (Feature::isEnabled('foo.bar')) { // Code}
Saturday, August 17, 13
![Page 100: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/100.jpg)
Ramp-up:1%, 5%, 25%, 100%
Saturday, August 17, 13
![Page 101: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/101.jpg)
Deployinator
Saturday, August 17, 13
![Page 102: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/102.jpg)
Saturday, August 17, 13
![Page 103: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/103.jpg)
Schemanator
Saturday, August 17, 13
![Page 104: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/104.jpg)
Saturday, August 17, 13
![Page 105: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/105.jpg)
Recent Improvements
Saturday, August 17, 13
![Page 106: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/106.jpg)
Saturday, August 17, 13
![Page 107: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/107.jpg)
Sandy Bridge
Saturday, August 17, 13
![Page 108: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/108.jpg)
Baseline Performance (median)
Saturday, August 17, 13
![Page 109: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/109.jpg)
Small code change
Baseline Performance (median)
Saturday, August 17, 13
![Page 110: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/110.jpg)
Small code change
PHP 5.4
Baseline Performance (median)
Saturday, August 17, 13
![Page 111: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/111.jpg)
Small code change
PHP 5.4
Disabled Hyperthreading
Baseline Performance (median)
Saturday, August 17, 13
![Page 112: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/112.jpg)
Challenges
Saturday, August 17, 13
![Page 113: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/113.jpg)
Saturday, August 17, 13
![Page 114: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/114.jpg)
Options
Saturday, August 17, 13
![Page 115: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/115.jpg)
Options
• Scale Horizontally (more shards)
Saturday, August 17, 13
![Page 116: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/116.jpg)
Options
• Scale Horizontally (more shards)• Scale Vertically (SSD, bigger drives)
Saturday, August 17, 13
![Page 117: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/117.jpg)
Options
• Scale Horizontally (more shards)• Scale Vertically (SSD, bigger drives)• Change architecture
Saturday, August 17, 13
![Page 118: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/118.jpg)
Saturday, August 17, 13
![Page 119: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/119.jpg)
Architecture Changes
Saturday, August 17, 13
![Page 120: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/120.jpg)
Architecture Changes
• Ongoing conversation
Saturday, August 17, 13
![Page 121: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/121.jpg)
Architecture Changes
• Ongoing conversation• Separate logical/physical shards
Saturday, August 17, 13
![Page 122: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/122.jpg)
Architecture Changes
• Ongoing conversation• Separate logical/physical shards• Complex
Saturday, August 17, 13
![Page 123: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/123.jpg)
Architecture Changes
• Ongoing conversation• Separate logical/physical shards• Complex• Server failure
Saturday, August 17, 13
![Page 124: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/124.jpg)
Architecture Changes
• Ongoing conversation• Separate logical/physical shards• Complex• Server failure
• Migrating data
Saturday, August 17, 13
![Page 125: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/125.jpg)
Architecture Changes
• Ongoing conversation• Separate logical/physical shards• Complex• Server failure
• Migrating data
• Consistency
Saturday, August 17, 13
![Page 126: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/126.jpg)
Monitoring
Saturday, August 17, 13
![Page 127: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/127.jpg)
Graphite
Saturday, August 17, 13
![Page 128: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/128.jpg)
Stacked Search Timers
Saturday, August 17, 13
![Page 129: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/129.jpg)
Ganglia
Saturday, August 17, 13
![Page 130: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/130.jpg)
Saturday, August 17, 13
![Page 131: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/131.jpg)
Nagios
Saturday, August 17, 13
![Page 132: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/132.jpg)
Saturday, August 17, 13
![Page 133: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/133.jpg)
Saturday, August 17, 13
![Page 134: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/134.jpg)
Takeaways
Saturday, August 17, 13
![Page 135: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/135.jpg)
K.I.S.S.
Saturday, August 17, 13
![Page 136: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/136.jpg)
Use Proven Technologies
Saturday, August 17, 13
![Page 137: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/137.jpg)
Saturday, August 17, 13
![Page 138: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/138.jpg)
Understand Your Stack
Saturday, August 17, 13
![Page 139: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/139.jpg)
Measure Everything
Saturday, August 17, 13
![Page 140: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/140.jpg)
Work at Etsy ;-)
Saturday, August 17, 13
![Page 141: Scaling PHP to 40 Million Uniques](https://reader034.fdocuments.us/reader034/viewer/2022051323/54b6d91f4a795994658b45a3/html5/thumbnails/141.jpg)
Connecthttp://www.meetup.com/Web-Performance-Boston/
www.etsy.com/careers
@jonathanklein
Saturday, August 17, 13