How Badoo Saved $1M Switching to PHP7 - Nikolay Krapivnyy - PHPDay Verona 2016
-
Upload
badoo -
Category
Technology
-
view
493 -
download
8
Transcript of How Badoo Saved $1M Switching to PHP7 - Nikolay Krapivnyy - PHPDay Verona 2016
About me✦ Nikolay Krapivnyy
✦ 10+ years PHP experience
✦ 5 years with Badoo, now backend team lead
✦ This is my first talk in English :)
46languages
data
300Musers190
countries
“ BADOO IS THE LARGEST DATING NETWORK IN THE WORLD— Forbes
Badoo tech:
✦ 4 datacenters, ~3,000 servers overall
✦ up to 80,000 RPS for php backend
✦ ~200 engineers (~70 php devs)
✦ >2M PHP LoC
✦ and growing….
Today I will:✦ Share our practical experience
✦ Talk about some of our tools
✦ Tell you a million dollar story :)
1. Problem 2. Transition 3. Results
1. Problem
DBs
CLOUD SERVICES
LTM
How it works:APP
HTTPHTTP
API
API
9 %
15 %
22 %22 %
32 %phpmysqlotherphotosc/go
Hardware
32% or 1000of our servers are running php
PHP Cluster Growth
0
125
250
375
500
Jan, 2014 Jul, 2014 Jan, 2015 Jul, 2015 Jan, 2016
servers in 1DC users
CPU is the main bottleneck for us
How do we measure php performance?
Pinba!✦ Php is not a bottleneck anymore
✦ RT performance monitoring tool
✦ Extension for php + plugin for mysql
✦ Open source! http://pinba.org/
Request time
✦ Some API call: GET_USER
✦ avg request time = 0.12 sec
✦ request time = cpu time + i/o time
✦ i/o time = mysql + services + FS i/o +…
Pinba
Particular requests time
Pinba report
= +
Request time
✦ 0.12 sec = 0.07 (cpu) + 0.05 (i/o)
✦ 0.05 (i/o) = 0.012 + 0.05 + 0.03 + …
✦ 0.012 (memcache) = ~26*0.0005
✦ Realtime!
CPU usage takes ~50% of our request time
PHP7 alternatives?✦ HHVM (Facebook, http://hhvm.com/)
✦ kPHP (VK.com, https://github.com/vk-com/kphp-kdb)
+ JIT, PHP5 compatible— C++, extensions API
+ faster then HHVM
— no OOP support at all
2. Transition
Platform team:✦ C developers
✦ Php developers
✦ Badoo infrastructure support
Transition steps:2.1. Extensions
2.2. Codebase
2.3. Deploy
PHP7: extensions
xdebug
apcu
lua
imagick
✦ ~30 of them
✦ Already supported:
PHP7: extensions
blitz
xhprof
handlersocketi
pinba
leptonica
tarantool
sphinx
memtrack
judy
&& 15 more…
PHP7: extensions
https://github.com/tony2001
All php7 versions available:
Check for php7 branch
We thought we were done, but…
Runkit
What is runkit?
✦ runkit_method_add/redefine
✦ runkit_constant_add/redefine
✦ in runtime
✦ but why?
Unit testing!but why?
Unit testing:✦ 60K Tests now
✦ Coverage >50%
✦ Introduced only a few years ago
✦ Runkit: to isolate old code
Runkit: summary✦ essential for us
✦ no support for php7
Runkit for php7?✦ It`s different
✦ Alternatives? • UOPZ (https://github.com/krakjoe/uopz)
✦ Lots of changes anyway
Let`s remove this dependency
Soft mocks!✦ Soft means no extension
✦ Plain PHP, compatible with all PHP
✦ Replaces Runkit for us
✦ Open source! (https://github.com/badoo/soft-mocks)
How it works:✦ Rewrites code on the fly
✦ Inserts "interceptors" in methods
✦ Wraps all constant usages
✦ Uses PHP-Parser
1 require_once(__DIR__ . '/path.inc');2 ini_set(‘pinba.enabled', 1);3 echo RRD_BASE_PATH;
1 require_once SoftMocks::rewrite( ROOT_PATH . '/path' . '/path.inc' );2 SoftMocks::call( 'ini_set', array('pinba.enabled', 1’) );3 echo SoftMocks::getConst(‘RRD_BASE_PATH');
Original
After SoftMocks::rewrite()
1 class A2 {3 public function getValue()4 {5 return 10;6 }7 }
1 class A2 {3 public function getValue()4 { if (SoftMocks::isMocked(A::class, __FUNCTION__)) { return SoftMocks::getResult(…); }
5 return 10;6 }7 }
Original
After SoftMocks::rewrite()
So…
✦ No extension dependency
✦ 1.5 weeks for development
✦ A bit longer to adapt tests
✦ More stable tests in general
https://github.com/YuriyNasretdinov
CreditsMore cool stuff:
Soft mocks:
https://github.com/badoo/soft-mocks
2.2. Codebase
php7mar
✦ https://github.com/Alexia/php7mar
✦ Migration Assistant Report
php7mar 469 * foreachByReference 230 * oldClassConstructors 156 * funcGetArg 22 * hexadecimalString 19 * variableInterpolation 14 * yield 7 * deprecatedFunctions 4 * reservedNames 3 * arrayValueByReference
2.3. Deploy
Deploy✦ Devel: separate sandbox
✦ Staging
✦ Production
Deploy: production ✦ Several hosts in each cluster
✦ Several weeks of quarantine
✦ CLI: OK, WEB: ERROR
✦ WEB: fix for opcache reset in FPM
3. Results
CPU usage
cluster CPU usage
memory usage
We got: 2x CPU usage 9x memory usage
✦ 700 servers running FPM
✦ 1/2 required
✦ ~$4K each
✦ 350*4K = $1,4M
Let's calculate
Links✦ Pinba: http://pinba.org/
✦ Soft mocks: https://github.com/badoo/soft-mocks
✦ php7mar: https://github.com/Alexia/php7mar
✦ php7 exts: https://github.com/tony2001
Summary:✦ PHP7 rocks!
✦ Know your bottleneck
✦ Find your way
✦ Share!
Grazie!
Questions?
Twitter: @BadooTech
Blog: https://techblog.badoo.com
LinkedIn: https://linkedin.com/in/nkrapivnyy
Email: [email protected]
Joind.in: https://joind.in/talk/948d4