1. Nginx and friendsaka Adding a turbo button to your site Wim Godden Cu.be Solutions

2. Who am I ?Wim Godden (@wimgtr)Owner of Cu.be Solutions (http://cu.be)Open source developer since 1997Developer of OpenXZend Certified EngineerZend Framework Certified EngineerMySQL Certified Developer 3. Who are you ?Developers ?System/network engineers ?Managers ? 4. The Microsoft box 5. Whats outside the box ?Operating systems ?Database servers ?Webservers ?Reverse proxies ? 6. Website XHeader Latest newsArticle content page NavigationArticle content 7. What is caching ? x = 5, y = 2 Same resultn = 50 CACHE select * from articlejoin user Doesnt changeon article.user_id = user.idall the time order by created desc limit 10 8. Caching storage - Memcache(d)Facebook, Twitter, YouTube, need we say more ?Distributed memory caching systemMultiple machines 1 big memory-based hash-tableKey-value storage systemKeys - max. 250bytesValues - max. 1Mbyte 9. Caching storage - Memcache(d)Facebook, Twitter, YouTube, need we say more ?Distributed memory caching systemKey-value storage system Keys - max. 250bytes Values - max. 1MbyteExtremely fast... non-blocking, UDP (!) 10. Memcache - installation & running itInstallation Distribution package PECL Windows : binariesRunning No config-files memcached -d -m -l -p ex. : memcached -d -m 2048 -l 172.16.1.91 -p 11211 11. Caching storage - Memcache - some notesNot fault-tolerant Its a cache ! Lose session data Lose shopping cart data ... 12. Caching storage - Memcache - some notesNot fault-tolerant Its a cache ! Lose session data Lose shopping cart data Firewall your Memcache port ! 13. Memcache in codeInitialization :MemcachedClient.Setup("MyCache", new string[]{ "server1.example.com", "server2.example.com"});Usage :MemcachedClient cache = MemcachedClient.GetInstance("MyCache");string myData = cache.Get("myKey") as string;if (myData == null) { MissmyData = GetMyDataFromDB();cache.Set("myKey", myData);}// Use myData 14. Memcache slabs(or why Memcache says its full when its not)Multiple slabs of different sizes : Slab 1 : 400 bytes Slab 2 : 480 bytes (400 * 1.2) Slab 3 : 576 bytes (480 * 1.2) (and so on...)Multiplier (1.2 here) can be configuredStore a lot of very large objects Large slabs : full Rest : free Eviction of data ! 15. Memcache - Is it working ?Connect to it using telnet STAT pid 2941 STAT uptime 10878 STAT time 1296074240 "stats" command STAT version 1.4.5 STAT pointer_size 64 STAT rusage_user 20.089945 Use Cacti or other monitoring tools STAT rusage_system 58.499106 STAT curr_connections 16 STAT total_connections 276950 STAT connection_structures 96 STAT cmd_get 276931 STAT cmd_set 584148 STAT cmd_flush 0 STAT get_hits 211106 STAT get_misses 65825 STAT delete_misses 101 STAT delete_hits 276829 STAT incr_misses 0 STAT incr_hits 0 STAT decr_misses 0 STAT decr_hits 0 STAT cas_misses 0 STAT cas_hits 0 STAT cas_badval 0 STAT auth_cmds 0 STAT auth_errors 0 STAT bytes_read 613193860 STAT bytes_written 553991373 STAT limit_maxbytes 268435456 STAT accepting_conns 1 STAT listen_disabled_num 0 STAT threads 4 STAT conn_yields 0 STAT bytes 20418140 STAT curr_items 65826 STAT total_items 553856 STAT evictions 0 STAT reclaimed 0 16. Website X news is changing ! HeaderLatest news Article content pageNavigation Article content 17. cache.Delete("Latest-news") 18. Updating data 19. Updating dataLCD_Popular_Product_List 20. Adding/updating datacache.Delete("LCD_Popular_Product_List") 21. Adding/updating data 22. Adding/updating data - Why it crashed 23. Adding/updating data - Why it crashed 24. Adding/updating data - Why it crashed 25. Cache stampeding 26. Cache stampeding 27. Memcache code ?Memcache code Visitor interfaceAdmin interface DB 28. Cache warmup scriptsUsed to fill your cache when its emptyRun it before starting Webserver !2 ways : Visit all URLs Error-prone Hard to maintain Call all cache-updating methods Make sure you have a warmup script ! 29. Cache stampeding - what about locking ?Seems like a nice idea, but...While lock in placeWhat if the process that created the lock fails ? 30. Website XHeader Latest newsArticle content page NavigationArticle content 31. Website XHeader(TTL = 2h) Latest newsArticle content page Navigation (TTL = 2h)Article content 32. Website XHeader(TTL = 2h) Latest news (TTL = 2m)Article content page Navigation (TTL = 2h) Article content (TTL = 15m) 33. Move the cache 34. VarnishNot just a load balancerReverse proxy cache / http accelerator / Caches (parts of) pages in memory 35. Varnish - ESIIn your article page output : Header (TTL : 60 min) /top Latest news (TTL : 2 min) /newsNavigationArticle content page(TTL :In your Varnish config : 60 min) Article content (TTL : 15 min) sub vcl_fetch { /nav/article/732if (req.url == "/news") { esi; /* Do ESI processing */ set obj.ttl = 2m; } elseif (req.url == "/nav") { esi; set obj.ttl = 1h; } elseif ..} 36. A simple benchmark 2KByte JPEG Apache 2.24210 IIS 7.5 3960 Varnish 3.0 11400 37. A dynamically generated, but static page Apache 2.2 + PHP (3 DB queries) 18 IIS 7.5 + .Net (3 DB queries) 16 Varnish 3.0 11400 38. Varnish - what can/cant be cached ?Can : Static pages Images, js, css Static parts of pages that dont change often (ESI)Cant : POST requests Very large files (its not a file server !) Requests with Set-Cookie User-specific content 39. ESI no caching on user-specific content ? Logged in as : Wim Godden TTL = 0s ?5 messagesTTL=1hTTL = 5min 40. Dynamic contentRequest Varnish Ideallyuser X : /top /top request to webserver sameuser Y : /top /top request to webserver sameuser X : /top /top request to webserver if unchanged : served from cacheuser Y : /top /top request to webserver if unchanged : served from cache 41. NginxWeb serverReverse proxyLightweight, fast12.2% of all Websites 42. NginxNo threads, event-drivenUses epoll / kqueueLow memory footprint10000 active connections = normal 43. Coming to Nginx soon... Logged in as : Wim Godden5 messages Menu NEWS 44. Coming to Nginx soon...ttl="1h" /> 45. Requesting /page (1st time) Nginx13Shared memory/page 2 4 /page 46. Requesting /page ESI subrequests (1st time)Nginx123/menu/news/top ($sessionid$) 47. Requesting /page (next time)Nginx 1 Shared memory /page2/page/menu/news/top ($sessionid$) 48. New message arrives...POST /send o ... se i nt t(.e rt ..) insDB top ($sessionid$) 49. AdvantagesNo repeated hits to webserver anymore ! Only the initial hit (unless you pre-heat/warm up the cache !) No hits for user-specific content Not even for non-specific content 50. News addedaddnews() methodo ...sei ntt(. e rt..)ins DB/news 51. AdvantagesNo repeated hits to webserver anymore ! Only the initial hit (unless you pre-heat/warm up the cache !) Not for user-specific content Not even for non-specific content No TTLs for non-specific content Imagine doing it for the bid status on Ebay items ;-) 52. How many Memcache requests ? Logged in as : Wim Godden5 messages ttl="1h" /> 53. Why Nginx ?Native Memcache supportExcellent and superfast subrequest system Including parallel subrequestsHandles thousands of connections per worker With minimal memory footprint 54. Whats the result ? 55. FiguresFirst customer : No. of web servers : 18 4 No. of db servers : 6 2 Total : 24 6 (75% reduction !)Second customer (already using Nginx + Memcache) : No. of web servers : 72 8 No. of db servers : 15 4 Total : 87 12 (86% reduction !) 56. AvailabilityStable at 2 customersStill under heavy developmentBeta : July 2012Final : Sep 2012 57. So... 58. So...Theres life outside the MS-box And its pretty awesome ;-)Need caching in .Net (for DB queries, config-files, etc.) ? Memcache Velocity ?Caching static pages Varnish with ESICaching for authenticated users Nginx with Dynamic ESITheres a lot more outside the box Maybe some other time ;-) 59. Questions ? 60. Questions ? 61. ContactTwitter@wimgtrWeb http://techblog.wimgodden.beSlideshttp://www.slideshare.net/wimgE-mailwim.godden@cu.be 62. Thanks !Please rate my talk :http://speakerrate.com/talks/11081