Trade me use case
-
Upload
varnish-software -
Category
Technology
-
view
64 -
download
1
Transcript of Trade me use case
![Page 1: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/1.jpg)
1
Trade Me: Solving Real-World Problems With Varnish Cache
April 2016
![Page 2: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/2.jpg)
About Trade Me
• NZ’s largest domestic website
• Auctions & Classifieds• Jobs, Property, Motors• Used to be just
www.trademe.co.nz• Now we have a public API
too
![Page 3: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/3.jpg)
Some Stats
• Over 60 million requests/day (desktop)• Same again for API• 700+ active 3rd party apps consuming our API• 5 first party apps (iOS/Android/websites)• Over 60% of our internet traffic is photos & other static
content• We achieve ~95% hitrate for static content with Varnish
![Page 4: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/4.jpg)
More Stats
![Page 5: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/5.jpg)
6
Problem #1
Integrating a web app with a CMS
![Page 6: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/6.jpg)
Problem statement
• Pages with static text were part of the main project• Text changes required a full deploy • Deploy requires involvement from several areas• Took too long to make simple changes, higher risk• Applied to several sections of the site, we wanted to split
into separate “microsites”• “Hey, we should use a CMS!”
![Page 7: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/7.jpg)
CMS – challenges
• Needed a way of keeping the dynamic header and footer from the main site, while adding CMS functionality for creating and editing content
• CMS microsites needed to reflect changes to main site’s CSS and JS
• TL;DR problem – how to seamlessly integrate a separate app (on different servers) into the main site
![Page 8: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/8.jpg)
![Page 9: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/9.jpg)
Header
Footer
Content
![Page 10: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/10.jpg)
![Page 11: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/11.jpg)
Custom logoGreeting
Shopping cart count
Search history
Dynamic drop-down menus
TimeAdvertising
![Page 12: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/12.jpg)
Normal page request
Load balancer
Main siteSite Content
Footer
Header
![Page 13: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/13.jpg)
Request for CMS page
Load balancer
Varnish (ESI)
CMSCMS Content
URL rule - eg /help/*
<esi:header.aspx>
<esi:footer.aspx>
![Page 14: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/14.jpg)
Request for CMS page
Load balancer
Main site
Varnish (ESI)
CMS
URL rule - eg /help/*
header.aspxfooter.aspx
CMS Content
<esi:header.aspx>
<esi:footer.aspx>
Header
Footer
![Page 15: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/15.jpg)
![Page 16: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/16.jpg)
17
Problem #2
Changing backend file structure (with no downtime)
![Page 17: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/17.jpg)
Problem statement
• File metadata performance concerns on storage• Needed to change how image files were stored on disk :• From 100 dirs. per image size to 1000
![Page 18: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/18.jpg)
Changing backend file structure – challenges
• Needed to change underlying file storage scheme• Changes to file upload/save code• Changes to image URLs in web application• Tidy up URLs at the same time (why not?)• Remove “_full” from Fullsize image filename• Remove directory ID from all URIs
• All without causing any user-facing errors
![Page 19: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/19.jpg)
Basic steps
1. Agree image ID for change2. Code changes (inactive)3. VCL changes (inactive)4. [activation happens]5. Backfill all active images to the new dir structure6. Remove all old logic
![Page 20: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/20.jpg)
Regular Expressions …are hard Even when you are staring
at them, it takes a while to process what you’re looking at.
![Page 21: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/21.jpg)
Regex – use a tool
![Page 22: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/22.jpg)
VCL Changes
• Extract image ID from URL (regex)• If image ID > $threshold• Rewrite URL for backend (add folder, remove _full from filename)• Log all changes to Varnishlog
![Page 23: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/23.jpg)
VCL
We’ll need this later
![Page 24: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/24.jpg)
VCL – new sub
Extract the image ID from the URLStore it in an HTTP header
![Page 25: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/25.jpg)
vcl_recv
If the image ID is new, ensure we use the thousand directories
![Page 26: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/26.jpg)
VCL
Rewrite From (client request) Rewritten To (backend request)/photoserver/thumb/12345.jpg /photoserver/thumb/
345/12345.jpg/photoserver/thumb/45/12345.jpg
/photoserver/thumb/345/12345.jpg
/photoserver/full/12345.jpg /photoserver/full/345/12345.jpg
/photoserver/45/12345_full.jpg /photoserver/full/345/12345.jpg
/photoserver/lv/0/1234500.jpg /photoserver/lv/500/1234500.jpg
Here’s some examples showing how we consistently rewrite requests for old and new formats to what we need on the backend
![Page 27: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/27.jpg)
What’s next for us with Varnish?
• Move servers to private cloud• Investigate object storage• Investigate using Varnish API Engine for our public API• Better stats and management
![Page 28: Trade me use case](https://reader035.fdocuments.us/reader035/viewer/2022070602/5878cf301a28ab917a8b4a4b/html5/thumbnails/28.jpg)
Questions