Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo
-
Upload
per-buer -
Category
Technology
-
view
225 -
download
1
description
Transcript of Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo
![Page 1: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/1.jpg)
![Page 2: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/2.jpg)
Varnish Cache 4.0Whats new
![Page 3: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/3.jpg)
Varnish 3.0
• Released in 2011
• Added modules
• gzip
• basic streaming capabilities
![Page 4: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/4.jpg)
![Page 5: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/5.jpg)
Security in V4
• -r for read only parameters
• Locks down CLI
• user, group and cc_command
• Inline C is default off
![Page 6: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/6.jpg)
![Page 7: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/7.jpg)
Threading in Varnish 4
Clientthread
Backend thread
Varnish Cache
Client Web server
![Page 8: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/8.jpg)
Varnish Cache
Grace in V4
Clientthread
Backend thread
Client Web server
GET /stale-object
![Page 9: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/9.jpg)
Varnish Cache
Grace in V4
Clientthread
Backend thread
Client Web serverStale object
delivered
Background fetch scheduled
![Page 10: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/10.jpg)
Varnish Cache
Grace in V4
Clientthread
Backend thread
Client Web server
Object refreshed
![Page 11: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/11.jpg)
![Page 12: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/12.jpg)
Streaming
• V3 will add a bit of delay before starting delivery on cache misses
• From “store and forward” to “cut through”
• Set do_stream = true in V4
• Beneficial for large objects and cache hierarchies
![Page 13: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/13.jpg)
Streaming in Varnish
Backend thread
Varnish Cache
Web server
ClientClientthread
ClientClientthread
Client
Clientthread
GET /big-object-to-be-streamed
![Page 14: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/14.jpg)
Upcoming in V4 Plus
• New storage backend
• Supports >1TB of storage
• Persists cache across crash/reboot
• Highly performant on SSDs
• Only available in Varnish Plus
![Page 15: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/15.jpg)
![Page 16: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/16.jpg)
Logging in V3
• Logging in V3 is limited
• Only regex matching using &&
• No req/bereq relationship
• Performance problems
![Page 17: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/17.jpg)
• Transactions and transactions groups
• Query language
• Output control
• Increased performance (zero copy)
Logging in V4
![Page 18: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/18.jpg)
Log transactions
• On work item for Varnish
• client request
• backend request
• ESI sub-request
• session
![Page 19: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/19.jpg)
Transactions groups
• Transactions (work items) can be grouped
• by VXID
• by request
• by session
• raw varnishlog -g (vxid | request | session | raw )
![Page 20: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/20.jpg)
Log query language"$ varnishlog -g request \
-q 'ReqURL eq "/"' """$ varnishlog -g request \
-q 'Backend ~ default'
* << Request >> 32770 - Begin req 32769 - ReqMethod GET- ReqURL /- ReqProtocol HTTP/1.1 - ReqHeader TE: deflate,gzip;q=0.3 ... - Link bereq 32771 - VCL_call DELIVER - VCL_return deliver - RespProtocol HTTP/1.1 - RespStatus 200 - RespResponse OK - ReqEnd 1385330985.979025126 1385330985.978960991 -0.001315594 0.001251459 -0.001315594 - End ** << BeReq >> 32771 -- Begin bereq 32770 -- VCL_call BACKEND_FETCH -- VCL_return fetch -- BackendOpen 18 default(127.0.0.1,::1,8020) 127.0.0.1 45989 -- Backend 18 default default(127.0.0.1,::1,8020) -- BereqMethod GET -- BereqURL / -- BereqEnd 1385330985.979187250 1385330985.980367422 0.000082792 0.000496101 0.000326045 0.000822146-- End
![Page 21: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/21.jpg)
Examples"
String matching, negation, logical operations
not (RespProtocol eq “HTTP/1.1”) or (RespProtcol eq “HTTP/1.0”))!
Regular expressions
! ReqMethod !~ "GET|POST"!
Integer matching
! (RespStatus >= 200 and RespStatus < 300)!
Float matching
! Timestamp:Process[2] > 0.5
![Page 22: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/22.jpg)
Mind blown?
![Page 23: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/23.jpg)
Output control
• A bit like “grep” for varnishlog
• Applied last, doesn’t affect queries
• -i <taglist> / -I <taglist:regex>
• -x <taglist> / -X <taglist:regex>
• Taglists supports globbing (e.g. Req*)
![Page 24: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/24.jpg)
![Page 25: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/25.jpg)
Load balancing
• Mostly feature parity… however
• Directors are VMODs now
• Directors typically defined in vcl_init
• Easy to implement new directors
• Directors are now stackable
![Page 26: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/26.jpg)
Summing up
• Streaming is much improved in V4
• New mind blowing logging facility
• Backend/frontend threading
• IMS towards backend
• Performance increase
• VCL changes
• Reworked documentation (varnish.org/docs)
![Page 27: Varnish Cache 4.0 / Redpill Linpro breakfast in Oslo](https://reader033.fdocuments.us/reader033/viewer/2022051610/54922e2fac795925288b470e/html5/thumbnails/27.jpg)
Other neat stuff
• Auto-tuner coming to Varnish Cache Plus
• Clustering for Varnish Cache Plus
• Real time analytics (VCS)