Post on 23-Aug-2014
description
Choosing A Proxy ServerApacheCon 2014
Bryan CallATS Committer / Yahoo
About Me• Yahoo! Employee
– WebRing, GeoCities, Personals, Tiger Team, Platform Architect, Edge Team, Research, ATS and HTTP
(HTTP/2 and TLS at IETF)• Working on Traffic Server for 7 years
– Since 2007• Part of the team that open sourced it in 2009• ATS Committer
Overview• Types of Proxies• Features• Architecture• Cache Architecture• Performance• Pros and Cons
How are you going to use a proxy server?
Reverse Proxy
Reverse Proxy• Proxy in front of your own web servers• Caching?• Geographic location?• Connection handling?• SSL termination?• SPDY support?• Adding business logic?
Forward Proxy
Intercepting Proxy
Forward / Intercepting Proxy• Proxy in front of the Internet• Configure clients to use proxy?• Caching?• SSL - CONNECT?• SSL - termination?
Choices
Plenty of Proxy Servers
PerlBal
Plenty of Proxy Servers
Features And Options
FeaturesATS NGiNX Squid Varnish Apache httpd
mod_proxy
Reverse Proxy Y Y Y Y YForward Proxy Y N Y N YTransp. Proxy Y N Y N YPlugin APIs Y Y partial Y YCache Y Y Y Y YESI Y N Y partial NICP Y N Y N NSSL Y Y Y N YSPDY Y* Y N N partial
* 5.0.0 (May 2014)
SSL Features
Source: https://istlsfastyet.com/ - Ilya Grigorik
What type of proxy do you need?• Of our candidates, only three fully supports all
proxy modes
HTTP/1.1 Compliance
HTTP/1.1 Compliance• Accept-Encoding - gzip• Vary• Age• If-None-Match
How things can go wrong: Vary$ curl -D - -o /dev/null -s --compress http://10.118.73.168/HTTP/1.1 200 OKServer: nginx/1.3.9Date: Wed, 12 Dec 2012 18:00:48 GMTContent-Type: text/html; charset=utf-8Content-Length: 8051Connection: keep-aliveCache-Control: public, max-age=900Last-Modified: Wed, 12 Dec 2012 17:52:42 +0000Expires: Sun, 19 Nov 1978 05:00:00 GMTVary: Cookie,Accept-EncodingContent-Encoding: gzip
How things can go wrong: Vary$ curl -D - -o /dev/null -s http://10.118.73.168/HTTP/1.1 200 OKServer: nginx/1.3.9Date: Wed, 12 Dec 2012 18:00:57 GMTContent-Type: text/html; charset=utf-8Content-Length: 8051Connection: keep-aliveCache-Control: public, max-age=900Last-Modified: Wed, 12 Dec 2012 17:52:42 +0000Expires: Sun, 19 Nov 1978 05:00:00 GMTVary: Cookie,Accept-EncodingContent-Encoding: gzip
EPIC FAIL!
Note: no gzip request
CoAdvisor HTTP protocol quality tests for reverse proxies
ATS 3.3.1
Nginx 1.3.9
Squid 3.2.5
Varnish 3.0.3
0 100 200 300 400 500 600
Failures Violations Success
49%
81%
51%
68%
CoAdvisor HTTP protocol quality tests for reverse proxies
ATS 3.3.1
Nginx 1.3.9
Squid 3.2.5
Varnish 3.0.3
0 100 200 300 400 500 600
Failures Violations Success
25%
6%
27%
15%
Architecture
Architecture And Process Models• Multithreading• Events• Process• Fibers
– Co-operative multitasking, getcontext/setcontext
Threads
Threads• Pros
– Easy to share memory– Lightweight context switching
• Cons– Easy to (accidently) share memory
• Overwriting another threads memory– Locking
• Deadlocks, race conditions, starvation
Event Processing
Problems with Event Processing• Doesn’t work well with
blocking APIs– open(), locking
• It doesn’t scale on SMP by itself
Process Model And ArchitectureATS NGiNX Squid Varnish Apache httpd
mod_proxy
Threads X X XEvents X X X partial XProcesses X X X
Caching Architecture
Cache• Mainly two types
– File system– Database like
• In memory index– Bytes per object
• Minimize disk seeks and system calls
CacheATS NGiNX Squid Varnish Apache httpd
mod_cache
File system X X Xmmap XRaw disk/direct IO X XRam cache X XMemory index X X X*Persistent cache X X X X
Performance Testing
ATS Configurationetc/trafficserver/remap.config:
map / http://origin.example.cometc/trafficserver/records.config:
CONFIG proxy.config.http.server_ports STRING 80CONFIG proxy.config.accept_threads INT 3
NGiNX Configurationworker_processes 24;access_log logs/access.log main;
proxy_cache_path /mnt/nginx_cache levels=1:2 keys_zone=my-cache:8m max_size=16384m inactive=600m;proxy_temp_path /mnt/nginx_temp;
server { set $ae ""; if ($http_accept_encoding ~* gzip) { set $ae "gzip"; }
location / { proxy_pass http://origin.example.com; proxy_cache my-cache; proxy_set_header If-None-Match ""; proxy_set_header If-Modified-Since ""; proxy_set_header Accept-Encoding $ae; proxy_cache_key $uri$is_args$args$ae; }
location ~ /purge_it(/.*) { proxy_cache_purge example.com $1$is_args$args$myae }
Squid Configurationhttp_access allow allhttp_port 80 accelworkers 24
cache_mem 4096 MBmemory_cache_shared oncache_dir rock /usr/local/squid/cache 1000 max-size=32768cache_peer origin.example.com parent 80 0 no-query originserver
Varnish Configurationbackend default { .host = ”origin.example.com”; .port = "80";}
Varnish Configuration (Cont)
sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=4000sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=200sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=1000 -p session_linger=0sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=1000 -p session_linger=10
Apache httpd ConfigurationLoadModule cache_module modules/mod_cache.soLoadModule cache_disk_module modules/mod_cache_disk.soLoadModule proxy_module modules/mod_proxy.soLoadModule proxy_http_module modules/mod_proxy_http.soInclude conf/extra/httpd-mpm.confProxyPass / http://origin.example.com/
<IfModule mod_cache_disk.c> CacheRoot /usr/local/apache2/cache CacheEnable disk / CacheDirLevels 5 CacheDirLength 3</IfModule>
MaxKeepAliveRequests 10000
Benchmark 1• 1,000 clients• 8KB response• 100% cache hit• Keep-alive on• 100K rps rate limited
• Squid used the most CPU and the worst median latency
• 95th percentile latency with NiGNX, Squid and httpd
ATS NGiNX Squid Varnish httpd0
500
1000
1500
2000
2500
RPS / CPU Usage
ATS NGiNX Squid Varnish httpd0
20000
40000
60000
80000
100000
120000
Requests Per Second
ATS NGiNX Squid Varnish httpd02468
1012141618
Latency
Median
95th
Benchmark 2• 1,000 clients• 8KB response• 100% cache hit• Keep-alive off
• Squid used the most CPU again
• NGiNX had latency issues• ATS most throughput
ATS NGiNX Squid Varnish httpd0
500
1000
1500
2000
2500
RPS / CPU Usage
ATS NGiNX Squid Varnish httpd0
5000
10000
15000
20000
25000
30000
Requests Per Second
ATS NGiNX Squid Varnish httpd0
5
10
15
20
25
30
35
40
Latency
Median
95th
ATS• Pros
– Scales well automatically, little config needed– Best cache implementation
• Cons– Too many config files– Too many options in the default config files
NGiNX• Pros
– Lots of plugins– FastCGI support
• Cons– HTTP/1.1 compliance– Latency issues around accepting new connections– Rebuild server for new plugins
Squid• Pros
– Best HTTP/1.1 compliance• Cons
– Memory index for cache using 10x that of ATS– Least efficient with CPU– Worst median latency for keep-alive benchmarks
Varnish• Pros
– VCL (Varnish Configuration Language)• Can do a lot without writing plugins
• Cons– Thread per connection– mmap for cache
• Persistence is experimental– No SSL or SPDY support
Apache httpd• Pros
– Lots of plugins– Most used http server– Best 95th percentile latency for non-keep-alive
• Cons– SPDY Support
Why ATS?• Scales well
– CPU Usage, auto config• Cache scales well
– Efficient memory index, minimizes seeks• Apache Community• Plugin support
– Easy to port existing plugins over
References• ATS - http://trafficserver.apache.org/• NGiNX - http://nginx.org/• Squid - http://www.squid-cache.org/• Varnish - https://www.varnish-cache.org/• Apache httpd - http://httpd.apache.org/