Building high traffic http front-ends. theo schlossnagle. зал 1

/

High-performanceRobustHTTPFront-ends

tips, tricks and expectations

Saturday, April 23, 2011

Who am I? @postwait on twitter

Author of “Scalable Internet Architectures”Pearson, ISBN: 067232699X

Contributor to “Web Operations”O’Reilly, ISBN:

Founder of OmniTI, Message Systems, Fontdeck, & CirconusI like to tackle problems that are “always on” and “always growing.”

I am an EngineerA practitioner of academic computing.IEEE member and Senior ACM member.On the Editorial Board of ACM’s Queue magazine.

2


Agenda

• Why only HTTP?

• HTTP-like protocols

• Performance

• Availability


HTTP

• Why only HTTP... it’s what we do.

• User-based, immediate, short-lived transactions occupy my life.

• So, not just HTTP.

• HTTPS

• SPDY (... we’ll get to this)


Performance

• ATS (Apache Traffic Server)

• supports SSL

• battle-hardened codebase

• very multi-code capable

• Varnish

• VCL adds unparalleled flexibility

• no SSL!

• nginx

• I don’t see much of this out on the edge


Performance Expectations

• from a single server, you should be able to:

• support 500k concurrent users

• this is only 40k sockets/core

• push in excess of 100k requests/second

• this is only 9k requests/core*second

• push close to 10 gigabits

• this is why 10G was invented


Performance Achievements

• Good load balancers achieve this performance

• with dual socket Westmere processors,we’re able to achieve insoftware ongeneral purpose hardwarewhat was only possible in hardware ASICs.

• ATS and Varnish can do this today.


The Basic Rules: Content

• You must serve content from cache

• Your cache should fit in memory

• If it does not, it should spill to SSD,not spinning media.


The Basic Rules: CPU

• You must cache SSL sessions

• SSL key negotiation is expensive.

• SSL encryption is not*

• Common cases must not cause state on the firewall.

• It’s hard enough to serve 150k requests/second.

• You will spend too much time in kernel in iptables, ipf, or pf.

• allow port 80 and port 443.

• enable SYN flood prevention

* crypto obviously costs CPU; symmetric crypto is relatively cheap


The Basic Rules: Network

• You must not run a stateful firewall in front

• too expensive

• too little value

• You must be directly behind capable router(s)

• expect anywhere from1MM to 20MM packets per second

• we need to run BGP for availability


Availability

• We learned in the performance section:

• 1 machine / 10Gbps uplink performs well enough

• We need redundancy:

• Linux HA?

• VRRP/HSRP?

• CARP?

• No...


Availability: Constraints

• Client TCP sessions are relatively short lived.

• The web is a largely idempotent place.

• Clients are capable of retrying on failure.

• This means:

• forget stateful failover.

• focus on availability for new connections.


Availability: Setup

• You are behind a capable router (it was a rule)

• Use routing protocols (BGP) to maintain availability.

10.1.0.0/2310.1.0.0/23

10.1.1.0/2410.1.0.0/24

BGP


Working Stacks

• Linux (OS/TCP stack)

• Varnish (HTTP)

• Quagga (BGP)

• Illumos (OS/TCP stack)

• ATS (HTTP/HTTPS)

• Quagga (BGP)


Future!

• This stuff is fast.

• In the end, we’re not looking for faster servers,we’re looking for improved user experience.

• Enter SPDY

• Google’s multi-channel HTTP super-protocol

• Allows multiplexing of concurrent HTTP(like) request/response on a single TCP session.

• Defeats slow startup

• Allows for content prioritization on server


Future: my thoughts

• SPDY is relatively simple to implement on the server

• SPDY is very very hard to leverage on the server

• If ATS implemented SPDY in and out

• and provided a robust configuration languageto leverage it

... the future would be today.


Thank you.

• Thank you Олег Бунин

• Thanks to the Varnish and ATS developers.

• Спасибо.


Building high traffic http front-ends. theo schlossnagle. зал 1

Technology

Transcript of Building high traffic http front-ends. theo schlossnagle. зал 1