Post on 29-Dec-2015
Akamai vs. Akamai vs. Flash Crowds andFlash Crowds and
Distributed Denial of Service Distributed Denial of Service
Akamai Technologies Akamai Technologies
&&
Carnegie MellonCarnegie Mellon
Bruce Maggs
OutlineOutline
• AkamaiAkamai
• Content Delivery on 9/11Content Delivery on 9/11
• Impact of the “Slammer” WormImpact of the “Slammer” Worm
• FirstPointFirstPoint
• SiteShieldSiteShield
Akamai Services and ProductsAkamai Services and Products
http://http://www.google.comwww.google.com
http://http://www.yahoo.comwww.yahoo.com
http://http://windowsupdate.microsoft.comwindowsupdate.microsoft.com//
http://http://www.apple.com/quicktime/qtv/mwsf04/www.apple.com/quicktime/qtv/mwsf04/
http://http://www.CRITICAL.govwww.CRITICAL.gov
Akamai’s Platform for Delivering Akamai’s Platform for Delivering Content and ApplicationsContent and Applications
Akamai Serversat Network Edge
ContentProviders
EndUsers
NAP
NAP
Current Installations
Network Deployment Network Deployment
15000+Servers
1000+Networks
65+Countries
Content Delivery Using AkamaiContent Delivery Using Akamai
<html><html><head><head><title>Welcome to xyz.com!</title><title>Welcome to xyz.com!</title></head></head><body><body><img src=“<img src=“<img src=“ <img src=“ <h1>Welcome to our Web site!</h1><h1>Welcome to our Web site!</h1><a href=“page2.html”>Click here to enter</a><a href=“page2.html”>Click here to enter</a></body></body></html></html>
http://www.xyz.com/logos/logo.gifhttp://www.xyz.com/logos/logo.gif”>”>http://www.xyz.com/jpgs/navbar1.jpghttp://www.xyz.com/jpgs/navbar1.jpg”>”>
Embedded URLs are Converted to ARLsEmbedded URLs are Converted to ARLs
akak
End User
Akamai DNS ResolutionAkamai DNS Resolution
Akamai High-Level DNS Servers
10g.akamai.net
1
Browser’s Cache
OS
2
Local Name Server
3
xyz.com’s nameserver
66ak.xyz.comak.xyz.com
77a212.g.akamai.net
9915.15.125.6
16
15
1120.20.123.55
Akamai Low-Level DNS Servers
12 a212.g.akamai.net
30.30.123.5 13
14
4 xyz.comxyz.com .com .net Root
(InterNIC)10.10.123.555
akamai.net
88
select cluster
select servers within cluster
Content Delivery on 9/11Content Delivery on 9/11
• Akamai’s network had capacity for all Akamai’s network had capacity for all content providers requesting servicecontent providers requesting service
• Total bits served on September 11 Total bits served on September 11 was approximately 3.5 times normalwas approximately 3.5 times normal
• Traffic was higher on September 12Traffic was higher on September 12
• (But not as high as January 7, 2002)(But not as high as January 7, 2002)
News Site A – FreeFlow TrafficNews Site A – FreeFlow Traffic
News Site A – FreeFlow StreamingNews Site A – FreeFlow Streaming
News Site B – EdgeSuite TrafficNews Site B – EdgeSuite Traffic
News Site B – FreeFlow TrafficNews Site B – FreeFlow Traffic
News Site B – FreeFlow StreamingNews Site B – FreeFlow Streaming
Portal A – FreeFlow trafficPortal A – FreeFlow traffic
Sports Site A – FreeFlow trafficSports Site A – FreeFlow traffic
Steve Jobs KeynoteSteve Jobs Keynote
Impact of Sapphire/Slammer WormImpact of Sapphire/Slammer Worm
•Web site performance severely impacted
•Congestion in core of Internet
•Significant route flapping
Military Web Site - PerformanceMilitary Web Site - Performance
AFB response times on 01/25/2003
0
1000
2000
3000
4000
5000
0:00
1:45
3:15
4:45
8:15
9:45
11:0
012
:30
14:0
015
:45
17:1
519
:00
20:1
521
:45
23:1
5
Time (GMT)
mS
ec
Akamai Origin
71 content providers; 17 agents71 content providers; 17 agents
Download Failure Rates on 01/25/2003
02468
1012141618
0:45
3:15
5:15
6:45
8:15
9:45
11:0
012
:15
13:4
515
:00
16:1
517
:45
19:0
020
:15
22:1
523
:45
Time (GMT)
Fai
lure
Per
cen
tag
e
Akamai Failure Percentage Origin Failure Percentage
Military Web Site - ReliabilityMilitary Web Site - Reliability
Download Failure Rates (AFB)
0102030405060708090
100
00:0
002
:45
05:0
009
:15
11:4
514
:00
16:4
519
:15
21:1
5
Time (GMT)
Fai
lure
Per
cen
tag
e
VideoVideo
Aggregate Routing ActivityAggregate Routing Activity
11:30 PM EST Friday
Routing Activity by NetworkRouting Activity by Network
11:30 PM EST Friday
DOS attacksDOS attacks
•Coordinated attacks
•From multiple compromised machines
•On website or upstream
•Goal – to overwhelm
•Hacker-based e.g., – Microsoft, Yahoo!
•Voluntary sit-ins e.g., – World Economic Forum
MicrosoftMicrosoft
What is FirstPointWhat is FirstPoint
• Traffic management system for Traffic management system for mirrored websitesmirrored websites
• Directs browser to the optimal mirrorDirects browser to the optimal mirror
• DNS basedDNS based
• Application level anycastApplication level anycast
Why FirstPointWhy FirstPoint
• Content providers have mirrored Content providers have mirrored websiteswebsites
• Content providers only want to Content providers only want to offload embedded contentoffload embedded content
- ControlControl- SecuritySecurity- PerformancePerformance
Mapping ProblemMapping Problem
How to improve user experience?
What is the Mapping ProblemWhat is the Mapping Problem
• Problem of directing requests to Problem of directing requests to servers so as to optimize end-user servers so as to optimize end-user experienceexperience
- reduce latencyreduce latency- reduce lossreduce loss- reduce jitterreduce jitter
• Assumption - servers are fineAssumption - servers are fine
• Applicable to 2 mirrors or 1500 Applicable to 2 mirrors or 1500 Akamai locationsAkamai locations
AttemptAttempt
• Measure which is closerMeasure which is closer- Closeness changes over timeCloseness changes over time
• Measure frequentlyMeasure frequently- Bothers peopleBothers people- Too many to doToo many to do
~500,000 unique nameservers on any given day10 sec per measurement cycle
IdeaIdea
• Topology Topology - relatively staticrelatively static- changes in BGP timechanges in BGP time- order of hours if not daysorder of hours if not days
• CongestionCongestion- dynamicdynamic- changes in round-trip timechanges in round-trip time- order of millisecondsorder of milliseconds
Topology Discovery - Proxy pointsTopology Discovery - Proxy pointsData exchange
Topology DiscoveryTopology Discovery
500,000 nameservers 500,000 nameservers
reduced to reduced to
90,000 proxy points (clusters)90,000 proxy points (clusters)
Congestion MeasurementCongestion Measurement
Problem - Still too many measurements to do. Problem - Still too many measurements to do. 90,000 measurements every 10s with 32B 90,000 measurements every 10s with 32B packets requires a few Mbps per mirror.packets requires a few Mbps per mirror.
Solution - Importance based samplingSolution - Importance based sampling
CDF of End-user LoadCDF of End-user Load
CDF
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
Number of Clusters
Load
Load EstimationLoad Estimation
500,000 nameservers 500,000 nameservers
reduced toreduced to
90,000 clusters 90,000 clusters
7,000 account for 95% end-user load! 7,000 account for 95% end-user load!
Mapping Problem – Solved?Mapping Problem – Solved?
Maps built every 10s
FirstPointFirstPoint
• Customers - how to tell?Customers - how to tell?- look for CNAME to akadns.netlook for CNAME to akadns.net
• Customers - who?Customers - who?- High traffic content providersHigh traffic content providers- Yahoo!, Microsoft, TicketMaster etcYahoo!, Microsoft, TicketMaster etc
• Price - don’t ask :)Price - don’t ask :)
• Competitors - whoCompetitors - who- one-of-a-kind serviceone-of-a-kind service- boxes: Cisco, F5, Foundryboxes: Cisco, F5, Foundry
FirstPoint - other aspectsFirstPoint - other aspects
• Load-balancingLoad-balancing- estimate-basedestimate-based- feedback-based : https, snmp feedback-based : https, snmp - cost-based: 95/5cost-based: 95/5
• Fast cutout in case of failoverFast cutout in case of failover
• Highly fault-tolerantHighly fault-tolerant- hardware duplication, leader electionhardware duplication, leader election- overlay routing, BGP-based anycastoverlay routing, BGP-based anycast
• Integration with other servicesIntegration with other services- DOS/Load failoverDOS/Load failover
SiteShieldSiteShield
Content provider’swebsite
Hacker!
Hacker!
Hacker!
AKAMAI
AKAMAI
AKAMAI
SiteShieldSiteShield
•IP address of origin shielded
•Akamai can be attacked
•But Akamai will respond by •Diffusion – load balancing, &•Resurrection – reviving unpinned servers