Flash Crowds & Denial of Service Attacks

Characterization and Implications for CDNs and Web sites

Jaeyeon JungMIT Laboratory for Computer Science

Balachander Krishnamurthy and Michael RabinovichAT&T Labs-Research

Motivation✔ Flash crowd is a sudden, large surge in traffic to a

particular Web site

– September 11, Ken Starr’s report, Victoria’s

Secret webcast

✔ Denial of Service (DoS) attack is an explicit attempt

to prevent legitimate users of a service from using

that service

– HTTP request flooding, attack to crack

password-protected web pages, ���� ���

worm, TCP SYN flooding, etc.

Slide 1

Questions✔ Part I: Flash events vs. DoS attacks

– What properties differentiate DoS attacks from

flash events?

– How can we use them to identify and separate

DoS attacks from flash events?

✔ Part II: Flash crowds and CDNs

– What is the locality of file reference like during

flash events and its implication for CDNs?

– How can we improve protection of Web servers

from flash crowds using CDNs?

Slide 2

Network-Aware Clusters [KW00]

✔ Clustering uses a large collection of unique

network prefix from BGP tables

✔ Classify all the IP addresses that have the same

longest matched prefix into a cluster

✔ It helps determine topological distribution of

clients in FE and DoS

Slide 3

Flash Events

0

1000

2000

3000

4000

5000

6000

7000

0 20000 40000 60000 80000

Num

ber

of r

eque

sts

per

seco

nd (

rps)

Time (second)

Play-along

0

100

200

300

400

500

600

700

0 20000 40000 60000 80000 100000 120000

Num

ber

of r

eque

sts

per

seco

nd (

rps)

Time (second)

Chile

Trace Requests Documents Clients Clusters

Play-along Total 13,018,385 7,084 53,745 14,100

FE 71.0% 68.9% 63.9% 61.6%

Chile Total 2,634,567 10,302 20,532 1,739

FE 88.2% 90.2% 89.0% 86.6%

Slide 4

DoS Attacks

0

20

40

60

80

100

120

140

5000 6000 7000 8000 9000 10000 11000 12000

Req

uest

rat

e

Time

Trace Requests Documents Clients Clusters

bit.nl 35,657 1 11,092 6,155

✔ ���� ��� : In the earlier variant, each instance

uses the same random number generator seed to

create the list of IP addresses it scans [CERT].

Slide 5

Part I: Flash Events vs. DoS Attacks

Slide 6

Client Characteristics

0

200

400

600

800

1000

1200

1400

1600

0 20000 40000 60000 80000

Num

ber

of d

istin

ct c

lient

s or

clu

ster

s pe

r se

cond

Time (second)

Play-along

clientsclusters

0

5

10

15

20

25

30

35

40

45

5000 6000 7000 8000 9000 10000 11000 12000

Num

ber

of d

istin

ct c

lient

s or

clu

ster

sTime

bit.nl

clientsclusters

✔ [FE] Clients can be effectively aggregated into

clusters

✔ [DoS] Distribution of DoS attackers is broad

Slide 7

Client Characteristics - contd.

✔ [FE] Many old clusters are represented in flash

events

Play-along : 42.7% and Chile: 82.9%

✔ [DoS] Very few previously seen clusters are

involved in DoS attacks

creighton: 0.6%, fullnote: 0%, spccctxus: 1.8%,

and rellim: 14.3%

Slide 8

Per-client Request Rate

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 20000 40000 60000 80000

Ave

rage

per

-clie

nt r

eque

st r

ate

Time (second)

Play-along

0

1

2

3

4

5

6

5000 6000 7000 8000 9000 10000 11000 12000

Ave

rage

per

-clie

nt r

eque

st r

ate

Time

bit.nl

✔ [FE] There is a decline in per-client request rate

during the flash event

✔ [DoS] The per-client request rate does not change

during the surge in requests

Slide 9

Server Strategy✔ Monitor the clients that access the site and their

request rate

✔ Periodically perform network aware clustering over

the client set accumulated over the past period

without flash or DoS events - old clusters

✔ When performance degrades to a threshold level,

discard packets that come from clients that do not

belong to old clusters as well as from non-proxy

clients whose request rate deviates significantly

from average

Slide 10

Part II: Flash Crowds and CDNs

Slide 11

File Reference Characteristics✔ Large number of documents are accessed only

during FEs (Play-along : 61% and Chile: 82%)

➔ Many cache misses at the beginning of FEs

✔ 10% of popular documents account for more than90% of requests.

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Fra

ctio

n of

req

uest

s

Fraction of documents

Play-alongChile

Slide 12

Flash Events and CDN

CDN node CDN node

CDN nodeCDN node

Client

Client

Client

Client

Client Client

Client

Client

Client

Client

Client

Client

OriginServer

✔ CDN with 1,000+ cache nodes might not be able to

provide an absolute protection against FE due to the

peaks in the beginning of FE

Slide 13

Flash Events and CDN

CDN node CDN node

CDN nodeCDN node

Client

Client

Client

Client

Client Client

Client

Client

Client

Client

Client

Client

OriginServer

✔ Limiting the number of caches would increase the

load on individual caches

Slide 14

Flash Events and CDN

CDN node CDN node

Client

Client

Client

CDN nodeClient

Client Client

Client

Client Client Client

Client

Client

OriginServer

✔ Limiting the number of caches would increase the

load on individual caches

Slide 14

Flash Events and CDN

CDN node

Client

CDN nodeClient

Client Client

Client

Client

Client

Client

Client

Client Client

Client

OriginServer

✔ Limiting the number of caches would increase the

load on individual caches

Slide 14

Adaptive CDN

✔ Lower the peak rate forwarded to an origin server

while spreading load over cache nodes

Delegate

Delegate

Primary

Primary Primary

Primary

Client Client

DNS

1

2

3

OriginServer

Slide 15

Adaptive CDN

✔ Lower the peak rate forwarded to the origin server

while spreading load over cache nodes

Delegate

Delegate

Primary

Primary Primary

Primary

Client Client

DNS

12

3

OriginServer

Slide 15

Adaptive CDN

✔ Lower the peak rate forwarded to the origin server

while spreading load over cache nodes

Delegate

Delegate

Client Client

Primary

Primary Primary

Primary

Client

DNS1

2

OriginServer

Slide 15

Dynamic Delegation

lhl <

lll <

ll lh< l <

lh, l(d) >

OverloadedUnderloaded

; release delegates ; redirect request to; add a new one if

; process requests

l > lh

of the lowest loadd

; HTTP request redirect to; add delegate d

d

d

✔ Load on primary cache, �, is computed as requests

per second averaged over two-second interval.

✔ Cache is assigned based on cluster and used for ttl

period.

Slide 16

Simulation Results

0

1000

2000

3000

4000

5000

6000

7000

0 20000 40000 60000 80000

Num

ber

of r

eque

sts

per

seco

nd (

rps)

Time (second)

Play-along

0

20

40

60

80

100

120

0 20000 40000 60000 80000

Num

ber

of r

eque

sts

per

seco

nd

Time (second)

Play-along

✔ Peak request rates are reduced by a factor of 50

(Play-along), and 20 (Chile)

✔ It ensures that load on each cache remains low (50

rps) and that proximity-based cache selection is not

compromised.

Slide 17

Conclusion

✔ Client clustering technique is useful for source

identification and for distinguishing legitimate

requests and malicious attacks.

✔ Per-client request rate drops and remains lower

during the FEs unlike DoS attackers who generate

requests independently of a server load

✔ Adaptive CDN is effective in terms of reduction of

flows from the main server and dynamic load

distribution over cache nodes.

Slide 18