Detecting Attacks in Routers Using Sketches

Dhiman Barman Piyush Satapathy Gianfranco Ciardo

Network Attack Detection

Network anomalies are prevalent Flash crowds, DoS, failures, worms, …

Detect Anamolies quickly and accurately Two basic approaches

Statistics-based: looking for abnormal behavior E.g., heavy hitters, big changes Prior knowledge not required

Signature based: Looking for known patterns Port scan, address scan, Mal wares

Problem Addressed Accurate, memory efficient and scalable

techniques to detect attacks worms, viruses, superspreaders and DoS.

Enabling change detection in the routers by looking only at the IP headers

A general methodology to use sketches to recognize attacks in the routing architectures

Example of Attack

Domain W to Domain J

Domain X to Domain J

Domain Y to Domain J

Domain Z to Domain K

Domain J

Domain K

RouterRouter

Router

Router

Router

Router A

Attacker

2

2

2

3

4

Link C

Link B

10 Mbps

15 Mbps

15 Mbps

35 MbpsLink Capacity: 50 Mb/s

1

1. Attacker Compromises Router A2. Update Message Sent Setting Link B’s cost to 10,0003. Traffic rerouted around link B (lower cost)

4. Congestion occurs on link C DoS on domains W, X, Y and Z

Outline

Introduction Attack Definitions Sketch Background Proposed Methods Simulation and Evaluation Conclusion

Background• Worm spreads and gains control of hundreds of computers in

a few minutes– Example: Code Red version 1 and 2, Nimda etc

State of the art:– PCA (offline) [Lakhina et. al SIGCOMM’04],– Data mining [KrishnaMurthy et. al IMC’03]– Time series analysis [Estan SIGCOMM’03]

• Our Focus: Detecting and filtering attacks at short time scale by online Methods

Architecture

• Put Sketch in the Router Line Card• Sketches will update the flow cache for each given time interval• Data Collection and Analysis part will analyze the change detection

based on the algorithm and distance calculation functions.

Data CollectionAnd

Analysis Over

ForwardingHard Ware

Software

NetFlowFlow Cache

Processor

TerminatedFlow Records

SketchesBus

UpdateAlarm / No Alarm

Attack Definitions

• S is a sequence of packets identified by (<i, j, p, q>, v) where

– i and j denote packet source IP address and port

– p and q denote the destination IP address and port

– v denotes size of the packet

• Flow given by Cisco's NetFlow, R(i, j, p, q) is the multi set containing all the packets corresponding to a given (i, j, p, q) combination

Port Scans

• Attacks where a particular IP address and port pair connects to a destination on several ports

PortScan(i, j, p) |{q : | R(i, j, p, q)| > 0}| > δPS

srcIP srcPort

destPort

destIP

Address Scans• Attacks where a particular IP address

connects to multiple destination IP addresses on a particularly vulnerable port

AddrScan(I, j, q) |{p : | R(i, j, p, q) > 0}| > δAS

or

AddrScan(i, q)| |{(j, p) : |R(i, j, p, q)| > 0}| > δAS

srcIP srcPort destPortdestIP

Malware

• Attacks where a number of sources try to connect to a particular destination or a set of destinations on any ports.WormMalwSpam (i) |{(j, p, q) : | R(i, j, p; q)| > 0}| > δWMS

srcIP srcPort destPort destIP

Sketch Background A sketch of a data stream is a compact

summarization Sketched summary is much smaller than the

data stream itself Sketching is randomized projection of a signal

(using hashing)• Examples:

• Count Min Sketch [Cormode 2003]• Count Bloom Filter [Broder Internet Mathematics

2004]• Multi Count Bloom filter [Broder IM 2004]• Flajolet Martin Sketch [Flajolet CS Journal1985]

Count Min Sketch CM sketch maintains an array of width (K) and

depth (H). Hashed by choosing a hash function uniformly at

random from a pair wise independent family of hash functions.

Update (Keyt, Valuet): Tj [ hj(keyt)] += Valuet (for all j)

1

j

H

0 1 K-1…

……

+Ct

+Ct

+Ct

hj(k)

hH(k)

h1(k)

Key t

Bloom Filter• An array of (m) bits initially all set to 0

• An incoming element is hashed through (k) hash functions and bit positions returned are set to 1

• Update (Keyt): Tj [ Hj(keyt)] = 1 (for all j)

• Update (Keyt, Valuet): Tj [ hj(keyt)] += Valuet (for all j) for Counting Bloom Filter

H1

H2

H3

Hk

1

1

1

1

1

m bit Vector

K hash Functions

Key t

Multi Count Bloom Filter

• Counting Bloom Filter with m counters and divided into k groups of size (m / k) each

• ith Hash function maps from [m*(i-1)/k]+1 to [m*i/k]

• Update (keyt, valuet): Tj [ hj(keyt)] += value t (for all j)

Key t

H1

H2

H3

Hk

m countersDivided intoK groups

K hash Functions

FM Sketch

• Estimates the number of distinct items in a stream of values from [0,…, M-1]

• Assume a hash function h(x) that maps incoming values x in [0,…, M-1] uniformly across [0,…, 2L-1], where L = O(logM)

• Let lsb(y) denote the position of the least-significant 1 bit in the binary representation of y– A value x is mapped to lsb(h(x))

• Maintain FM Sketch = BITMAP array of L bits, initialized to 0– For each incoming value x, set BITMAP[ lsb(h(x)) ] = 1

Data stream: 3 0 5 3 0 1 7 5 1 0 3 7

Number of distinct values: 5

x = 5 h(x) = 101100 lsb(h(x)) = 2 0 0 0 001

BITMAP5 4 3 2 1 0

Space-Time ComplexitySketches Space Update Time Query Time

Count-Min 1/ε ; O(K) 1 1

Bloom Filter m ; Constant k k

Count Bloom Filter

mC; O(m) k k

Multi Counting Bloom filter

mC; O(m) k k

FM ML O(M) M M

C = Number of Bits in the Counter in Bloom FilterM = Number of Bit Maps used in FM Sketch

L = Number of Bits in FM SketchAll Notations Described Earlier

Our proposals Linear Combination Approach

Change between two sketches S1(α1; β1) and S2(α2; β2) is given as Sd = S2 - kS1 (sketches are closed under linear combination)

If Sd > δ then there is an anomaly and Alarm raised Use Count Min sketch, Count and Multi Count Bloom Filter

Change Detection By Sliding Window Method Using two windows, one static and one moving Parallel Execution of all the given window lengths. Use FM, Change Detection Algorithm and Distance function

Change Detection using FM Sketch

A bit vector of length equal to kK different windows sizes

Forming 2 diff. windows

M

5 10 0.05

6 8 0.04

7 4 0.03

Set the bit to 1

Inputs: (A,M); A is a mX1 matrix and M is a KX3 matrix1: c0 02: for i = 1 do3: si FMi

4: Windowx,i mx,i intervals from time c0

5: Windowy,i my,i intervals from in coming data streams6: end for7: while more flow counts to process do8: slide windowy,i by 1 sample9: if distance (windowx,i,windowy,i) ≥ αi then10: c0 current time11: Output change at time c0

12: clear all windows and GOTO step 113: end if14: end whileOutput: c0

Distance Function

• Difference between two windows calculated using:– Probability Distribution [Kifer VLDB 2004]

– L1 Distance

– KL Distance (Empirical Distribution Function)

Sketches in Action

Traces ofgivenTime slots

SketchingChange detectionmodule

(k,u) … Alarms

Pre Processing

sketches

W, d, m

K

A, M

δ

HeaderPayload

Time Key Count

LBNL &NLANRTraces

CAIDA’sCoral Software

Sketch Library

Functions

Experimental Issues

Experiments Run on Intel Xeon 1.4Ghz processor with 512 MB of RAM

Implementation of Sketches Hash functions are created randomly Random numbers are generated with the max number of 261-1

Following Hash Functions for all the 4 sketches Update (key,value) getFrequency (key)

Synthetics traces generated where attack flows are injected

Evaluation (Accuracy)

Sketches of 10 heavy ranked flowsSketching changes in heavy ranked flows between two time intervals

Evaluation (Accuracy)

FM sketch of 10 Intervals of normal traffic

FM sketch of 10 Intervals of malicious traffic trace

Scatter Plot (Accuracy)

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 100000

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

Exact Flow Size (Bytes)

Est

. Cou

ntB

loom

Flo

wS

ize(

Byt

es)

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 100000

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

Exact Flow Size (Bytes)

Est

. M

ultC

ou

ntB

loo

m S

ize

(Byt

es)

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 100000

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

Exact Flow Size (Bytes)

Est

Co

unt

Min

Flo

w S

ize

(B

yte

s)

Scatter plot of Exact Vs Est. CountMin

Count Min Sketch in Attack Detection

0 1 2 3 4 5 60

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Number of Hash Functions (K)

Fre

qu

en

cy E

rro

r (%

)

Port Scan Attack Detection By CountMin Sketch

0 1 2 3 4 5 6

0.05

0.1

0.15

0.2

0.25

0.3

Number of Hash Functions (K)

Fre

quen

cy E

rror

(%

)

Address Scan Attack Detection By CountMin Sketch

0 1 2 3 4 5 60

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Number of Hash Functions (K)

Fre

qu

en

cy E

rro

r (%

)

Malware Attack Detection By CountMin Sketch

Attacks Vs Sketches(Efficiency)

1 1.5 2 2.5 3 3.5 4 4.5 50

0.05

0.1

Number of Hash Functions (K)

Fre

qu

en

cy E

rro

r (%

)

Port Scan Attack Detection By Different Sketches

CountMin

CountingBloom

MultiCountingBloom

1 1.5 2 2.5 3 3.5 4 4.5 50

0.02

0.04

0.06

0.08

0.1

0.12

0.14

Number of Hash Functions (K)

Fre

quen

cy E

rror

(%

)

Address Scan Attack Detection By Different Sketches

CountMinCountingBloomMultiCountingBloom

1 1.5 2 2.5 3 3.5 4 4.5 50

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Number of Hash Functions (K)

Fre

quency E

rror

(%)

Malware Detection By Different Sketches

CountMinCountingBloomMultiCountingBloom

Evaluation by FM Sketch (Accuracy)

Change Detection Accuracy by FM sketch using probability distribution

distance function

Comparison of two different distance functions on exact

flow counts

Evaluation by FM Sketch (Efficiency)

Accuracy of two distance functions on synthetic traces. The x-axis represents values such as threshold is mean(X)+ std(X) where X is the flows counts.

Conclusions

• Sketches can detect heavy changes quickly and accurately

• Compact sketch-like data structures can be implemented on-chip hardware

• Reverse hashing of sketches to identify malicious flows is challenging

Thank you!