Topology Inferencefrom BGP Routing Dynamics

David Andersen, Nick Feamster, Steve Bauer, Hari Balakrishnan

MIT Laboratory for Computer Science

October 2002

http://nms.lcs.mit.edu/ron/

Current Topologies: AS Topologies

MIT

Sprint

AT&T

BBN

UUNET

✔ Simple to construct

✔ Completely passive - BGP snapshot

✘ Obnoxiously free of interesting detail

A few paths contain most prefixes

AT&T (7018): 1250UUNET (701 702): 1250

Source (AS) #prefixes

Supernet (3908): 793

(hong Kong)REACH (1221): 1282UUNET (701): 2053

14000

0.80.9

1

0 2000 4000 6000 8000 10000

Number of origin AS’s

Cumulative distribution

0.60.50.40.30.20.1

0

Fra

ctio

n of

ann

ounc

ed p

refix

es

0.7

� 13 common paths contain 10% of prefixes

� Binning large ISPs misses critical detail

Current Topologies: Router-Level

ATT−1

mit1 mit2 BBN1

BBN2

BBN3

BBN4

UU−1

✔ Lots of juicy detail

✘ Requires active probing

- Annoys the paranoid (and can be blocked)

- Consumes time and bandwidth

➔ Best of both worlds?

New: Implied Logical Topologies

Net 2

Net 1

Net 4

Net 3

� Group prefixes that “behave similarly”

� What do the resulting clusters mean?

BGP update streams2002-01-10 23:51:05 198.140.178.0/24

2002-01-10 23:51:05 192.107.237.0/24

2002-01-10 23:55:53 199.230.128.0/23

2002-01-10 23:56:21 216.9.174.0/23

2002-01-10 23:56:21 216.9.172.0/24

� Colored prefixes updated at (nearly) same time

➔ Cluster prefixes that often do this

Mechanics2002-01-10 23:51:05 198.140.178.0/24

2002-01-10 23:51:05 192.107.237.0/24

2002-01-10 23:55:53 199.230.128.0/23

2002-01-10 23:56:21 216.9.174.0/23

2002-01-10 23:56:21 216.9.172.0/24

� Group by 30-second intervals(in practice, bin length choice flexible) (BGPmin-route-adver time)

Creating BGP update vectors

time

p1 updates

p2 updates

u

u

I seconds

1

1

1

1

10

0 0

(t)p1

(t)p2

� Update stream is a 0/1 signal

Did an update happen in time [t; t+ 30s]?

� Now we have a bunch of 0/1 vectors to

compare...

BGP update vectors

time �!

Prefix A 0 0 1 0 1 0 0

Prefix B 1 0 1 0 0 0 1

Prefix C 1 0 1 0 0 0 0

How close are two vectors?

� Correlation coefficient

Correlation CoefficientA 0 0 1 0 1 0 0

B 1 0 1 0 0 0 1

C 1 0 1 0 0 0 0

corr(p1; p2) =

E[(p1 � p1)(p2 � p2)]

�p1�p2

� Expresses correlation well

� Susceptable to some “coincidental” correlation

How to Group Prefixes?

E−A: 0.001

A B C D E

Resulting ClusterInput DistancesA−B: 1

...

A−C: 0.75B−C: 0.5D−E: 0.25

Single-linkage clustering

� Simple and efficient

� Creates a similarty hierarchy: A & B mostsimilar, etc.

How to Group Prefixes?

E−A: 0.001

A B C D E

Resulting ClusterInput DistancesA−B: 1

...

A−C: 0.75B−C: 0.5D−E: 0.25

Single-linkage clustering

� Simple and efficient

� Creates a similarty hierarchy: A & B mostsimilar, etc.

How to Group Prefixes?

E−A: 0.001

A B C D E

Resulting ClusterInput DistancesA−B: 1

...

A−C: 0.75B−C: 0.5D−E: 0.25

Single-linkage clustering

� Simple and efficient

� Creates a similarty hierarchy: A & B mostsimilar, etc.

How to Group Prefixes?

E−A: 0.001

A B C D E

Resulting ClusterInput DistancesA−B: 1

...

A−C: 0.75B−C: 0.5D−E: 0.25

Single-linkage clustering

� Simple and efficient

� Creates a similarty hierarchy: A & B mostsimilar, etc.

Data Capture and AnalysisBBN

AS 10578Collection Host

Border Router

AS 3 (MIT)

� Studied 90 days of BGP traffic at MIT

� Examined 2 “huge” origin ASes

– UUNET: 2338 prefixes

– AT&T: 1310 prefixes

� How do clusters relate to real-word features?

Anecdotes

� Many “expected” results - same city, etc.

We’ ll get to those in a second.

� 135.36.0.0/16, 135.12.0.0/14. Denver vs. New

Jersey. Lucent vs. Agere – a spinoff in 2000,

identical network behavior. (... CIA?)

� 6 Sandia labs prefixes - internet2 routes, but

flapped to backup UUNET route.

� Many transient discoveries: backups, etc.

Topological similaritiesMeasureable quantities: path, location

� Compute pairwise similarity for metric (sharedpath length, or shared pop)

� Average similarity as clustering proceeds

� If match with logical clustering,similarity strongest for leaf clustering,weakest at end.

➔ Logical topology: integration of topological,organizational, and administrative factors.

Leaves share more hops in traceroute

8

10

12

14

16

18

20

22

0500100015002000

Num

ber

of tr

acer

oute

hop

s

Number of clusters

UUNET max hopsUUNET shared hops

� Path length varies less with clustering

� More shared hops in earlier clustering

� Data noisy: loops, etc., but still works

Leaves often share the ISP POP

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0500100015002000

Avg

. fra

ctio

n of

sam

e-P

OP

clu

ster

ing

Number of clusters

UUNETAT&T

� UUNET: 50% clustered at 95% accuracy

� AT&T: 30% clustered at 97% accuracy

What does it all mean?

� Update clusters reflect reality:

– Topology

– Prefix assignment

– Fate sharing

� Passive window into remote networks

� Facilitate network mapping and data collection

� What else can be extracted from this signal?

Similar signals?

Weighted sumA 0 0 1 0 1 0 0

B 1 0 1 0 0 0 1

C 1 0 1 0 0 0 0

score(p1; p2) =

Xi2sets

(1

size(i) if p1; p2 2 i

0 otherwise

AB: 1

AC: 1

BC: 2

Cluster Size Evolution

0

500

1000

1500

2000

2500

0 10000 20000 30000 40000 50000 60000

Num

ber

of c

lust

ers

Number of pairwise comparisons

UUNET - correlationUUNET - weighted sum

AT&T - correlationAT&T - weighted sum

� Formation speed drops off rapidly

Clustered prefixes are “near” each otherNumeric distance between two prefixes

2^42^62^8

2^102^122^142^162^182^202^222^242^262^28

0500100015002000

Num

eric

dis

tanc

e fo

r cl

uste

red

addr

esse

s

Number of clusters

UUNETAT&T

� Many early clusters separated by /16s