Slide -1-February, 2006

Interdomain Routing

Gordon WilfongDistinguished Member of Technical Staff

Algorithms Research DepartmentMathematical and Algorithmic Sciences Center

Bell Laboratories

February, 2006

Slide -2-February, 2006

Current Internet Routing

Autonomous Systems (ASes)– Subnetwork under single administrative control

• Each AS has its own economic incentives to:

– cooperate so as to achieve connectivity and to abide by economic contracts (SLAs)

– to hide information about its internal network

Routing between ASes achieved by protocol where– AS selectively announces to neighbors its chosen route to

destinations

– AS selects highest ranked route announced to it

Slide -3-February, 2006

Border Gateway Protocol (BGP)

“I routedirectly to d”

AS1 AS2

“The route I use to d goes through AS0”

AS0

d

“The route I use to d goes throughAS1 then AS0”

AS3

Data packets

Slide -4-February, 2006

Routing Protocol Requirements

Protocol must scale

– Routing table entries (destinations): ~150,000 and increasing

– Autonomous Systems: ~20,000 and increasing

Administrators require expressiveness for routing policies

– Local AS policy (based on SLAs, traffic engineering etc) determines

• route selection (effects where AS sends packets)

• route announcements (effects how/if packets from other ASes traverse it)

Slide -5-February, 2006

External BGP (E-BGP) is the mode of BGP that propagates routes between autonomous systems.

E-BGP

Slide -6-February, 2006

Stable Paths Problem

1

3

2

3 2 03 1 03 2 1 0

1 01 2 0

0

2 1 02 0

1 0210

310

0

Slide -7-February, 2006

1 2 01 0

Independent policies can lead to nonconvergence

0

1

23 2 3 02 0

3 1 03 0

Bad Gadget

Each AS chooses“favorite” availableroute.

0

Slide -8-February, 2006

May Be Multiple Solutions

First solution

1

0

2

1 2 01 0

1

0

2

1

0

2

2 1 02 0

1 2 01 0

2 1 02 0

1 2 01 0

2 1 02 0

Second solutionDISAGREE

Slide -9-February, 2006

What can be done?

Modify protocol

– Design a protocol that has provably good properties. e.g., [Griffin, W.]

Constrain topology and/or policy

– Prove good protocol behavior assuming constrained configurations. e.g., [Gao, Rexford]

• These constraints give rise to interesting “valley-free” routing problems (from customer to local provider to regional provider to national provider and then down chain again to another customer). e.g. [Erlebach et al]

Slide -10-February, 2006

Possible Research Directions

Complexity issues

– What is the complexity of deciding if a configuration is “safe” (i.e., will it always converge)?

• NP-complete to determine if there is a solution it can converge to [Griffin, W.]

Extending model

– Model assumes destinations can be considered independently

• Aggregation has not been taken into consideration

Trade-offs between protocol expressivity and safety

– Formal model to study such trade-offs

Slide -11-February, 2006

Internal BGP (I-BGP) is the protocol used to propagate external routes within an autonomous system.

I-BGP

Slide -12-February, 2006

Fully Meshed

RR

RR

RR

RR

RR

RR

A router only announcesits own routes

Slide -13-February, 2006

Route Reflectors

RR

RR

RR

RR

RR

RR

Route Reflectors must be fullymeshed

Route Reflectors pass along updatesto client routers

Slide -14-February, 2006

Route Selection Summary

Highest Local Preference

Shortest ASPATH

Lowest MED (if same next AS)

I-BGP < E-BGP

Lowest IGP cost to BGP egress

Lowest router ID

Traffic engineering

Enforce relationships

Throw up hands andbreak ties

Slide -15-February, 2006

What Are MEDs?

MED=2MED=0

AS1

AS2

CP

Client

“Hot Potato Routing” vs.“Cold Potato Routing”

MED=1

AS3

Slide -16-February, 2006

Persistent Route Oscillations

135.104.54.4

5040

120

1

MED=1

MED=2MED=1

r1

r2

r3

A B

Slide -17-February, 2006

Oscillations in I-BGP

135.104.54.4

1

115 5 5

RR1 RR2 RR3

C1 C2 C3

P1 P2 P3I-BGP connection

Physical link

AS0

Slide -18-February, 2006

Deflections

135.104.54.4

3

1

1

P

Q

RR

A B

Packets can be diverted out of a network unexpectedly.

AS0

Slide -19-February, 2006

Routing Loops

135.104.54.4

5

11

1

RR1RR2

C1 C2

Badly configurednetworks can alsoexperience routingloops

AS0

Slide -20-February, 2006

A Real-world MED Oscillation Example

• A functioning network breaks into a state of persistent route oscillations when a BGP session goes down• First thought to be a hardware problem• Analysis shows that route oscillations caused by the use of the MED attribute• The example was actually predicted by the theoretical analysis [Griffin, W.]

Slide -21-February, 2006

Initial State

AS 4

A B

E F

AS 3

AS 1

AS 2

reflector reflector

I-BGP

E-BGP

(1)(2)

C

MEDs

IGP12

D

21

(2)

70000

Only AS 2 sends MEDs to AS 4

Slide -22-February, 2006

Initial Routing

AS 4

A B

E F

AS 3

AS 1

AS 2

reflector reflector

I-BGP

E-BGP

(1)

C

IGP12

D

21

(2)MEDs (2)

70000

D prefers AS 2path due to router IDtie breaking

Slide -23-February, 2006

B Changes Its Route

AS 4

A B

E F

AS 3

AS 1

AS 2

reflector reflector

I-BGP

E-BGP

(1)

C

IGP12

D

21

sessi

on down!

The AS 4 AS 3 BGP Session is dropped

(2)MEDs (2)

70000

Slide -24-February, 2006

A Changes Its Route

AS 4

A B

E F

AS 3

AS 1

AS 2

reflector reflector

I-BGP

E-BGP

(1)

C

IGP12

D

21

sessi

on down!

(2)MEDs (2)

The MED 1 route from B beats the MED 2 routesthat A sees from its clients….

70000

CALL THISSTATE ZERO

Slide -25-February, 2006

C & D Change Routes

AS 4

A B

E F

AS 3

AS 1

AS 2

reflector reflector

I-BGP

E-BGP

(1)

C

IGP12

D

21

sessi

on down!

(2)MEDs (2)

The MED 1 route from A knocks both MED 2 routesout of the picture for C & D …

70000

Slide -26-February, 2006

A Changes Route Again

AS 4

A B

E F

AS 3

AS 1

AS 2

reflector reflector

I-BGP

E-BGP

(1)

C

IGP12

D

21

sessi

on down!

(2)MEDs (2)

A now sees the route from D through AS 3, and it is closer IGP-wise than the route from B…

70000

Slide -27-February, 2006

C&D Return to Initial Routes

AS 4

A B

E F

AS 3

AS 1

AS 2

reflector reflector

I-BGP

E-BGP

(1)

C

IGP12

D

21

sessi

on down!

(2)MEDs (2)

C & D no longer see MED 1 route from A, so they return to the eBGP routes with MED 2…

70000

Slide -28-February, 2006

Back to State Zero!

AS 4

A B

E F

AS 3

AS 1

AS 2

reflector reflector

I-BGP

E-BGP

(1)

C

IGP12

D

21

sessi

on down!

(2)MEDs (2)

A switches back to MED 1 route through B.

70000

Slide -29-February, 2006

What Can Be Done?

Possible Approaches:

Use only configurations that guarantee no problems

No modification to BGP required

Previous example shows this might be difficult

Prevent problems for any configuration

Modification to BGP required, e.g. [Basu et al]

Minimize overhead (number of messages, memory requirements, …)

Slide -30-February, 2006

• BGP is extremely flexible, allowing operators to easily make obscure errors that are difficult to find and correct.

• Policy based routing is difficult to get right.

• Defining a formal model of path vector protocols helps to understand what features can cause problems and what can be done to avoid errors.

Conclusions

Slide -31-February, 2006

• BGP is the routing protocol in today’s internet

• What about the future?• new Internet architectures• new architectures within an AS (e.g., Routing Control Platform [J. Rexford, et al])

• In any case, a formal theoretical approach is essential to getting it right.

What about the future?

Slide -32-February, 2006

Extras

Slide -33-February, 2006

Persistent Route Oscillations

135.104.54.4

5040

120

1

MED=1

MED=2MED=1

r1

r2

r3

A B

r2r1r3

r1r3

r1r2r3

r3r2

r1r2r3

r1r3

r2r13

r1r3

r2r1rr3

r3r2