Yaping Zhu Advisor: Prof. Jennifer Rexford With: Andy Bavier and Nick Feamster (Georgia Tech) UFO: A...

Yaping ZhuAdvisor: Prof. Jennifer Rexford

With: Andy Bavier and Nick Feamster (Georgia Tech)

UFO: A Resilient Layered Routing Architecture

2

Scalability + High Availability ?

Scalability: Scalability of routing control planeEfficiency of routing data plane

High Availability: Quick adaptation and re-route

3

Can We Have the Best of Both Worlds?

Scalability Availability

Internet Routing Yes No

Overlay Routing No Yes

UFO Routing Yes Yes

Basic Idea: 1. Layered routing architecture (borrowing idea from overlay routing)2. Underlay Support for efficient and scalable overlay routing

4

Outline

• Background– Internet routing architecture– Overlay routing (Resilient Overlay Networks)– Basic idea of Layered routing architecture

• Efficient overlay forwarding

• Scalable overlay monitoring

• Enhancing the scalability of UFO

• Implementation and Evaluation

• Conclusion and deployment

5

Internet Routing designed for Scalability

Autonomous System (AS)

AS

AS

AS

AS

AS

AS

AS

AS

AS

AS

AS

AS

AS

ASAS

AS

AS

ASASASAS

AS

Peering

Transit

6

Internet Routing without High Availability

• Scalability– Statistics: 25K ASes, 200K prefixes, millions of routers– Hierarchical: intra-domain / inter-domain routing– Prefix aggregation

• Routing protocols oblivious to performance– Intra-domain: static link weights– Inter-domain: routing policies

• Slow outage detection and recovery – Disruptions during convergence– Performance suffers from black-holes and loops

7

Scalable Internet Routing without Customization

• IP does destination-based forwarding– All traffic follows the same paths– Independent of the application requirements

• Yet, applications have different needs– Voice and gaming: low latency and loss– File sharing: high throughput

High throughput,but high latency

low latency,but low throughput

8

Outline







9

RON: Resilient Overlay Networks (by D. Andersen)

ScalableIP routing substrate

10

RON: Resilient Overlay Networks System Components

• Overlay Control Plane– Probing, overlay path evaluation– Disseminate routing messages, update routes

• Overlay Data Plane– Tunnel setup: packet

encapsulation/decapsulation

• User Opt-in Method– DNS redirection to overlay server– Connection to overlay server: tunnels (e.g

VPN)

11

Overlay Routing

• Pros:– High availability: End hosts discover network-level path

failure and cooperate to re-route.– Customization: Forwarding paths tailored to the

application

• Applications:– Content distribution (e.g. Akamai SureRoute)– Application layer multicast

12

Overlay Routing: Poor Efficiency

• Problem: traffic must traverse bottleneck link both inbound and outbound– Additional latency overhead– Additional traffic consumption

Upstream ISP

13

Overlay Routing: Poor Scalability

ScalableIP routing substrate

I don’t know when

failure happens

Let’s just keep

probing

Shall I re-route if one

packet lost?

14

Overlay Routing: Poor Scalability

• Fundamental trade-off between probing freq and adaptation– To get Quick adaptation

-> aggressive probing at short time interval-> poor scalability:->RON only supports for a small (i.e.,< 50 nodes) set of connected hosts

• Can not differentiate packet lost due to different events– Failure -> fast re-route– Congestions -> may slower? -> oscillation?

15

Outline







16

Can We Have the Best of Both Worlds?

Scalability Availability Customization

Internet

Routing

Yes No No

Overlay

Routing

No Yes Yes

UFO

Routing

Yes Yes Yes

17

A Resilient Layered Routing Architecture

• Combination of underlay and overlay routing

18

UFO: Underlay Friendly to Overlays

Underlay Friendly to Overlays

• In-network support for overlays

19

A Resilient Layered Routing Architecture

• Questions:– Which functionality belong to which layer?– What are the interfaces between both layers?

• Cross-layer design– Efficiency improvement:

• Direct control over forwarding table entries

– Scalability improvement:• Explicit notification about changing network

conditions

20

Outline

• Efficient overlay forwarding– Overlay forwarding on line cards– Hosting the overlay control plane

• Scalable overlay monitoring– Registration of overlay links– Notification of network events– Lazy recovery

• Enhancing the scalability of UFO• Implementation and Evaluation • Conclusion and deployment

21

Outline




22

Efficient Overlay Forwarding

• Problem: traffic must traverse bottleneck link both inbound and outbound

• Solution: reflection points in routers

Upstream ISP

23

Overlay Forwarding on Router Line Cards

• Building block: tunnels

24

Where the overlay control plane runs? On Routers

• On Routers: by Router virtualization– Pros: fast updates of forwarding tables– Pros: efficient transmission of control

messages– Pros: fate-sharing

Processors

Switching Fabric

Line Cards

Router

25

Where the overlay control plane runs? On Servers

26

Where the overlay control plane runs? On Servers

• On separate set of servers– Update forwarding table on router line cards– Data packets reflected in-network

• Pros:– Pros: cheap compared to router– Pros: compatibility with legacy overlay server

• Cons:– Lack of fate sharing

27

Outline


• Scalable overlay monitoring– Registration of overlay links– Notification of different kinds of network events– Lazy recovery


28

Scalable Overlay Monitoring

Assumption: Rich connectivity, multiple alternative overlay pathsOverlays could even tolerate “false positive” notification

What to notify? Different applications may want notification of different events

Notification Benefits: Accurate adaptation (compared with RON)Reduce probing overhead, and increase scalability

29

Scalable Overlay Monitoring

• Notification preserve overlay link abstractions– Message format:

(overlay source, overlay destination, event)– Routers store states by explicit overlay registration

• Explicit notification about events which affect performance of overlay applications– Physical failures of routers or links– Reachability failures: route withdraw, routing session

failure– Network congestion– few “hello” packets lost

30

Registration of Overlay Links

1A 2 3

4

B

C

Overlay Nodes: A, B, C

Routers: 1, 2, 3, 4

Register for uni-directional overlay links A->B and A->C

31

Periodical Registration of Overlay Links

1A 2 3

4

B

C

ACK for successful registration

(A,B) (A,B) (A,B) (A,B)

32

Periodical Registration of Overlay Links

1A 2 3

4

B

C

(A,B)(A,C)

(A,B)(A,C)

(A,B) (A,B)

(A,C)

(A,C)

Registration kept as soft state

Periodical re-registration

33

Notification of Network Events

1A 2 3

4

B

C

(A,B)(A,C)

(A,B)(A,C)

(A,B) (A,B)

(A,C)

(A,C)

34

Reactive Routing and Lazy Recovery

• Assumption: rich connectivity

• Reactive routing after notification– Re-route via alternative overlay paths– Disseminate notification message to peers

• Lazy recovery– Stick to alternative overlay paths (e.g. for mins)– Re-register for failed overlay– Reason: transient period during convergence of

recovery, causing loops and blackholes

35

Outline




36

Unicast Registration is Inefficient

1A 2 3

4

B

C(B,A)(C,A)(D,A)(E,A)

D

E

(B,A)(C,A)(D,A)(E,A)

(B,A)(C,A)

(D,A)(E,A)

(B,A)

(C,A)

(D,A)

(E,A)

• Overlay Nodes: A, B, C, D, E and Routers: 1, 2, 3, 4• Register for overlay links B->A, C->A, D->A, E->A

37

Unicast Notification is inefficient

1A 2 3

4

B

C(B,A)(C,A)(D,A)(E,A)

D

E

(B,A)(C,A)(D,A)(E,A)

(B,A)(C,A)

(D,A)(E,A)

(B,A)

(C,A)

(D,A)

(E,A)

38

Multicast Registration

1A 2 3

4

B

C

D

E

GroupA GroupA GroupA

GroupA

GroupA

GroupA

GroupA

39

Multicast Notification

1A 2 3

4

B

C

D

E

GroupA GroupA GroupA

GroupA

GroupA

GroupA

GroupA

40

Benefits of Multicast registration/notification

• Reduce registration states stored at routers– Unicast: store state for each (src, dst) pair,

O(n2)– Multicast: store state each mcast group, O(n)

• Reduce notification message overhead

• Deployment Benefits:– Exploit IP-Multicast (which routers already

have)

41

Outline




42

Prototype Implementation on VINI

• What’s finished?– RON

• Control plane: probing and reactive routing• Data plane: overlay tunnel setup• User Opt-in: user data packets delivered by overlays

– UFO: Notification of link failure

• What to do next?– UFO

• Evaluate inter-domain routing convergence• Notification of link congestion

– Run applications: e.g. VoIP

43

Prototype Implementation on VINI

• Overlay: RON• Overlay FIB• Client opt-in• Notification by Filter

XORPIP Router

UML

eth1 eth3eth2eth0

Click

PacketForwardEngine

Control

DataUmlSwitch

element

Tunnel table

Filters

PlanetLab VM

RON

OverlayFIB

VPN ServerClients

44

Evaluation Setup

• Topology– Routers and Overlay nodes

s

d

r

45

Evaluation1: Reactive Routing of RON

• How much time does RON spend to detect outage? – RON probe interval : 12s– RON probe timeout: 3s– Average detection time =

Probe interval / 2 + probe timeout * 3

• What to evaluate?– Fundamental trade-off between probe

frequency and detection time– Parameters: probe interval

46

Evaluation1: Reactive Routing of RON

• Detection time = probe interval / 2 + probe timeout *3

47

Evaluation2: comparison of Convergence Speed

• Controlled Experiment– Fail a link by filtering all the packets

• Comparison of Convergence Speed– IP routing (XORP)– RON reactive routing– Reactive routing with UFO notification

48


– IP Routing (XORP)• Hello-interval: 15s• Router-dead-interval: 45s

Link down Link up

49


– RON• Probe interval: 12s• Probe timeout: 3s• Re-route immediately after outage detection

Link down Link upRON up

50


– UFO routing with explicit notification• Re-route immediately after outage notification

Link down Link upRON upUFO up

51

Outline




52

Deployability Benefits

• Forwarding Support:– Low barriers to entry– Routers already have hardware for setting

tunnels– Upgrade small fraction for overlay forwarding

• Notification Support:– Upgrade all routers to support notification

(could start with one AS)– Performance benefits and business incentives– Better real-time applications: VoIP

53

Related Work

• Overlay routing– Detour (Collins98)– Resilient Overlay Networks (Andersen01)

• Improving forwarding efficiency– Path reflection and path painting (Jannotti02)

• Reducing probing overhead– Routing Underlay for Overlays (Nakao03)

• Network virtualization– VINI, GENI, CABO, VERA

54

Conclusion

• Contributions– Scalable overlay routing is feasible with in-

network support– UFO provides strong reliability and a

compelling deployment model

• Future Work– Further performance evaluation– Applications: VoIP– Application Layer Multicast (with NEC Lab)

55

Acknowledgement

• General Exam Committee:– Prof. Jennifer Rexford (Advisor)– Prof. Larry Peterson– Prof. Vivek Pai

• Collaborators:– Andy Bavier and Nick Feamster (Georgia Tech)

• Cabernet Research Group

• VINI Support, Planetlab Operations

56

Questions?

57

FAQ: recovery notification ?

• UFO does NOT support notification of recovery, because:– Alternative overlay paths available (overlays

don’t care !)– Hard for routers to determine intra-domain

convergence: synchronization to determine data-plane convergence

– Hard for routers to determine inter-domain convergence

Yaping Zhu Advisor: Prof. Jennifer Rexford With: Andy Bavier and Nick Feamster (Georgia Tech) UFO: A...

Documents

Transcript of Yaping Zhu Advisor: Prof. Jennifer Rexford With: Andy Bavier and Nick Feamster (Georgia Tech) UFO: A...