Infocom 2003

An Approach to Alleviate Link Overload as

Observed on an IP BackboneTuesday, April 1st

Infocom 2003

Sundar Iyer1,2, Supratik Bhattacharrya2, Nina Taft2,

Christophe Diot2

1Stanford University, 2ATL SprintLabs

2Infocom 2003

Contents

1. Introduction

2. Pathology of link overload

3. Alleviate overload - deflection routing

4. Performance analysis

3Infocom 2003

There should be no link overload

IP backbones are Overprovisioned low average utilization Have multiple paths

Routing algorithms balance load across multiple shortest paths should reduce the likelihood of overload

Overload: More than 50% utilization

4Infocom 2003

But there is link overload

Shortest path routing puts load on a small set of equal cost shortest

paths causes unequal use of link capacity

Unpredictable traffic Short term load fluctuations e.g. hotspots

Failure Link failures, fiber cuts, network maintenance

Hard to predict all factors apriori

5Infocom 2003

Why bother about link overload?

Operators upgrade persistently overloaded links Peaks in link utilization cannot

increase average utilization

Severe link overload causes packet drops

Interactive, real-time applications make it mandatory to overcome overload

6Infocom 2003

Contents

1. Introduction

2. Pathology of link overload

3. Alleviate overload - deflection routing

4. Performance analysis

7Infocom 2003

Methodology

Measurement of data from the Sprint backbone

Analyzed 138 backbone links for 9 months SNMP link utilization data polled every 5

minutes The link utilization is an exponentially

weighted moving average (EWMA) Measurements under-estimate overload Short term fluctuations are missed

8Infocom 2003

Maximum load

Observation 1: There is always some overloaded link

Maximum Load

9Infocom 2003

Contribution of links to overload

Observation 2: Most of the links are not overloaded

Non-Overloaded linksOverloaded links

10Infocom 2003

Types of link overload

Observation 3: Two types — Persistent

Periods of link overload

and temporary overload

Observation 4: Often just 1-2 links are simultaneously overloaded

11Infocom 2003

Causes of temporary link overload

Observation 5: Link failures cause temporary overload

Link Utilizations

Observation 6: Fiber cuts cause severe overload

12Infocom 2003

Contents

1. Introduction

2. Pathology of link overload

3. Alleviate overload - deflection routing

4. Performance analysis

13Infocom 2003

The case for deflection routing

Previous techniques useful for long term overload change normal functioning of the network useful when overload is common

We observe that link overload is relatively rare (0.1% of the time on any link) are typically caused due to link failures/maintenance lasts for minutes-hours on average occurs on maximum of 1-2 links simultaneously can be easily overcome by deflecting packets

Allow normal network operation most of the time

14Infocom 2003

Problem

Problem:

How can we design a simple, stateless, loop-free deflection algorithm to overcome link overload?

Theorem 1: (sufficiency)

Any deflection algorithm which deflects packets with “strictly decreasing cost” is loop-free

15Infocom 2003

Explanation of Theorem 1

A packet is forwarded from node s to d according to the strictly decreasing cost criteria as follows

1. If shortest path not overloadedForward the packet on the shortest path with cost C

2. If link to neighboring node n is not overloadedForward the packet to n if n’s cost to d is C

3. ElseForward the packet on the shortest path

16Infocom 2003

Intuition for Theorem 1

Shortest path routing: forward packet on the shortest path the sequence of costs to a destination is strictly decreasing

30

Router: s

10

25

20

20

10

Router: n3

Router: n2

Router: n1

Router: d

15

Loop-free deflection routing:

Yes

No

we do not consider the cost of reaching the deflection node

17Infocom 2003

Problem

Problem:

Can we always find loop-free deflection paths according to the strictly decreasing cost criteria?

Theorem 2: (sufficiency)

A network with redundant equal length paths always has a loop-free deflection path if the link weights are in a ratio 1 + 1/(d-1), where d is the diameter of the network

18Infocom 2003

Requirements

Intuition: All link weights are in the range [Wmin ,Wminx]

the minimum cost of the shortest path is dWmin

the maximum cost of the deflection path is (d-1)Wminx

(d-1)Wminx dWmin x 1 + 1/(d-1)

Criteria for Theorem 2 Need equal length shortest paths between any two

nodes Weights need to be within a bounded ratio “1 + 1/(d-1)” The diameter d of the network should be small

19Infocom 2003

Topology ConsiderationsInter-PoP Network

Large inter-POP weights are within ratio

Redundant equal length paths are guaranteed

NYC-2 NYC-4

NYC-1

NYC-3

RTP-2 RTP-4

RTP-1

RTP-3

FW-2 FW-4

FW-1

FW-3

CHI-2 CHI-4

CHI-1

CHI-3

ANA-2 ANA-4

ANA-1

ANA-3

SJ-2 SJ-4

SJ-1

SJ-3

PoPSan Jose

PoPAnaheim

PoPChicago

PoPFort-Worth

PoPNew York

PoPRTP

Small diameter, d=3

20Infocom 2003

Topology ConsiderationsComplete Network

Large Inter-POP Weights

NYC-2 NYC-4

NYC-1

NYC-3

RTP-2 RTP-4

RTP-1

RTP-3

FW-2 FW-4

FW-1

FW-3

CHI-2 CHI-4

CHI-1

CHI-3

ANA-2 ANA-4

ANA-1

ANA-3

SJ-2 SJ-4

SJ-1

SJ-3

Perfect Mesh in PoPs

Small (wmax) Intra-POP Weights

Diameter is larger

Redundant equal length paths not guaranteed

21Infocom 2003

Problem

Inter-PoP Network: PoPs as a single ‘logical node’

+ All criteria for theorem 2 are satisfied

The complete network- Equal length redundant paths does not exist

- Diameter of the network is not small

- Maximum intra-PoP link weight wmax is unrelated and very small compared to inter-PoP link weights

Problem- Cannot satisfy theorem 2 for the complete network

22Infocom 2003

Practical deflection routing algorithm

Solution: Clumping a PoP A packet is forwarded from node s to d as follows,

where wgain = wmax

1. If shortest path not overloadedForward the packet on the shortest path (with cost C)

2. If link to neighboring node n is not overloadedForward the packet to n if n’s cost to d is C – wgain

3. Else if link to (intra-PoP) node n’ is not overloadedForward the packet if its cost to d is C + wmax

4. Forward the packet on the shortest path

Inter-PoP

Intra-PoP

23Infocom 2003

Theorem 3

Theorem 3:

The practical deflection routing algorithm has no inter-PoP loops

Comments The sequence of costs strictly decreases across

PoPs This is in keeping with the idea of ‘PoPs’

Link failures The algorithm is extended by setting wgain = (n-1)wmax

24Infocom 2003

Contents

1. Introduction

2. Pathology of link overload

3. Alleviate overload - deflection routing

4. Performance analysis

25Infocom 2003

Simulations

Simulation parameters

14 node inter-PoP network and 4-5 node intra-PoP network Estimated traffic matrix with gravity models & link

measurements Deflection threshold was set to 45% Deflection based on fast EWMA Simulations for link failures and fiber cuts

26Infocom 2003

Link overload due to a fiber cut

Deflection routing decreases the maximum load amongst all links in the backbone

27Infocom 2003

Conclusions

Deflection routing algorithm Based on practical considerations and overload pathology Exploits backbone architecture, meshed topology Mandates a condition on weights which is not too

restrictive Is loop-free across PoPs

Note Needs a redundant backbone network with equal-length

paths Useful when average utilization is low

Future Work Stability needs to be investigated