1

Cross-layer Visibility as a Service

Ramana Rao Kompella

Albert Greenberg, Jennifer Rexford

Alex C. Snoeren, Jennifer Yates

2

Layering in the current Internet

IP

Optics

Fiber

MPLS

OVERLAYS

Ethernet

Fiber-spans

3

Layering is a mixed blessingLayering allows us to contain complexity

Each layer evolves independently without affecting any other layer

Allows us to focus on one layer at a time

There are associated challenges too… Routine operational tasks need associations

across layers Example: mapping an IP link to optical circuit,

overlay link to an IP path Lack of accurate cross-layer associations can

affect the reliability of the network

4

Intended planned maintenance

Los Angeles

San Francisco

Seattle

Denver

Dallas

St Louis

Chicago

Boston

New York

Orlando

Planned maintenance

on optics

5

Intended planned maintenance Optical component is

on circuit id A Lookup database to

map circuit id A to IP link

Due to mis-association, incorrectly maps it to LA to Dallas

Increase OSPF weight on LA to Dallas link

Disconnect component Causes failure

Los Angeles

San Francisco

Denver

Traffic from LA to Dallas is rerouted

via Denver

Dallas

X

LA to San Francisco link is

congested

High OSPF weight

Planned maintenance can induce faults if

accurate associations are not maintained

6

Customer Fault Tolerance

Customer in NJ

New York

Philadelphia

InternetInternet Shared optical element

INTRA-CARRIERDIVERSITY

Customer in NJ

New York

InternetInternet

Sprint

Level 3

INTER-CARRIERDIVERSITY

Going through same conduit or Holland tunnel ?

Customer diversity information requiresaccurate cross-layer

associations, sometimes across domains

7

Fault diagnosis

Los Angeles

San Francisco

Seattle

Denver

Dallas

St Louis

Chicago

Boston

New York

Orlando

What happened

?!!

X

Because of a bug, IP forwarding path changed, but MPLS

did not !

MPLS circuit between LA and

New York

Diagnosing faults requiresaccurate cross-layer

associations

8

Why is it hard ?Can’t the operators maintain associations in a

centralized fashion ? Maintain database as links are provisioned Update as and when interfaces are re-homed

Hard due to flux in topology Churn because of dynamic topology changes Human errors during re-homing interfaces Operational realities – separation of concerns

9

How it is done today ?A combination of non-standard databases

Human-generated inventory data Measurement data obtained from probes Configuration state from network elements

Policies implemented in network elements Higher complexity and overhead

No compatibility across ASes Difficult to evolve a network Difficult to integrate two networks after acquisition Difficult to incorporate third-party tools

10

Why not concentrate on restoration?Advantages of lower-layer restoration

Hides lower-layer failures from impacting upper layers

Obviates to some extent need for cross-layer visibility

Cross-layer visibility still important Lower-layer restoration more expensive than IP

restoration Subtle performance changes (e.g., RTT) need

diagnosis

11

Why not fatten the interfaces ? Fattening interfaces to make layers aware of

the entire topologies above and below Layers discover and propagate mappings

automatically Management system can query the network to

obtain mappings

Fattening results in high complexity Interoperability is a big challenge – long

design and test cyclesWider interfaces impact security

12

Cross-layerPolicyServer

Architecture for cross-layer visibility

Ping

Trace-route

Backboneplanning

Customer diversity

Backbonemaintenance

FaultdiagnosisDB

MANAGEMENTAPPLICATIONS

BOW-TIE

IP

Optics

Fiber

MPLS

OVERLAYS

IP HOUR-GLASS

Ethernet

Fiberspans

13

IP

Optics

FIBER, FIBERSPAN

MPLS

OVERLAYS

Standardize what goes in !

IP

Optics

FIBER, FIBERSPAN

MPLS

OVERLAYS Standardize what goes in(e.g. IP topologies)

AS1Facilitates interactionbetween ISP policy servers

AS2

14

Advantages of the bow-tieTopology, routing information and other

associations can be queried for maintenance, diversity, and fault diagnosis

Cooperation across ASes to present better visibility across domains

Policies easily enforced through the serverLower overhead on network elements

Caching of common queries possible Historical questions can be answered

15

Evolution path to improve accuracyA lot of room for improvementArchitecture accommodates evolution so that

accuracy can be improved over timeEvolution path for individual layers

Fiber & Fiber-spans Optical components IP links MPLS and overlay paths

16

Fiber & Fiberspans Automated mechanisms

[sebos02] Inject labels through

fibers or use RFID GPS to determine the

location of fibers Transmit this information

to the DB More coverage results in

better accuracy but expensive

FIBER

DB

GPSOPTICALTAPS / RFID

FIBER

17

Optical componentsManual mechanisms

Basic consistency checks Automatic correlation mechanisms such as

[kompella05nsdi] to output errors

Automatic mechanisms Neighbor discovery for active optical devices Configuration state from “intelligent” optical

networks (that support dynamic restoration)

18

Optical components

Neighbor discoverythrough periodic

broadcasts at opticallayer

DB

Configuration state during restoration

Intelligent Optical

Network

Intelligent Optical

Network

ROUTER A ROUTER B

19

Other layers IP layer

Periodically obtain configuration information to construct topology

Automatically collect up/down messages to provide up-to-date view

MPLS and overlay paths Static paths obtained from configuration Dynamic paths obtained by monitoring signaling

messages

20

SummaryAccurate associations critical to many

operational tasksA bow-tie architecture for cross-layer visibility

Provides the cross-layer associations as a service to various applications

Allows better cooperation among ASes through standardizing what goes into the database

Policy controlled export of these associations Lower overhead on network elements Allows for innovation while containing complexity

21

Future research directionsDesign automated mechanisms at each layer

to improve cross-layer visibility What frequency should information be obtained? How do we resolve conflicts (minimal edits) in the

database?

Identify higher-level models that we need to standardize

Devise incentives for cooperation among ASes

Define a language to specify policies

22

Questions ?