MPLS

and

Traffic Engineeringby

Sunit Chauhan

Sunil Bakhru

Copyright, 1996 © Dale Carnegie & Associates, Inc.

Why need MPLS?

• Limitations of existing IP Network• Network Scaling• Traffic Engineering• Provisioning of QoS

We need better control over the network.

• MPLS stands for MultiProtocol Label Switching.• Convergence of connection oriented forwarding

techniques and Internet’s routing protocols

MPLS Network

R1

R2 R5

Access A1

Access A2

Access A3

IP backbone R6

R4

R3

Destination D

Desired route from Ax to DActual route from Ax to D

Traffic Engineering to override shortest path route

Terminology

• Label - a short fixed length identifier used to identify a FEC, usually of local significance

• FEC - Forwarding Equivalence Class represents set of packets with common cross core forwarding requirements

• LSR - Label Switched Router• LER - Label Edge Router• NHLFE - Next Hop Label Forwarding Entry• ILM - Incoming Label Map

Maps label to a set of NHLFE entries• LSP - Label Switched Path

Path through one or more LSRs at one level of hierarchy followed by packets in a particular FEC

Key concept in MPLS

• Seperation of IP router’s function into Forwarding and Control

• Forwarding - deals with how data packets are relayed between IP routers, uses label swapping.

• Control - consists of network layer routing protocols to distribute routing information between LSR’s and label binding procedures for converting this routing information into forwarding tables needed for Label Switching.

Label Based Forwarding

• At each LSR, forwarding is done by the single index lookup into the switching table using the packet’s MPLS label.

• The switching table is loaded a priori with a unique next-hop label, output port and queuing and scheduling rules.

• The establishment of mapping information is responsibility of control part - done using Label Distribution Protocols

MPLS MPLS PayloadLabel

SwitchingTable Next hop +

portQueuing and Scheduling

Output Ports

InputPorts

Label Stack

• MPLS allows hierarchical labels supported as LIFO stack.

• A packet is always processed based on the top label regardless of other labels that may be below it.

• Each label stack entry is 32 bits.• 20 bits for label • 3 bits for experimentation • 8 bits for TTL and 1 stack

bit.

Label (20 bits)

Exp (3 bits)

S(1 bit)

TTL ( 8 bits)

Label (20 bits)

Exp (3 bits)

S(1 bit)

TTL ( 8 bits)

Label (20 bits)

Exp (3 bits)

S(1 bit)

TTL ( 8 bits)

Original Packet

MPLS stack encoding for packet-oriented transport

Control in MPLS

• Consists of • Network Layer routing protocols to distribute routing

information between LSRs.• Label binding procedures to convert this routing

information into the forwarding tables needed for label switching

• QoS routing requires additional information about availability of resources in the network and QoS requirements of each flow.

• A signaling protocol is also needed for reserving needed resources along the a selected routee.g. CR-LDP, M-RSVP

Label Distribution Control

• Ordered Vs Independent• In Independent control, each LSR upon noting

that it recognizes a particular FEC makes an independent decision to bind a label to that FEC and to distribute the binding information to its label distribution peers.

• In Ordered control, an LSR only binds a label to a particular FEC if it is egress LSR for that FEC or if it has already received label binding for that FEC from its next hop for that FEC.

• For an LSP to be set up with specified set of properties, ordered control must be used.

E.g. FEC = Traffic pertaining to an address prefix

Label Assignment Strategies

• Two LSRs must establish a bidirectional LDP session to exchange label binding information.

• Labels are always “downstream assigned”. • Label information flows in the direction opposite to

that of data packets

• Two modes of label distribution• Unsolicited Downstream - each LSR distributes

label bindings to its upstream LSRs (even if they haven’t explicity requested them).

• Downstream-on-demand - each LSR allows LSR upstream to explicitly request the label binding for a particular FEC.

Whom to send label binding information?

• Each LSR must distribute label bindings for an address prefix(FEC) to its label distribution peers for that FEC, which are• IGP neighbors (local peers)• BGP neighbors(remote peers)• Tunnel’s transmit endpoint(remote peer)

• Basically all this is topology driven MPLS where, each LSR must,• bind one or more labels to address prefix that appears

in the routing table• for each such FEC use label distribution protocol to

distribute the bindings to its Label distribution peers

Aggregation

• Procedure of binding a single label to a union of FECs which is itself an FEC and of applying that label to all the traffic in the union is known as “aggregation”• e.g. set of distinct address prefixes might have same

egress node. Please note that each address prefix forms an FEC and the union of those FECs is also an FEC.

Label Edge Router

• LER terminates and originates LSP’s and performs both label based forwarding and conventional NIF functions.

• Ingress LER - labels unlabelled packets and creates an initial MPLS frame by pushing one or more MPLS label entries.

• Egress LER - terminates LSP by popping the top MPLS stack entry.

Conventional IP ports

Output port, label, queuing and scheduling rules

Forwarding Table

LSP Table(s)

Packet Labeling

Packet classification

IP IP payloadheader

MPLS IP PacketLabel

Queuing and scheduling

Context

MPLS core ports

Ingress LER

Recap

• Packet processing based on the top level label regardless of the label underneath

• FECs can be• coarse grained consisting of all the packets with

same destination address • Allow the overall system of be scalable where it is

useful to handle large bundle of flows as a single class of traffic

• Help in rerouting in event of occurrence of a fault

• fine grained as in packets belonging to a particular application running between two hosts.

• Help in providing different QoS to different flows.

Recap Contd.

• Mapping of packets to an FEC done only once at the Ingress router upon entry into an MPLS domain

• Subsequent packets are forwarded strictly according to their labels

• label is removed by egress LSR• Each LSR maintains label to NHLFE mapping giving

a set of entries for each FEC. • Mapping can be changed for

• load balancing over multiple paths• rerouting from a failed path to an alternate path

Route Selection

• Method used for selecting the LSP for a particular FEC.• Hop by Hop is the same as topology driven.• Explicit Routing

• Explicit route need to be specified only at the time that labels are assigned and not with each IP packet, as in case of IP routing

• Tunneling • A router Ru takes explicit action to cause a particular packet

to be delivered another router Rd even though Ru and Rd are not consecutive routers on the hop-by-hop path for that packet and Rd is not the packet’s ultimate destination. This concept is called tunneling.

• Hop-by-Hop routed tunnel • Explicitly routed tunnel

LSP Tunnels

• Implement a tunnel as a LSP, and use label switching rather than network layer encapsulation to cause a packet to travel through the tunnel.

• Set of packets sent through the LSP tunnel constitutes a FEC and each LSR in the tunnel must assign a label to that FEC.

• If a tunnel from Ru to Rd, then• Ru is transmit endpoint of the tunnel• Rd is receive endpoint of the tunnel

Hierarchy: LSP Tunnels within LSPs

• Consider an LSP <R1, R2, R3, R4>• R1 receives unlabelled packet P • R2 and R3 are not directly connected but are

neighbors by virtue of being endpoints of an LSP tunnel

• Actual sequence of LSRs traveled by P is <R1, R2, R21, R22, R23, R3, R4>

• Traveling from R1 to R2 packet P has label stack of depth 1

• Another label push at R2 as it enters the tunnel making label stack of depth 2 inside the tunnel.

• At R3 label stack is again of depth 1

• LSP tunneling can be nested to any depth

Traffic Engineering

Application of technology and scientific principles to the measurement, modeling, characterization and control of internet traffic. (Fundamentally, a control problem)

e.g mapping of traffic on IP network infrastructure

A network consists of :• Demand System (Traffic)• Constraint System (Interconnected N/W Elements)• Response System (N/W Protocols and Processes)

Traffic Engineering Process Model

• Performance Objective• Resource Oriented

• Efficient Link Utilization (Congestion Control)

• Traffic Oriented• Packet Loss• Delay / Delay Variation• Throughput

• Adaptive And Iterative Process

Formulation ofControl Policy

Observation ofNetwork State

Characterization oftraffic and network

state

Optimized?

ReviseControlPolicy

No

Optimized NetworkPerformance

No

Yes

Traffic Engineering Process Model

Response to Internet Growth by ISP’s

Capacity Expansion• DS3(44.736 Mbps) 1996

• OC-12c(622Mbps) 1997-98

• OC-48c(2.488Gbps) 1999

• OC-192c(9.953Gbps) over DWDM

• Traffic Engineering• Metric-Based Traffic Control till 1994-95

IP Overlay Networks

Physical Topology of a Large ISP Network in 1997-98

• DS-3 point-to-point links replaced by OC-3 ATM SAR interfaces at the edge and OC-12 in the core.

Operation of IP over ATM Network

PVC 1 PVC 2 PVC 3

• ATM infrastructure used only for High Speed transport• Physical paths for PVC overlay calculated by an offline

configuration utility• IP prefixes Router Interfaces ATM PVCs

IP over ATM

Benefits• High Speed interfaces

• Greater aggregate Bandwidth

• Deterministic Performance

• PVC functionality and hence traffic engineering

IP over ATM

Limitations• Administrative expense

• Equipment expense

• Operational stability

• Routing and traffic engineering on different sets of systems

• Cell Tax and n-squared problem

IP over ATM in Optical Network

The fastest commercially available ATM SAR router interface is OC-12. OC-48 packet over SONET(POS) interfaces are available but OC-48 or OC-192 ATM router interfaces will not be available in the near future.

Reason : Expense and complexity of implementing SAR functions at high speeds

MPLS and Traffic Engineering

• MPLS supports origination connection control through explicit LSP’s

• Traffic trunk : Aggregation of traffic belonging to the same class.

• Mapping of traffic trunks on to the network topology is done by selection of routes for explicit LSP’s.

• LSP tunnels provide• rerouting in congested conditions• Flexible cost effective survivability• Provide statistics for Traffic Matrix• Parameterized resource allocation

Components of MPLS TE Model

• Network State Information Dissemination• Extending conventional IGP’s link state advertisements

• OSPF extensions implemented with Opaque LSAs• IS-IS extensions implemented using Type Length Values (TLVs)

• Traffic Engineering Database(TED) maintained by each LSR

Components of MPLS TE Model

• Path Management• Selection

• Explicit route for LSP tunnel generated• Strict or Loose path(Abstract node) is specified• May be defined administratively or computed automatically by

a constraint-based routing entity(CSPF).

Components of MPLS TE Model

• Path Management• Instantiation or Placement

• Signaling Protocol which serves as an Label Distribution Protocol• Resource Reservation Protocol (RSVP) extensions• Constraint Routed Label Distribution Protocol (CR-LDP)

• Maintenance of LSP tunnelsSustain, Reroute or Terminate LSP tunnel

Components of MPLS TE Model

• Traffic Assignment• All aspects associated with allocation of traffic to

established LSP’s• Partitioning Function

• Network Management• Online management is

Non-deterministic• Offline management tools interfaced with MPLS to provide external feedback

Requirements for a Signaling Protocol

• Robustness

• Scalability

• LSP establishment/ teardown/maintenance

• Specification of QOS

• LSP Priority / Preemption

• Flexibility in Path Setup Options

U Message Message Message Mandatory Optional Type Length Id Parameters Parameters

1 5 16 32 variable variable

Format of LDP Messages

LDP Message Format

• Four classes of messages• Discovery messages to advertise the presence of LSRs

• Session messages to establish and maintain LDP sessions

• Advertisement messages to create, change and delete label mappings for FECs

• Notification messages to inform about faults

TLV Encoding

• Mandatory and Optional parameters• use TLV encoding with the format shown in figure• U bit : If the LSR does not recognize a particular TLV , U bit

tells whether to notify the sender and ignore the message or ignore TLV and process remainder of message

• F bit : Tells LSR whether to forward the unknown TLV• Type field - 14 bit to identify seven defined TLV types

FEC, Label, Address List, COS(Class of Service), Hop count, Path Vector and Status.

U F Type Length Value

1 1 14 16 variable

TLV Encoding

CR-LDP

• Extension of the LDP approach• Hard State Protocol• UDP used for peer discovery• TCP used for session, advertisement, notification, and LDP messages• Supports Diffserv and Operator configurable QOS classes• Failure reported using the reliable TCP

Extended RSVP

• Extension of the classical connectionless RSVP

• Path and Resv messages used with• Label_Request Object• Explicit_Route Object• Label Object

• Aggregation of flows to reduce state information in routers

• Soft State Control and scalability concerns

CR-LDP Vs RSVP

• Choice of Transport Protocol• UDP + TCP sessions Vs Raw IP

• Multipoint Support• Multipoint-to-Point : (Both support)

• Multicast (Point-to-Multipoint) : Yet to be addressed

• Scalability• Network Flows

• RSVP extensions for refreshing many LSPs in a single RSVP BUNDLE message and ability to indicate status quo rather than having to send the entire payload

• CPU Utilization

• Data Storage Requirements

CR-LDP Vs RSVP

• Link and Peer Failure • Both use HELLO extensions• RSVP uses the State Refresh Processing.• TCP failure implies re-establishment from

ingress• Policy Control

• RSVP messages carry policy objects• Security

• TCP is vulnerable to DOS attacks• RSVP has built in authentication and policy

control

LSP Rerouting

• Strictly specified route re-routed by the ingress node

• Loosely specified route :

• Refresh path for the next Hop of alternate route(RSVP)

• Make-before-Break using

• ’Modify’ flag in LABEL_REQUEST message (CR-LDP)

• Shared explicit filters may be used(RSVP)

• Problem of ‘Thrashing’ in unstable networks

• Loosely specified route may be ‘pinned’• Flagging the loose part of explicit route (CR-LDP)

• Record_Route Object is used in RSVP

LSP Modification and Protection

• Modification implies changing reservation parameters.

• Similar to rerouting

• Protection implies pre-programming of alternate paths

• Layer 2 protection transparent to the LSP is assumed

• Schemes for pre-programmed alternate paths from the ingress or intermediate nodes are under consideration.

MPLS QoS

• IP QoS can be based on:• Multifield IP header classification (up to 104 bits)

• Diffserv (DS field with 6 bits)

• MPLS performs classification and scheduling on the basis of 20 bit label and 3 bit experimental field

Per hop classification, queuing and scheduling

Port M

Queue

Queue

Queue

Queue

Port 1

Port N

Schedule

MPLS - QoS

• In MPLS, forwarding is independent of control and we have many different control planes .

• All control modules share a single QoS paradigm• same label lookup and 3 bits of experimentation determine

both the output queue and priority. • Unique to MPLS - same control mechanisms are invoked

regardless of which control plane assigned the labels.

• LER does multified classification to assign packets to LSPs with specific attributes(map EXP bits in the MPLS header)

• 3 bit E field => upto 8 different Diffserv code points will be available over a single tunnel. Work is in progress to define these code points.

Reliability in MPLS

• Current Internet has inherent survivability because of connectionless IP protocol.

• Loss of QoS is not a issue because the service is best-effort.

• In MPLS, Reliability is important because of support for integrated Services

• Fault Recovery• Centralized by a network Manager• Distributed control by having the LSRs automatically

initiate fault recovery procedures upon fault detection.

Requirements for Fault Recovery

• Fault Notification to all the affected LSRs• Search for an alternate path for the affected traffic• Rerouting to an alternate path• (Optional) redistribution of the network traffic to

ensure that capacity will be available to recover from subsequent faults

Notification

• Notification Message generated by the LSR downstream from where the fault is detected

• Notification message carries• type of error• Status TLV identifying

• failed link• cause of failure• affected FECs

Rerouting

• Recipient of Notification message depends on whether the recovery is done by link rerouting or edge-to-edge rerouting

• Link Rerouting• Alternate path is found between two LSRs on the ends of failed

link

Rerouting

• fast recovery • alternate path may be pre-established in the form of

another entry in NHLFE mapping for the label. Only for guaranteed services otherwise wasteful of resources

• For more efficient resource utilization, resource may not be reserved

• The notification message should check the availability of resources along the pre-established alternate path as it travels the upstream LSR

• dynamic creation of alternate path• Downstream LSR has QoS information, may select the

alternate path and send appropriate Label_Mapping message to reserve resources along the alternate path

Edge to Edge Rerouting

• More complex approach• Finds another link disjoint path from Ingress to

Egress LSR• Advantage of handling multiple link failures• Each effected FEC may have different

Ingress/Egress LSR pairs and must be rerouted individually• Hence the use of coarse grained FECs to minimize

the number of FECs to reroute.

Loop Prevention in MPLS

• Three levels of control • Loop survival

• allows forming of loop but looping packets does not impact transmission of non-looping packets. A node capable of TTL decrement has this capability.

• Loop detection• allows LSP to form a loop but detects and disconnects

in a short time, e.g. using LDP

• Loop prevention• most rigorous form of control

Loop prevention

• LSP can be• Non state merging LSPs : Distinct outgoing link state

exists for each incoming link state. E.g. Explicitly routed LSPs

• State merging LSPs : Multiple incoming states can be merged into a single outgoing states. E.g. Hop by hop routed LSPs

• NSM LSPs : Control message used to request a label contains list of addresses added by all the LSRs in the path. An LSR on detecting its address in the receiving message comes to know of the loop.

• Loop prevention in state-merging LSPs is non trivial• multiple state merging events may occur simultaneously

Path vector diffusion algorithm

• when a node receives a new outgoing label mapping while it has one or more incoming label mappings, it starts a diffusion computation

• creates a new query, for each incoming label, with a path vector containing creator’s address only

• and sends it corresponding upstream node

• waits for the acknowledgment of the query

Immediate Consequences of MPLS

• Efficient transit core network• Improved economy of scale• Better utilization of Network• Fast recovery from faults

References Web Sites

www.mplsrc.com

www.ietf.org/internet-drafts

www.datacon.co.uk

IEEE Communications Magazine – December 1999

MPLS and Traffic Engineering in IP Networks - discusses the applications of MPLS to traffic engineering in IP networks. By Daniel O. Awduche of UUNET, MCI WorldCom

MPLS Advantages for Traffic Engineering - discusses the architectural aspects of MPLS which enable it to address IP traffic management. By George Swallow.

Traffic Engineering Standards in IP Networks Using MPLS - overview of the various approaches for label distribution.

MPLS and the evolving Internet Architecture – By Toni Lee, Procket Networks, Inc.

Issues on Loop Prevention in MPLS Networks - provides an overview of various methods underdevelopment for preventing LSP loops. By Yoshihiro Ohba.

Reliable Services in MPLS - examines distributed methods for fast fault recovery using modified LDP messages. By Thomas M. Chen and Tae H. Oh

IEEE Communications Magazine - January 2000

MPLS : The Magic behind the Myths - By Grenville Armitage, Bell Labs Research Silicon Valley, Lucent Technologies

References

IETF Drafts

draft-ietf-mpls-arch.txt

draft-ietf-mpls-rsvp-lsp-tunnel.txt

Others

IP Traffic Engineering for Carrier Networks: Using Constraint-BasedRouting to Deliver New Services - White Paper on constraint-based routing. By Nortel Networks

Traffic Engineering for the New Public Network - An overview of the present and future of traffic engineering. By Juniper Networks

MPLS Traffic Engineering : A Choice of Signaling Protocols - White paper comparing RSVP and CR-LDP for label path distribution. By Data Connection

RSVP Signaling Extensions for MPLS Traffic Engineering - Describes extensions to RSVP to automate the establishment of LSP's across service provider networks. By Juniper Networks

• DiffServ and MPLS: A quality choice - Data Communications article comparing MPLS and DiffServ for use in building networks capable of delivering QoS

Daniel Obi Awduche's home page - Collection of papers and drafts on traffic engineering, optical networking and MPLS written by the manager of advanced technology for UUNET

Deploying MPLS Traffic Engineering - An overview of the process for deploying traffic engineering using MPLS. By Juniper Networks

Native IP Forwarding

• Longest-prefix match based on packet’s destination IP address

Forwarding Table

PacketClassification

IP header IP Payload

OutputPorts

Next hop + port

InputPorts

Queuing and Scheduling

NIF node forwarding Engine

Label Distribution Protocols

• No hard and fast rules for choosing a Label distribution protocol

• Requirements from a Label Distribution protocol• establish and maintain label bindings• information to be transmitted reliably and label

distribution protocol messages pertaining to a particular FEC need to be transmitted in sequence

• Flow control• Capability to carry multiple label messages in a

single datagram

• BGP speakers use BGP to distribute labels among themselves while routes are being distributed.

LDP

• Four classes of messages• Discovery messages to advertise the presence of

LSRs• Session messages to establish and maintain LDP

sessions• Advertisement messages to create, change and

delete label mappings for FECs• Notification messages to inform about faults

U Message Message Message Mandatory Optional Type Length Id Parameters Parameters

1 5 16 32 variable variable

Format of LDP Messages

LDP Message Format

• 16-bit message length field • gives total length of message in bytes

• 32 bit message id • uniquely identifies a particular message

• Mandatory parameters• are ordered set of required fields that depend on a

particular message type

TLV Encoding

• Mandatory and Optional parameters• use TLV encoding with the format shown in figure• U bit : If the LSR does not recognize a particular TLV , U bit

tells whether to notify the sender and ignore the message or ignore TLV and process remainder of message

• F bit : Tells LSR whether to forward the unknown TLV• Type field - 14 bit to identify seven defined TLV types

FEC, Label, Address List, COS(Class of Service), Hop count, Path Vector and Status.

U F Type Length Value

1 1 14 16 variable

TLV Encoding