slides

QoS Enablers for Next-Gen Networks

Krishan Sabnani

Bell Labs

04/13/23

2

Today’s Data Networks

AccessRouterAccessRouter

3G CellularNetworks

RadioController

RadioController


EnterpriseNetworks

HomeNetworks

MetroNetworks Packet-Based

Network

EdgeRouterEdge

Router

Edge RouterEdge Router

EdgeRouterEdge

Router


Ad hocNetworks

Not QoS enabled, secure, manageable, and reliable

04/13/23

3

The Network Evolution

Networks were designed to carry voice traffic

Data traffic mostly overlaid on voice networks (using Modems)

Networks are designed to carry primarily data traffic

Voice traffic overlaid on data networks (e.g. VoIP)

Future networks should be designed primarily for efficient content distribution and content search/location

– Content distribution should not only be overlaid, but built in from ground up

Future networks should also be able to effectively carry best-effort data traffic and QoS-sensitive voice.

Yesterday… …Today...

…Tomorrow ...

Volume of data trafficexceeds voice traffic

Content trafficbecomesdominant

04/13/23

4

Impact of Video on Network Traffic

VoD/IPTV will grow by an order of magnitude over the next 5 years

– 2005: 90% best-effort data traffic

– 2010: 40% high-priority VoD traffic; 50% best-effort traffic

– Carrier-class links with capacities of 40G and 100G required in MAN

Broadcast IPTV and streaming VoD is high-priority traffic, close to TDM

– Very different from today’s best-effort bursty data traffic

Source: Lucent Bell Labs IPTV/VoD study, May 2006

04/13/23

5

Tomorrow’s Converged Network

EdgeRouterEdge

Router



3G CellularNetworks

RadioController

RadioController


EnterpriseNetworks

HomeNetworks

Next-GenMetro

Networks

4G/Mesh

UserMobility

Traffic Type(Multimedia)

Qualityof Service

(e.g. for voice)

Network Intelligence

QoS-EnabledPacket Core Network

EdgeRouterEdge

Router

EdgeRouterEdge

Router

Services Enablement Layer

Always-OnAlways-OnGlobal RoamingGlobal Roaming

PersonalizationPersonalization

04/13/23

6

Enabling technologies

Future converged network will put significant demands on next generation network elements

– Carrier-grade and increased complexity demands

• QoS support, Scalability, Reliability, Security, Manageability

– SoftRouter

– Capacity demands

• Switching capacity > 100Tb/s will be required in a single switching element

– Optical Data Router

– High-speed wireless data access

• Simplify RAN to improve QoS

– Base Station Router

04/13/23

7

Motivation


– Carrier-grade demands


– SoftRouter


• Switching capacity > 100Tb/s may be required in a single switching element



• All wireless services will converge to IP


04/13/23

8

Routers are becoming increasingly complex

Complexity is an IP “Middle-Age” problem!

– IP provides end-to-end datagram delivery service to protocols/applications

– IP can use any link-layer technology that delivers packets

Emerging applications are driving more functions into IP, expanding the “waist” of the IP hour glass

Router vendors incorporate all new IP functions into routers

Complexity is spread throughout the network

• Achieving network-wide objectives such as traffic engineering requires complex translation of global objectives to configuration information in numerous individual routers

• Misconfiguration or uncoordinated configuration can result in poor performance or even network instability

email WWW phone...

SMTP HTTP RTP...

TCP UDP…

IP

ethernet PPP…

CSMA async sonet...

copper fiber radio...

email WWW phone...

SMTP HTTP RTP...

TCP UDP…

IP

ethernet PPP…

CSMA async sonet...

copper fiber radio...

IPmobile mcast

IPsecdiff-serv

NAT

04/13/23

9

Solution: SoftRouter

Disaggregation of router hardware from software addresses this problem and has the potential for major additional advantages

Bell Labs has a research program that disaggregates router control and transport planes (called SoftRouter-based approach)

• Transport plane: packet forwarding element

• Control plane: control element server and feature server

• Control plane servers and transport plane communicate using standard protocols

• Approach similar to SoftSwitch-based disaggregation of class 5 switches

04/13/23

10

New Router Architecture: SoftRouter

3 key components of SoftRouter approach– Decoupling: Separate complex control plane processing from the transport

plane

– Servers: Implement control plane processing functions on dedicated external control plane servers

– Standard Interface: Define standard protocol for control plane servers to interface to the forwarding elements

Control plane

processing

Forwarding plane

processing

ProprietaryAPI

Standardprotocol

Current Router Model SoftRouter Model

Control Plane

Transport Plane

FeatureServer

PacketForwarding

Element

Control ElementServer

04/13/23

11

SoftRouter Network Architecture

Router

PacketForwarding

Element

ControlElementServer

Traditional Router-based Network

SoftRouter-basedNetwork

The SoftRouter approach separates and centralizes the software-intensive control element and feature servers from

hardware-centric transport and packet forwarding

FeatureServer

04/13/23

12

SoftRouter Benefits

1. Lower Costs

– Commoditized, standards-based hardware (lower capex)

– Dedicated control element servers imply fewer management points (lower opex)

2. New Features

– Network-based features to support new services more easily added using open APIs

– Incremental deployment made simpler through centralized management

3. Better Scalability

– Centralized control element servers easier to scale using well-established server scaling techniques

4. Enhanced Stability, Controllability, and Reliability

– Network instability problems due to BGP Route Reflectors are eliminated

– SoftRouter-based network can be designed to be more reliable than a traditional network

5. Increased Security

– Fewer control element servers easier to secure using perimeter defense systems, e.g., firewalls

04/13/23

13

Technical Challenges: Summary

Protocol aggregation: how would protocols like OSPF/BGP operate when a single protocol instantiation at the control element server manages multiple forwarding elements?

– OSPF: Preliminary results indicate 50ms failure recovery time is feasible when the SoftRouter network is managed by one or two primary CE/OSPF processes and the propagation delay from CE to its FEs is small

– BGP: Full I-BGP mesh can be maintained among the few control servers, eliminating network instability possible under the Route Reflector architecture

Network design: where do we place the control element servers and how do we determine which control element servers manage which forwarding elements?

– Algorithm based on recursive graph bisection appears to work reasonably well in identifying where to place the CEs and which set of FEs that each CEs manage

Bootstrapping paradox: the forwarding element needs updated forwarding tables in order to route packets to its control element server but only the control element server can update the forwarding tables

– We develop a new discovery protocol to break the above circularity

– This protocol allows each FE to bind to its best CE and provides simple routing capability between them

04/13/23

14

Motivation




– SoftRouter







04/13/23

15

Router Capacity over Time

Growth in Router Capacity

Internet traffic doubles every 12 months

Router Capacity doubles every 18 months

Routers will reach 100Tb/s in 2010

Single Rack <1Tb/s Multi-rack solutions >1Tb/s

e.g. Juniper TX640or Cisco XR12416

e.g. Juniper TX Matrix or Cisco CRS-1

Routers2x every

18 months

Internet2x every

12 months

Source: Nick McKeown, Stanford

Currently Third Generation of Packet Routers : 1Tb/s capacity in a Single Rack

04/13/23

16

Why do current packet routers not scale?

Power and heat dissipation limit density.

– Require multiple shelves and racks of Line Cards

– Increases distance between Line Cards and Switch Fabric

SwitchFabric

CentralScheduler

ToNetwork

Line Cards

PacketProcessing

Buffer

Optics

I/OPacket

Processing

Buffer

Optics

I/OPacket

Processing

Buffer

Optics

I/OPacket

Processing

Buffer

Optics

I/O

Centralized Switch Fabric

– Switch Fabric alone pushes power density and heat dissipation limits

– Massive wiring density between Line Cards and Switch Fabric

Centralized Control and Scheduling complexity grows nonlinearly

Centralizedfunctions

scale poorly

Massive Wiring

04/13/23

17

SwitchFabric

CentralScheduler

ToNetwork

Line Cards

PacketProcessing

Buffer

Optics

I/OPacket

Processing

Buffer

Optics

I/OPacket

Processing

Buffer

Optics

I/OPacket

Processing

Buffer

Optics

I/O

Bell Labs Solution: Distributed Optical Switch Fabric

Optical fibers eliminate massive wiring density and allow higher bandwidths

CentralScheduler

ToNetwork

Line Cards

PacketProcessing

Buffer

Optics OpticsPacket

Processing

Buffer

Optics OpticsPacket

Processing

Buffer

Optics OpticsPacket

Processing

Buffer

Optics

T-TX Optics AWGFiber

Central Switch fabric replaced by passive optical device that directs signals according to their wavelength

– Called Arrayed Waveguide Grating (AWG) Switching using Fast Wavelength Tunable Transmitters (T-TX)

distributed on each line card– Deployed hardware scales with deployed capacity– Passive Optical Backplane

Fixes HardwareScaling

Passive Optical Backplane

04/13/23

18

Bell Labs Solution: Distributed Scheduler

Eliminate central scheduler without sacrificing throughput

– Distribute traffic uniformly to Line Cards

– Technique called “load balancing”

ToNetwork

Line Cards

AWG

PacketProcessing

Buffer

Optics Optics

LocalScheduler

PacketProcessing

Buffer

Optics Optics

LocalScheduler

PacketProcessing

Buffer

Optics Optics

LocalScheduler

PacketProcessing

Buffer

Optics Optics

LocalScheduler

Each line card works like a small router with its own scheduler– Scales gracefully since complexity of scheduling is constant– Deployed hardware scales with deployed capacity

Combined with Distributed Optical Switch Fabric allows scaling to very large routers

RouterScales to >100Tb/s

Lucent Technologies – Proprietary

Use pursuant to company instruction

04/13/23

19

Optical Data Router

DARPA-funded program Linecards

– Highly integrated optical components• Denser integration of optical interfaces on line

card

• Data never converted to electronics for lower power

– Shallow Buffers• Demonstration of high throughput with buffers

20 packet deep

Scalable switch fabrics– Highly scalable nonblocking switch fabrics

• 5Tb/s using AWG plus tunable lasers and 100Gb/s Transmitter

– Use of load balancing to allow blocking switch fabrics to be used and simplifies scheduling

• Scalability to 256 Tb/s throughput

• Transparent optical packet router demonstrated

100Gb/s Packet

switching

Space SwitchSpace Switch Time Buffer

AWG3

drop

WC1

WC2

AWG1

WC3

WC4

AWG4

AWG2

TL4

TL3TL1

TL2

MZM EAM PD

PD

40GPG

40GCLK

40GBERT

NR

ZD

ata

RR

RR

TB

Sch

edul

e

FDLs

CSG

Optical Packet Router

Highly integrated photonic

chips

Effects of shallow buffers

04/13/23

20

The Optical Data (IRIS) Router in a Carrier Network

IRIS intermediate

node

IRIS intermediatenode

IRIS Router

Riverstone15008

IRISintermediate

node

Riverstone15008

IRISedgecard

IRISedgecard

10 GbE

10GbE

40 Gb/s optical packets

Riverstone15008

Riverstone15008

10GbE 10

GbE

04/13/23

21

Motivation




– SoftRouter







04/13/23

22

Current wireless networks are complex, involving many network elements, and result in high cost and high latency

Base Station Router terminates all air interface-specific functionality in the base station

Base Station Router: push intelligence to the edge

Collapsing Radio Access Network elements into the base station simplifies network and reduces latency

Pushing IP intelligence to the base station results in better Quality of Service support

Base Station

O

MobileSwitching

Center

Base Station

O

MobileRouter

O

RadioController

PacketBackhaul

Packetbackhaul

Telephone Network

Internetpacketdata

circuitvoice

Base Station Router

O

MobileRouter

04/13/23

23

BSR Motivation

BSR provides advantages in the following areas

reduces system OpEx by 75%

increases VoIP Capacity by 25%

reduces delay by 40%

O

OO

O

PSTN

IP

OO

OOOO

O

PSTNPSTN

IPIP

BSR

Collapse access, control, and gateway elements to a single IP Access Point

AccessControl

Gateway

04/13/23

24

Lucent Technologies' Base Station Router Receives CTIA Emerging Technology Award

Revolutionary Product Takes Top Honors for Most Innovative In-Building Solution

LAS VEGAS – Lucent Technologies (NYSE:LU) today announced that its Base Station Router (BSR) product was selected as the first place winner of a CTIA WIRELESS 2006 Wireless Emerging Technologies (E-tech) Award in the category of “Most Innovative In-Building Solution.” Award recipients were announced yesterday in a ceremony at the Las Vegas Convention Center during the CTIA WIRELESS trade show.

The Wireless E-tech Awards program is designed to give industry recognition and exposure to the best wireless products and services in the areas of Consumer, Enterprise and Network technology. Nearly 200 applications were submitted and reviewed by a panel of recognized members of the media, industry analysts and executives, as well as select show attendees. Products were judged on innovation, functionality, technological importance, implementation and overall “wow” factor.

04/13/23

25

State of QoS Deployment and Research

04/13/23

26

Typical Objections to QoS Deployment

QoS can be avoided by over-provisioning– May work for core network, not true for access network (difficult to add

capacity) and wireless network (limited air spectrum)

– Instantaneous congestion can still occur, disrupting flows that have strict QoS requirements

QoS is too expensive to implement– Range of QoS mechanisms available, some scale better than others

– Fine grain QoS at the edge and aggregate based QoS in the core

QoS is too complicated to manage– Results show that many benefits of traffic management can be achieved by

placing QoS-awareness in a small (5-10%) number of network elements

04/13/23

27

State of QoS Deployment

Core: MPLS, some with traffic engineering, is widely implemented in core networks

Access: Most access network elements have built-in QoS features such as policing. New services such as VoIP/IPTV will need better QoS support such as diff-serv.

Enterprise: Enterprises are increasingly deploying QoS-aware boxes at their edges. Main functions are application prioritization and acceleration.

Cellular networks: Limited bandwidth has made QoS Support an absolute must in air interfaces.

– Air interfaces use mechanisms to ensure fairness among mobiles.

– Radio access networks (RAN) differentiate among mobile flows.

04/13/23

28

EthernetAccess

DSL/CableAccess

TelephoneAccess

802.11Access

CircuitMobile Network

PacketMobile

Network

IAD

IP/MPLS IP/MPLS BackboneBackboneIP/MPLS IP/MPLS BackboneBackbone

Focus on Focus on Core networkCore network

Focus on Focus on Core networkCore network

Core Networks are implementing MPLS

MPLS label-switched paths

Bandwidth guaranteed

Strong traffic isolation and flexible routing

Allows traffic to be routed away from congestion points

Key tools for traffic engineering and policy-based routing

04/13/23

29

Access Networks are implementing Differentiated Services

Differentiated services: Connection less IP-QoS mechanism– Many traffic classes

• EF: Expedited Forwarding

• AF: Assured Forwarding (several classes)

• Best-Effort

– Priority scheduling among different classes• E.g., EF traffic given highest priority, current best-effort traffic carried as

lowest priority

– Admission control for each class• E.g., Overall levels of EF traffic in network kept low to avoid starvation

– Network traffic shaped at edge using leaky-bucket controllers

– Network routing can be determined by IP routing protocols\

Pros:– No per-flow or per-LSP state needed in network

Cons:– Difficult to provide bandwidth guarantees finer than aggregate level

04/13/23

30

Enterprises are implementing Application Acceleration Deploy deep packet inspection at enterprise edge routers to

– Classify packets by applications

– Provide prioritization among different application packets

– E.g., VoIP flows given priority over email traffic

Some large corporations implementing large-scale VPN or even private network to enable control over its internal routing

– E.g., Google

04/13/23

31

Wireless NetworksNeed More Than Diff-Serv and MPLS Air Interface: Time-varying bandwidth constrained channel

– Diff-serv and MPLS only differentiate between traffic classes

– Wireless scheduling needs to take into account the channel conditions for maximizing cell-capacity: fundamentally different from scheduling diff-serv and MPLS traffic on wireline links

– New scheduling mechanisms use information on channel condition and current queue-lengths to maximize throughput while maintaining QoS

Radio access network: Low bandwidth network transports control, real-time, streaming, and best-effort traffic– Requires QoS mechanisms but all traffic is encapsulated in tunnels

– Diff-serv and MPLS mechanisms can be used if different traffic classes are carried over different tunnels and not on the same tunnel

04/13/23

32

Provides minimum and maximum user throughput bounds

Provides “hogging prevention”

Used for inter-user throughput allocation. (Can be used for intra-user as well)

Proportional Fair with Minimum Rate (PFMR) Algorithm

0)( if

0)( if )(

otherwise 0

slot in served user if )()(

)()()()1(

)(

)(max

max

min

)(

nTR

nTRnX

ninDRCnr

nrnXnTnT

enR

nDRC

ii

iii

ii

iiii

nTa

i

ii

ii

miniRmax

iRir

0Research work of:Research work of:

Alexander Stolyar; Matthew Andrews; Lijun Qian (Bell Labs)Alexander Stolyar; Matthew Andrews; Lijun Qian (Bell Labs)Research work of:Research work of:

Alexander Stolyar; Matthew Andrews; Lijun Qian (Bell Labs)Alexander Stolyar; Matthew Andrews; Lijun Qian (Bell Labs)

Ti is a virtual “token queue” corresponding to each flow i

Tokens arrive in the token queue at the rate Xi which is Ri

min or Rimax per slot

If user i is served in slot n, then DRCi (n) tokens are removed from the queue

If in a time interval, the average service rate of flow i is less than Ri

min, the token queue size has a positive drift, so the chances of flow i being served in each time slot gradually increase

Ti is a virtual “token queue” corresponding to each flow i

Tokens arrive in the token queue at the rate Xi which is Ri

min or Rimax per slot

If user i is served in slot n, then DRCi (n) tokens are removed from the queue

If in a time interval, the average service rate of flow i is less than Ri

min, the token queue size has a positive drift, so the chances of flow i being served in each time slot gradually increase

04/13/23

33

Provides minimum and maximum user throughput bounds

Provides “hogging prevention”

Provides individual stream minimum throughput and latency control

Provides unified mechanism for inter-user and inter-flow (intra-user) resource allocation. This has its pluses (universality, higher overall throughput) and minuses (complexity)

Key to supporting VoIP in Ev-DO networks

PFMR with Latency Control (PFMR-LC) algorithm:PFMR with Latency Control (PFMR-LC) algorithm:

04/13/23

34

State of QoS Research

QoS research is as old as networking itself

– Lots of papers have appeared, most have analytical focus

Most skeptics of QoS are also in academic

– QoS mechanisms are being implemented and used, adoption is mostly driven by real-world requirements, not by elegance of the theory

Is more QoS research needed?

– How QoS mechanisms can be implemented in the networks of future?

– Inter-domain QoS peering

04/13/23

35

Net Neutrality

Should transport providers be allowed to differentiate traffic transported over their networks?

– Verizon, AT&T, Comcast, … : ability to offer service differentiation provides the incentive for large capital investment in building out new access networks

– Google, Yahoo, Microsoft … : service differentiation can be used by transport providers to favor their own traffic or traffic from their favored content aggregators, giving them an unfair advantage over 3rd party content providers, and thus slowing competition and innovation

My position:

– Transport providers should offer services with QoS

– Such services should however be universally available to all content aggregators

04/13/23

36

Conclusions

Network convergence is happening QoS is a necessity for next generation converged networks Bell Labs has several programs to produce key assets for creating next

generation converged networks– SoftRouter– Optical Data Router– Base Station Router

QoS is being deployed in service provider networks and enterprise networks

QoS research is still needed – Inter-domain QoS– Simpler implementation

slides

Documents

Transcript of slides