Routeurs: Evolution ettechnologies

Cours de C. Pham

Transparents réalisés parNick McKeown, Pankaj Gupta

nickm@stanford.eduwww.stanford.edu/~nickm

2

“The Internet is a mesh ofrouters”

The Internet Core

IP Core router

IP EdgeRouter

3

The Internet is a mesh of IProuters, ATM switches, frame

relay, TDM, …

AccessNetwork

AccessNetwork

AccessNetwork

AccessNetwork

AccessNetwork

AccessNetwork

AccessNetwork

4

The Internet is a mesh ofrouters mostly interconnected by

(ATM and) SONET

TDMTDM

TDMTDM

Circuit switchedcrossconnects, DWDM etc.

5

Where high performance packetswitches are used

Enterprise WAN access& Enterprise Campus Switch

- Carrier Class Core Router- ATM Switch- Frame Relay Switch

The Internet Core

Edge Router

6

Ex: Points of Presence (POPs)

A

B

C

POP1

POP3POP2

POP4 D

E

F

POP5

POP6 POP7POP8

7

What a Router Looks LikeCisco GSR 12416 Juniper M160

6ft

19”

2ft

Capacity: 160Gb/sPower: 4.2kW

3ft

2.5ft

19”

Capacity: 80Gb/sPower: 2.6kW

8

Basic Architectural Components

OutputScheduling

Control Plane

Datapath”per-packet processing

SwitchingForwardingTable

ReservationAdmissionControl Routing

Table

Routing Protocols

Policing& AccessControl

PacketClassification

Ingress EgressInterconnect

1. 2. 3.

9

Basic Architectural ComponentsDatapath: per-packet processing

2. Interconnect 3. EgressForwarding

Table

ForwardingDecision

1. Ingress

ForwardingTable

ForwardingDecision

ForwardingTable

ForwardingDecision

10

Generic Router ArchitectureLookup

IP AddressUpdateHeader

Header Processing

AddressTable

LookupIP Address

UpdateHeader

Header Processing

AddressTable

LookupIP Address

UpdateHeader

Header Processing

AddressTable

Data Hdr

Data Hdr

Data Hdr

BufferManager

BufferMemory

BufferManager

BufferMemory

BufferManager

BufferMemory

Data Hdr

Data Hdr

Data Hdr

11

RFC 1812: Requirements forIPv4 Routers

• Must perform an IP datagram forwardingdecision (called forwarding)

• Must send the datagram out theappropriate interface (called switching)

Optionally: a router MAY choose to perform specialprocessing on incoming packets

12

Examples of special processing

• Filtering packets for security reasons• Delivering packets according to a pre-

agreed delay guarantee• Treating high priority packets

preferentially• Maintaining statistics on the number

of packets sent by various routers

13

Special Processing RequiresIdentification of Flows

• All packets of a flow obey a pre-definedrule and are processed similarly by therouter

• E.g. a flow = (src-IP-address, dst-IP-address), or a flow = (dst-IP-prefix,protocol) etc.

• Router needs to identify the flow of everyincoming packet and then performappropriate special processing

14

Flow-aware vs Flow-unawareRouters

• Flow-aware router: keeps track offlows and perform similar processingon packets in a flow

• Flow-unaware router (packet-by-packet router): treats each incomingpacket individually

15

Why do we Need Faster Routers?

1. To prevent routers becoming thebottleneck in the Internet.

2. To increase POP capacity, and toreduce cost, size and power.

16

0,1

1

10

100

1000

10000

1985 1990 1995 2000

Sp

ec95In

t C

PU

resu

lts

Why we Need Faster Routers1: To prevent routers from being the bottleneck

0,1

1

10

100

1000

10000

1985 1990 1995 2000

Fibe

r Cap

acity

(Gbi

t/s)

TDM DWDM

Packet processing Power Link Speed

2x / 18 months 2x / 7 months

Source: SPEC95Int & David Miller, Stanford.

17

POP with smaller routers

Why we Need Faster Routers2: To reduce cost, power & complexity of POPs

POP with large routers

Ports: Price >$100k, Power > 400W. It is common for 50-60% of ports to be for interconnection.

18

Why are Fast Routers Difficult toMake?

1. It’s hard to keep up with Moore’s Law:– The bottleneck is memory speed.– Memory speed is not keeping up with

Moore’s Law.

19

Memory BandwidthCommercial DRAM

1. It’s hard to keep up with Moore’s Law:– The bottleneck is memory speed.– Memory speed is not keeping up with

Moore’s Law.

0,0001

0,001

0,01

0,1

1

10

100

1000

1980 1983 1986 1989 1992 1995 1998 2001

Access

Tim

e (ns) DRAM

1.1x / 18months

Moore’s Law2x / 18 months

Router Capacity2.2x / 18months

Line Capacity2x / 7 months

20

Why are Fast Routers Difficult to Make?

1. It’s hard to keep up with Moore’s Law:– The bottleneck is memory speed.– Memory speed is not keeping up with

Moore’s Law.2. Moore’s Law is too slow:

– Routers need to improve faster thanMoore’s Law.

21

Router Performance Exceeds Moore’s Law

Growth in capacity of commercial routers:– Capacity 1992 ~ 2Gb/s– Capacity 1995 ~ 10Gb/s– Capacity 1998 ~ 40Gb/s– Capacity 2001 ~ 160Gb/s– Capacity 2003 ~ 640Gb/s

Average growth rate: 2.2x / 18 months.

22

Things that slow routers down• 250ms of buffering

– Requires off-chip memory, more board space, pins and power.• Multicast

– Affects everything!– Complicates design, slows deployment.

• Latency bounds– Limits pipelining.

• Packet sequence– Limits parallelism.

• Small internal cell size– Complicates arbitration.

• DiffServ, IntServ, priorities, WFQ etc.• Others: IPv6, Drop policies, VPNs, ACLs, DOS traceback,

measurement, statistics, …

23

First Generation Routers

Shared Backplane

LineInterface

CPU

Memory

RouteTableCPU Buffer

Memory

LineInterface

MAC

LineInterface

MAC

LineInterface

MAC

Fixed length “DMA” blocksor cells. Reassembled on egress

linecard

Fixed length cells or variable length packets

Typically <0.5Gb/s aggregate capacity

Most Ethernet switches and cheappacket routersBottleneck can be CPU, host-adaptor or I/O bus

24

Output 2

Output N

First Generation RoutersQueueing Structure: Shared Memory

Large, single dynamicallyallocated memory buffer:N writes per “cell” timeN reads per “cell” time.

Limited by memorybandwidth.

Input 1 Output 1

Input N

Input 2

Numerous work has proven andmade possible:– Fairness– Delay Guarantees– Delay Variation Control– Loss Guarantees– Statistical Guarantees

25

Limitations• First generation router built with 133 MHz Pentium

– Mean packet size 500 bytes– Interrupt takes 10 microseconds, word access take 50 ns– Per-packet processing time is 200 instructions = 1.504 µs

• Copy loopregister <- memory[read_ptr]

memory [write_ptr] <- register

read_ptr <- read_ptr + 4

write_ptr <- write_ptr + 4

counter <- counter -1

if (counter not 0) branch to top of loop

• 4 instructions + 2 memory accesses = 130.08 ns• Copying packet takes 500/4 *130.08 = 16.26 µs; interrupt 10 µs• Total time = 27.764 µs => speed is 144.1 Mbps• Amortized interrupt cost balanced by routing protocol cost

26

Second Generation Routers

RouteTableCPU

LineCard

BufferMemory

LineCard

MAC

BufferMemory

LineCard

MAC

BufferMemory

FwdingCache

FwdingCache

FwdingCache

MAC

Slow Path

Drop PolicyDrop Policy OrBackpressure

OutputLink

Scheduling

BufferMemory

Typically <5Gb/s aggregate capacity

Port mappingintelligence in linecards-better forconnection modeHigher hit rate inlocal lookup cache

27

RouteTableCPU

Second Generation RoutersAs caching became ineffective

LineCard

BufferMemory

LineCard

MAC

BufferMemory

LineCard

MAC

BufferMemory

FwdingTable

FwdingTable

FwdingTable

MAC

ExceptionProcessor

28

Second Generation RoutersQueueing Structure: Combined Input and Output

Queueing

Bus

1 write per “cell” time 1 read per “cell” timeRate of writes/reads determined by bus speed

29

Third Generation Routers

LineCard

MAC

LocalBufferMemory

CPUCard

LineCard

MAC

LocalBufferMemory

Switched Backplane

LineInterface

CPUMemory Fwding

Table

RoutingTable

FwdingTable

Typically <50Gb/s aggregate capacity

Third generation switch provides parallel paths (fabric)

30

Arbiter

Third Generation RoutersQueueing Structure

Switch

1 write per “cell” time 1 read per “cell” timeRate of writes/reads determined by switch

fabric speedup

31

Arbiter

Third Generation RoutersQueueing Structure

Switch

1 write per “cell” time 1 read per “cell” timeRate of writes/reads determined by switch

fabric speedup

Per-flow/class or per-output queues (VOQs)

Per-flow/class or per-input queues

Flow-controlbackpressure

32

Fourth Generation Routers/Switches

Switch Core Linecards

Optical links

100’sof feet

33

Physically Separating SwitchCore and Linecards

• Distributes power over multiple racks.• Multistage, clustering• Allows all buffering to be placed on

the linecard:– Reduces power.– Places complex scheduling, buffer mgmt,

drop policy etc. on linecard.

34

Do optics belong in routers?

They are already there.– Connecting linecards to switches.

Optical processing doesn’t belong onthe linecard.– You can’t buffer light.– Minimal processing capability.

Optical switching can reduce power.

35

Optics in routers

Switch Core Linecards

Optical links

36

Complex linecards

PhysicalLayer

Framing&

Maintenance

PacketProcessing

Buffer Mgmt&

Scheduling

Buffer Mgmt&

Scheduling

Buffer & StateMemory

Buffer & StateMemory

Typical IP Router Linecard

10Gb/s linecard: Number of gates: 30M Amount of memory: 2Gbits Cost: >$20k Power: 300W

LookupTables

SwitchFabric

Arbitration

Optics

37

Replacing the switch fabric with optics

SwitchFabric

Arbitration

PhysicalLayer

Framing&

Maintenance

PacketProcessing

Buffer Mgmt&

Scheduling

Buffer Mgmt&

Scheduling

Buffer & StateMemory

Buffer & StateMemory

Typical IP Router LinecardLookupTables

OpticsPhysicalLayer

Framing&

Maintenance

PacketProcessing

Buffer Mgmt&

Scheduling

Buffer Mgmt&

Scheduling

Buffer & StateMemory

Buffer & StateMemory

Typical IP Router LinecardLookupTables

Opticselectrical

SwitchFabric

Arbitration

PhysicalLayer

Framing&

Maintenance

PacketProcessing

Buffer Mgmt&

Scheduling

Buffer Mgmt&

Scheduling

Buffer & StateMemory

Buffer & StateMemory

Typical IP Router LinecardLookupTables

OpticsPhysicalLayer

Framing&

Maintenance

PacketProcessing

Buffer Mgmt&

Scheduling

Buffer Mgmt&

Scheduling

Buffer & StateMemory

Buffer & StateMemory

Typical IP Router LinecardLookupTables

Optics

optical

Req/Grant Req/Grant

Candidate technologies1. MEMs.

2. Fast tunable lasers + passive optical couplers.3. Diffraction waveguides.4. Electroholographic materials.

38

Evolution to circuit switching

• Optics enables simple, low-power,very high capacity circuit switches.

• The Internet was packet switchedfor two reasons:– Expensive links: statistical multiplexing.– Resilience: soft-state routing.

• Neither reason holds today.

39

Fast Links, Slow Routers

0,1

1

10

100

1000

10000

1985 1990 1995 2000

Fib

er

Ca

pa

cit

y (

Gb

it/s

)

Fiber optics DWDM0.1

1

10

100

1000

10000

1985 1990 1995 2000

Sp

ec95In

t C

PU

resu

lts

Processing Power Link Speed (Fiber)

2x / 2 years 2x / 7 months

Source: SPEC95Int; Prof. Miller, Stanford Univ.

40

Fewer Instructions

1

10

100

1000

1996 1997 1998 1999 2000 2001

(lo

g s

ca

le)

Instructions per packet since 1996

41

Some Mc Keown's predictionsabout core Internet routers

• The need for more capacity for a given power and volumebudget will mean:

• Fewer functions in routers:– Little or no optimization for multicast,– Continued overprovisioning will lead to little or no support for

QoS, DiffServ, …,• Fewer unnecessary requirements:

– Mis-sequencing will be tolerated,– Latency requirements will be relaxed.

• Less programmability in routers, and hence no networkprocessors.

• Greater use of optics to reduce power in switch.

42

What McKeown believe is mostlikely

The need for capacity and reliability will mean:

• Widespread replacement of core routerswith transport switching based on circuits:– Circuit switches have proved simpler, more

reliable, lower power, higher capacity and lowercost per Gb/s. Eventually, this is going to matter.

– Internet will evolve to become edge routersinterconnected by rich mesh of DWDM circuitswitches.

43

Click here to view forwardingtechnologies