Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All...

51
Cisco Public © 2012 Cisco and/or its affiliates. All rights reserved. 1 Cisco Expo Cisco Expo 2012 Architektura Carrier Routing System T-SP4 David Jakl Cisco

Transcript of Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All...

Page 1: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved. 1Cisco Expo

Cisco Expo

2012

ArchitekturaCarrier Routing SystemT-SP4

David Jakl – Cisco

Page 2: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

2© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

• Overview

• Architecture

• Line Card

• Switch Fabric

• Multi-chassis

• Summary

Page 3: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

3© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Page 4: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

4© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Carrier Routing System 1

Multi-chassis

10.2 (8xLCC) 92 Tbps (72xLCC)

4-slot

320Gbps8-slot

640Gbps

16-slot

1280Gbps

Core, Peering, L3 (L2) Edge Router

40Gbps/slot Full Duplex

Redundant 3-stage non-blocking Beneš Switch Fabric

IOS XR - Highly modular, Microkernel-based

SDR - Secure Domain Routers

IPoDWDM 10GE/40G, Tunable, G.709, (E)FEC,

40G over 10G DWDM System (single lambda)

Control Plane:

Redundant Route Processors (RP PRP)

More RPs for scale (DPRs) for "Process Placement“ or SDR

PLIMs (Front):

Modular: SIP-800/6xSPA: 1/10GE, POS 155M->10G, E3, ATM

Fixed: 1/10GE, POS 2.5G->40G, 10GE/40G IPoDWDM

Services: CGSE = Carrier Grade Services Engine

Forwarding Line Cards (Back):

MSC40 – Core/Peering/Edge (4/8/16/MC)

2M routes, H-QoS 8000 queues/port, 2000 intfs

FP40 – Thin Core/Peering (4/8) +licenses

2M routes, 8 queues/port, 100 intfs

TODAY MAX

Page 5: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

5© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Carrier Routing System 3

Multi-chassis

35.8 (8xLCC) 322 Tbps (72xLCC)

4-slot

1.12Tbps8-slot

2.24Tbps

16-slot

4.48Tbps

140Gbps/slot Full Duplex 140G Switch Fabric (4/8/16/MC)

PLIMs (Front):

1x100GE Line Rate, CFP (L4)

14x10GE Line Rate, LAN/WAN PHY, XFP

20x10GE Oversubscribed 140G, LAN/WAN PHY, XFP

2012: 100GE OTN IPoDWDM PM-QPSK over 10G DWDM

System (single lambda)

Forwarding Line Cards (Back):

MSC140 – Edge (4/8/16/MC)

4M routes, H-QoS 64000 queues/port, 12000 intfs

FP140 – Core/Peering (4/8/16/MC) +licenses

1/4M routes, 8 queues/port, 250 intfs

LSP140 – MPLS Core (limited IP) (4/8/16/MC) +licenses

HW Ready: E-OAM, Video monitoring, Time stamping, SyncE

CRS-1 CRS-3 NO Chassis/Power/Cooling/RP upgrade

Switch Fabric upgrade: Hitless = Zero packet loss

40G & 140G Cards in the same chassis MAX

Powered by QuantumFlow Array

TODAY

Page 6: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

6© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

1x 100GE

14x 10GE

20x 10GE

CFP

CRS-3 #100GE LR #10GE LR #10GE OS

4-slot 4 56 80

8-slot 8 112 160

16-slot 16 224 320

MC 8 LCC 128 1 792 2 560

MC 72 LCC 1 152 16 128 23 040

Page 7: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

7© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

2012

• CRS-3 16-slot B2B (2+0) Multichassis

• 100GE SR10 CFP

• 100GE IPoDWDM PLIM

• 4x40GE OTN PLIM – CFP: SR4, LR4, FR

• Tunable DWDM XFP

• FlexPLIM – 6x10GE LAN/WAN/OTN + 4xSPA

2013

• CRS-3 8-slot B2B (2+0) Multichassis

• CGSE+ 80Gbps

• 400G/slot – Switch Fabrics, Line Cards, PLIMs

Subject to change

Page 8: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

8© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Page 9: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

9© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

RP (active)

PLIM F

A

B

R

I

C

S1

PLIM

PLIM

PLIM

MSC

MSC

MSC

MSC

M

I

D

P

L

A

N

E

PLIM

PLIM

PLIM

PLIM

MSC

MSC

MSC

MSC

M

I

D

P

L

A

N

E

F

A

B

R

I

C

S1

F

A

B

R

I

C

S2

F

A

B

R

I

C

S3

F

A

B

R

I

C

S3

PLIM

PLIM

PLIM

PLIM

MSC

MSC

MSC

MSC

PLIM

PLIM

PLIM

PLIM

MSC

MSC

MSC

MSC

M

I

D

P

L

A

N

E

M

I

D

P

L

A

N

E

RP (standby)

Packets

In

Packets

Out

M

I

D

P

L

A

N

E

M

I

D

P

L

A

N

E

M

I

D

P

L

A

N

E

M

I

D

P

L

A

N

E

PLIM – Physical Layer Interface Module

MSC – Modular Service Card

RP – Route Processor

Switch Fabric

- 4 or 8 redundant planes

- Multi-Chassis option

Page 10: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

10© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Switch Fabric – LC/LC & RP/LC Path

PLIM & MSC – Packet Forwarding

Fans and Controllers - Cooling

Air Intake and Exhaust - Cooling

Power Supplies – AC or DC

RP – System Master

Front View Rear View16 Slot 8 Slot 8 Slot 16 Slot

F F

A A

B B

F F

A A

B B

F

A

B

R

I

C

F

A

B

R

I

C

MSCMSC

MSCMSC

MSCMSC

PLIMPLIM

PLIMPLIMPLIMPLIM

CABLE MGMT

CABLE MGMT

AIR OUT

FAN TRAY

FAN TRAY

POWER SUPPLIES

AIR OUT

FAN TRAY

FAN TRAY

POWER

SUPPLIES

CABLE MGMT

POWER

SUPPLIES

AIR

INTAKE

FAN

CTRL

AIR INTAKE

CABLE MGMT

POWER SUPPLIES

D

R

PR R

P P

R R

P P

D

R

P

P

L

I

M

CRS-1

CRS-1

M

S

C

AIR INTAKE

POWER SUPPLIES

R

P PLIMM

S

C

R

P

F F F F

A A A A

B B B B

AIR OUT

4 Slot 4 Slot

Page 11: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

11© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Page 12: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

12© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

• 1 4 PLIM ASICs (PLAs) depending on card type

• Nominally 40 Gbps for CRS-1

• Oversubscription allowed when all Ethernet

• 96 Gbps aggregate bandwidth into PSE = no bandwidth bottleneck even when oversubscribed

PLIM PLAs BW to PSE (per PLA)

4xOC192 POS 2 48 Gbps

16xOC48 POS 4 24 Gbps

1xOC768 POS 1 96 Gbps

4/8xTenGE 2 48 Gbps

SIP-800 2 48 Gbps

PLIM

PLA

PLA

PSE

HW CEF TCAMM

I

D

P

L

A

N

E

Page 13: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

13© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

M

I

D

P

L

A

N

E Egress

PSE

TX Forwarding

& Features

Multicast

Replication

IngressQ

Input Queuing

Fabric Queuing

Segmentation to Cells

Shaping

2 FabricQs

Cell Reassembly &

Queuing into PSE

Ingress

PSE

RX Forwarding

& Features

EgressQ

Output

Queuing

CPU

F

A

B

R

I

C

S1 F

A

B

R

I

C

S2F

A

B

R

I

C

S3

M

I

D

P

L

A

N

E

P

L

I

M

Page 14: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

14© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

M

I

D

P

L

A

N

E

PSE

IngressQ

FabricQs

FQ

PSE

EgressQ

CPU

FQ

HW CEF TCAM

Shape Queues Fabric Dest Qs

Priority

Assured Forwarding

Assured Forwarding

Assured Forwarding

Assured Forwarding

Best Effort

Priority

Assured Forwarding

Best Effort

Priority

Best Effort

Priority

Assured Forwarding

Assured Forwarding

Best Effort

HP

BE

HP

BE

HP

BE

HP

BE

HW CEF TCAM

F

A

B

R

I

C

S1 F

A

B

R

I

C

S2F

A

B

R

I

C

S3

M

I

D

P

L

A

N

E

Cells

P

L

I

MHP

AF

BE

HP

AF

BE

HP

AF

BE

HP

AF

BE

Page 15: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

15© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

• 188 Parallel Processing Engines (PPE)

Independent operation, not pipelined

All PFEs can access forwarding resources

• Micro-coded for Service Flexibility

• Performs per packet operations

IPv4, IPv6, MPLS, and Multicast lookups

Statistics

ACLs and Netflow accounting

Policing, Marking and WRED

• Access to memories and TCAMs

• Adds 8-byte buffer headerCEF

TABLETCAM POLICER

Page 16: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

16© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

R

E

S

O

U

R

C

E

F

A

B

R

I

C

TLU

TCAM

STATS

POLICER

PLU

Packets

from

PLAs

Input

Stream

Interface

Packet

Distributor

Packet

Processing

Engine

Cluster – 12 PPEsInstruction Memory

Packet

Buffer

MUX to access Resource Fabric Resources

PPE

Page 17: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

17© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Assign Packet (Header) to a PPE

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

R

E

S

O

U

R

C

E

F

A

B

R

I

C

TLU

TCAM

STATS

POLICER

PLU

Packet

Buffer

Header

Rest of

Packet

Busy PPE

(Red)

Available PPEs

(Green)

Page 18: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

18© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Perform lookup and features using Resources

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

R

E

S

O

U

R

C

E

F

A

B

R

I

C

TLU

TCAM

STATS

POLICER

PLU

Packet

Buffer

Page 19: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

19© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Recombine header and tail and send to IngressQ

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

IMEM MUX

R

E

S

O

U

R

C

E

F

A

B

R

I

C

TLU

TCAM

STATS

POLICER

PLU

Packet

Buffer

Page 20: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

20© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Input Shaping Queues

Per Interface

HP & LP per class if configured (max 8k queues)

Fabric Destination Queues

HP & LP for every FabricQ in system

4 queues for every MSC in entire system

Queue determined by Ingress QoS or Fabric QoS

Segmentation of packets into cells

45 Gbps limit between Shape Queues and Fabric Destination Queues (140 Gbps in CRS-3)

Discard bitmap

IngressQ

Shape Queues Fabric Dest Qs

HP

BE

HP

BE

HP

BE

HP

BE

CellsP

S

E

F

A

B

R

I

C

S1

Page 21: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

21© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

136 byte cells with

12 byte header, 120 byte payload, 4 byte CRC

1 or 2 packets per cell

Packets must start on a 30 byte boundary

Packets sharing a cell must be same priority and cast

Entire cell travels over 1 fabric plane

Round Robin among 8 fabric planes (4 for 4 slot CRS)

Header Payload CRC

12b 30b 30b 30b 30b 4b

120 bytes

136 bytes

Page 22: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

22© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Cells reassembled into packets

Packets queued prior to PSE

Unicast queues

Per type of service (HP/AF/BE)

Per output interface

Multicast queues

Per type of service

Raw (to CPU) queues

8 queues

FQ0

INT0 HP

INT1 AF

INT0 BE

INT1 HP

INT0 AF

INT1 BE

MCAST BE

MCAST HP

MCAST AF

To CPU

FQ1

INT2 HP

INT3 AF

INT2 BE

INT3 HP

INT2 AF

INT3 BE

MCAST BE

MCAST HP

MCAST AF

To CPU

E

G

R

E

S

S

P

S

E

F

A

B

R

I

C

S3

Page 23: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

23© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Drop on Ingress when a FabricQ queue is congested

M

I

D

P

L

A

N

E

PSE

IngressQ

FabricQs

FQ

PSE

EgressQ

CPU

FQ

HW CEF TCAM

Shape Queues Fabric Dest Qs

Priority

Assured Forwarding

Assured Forwarding

Assured Forwarding

Assured Forwarding

Best Effort

Priority

Assured Forwarding

Best Effort

Priority

Best Effort

Priority

Assured Forwarding

Assured Forwarding

Best Effort

HP

BE

HP

BE

HP

BE

HP

BE

HW CEF TCAM

F

A

B

R

I

C

S1 F

A

B

R

I

C

S2F

A

B

R

I

C

S3

M

I

D

P

L

A

N

E

Cells

P

L

I

MHP

AF

BE

HP

AF

BE

HP

AF

BE

HP

AF

BE

?

Page 24: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

24© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Identical hardware to Ingress PSE

Performs lookup for output adjacency

Performs output features: Policing, Marking, WRED, Netflow, ACLs, Statistics, …

HW multicast replication for multiple ports on same line card

CEFTABLE

TCAM POLICER

PSE

IngressQ

FabQ

PSE

EgressQ

FABRIC

Page 25: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

25© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Ingress PSE selects• Output Line Card

• Output Interface

• Fabric Dest Q (HP/LP)

• FabricQ (FQ0 or FQ1)

• IngressQ Queue

•HP, AF, or BE

Egress PSE selects• Output Interface

• Output Queue

• Adjacency

• Dest MAC address

PSE

IngressQ

FQ

PSE

EgressQ

CPU

FQ

HW CEF TCAM

Shape Queues Fabric Dest Qs

Priority

Assured Forwarding

Assured Forwarding

Assured Forwarding

Assured Forwarding

Best Effort

Priority

Assured Forwarding

Best Effort

Priority

Best Effort

Priority

Assured Forwarding

Assured Forwarding

Best Effort

HP

AF

BE

HP

AF

BE

HP

AF

BE

HP

AF

BE

HP

BE

HP

BE

HP

BE

HP

BE

HW CEF TCAM

Cells

FabricQs

Page 26: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

26© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

HW Replication within fabric planes

For cells going to multiple line cards

No performance impact for additional replications

HW Replication on Egress PSE

For multiple ports on a line card

Efficient scale for high fan-out

No increase in load on MSC

1 plane of 8 (4) shown

F

A

B

R

I

C

RP

MSC

MSC

MSC

MSC

RP

Page 27: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

27© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

• Per interface/sub-interface queuing

• 8k queues

• 1GB packet buffer

• P2PMDRR

• MQC configurationStrict HP queue

Bandwidth guarantees

Shaping

Bandwidth remaining

• 3 Level HierarchyPort – Highest Level Queuing Engine

Group – Middle Level Queuing Engine

Queue – Lowest Level Queuing Engine

EgressQ

Priority

Assured Forwarding

Assured Forwarding

Assured Forwarding

Assured Forwarding

Best Effort

Priority

Assured Forwarding

Best Effort

Priority

Best Effort

Priority

Assured Forwarding

Assured Forwarding

Best Effort

P

S

E

P

L

I

M

Page 28: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

28© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

M

I

D

P

L

A

N

E FabQsEgressQ

Accel

FPGA

Accel

FPGA

PLA

iPSE

ePSE

IngressQ M

I

D

P

L

A

N

E

F

A

B

R

I

C

Modular Services CardService Engine PLIM

Octeon CPUs Same PLA as SIP-800

Page 29: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

29© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

M

I

D

P

L

A

N

E

FabQsEgressQ

Accel

FPGA

Accel

FPGA

PLA

iPSE

ePSE

IngressQ

M

I

D

P

L

A

N

E

F

A

B

R

I

C

FabQsEgressQ

PLAiPSE

ePSE

IngressQ

PLA

SPA

SPA

SPA

SPA

SPA

SPA

Page 30: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

30© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

from PLIM

120G2x80G

PSE 160G

160GEgressQ PSE

100G

100G

Intel CPUSub-system

141G

113G

113GFabricQ

FabricQ

IngressQ

Output Queuing Output FeaturesMulticast Replication Cell Reassembly

Forwarding LookupInput Features

Queuing for FabricCell Segmentation Input Shaping

toPLIM

to Fabric

from Fabric

160G2x100G

Page 31: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

31© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Page 32: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

32© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

1 of 8

2 of 8

8 of 8

1

2

8

1

2

16

Line Card Line Card

S1 S2 S3

S1 S2 S3

S1 S2 S3

3 stage switching fabric

High and low priority cells

Set on control by default

Set on transit pkts via CLI

Vital Bit for IPC

40 Gbps 100 Gbps

8 independent fabric planes (4 on CRS-1/4)

2.5x speedup through fabric

Support for 72 chassis & 1296 RP/MSC clients

Page 33: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

33© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

S2 S3S1

Send to any S2. No need to look at header

Look at cell header. Send to S3 based on chassis/LC

Look at cell header. Send to specific LC and FabricQ

Page 34: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

34© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Full View of 1 Plane

S1

S2S1

S2

S3

Fabric Plane (1 of 8)Upper shelf

Lower shelf

Ingress LC& DRP x 8

RP X 2

S3

S3

S3

= 18 x 2.5G links

Upper shelf

Lower shelf

Ingress LC& DPR x 8

Egress LC& DRP x 8

Egress LC& DPR x 8

= 1 x 2.5G links

RP X 2

Lower shelf

Fabric speedup from S2 to S3

(16 LCs x 4 links) + (2 RPs x 2 links) (16 LCs x 4 links) + (2 RPs x 2 links)

Ing

res

sQ

Ing

ressQ

FQ

FQ

FQ

FQ

IQ FQ

Page 35: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

35© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Each destination and HP/LP queued separately

S3S2

S1

S1

S2

Control

Data

LC0

LC1

LC2

LC4

LC3

LC5

LC6

LC7

LC8

LC9

LC10

LC12

LC11

LC13

LC14

LC15

RP1

RP0

Control

Data S3

LC0

LC1

LC2

LC4

LC3

LC5

LC6

LC7

LC8

LC9

LC10

LC12

LC11

LC13

LC14

LC15

RP1

RP0

LC0

LC1

LC2

LC3

LC4

LC5

LC6

LC7

LC8

LC9

RP1

LC11

LC12

LC13

LC14

LC15

LC10RP0

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

mcast-High

mcast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

mcast-High

mcast-Low

mcast-High

mcast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

mcast-High

mcast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Control

ucast-High

ucast-Low

Page 36: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

36© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

• CRS-1 provides efficient multicast replication via 3 operationsS2 can replicate cells to registered (via FGID) S3 SEAs

S3 can replicate cells to registered (via FGID) FabricQs

Egress PSE can replicate packets for each output port

• 1 million Fabric Group IDsProgram fabric for mcast

• S2 and S3 lookup FGIDS2 S3S1

S1 switches to all S2s

Mcast replication at S2based on FGID

Replication at S3based on FGID

Page 37: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

37© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Page 38: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

38© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

1 of 8

2 of 8

8 of 8

1

2

8

1

2

16

40 Gbps

Line Card Line Card

40 Gbps

136 Bytes cellsFabric Chassis

100 Gbps

(2.5X Speedup)

S1 S2 S3

S1 S2 S3

S1 S2 S3

S2 stage moves into Fabric Card Chassis (FCC)

Logical operation of fabric remains the same

Page 39: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

39© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

S3

S1

S2

S3

S2 Stage inFabric Card Chassis

S1 and S3 Stages in Line Card Chassis

Logical decisions same

as standalone system

LCC #1

LCC #2

Page 40: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

40© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

POWERSUPPLIES

AIRINTAKE

SWITCHFABRIC

SHELF

CTRL

FAN TRAY

FAN TRAY

SWITCHFABRIC

SHELF

CTRL

CRS

Fabric ChassisFront View

LEDS

LEDS

POWERSUPPLIES

FIBEROIMS

FIBEROIMS

AIROUT

CABLE MGMT

Line CardChassis

Fabric ChassisRear View

Page 41: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

41© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

• Single to Multi-chassis upgrade w/o packet loss

Moves fabric stage 2 to separate chassis

Upgrade LCC fabric cards

Add fabric card chassis (FCC)

Connect control GE

Connect LCC and FCC with fiber bundles

FCC OpticalConnections

72 Fiber Bundle

Optical Array Cable (100m)

Page 42: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

42© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

FCCLCC 0 LCC 1

8 S2 boards & OIMs

8 S13 boards

Page 43: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

43© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Page 44: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

44© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

2 3 41

6 7 85

2 3 41

2 3 41

6 7 85

6 7 85

1 2 21

1 2 21

3 4 43

3 4 43

5 6 65

5 6 65

7 8 87

7 8 87

5 5 55

5 5 55

6 6 66

6 6 66

7 7 77

7 7 77

8 8 88

8 8 88

1 1 11

1 1 11

2 2 22

2 2 22

3 3 33

3 3 33

4 4 44

4 4 44

1 FCCMax 9 LCC

2 FCCMax 18 LCC

4 FCCMax 36 LCC

8 FCCMax 72 LCC

Page 45: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

45© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Page 46: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

46© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

M

I

D

P

L

A

N

E

PSE

IngressQ

FabricQs

FQ

PSE

EgressQ

CPU

FQ

HW CEF TCAM

Shape Queues Fabric Dest Qs

Priority

Assured Forwarding

Assured Forwarding

Assured Forwarding

Assured Forwarding

Best Effort

Priority

Assured Forwarding

Best Effort

Priority

Best Effort

Priority

Assured Forwarding

Assured Forwarding

Best Effort

HP

BE

HP

BE

HP

BE

HP

BE

HW CEF TCAM

F

A

B

R

I

C

S1 F

A

B

R

I

C

S2F

A

B

R

I

C

S3

M

I

D

P

L

A

N

E

Cells

P

L

I

MHP

AF

BE

HP

AF

BE

HP

AF

BE

HP

AF

BE

?

Page 47: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

47© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

M

I

D

P

L

A

N

E

PSE

IngressQ

FabricQs

FQ

PSE

EgressQ

CPU

FQ

HW CEF TCAM

Shape Queues Fabric Dest Qs

Priority

Assured Forwarding

Assured Forwarding

Assured Forwarding

Assured Forwarding

Best Effort

Priority

Assured Forwarding

Best Effort

Priority

Best Effort

Priority

Assured Forwarding

Assured Forwarding

Best Effort

HP

BE

HP

BE

HP

BE

HP

BE

HW CEF TCAM

F

A

B

R

I

C

S1 F

A

B

R

I

C

S2F

A

B

R

I

C

S3

M

I

D

P

L

A

N

E

Cells

P

L

I

MHP

AF

BE

HP

AF

BE

HP

AF

BE

HP

AF

BE

More PPEsFaster PPEs

PLIMQoS

140GPLIMs

More memoryHigher scale

More Queues8K -> 64K

Faster CPU

Page 48: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

48© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

Rea

sse

mbly

FabricQ

Rea

sse

mbly

FabricQ

S1 S2 S3

S2 Queues per priority per fabric

group

S3 Queues per priority per fabric

destination

WRED

PSE

Dis

ca

rd F

ilte

r

8k shaped Queues

…..

IngressQ

Fab

ric D

estina

tio

n B

P

3072 High priority fabric

destination queues

8k shaped Queues

…..

EgressQWRED

PSE

3072 Low priority fabric destination

queues512 raw queues in fabricQ; .5GB total per fabricQ

64K queues, 16K groups;

1GB total

64k Input rate shaping

queues; 1GB total

PLA

2 queues/port; 75MB total

PLA

~110 MTUs

Page 49: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System

49© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo Cisco Public© 2012 Cisco and/or its affiliates. All rights reserved.Cisco Expo

#1 The Fastest Industry Leading Core Router

Carrier Class Architecture

• Fully Modular

• Scalable

• Reliable

• Predictable

• Backward compatible

Page 51: Architektura Carrier Routing System - Cisco · Cisco Expo © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 1 Cisco Expo 2012 Architektura Carrier Routing System