1

Carrier Routing System (CRS) Multi-Chassis

Hitesh Kumar CCIE SP (#38757)

HTTS

Rahul Rammanohar CCIE R&S, SP (#13015)

HTTS

Vinay Kumar CCIE R&S (#35210)

HTTS

Cisco Support Community PresentsTech-Talk Series

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 2

CRS Chassis Scaling OptionsStandalone Chassis Back to Back (2+0) Multi-Chassis (X+1, X+2, X+4)

X is from 3 to 9

CRS-16/S, CRS-16/S-B

CRS-8/S, CRS-8/S-B

CRS-4/S

CRS-16/S, CRS-16/S-B

CRS-8/S, CRS-8/S-B

CRS-16/S, CRS-16/S-B

CRS1 - 1.12TbpsCRS3 - 4.48Tbps

CRS1 - 620GbpsCRS3 - 2.24Tbps

CRS1 - 320GbpsCRS3 - 1.12Tbps

CRS1 - 1.12TbpsCRS3 - 4.48Tbps

CRS1 - 2.24TbpsCRS3 - 8.96Tbps

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 3

CRS-1 Fabric Hardware Architecture

. .

. .

.Line Card

0-71

2

8

Line Card8-15

12

8

. . .

Ingress --> 8 (planes) * 4 links * 2.5G * 8b/10b (encoding) * 120/136 (overhead) = 56 Gbps

Egress <-- 8 (planes) * 4 links * 2.5G * 8b/10b (encoding) * 120/136 (overhead) * 2 S3 ASICs = 112 Gbps

S2

S3

S1S3

S3

S2S1

S3

8 of 8

S2

S3

S1S3

S3

S2S1

S3

2 of 8

S2

S3

S1S3

S3

S2S1

S3

1 of 8

. . .

Route Processor1

2

8

. . .

2 connections to each plane to lower S1 only.

4 connections of 2.5G to each plane.

18 Links18 Links

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 4

S2

S3S1

S2

S1 S3

8 of 8

S2

CRS-3 Fabric Hardware Architecture

. .

. .

.

Line Card1

2

8

Line Card1

2

8

. . .

. . .

Route Processor1

2

8

. . .

S2

S3S1

S2

S1 S3

2 of 8

S2

S2

S3S1

S2

S1 S3

1 of 8

S2

5 connections of 5G to each plane.

Ingress --> 8 (planes) * 5 links * 5G * 8b/10b (encoding) * 120/136 (overhead) = 141 Gbps

Egress <-- 8 (planes) * 8 links * 5G * 8b/10b (encoding) * 120/136 (overhead) * 2 S3 ASICs = 225 Gbps

12 Links 24 Links

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 5

S2 Fabric CardS13 Fabric CardMSC Card

IngressQ

FabricQ

CRS-3 Multi-chassis Hardware Architecture

S2

S2

S1

S1

S3

S3

S2

Fabric Chassis

OIM LED ModuleLine Card Chassis

Array Cables

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 6

S13 Fabric Card MSC Card

IngressQ

FabricQ

Line Card Chassis

S13 Fabric CardMSC Card

IngressQ

FabricQ

CRS-3 Back to Back Hardware Architecture

S1

S1

S3

S3

S2

S2

S2

Line Card ChassisBack to Back

Cables

S1

S1

S3

S3

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 7

Control Ethernet

Inter-rack

Inter-rack

SC0

SC1

RP0

RP1

RP0

RP1Forwarding Links

Blocked Links

LCC0

LCC1

FCC0

Intra-rack

Intra-rack

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 8

Fabric Plane Configuration

Multi-Module Topology

• Three Plane Configuration

• Three S2 SFC’s are used to create a plane The array cables for each fabric plane are connected to all THREE cards

• Each S13 card is connected to 3 different S2 cards

To LC rack # 0

To LC rack # 2

To LC rack # 8

To LC rack # 6

Single Module Topology• Full Plane Configuration

• Each S2 SFC serves one plane• The array cables for each fabric plane are

connected to a ONE SFC

• Each S13 card is connected to a single S2 card

To LC rack # 1

To LC rack # 0

Switch Fabric Card (SFC)

SwitchFabric Cards

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9

Config Plane Topology Configuration

Planes are not tied to specific slots in fabric rack

Admin-level config defines the slots for each plane

uses (and how big the plane is)controllers fabric plane 0

oim count 1

oim width 1

oim instance 0 location F0/SM0/FM

Count 1- All cables in plane connect to the same OIM.

Position of 1st card within the plane in fabric rack.Plane 0 uses rack F0 slot SM0.

controllers fabric plane 1oim count 3oim instance 0 location F0/SM8/FMoim instance 1 location F0/SM7/FMoim instance 2 location F0/SM6/FM

Count 3 - The cables from each LCC for that plane connect to 3 different OIMs.

3 different cards makes one plane. Plane 1 uses rack F0 slot SM6, SM7 and SM8.

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10

Configuration Rack numbers

RP/0/RP0/CPU0:CRS3#admin show run | i dscBuilding configuration...dsc serial TBA10080000 rack 1dsc serial TBA10090001 rack 0dsc serial TBA10120000 rack f0

Assign a rack number to the S/N of each chassis. Serial numbers can be obtained from,- ‘show diag chassis’- rommon with ‘dumpplaneeeprom’- rear of the FCC and front of the LCC

• Rack 0 or 1 = dSC (also an LCC)• Rack 1 -> 127 = LCC• Rack f0 -> f4 = FCCNote: Rack numbers have to be unique

Example:

Thank you.