Page 1
EE384M Switch Design
Tom Edsall
Page 2
Bus Architecture
MAC MAC MAC MAC MAC MAC
FWD
Page 3
MACMACMAC
Bus Architecture
MAC MACMACMACMACMACMAC MACMAC
MACMACMACMAC MACMAC
FWD
Page 4
Bus ArchitectureMACMACMAC
MACMACMAC
MACMACMAC
MACMACMAC
MACMACMAC
MACMACMAC
FWD
Page 5
Cross-bar ArchitectureMACMACMAC
MACMACMAC
MACMACMAC
MACMACMAC
MACMACMAC
MACMACMAC
FWD
Page 6
Cross-Bar Model
FWD
FWD
FWD
MACMACMAC
MACMACMAC
MACMACMAC
MACMACMAC
MACMACMAC
MACMACMAC
Page 7
Performance Issue with Cross-Bars
Source: M. J. Karol, M.G. Hluchyj, S. P. Morgan, “Input Versus Output Queueing [sic] on a Space-Division Packet Switch”, IEEE Transactions on Communications, Vol COM-35, No 12, December 1987, page 1353
58.6%
Page 8
3X Speed-up in cross-bar
ClassicCross-Bar2.16Tbps
FIFO 1
FIFO 2
FIFO 3
FIFO 18
20 GbpsEach
60 GbpsEach
FIFO 1
FIFO 2
FIFO 3
FIFO 18
60 GbpsEach
20 GbpsEach
Page 9
Cross-Bar Model
FWD
FWD
FWD
MACMACMAC
MACMACMAC
MACMACMAC
MACMACMAC
MACMACMAC
MACMACMAC
Page 11
Head of Line Blocking
Stadium
Beer/Soda/Chips
Kwiky Mart
Page 12
Stadium
Output Queuing
Beer/Soda/Chips
Kwiky Mart
Page 13
Head of Line Blocking
BCACB
A
B
C
Page 14
Head of Line Blocking
BCACBCAB
A
B
C
Page 15
Head of Line Blocking
CB CBCABCB A
A
B
C
Page 16
A
B
C
VOQ—Virtual Output Queues
BCACB
ARB
Page 17
VOQ—Virtual Output Queues
B
C
AA
A
B
C
ARB
CBCAB
Page 18
VOQ—Virtual Output Queues
B
C
A
BACCB
AAA
A
B
C
ARB
Page 21
Completing the Swtich
FWD
FWD
FWD
ARB
Page 22
Completing the Swtich
ARB
FWDMAC
FWDMAC
FWDMAC
MAC
MAC
MAC
Page 23
Completing the Swtich
Port
ARB
FWDMAC
FWDMAC
FWDMAC
MAC
MAC
MAC
Page 24
Re-Draw
MAC MAC MAC
ARB
FWD FWD FWD
Page 25
MDS 9500 ImplementationEth
FlashCard
Console
Cross-Bar
Supervisor/Fabric
Back Plane
Arb
I/F uP
M M M
F
Q
F
Q
F
Q
I/F uP
M M M
F
Q
F
Q
F
Q
I/F uP
M M M
F
Q
F
Q
F
Q
I/F uP
M M M
F
Q
F
Q
F
Q
uP
Page 26
ARB
FWDMAC
FWDMAC
FWDMAC
MAC
MAC
MAC
Page 27
ARBREQ
GNT
Credit
Packet
Page 28
Failures
• Arbitration– Arbiter logic• duplicate arbiters
– Message failure• Cross-bar– duplicate x-bars– load-balance
Page 29
Scaling further
Linecard Linecard Linecard Linecard Linecard
ARB ARB
Page 30
Scaling further
Linecard Linecard Linecard Linecard Linecard
ARB ARB
Page 31
ARBREQ
GNT
Credit
Packet
Packet Spraying
Page 32
Other Algorithms
• Table lookups• Forwarding• Output Interface (OIF) Lists• Packet Buffering/Queuing• Arbitration between queues• Load balancing• Parsing• Link aggregation