Post on 20-Mar-2020
QFABRIC TECHNOLOGY OVERVIEW AND HPC WORKLOAD EXPERIENCE
2 Copyright © 2013 Juniper Networks, Inc. www.juniper.net
SYSTEMS PERFORMANCE TRENDS
‘88 ‘89 ‘90 ‘91 ‘92 ‘93 ‘94 ‘95 ‘96 ‘97 ‘98 ‘99 ‘00 ‘01 ‘02 ‘03 ‘04 ‘05 ‘06 ‘07 ‘08
22
20
24
26
28
210
212
214
216
218
220
Meg
aher
tz /
MFl
ops
222
224
System CAGR: 1.9x /year
Microprocessor CAGR: 1.3x /year
Super Computers
226
228 500,000 X over 20 years
‘88 ‘89 ‘90 ‘91 ‘92 ‘93 ‘94 ‘95 ‘96 ‘97 ‘98 ‘99 ‘00 ‘01 ‘02 ‘03 ‘04 ‘05 ‘06 ‘07 ‘0822
24
26
28
210
212
214
216
218
220
222
M B
ytes
224
226
228
230
232
System CAGR: 2.4x /year
Disk CAGR: 1.71x /year
16,000,000 X over 20 years
22
20
24
26
28
210
212
214
216
218
220
‘88 ‘89 ‘90 ‘91 ‘92 ‘93 ‘94 ‘95 ‘96 ‘97 ‘98 ‘99 ‘00 ‘01 ‘02 ‘03 ‘04 ‘05 ‘06 ‘07 ‘08
Meg
abits
per
sec
ond
222
224
M40
M160
T640
TX
Pre-ASIC era: 1.6x /year
Post-ASIC era: 2.2x /year
Interface CAGR: 1.7x /year
1000,000 X over 20 years (2x /year)
T1600
Computer Performance: 1988-2008 Storage Capacity: 1988-2008
Router Performance 1988 – 2008
3 Copyright © 2013 Juniper Networks, Inc. www.juniper.net
QFABRIC ARCHITECTURENETWORK PERFORMANCE ACCELERATION
StorageNodes
ComputeNodes
Core Routers
SecurityServices
High speed interfacesAny to any connectivity
Non blocking & lossless fabric
4 Copyright © 2013 Juniper Networks, Inc. www.juniper.net
40 GB
LEGEND
1 GB
QFabric Interconnect
Node #1 Node #3Node #2 Node #16
QFabric Director
QFABRIC DESIGN BLOCKS
…Line Card 1
Fabric FabricFabricFabricFabric FabricFabricFabric
Line Card 15
RE0RE0
RE1RE1
Route Engine (RE)
Linecards
Fabric
QFabric InterconnectQFabric Director
QFabric Nodes
5 Copyright © 2013 Juniper Networks, Inc. www.juniper.net
QFABRIC EVOLVING THE SINGLE SWITCH MODEL
Chassis Switch
Interconnect
Node
QFabric - Distributed System
Director
I/O Modules
I/O Modules
FabricFabricRoute EngineRoute Engine
6 Copyright © 2013 Juniper Networks, Inc. www.juniper.net
QFABRIC FUNCTIONAL ELEMENTS
Interconnect
Fabric switches
Node
I/O devices
Director
Control devices
40G Interface 1Tbps/slot 40Tb Fabric <5µSec latency Single UI
7 Copyright © 2013 Juniper Networks, Inc. www.juniper.net
Director
QFABRIC: SEPARATE CONTROL AND DATA
ControlPlane1
GbE network connectedTo EX4200
ControlPlane 2
GbE network connectedTo EX4200
8 Copyright © 2013 Juniper Networks, Inc. www.juniper.net
Interconnect
Solution for 6,000 10GE ports
1GE connections to the Director cluster
40 Gig uplink from each Node to Interconnect
Node #1 Node #128Node #3Node #2
QFABRIC CONFIGURATION FOR LARGE DEPLOYMENT
Control Plane Control Plane
Director
9 Copyright © 2013 Juniper Networks, Inc. www.juniper.net
POWER & ENERGY CHALLENGESDemand for network capacity & performance will continue to grow
Power usage ratio for networking in DC needs to be sustained
Today’s DC network design approach is not energy efficient
Servers NAS
Hot spots
10 Copyright © 2013 Juniper Networks, Inc. www.juniper.net
ENERGY EFFICIENCY IN NETWORK DESIGN
Servers NAS
Hot spots
Servers NAS
11 Copyright © 2013 Juniper Networks, Inc. www.juniper.net
NETWORK PERFORMANCE WITH QFABRIC
Scale
Latency
Traditionaldesign
Juniper
>100 µSec
<5 µSec
12 Copyright © 2013 Juniper Networks, Inc. www.juniper.net
Node #2 QFabric Nodes
QFabricInterconnect
QFABRIC: L2 FORWARDING
Interconnect #1 Interconnect #2
DA-Y SA-X Payload
FH DA-Y SA-X Payload
DA-Y SA-X Payload
MAC Node
X #1
Y #2
Z #340 GB
LEGEND
1 GB
VLANs22 33 44
Node #1 Node #3 Node #4
VLAN 2VLAN 2 22 4422 33 33 44 33
Network Node GroupServer Node Groups
Host A MAC: X VLAN: 2
Host B MAC: YVLAN: 2
Host C MAC: ZVLAN: 4
MAC Node
X Local
MAC Node
X #1
MAC Node MAC Node
X #1
Y #2 Y Local Y #2
Z Local
Z #3
QFabric Director
13 Copyright © 2013 Juniper Networks, Inc. www.juniper.net
Node #2 QFabric Nodes
QFabricInterconnect
QFABRIC: L3 FORWARDING
Interconnect #1 Interconnect #2IP Node
X #1
Z #3 40 GB
LEGEND
1 GB
VLANs22 33 44
Node #1 Node #3 Node #4
VLAN 2VLAN 2 22 4422 33 33 44 33
Network Node GroupServer Node Groups
Host A MAC: X VLAN: 2IP: X
Host B MAC: YVLAN: 2IP: Y
Host C MAC: ZVLAN: 4IP: Z
IP Node
X Local
IP Node
X #1
IP Node IP Node
X #1
Z #3 Z #3
Z Local
Z #3
DIP-Z SIP-X Payload
FH DIP-Z SIP-X Payload
DIP-Z SIP-X Payload
QFabric Director
14 Copyright © 2013 Juniper Networks, Inc. www.juniper.net
QFABRIC & LAYER-2 EXTENSION DEMONSTRATIONMPLS Backbone Data Center 2Data Center 1
Workload10GbE
MX-80
T-4000
MC-LAG10GbE/40GbE
MPLS/VPLS100GbE
MX-240
Workload10GbE
Spirent
Spirent
Scales to 6,144 10GbE or 1,536 40GbE< 5 µSec latency
100GbE end to endMulti-point Layer-2/3
ORNL Booth
MX-240