Inspur OAI Reference System
Transcript of Inspur OAI Reference System
Inspur OAI Reference System
YJ Huang, Development Manager, Inspur
OAI
UBB Common Tray
OAM with Heatsink
Inspur Fab2 System
Compute Node: ON5263M5
UBB reference System: MX1
High DensityWhisper Cable
三視圖
141mm
Inspur System Overview
Front View
Rear View
Front IO connector QSFP-DD Connector for OAM Expansion
Whisper ConnectorHHHL PCIe Expansion
54V BUSBAR Clips 9256 Fan
Product model: MX1
Chassis 21” 3OU Rack mount
Dimensions 537W*141H*803D (mm)
Connection with Compute node Up to PCIe Gen4 x32
OAMSupport Max 8pcs 48~54V OAM(up to 450W each);Support Max 8pcs 12V OAM (up to 350W each)
Power without OAM 1570W
PCIe Switch Support PCIe Gen4 (100lanes/chip)
PCIe re-timer Support PCIe Gen4 x16
Phy re-timer 56Gbps PAM-4 or 10/28Gbps NRZ x16
Expansion slots Up to 4 x PCle Gen4 x16 low profile standard card
BMC AST2520
I/OUSB port for UART debug, RJ45 port for BMC dedicated, UID, PWR Button
Ambient Working Temperature 5-35 ℃
Inspur OAI System Features
System Explode View Top cover
PDB
HIBUBB
9256 Fan
Side cover
FIO
UBB Common Tray
Side cover
Air Duct
System Block DiagramCompute
NodeHIB
BMC AST2520
(Management)
PCIe Switch PM42100
PCIe Switch PM42100
PCIe Switch PM42100
PCIe Switch PM42100
CPLD
IB Module
UBB
PCIeRe-timerPT4161L
PCIeRe-timerPT4161L
PCIeRe-timerPT4161L
PCIeRe-timerPT4161L
PCIeRe-timerPT4161L
PCIeRe-timerPT4161L
PCIeRe-timerPT4161L
PCIeRe-timerPT4161L
OAM0 OAM1 OAM2 OAM3 OAM4 OAM5 OAM6 OAM7
QS
FP-
DD
QS
FP-
DD
QS
FP-
DD
QS
FP-
DD
QS
FP-
DD
QS
FP-
DD
QS
FP-
DD
QS
FP-
DD
IB Module
Signal symbol Signal Type
PCIe x16
Management
Serdes
I2C
PDB
Power Monitor
I2C
I2C
54V to 12VVR x6
54V HSC x9
I2C/JTAG
CPLD
CPLD
PhyRe-timer
CRT50216P
PhyRe-timer
CRT50216P
PhyRe-timer
CRT50216P
PhyRe-timer
CRT50216P
System Placement
OAM6
OAM5
OAM2
OAM1
OAM7
OAM4
OAM3
OAM0
UBB HIB PDB
9256 Fan x4
Power Clip
QSFP-DD x8
Whisper conn. to IB
Whisper conn. to Compute node
PCIe x16 Conn. x4
Whisper Cable
FIO
IB Card
Universal Baseboard Placement
585mm
417mm
QSFP-DD Connector Amphenol UE36-A 1070-3000T
QSFP-DD Cage Amphenol UE36-B 16221-06A5A
12V Connector Amphenol JX410-513xx
54V Connector Amphenol 10028917-001LF
Guide Pin Amphenol 10037909-101LF
High Density Connector
Amphenol 10131762-101LF
Mezzanine Connector Molex 2093111115
High-speed Interface Board Placement
528mm
188.4mm
12V Connector Amphenol JX412-50432
54V Connector Amphenol 10052620-5544P00LF
Guide Pin Amphenol 10044314-101LFC
High Density Connector
Amphenol 10124558-101LF
Whisper Conn. to compute node
Whisper Conn. to IB
Power Conn. to PDB
System Access Solution – UBB tray
Press button to release lever handle.
Then pull out UBB tray from system.
OAM Heatsink Design For Common Tray✓Add top cover & HS module handle for accessing✓Reduce fin numbers to meet common UBB tray specification
Preliminary version Current version
Fin area height63.3mm
Fin area height 66.1mm
OAM x4HS: detail model
Switch x4
OAM x4HS: compact model
VR x6
QSFP-DD x8
Vent: 40%
Power Deliver Board PlacementThermal Design Power
Component Unit Power Q’ty Power (W)
OAM 450 8 3600
Re-timer 9.6 8 76.8
Switch 44 4 176
PHY 18.7 4 74.8
QSFP-DD 12.5 8 100
CX5 18.1 4 72.4
VR 25 6 150
9256 Fan 140 4 560
Total Power (watt) 4810
Re-timer x8
PHY x4
Top view
CX5 x4
Simulation Results (Temperature Field)Simulation Boundary Condition Setup
✓ TDP at 35C ambient
✓ Fan normal
✓ Sea-level
Cut plane: y=17.5mm
Airflow
Simulation Result
Location Spec (C) Temp.
OAMASIC 115.0 76.5
HBM 95.0 93.3
PCIe Re-timer 85.0 79.5
PCIe Switch 115.0 90.8
PHY Re-timer 125.0 78.7
QSFP-DD 70.0 66.9
CX5 105.0 94.1
VR 125.0 108.7
✓ Compact model for upper stream OAM HS
✓ Compact model HS for VR
✓ Symmetric placement for switch HS two for
detail model; two for compact model
Inspur 2OU Compute Node
Re-timer card with Whisper connector
2.5” HDDx4
8x10 Whisper Cable
Hea
d n
od
e 1
Hea
d n
od
e 2
54V->12VVR
54V->12VVR
Compute Node Explode View
Rack Level Solution
Cable symbol Cable Type
Whisper Cable (PCIe)
Eth cable
QSFP-DD (Serdes)
48VDC Open Rack
✓ 1 pairs 48V Bus Bar
✓ 1 shelf per Rack
Power Shelf
✓ 33KW(12xPSU)
✓ 40V-58V
✓ 93mm (H, 2OU) x 537mm (W, 21”) x 586 (D) mm
System Devices
✓ Inspur 3OU OAI systems x4
✓ Inspur 2OU compute node x4
100G Switch
OAI system
Compute node x2
OAI system
Power Shelf
OAI system
Compute node x2
OAI system
OAM Vendor
✓ Habana Gaudi
✓ Nvidia
✓ AMD
…etc
System Power Subtotal
Inspur OAI system20kW
(5kW x4)22.8kW
Inspur Compute Node2.8kW
(700W x4)
Total Power/Rack 33kW
1G Mgmt Switch
Call to Action
• Power on sample is already available @TWN office• Have Inspur reference system sample review in Experience lab
online booth• Project Wiki with the latest specification :
https://www.opencompute.org/wiki/Server/OAI• Mailing list:
https://ocp-all.groups.io/g/OCP-OAIV