Sun sparc enterprise t5120 and t5220 servers technical presentation
-
Upload
xkinanx -
Category
Technology
-
view
791 -
download
4
description
Transcript of Sun sparc enterprise t5120 and t5220 servers technical presentation
1
Sun SPARC Enterprise T5120 and T5220 Rack Server TTTPeter A. WilsonTechnical MarketingSun Microsystems, Inc.
Sun Internal and Approved Partners Only 2
Agenda• Introduction• Overview of UltraSPARC T2• Sun SPARC Enterprise T5x20 servers
> Comparison to Sun Fire T1000/T2000> System Architecture and Tours> Memory, Networking and IO Expansion> Disks, Fans and Power Supplies> Solaris for T5x20
• ILOM 2.0• LDOMs for T5x20 • CRUs/FRUs and Configuration Options• Tuning and Performance• Tools, Information and References• Q & A ...
Sun Internal and Approved Partners Only 3
CMT Processor StrategyFirst True System on a Chip
• Scaling performance with frequency is a dead-end
• Multi-core/Multi-thread processors can exploit application parallelism
• Able to Aggressively attack memory latency
• Provide rich availability feature set• Integrate networking, I/O and security
capability• Virtualize the architecture
Sun Internal and Approved Partners Only 4
Agenda• Introduction• Overview of UltraSPARC T2• Sun SPARC Enterprise T5x20 servers
> Comparison to Sun Fire T1000/T2000> System Architecture and Tours> Memory, Networking and IO Expansion> Disks, Fans and Power Supplies> Solaris for T5x20
• ILOM 2.0• LDOMs for T5x20 • CRUs/FRUs and Configuration Options• Tuning and Performance• Tools, Information and References• Q & A ...
Sun Internal and Approved Partners Only 5
UltraSPARC T2: TrueSystem On a Chip
• Up to 8 cores @1.2GHz or 1.4GHz• Up to 64 threads per CPU • Up to 16 FB-DIMMs, 4 memory
controllers> Up to 64GB memory (4GB DIMMs)> 2.5x memory BW of UltraSPARC T1
• 8 x fully pipelined Floating Point units / core, 1 per core
• Dual 10Gbit Ethernet and PCI-E integrated onto chip
• 4MB L2 (8 banks) 16 way associative
• Enhanced MAU/Security co-processor per core> DES, 3DES, AES, RC4, SHA1,
SHA256, MD5, RSA to 2048 key, ECC,CRC32
• Advanced Power saving features• 65nm process technology
x8 @2.5GHz4GBytes/s bi-directional
Full Cross BarC0 C1 C2 C3 C4 C5 C6 C7FPU FPU FPU FPU FPU FPU FPU FPU
L2$ L2$ L2$ L2$ L2$ L2$ L2$ L2$
FB DIMM FB DIMM FB DIMM FB DIMM
FB DIMM FB DIMM FB DIMM FB DIMM
PCI-ENIU(E-net+)
Sys I/FBuffer Switch Core
2x 10GE EthernetPower ~95W<1.5W/thread
MCU MCU MCU MCU
MAU MAUMAUMAUMAUMAUMAUMAU
DMA Sub-System
Sun Internal and Approved Partners Only 6
UltraSPARC T2 ProcessorL2 Data
Bank 0
SPARC
Core 0
SPARC
Core 1
SPARC
Core 5
SPARC
Core 4L2 Data
Bank 1
L2 Data
Bank 4
L2 Data
Bank 5
L2 Data
Bank 7
L2 Data
Bank 6
L2 Data
Bank 3
L2 Data
Bank 2
L2B0
L2B1
L2B2
L2B3
L2B5
L2B4
L2B6
L2B7
SPARC
Core 2
SPARC
Core 3
SPARC
Core 7
L2
TAG2
L2
TAG3
L2
TAG7
L2
TAG6
L2
TAG0
L2
TAG1
L2
TAG5
L2
TAG4MCU0
MCU1
MCU2
MCU3
DMU
PEU
RTX
RDP TDS
CCXSII
SIO
CCU
NC
U
EF
U
SPARC
Core 6
MACFSR
FSR
FSR
PSR ESR
For a video tour with Rick Hetherington, go to http://www.youtube.com/watch?v=B8DNexYcsa4
Sun Internal and Approved Partners Only 7
UltraSPARC T2 Core Block Diagram
EXU1
IFU
LSUSPU
TLU
MMU/HWTW
FGU
Gasket
xbar/L2
EXU0
• IFU – Instruction Fetch Unit> 16 KB I$, 32B lines, 8-way SA> 64-entry fully-associative ITLB
• EXU0/1 – Integer Execution Units> 4 threads share each unit> Executes one integer
instruction/cycle• LSU – Load/Store Unit
> 8KB D$, 16B lines, 4-way SA> 128-entry fully-associative
DTLB• FGU – Floating/Graphics Unit• SPU – Stream Processing Unit
> Cryptographic acceleration• TLU – Trap Logic Unit
> Updates machine state, handles exceptions and interrupts
• MMU – Memory Management Unit> Hardware tablewalk (HWTW)> 8KB, 64KB, 4MB, 256MB pages
Sun Internal and Approved Partners Only 8
Data
• Each Core has its own Floating Point Unit. This removes completely the 2% FP restriction of T1.
• Only a 6 cycle penalty to the FPU• Fully-pipelined (except divide/sqrt)
> Divide/sqrt in parallel with add or multiply operations of other threads
• Full VIS 2.0 implementation• FPU performs integer multiply, divide, population count• Multiplier enhancements for modular arithmetic operations
in the Crypto Unit• Performance is impressive...testing and tuning still
happening
Floating Point Unit
Sun Internal and Approved Partners Only 9
Niagara 2 Crypto Performance
Data
881---
1616166464644
2 rounds/cycle2 rounds/cycle
3 overlapped cycles4 words into 44 words6 words into 52 words8 words into 60 words
2 column over two cycles2 column over two cycles2 column over two cycles
2 cycles per step2 cycles per step2 cycles per step32 bits per cycle
16 rounds48 rounds
1---
10 rounds12 rounds14 rounds80 steps64 steps64 steps
1
8/9+25/2437683
323741
30/3036/3642/42
160/160128/128128/128
1
12.04.012.0
---
6.365.284.564.806.006.0048.00
CommentIterations/
BlockBlock(bytes)
Cycles/Block
GB/S
New
Ciphers
with U
ltraSPARC
T2• Supports ten most common ciphers and
secure hashing functions> RC4, DES, 3DES, AES-128, AES0192, AES-256, MDS, SHA-1,
SHA-256• On-chip crypto accelerators and built-in 10GbE vs.
accelerator cards> <36% crypto performance> >23% CPU overhead
• Leading cryptography performance> Over 30x greater RSA1024 performance than 2-socket IBM p510
• Best SSL Performance> 2.2x better performace/watt than 4-socket HP DL585G2> 1.74x better performance/watt than 2-socket HP DL380G5> 15.6x better AES128 performance than crypto acclerator card
Sun Internal and Approved Partners Only 10
Data
FB-DIMM Memory Interface
Feature UltraSPARC T1 UltraSPARC T2Require Pins 1000 440Available Bandwidth 23GBytes/second 50Gbytes/second
Memory Capacity up to 16x DIMM slots
Memory Controllers 4 4
DIMMS/Channel/Controller 02/02/04 08/02/04
Memory Latency 96ns 126nsDIMM speeds 400MHz 667MHzDIMM Power 7watts/DIMM 12.5Watts/DIMM
Up to 64x DIMM slotsT5x20 uses 16 DIMM slots
Sun Internal and Approved Partners Only 11
Data
10GbE Network Interface Unit• Fully Integrated Network Interface Unit
> Eliminates PCI-E chipset latencies > 30% better performance vs PCI-E NIC
– Mainly through reduced latency• Two 10 GbE ports (XAUI)• 16 Rx and 16 Tx DMA channels for virtualization• Packet classification
> Incoming flows are classified at layer 2 ,3 and 4 and put into the Rx DMA buffer according to the classification rules that matched the flow
• nxge device driver, source base shared with Atlas• Performance estimates, testing/tuning ongoing
– 1 port TX/RX = about 9.4Gbps in each direction with about only 40-50% CPU utilisation
– 2 port TX/RX = 12GbpsTX, 18GbpsRX with 70-80% CPU utilisation (still 20 threads available for application)
Sun Internal and Approved Partners Only 12
Agenda• Introduction• Overview of UltraSPARC T2• Sun SPARC Enterprise T5x20 servers
> Comparison to SunFire T1000/T2000> System Architecture and Tours> Memory, Networking and IO Expansion> Disks, Fans and Power Supplies> Solaris for T5x20
• ILOM 2.0• LDOMs for T5x20 • CRUs/FRUs and Configuration Options• Tuning and Performance• Tools, Information and References• Q & A ...
13Sun Internal Only
Sun SPARC Enterprise T5x20UltraSPARC T2 Platform
Extreme Rackmount Density• 1/2U chassis, 28” depth• Up to 8 cores @ 1.2 - 1.4GHz
> Up to 64 threads • 16 FB-DIMM memory slots
> 64GB of memory, (4GB DIMMs)High Reliability• 4 to 16 hot plug SAS 2.5” disk
drive (RAID 0,1)• N+N Redundant, hot-swappable
PSUs and Fans. DC PSU option for 1U systems only
Expandable• 3/6 PCI-E expansion slots
(including 2 x 10 GBE I/0 slots)• 4 x 1GbE Ethernet as standard• 4 USB ports
• Optimized for best price / performance & performance / watt
• Scale-Out of web & network infrastructure
• Extension to T1000/T2000 product line
Available Now
Sun Internal and Approved Partners Only 14
Sun Fire T1000 / T5120 Differences
Feature Sun Fire T1000 SPARC Enterprise T5120Form Factor 1U, 18¾” deep 1U, 28” deep
CPU
Memory
Network 4x GbE 4x GbE + 2x 10GbE
Internal Storage
Removable Media None 1x DVD-ROMSerialPCI Express slots
Power Supply
Fans Non-Hot-Swap Redundant/Hot-Swap
UltraSPARC T11.0 GHz, 32 threads
UltraSPARC T21.2/1.4 GHz 64 threads
DDR2 32GB Max8x slots
FB-DIMM 64GB MAX16x slots
1x 3.5” SATA, non-hot-swapOR 2x 2.5” SAS Up to 8 x 2.5” SAS, hot-swap
1x RS-232, No USB 1x RS-232, 4x USB1x (low profile) 3x(low profile)
1x 300W ACNon-Redundant
2x 650W AC & DC optionRedundant/Hot-Swap
Sun Internal and Approved Partners Only 15
Sun Fire T2000 / T5220 Differences
Feature Sun Fire T2000 SPARC Enterprise T5220Form Factor 2U, 24” deep 2U, 28” deep
CPU
Memory
4x GbE 4x GbE + 2x 10GbE
Internal Storage
Removable Media 1x DVD-ROM 1x DVD-ROMSerialPCI Express slotsPCI-X slots None
Power Supply
UltraSPARC T11.0 - 1.4GHz 32 threads
UltraSPARC T21.2/1.4 GHz 64 threads
DDR2, 64GB MAX16x Slots
FB-DIMM, 64GB MAX16x Slots
Network(10/100/1000)
4x 2.5” SAS, hot-swap Up to 16 x 2.5” SAS, hot-swap
1x RS-232, 4x USB 1x RS-232, 4x USB3x(low profile) 6x(low profile)2 x (low profile)
2x 450W ACRedundant/Hot-Swap
2x 750W ACRedundant/Hot-Swap
Sun Internal and Approved Partners Only 16
Front Panel
DISK 0 DISK 1 DISK 2 DISK 3
DISK 0
DISK 1 DISK 3
DISK 2
DISK 5
DISK 4 DISK 6 DISK 7
Locator LED/Button
Fault LEDStatus LEDPower Button
Locator LED/Button
Fault LEDStatus LEDPower Button
Serial Number label
Serial Number
Disk Drive MapTop Fault LEDPSU Fault LEDTemp Fault LED
Top Fault LEDPSU Fault LEDTemp Fault LED
Disk Drive Map
DVD-ROM
DVD-ROM
Dual USB 2.0 ports
Dual USB 2.0 ports
DISK 0
DISK 1 DISK 3
DISK 2
DISK 5
DISK 4 DISK 6 DISK 7
DVD-ROM
DISK 0DISK 1
DISK 3DISK 2
DISK 5DISK 4 DISK 10 DISK 13
DVD-ROMDISK 6
DISK 7
DISK 8DISK 9
DISK 11DISK 12 DISK 15
DISK 14
Sun Internal and Approved Partners Only 17
Rear Panel
PSU 0 PSU 1
PSU 1
PSU 0
PSUStatusLED's
SPSerialPort
ChassisStatus LED's
and Locator Button SP
NetworkPort
Quad GigabitEthernet Ports Dual
USB 2.0Ports
POSIX DB-9Serial Port
PCI-E Slot 0/XAUI 0x8 Slot/x4 Electrical
PCI-E Slot 1/XAUI 1x8 Slot/x4 Electrical
PCI-E Slot 2x8 Slot/x8 Electrical
PCI/XAUI 0 PCI/XAUI 1 PCI 2
PCI/XAUI 0 PCI/XAUI 1 PCI 2
PCI 5PCI 4PCI 3
PSUStatusLED's SP
SerialPort
ChassisStatus LED's
and Locator Button SP
NetworkPort
Quad GigabitEthernet Ports Dual
USB 2.0Ports
POSIX DB-9Serial Port
Sun Internal and Approved Partners Only 18
Architecture Block Diagram - T5120
x8
16 x FB-DIMM (DDR2-667)
4 Memory Controllers, 2 Channels, 2 DIMMS
Disk Chassis1RU
x4LSI1068Ex4SAS links
x4
x1USB
to IDE
DVD
USB
2.0
2.0
2.0 IntelDualGbE
x4 x4
0 1 2 30
10GbESerDes
BCM8704
XFP
10GbEFibrePlugin
10GbECu PHY
BCMxxxx
x8
x4x8
x4
USB 2.0Hub
PCI-ESwitch
PLX 8533PCI-ESwitch
PLX 8533PCI-ESwitch
PLX 8517
PCI-Eto
USB
IntelDualGbE
10GbE10GbE
MPC885ILOM
ServiceProcessor
FPGA
Front Panel
USB Quad GbE Connectors
PCI-Ex16
PCI-Ex8
PCI-Ex8
SerialMgt
NetworkMgt
POSIXSerial DB-9Rear Panel
SSI
x4
Sun Internal and Approved Partners Only 19
Architecture Block Diagram - T5220
x8
16 x FB-DIMM (DDR2-667)
4 Memory Controllers, 2 Channels, 2 DIMMS
Disk Chassis1RU 2RU/8
x4LSI1068E
x4x4SAS links
x4
x1USB
to IDE
DVD
USB
2.0
2.0
2.0 IntelDualGbE
x4 x4
0 1 2 30
10GbESerDes
BCM8704
XFP
10GbEFibrePlugin
10GbECu PHY
BCMxxxx
x8
x8
x4 x4 x4x8
x4
USB 2.0Hub
PCI-ESwitch
PLX 8533PCI-ESwitch
PLX 8533PCI-ESwitch
PLX 8517
PCI-Eto
USB
IntelDualGbE
10GbE10GbE
MPC885ILOM
ServiceProcessor
FPGA
Front Panel
USB Quad GbE Connectors
PCI-Ex16
PCI-Ex8
PCI-Ex8
PCI-Ex8
PCI-Ex8
PCI-Ex8
2RU Only
SerialMgt
NetworkMgt
POSIXSerial DB-9Rear Panel
SSI
Sun Internal and Approved Partners Only 20
Internal View (T5120)
DiskChassis
SAS x4 cable to
Disk Chassis 4 Fan Assemblies are required(chassis can take up to 8)
System Configuration Card (SCC)EEPROM
PCI-E Slot 2x16(m)/x8(e)
Chassis Intrusion Reed Switch
(magnet on underside of lid)X4 SAS Connector 0 (cable attached)
X4 SAS Connector 1 (not used in 1RU)
Service Processor
FB-DIMMs
UltraSPARCT2
Power Supplies
FB-DIMMs
FB-DIMMs
FB-DIMMs
PowerBoard
PSU1
PSU0
PCI-E Slot 1x8(m)/x4(e)
orXAUI 1
PCI-E Slot 0x8(m)/x4(e)
orXAUI 0
Power BUS BarsRED = +12VoltsBLUE = Ground
FB0/FM1
FB0/FM1
FB0/FM1
FB1/FM0
FB1/FM1
FB1/FM2
Fault RemindButton
Sun Internal and Approved Partners Only 21
Internal View (T5220)
DiskChassis
Two SAS x4 cables
to Disk Chassis
3 Fan Assemblies are required(chassis can take up to 6)
System Configuration EEPROM
PCI-E Slot 2x8(m)/x8(e)x16(m)/x8(e)
Chassis Intrusion Reed Switch
(magnet on underside of lid)X4 SAS Connector 0 (cable attached)X4 SAS Connector 1 (cable attached)
Service Processor
FB-DIMMs
UltraSPARCT2
Power Supplies
FB-DIMMs
FB-DIMMs
FB-DIMMs
PowerBoard
PSU's Vertically Stacked
PCI-E Slot 1x8(m)/x4(e)x8(m)/x4(e)
orXAUI 1
PCI-E Slot 0x8(m)/x4(e)x8(m)/x4(e)
orXAUI 0
Power BUS BarsRED = +12VoltsBLUE = Ground
FB0/FM1
FB0/FM2
FB0/FM0
Fault RemindButton
Sun Internal and Approved Partners Only 22
Memory Physical LayoutChassis Front
Chassis Rear
Channel1/DIMM 1/Branch 1Channel1/DIMM 0/Branch 1Channel0/DIMM 1/Branch 1Channel0/DIMM 0/Branch 1
Channel1/DIMM 1/Branch 0Channel1/DIMM 0/Branch 0Channel0/DIMM 1/Branch 0Channel0/DIMM 0/Branch 0
Channel0/DIMM 0/Branch 2Channel0/DIMM 1/Branch 2Channel1/DIMM 0/Branch 2Channel1/DIMM 1/Branch 2Channel0/DIMM 0/Branch 3Channel0/DIMM 1/Branch 3Channel1/DIMM 0/Branch 3Channel1/DIMM 1/Branch 3
Memory Population Rules1. All Branches must have at least one DIMM populated
- min config is 4 x 1GB DIMMS2. Populate Branches starting with the nearest slot to the CPU first
- See “1” in the diagramC0D0B0, C0D0B1, C0D0B2, C0D0B3
3. All DIMMS must be the same size4. In Branch all DIMMS must be indentical (SUN Part No.)
- Upgrading from 4 to 8 DIMM configs means moving DIMMS to ensure sameness
5. 12 DIMM configurations are NOT supported.6. Only 1GB, 2GB and 4GB DIMMS are supported7. Memory is FB-DIMM DDR2-667, there is no clocking down of memory in any config.
11
11
11
11
22
22
22
2233
33
33
33
33
33
33
33
UltraSPARCT2
Sun Internal and Approved Partners Only 23
Memory Configuration Rules• Populate at least 4 DIMMS at a time• First
> Fill DIMM0 in Channel 0 in each Branch• Second
> Fill DIMM0 in Channel 1 in each Branch• Third
> Fill remaining DIMM slots.• All DIMMS must be the same size(i.e all 1GB, 2GB or 4GB, no
mixing)• In a Branch all DIMMS must have the same SUN Part No.• 12 DIMM configurations are NOT supported.• No clocking down of memory in any config.
Sun Internal and Approved Partners Only 24
Memory Population Examples
Channel1/DIMM 1/Branch 1Channel1/DIMM 0/Branch 1Channel0/DIMM 1/Branch 1Channel0/DIMM 0/Branch 1
Channel1/DIMM 1/Branch 0Channel1/DIMM 0/Branch 0Channel0/DIMM 1/Branch 0Channel0/DIMM 0/Branch 0
Channel0/DIMM 0/Branch 2Channel0/DIMM 1/Branch 2Channel1/DIMM 0/Branch 2Channel1/DIMM 1/Branch 2Channel0/DIMM 0/Branch 3Channel0/DIMM 1/Branch 3Channel1/DIMM 0/Branch 3Channel1/DIMM 1/Branch 3
11
11
11
11
22
22
22
2233
33
33
33
33
33
33
33
UltraSPARCT2
Channel1/DIMM 1/Branch 1Channel1/DIMM 0/Branch 1Channel0/DIMM 1/Branch 1Channel0/DIMM 0/Branch 1
Channel1/DIMM 1/Branch 0Channel1/DIMM 0/Branch 0Channel0/DIMM 1/Branch 0Channel0/DIMM 0/Branch 0
Channel0/DIMM 0/Branch 2Channel0/DIMM 1/Branch 2Channel1/DIMM 0/Branch 2Channel1/DIMM 1/Branch 2Channel0/DIMM 0/Branch 3Channel0/DIMM 1/Branch 3Channel1/DIMM 0/Branch 3Channel1/DIMM 1/Branch 3
11
11
11
11
22
22
22
2233
33
33
33
33
33
33
33
UltraSPARCT2
Channel1/DIMM 1/Branch 1Channel1/DIMM 0/Branch 1Channel0/DIMM 1/Branch 1Channel0/DIMM 0/Branch 1
Channel1/DIMM 1/Branch 0Channel1/DIMM 0/Branch 0Channel0/DIMM 1/Branch 0Channel0/DIMM 0/Branch 0
Channel0/DIMM 0/Branch 2Channel0/DIMM 1/Branch 2Channel1/DIMM 0/Branch 2Channel1/DIMM 1/Branch 2Channel0/DIMM 0/Branch 3Channel0/DIMM 1/Branch 3Channel1/DIMM 0/Branch 3Channel1/DIMM 1/Branch 3
11
11
11
11
22
22
22
2233
33
33
33
33
33
33
33
UltraSPARCT2
4 DIMM configuration
8 DIMM configuration
16 DIMM configuration
Sun Internal and Approved Partners Only 25
Network Ports and Options
XAUI 0
NOTESNET 0 and NET 1 share a common Intel Ophir chip, as do NET 2 and NET 3.
XAUI Adapter cards are required to take advantage of the built-in 10GbE networkingThere are two variations, Fibre and Copper
Fibre XAUI cards take one of three current 10GbE transceiversShort Reach(SR), Long Reach(LR) and Long Reach Multimode(LRM)
Copper versions of the XAUI card are expected Post-RR XAUI Cards consume the same physical space in the chassis as a PCI card, so are mutually
exclusive with PCI cards in the same location
Device NET 0 NET 1 NET 2 NET 3 XAUI 0 XAUI 1Driver e1000g0 e1000g1 e1000g2 e1000g3 nxge0 nxge1PCI Device PCI 0/0/1/0/2 PCI 0/0/1/0/3 NIU@80/0 NUI@80/1
XAUI 1
Sun Internal and Approved Partners Only 26
10GbE XAUI Option Cards• Interface to the outside world for the Onboard 2 x 10GbE interfaces
from the UltraSPARC T2• Cards each consume a PCI-E slot if installed• Attach to dedicated XAUI sockets on special risers• 10Gbe XAUI Card
– SESY7XA1Z> Requires Fibre Tranceivers
– 10GbE Short Reach(SR)– SESY7XT1Z
– 10GbE Long Reach (LR)– SESY7XT2Z
– 10GbE Long Reach multimode (LRM)– part No. TBD
• Copper connector XAUI versions to follow post-RR
Sun Internal and Approved Partners Only 27
PCI Riser Cards• Various Riser Cards are used• Risers should not be moved between slots• All slots accommodate low Profile PCI-Express cards• T5120 – 25 Watts per card, 75 Watts Max (3 Slots)• T5220 – 25 Watts per card, 102 Watts Max (6 Slots)
> Riser Card Mechanical and Electrical widths for T5120 and T5220
T5120 Slot 0 Slot 1 Slot 2Only Slot x8/XAUI(m) x4(e) x8/XAUI(m) x4(e) x16(m) x8(e)
T5220 Slot 1 Slot 1 Slot 2Top Slot x8(m) x4(e) x8(m) x4(e) x8(m) x8(e)
Sun Internal and Approved Partners Only 28
Disk Cage• Disk chassis is a FRU
– 4 screws and disconnect cables• Contains DVD-ROM/Dual USB module as a
sub-FRU/CRU– Standard in all default configs– remove disk 3/7 to access release latch, pre-RR
• Non-DVD module available as X-Option– Still provides Dual USB ports
• T5120 will be available initially with only a 4 disk chassis, 8 disk in qualification > Only uses one or two, of the x4 SAS links
• T5220 will be available initially with only an 8 disk chassis, 16 disk in qualification> Uses both of the x4 SAS link cables
Remove Disk and locate latch beneath DVD
Press up and pull DVD out
Sun Internal and Approved Partners Only 29
Disk Cage - T5120• Chassis accommodates up to 4 or 8 SAS or SATA 2.5”
drives– No mixing in a chassis– No chassis changes needed
• Disk Chassis requires only a single cable
• Connects to Fan Paddle board with press fit connector
• Mounting ensures fan to disk vibration isolation
'T' headlocator bolts
Locator Tabs
Fan Paddle Board press-fit connector
x4 SAS cable and connector
Chassis Screws
Chassis Screws
Sun Internal and Approved Partners Only 30
Disk Cage - T5220• Accommodates up to 8 or 16 SAS or SATA
2.5” drives• Requires both SAS cables be connected• Two x4 SAS connector on chassis• Even 16 disks provides lots of airflow• Cage and Fan install ensures vibration
isolation
'T' headlocator bolts
Locator Tabs
Paddle Board press-fit
connector x4 SAS cables and connectors
Chassis Screws
Chassis Screws
Sun Internal and Approved Partners Only 31
Disk Drives• Disk drives are all 2.5” form factor• Chassis support both SAS and SATA (post-RR)• SAS Drives
– 73GB @ 10K RPM– 146GB @ 10K RPM– 73GB @ 15K RPM (post-RR)– 146GB @ 15K RPM (post-RR)
• SATA Drives– 200GB @ 5400 RPM (soon after RR)– 300GB @ 5400 RPM (H1CY08)– 500GB @ 7200 RPM (end CY08)
• New mounting bracket “Marlin”> Allows better cooling in denser chassis
Ready to Remove
FaultStatus
Disk LED's
Improved Airflow above
and below disks
Sun Internal and Approved Partners Only 32
Disk Controller Options• T5x20 has embedded LSI1068E on the motherboard
– JBOD up to 8 disks– Controls internal disks only– RAID 0 (striping) and RAID 1(mirroring)– Hot-plug supported– Hot-swap in non-RAID configs is done with 'cfgadm'
• Optional Extended RAID functionality cards (available Q4CY07)> 'Cougar' - SGXPCIESAS-R-INT-Z
– 8 port SAS RAID 0,1,1E,5,5EE,6,10,50,60 (depending on No. disks available)– Adaptec Card and Firmware with Intel SunRiseLake chipset– Battery Backed Write Cache (BBWC) of 256MB– BBWC battery at least 72 hours (much longer when new/fully charged)– Only for internal disks, replaces cables to route to standard disk chassis
> 'Prometheus' – SGXPCIESAS-R-EXT-Z– External port version of 'Cougar', for external disk arrays only (no internal ports)
Sun Internal and Approved Partners Only 33
Fans and Cooling• T5120
> Requires 4 Fan modules> Two cooling groups
– Group 1 – N2 cooling– FANBD0/FM1– FANBD1/FM1
– Group 2 – FB-DIMM cooling– FANBD1/FM0– FANBD1/FM2
– Blank Module in FANBD1/FM3• T5220
> Requires 3 Fan Modules> Single group for N2 and FB-DIMM
– FANBD0/FM0– FANBD0/FM1– FANBD0/FM2
• Higher disk densities may require more fans
PSU'sPSU'sFBD
1FBD
1FM
3FM
3FBD
1FBD
1FM
2FM
2FBD
1FBD
1FM
1FM
1FBD
1FBD
1FM
0FM
0
FBD0
FBD0
FM0
FM0
FBD0
FBD0
FM1
FM1
FBD0
FBD0
FM2
FM2
FBD0
FBD0
FM3
FM3
MotherboardMotherboard
Disk C
hassisD
isk Chassis
PSU'sPSU's
MotherboardMotherboard
Disk C
hassisD
isk Chassis
FBD0
FBD0
FM1
FM1
FBD0
FBD0
FM0
FM0
FBD0
FBD0
FM2
FM2
FBD1
FBD1
FM1
FM1
FBD1
FBD1
FM0
FM0
FBD1
FBD1
FM2
FM2
FrontBack
Sun Internal and Approved Partners Only 34
Fans modules and boards• Fans are speed controlled by the Service
Processor• Each fan has an Amber FAULT LED
– if lit the fan module is faulty• 1RU modules
– 2 Fans at 15K RPM Max.• 2RU modules
– 2 Fans at 12K RPM Max• Monitored through ILOM/IPMI/Web
Fan Fault Amber LED
Lit if fan faulty
Vibration Absorbing
Silicon Dampers
Sun Internal and Approved Partners Only 35
Power Supplies• PSU's are high efficiency• Output 12V (and 3.3v standby)• Most DC-DC conversion done in MBD• Shared with Doradi/o, Tucani/a• Integrated Fan• T5120
> Side-by-Side> AC 650Watts> DC 660 Watts option coming post-RR> DC or AC, no sharing
• T5220> Vertically Stacked> 750Watts (may require the 1050Watt Psu for the 16 disk chassis, pending quals)> No DC option> BEWARE... Same form factor as SunFire X4200 PSU's ... DO NOT MIX
Ready to Remove
Fault
AC Present
PSU LED's
Sun Internal and Approved Partners Only 36
Rack Mounting Options• Two Rack mounting options are available..• Standard Rack Mount (4 post)SESY9RK1Z
> Same as Galaxy servers, T2000, etc..> Standard configs contain this kit
– ATO : SESY9RK1Z, X-ATO : SESY9RK1Z• Tool-less Rack Kit (4 post)
> Optional kit, purchased seperately> Snaps into most 3rd party Racks> Requires no tools to fit or remove
– ATO : SESY9RK2Z, X-ATO : SESX9RK2Z
Sun Internal and Approved Partners Only 37
Solaris for Sun SPARC Enterprise T5x20• T5x20 servers will require Solaris10 8/07 (update 4)• This incorporates several UltraSPARC T1/T2
optimisations for CMT servers in general• Also includes some specific T5x20 optimisations
> UltraSPARC T2 Crypto optimisations> nxge driver for 10GbE devices> UltraSPARC T2 performance counters> Processor grouping, page coloring and default sizing and
bcopy/memcpy
Sun Internal and Approved Partners Only 38
Solaris FMA for T5x20• Now aware of CPU Core oflining• PCIe Fabric diagnostics improved• Crypto Unit offlining• Network interface Units (10GbE)• POST memory CE tolerance improved• FRU identity info improved in fault reports (part/ser #)• Diagnostic Engines (DE)
– Existing DE have been extended for T5x20– New Eversholt DE used for HostBridge, PCIe and NUI diagnostics
• Limitations...– Can't isolate Uncorrectable Errors to a single DIMM– PCIe riser cards not in topology– XAUI card and XFP not in the topology– SouthBridge chip is not diagnosed at this time (DVD, USB,...)
Sun Internal and Approved Partners Only 39
Agenda• Introduction• Overview of UltraSPARC T2• Sun SPARC Enterprise T5x20 servers
> Comparison to SunFire T1000/T2000> System Architecture and Tours> Memory, Networking and IO Expansion> Disks, Fans and Power Supplies> Solaris for T5x20
• ILOM 2.0• LDOMs for T5x20 • CRUs/FRUs and Configuration Options• Tuning and Performance• Tools, Information and References• Q & A ...
Sun Internal and Approved Partners Only 40
ILOM Features at a Glance• Hardware Control, Inventory, Monitoring• Host Configuration and Administration• SP Configuration and Administration• Logging, Auditing and Alerting• Authentication :
– LDAP, RADIUS, Active Directory, Local Users
• Services :– SSH, HTTP(S), NTP, Syslog, SMTP
• Out-of-Band– Via Serial port, or dedicated Ethernet port
• In-Band– Via Host OS, firmware upload, Explorer, Syslog
Sun Internal and Approved Partners Only 41
ILOM – Industry Standard Manageability• ILOM enables Sun systems to be easily integrated into
third party management infrastructures• Key functionality
> IPMI 2.0> SNMP V1, V2c, V3> SSH 2.0> LDAP, RADIUS & MS Active Directory authentication> Browser UI, SMASH DMTF CLP + ALOM style CLI
• NOTE : No RKVMS functionality like Galaxy ILOM> No Java plugin required
• Browsers tested include– Mozilla 1.7 on Solaris 10, IE 6.0 on W2K Server,
Firefox 1.0 and 1.5 on W2K server
Sun Internal and Approved Partners Only 42
ILOM – Industry Standard Manageability
• Default user is 'root' with password 'changeme'> Use for ILOM WEB GUI access and SSH access and
IPMI• The ALOM 'admin' user is not defined by default, create
this user with an ALOM CLI• Lost 'root' password and ILOM config is reset using
Motherboard Jumper J6801• Note... 'sunservice' user shares the password of the
'root' user ONLY FOR SERVICE ACCESS*• Passwords must be 8 to 16 chars (used to be max 8)• Max of 10 users allowed (used to be 16)
Sun Internal and Approved Partners Only 43
ILOM Common Tasks• Setting up an ALOM 'admin' user from the ILOM CLI
> SUNSP00144F4641BD login: rootPassword: changemeWaiting for daemons to initialize...Daemons readySun(TM) Integrated Lights Out ManagerVersion 2.0.4.0Copyright 2007 Sun Microsystems, Inc. All rights reserved.Use is subject to license terms.Warning: password is set to factory default.->create /SP/users/admin role=Administrator cli_mode=alomCreating user...Enter new password: ********Enter new password again: ********Created /SP/users/admin2-> exitSUNSP00144F4641BD login: adminPassword: ********Waiting for daemons to initialize...Daemons readySun(TM) Integrated Lights Out ManagerVersion 2.0.4.0Copyright 2007 Sun Microsystems, Inc. All rights reserved.Use is subject to license terms.sc>
Sun Internal and Approved Partners Only 44
ILOM Common Tasks• Setting up ILOM IP configuration from the ILOM CLI
> SUNSP00144F4641BD login: rootPassword: changemeWaiting for daemons to initialize...Daemons readySun(TM) Integrated Lights Out ManagerVersion 2.0.4.0Copyright 2007 Sun Microsystems, Inc. All rights reserved.Use is subject to license terms.Warning: password is set to factory default.-> set /SP/network state=enabledSet ’state’ to ’enabled’-> set /SP/network pendingipaddress=10.8.31.163Set ’pendingipaddress’ to ’10.8.31.163’-> set /SP/network pendingipdiscovery=staticSet ’pendingipdiscovery’ to ’static’-> set /SP/network pendingipnetmask=255.255.252.0Set ’pendingipnetmask’ to ’255.255.252.0’-> set /SP/network pendingipgateway=10.8.31.248Set ’pendingipgateway’ to ’10.8.31.248’-> set /SP/network commitpending=trueSet ’commitpending’ to ’true’->
-> set/SP/pendingipdiscovery=dhcpSet ’pendingipdiscovery’ to ’dhcp’-> set /SP/network commitpending=trueSet ’commitpending’ to ’true’->
Static IP configuration
Dynamic IP (DHCP)
configuration
Sun Internal and Approved Partners Only 45
ILOM IPMI functionality• IPMI is a widely used management protocol for x86 servers• ... But its new for SPARC boxes...• Download the required tools from
> The Solaris Supplemental CD (an older binary package)> Latest is available at http://ipmitool.sourceforge.net
– Binary packages (usually older versions) are available for Solaris x86 and SPARC, as well as Linux, Windows, Mac...etc
> There is a Blueprint available from Sun also (mainlyx64 centric, but applies to SPARC now too)
– http://www.sun.com/blueprints/0107/820-1011.pdf-bash-ventus2$ /opt/ipmitool/bin/ipmitool -U root -P changeme -H 10.10.10.10 sdr/MB/V_+1V0_VDD | 1.01 Volts | ok/MB/V_+3V3_STBY | 3.36 Volts | ok/MB/V_+3V3_MAIN | 3.36 Volts | ok/MB/V_+5V0_VCC | 4.97 Volts | ok/MB/V_+12V0_MAIN | 12.16 Volts | ok....-bash-ventus2$ /opt/ipmitool/bin/ipmitool -U root -P changeme -H 10.10.10.10 sunoem sbled set /SYS/LOCATE on/SYS/LOCATE | ON-bash-ventus2$
Get the Sensor Data Repository
Set the System Locator LED
flashing
Sun Internal and Approved Partners Only 46
ILOM Common Tasks• Logging into the ILOM Web GUI
• Point Web Browser at the IP address of the ILOM> Will automatically redirect to a
secure HTTPS port> Best to disable any Web
Proxies for best performance> Make sure port 80, 443 are
open on any firewall
• Log in as any defined user, default Administration user is 'root'
Sun Internal and Approved Partners Only 47
ILOM/ALOM Common Tasks• A Tour of the ILOM Web GUI • GUI provides ways to set up IP address
configurations for ILOM
• Configure additional ILOM/ALOM users
• Access Sensor Readings
• Control System configurations
• Carry out firmware updates.
• etc....
Sun Internal and Approved Partners Only 48
ALOM 2.0 vs. 1.4 CLI Differences• Default CLI appearance is ILOM
– User is 'root'– Must create an 'admin' Administrator user with the ALOM CLI
• Flashupdate in new ALOM expects a TFTP server not an FTP server • ALOM IP address changes now active only after running ...
– 'setsc netsc_commit true'– Previously changes were active only after SC reset– DHCP is enabled by default
• View firmware versions with – 'showhost version'
• View FPGA version with – 'fpga version'
• Only Admin and Operator rolesare defined
sc> showhost version Sun System Firmware 7.0.0_build_13 2007/08/10 23:44
Host flash versions: Hypervisor 1.5.0.build_13**PROTOTYPE** 2007/08/10 22:54 OBP 4.27.0.build_04***PROTOTYPE BUILD*** 2007/08/10 13:25 Sun Fire[TM] Huron POST 4.27.0.build_04***PROTOTYPE BUILD*** 2007/08/10 13:51
sc> fpga versionFPGA Version: 1.5.0.0Platform ID : 1Major revision : 5Minor revision : 0Lab Debug Version: 0sc>
Sun Internal and Approved Partners Only 49
ALOM backward Compatibilty CLI• Uses ALOM-CMT 1.2 for a base feature set• Includes ALOM-CMT 1.4 features...
> break -d> Host Watchdog Timer> Bootmode LDOMs awareness> Forward SP events to Hot/Solaris syslog via inband
channel
Sun Internal and Approved Partners Only 50
Agenda• Introduction• Overview of UltraSPARC T2• Sun SPARC Enterprise T5x20 servers
> Comparison to SunFire T1000/T2000> System Architecture and Tours> Memory, Networking and IO Expansion> Disks, Fans and Power Supplies> Solaris for T5x20
• ILOM 2.0• LDOMs for T5x20 • CRUs/FRUs and Configuration Options• Tuning and Performance• Tools, Information and References• Q & A ...
Sun Internal and Approved Partners Only 51
Logical Domains 1.0.1 Features
• UltraSPARC T2 Platform Support> 32 domains at RR, 64 domains following
• I/O domain can reboot while the other domains keep running> Dependant domains would wait for IO transactions to complete
• LDoms SNMP MIB> Modeled after industry standard DMTF CIM (Common Information
Model)> Running on control domain to enable remote SNMP monitoring
• Domain minimization support> Minimal/reduced Solaris installation> Base minimal packages plus optional ones for guest, control, I/O or
service domains
Released, aligned with UltraSPARC T2/T5x20
Sun Internal and Approved Partners Only 52
Logical Domains 1.0.1 Requirements • Solaris 10 08/07 (update 4)• Systems firmware version
– T5x20/Glendale Firmware 7.0.x = SP FW 2.0.4.0+– T2000/T1000/T6300/Pelton Firmware 6.5.x
• SUNWldm.v LDOMs Manager Package– Version 1.0.1
• T5x20 System FPGA– Version 1.5.0+ (Not a concern for Post-RR systems)
Sun Internal and Approved Partners Only 53
Agenda• Introduction• Overview of UltraSPARC T2• Sun SPARC Enterprise T5x20 servers
> Comparison to SunFire T1000/T2000> System Architecture and Tours> Memory, Networking and IO Expansion> Disks, Fans and Power Supplies> Solaris for T5x20
• ILOM 2.0• LDOMs for T5x20 • CRUs/FRUs and Configuration Options• Tuning and Performance• Tools, Information and References• Q & A ...
Sun Internal and Approved Partners Only 54
CRU/FRU List - T5120• Lots shared with other Jake and Elwood systems
I tem Description FRU/CRUDisk Backplane 4 drive Disk Backplane to support 4 Drives only FRUFan Module 1U Dual Fan module, 4 populated in system CRUFan Board 1U Fan connector board to accommodate Fan Modules, 2 per system FRUDVD Assembly DVD Assembly including front USB board FRUPaddle board 1U Conection between PDB, fan boards and Disk Backplane FRUPDB 1U FRUPSU 1U Power supply (650 Watts) CRUX8 &XAUI, 1U PCIe Riser PCI Riser assembly for left and centre motherboard slots CRUx16 1U PCIe Riser PCI Riser for right motherboard slot CRURack mount kit (requires tools) Rack mount kit CRURack Mount Kit (tooless) Rack mount kit, no tools required for installation CRUCMA Cable management arm CRUBus Bars Power Bus Bars between PDB and Motherboard FRUPDB to MB Ribbon Cable Ribbon cable for control connections between PDB abd Motherboard FRULight pipe kit Light pipe assemblies for left and right of disk cage FRUMotherboard assembly Motherboard assembly, part no varies depending on Core count FRUDIMMs FBDIMMs (three different part numbers for 1GB, 2GB, 4GB) CRUXAUI board optical/copper XAUI Adapter card (two different parts for copper/optical) CRUXFP module XFP GBIC module for use in XAUI cards (three different parts SR, HR, HRMM) CRUBattery Battery backup for NVRAM of Service Processor CRUHDD 73GB/146GB SAS Hard Disk drive, different part numbers for capacities CRUSystem Configurations PROM System Configuration PROM FRU
Power distribution board, contains master chassis serial number
Sun Internal and Approved Partners Only 55
CRU/FRU List - T5220• Lots shared with other Jake and Elwood systems
I tem Description FRU/CRUDisk Backplane 8 drive Disk Backplane to support 8 Drives only FRUFan Module 2U Dual Fan module, 3 populated in system CRUFan Board 2U Fan connector board to accommodate Fan Modules, 2 per system FRUDVD Assembly DVD Assembly including front USB board FRUPaddle board 2U Connection between PDB, fan boards and Disk Backplane FRUPDB 2U FRUPSU 2U Power supply (750 Watts) CRUX8,X8, XAUI, 2U PCIe Riser PCI Riser assembly for left and centre motherboard slots CRUX8, x16 2U PCIe Riser PCI Riser for right motherboard slot CRURack mount kit (requires tools) Rack mount kit CRURack Mount Kit (tooless) Rack mount kit, no tools required for installation CRUCMA Cable management arm CRUBus Bars Power Bus Bars between PDB and Motherboard FRUPDB to MB Ribbon Cable Ribbon cable for control connections between PDB abd Motherboard FRULight pipe kit Light pipe assemblies for left and right of disk cage FRUMotherboard assembly Motherboard assembly, part no varies depending on Core count FRUDIMMs FBDIMMs (three different part numbers for 1GB, 2GB, 4GB) CRUXAUI board optical/copper XAUI Adapter card (two different parts for copper/optical) CRUXFP module XFP GBIC module for use in XAUI cards (three different parts SR, HR, HRMM) CRUBattery Battery backup for NVRAM of Service Processor CRUHDD 73GB/146GB SAS Hard Disk drive, different part numbers for capacities CRUSystem Configurations PROM System Configuration PROM FRU
Power distribution board, contains master chassis serial number
Sun Internal and Approved Partners Only 56
Option Parts List
Part Number Option Description Avialable
SECY9PS11Z AC PSU 1RU (650 Watts) RRSEDY9PS31Z AC PSU 2RU (750 Watts) RRSECY9PS21Z DC PSU 1RU (660Watts) post-RRSESY2A1Z 2GB Memory (2x1GB) RRSESY2B1Z 4GB memory (2x2GB) RRSESY2C1Z 8GB Memory (2x4GB) RRSESY3A11Z 73GB, 10KRpm 2.5” SAS Drive RRSESY3C11Z 146GB, 10KRpm 2.5” SAS Drive RR- SATA Drives (200, 300,500GB) post-RRSESY9DV1Z DVD, 8X RW, Slot Load RRSESY9RK1Z Slide Rail Kit, requires screwdriver (1RU and 2RU) RRSESY9RK2Z Slide Rail Kit, Tool-less (1RU and 2RU) RR
Sun Internal and Approved Partners Only 57
Option Parts List (cont..)Part Number Option Description Avialable
SG-PCIE1FC-QF4 Qlogic, 4Gb, PCI-E Single Port (up to 3/6) RR
SG-PCIE2FC-QF4 Qlogic, 4Gb, PCI-E Dual Port (up to 3/6) RRSG-PCIE1FC-EM4 Emulex, 4Gb, PCI-E Single Port (up to 3/6) RR
SG-PCIE2FC-EM4 Emulex, 4Gb, PCI-E Dual Port (up to 3/6) RRSG-PCIE2SCSI-U320Z U320 SCSI, PCI-E Dual Port (up to 3/6) RR
SG-PCIE8SAS-E-Z SAS, 8 Port, PCI-E (up to 1) RR
SGXPCIESAS-R-INT-Z SAS, 8 Port, RAID, PCI-E Internal Drive Controller (up to 1) RR+3
SGCPCIESAS-R-EXT-Z SAS, 8 Port, RAID, PCI-E, External Drive Controller (up to 1) RR+3
7280A-2 Northstar Dual port GigE MMF Low profile PCI-E (up to 3/6) RR
7281A-2 Northstar Dual port GigE UTP Low profile PCI-E (up to 3/6) RR
1236A-Z Infiniband HBA, Dual 4x ports, PCI-E (up to 3/6) RR
4447A-X Atlas Quad Port, GigE Copper, PCI-E (up to 3/6) RR
X4240A XVR-300, PCI-E x16/x8 Graphics cards (up to 1) TBD
Sun Internal and Approved Partners Only 58
Agenda• Introduction• Overview of UltraSPARC T2• Sun SPARC Enterprise T5x20 servers
> Comparison to SunFire T1000/T2000> System Architecture and Tours> Memory, Networking and IO Expansion> Disks, Fans and Power Supplies> Solaris for T5x20
• ILOM 2.0• LDOMs for T5x20 • CRUs/FRUs and Configuration Options• Tuning and Performance• Tools, Information and References• Q & A ...
Sun Internal and Approved Partners Only 59
Tuning Out of the Box• Using Solaris 10 08/07...
> Default pages sizes for – Initdata Segments is 64kB pages– Private Anon, Heap, Process Stack, Text, Shared Anon segments are
4MB pages– ISM and DISM use 256MB pages– New variables introduced to control these sizes if changes are needed
– max_uheap_lpsize,max_ustack_lpsize, max_privmap_lpsize, max_shm_lpsize, max_uidata_lpsize, max_utext_lpsize, max_shm_lpsize
> L2 Cache hashing implemented– helps randomise accesses to L2 caches and ease thrashing
> Stack biasing implemented– D-Cache is only 4 Way associative... 8 threads– Running multiple copies of a binary can lead process stacks aligning in
the cache and to hot lines in the cache– Stack Bias slews the stack variable for processes.
>
Sun Internal and Approved Partners Only 60
Tuning Out of the Box (continued..)> Processor Groups introduced
– Helps associate CPUs with some shared resources (int/FP pipelines, Crypto units etc)
– Enables improved multi-level CMT load balancing and affinity policies> 'bcopy'/'uiomove' Optimizations
– Key kernel operations– new implementation about 2x faster than previous
• Some optimisations did not make it into S10 8/07> These will be post RR patches as they become available> Most fixes are already committed in update 5> Check patch levels for customers with performance issues> Also check Firmware versions as this can improve
performance too...
Sun Internal and Approved Partners Only 61
Tuning Recommendations• Less involved than with T2000• S10 8/07 is more 'automatic'
– OOB optimised page size selection, thread scheduling, block copy, IP instances for zones, page coloring etc...
• If tuning is necessary remember...> Improved pipeline results in Single threaded performance around 20% better> Floating Point is no longer an issue> Start with T2000 tuning options> Use updated trapstat, cpustat,
• LDOMs may be an easier way to scale poorly threaded applications• Solaris Crypto framework is still being tuned
> Great for big packets, working on improving small packets• A Blueprint is being prepared (Denis Sheahan)
Sun Internal and Approved Partners Only 62
Tuning Recommendations (cont..)• Refer to UltraSPARC T1 applications tuning best practices...
> http://www.sun.com/servers/coolthreads/tnb/applications.jsp• CoolTools are being updated for UltraSPARC T2
> http://www.sun.com/servers/coolthreads/overview/cooltools.jsp• Sun Studio 12 is UltraSPARC T1 and T2 aware
> Recompiling without code changes should benefit T1/T2 apps• Processes are being worked on for LDOMS migrations 1.0 to 1.0.1
> LDOMs configs can benefit from a transition to T5x20 platform> Make sure appropriate LDOMs tuning and sizing is carried out
• Wider Crypto capability, so ensure code is using PKCS#11 framework> Ensure that the ncp (SSL, RSA, DSA etc) and n2cp (bulk ciphers),
n2rng(random number generator) drivers are used appropriately•
Sun Internal and Approved Partners Only 63
Agenda• Introduction• Overview of UltraSPARC T2• Sun SPARC Enterprise T5x20 servers
> Comparison to SunFire T1000/T2000> System Architecture and Tours> Memory, Networking and IO Expansion> Disks, Fans and Power Supplies> Solaris for T5x20
• ILOM 2.0• LDOMs for T5x20 • CRUs/FRUs and Configuration Options• Tuning and Performance• Tools, Information and References• Q & A ...
Sun Internal and Approved Partners Only 64
Tools and Information• Cool Tools
– http://www.sun.com/servers/coolthreads/overview/cooltools.jsp
• Architecture WhitePaper– http://www.sun.com/
• SUNWIN Tokens> Sun Intro : 514416> Architecture Whitepaper : 512750> This Technical Presentation: 512749
Sun Internal and Approved Partners Only 65
Reference Information• T5120 and T5220 (Huron) Engineering TOI
– http://ittdev.east.sun.com/TechTalk2/EngineeringTOIs/Huron_Apr07/• ILOM/ALOM User Guides
– http://www.sun.com/products-n-solutions/hardware/docs/pdf/819-1160-13.pdf– http://www.sun.com/products-n-solutions/hardware/docs/pdf/819-7981-10.pdf
• Solaris FMA– http://fma.eng– http://lightside.sfbay/n2-fma/docs.php
•
66
Peter A. WilsonSenior Technical Marketing EngineerSun Microsystems
66
Sun Internal and Approved Partners Only 67
Huron PCI Topology
Sun Internal and Approved Partners Only 68
T5x20 PCI Device TreeUltraSPARC T2 PCI bus
To First Level PCI-E Switch ChipTo Second level PCI-E Switch Chip
TO SouthBridge USB Devices
Standard USB Device managerEnhanced USB Device mangerUSB Dual Port HUB
Rear Port 0Rear Port 1
USB to IDE ConverterDVD ROM Drive
Front USB Port 0
Front USB Port 1To Dual GbE Controller 0
NIC 0NIC 1
To Dual GbE Controller 1NIC 2NIC 3
To SAS ControllerSAS Controller
SAS Disk DeviceTo Second Level PCI-E Switch Chip
PCI-E Slot 1/2RUPCI-E Slot 0/2RUPCI-E Slot 2/2RUPCI-E Slot 1/1RUPCI-E Slot 0/1RU
PCI-E Slot 2Network Interface Unit
10Gb_NIC 010Gb_NIC 1
“/pci@0" 0 "px"“/pci@0/pci@0" 0 "pxb_plx"“/pci@0/pci@0/pci@1" 1 "pxb_plx"“/pci@0/pci@0/pci@1/pci@0" 3 "pxb_plx"“/pci@0/pci@0/pci@1/pci@0/pci@1" 4 "pxb_plx"“/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0" 0 "px_pci"“/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0" 0 "ohci"“/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,1" 1 "ohci"“/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2" 0 "ehci"“/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4" 0 "hubd"“/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/keyboard@1" 2 "hid"“/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/keyboard@2" 0 "hid"“/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@1" 0 "scsa2usb"“/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@1/disk@0,0" 0 "sd"“/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@1/disk@0,1" 3 "sd"“/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@2" 1 "scsa2usb"“/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@2/disk@0,0" 2 "sd"“'pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@3" 2 "scsa2usb"“”pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@3/disk@0,0" 4 "sd"“/pci@0/pci@0/pci@1/pci@0/pci@2" 5 "pxb_plx"“/pci@0/pci@0/pci@1/pci@0/pci@2/network@0" 0 "e1000g""/pci@0/pci@0/pci@1/pci@0/pci@2/network@0,1" 1 "e1000g""/pci@0/pci@0/pci@1/pci@0/pci@3" 6 "pxb_plx""/pci@0/pci@0/pci@1/pci@0/pci@3/network@0" 2 "e1000g""/pci@0/pci@0/pci@1/pci@0/pci@3/network@0,1" 3 "e1000g""/pci@0/pci@0/pci@2" 2 "pxb_plx""/pci@0/pci@0/pci@2/scsi@0" 0 "mpt""/pci@0/pci@0/pci@2/scsi@0/sd@0,0" 1 "sd""/pci@0/pci@0/pci@8" 7 "pxb_plx""/pci@0/pci@0/pci@8/pci@0" 9 "pxb_plx""/pci@0/pci@0/pci@8/pci@0/pci@1" 10 "pxb_plx""/pci@0/pci@0/pci@8/pci@0/pci@2" 11 "pxb_plx""/pci@0/pci@0/pci@8/pci@0/pci@8" 12 "pxb_plx""/pci@0/pci@0/pci@8/pci@0/pci@9" 13 "pxb_plx""/pci@0/pci@0/pci@8/pci@0/pci@a" 14 "pxb_plx""/pci@0/pci@0/pci@9" 8 "pxb_plx""/niu@80" 0 "niumx""/niu@80/network@0" 0 "nxge""/niu@80/network@1" 1 "nxge"