Quadrics QsNet: A network for Supercomputing Applications

ANNARAYA S N – 1BY08EC007

UNDER THE GUIDANCE OF:

SANEESH CLEATUS T SABINA RAHAMAN ASST. PROF., ECE, ASST. PROF., ECE, BMSIT BMSIT

A Quadrics Supercomputer Network

§ Ultra low user process to user process latency

§ Highest possible (affordable?) compute

communications ratio

§ Seamless scaling to many 1000s of nodes

§ High availability

§ Reliable data transfer

§ Mixed system and multiple user traffic on one network

QsNet’s contributions:

Integration of the virtual-address spaces.

Network fault detection and fault tolerance.

QsNet I Components:

Elan Network InterfaceElite SwitchGlobal Virtual MemoryNetwork Fault Detection and Fault Tolerance

Improvements in QsNet II:

Increase in the processor performanceIncreased link speed.Enhancement in the adapter and switch functionality.The bidirectional data communication speed can be increased to 1.3 Gbyte/s.

Elan4

Packet Generation

For receiving:Elan4 performs master PCI-X read .Node CPUs can issue writes directly.

For sending:For large messages, the Elan4 DMA engine can read from the PCI-X.For small messages, Elan4 uses STEN processor.

Elan4 Command Queues

Up to 8000 command queues can be active at any one time. No practical limit to the command queues that Elan4 device driver can allocate. Command queues provide protected access to the Elan4 hardware.

Elan4 Command-queue model

Elite 4

•Elite 4 Switch Component

•8 QsNetIIlinks ×2 virtual channels•Broadcast to range of outputs•Full automatic error detection / recovery •Arbitration based on age of packet•Two levels of priority•Adaptive routing support•Unblocked latency of ~20ns•Trace route transaction for interrogating the network

Performance Evaluation Based on Coding

Performance Evaluation Based on Platforms

Improvements in QsNet III

QsNet III is a multi-stage switch network.Elan4 is enhanced to Elan5. It has 2 adapters for routing.The improved version of Elite takes less than 40 ns to route a packet.HPC Specific Features of QsNet III -Barrier and Broadcast -Small Packet Support

Implementation

Thunder an 4096-way Itanium II system at Lawrence Livermore National Laboratory, the most powerful system in the US and the second most powerful system in the world.Quadrics hardware and software is an integral part of many of the leading production HPC clusters.

Advantages

*Full, pageable 64 bit virtual memory support*Multiple, virtual, programmable network interfaces*Ultra-low latency short messaging*Optimized support for scalable global operations*Ability to scale number of network connections with number of CPUs for SMP nodes.*Proven scalability to many 1000s of processors.

Conclusion

QsNet generations provide a high performance data network for HPC apps.

Quadrics have increased link speeds significantly in each generations.

Quadrics have enhanced both adapter and switch functionality.

No. of components required is reduced by using high radix routers.

Cost and complexity of routing data is reduced. Features such as h/w barrier and broadcast enhance the

applicability to HPC high end applications.

References

The Quadrics Network (QsNet): High-Performance Clustering Technology- Fabrizio Petrini, Wu-chun Feng, Los Alamos National Laboratory, IEEE Computer society, 2001.QsNet II: Defining High-Performance Network Design- Jon Beecroft, David Addison, Pacific Northwest National Laboratory, IEEE Computer society, 2005, p.34 to p.47.QsNet III an Adaptively Routed Network for High Performance Computing, Duncan Roweth, Trevor Jones, Quadrics Limited, Bristol, UK, IEEE Computer society, 2008, p. 157 to p. 164.

Thank You