University of Southampton

Clusters: Changing the Face of Campus

ComputingKenji Takeda

School of Engineering Sciences

Ian Hardy

Oz Parchment

Southampton University Computing Services

Simon Cox

Department of Electronics and Computer Science

University of Southampton

• Introduction• Clusters background• Procurement• Configuration, installation

and integration• Performance• Future prospects• Changing the landscape

Talk Outline

University of Southampton

Introduction• University of Southampton

– 20,000+ students (3000+ postgraduate)

– 1600+ academic and research staff

– £182 million turnover 1999/2000

University of Southampton

"to acquire, support and manage general-purpose computing, data

communications facilities and telephony services within available

resources, so as to assist the University to make the most

effective use of information systems in teaching, learning and research

activities".

University of Southampton

HEFCE Computational and Data Handling Project

• Existing facilities outdated and overloaded• £1.01 million total bid, including infrastructure costs and Origin 2000 upgrade

• Large compute facility to provide significant local HPC capability

• Large data store – several Terabytes• Upgraded networking: Gigabit to the desktop• Staff costs to support new facility

University of Southampton

Cluster Computing

• Extremely attractive price/performance• Good scalability achievable with high performance memory interconnects

• Fast serial nodes with lots of memory (up to 4 Gbytes) affordable

• High throughput, nodes are cheap• Still require SMP for large (>4 Gigabytes) memory jobs – for now

University of Southampton

Clusters at Southampton

• ECS: 8 node Alpha NT and 8 node AMD Athlon clusters• Social Statistics/ECS/SUCS: 19 node Intel PIII cluster• Chemistry: 39 AMD Athlon and 4 dual Intel PIII node cluster• Computational Engineering and Design Centre: 21 dual

node and 10 dual node Intel PIII clusters • Aerodynamics and Flight Mechanics Group: 11 dual node

Intel PIII cluster with Myrinet 2000• ISVR: 9 dual node Intel PIII Windows 2000 cluster• Several high throughput workstation clusters on campus

• Windows Clusters research

University of Southampton

User Profiles

• Users from many disciplines:– Engineering, Chemistry,

Biology, Medicine, Physics, Maths, Geography, Social Statistics

• Many different requirements:– Scalability, memory,

throughput, commercial apps

• Want to encourage new users and new applications

University of Southampton

Procurement• Ask users what they want - open discussion• General-purpose cluster specification • Open tender process• Vendors, from big iron companies to home PC suppliers

• Shortlist vendors for detailed discussions

Oct99 Aug00 Dec00 Jan01 Feb01 Apr01 Jun01

Bid Go-ahead PTQ Replies Clarifications Order Install

University of Southampton

Configuration

• Varied user requirements• Limited budget – value for money crucial• Heterogenous configuration optimum• Balanced system: CPU, memory, disk• Boxes-on-shelves or racks?• Management options: serial network, power strips, fast ethernet backbone

University of Southampton

IRIDIS Cluster• Boxes-on-Shelves• 178 Nodes

– 146 × dual 1GHz PIIIs– 32 × 1.5GHz P4s

• Myrinet2000– Connecting 150 cpu’s

• 100 Mbit/s fast Ethernet• APC Power strips• 3.2 Tb IDE-Fibre disk

University of Southampton

Installation & Integration

• Initial installation by vendor – Compusys plc• One week burn-in, still had 3 DOAs• Major switch problem fixed by supplier• Swap space increased on each node

No problems since• Pallas, Linpack, NAS benchmarks and user codes for thorough system shakedown

• Scheduler for flexible partitioning of jobs

University of Southampton

NAS Serial Benchmarks

0 50 100 150 200 250 300

Mops

LU

MG

SP

1 GHz Pentium III

1.5 GHz Pentium 4

Origin 3000 (R12K)

Origin 2000 (R12K)

Bigger is better

University of Southampton

Chemistry Codes

0 500 1000 1500 2000

w all clock time (secs)

Amber 6

gaussian94(II)

gaussian94 (I)

BossSG-R12K (400Mhz)

1 GHz Pentium III

1.5 GHz Pentium 4

Smaller is better

University of Southampton

Amber 6 Scalability

University of Southampton

Future Prospects

• Roll-out Windows 2000/XP service– In response to user requirements– Increase HPC user-base– Drag-and-drop supercomputing

• Expand as part of Southampton Grid– Integration with other compute resources on

and off campus– Double in size over next few years

University of Southampton

Changing the Landscape

• Availability of serious compute power to many more users – HPC for the masses

• Heterogenous systems - tailored partitions for different types of users easy to cater for

• Compatability between desktops and servers improved – less intimidating

• New pricing model for vendors – costs are transparent to the customer

Affordable, Expandable, Grid-able