High Performance Computing at EPCC Alan D Simpson Technical Director

1October 2003HPC@EPCC

High Performance Computing at EPCC

Alan D Simpson

Technical Director

Telephone: +44 131 650 5120 Fax: +44 131 650 6555 Email: [email protected]

http://www.epcc.ed.ac.uk/


Overview

Background

HPC Facilities at EPCC

HPCx

– Current Status

– HPCx and the Grid

Training and Research in HPC

Summary


EPCC

Founded in 1990 as a focus for the University of Edinburgh activities in HPC

Mission“to accelerate the effective exploitation of novel computing

in industry, academia, and commerce”

One of leading HPC Centres in Europe 65 staff

– 40 applications consultants + support staff

Income £2.7M per annum; 30% from Industry Academic and industrial clients from UK, Europe and

beyond


Technology Transfer

Technology Transfer

Academic:o National HPC Facilities o Researcho Support

Training:o Academiao Industryo MSc

Europe:o Visitor Programmeso Technology Transfero Strategic Planning

Industry:o Projectso Consultancyo Middleware


Industrial Consultancy

Provide project-based consultancy to industry and commerce

Over 30 clients in 3 years Large enterprises...

– eg, UK Met Office, Sun, C&G, AEA, Cisco

...to local SMEs– eg, Weidlinger, Quadstone, Jardine

40% of technical staff Funded by direct contracts with business, local

government and European Commission


Industrial Clients

USA:o Cisco Systemso Cray Research Inco Schlumberger Geoquesto Sun Microsystems

Japan:o Fujitsu Research Laboratorieso Hitachi

UK:o AEA Technologyo AlphaData Ltdo Applied Research & Technology Ltdo Avro International plco British Aerospace plco CN Software Ltdo Cray Research (UK) Ltdo Crown Officeo DTIo Digital Equipment Corpo Edinburgh Old Town Renewal Trusto Edinburgh Petroleum Services Ltdo Enterpris Ltdo EPSRCo High Speed Productions Ltdo Integriti Solutions Ltdo Kwik-Fit Holdings plco LEELo MCS/Hampcoo Peter Tilling Plastics Ltdo Quadstone Ltdo Rolls Royce plco SCI Ltdo Scottish Enterpriseo Scottish Officeo SIAS Ltdo Silicon Graphics (UK) Ltdo UK Meteorological Officeo Upstream Systems Ltdo 3L Ltd

Europe:o AGIP S.p.A, Italyo Digital Equipment BV, Irelando European Commissiono Hitachi Dublin Laboratoryo Kjaergaard Industri Automatico Statoil, Norway


European Programmes

Collaborative research

– HPC-Europa: EPCC coordinates pan-European visitor programme

– DEISA: connecting national centres across Europe

IST (industrial) projects– EUTIST-IMV: co-ordination of 80 machine vision

organisations– Gridstart: co-ordination of all EU Grid development

projects


• 1997 Cray T3E (344 processors)• 1997 Hitachi SR2201• 2000 Sun UltraSPARC III Cluster

HPC Facilities at EPCC

• 1990 Meiko i860 CS (64 processors)

• 1991 TMC CM-200 (16K processors)

• 1992 Meiko i860 CS (16 processors)

• 1982 ICL DAPs• 1986 Meiko T800 CS (400 processors)• 1988 AMT DAP608

• 1994 Cray T3D (512 processors)

• Cray Y-MP

• 1995 Meiko CS-2

• 2002 Sun E15000 (54 processors)• 2002 IBM p690 Cluster (1280 processors)• 2004 QCDOC


UoE HPC Service

Funded by £400K JREI grant– awarded to EPCC in 1998– freely available to local researchers

Service based on Sun SMP clusters– familiar software and easy porting– recently upgraded to Sunfire E15K– large memory and CPU with a single

system image

EPCC is a Sun Centre of Excellence in HPC and Grid Computing


QCDOC

QCDOC is a collaborative project to develop a special-purpose computer for QCD– involving EPCC, Physics,

Columbia University, IBM,… QCD: Quantum ChromoDynamics

– key part of Standard Model of particle physics– has very extreme computing requirements

Price-performance is critical– may be cheaper to design special purpose machines for

particular problems– only pay for what you use– put extra effort into what is important to you


QCDOC

Each node is small and consists of a single specially designed chip plus some memory– very large numbers of

nodes are possible

Equivalent general purpose machine would be huge and expensive

Difficulty of chip design reduced by including components (eg, CPU) from IBM design library

10TF machine to be installed at EPCC in 2004


HPCx Overview

UK’s major HPC facility, funded by EPSRC £53M/6 year contact awarded to UoE HPCX Ltd

– wholly-owned subsidiary of University of Edinburgh– work subcontracted to CCLRC (DL), EPCC and IBM

Largest academic supercomputer in Europe– doubling in performance every 2 years


HPCx Objectives

Capability computing for world-leading science– Capability computing: jobs which use a significant

fraction of the resource, eg, at least 512 CPUs

Maximise benefits to the UK’s computational science and engineering community

IBM technology roadmap:– 12/02: 40x32-way Regatta H frames + Colony Switch

• initially #9 on Top 500 list

– 07/04: 48x32-way Regatta H+ frames + Federation switch

– 11/06: 96x32-way Regatta H+ frames + Federation switch

Science support is key for effective use


Partnership

EPCC and CCLRC– are partners in C3ES (Consortium for Capability

Computing and e-Science)– providing science support and systems management for

HPCx– underpinned by MoU between UoE and CCLRC– combines Europe’s foremost academic HPC, e-Science

and technology transfer centres– significant experience of:

• operating national HPC services• developing capability applications

– the strongest UK partnership ever to support scientific computing


Virtual Organisation

Outreach

Life sciences

New applications

Applications Support

Helpdesk Training

Liaising with users

Users

Technology

Software EngineeringUnderpinning technology Grid/e-Science

Systems & Networking Flexible and responsive capability service

Smooth transitions between phases

Terascaling Capability applications

Scalable algorithms Performance optimisation


HPCx and the Grid

Key responsibility for Software Engineering team HPCx is committed to support access via Grid

– currently provided through Globus 2– Globus 3 support when appropriate

HPCx is key part of UK collaboration with Extensible Teragrid Facility project in the US– focus is exploiting unique features of Grid + HPC systems

for capability computing– initial experiment planned for SC2003

• RealityGrid computational steering

• HPCx is major compute resource


HPCx Status

HPCx builds on significant complementary experience at EPCC and DL

Very successful start– averaging >75% utilisation– …with capability usage already up to 35%

Committed to e-Science and the Grid– ETF experiment at SC2003

HPCx is focussed on capability computing– world-class service for world-class research


MSc in HPC

£400K grant from UK research council– runs for 5 years– just started year 3

One of a very few such courses in the world Each year an increasing number of students,

especially overseas students


Training in HPC

Courses include– Fundamental Concepts of HPC – Practical Software Development – Message Passing Programming – Shared Memory Programming – Parallel Decomposition– Applied Computer Science– Object Oriented Programming for HPC – Exploiting the Computational Grid – Applied Numerical Algorithms – Performance Optimisation – Scientific Visualisation

Remote runs at, eg, Cambridge, Daresbury, …


HPC Research

Java Grande Forum– EPCC leads the benchmarking activity– including parallel benchmarks and language comparisons– have taught Java tutorials at Supercomputing

OpenMP– EPCC a full member of OpenMP

Architecture Review Board– OpenMP microbenchmarks

• tests quality of the compiler implementation

• becoming a de-facto standard


HPC Research

JOMP– an OpenMP-like standard for Java– research implementation available for download

Mixed Mode– combined OpenMP + MPI becoming popular– topic of investigation at EPCC for over 3 years

Single Sided MPI– EPCC produced implementations for Cray, Sun, …

Optimised Libraries– BLAS, FFTs, ScaLAPACK,…


Summary

EPCC is multidisciplinary and multi-funded– ... supporting a large spectrum of activities ...– … and a critical mass of expertise

Proven track record in Technology Transfer– business-like approach benefits whole organisation

New initiatives– MSc in HPC– European programmes– Grid middleware– HPCx

EPCC has a unique breadth of expertise

High Performance Computing at EPCC Alan D Simpson Technical Director

Documents

Transcript of High Performance Computing at EPCC Alan D Simpson Technical Director