Tuning Parallel Applications to Accelerate Scientific Discoveries

Robert Henschelrhensche@indiana.edu

October 2009

Robert Henschel

Contents• PTI / High Performance Applications• Performance of Scientific Codes• IU and TeraGrid Compute Resources• Optimizing for IU's HPC Systems• Using TeraGrid HPC Systems• HPA is Here to Help

Robert Henschel

What this talk will be about• Making you aware of compute resources that you

can use for your work, to make you more productive.

• Introducing the High Performance Applications group and how we can help get research done faster.

• Give you examples of what we have done for researchers to make them more competitive in their field.

Robert Henschel

PTI and High Performance Applications• Pervasive Technology Institute

– Develop and deliver innovative information technology to advance research, education, industry and society.

– School of Informatics– School of Law– University Information Technology Services

• High Performance Applications– Part of the Digital Science Center of PTI– Part of the Research Technologies of UITS– Seven people that help IU researchers make efficient

use of IU and TeraGrid compute resources

Robert Henschel

Performance of Scientific Codes• Supercomputing, or High Performance Computing (HPC),

is not just for computer geeks!

• Performance for computer scientists– Amdahls law and scalability– Efficient usage of functional units of processors– Optimally using memory bandwidth– Trying to avoid I/O as much as possible

• Performance for researchers– When do I get the answer to my problem?– When does my job run and when is it done?

Robert Henschel

IU and TeraGrid Compute Resources• Two HPC systems at IU

– BigRed 30 TFLOPS (3000 cores)– Quarry 7 TFLOPS (1000 cores)

• Several special purpose systems– Small Cell B.E. Cluster– MDGRAPE-2 machine

• Several storage resources– IU Data Capacitor– GPFS, RFS, HPSS

• Policy of open access to compute resources

Robert Henschel

IU and TeraGrid Compute Resources cont'd

Robert Henschel

IU and TeraGrid Compute Resources cont'd• TeraGrid

– NSF funded HPC systems and support infrastructure– 11 resource providers– More than 1,500 TFLOPS (150,000 cores)

• Central allocation and support structure

Robert Henschel

Optimizing for IU's HPC Systems• Help researchers access the central systems and

determine what system to use• Install and optimize applications• Provide guidance on compiler and library optimization• Help with job submission, especially running many

thousands of jobs

Robert Henschel

Using TeraGrid HPC Systems• Low barrier of entry

• Identify if a problem and workflow will work on the TeraGrid

• Get a startup allocation• Use it and identify if it is worth pursuing this further• Submit a full allocation request

Robert Henschel

Contents – HPA is Here to Help• HPA is Here to Help

– What We Do

• Recent Examples– Integrating HPC Systems into an Electron

Microscope Workflow– Migrating Research in Gas Giant Planets from IU

to TeraGrid HPC Systems– Developing Computational Models to Predict

Drug-Drug Interactions

Robert Henschel

What We Do• Consulting about HPC system usage

– From start to finish– Optimize source code for architectures

• Help with TeraGrid allocation proposals• Adapting and creating workflows for new environments• Consulting for grant proposals

Robert Henschel

HPC Systems and an Electron MicroscopeGeneral Case

– Users have an instrument that produces a lot of data on a daily basis

– This data needs to be stored and analyzed

Electron Microscope in Simon Hall (IU Bloomington)– Microscope stores data on a Windows workstation– Researcher does quality checks on local workstation– IU Data Capacitor links workstations, IU HPC systems

and the IU long term archive together

Robert Henschel

Gas Giant Planets on the TeraGridGeneral Case

– Users have a set workflow for analyzing data– Locally available compute resources are not big

enough to keep up with demand

Understanding Gas Giant Planets– IDL is used to visualize simulation data

• Commercial software, IU Astronomy has a license– Simulation application needs to run on a large shared

memory system– TeraGrid and IU Data Capacitor tie this workflow

together

Robert Henschel

Predicting Drug-Drug InteractionsGeneral Case

– Researchers implement proof of concept research algorithms

– Scaling from proof of concept to production science is difficult

– The ability to add HPC expertise to grant proposals will make the proposal more competitive

Computational Models to Predict Drug-Drug Interactions– Drug exposure model developed in R– Scaling to real world data sets not possible without

using HPC systems– Porting to C and running on UITS hardware

Robert Henschel

What this talk was about• Made you aware of compute resources that you can

use for your work, to make you more productive.

• Introduced the High Performance Applications group and how we can help get research done faster.

• Gave you examples of what we have done for researchers to make them more competitive in their field.

Robert Henschel

AcknowledgmentsThis material is based upon work supported by the National Science Foundation under Grant Numbers 0116050 and 0521433. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation (NSF).

This work was support in part by the Indiana Metabolomics and Cytomics Initiative (METACyt). METACyt is supported in part by Lilly Endowment, Inc.

This work was support in part by the Indiana Genomics Initiative. The Indiana Genomics Initiative of Indiana University is supported in part by Lilly Endowment, Inc.

This work was supported in part by Shared University Research grants from IBM, Inc. to Indiana University.