PERFORMANCE COMPUTING FOR RESEARCH Frank J. Wessel, Tony Soeller, Evan Klinger, and Afshin Mahini...

PERFORMANCE COMPUTING FOR RESEARCH

Frank J. Wessel, Tony Soeller, Evan Klinger, and Afshin Mahini

Research Computing Support

Network and Academic Computing Services

University of California, Irvine

QuickTime™ and aGraphics decompressorare needed to see this picture.

NETWORK AND ACADEMIC COMPUTING SERVICES

(http://www.nacs.uci.edu)

• NETWORK & TELEPHONE– Network, wireless, remote user access– Telephone, cell sites, life-line modem– Campus directory services

• CAMPUS WIDE SUPPORT– Response center, training,– Instructional computing,– Software licensing, – Web development,– Calendaring, mailing lists,– Department support,– Research computing support


http://www.nacs.uci.edu/



RESEARCH COMPUTING SUPPORT http://www.nacs.uci.edu/rcs

• MISSION:– To facilitate the efficient use of information

technologies and campus resources in the UCI research community

• OBJECTIVES– Extend NACS support to research, – Promote awareness of campus resources,– Provide custom assistance as needed– Large and small “consumers”


http://www.nacs.uci.edu/rcs




MOTIVATION

• Increasing demands placed on researchers– Discipline specific needs– Technology/hardware/software trends– Expertise available within NACS/elsewhere

• Centralized vs. distributed support – More efficient use of resources– Machine room, networking, etc– Campus collaborations help win grants

• Shared expertise – Awareness of projects and implementations– Disseminate information to campus– Promote researcher liaisons


RESEARCH “DEMOGRAPHICS” AT UCI

• Few examples of centralized support

• Mostly at the department/IRU/ORU level

• School support largely for Beowulf clusters

• SMP installations are few

• SDSC (UCSD) is a notable exception


Education

ICSGSM Arts

SocEco

Engineering

Bio Sci SocSciPhysSci

Humanities

CoM

0

20

40

60

80

100

120

140

160

180

200

# FTE Faculty Vs. Research Funds-by Academic Unit '01-'02

Res. Funds $M

# FTE Reg. Faculty

UCI PERFORMANCE COMPUTING RESOURCES

• NACS– ESMF supercomputer, 7x8cpu, 32cpu– Krein 32cpu, SGI Origin 2000– MPC - 200+cpu Beowulf– GRADEA - 16cpu Beowulf

• SDSC/NPACI– Academic Associates Program– “Access grants” for HPC resources

• Other – NCAR, NCSA, DoE Labs, etc


NACS MACHINE ROOM

4500 sq. ft. floor spaceClimate controlled24/7 operator attended100 kVA UPS

krein.math32p

SGI Origin 2500

http://www.nacs.uci.edu/computing/hpc/krein.html

http://www.nacs.uci.edu/computing/hpc/krein.html

MPC Beowulf Clustermpc.uci.edu

16p Athlons150p “night nodes”52p dual Opterons

GigE

mpc.uci.edu

• Campus collaboration with researchers– 25% “donation” to mpc– 75% dedicated queue– Free sys. admin., – Free “co-location” – Software– Racks/switches

• Installed software– OPBS, MPICH, PGI CDK, Gaussian, SAS,– Mathematica, Fluent, MATLAB, IDL,– CFDACE+,IMSL, etc.


http://mpc.uci.edu/

http://mpc.uci.edu/

Computational Environmental Sciences LaboratoryDepartment of Mechanical and Aerospace Engineering

Air Quality Modeling of the South Coast Air Basin of California Tyler August Reid, Marc Carreras, Prof. Donald Dabdub

QuickTime™ and aMicrosoft Video 1 decompressorare needed to see this picture.

The CIT Airshed Model

( ) ( )chemistry

km

aerosol

km

kssources

kmk

mkm

km

t

Q

t

Q

t

QQKuQ

t

Q⎟⎟⎠

⎞⎜⎜⎝

⎛+⎟⎟

⎠

⎞⎜⎜⎝

⎛+⎟⎟

⎠

⎞⎜⎜⎝

⎛+∇⋅∇=⋅∇+

∂

∂

∂

∂

∂

∂

∂

∂

sin/

1100 m

40 m150 m

0 m

310 m

670 m

80 Cells

30Cells

47 Gas Species152 Aerosols: 19 species, 8 sizes121 Reactions

47 Gas Species152 Aerosols: 19 species, 8 sizes121 Reactions

Each Cell: 5 x 5 km2

General Dynamics Equation:

Aero

sol P

hase

O3NO3

NO

O3

hv

HO2

H2ONH3

HNO3

SO2

H2SO4

OH

CO, RH, RCHO

SOA

N2O5

NO3 NO2

Aerosol Phase

H2O

RO2

QuickTime™ and aAnimation decompressorare needed to see this picture.

One Time Purchase vs. 4 Sequential Quarterly Purchases

Throughput

1218

2436

phased purchase

one-time purchase

1200

1800

2400

3600

672

1447

2222

3772

0

500

1000

1500

2000

2500

3000

3500

4000

Months

EARTH SYSTEM MODELING FACILITY (

ESMF)•7 each, p655 8-CPU MCMs, 1.5 GHz, 16 GB (32GB)

•1 each, p690 32-CPU SMP cluster, 1.5 GHz, 64 GB

http://www.ess.uci.edu/esmf




32 TB Disk:RAID 5 +LUSTRE

SINTINAGFS

GNBD

ESMF MISSION

To simulate climate change and answer questions how global warming, man-made pollutants, polar-ice movements, chemical cycles, etc. impact the Earth and its inhabitants over future decades and centuries.

ESMF

7EACH, 8-CPU, P655s, 1 EACH P690, 32 TB RAID STORAGE

Power4 Architecture

Shared L2

Chip to chip

8-way SMP on a Multichip Module

(MCM):The basic processor building block of the

pSeries 690

40+GFLOPS /MCM

World's fastest 64-bit processor

First SMP server system-on-a-chip

Reliability, Availability, Serviceability on the processor

IBM copper and SOI technology delivers more performance at lower power levels, saving on licensing fees and operating costs

POWER 4: The Heart of pSeries' Performance Advantage

2000 Technology Award "Advanced Chip Projects"From: Micro Design Resources/Microprocessor Report

POWER464-bit 1.1 or 1.3GHz

Microprocessor

LPAR

ESMF ACQUISITION

• Earth System Modeling Facility (ESMF)• High Performance (Super) Computer

– Benchmarked on CCSM and CAM codes

• High Performance Disk System- 32 TB disk, single name-space

• Network Interconnect– As fast as possible

• Purchase Price $1,000,000.00– Including tax, maintenance, and shipping

ESMF HARDWAREACQUISITION (contd.)

• 12/20/02 - MRI Pre-proposal to RGS• 1/23/03 - Proposal submitted to NSF• 5/29/03 - NSF award to UCI• 7/16/03 - RPF issued to vendors• 8/5/03 - Bidders conference• 9/17/03 - Bids received

– HP - 96 CPU, dual I2 Beowulf cluster, w/ Myrinet– SGI - 96 CPU Altix, I2 SMP cluster– IBM - 88 CPU, Power 4+, SMP cluster– SUN - requested private negotiations

• 10/1/03 - IBM selected• 12/1/03 - ESMF delivered

Who

Prof. Charles Zender

Atmospheric Physicist

Microphysical and general circulation models

Trace gases, aerosols,dust, clouds

Satellite remote sensing

Open Source Initiative

"The ESMF IBM Supercomputer will provide sustained

compute capability, speed and storage capacity necessary to best meet the challenges involved in understanding the atmosphere and its interactions with the Earth system, as well as developing methods for predicting its behavior. The ESMF will also allow researchers to pursue data-

intensive research utilizing the large geophysical datasets from current and next generation

numerical models and satellite observations.

Who

Prof. J. Keith Moore

Oceanographer

Biogeochemical cycles

Marine biota influence

Marine ecosystem dynamics

Satellite remote sensing

HIPERWALL

32 TB RAID

PRO-MICRO

7 IBM P655

CLUSTERS

1 IBM P690

CLUSTER

Who

• Prof. Falko Kuester• Scientific visualization• Real-time computer graphics• Virtual reality• Collaborative virtual environments• Distributed/remote visualization• Biomedical imaging• Computer aided design

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

Who

• Prof. Steven Jenks• Parallel and distributed

computing• Large-scale distributed

memory systems• Thread migration to exploit

locality• Real-time distributed systems

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Consortium of Universities for the Advancement of Hydrologic Science,

Inc.(CUASHI)

• Proposal to CUASHI to support the national community of hydro-geologists

• UCI is one of four finalists

• Faculty in Civil and Environmental Engineering and Earth System Science

• Develop a National Center for Hydro-Geologic Synthesis (NCHS)

NATIONAL CENTER FOR HYDRO-GEOLOGIC SYNTHESIS

• Real-time integration of process models and visualizations

• Capable of rapidly deploying evolving data models

• Provide and online collaborative environment for complex modeling.

• http://hydis8.eng.uci.edu/hydis-unesco/

http://www.chrs.nacs.uci.edu/

http://www.chrs.nacs.uci.edu/

Who

• Prof. Saroosh Saroosian• Hydrology• Hydrometeorology• Hydroclimate modeling• Remote sensing• Water sources management



Who



• Prof. Bisher Imam• Hydro-logic data and

information systems• Watershed processes• GIS modeling and analysis

RCS CONSULTING

• Software specific requests• Grant-writing activities

– Hardware, software

• New technologies– Videoconferencing– Visualization– Benchmarking

• Training• Access to research forums


TRAINING

• NACS RCS summer workshops

• Workshop1 - Supercomputing resources and HPC programming

• Workshop2 - Data analysis and visualization

• Workshop3 - Computational software in biological and medical research


NACS RESEARCH SOFTWARE

• NACS “licensed software”

• Coordinate “critical-mass usage”


• Mathematical– Mathematica– Matlab– Maple– IMSL (on GradEA)

• Statistical– SAS– SPSS– SigmaPlot/SigmaStat– S-Plus

• GIS & Image Processing– ArcGIS / ArcIMS / ArcPad– IDL (on GradEA)– Matlab

• Laboratory– LabVIEW– Matlab

• Bioinformatics– Lasergene

NACS RESEARCH SOFTWARE

MATLAB: Import DataMATLAB: Import DataImporting from Excel:

1) Goto File2) Click on Import3) Select the desired file4) Click “ok”5) And Import Wizard will pop up6) Preview to see if data are

imported correctly7) Click “Finish”

Data are loaded in workspace

MATLAB Plotting BasicsMATLAB Plotting Basics

x = 0:0.05:6 plot(x, cos(x),'k',x,1./cosh(x),'k',[4.73 4.73],[-1 1],'k')

xlabel('x')

ylabel('Value of functions')

title('Visualization of two intersecting curves')

text(4.8,-0.1,'x=4.73')

text(2.1,0.3,'1/cosh(x)')

text(1.2,-0.4,'cos(x)')

legend('cos(x)', '1/cosh(x)',3)

Octave

Open source mathematical package

OpenDXOpen source visualization package

Most earth and space science data sets contain multiple parameters. Along with the temperature data there are horizontal winds. To add this data to the picture, place an Import module on the canvas

CONCLUSIONS

Be involved, understand research needs


Remain flexible, be willing to compromise

Think big, take risks

Expand your horizons, but be patient

PERFORMANCE COMPUTING FOR RESEARCH Frank J. Wessel, Tony Soeller, Evan Klinger, and Afshin Mahini...

Documents

Transcript of PERFORMANCE COMPUTING FOR RESEARCH Frank J. Wessel, Tony Soeller, Evan Klinger, and Afshin Mahini...