Grids and the Home Institution

A Campus Grids Overview

by

Laura F McGinnis

Pittsburgh Supercomputing Center

26 June 2006 © 2006 PSC/CMU lfm@psc.edu

Agenda

What is a campus grid? Who uses campus grids?Why are they using them?Where are the resources coming from?When can resources be used as a grid?How can we set up a grid on our campus?Bonus: What next?

26 June 2006 © 2006 PSC/CMU lfm@psc.edu

What is a campus grid?

Heterogeneous dedicated and non-dedicated resources

Designed to enable resource sharing and (over time) cost saving

A system to federate compute and data resources across a campus

Abstracts the interface to each from users Resources contributed from diverse campus

organizations, but with centralized coordination

Thanks to David Wallom, Oxford University

Campus Windows Machines

(WSRF.NET)

UVA Campus Grid Overview

Web portal

OGCE / uPortal

UVA PubCookie

Server

ITC LDAP Server

GGF OGSA SAML Authz Service

MyProxy Server

PubCookie compatible

Campus Linux Machines

(NMI / GT4)

Existing We built

Grids at Indiana University

Gary

PurdueWest Lafayette

Indianapolis

Bloomington

Richmond

Chicago

1 TFLOPS SP – INGEN, IBM2 TFLOPS AVIDD – NSF, IBM2.2 PB Storage – STK, NSF, INGEN

Indiana/Purdue University

Partnership

26 June 2006 © 2006 PSC/CMU lfm@psc.edu

Who uses campus grids?

Harvard University, CrimsonGrid

University of Wisconsin-Madison, GLOW

University of Virgina, UVaCG University of Alabama-

Birmingham, UABGrid University of Oxford, UK e-

Science Campus Grids Indiana University, Hydra

Indiana University, Campus Grids at Indiana

Purdue University, Campus Grids at Purdue

Langston University, DOSAR Fermi Lab, SAMGrid Greek Research & Technology

Network, EGEE Texas Tech, TIGRE, THEGrid Cambridge University,

CamGrid

Global Grid Forum – GGF15 Campus Grids Workshop & Followup

University of Iowa: HawkGridUniversity of Michigan: M-Grid

Example Uses• ATLAS

– Over 15 Million proton collision events simulated at 10 minutes each

• CMS– Over 10 Million events simulated in a month - many more events

reconstructed and analyzed

• Computational Genomics– Prof. Shwartz asserts that GLOW has opened up new paradigm of

work patterns in his group• They no longer think about how long a particular computational job will take -

they just do it

• Chemical Engineering– Students do not know where the computing cycles are coming

from - they just do it

DOSAR – Grids on CampusJoel Snow Langston University

Desktop Analysis Stations

Institutional Analysis Centers

Regional Analysis Centers

Normal InteractionCommunication PathOccasional Interaction Communication Path

Central Analysis Center (CAC)

DAS DAS…. DAS DAS….

IAC ... IAC IAC…IAC

RAC….

RAC

DØ Remote Analysis Model (DØRAM)Fermilab

•Data and Resource hub•MC Production•Data processing•Data analysis

26 June 2006 © 2006 PSC/CMU lfm@psc.edu

Why are they using them?

Applications – Workflow Parallel programming

Loosely coupled – “pleasingly parallel” Tightly coupled - mpich-g

Coursework – Expand pool of resources available for class work Reduce bottlenecks during peak academic usage

Requirements for our GRIDRequirements for our GRIDData collected by each running experiment 1 PetaByte p.a. experiments with a High Energy physics user community

Consequences for our GRID: data driven GRID interoperability possible

data

people

CPUFermiLab, SAMGrid

Enabling Grids for E-sciencE

INFSO-RI-508833

EGEE Applications

Pilot New

26 June 2006 © 2006 PSC/CMU lfm@psc.edu

Where are the resources coming from?

Campus compute pools Underutilized departmental resources Student machines

University of Iowa: HawkGrid

University of Michigan: MGrid

26 June 2006 © 2006 PSC/CMU lfm@psc.edu

26 June 2006 © 2006 PSC/CMU lfm@psc.edu

When can resources be used as a grid?

Opportunistic Off-hours Scheduling by policy

Utilization of Idle CyclesUtilization of Idle Cycles

Red: total owner Blue: total idle Green: total Condor

Hydra – Indiana University

26 June 2006 © 2006 PSC/CMU lfm@psc.edu

Caveats: Things to Beware of

Security System administration Resource management – “fair share” No “plug and play” solutions (yet)

26 June 2006 © 2006 PSC/CMU lfm@psc.edu

How can we set up a grid on our campus?

Condor Globus Commercial Vendors

Univa Platform Computing Microsoft

26 June 2006 © 2006 PSC/CMU lfm@psc.edu

Bonus: What next?

Grids of Grids – interoperability Open Science Grid TeraGrid Campus Affiliates SURAGrid

Global Grid Forum

ArchitectureArchitecture

26 June 2006 © 2006 PSC/CMU lfm@psc.edu

26 June 2006 © 2006 PSC/CMU lfm@psc.edu

Acknowledgements

Jayanta Sircar, Harvard University, CrimsonGrid Sridhara Dasu, University of Wisconsin-Madison, GLOW Glenn Wasson, University of Virgina, UVaCG Jill Gemmill, University of Alabama-Birmingham, UABGrid David Wallom, University of Oxford, UK e-Science Campus Grids Arvind Gopu, Indiana University, Hydra Scott McCaulay, Indiana University, Campus Grids at Indiana Preston Smith, Purdue University, Campus Grids at Purdue Joel Snow, Langston University, DOSAR Valeria Bartsch, Fermi Lab, SAMGrid Ognjen Prnjat, Greek Research & Technology Network, EGEE Alan Sill, Texas Tech, TIGRE, THEGrid

26 June 2006 © 2006 PSC/CMU lfm@psc.edu

For More Information

Condor: www.cs.wisc.edu/condor Globus: www.globus.org TeraGrid: www.teragrid.org SURAGrid: www1.sura.org/3000/SURAgrid.html Open Science Grid: www.opensciencegrid.org

GGF: gridforum.org Campus Grids Workshop Slides:

www.psc.edu/~lfm/PSC/Grid/PGS-RG