Post on 18-Jan-2016
description
www.cactuscode.org www.gridlab.org
Introduction to Grid Computing
Ed SeidelMax Planck Institute for
Gravitational Physicseseidel@aei.mpg.de
Grid Intro 2
OutlineOutline
What is the Grid. What does the Grid mean for application
users? What is a Virtual Organization? Why is the Grid different from the Internet? What should application users and
developers be doing right now to be ready for the Grid?
High level overview of today’s testbeds.
Grid Intro 3
History and MotivationHistory and Motivation
1970s - 80s: Internet Email, ftp, etc
1980s - 90s: Remote-, Distributed-, Meta-computing Beginning to think of ways to exploit distributed resources US SC Centers Programs
– Had to access remotely– Think of harnessing together
See examples later SC95: I-Way
First large scale experiments Most failed, but I-Way a success!
Post I-Way: The Grid Explosion of activity, but still immature. Get prepared for
future
Grid Intro 4
Grid Computing: A New ParadigmGrid Computing: A New Paradigm
Computational Resources Scattered Across the World Compute servers (double each 18 months) Handhelds File servers Networks (double each 9 months) Playstations, cell phones etc…
How to take advantage of this for scientific/engineering simulations?
Harness multiple sites and devices Simulations at new level of complexity and scale, interacting with data
QuickTime™ and aPhoto - JPEG decompressor
are needed to see this picture.
QuickTime™ and a Photo decompressor are needed to see this picture.
Grid Intro 5
Grid vs. Internet/Web Services?Grid vs. Internet/Web Services?
We’ve had computers connected by networks for 20 years
We’ve had web services for 5-10 years The Grid combines these things, and brings
additional notions Virtual Organizations Infrastructure to enable computation to be carried
out across these – Authentication, monitoring, information, resource
discovery, status, coordination, etc
Can I just plug my application into the Grid? No! Much work to do to get there!
Grid Intro 6
All have to work for real applicationsCommunities Teams with a need to share data, or develop common code Communities can be very diverse
Experimentalists, computer scientists, simulation scientists, network researchers
All may need to work together to solve a specific problem May co-develop a code, or contribute modules that should work
together These communities may have very different resources
Within company or university Across the world
Grids aim to bring them together, allow them to harness their resources
These communities may form a type of (virtual) organization
Components for Grid ComputingComponents for Grid Computing
Grid Intro 7
Virtual OrganizationsVirtual OrganizationsResources Machines, networks, archives, file systems, etc at different sites What is a Virtual Organization (VO)?
groups of organizations that use the Grid to share resources for specific purposes
– EU DataGrid, Alliance, TeraGrid, SC02 Global Grid Testbed, etc Typically deploy same technology Deploy directory service: resources registered, and may be
“discovered”. Globus has MDS (Monitoring and Discovery Service)– GIIS (Grid Index Information Service) provides info for entire VO– GRIS (Grid Resource Information Service), installed locally, reports to
GIIS so people or applications can search GIIS for info Authentication: Certificates and Gridmap file
– Sometimes a VO offers a “certificate” for individuals and resources– Sometimes they use other VO’s certificates
So far, there is no Global GIIS that links together all VOs
Grid Intro 8
Software InfrastructureSoftware Infrastructure
Infrastructure: Globus Metacomputing Toolkit Low Level
Fundamental technologies needed to build computational grids. Security: logins, data transfer Communication Information (GRIS, GIIS) More generally: Grid Services
Middleware Data movers Resource Brokers Portals Application Monitoring systems
High Level Application Toolkits
We focus today on Globus, Cactus, and GridLab Other: Legion, Unicore, Juxta…
Grid Intro 9
Grid Aware Applications (Cactus example): Grid Enabled Modular Toolkits for Parallel
Computation: Provide to Scientist/Engineer, etc.. Plug your Science/Eng. Applications in! Must Provide, Register as Grid Services
Ease of Use: automatically find resources, given need! Distributed simulations: use as many machines as
needed! Remote Viz and Steering, tracking: watch what happens! Take advantage of infrastructure
Collaborations of groups with different expertise: no single group can do it! Grid is natural for this…
Components for Grid ComputingComponents for Grid Computing
Grid Intro 10
Example GridsExample Grids
GridLab Testbed Ten machines in Europe for developers of Grid tools
SC2001 ARG Testbed & Global Grid Testbed Collaboration Hastily assembled loose federation of world machines for SC2001
and now SC2002 demonstrations NCSA Virtual Machine Room and PACI Grid
Production resources TeraGrid (www.teragrid.org)
USA distributed terascale facility at 4 sites for open scientific research
Information Power Grid (www.ipg.nasa.gov) NASAs high performance computing grid
Grid Intro 11
Example GridsExample Grids
NSF PACI Grid
NorduGrid
Grid Intro 12
The NSF TeraGridThe NSF TeraGrid
Some Grids are tightly coupled facilities, like TeraGrid Some are testbeds, like Global Grid Testbed Collaboration Some are long term, loosely coupled: PACI Grid (includes
TeraGrid) All try to provide organized access to distributed resources Most are depending on Globus as the base infrastructure
2624
85
NCSASDSC
Caltech Argonne
13.6 Tflop facility
Grid Intro 13
What does this mean now for users and developers?What does this mean now for users and developers?
There is a grand vision of the future Collecting resources around the world into Vos Seamless access to them, with a single signon NEW applications to exploit of them in unique ways! Today we want to help you prepare for this
There is a frustrating reality of the present These technologies are not yet fully mature Not fully deployed Not consistent across even single Vos
But centers and funding agencies worldwide are pushing this very, very hard Better get ready now You can help! Work with your centers to get this deployed
Grid Intro 14
Getting Ready for the GridGetting Ready for the Grid
Need to start imagining how your applications can exploit the Grid (simplify use of HPC, provide more processing power, better access to data, allow new scenarios to forward your science, better collaborations).
Applications codes will need modifications for the Grid, depending on much of it you want to be able to exploit!
Write new codes with the Grid in mind. Consider using frameworks which are
already Grid-compatible.