Globus Overview CS-780-3 Lecture Notes In courtesy of Andy Kowalski.
-
Upload
elisabeth-floyd -
Category
Documents
-
view
216 -
download
0
Transcript of Globus Overview CS-780-3 Lecture Notes In courtesy of Andy Kowalski.
What Defines a Grid?
“We will probably see the spread of ‘computer utilities, which, like present electric and telephone utilities, will service individual homes and offices across the country”
-Len Kleinrock. 1969 [1]
What Defines a Grid?
“A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities.”
- Ian Foster and Carl Kesselman, 1998 [2]
Ian Foster’s Three Point Checklist (2002)
A Grid is a system that:1. “coordinates resources that are not
subject to centralized control” [2]2. “using standard, open, general-
purpose protocols and interfaces” [2]
3. “to deliver nontrivial quantities of service” [2]
Globus Project
Multi-institutional research effort on the technology required to build a grid or grids
Mathematics and Computer Science Division at Argonne National Laboratory
Information Sciences Institution at the University of Southern California
University of Chicago University of Edinburgh Center for Parallel Computers
Principle Investigators Ian Foster - Argonne National Laboratory and
University of Chicago Carl Kesselman - University of Southern California
Globus Project
Seek to understand application requirements for a usable grid and …Conduct research and development to create the technologies required to build a grid Globus toolkit
Globus ToolkitBag of services Developers can use these to build grid applications
Makes use of existing standards when possible Interfaces and implementations
Distinguishes between local and global servicesDefines interfaces to manage heterogeneity Does not hide heterogeneity
Information service to deal with constant changes Utilization, failures, new resources, etc.
Hourglass Principle
A layered architecture with and hourglass shapeGrid services should provide a simple well-defined interface between applications and local services
GRAM
Globus Resource Allocation ManagerProvides a single interface for requesting and using remote system resources for executing jobsEach GRAM is responsible for the resource management systems local to it
LSF (Load Sharing Facility), Condor, PBS Provides a standard interface to these different systems
Applications can access these different systems using a standard API
Within GRAM resource requests are described by the Resource Specification Language (RSL)
GRAM
Application
Broker
InformationService
LSF
Co-allocator
GRAM
PBS
GRAM
RSL
Queries & Info
Ground RSL
Single Ground RSL - Ground RSL is an expanded description of the application’s request. It may represent more than on job.- Single Ground RSL is a description of a singe job created from the Ground RSL. There may be more than one created from the original Ground RSL.
RSL
Resource Specification LanguageProvides a common language to describe resources Uses attribute, value pairs
A simple example& (executable = a.out (* <-- that is an unquoted
literal *)) (directory = /home/nobody ) (arguments = arg1 "arg 2") (count = 1)
Nexus
Low-level communications API that supports other communication librariesCan select alternative low-level protocols based on network topology Message passing, IP, shared memory Can introduce encryption based on source and
destination of a message
Intended for use by compiler writers and library developers CC++, HPC++, and MPICH-G
MDSMetacomputing Directory Service (GT1)Monitoring and Discovery Service (GT2&3)Provides information about the systems in the grid CPU type, memory, OS, network, etc. Queue information, utilization, etc
Basically a local LDAP serverMDS ties all the LDAP servers together so everyone has access to the same informationHas an API for discovering, publishing, and accessing information
MDS Architechture
Grid Index Information Service (GIIS)
Provides an aggregate directory of lower level data
Grid Resource Information Service (GRIS)
Runs on a resource and acts like a content gateway to the resource
Information Provider (IP) Interface to data
collection services
GSI
Globus Security InfrastructureDeals only with authentication Users and services
Uses x.509 certificates Also used username and password (rlogin)
Uses Secure Socket Layer (SSL) for the connection and authentication No longer true with GT3
All Globus services use GSI to authenticate users
HBM
Heartbeat MonitorAPI to register with the service and collect dataService expects updates from clients that have registered If no update is received, attempts to
determine if the service itself is down or if there are other factors
GASS
Global Access to Secondary StorageAllows programs to use the standard C I/O library to read and write files on remote computersReads
File is copied locally and then read
Writes File is written locally and then copied to their
destination
Appends Communicated to the remote file as soon as it is written
GEM
Globus Executable ManagementSupport identification, location, and creation of executables in heterogeneous environmentsPre-stages executables to multiple remote sites Removes them after execution
GridFTP
Secure data transfer protocol GSI authentication
Optimized for high-bandwidh wide-area networks Parallel streams - multiple data channels
Partial file transferThird-party transfers Server to server
Globus Toolkit 3
Based on the Open Grid Services Architecture (OGSA) Attempts to leverage industry standards Makes use of web services
Preserves backward compatibility with GT2 Future releases will begin eliminating
compatibility
Client interfaces to services are now WSDL/OGSI based
Issues
Protocols and interfaces Where the GT1 and GT2 implementation of a grid
service was inadequate for certain grids, others would developed competing services using different interfaces and protocols Defeated the goal of compatibility
OGSA/OGSI and web services are hoped to resolve this
Don’t need the whole thing Want to use GSI, need to install all of Globus COGs helped eliminate this burden
Use only what you need
Issues
Globus originally seemed focused on compute gridsOther groups were building data grids data grids represent a larger customer base
GridFTP was over extended Was once proposed that it be used for more
than file transfers Message passing
VO Management
References
[1] Ian Foster. What is the Grid? A Three Point Checklist. July 20, 2002.[2] Ian Foster and Carl Kesselman, editiors. “The Grid: Blueprint for a New
Computing Infrastructure.” 1998.[3] Ian Foster and Carl Kesselman. The Globus Project: A Status Report.
Proceedings of IPPS/SPDP’98 Heterogeneous Computing Workshop. 1998.
[4] Ian Foster and Carl Kesselman. Globus: Ametacomputing Infrastructure Toolkit. International Journal of Supercomputer Applications. 1997
[5] Globus Project. Globus Toolkit 2.2 MDS Technology Brief. January 30, 2003.
[6] Globus Project. Community Authorization Service (CAS) Overview. December 18, 2001.
[7] Ian Foster. Open Grid Services Architecture. Plenary Talk at CHEP 2003. March 26, 2003.
[8] Ian Bird. Deploying the LHC Computing Grid. Plenary Talk at CHEP 2003. March 26, 2003.