The Anatomy of the Grid: An Integrated View of Grid

ArchitectureCarl Kesselman

USC/Information Sciences Institute

Ian Foster, Steve Tuecke

Argonne National Laboratory

Outline and Goals

Define what is (and isn’t) a grid in terms of a layered protocol architecture

Understand the role of protocols, services, APIs and tools in the deployment and use of Grids

Understand what role Globus plays Understand the role of other technologies

What is a Grid?

Grid language has been driven by genesis from metacomputing, but…

In practice, the Grid is about resource sharing and coordinated problem solving in dynamic, multi-institutional virtual organizations

Focus on how to enable, maintain and control the sharing of resources to achieve a common goal

Some Useful Definitions

Network Protocol A formal description of message formats and

a set of rules for exchange of messages Rules define sequences of message

exchange, and potentially resulting behavior Protocol may define state-change in endpoint

Network Enabled Services Defines a set of capabilities Protocol defines interaction with service

All services require protocols, although not all protocols are to services

More definitions

Resource Entity that is to be shared Provides some capabilities, that can be

accessed via interface (API) or protocol Application Programmer Interface (API) Software Development Kit (SDK)

Package that enables application development, consisting of one or more APIs, and programming tools

Protocols Make the Grid

Protocols and APIs Protocols enable interoperability APIs enable portability

Sharing is about interoperability, so … Grid architecture should be about protocols

Grid Services Architecture: Previous Perspective

Grid-enabled archives, networks, computers, display devices, etc.; associated local services

Protocols, authentication, policy, resource management, instrumentation, discovery, etc., etc.

GridFabric

GridServices

ApplnToolkits

Applns

...

… a rich variety of applications ...

Remoteviz

toolkit

Remotecomp.toolkit

Remotedata

toolkit

Remotesensorstoolkit

Async.collab.toolkit

Characteristics of Grid Services Architecture

Identifies separation of concerns Isolates Grids from languages and specific

programming environments Makes provisions for generic and

application specific functionality Protocols not explicit in architecture

fails to make clear distinction between language, service and networking issues

Layered Grid Protocol Architecture

Connectivity

Fabric

Resource

Grid

Application

User

Important Points

Being Grid-enabled requires speaking appropriate protocols Protocol only requirement, not reachability Protocols can be used to bridge local resources

or “local Grids” Intergrid as analog to Internet

Built on Internet protocols Independent of language and implementation

Focus on interaction over network Services exist at each level

Protocols, services and interfaces

Languages/Frameworks

Fabric Layer

Applications

Local Access APIs and protocols

Grid Service APIs and SDKs

Grid ServicesGrid Service Protocols

Resource APIs and SDKs

Resource ServicesResource Service Protocols

User Service ProtocolsUser Service APIs and SDKs

User Services

Connectivity APIs

Connectivity Protocols

How does Globus fit in?

Defines connectivity and resource protocols

Enables definition of grid and user protocols Globus provides some of these, others

defined by other groups Defines range of APIs and SDKs that

leverage Resource, Grid and User protocols

Fabric

Local access to logical resource May be real component, e.g. CPU, software

module, filesystem May be logical component, e.g. Condor pool

Protocol or API mediated Fabric elements include:

SSP, ASP, peer-to-peer, Entropia-like, and enterprise level solutions

Connectivity Protocols

Two classes of connectivity protocols underlie all other components

Internet communication Application, transport and internet layer

protocols I.e., transport, routing, DNS, etc.

Security Authentication and delegation Discussed below

Security

Protocols TLS with delegation

Services K5ssl, Globus Authorization Service

APIs GSS-API, GAA, SASL, gss_assist

SDKs GlobusIO

Resource Protocols

Resource management, Storage system access Network quality of service Data movement Resource information

Resource Management

Protocols GRAM+GARA (on HTTP)

Resource services Gatekeeper, JobManager, SlotManager

APIs and SDKs GRAM API, JavaCog Client, DUROC

Data Transport

Protocols Grid FTP, LDAP for replica catalog

Services FTP, LDAP replica catalog

APIs and SDKs GridFTP client library, copy URL API, replica

catalog access, replica selection

Resource Information

Protocol LDAP V3, Registration/Discovery protocol

Service GRIS

APIs & SDKs C API; JNDI, PerlLDAP, ….

Grid Protocols Grid Information Index Services

LDAP and Service registration protocol, … GIIS service LDAP APIs and specialized information API

Co-allocation and brokering GRAM (HTTP+RSL) DUROC service DUROC client API, end-to-end reservation API

Grid Protocols (cont)

Online authentication, authorization services HTTP MyProxy, Group policy servers Myproxy API, GAA API,

Many others (e.g.): Resource discovery (Matchmaker) Fault recovery

User Protocols

In general, there are many of these, they tend to be on off, and not well defined

Examples: Portal toolkits (e.g. Hotpage) Netsolve Cactus framework

Summary

Grids are about sharing, not making big MPI jobs

Well defined protocol architecture is essential to understanding and progress Provides a framework for figuring out where

the pieces fit Globus provides many basic protocols and

services Interoperability between different Grid

components