Distributed Systems Architecture Research GroupUniversidad Complutense de Madrid

EGEE UF4/OGF25Catania, Italy

March 2nd, 2009

State and Future Plans for GridWay Metascheduler

Eduardo Huedo

EGGE UF4 / OGF25Catania (Italy), March 2nd, 2009

2/10

Contents

Architecture

Components

Scheduling Policies

Interoperation through Adapters

Infrastructures Using GridWay

Status and Future Plans

More Information

3/10

PBS SGE

$>CLI

GridWay

.C, .java

DRMAA

.C, .java

Infrastructure

Gri

d M

idd

lew

are

Ap

plic

atio

ns

Globus Services

Gri

d M

eta-

Sch

edu

ler

• Advanced scheduling policies• Accounting• Fault detection & recovery

Ap

plic

atio

n-I

nfr

astr

uct

ure

dec

ou

plin

g

• LRM-like Command Line Interface• OGF DRMAA C, Java, Perl… bindings• OGF JSDL (POSIX & HTC profiles)• Job arrays, DAG workflows and MPI jobs

• Straightforward deployment•Deploy different kinds of grids• Interoperation between grids

• Highly dynamic & heterogeneous• High fault rate

Architecture

4/10

ExecutionManager

TransferManager

InformationManager

DispatchManager

RequestManager

Scheduler

Job Pool Host Pool

DRMAA library CLI

GridWay Core

File TransferServices

ExecutionServices

GridFTP RFTpre-WSGRAM

WSGRAM

InformationServices

MDS2MDS2GLUE

MDS4

Resource DiscoveryResource MonitoringResource DiscoveryResource Monitoring

Job PreparationJob TerminationJob Migration

Job PreparationJob TerminationJob Migration

Job SubmissionJob MonitoringJob ControlJob Migration

Job SubmissionJob MonitoringJob ControlJob Migration

Components

5/10

Matching resources for each job (user)PendingJobs

Grid Scheduling = Job + Resource Policies

Resource Policies

• Rank Expressions

• Fixed Priority

• User Usage History

• Failure Rate

Job Policies

• Fixed Priority

• Urgent Jobs

• User Share

• Deadline

• Waiting Time

Scheduling Policies

6/10

SGE Cluster

Users

Globus

GridWay

Globus gLite gLite

SGE ClusterPBS ClusterPBS Cluster

• Different execution interfaces (GRAM2, GRAM4, CREAM, ARC...)

• Different cluster configurations (shared FS or not)• Different transfer models (direct, reverse)• Different information models (Globus, GLUE...)• A&A through globally recognized CAs and DN’s

Globus Globus

SGE ClusterPBS Cluster

Interoperation through Adapters

7/10

Interoperation through Adapters

A real example of gwhost output (OGF23 demo):

8/10

Infrastructures Using GridWay

9/10

Status and Future Plans

Latest developments (for GridWay 5.5)• A single account to manage several grid users/proxies, useful for portals

• Execution adapter for CREAM

• Improved information adapter for EGEE

Other developments (related components)• Information and execution adapters for ARC, thanks to Dmytro Karpenko

(BITP)

• Execution adapter for SGE, thanks to CESGA

• Plug-in for SLA negotiation, thanks to BEinEIMRT project

Future developments• Scheduling policies for federated grids

• Execution adapter for DRMAA, i.e. for any DRMS supporting it

• Adapters for UNICORE

• Data caching and data management

10/10

More Information

Guides, howtos, mailing lists...• http://www.gridway.org

Globus guides• http://www.globus.org/toolkit/docs/4.2/4.2.0/execution/gridway

Open community development process• http://dev.globus.org/wiki/GridWay

Research publications• http://dsa-research.org

More about GridWay at EGEE UF4/OGF25• Tutorial about "Porting Applications with Globus GridWay”, here! next!

• Presentation "Vashra-T: Grid Ray Tracing for the Fusion Physics ASTRA Code", Tuesday 14:25

• Presentation "Porting AITALC product through a master/worker scheme", Wednesday 14:00

• Poster "Support for CREAM-based CEs in GridWay Metascheduler“, Tuesday 16:00

Any question now?