Post on 31-Dec-2015
description
Distributed Systems Architecture Research GroupUniversidad Complutense de Madrid
EGEE UF4/OGF25Catania, Italy
March 2nd, 2009
State and Future Plans for GridWay Metascheduler
Eduardo Huedo
EGGE UF4 / OGF25Catania (Italy), March 2nd, 2009
2/10
Contents
Architecture
Components
Scheduling Policies
Interoperation through Adapters
Infrastructures Using GridWay
Status and Future Plans
More Information
3/10
PBS SGE
$>CLI
GridWay
.C, .java
DRMAA
.C, .java
Infrastructure
Gri
d M
idd
lew
are
Ap
plic
atio
ns
Globus Services
Gri
d M
eta-
Sch
edu
ler
• Advanced scheduling policies• Accounting• Fault detection & recovery
Ap
plic
atio
n-I
nfr
astr
uct
ure
dec
ou
plin
g
• LRM-like Command Line Interface• OGF DRMAA C, Java, Perl… bindings• OGF JSDL (POSIX & HTC profiles)• Job arrays, DAG workflows and MPI jobs
• Straightforward deployment•Deploy different kinds of grids• Interoperation between grids
• Highly dynamic & heterogeneous• High fault rate
Architecture
4/10
ExecutionManager
TransferManager
InformationManager
DispatchManager
RequestManager
Scheduler
Job Pool Host Pool
DRMAA library CLI
GridWay Core
File TransferServices
ExecutionServices
GridFTP RFTpre-WSGRAM
WSGRAM
InformationServices
MDS2MDS2GLUE
MDS4
Resource DiscoveryResource MonitoringResource DiscoveryResource Monitoring
Job PreparationJob TerminationJob Migration
Job PreparationJob TerminationJob Migration
Job SubmissionJob MonitoringJob ControlJob Migration
Job SubmissionJob MonitoringJob ControlJob Migration
Components
5/10
Matching resources for each job (user)PendingJobs
Grid Scheduling = Job + Resource Policies
Resource Policies
• Rank Expressions
• Fixed Priority
• User Usage History
• Failure Rate
Job Policies
• Fixed Priority
• Urgent Jobs
• User Share
• Deadline
• Waiting Time
Scheduling Policies
6/10
SGE Cluster
Users
Globus
GridWay
Globus gLite gLite
SGE ClusterPBS ClusterPBS Cluster
• Different execution interfaces (GRAM2, GRAM4, CREAM, ARC...)
• Different cluster configurations (shared FS or not)• Different transfer models (direct, reverse)• Different information models (Globus, GLUE...)• A&A through globally recognized CAs and DN’s
Globus Globus
SGE ClusterPBS Cluster
Interoperation through Adapters
7/10
Interoperation through Adapters
A real example of gwhost output (OGF23 demo):
8/10
Infrastructures Using GridWay
9/10
Status and Future Plans
Latest developments (for GridWay 5.5)• A single account to manage several grid users/proxies, useful for portals
• Execution adapter for CREAM
• Improved information adapter for EGEE
Other developments (related components)• Information and execution adapters for ARC, thanks to Dmytro Karpenko
(BITP)
• Execution adapter for SGE, thanks to CESGA
• Plug-in for SLA negotiation, thanks to BEinEIMRT project
Future developments• Scheduling policies for federated grids
• Execution adapter for DRMAA, i.e. for any DRMS supporting it
• Adapters for UNICORE
• Data caching and data management
10/10
More Information
Guides, howtos, mailing lists...• http://www.gridway.org
Globus guides• http://www.globus.org/toolkit/docs/4.2/4.2.0/execution/gridway
Open community development process• http://dev.globus.org/wiki/GridWay
Research publications• http://dsa-research.org
More about GridWay at EGEE UF4/OGF25• Tutorial about "Porting Applications with Globus GridWay”, here! next!
• Presentation "Vashra-T: Grid Ray Tracing for the Fusion Physics ASTRA Code", Tuesday 14:25
• Presentation "Porting AITALC product through a master/worker scheme", Wednesday 14:00
• Poster "Support for CREAM-based CEs in GridWay Metascheduler“, Tuesday 16:00
Any question now?