11 Application of CSF4 in Avian Flu Grid: Meta-scheduler CSF4. Lab of Grid Computing and Network...

11

Application of CSF4 in Avian Flu Grid:

Meta-scheduler CSF4.

Lab of Grid Computing and Network SecurityJilin University, Changchun, China

Hongliang Li (Simon) 2010-9-13

PRAGMA 19 workshop, Changchun, Jilin, China, Sep.13-15, 2010.

22PRAGMA 19 workshop, Changchun, Jilin, China, Sep.13-15, 2010.

44

CSF4 introduction

• Cross-domain meta-scheduler (grid-enable)• Grid protocol (portable)

– WS-GRAM, pre-WS-GRAM– Organizing resources from different domain under control of

diverse local schedulers

• Scheduling plugin framework (extendable)– Default plugin– Arrayjob plugin– Workflow plugin– DataAware plugin– OPAL service plugin– Parallel job plugin


55

CSF4 modules

• Job Service, Queue Service, Resource Managers• Supporting diverse local schedulers by grid protocols


Local Machine

PBS SGE CondorLSFLocal

MachinePBS SGE Condor

: Adapter : Local Scheduler

CSF4 Services

Queuing Service

Resource Manager LSF Service

GramPBS GramCondorGramFork GramSGE

WS-GRAM

gabd

Resource Manager Factory Service

Job Service

Reservation Srevice

GT2 Environment

GateKeeper

GramPBS GramSGE GramCondorGramFork

Resource Manager Gram Service

WS-MDSMeta Information

Grid Envi ronment

GramLSF

66

Scheduling framework

• Support multiple scheduling plugins co-operate together


77

Default and Arrayjob plugins

• Arrayjob consists of multiple subjobs(SIMD)


Job Description Job Job Object in Memory

MetaScheduler

Default Plugin

Job

Job

Job

Cluster

Cluster

Cluster…….

…….

…….

MetaScheduler

Array Job Plugin

Cluster

Cluster

Cluster

…….

Dispatch Split & Dispatch

Job

(1) (2)

Job

Job

Subjob

Subjob

Subjob

88

Two plugins working together

• Workflow jobs are spitted to subjobs by Workflow plugin• DataAware plugin allocate resources for these subjobs


File location info/

operations

Non workflow job(RSL) Workflow job (XPDL)

Ready job Non ready job

Real job (RSL) Available hosts

...

Workflow Plugin Data Aware Plugin

CSF4 Framework

..

.

Job Dispatch Resource

List

Job List

Gfarm APIs

map

1010

Integration of CSF4 and OPAL

• OPAL-CSF4 biomedical cloud– Enable large scientific applications (Virtual

screening, Autodoc, 2000 Arrayjobs)

– OPAL deals with service management and user interfaces

– CSF4 deals with cross-domain job scheduling


1111

OPAL-CSF4 cloud model

• CSF4 as a job manager of OPAL


1212

System structure


• Application management

• Cross-domain scheduling

• Input/Output file transfer

1313

CSF4 stagein&stageout


User

Cluster

Cluster

Cluster

Input DataOutput

Data

Manual Stage In

Submit Job

Manual Stage Out

1414

CSF4 stagein&stageout


User Cluster

Cluster

Cluster

Submit Job

Input Data

Output Data

Submit Job

Gridftp

MetaScheduler

1515

Improvements of CSF4

• Cross-domain dynamic file transfer• Recursively transmit files and folders for each job

(subjob)

• Job re-submission• Max walltime

• Default values in configuration file• User defined with RSL files

• 2000 array jobs stable• PRAGMA Grid testbed• Latest CSF4 release(Version 4.0.5.1 and 4.0.6).


• OPAL as resource manager of CSF4• CSF4 allocate service instances of OPAL for jobs

1717

New OPAL-CSF4 Cloud model


1818

New OPAL-CSF4 Cloud model

• OPAL as virtual resource manager in CSF4– Job submission, job monitoring

• CSF4 managing multiple OPAL sites– Site status (CPU, service) updates (modifying in OPAL)

• CSF4 allocate service resource of multiple sites– New interface of Job submission (URL to entire directory,

URL to list file of directories) (modifying in OPAL)

• Scheduling OPAL service jobs and maintaining lifecycle of jobs– New scheduling plugin (OPAL Service plugin)– Monitoring job status using status files


1919

New Resource manager

• Extend a new resource manager: – “Resource Manger Opal Service”


2020

Scheduling plugin

• A: Select Opal sites according to service requirement;

• B: Sort opal resources according to CPU numbers; • C: Spread arrayjobs to different sites


2121

Communication mechanism

• Using SOAP protocol to cooperate with OPAL (URLs)• Monitoring job status using status files


2222

Configuration and Experiments


<cluster> <name> vm2-opal </name> <type> OPAL </type> <host> vm2.jlu.edu.cn </host><port> 8080 </port> <version>2.4</version> <home>/home</home> </cluster> <cluster> <name> vm4-opal </name><type> OPAL </type> <host> vm4.jlu.edu.cn </host> <port> 8080 </port> <version>2.4</version> <home>/home</home></cluster>

2424

EVC model

• Customized, isolated and secure executing environment for parallel applications.


• Resource manager

• Virtual Infrastructure

• VMM

2525

Support EVC in CSF4

• Objectives– Parallel job co-allocation – dynamic executing environment deployment (VJM)

• Extend VJM module to manage EVC (EVC manager)– Resource reservation using Vjobs, Vjobs manage virtual

machines, EVC manages virtual clusters– Creating, reconstructing and rearranging virtual clusters

• New scheduling plugins: parallel job plugin– Parse VC requirements of jobs; prepare VCs dynamically in

runtime; distribute parallel jobs to VC

• Others– Integrate VJM as a separate service in CSF4– VC status monitoring using VJM– Real job monitoring


2626

Parallel job scheduling in CSF4

• Two phase resource allocation in parallel job plugin– Construct virtual clusters according to job requirements– Distribute real jobs to virtual clusters


2727

Module design of EVC manager

• Interfaces and internal modules

• Organize VCs in a pool

• VM configuration (IP, image)

• VC configuration (subnet, cluster software, …)

• Support multiple VMMS (Xen, VMwareServer, etc.)


• Two-phase scheduling are all based on GSI.– Resource co-allocation– Real job distribution

2828

Process of parallel job scheduling


2929

Image management• Image configuration file (XML)• Support image compression to save transmission time• Support dedicated applications by dynamic installation (yum…)


3131

Conclusion

• CSF4 have been evolved from traditional grid enabled to cloud support.– Powerful, usable, extendable

• New OPAL-CSF model– Sharing service resources by multiple OPAL sites.

• Elastic virtual cluster– Parallel job co-allocation– Dynamic executing environment pre-deployment


3232

Ongoing works and plans

• Virtual cluster live migration strategy– Concurrent migration protocol

• Multi-domain service scheduling policies– Monitoring service utilization rate– Scheduling policies

• Elastic virtual cluster management strategies– Reconstruction– Virtual cluster pool– Multi-VO users


11 Application of CSF4 in Avian Flu Grid: Meta-scheduler CSF4. Lab of Grid Computing and Network...

Documents

Transcript of 11 Application of CSF4 in Avian Flu Grid: Meta-scheduler CSF4. Lab of Grid Computing and Network...