Enabling Grids for E-sciencE

Batch System IntegrationJan Just Keijser

Nikhef AmsterdamEGEE III SA3 Kickoff Meeting, May 15-16 2008

EGEE III SA3 Kickoff Meeting, May 15-16 2008 2

Outline

• Targets/Milestones• Work done so far• Next steps• Issues• << Talk by Christan >>• Other Interesting System Integration Issues

EGEE III SA3 Kickoff Meeting, May 15-16 2008 3

Targets & Milestones

First milestone• Take part in writing MSA2.3,

'Strategy and roadmap of the EGEE multi-platform support’

Near-Future targets• Full support for 2 extra batch systems in gLite (apart from PBS and LSF)• Both lcg-CE and CREAM-CE supported• Develop a

Batch System Integration Checklist (BaSIC ;-))• Write a 'cookbook'

How to integrate Your Favourite Batch Systems in 6 Easy Steps

EGEE III SA3 Kickoff Meeting, May 15-16 2008 4

Work done so far...

• PIC:– Condor integration with lcg-CE: see Christian’s talk– CREAM-CE integration in the works

• CESGA: – Sun Grid Engine (SGE) integration with lcg-CE– SGE already certified by SA3

minor note: info-dynamic-scheduler ?

• Nikhef:– Inventorized existing batch system integration for Condor, SGE,

PBS– Set up testbeds for off-developer-site testing of BSI

CentOS 5.1 host OS, Xen 3.2, LVM Testbed consists of Batch System + WNs, CE, RB/WMS, MON, UI

EGEE III SA3 Kickoff Meeting, May 15-16 2008 5

Inventorizing...

• EGEE-UF03 in Clermont-Ferrand was very useful:

EGEE III SA3 Kickoff Meeting, May 15-16 2008 6

Next steps...

At Nikhef– we will set up a testbed with Batch System Of Choice– Next, we will use YOUR installation guidelines/Twiki pages on

how to integrate this batch system with gLite

What we need from you:– installation packages– installation guides/Twiki pages– Time & effort– your cooperation

EGEE III SA3 Kickoff Meeting, May 15-16 2008 7

Longer-term ideas

• Possible ARC integration– ARC-glite interface is at CE-level– Thus, ARC integration could be treated as

'Just Another Batch System'• Other batch systems

– ??• Other platforms

– 32bit Linux flavours, RPM-based, .deb-based– 64bit Linux flavours, RPM-based, .deb-based– Others??

EGEE III SA3 Kickoff Meeting, May 15-16 2008 8

Issues

• Which site will develop and support LSF integration?

• Interaction with JRA1 developers is limited thus far– Who will/should own the interface between a batch

system and the affected gLite components?– How to address issues like the current blahp

interface issue, where PBS+LSF are treated differently from Condor

• CREAM-CE is very much a “moving target” • Mismatch between FTEs available for doing

coordination and FTEs available for technical/development work

EGEE III SA3 Kickoff Meeting, May 15-16 2008 9

Concluding Remark

To make Batch System Integration a success it is vital that we have stable interfaces between the Batch System and the glite Middleware and that these are “owned” by SA3

DevelopmentJRA1

Batch system co-ordinationNikhef, SA3

OperationsSA1

SGECESGA, ..

.

LSF CondorPIC

Torque/MauiNikhef

Batch system integration