EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE Application Porting Support in EGEE Gergely...

EGEE-III INFSO-RI-222667

Enabling Grids for E-sciencE

www.eu-egee.org

Application Porting Support in EGEE

Gergely SiposMTA [email protected]

EGEE’08 ConferenceIstanbul, 22-26 September 2008


EGEE-III-INFSO-RI-222667

The Application Porting team

• MTA SZTAKI, Budapest– Grid Application Support Center (GASuC)

http://www.lpds.sztaki.hu/gasuc

• INFN, Catania– GILDA Team

https://gilda.ct.infn.it/

• UCM, Madrid– Distributed Systems Architecture Research Group

http://asds.dacya.ucm.es/doku.php?id=start

• CSIC, Santander– Institute of Physics of Cantabria

http://grid.ifca.es/

• ASGC, Taipei– Academia Sinica Grid Computing

http://grid.sinica.edu.tw/



Support cycle and services

Interviews

Apply online at www.lpds.sztaki.hu/gasuc

Personalized training

Porting specifications

Problem analysis

Writing publications and

case studies

Prototyping grid applications

Fine tuning applications on production grids

Visit porting centre to make the cycle more effective!



Application porting group webpagewww.lpds.sztaki.hu/gasuc



Application analysis

Application Description Form at www.lpds.sztaki.hu/gasuc



Dissemmination and outreach



Main tools and technologies

• gLite command line tools, scripts, APIs– Interfacing with the middleware services

• P-GRADE Portal (SZTAKI)– Workflows and parameter studies

– Application specific interfaces

• GridWay (UCM)– Metascheduling on clusters and grids

– Programming abstractions

• GILDA services (INFN and others)– gLite infrastructure for training and prototyping

– Training modules and services

• GEMLCA service (SZTAKI)– Exposing legacy component as Grid services

• GANGA and DIANE (CERN)– Parameter sweeps

– Support is provided by external partner



Manual vs. automated porting

• Manual– Writing scripts and wrapper programs from scratch– Quite often a necessary first step to test new concepts– Often includes new discoveries and errors GGUS

• Automated porting– Interfacing with gLite/LCG services through high level tools– Ready to use solutions for several typical scenarios

Fault tolerance Handling large number of jobs Composing data-parallel applications Parameter sweeps Composing task-parallel applications Workflows



Typical goals of porting 0.

• Porting for “myself”– Demand for large capacity

Porting from another grid Coming from a local

environment

Porting Execution

• Example– Simulate ionized cloud around solid

particles shot into plasma (pallets) – Parameter-study job with

P-GRADE Portal, input/output using SE and File catalog

– Few hundred simulations, each running for ~1 hour

– Porting done in a few days– Output is a presentation and

publication at “Fuelling of Magnetic Confinement Machines” conference




• Porting and improving for “myself”– Demand for large capacity

Porting from another grid Coming from a local

environment

Porting Execution

Improvement

Execution . . .

• Example– Numerical modeling of ElectroMagnetic

field distribution in human tissues– 1-2 mm digital human phantom from MRI

data– Parameter-study execution with scripts – Current difficulties:

MPI program – too few Biomed sites support it

Too many files – establishing reliable access to SEs and LFC over long time




• Porting for my group/VO/project– Customized (G)UIs are often

required– Typically motivated by a grid

project

Porting

Customization

Execution

• Example

– Dimensional Frequency Domain Electromagnetic Numerical Modeling

– Geoinformatics, metallurgy– Parameter study workflow

in P-GRADE




• Porting to operate grid application as a service– Application developer is not

the end user– End users are already known

They are typically non-IT people

• Example

– Solution of Schrodinger equations for triatomic systems

– Parameter study based on Fortran90 code

– Data size is manageable– Application ported with P-GRADE

in ~2 weeks (incl. consultancy!)– Custom interface developed by

application owner

Porting

CustomizationExecution

Execution Execution



Application porting and training

• Collaborating with NA3 Training activity on porting courses and schools:

– First part is dedicated to learn the basics of gLite and high level tools

– Second part is dedicated to application porting Working in groups Reporting, regular presentations

– Concept piloted by INFN and adapted since by SZTAKI and others Next event at Paris Observatory

in December



Other aspects of porting

• Conference papers• Tutorials, training

materials• Case studies



P-GRADE Portalapplication module

Programming API

Howto example:Develop an application specific grid portal

P-GRADE Portal 2.7

EGEE and Globus Grid services (gLite WMS, LFC, …; Globus GRAM, GridFTP, …)

End user Web page

code

Parameters Results

Fault tolerantworkflowsparameter sweepsparameter sweep workflows

publish



Separating the roles

Portal Administrator• Installs P-GRADE Portal 2.7

Grid Application Developer• Port application to grid with P-GRADE:

• Job, workflow, parameter study or combination of these• No need to modify your code

End Users• Executes the published application with custom input parameters• Benefits from fast execution and richer result

Grid Portal Developer• Develops a Web page for end users• Installs and configures Application Module to P-GRADE• Registers P-GRADE application in application module



Conclusions

If you have an application but• you do not know EGEE• you face any difficulty with gLite• do not know what is available “out there”• work for an external project that involves porting code to EGEE

apply for assistance: www.lpds.sztaki.hu/gasuc

Visit a porting support centre– EGEE can provide financial support for your visit

• Porting is an efficient way to– Extend the user community– Represent user communities– Transfer knowledge between various groups



EGEE Application Porting Support Groupwww.lpds.sztaki.hu/gasuc

Questions?

EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE Application Porting Support in EGEE Gergely...

Documents

Transcript of EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE Application Porting Support in EGEE Gergely...