Executing Workflow-Based Grid Applications with the Collaborative P-GRADE Portal

Gergely Sipos1, Csaba Nemeth1, Gareth J. Lewis2,Vassil N. Alexandrov2 ,Peter Kacsuk1,

1MTA SZTAKI (Hungary)2University of Reading (UK)

ContentsNeed for collaborative grid applicationsOur proposed conceptImplementation: Collaborative P-GRADE PortalWorkflow execution in single-VO environmentWorkflow execution in multi-VO environmentConclusions

Programming the GridGrid a highly complex infrastructureDifferent types of services multiple APIsMultiple VOs Different certificatesConcurrency, transactions, etc.Only complex applications can efficiently utilize the Grid.

Programming the Grid collaboratively Current collaborative environmentsGrid infrastructureGrid programming environment

Programming the Grid collaboratively Current collaborative environmentsGrid infrastructureGrid programming environment

Application development: Centralised information pieces must be collected

Application execution: Centralised responsibility is taken by a single person

Grid infrastructureGrid programming environmentProgramming the Grid collaboratively Our proposed solution

Collaborative Grid PortalsConcurrent development of Grid applications

Collaborative execution of Grid applications share responsibility!

The Collaborative P-GRADE PortalG S IGlobus GRAMGlobus MDSGlobus GridFTPMercury monitor (GridLAB)Workflow developmentWorkflow executionGrid servicesPortal serverUserGEMLCA (Uni. Of Westminster)

The workflow is a graph whereNodes are jobs (or services)Arcs represent file transfer between the jobs (services)Semantics of the workflow enables two levels of parallelism: Parallel execution inside a workflow nodeParallel execution among workflow nodesThe portal server contains a DAGMan-based workflow manager:File transferJob submission (service invocation)A Collaborative P-GRADE Portal workflowJobs can be parallel

The result of collaborative workflow developmentWorkflow definition

Contribution table

Each workflow component has several propertiesAll these properties can be defined by different users

Workflow componentPropertyDefined by (user)Job deltaExecutable fileTomJob deltaExecutor siteJamesJob deltaInput file 1Kevin

Sharing responsibilities 1:Accessing filesWF manager: Whose proxy to use to access a file?The only rational answer: the proxy that belongs to the user who defined the file location.If the source and target files were defined by different users transfer the file through the portal server indirectly

Sharing responsibilities 2:Submitting jobs single VO caseWF manager: Whose proxy to use to submit a job?If the Portal is connected to a single VO:Every portal user is allowed to submit jobs to any of the resorces several possible answersThe developer of the binary executableThe provider of the input parametersEtc.P-GRADE Portal: the proxy that belongs to the user who defined resource for the job (no brokering!)

Sharing responsibilities 3:Submitting jobs multi VO caseWF manager: Whose proxy to use to submit a job? If the Portal is connected to multiple VOs:Different collaborative users may have access to different resources

the only rational answer: the proxy that belongs to the user who defined resource for the job (no brokering!)

Multi-Grid executionP-GRADE Portal serverVO 1VO 2VO 3

ConclusionsCollaborate through Grid applications!The Collaborative P-GRADE Portal is the first implementation of the concept:Workflow applicationsExecution in Globus GridsMulti-VO accessPrototype version, release in 2006Non collaborative version is already connected to the UK NGS: www.cpc.wmin.ac.uk/ngsportal/gngsportal/index.php

Thank you for your attention!

Questionssipos@sztaki.hu www.lpds.sztaki.hu www.cpc.wmin.ac.uk/ngsportal/gngsportal/index.phpP-GRADE NGS GEMLCA Portal Demo 6pm, booth of Grid Operations Support Centre