Enabling Grids for E-sciencE Astronomical data processing workflows on a service-oriented Grid...
-
Upload
cameron-barnett -
Category
Documents
-
view
219 -
download
0
Transcript of Enabling Grids for E-sciencE Astronomical data processing workflows on a service-oriented Grid...
Enabling Grids for E-sciencE
www.eu-egee.org
Astronomical data processing workflows on a service-oriented Grid architecture
Valeria Manna INAF - SI
The 17th Global Grid Forum - GGF17Tokyo, JapanMay 10-12, 2006
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006 2
Enabling Grids for E-sciencE
Motivation and Objectives
Working Group:
Claudio Vuerli, Giuliano Taffoni, Valeria Manna, Andrea Barisani, Fabio Pasian
• Problem analysis related to the integration of astronomical applications in a Grid environment
• Description of astronomical tasks sequences with workflow languages to process complex compositions of astronomical applications
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006 3
Enabling Grids for E-sciencE
Astrophysics and distributed computing
Computational problems
• Theoretical Codes
• Computation and analysis of many TBs of images
• Database of catalogues of the detected elements
• data-mining tools
- Data source and organization interaction- Geographically distributed resources (database, telescope, storage, etc)
Request of distributed computing to provide:•User virtualisation
•Resource virtualisation
GRID
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006 4
Enabling Grids for E-sciencE
Astrophysics Application
TasksTasks::
•Database AccessDatabase Access
•Data ReductionData Reduction
•Pure Computational Pure Computational
•Instrument MonitoringInstrument Monitoring
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006 5
Enabling Grids for E-sciencE
Workload Management
The task sequence of the astronomical application is managed by a Workload Management System that makes a task- job mapping
User :– Submit a job
– Create some script for the monitoring of job status
– Create script to create some task connecting link
– Recover I/O data
Need to a Workflow
Management System
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006 6
Enabling Grids for E-sciencE
Workflow Management
A workflow language allows to describe the execution flow of pipeline tasks :
–Query astronomical catalogues–Data transfer–Synchronize job –Analyse the resulting data
Mapping task → service
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006 7
Enabling Grids for E-sciencE
Web Services CompositionWeb Services Composition
The composition of WS consists of providing logic around a set of interactions between the composition and the WS that participate in it. These interactions are simply invocations to the operations offered by the services in play. One approach to providing the control and data logic is the use of workflow specifying:
– the execution order of operations from a collection of WS – the data shared among composed WS– partner involved– how the partner are involved in the flow process
Service orchestrationweb services can interact with each other at the message level, including the flow logic and execution order of the interactions. For orchestration, the process is always controlled by the perspective of one of the parties.
Service choreographytypically associated with the public message exchanges that occur between multiple web services, rather than a specific business process that is executed by a single party. Each part involved in the process describes the part it plays in the interaction.
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006 8
Enabling Grids for E-sciencE
A workflow language: Bpel
Bpel (Business Process Execution Language) is a workflow-based composition language for Web Services
• Standard• Open Source• Support the Service Orchestration
Feature:– Interaction: define WSDL port types for each interface that is
used in the workflow definition– Basic Activity: to allow for interaction with the applications
being composed invoke, reply and receive – Structured Activity : conditional execution and iteration /
recursion of a sequence of activities to manage process flow
Likely to be adopted as EGEE standard
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006 9
Enabling Grids for E-sciencE
Bpel: process flow
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006 10
Enabling Grids for E-sciencE
Hypothetical Scenario using Bpel
BPEL ENGINE
CREAMEGEE
SERVICE
WEBSERVICES
GRID/WEBSERVICES
ASTROGRIDSERVICES
WF
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006 11
Enabling Grids for E-sciencE
WSIF
ProblemCurrently the LCG middleware is not Web Service oriented and in GLite (middleware of EGEE project) the WM and the other services are expected to offer a Web Service interface.In this scenario the use of BPEL is not feasible
In a BPEL workflow it is very useful to provide a set of services that are always available.
SolutionTo use BPEL partner abstraction that are services that workflow needs to use and typically are mapped to Web (or Grid) services. It is possible to map some BPEL partners to locally implemented services as long as a WSDL port type for a partner is provided.
WSIF (Web Service Invocation Framework) is a simple Java API for invoking Web services, enables to interact with abstract representations of Web services through their WSDL descriptions
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006 12
Enabling Grids for E-sciencE
Current Scenario with WSIF
WSIF
BPEL ENGINE
WSIF
LCG
WEBSERVICES
GLOBUS 2x
WEBSERVICES
WF
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006 13
Enabling Grids for E-sciencE
Conclusions
• Astronomical applications described using workflow language
• A solution to integrate astronomical task sequences in gLite middleware
• Use of Service Oriented technology• Use of Grid Service composition
Integrated solution in the current middleware and in its future development
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006 14
Enabling Grids for E-sciencE
The End
Thank you for your attention