© 2006 Open Grid Forum Workflow Management Research Group - WFM-RG OGF 22 Ian Taylor, Andrew...

© 2006 Open Grid Forum

Workflow Management Research Group - WFM-RG

OGF 22Ian Taylor, Andrew Harrison and Ewa Deelman

2© 2006 Open Grid Forum

OGF IPR Policies Apply

• “I acknowledge that participation in this meeting is subject to the OGF Intellectual Property Policy.”• Intellectual Property Notices Note Well: All statements related to the activities of the OGF and addressed to

the OGF are subject to all provisions of Appendix B of GFD-C.1, which grants to the OGF and its participants certain licenses and rights in such statements. Such statements include verbal statements in OGF meetings, as well as written and electronic communications made at any time or place, which are addressed to:

• the OGF plenary session, • any OGF working group or portion thereof, • the OGF Board of Directors, the GFSG, or any member thereof on behalf of the OGF, • the ADCOM, or any member thereof on behalf of the ADCOM, • any OGF mailing list, including any group list, or any other list functioning under OGF auspices, • the OGF Editor or the document authoring and review process

• Statements made outside of a OGF meeting, mailing list or other function, that are clearly not intended to be input to an OGF activity, group or function, are not subject to these provisions.

• Excerpt from Appendix B of GFD-C.1: ”Where the OGF knows of rights, or claimed rights, the OGF secretariat shall attempt to obtain from the claimant of such rights, a written assurance that upon approval by the GFSG of the relevant OGF document(s), any party will be able to obtain the right to implement, use and distribute the technology or works when implementing, using or distributing technology based upon the specific specification(s) under openly specified, reasonable, non-discriminatory terms. The working group or research group proposing the use of the technology with respect to which the proprietary rights are claimed may assist the OGF secretariat in this effort. The results of this procedure shall not affect advancement of document, except that the GFSG may defer approval where a delay may facilitate the obtaining of such assurances. The results will, however, be recorded by the OGF Secretariat, and made available. The GFSG may also direct that a summary of the results be included in any GFD published containing the specification.”

• OGF Intellectual Property Policies are adapted from the IETF Intellectual Property Policies that support the Internet Standards Process.


Full Copyright Notice

Copyright (C) Open Grid Forum (2008). All Rights Reserved.

This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works.

The limited permissions granted above are perpetual and will not be revoked by the OGF or its successors or assignees.


Boston OGF - OGF22

• 1:45 Session Overview (Taylor)• Background on Research Document• Past Efforts on interoperability

• 2:10 OGF Survey on interoperability (Harrison)• Responses from community• Conclusions

• 2:30 Discussion of Research Document• The way forward• Community involvement

• 3:00 Session Talks/Presentations• Farrukh Nadeem - Characterizing, Modeling and Predicting resource

availability in the Grid

Focus - Research Document


Motivation for Research Document

• The Story so far …• Focus on sharing and Interoperability

• Accept co-existing workflow representations/ environments

• How do we share/reuse workflows?• Focus on the scientist performing the experiment

• How can sharing help him/her

• Workflow Interoperability• Do we need this - use cases?• Workflow embedding?


Motivation: Types of Sharing

• Intra domain for collaboration on the same workflow • Inter domain - reuse of algorithms/services• Provenance issues/living documents/logging/storage• Sharing strategies for optimising/design of workflows• Support full research cycles/iteration• Search/discovery of services/workflows• Exposing/organising research -

workflows/papers/people• Tagging/annotation


WFM-RG

• Past Efforts• Related Meetings - background and

further motivation for proposed research


Manchester - OGF 20

• 10:35 “Levels of the Grid Workflow Interoperability” - Adrian Toth, University of Miskolc.

• 10:45 "Workflow Optimization and Sharing Using Performance Information" - Omer Rana, Cardiff University

• 10:55 "Scheduling Data Intensive Workflows Onto Storage-Constrained Distributed Resources" - Rizos Sakellariou, University of Manchester

• 11:05 "myExperiment - social software for workflow sharing" - David De Roure, University of Southampton and OMII-UK

• 11:15 "The WHIP Plugin for Workflow and Artefact Sharing" - Andrew Harrison, Cardiff University

• 11:25 "Shibboleth Protection and Management of Workflows" - Richard Sinnott, National e-Science Centre, University of Glasgow

• 11:35 "A Workflow Mapping Mechanism for establishing Quality of Service Guarantees", Dimosthenis Kyriazis, Telecommunications Laboratory, National Technical University of Athens

• 11:35 Discussion on Document Focus

Focus - Workflow Sharing


Seattle OGF - OGF21

• 9:00 Session Overview - Ian Taylor• 9:05 Matt Shields – Sharing Workflows Recap of OGF 20• 9:15 Research document

• use cases for workflow sharing, volunteers

• 9:20 Ewa Deelman— NSF Workflow Interoperability workshop and what transpired in the execution breakout

• 9:35 Andrew Harrison— Kepler/Triana integration - interoperability Use Case 1

• 9:50 Maurizio Melato--- Web Based Grid Workflow System: A-WARE project

• 10:05 Dave De Roure—myExperiment• 10:20 All---Future plan (Use cases documents)

Focus - Workflow Interoperability


WORKS ‘07

• The 1st (Paris) and 2nd Workshop on Workflows in Support of Large-Scale Science (WORKS07)

• 2nd on 25th June @ HPDC (Monterey, California)

• Deelman and Taylor, co-chairs

• Areas:

• Workflow Systems• Adaptation and Integration • Workflow Applications and Models• Short Papers


Workshop on Scientific and Scholarly Workflow Cyberinfrastructure

• Baltimore, Maryland, Oct 3-5, funded by the National Science Foundation

• The goal of our workshop was to address the issue of interoperability, sustainability and platform convergence in scientific and scholarly workflow.

• Bring together key parts of the developer/user/supporter communities and see if there are concrete steps that can be taken to a shared, sustainable workflow cyberinfrastructure.

• Much discussion, current interoperability between systems is weak

Lots and lots of Workflow Systems

• 27 Chapters - from applications and environments• BPEL, Taverna, Triana, Pergasus, Kepler, P-Grade, Sedna, ICENI, Java

CoG Workflow, Condor, ASKALON, Swift, Petrinets, and so on …• Successfully used - choice depends on requirements and politics :)


WFM-RG

• Current State-of-the-art• Related past efforts• Need more contribution from eScience

community on sharing • On-going• See survey later


Past Efforts by the WfMC

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.


WfMC - Types of Interoperability




WfMC - Types of Interoperability 2






Existing Workflow Standards

• Workflow Management Coalition (WfMC) identifies five functional interfaces, that connect a workflow management system with external application systems, of which Interface 4 deals with workflow interoperability, which• defines abstract protocol for peer-to-peer interaction of workflow

enactment services, potentially across business domain boundaries• Supports scenario where an activity in a workflow process hosted by

an enactment service is realized by a subprocess hosted by another enactment service.

• A typical example is an interenterprise workflow, such as those found in supply-chain interactions.

• Provides interoperability on the workflow-process level only; internals of a process instance need not be known across business domain boundaries.


WfMC Interface 4 Efforts

• SWAP (1998) - define an Internet-based workflow access protocol to instantiate, control, and monitor workflow process instances• SWAP is a binding of the jointFlow object mode• Slow adoption, community if favour of XML-based

comms

• Wf-XML - SWAP successor (1999)• interoperability standard that combines the

elementary concept of SWAP with the abstract commands defined by the WfMC Interface 4

• Defines an interoperability contract• HTTP is core transport mechanism for Wf-XML

messages


Wf-XML

• WfMC defined (2000) a binding of the standard, which uses asynchronous interaction via e-mail as the transport, with MIME encoding of the information to be exchanged.

• Asynchronous Messaging - loosely coupled• Other bindings exist


WfMC Other Related Work - XPDL

• XML Process Definition Language• Describes workflows at an abstract level• Use-case - a meta language that acts as a bridge between

proprietary languages understood by different engines.

• Benefits• A many-to-one mapping between engines• It exists

• no new specs• A number of implementations

• Drawbacks• Complex spec

• 30 pages of XML Schema• Each translation to a workflow must be hand-crafted (by

someone who understands XPDL and the engine specific language).


WfMC - Conclusions

• We should build on or reuse lessons learned and previous efforts

• They focus on the sub-process level (embedding) for interoperability

• Also, defining a meta language for modelling workflows (e.g. XPDL)• This is the b2b case• Are current eScience scientific workflow

systems too different (heterogeneous components, different models of execution and so forth)


Questionnaire - eScience Input

• Gathering Data for research Document• From eScience community experience on

interoperability - requirements analysis

• We asked the community to contribute to issues surrounding workflow interoperability and sharing• Had a handful of responses• You can still contribute at:

• http://bender.astro.cf.ac.uk/wfmrg/


Summary

• Feedback from developers and users of:• GridBus Workflow Engine• UNICORE• Inforsense• Triana• Taverna• Kepler• BPEL


Interoperability

• Requirement for interoperability between workflow systems• Comparing workflow systems with one another• Embedding existing workflows that run in a different environment• Using desktop interface to design in a chosen language

• Challenges to workflow interoperabiltiy• Language used to describe Scientific Workflows• Definition of workflows at an abstract level• Semantics of workflow and data• Security• The term ‘religious’ was mentioned more than once

• Benefits of interoperability• Reuse• A wider audience for workflow designers

• Achieving interoperability, two approaches idenitified• Defining workflow schema at an abstract level• Allowing workflow engines to be forked as a sub-process

(embedding)


Sharing

• Requirement of sharing of workflows• Reuse, modification, refining

• Challenges to workflow sharing• Language used to describe Scientific Workflows• Native engine enactment of shared workflows (embedding)• Trust and security• Workflow dependencies/time sensitivity

• Benefits of sharing• Reuse• Need driven approach to standardisation - sharing capability

as an incentive to achieving interoperability• Understanding what can be shared• Faster discovery

• Achieving sharing• Standards for semantics/description• Community portal


Conclusions

• Projects do have a need for interoperability and sharing• Not all projects, and not necessarily a need for both

• Primary reasons:• Reuse• Extending availability of existing work

• Primary routes:• Embedding workflows as (distributed) sub-processes• Shared abstract (meta) descriptions


Issues

• Shared meta-language (e.g. XPDL) is a good idea but…• Need commitment from workflow developers to implement a

binding• I want to run a workflow defined in the meta-language

• Will it really run?• Or does it require (aspects of) the environment in which it was

initially defined?Depends on the nature of the workflow, e.g. just WS

might be easier to port. But what if the client machine expects to receive notifications (i.e. can act as a server)?

• Introduces concept of modeling/bundling dependencies/runtime

• Reuse is about getting something for nothing• But writing lots of bindings to a meta-language is not getting

something for nothing


Issues

• Semantics• How do I dermine what a workflow is/does?

• Is it semantically the same but functionally different from another?

• Even if I can embed a foreign workflow into my favorite tool, what about data mediation/massaging?

• Understanding/mapping inputs/outputs


Approaches

• Try and solve the world?• Impose order on chaos• Define how it should be and hope individual

projects follow suit

• Try and solve small things?• But within a wider context• Fragmented, but not ignorant of other fragments.

• Start with what is there?• Sharing e.g. myExperiment• Execution e.g. BES, Atom• Meta description e.g. XPDL


Discussion

• Community Effort?• We’d be happy to lead a research document but

need volunteers• Need at least 5 volunteers not including ourselves …

• State-of-the-art of workflow interoperability and sharing

• use cases and proposals• Area is vague at present• Document would help firm up the area and push things

forward.

© 2006 Open Grid Forum Workflow Management Research Group - WFM-RG OGF 22 Ian Taylor, Andrew...

Documents

Transcript of © 2006 Open Grid Forum Workflow Management Research Group - WFM-RG OGF 22 Ian Taylor, Andrew...