S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor
-
Upload
virtual-campus -
Category
Technology
-
view
384 -
download
1
description
Transcript of S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor
www.s-cube-network.eu
S-Cube Learning Package
Chemical Modeling: Workflow Enactment based on the Chemical Metaphor
INRIA, CNR, SZTAKI
Zsolt Németh, SZTAKI
Learning Package Categorization
S-Cube
Service Infrastructure
Multi-level and self-adaptation
Supporting adaptation of
service-based applications
Learning Package Overview
Workflow, workflow management
Problem and requirement analysis
– enactment in large-scale heterogeneous environments
A chemical metaphor for workflow enactment
A coordination model for workflow enactment formalized in
the -calculus
Further challenges
This talk is about workflow but…
Workflow is just an example
– It is a common programming model for grids
– Features many coordination problems
Focus: coordination
– Large-scale distributed, dynamic, error prone environment and
applications
– Some degree of autonomy, adaptability, self-* must be provided
Workflow
“The computerized facilitation or automation of a
business process in whole or in part” – The Workflow Management
Coalition
a collection of activities that are processed in some order
and where both data-flow and control-flow relationships
may be present
The workflow Management Reference Model (Workflow Management Coalition)
Process Analysis, Modeling
& Definition Tools
Process Definition
Workflow Enactment Service
Applications &
Tools User (human)
Process Design
& Definition
Build time
Run time
Process Instantiation
& Control
Interaction with
Users & Application Tools
Scope of enactment
Abstract workflow
standalone application components
the order in which they are executed
names, references, etc. are logical
Represented by a graph
typically: DAG
Problem
Abstract workflow
Concrete workflow
Physical environment
Scope of enactment
Concrete workflow
standalone application components
the order in which they are executed
selected resources, services
additional activities (e.g. file transfer, staging, etc.)
all names, references, etc. are physical
Problem
Abstract workflow
Concrete workflow
Physical environment
Learning Package Overview
Workflow, workflow management
Problem and requirement analysis
– enactment in large-scale heterogeneous environments
A chemical metaphor for workflow enactment
A coordination model for workflow enactment formalized in
the -calculus
Further challenges
A common scenario (e.g. grid workflow)
Abstract workflow
Resources
Workflow engine
A common scenario (e.g. grid workflow)
Abstract workflow
Resources
Information system
Workflow engine
A common scenario (e.g. grid workflow)
Abstract workflow
Resources
Information system
Workflow engine
A common scenario (e.g. grid workflow)
Abstract workflow
Resources
Information system
Workflow engine
Concrete workflow
A common scenario (e.g. grid workflow)
Abstract workflow
Resources
Information system
Workflow engine
Concrete workflow
Problem analysis (current approaches)
The abstract workflow is static
(Typically) no advanced control structures
A priori mapping/ partly a priori mapping
Mapping based on stored information
A priori simulation/ a priori test/ a priori optimisation
Centralised engine
Human interaction
Limited autonomy, limited ability for adaptation
Overall lack of any high level model
Requirement analysis
Workflow enactment in large-scale heterogeneous environments
– should provide a higher level of autonomy
– should be able to adapt to changing conditions
– should be distributed
– should be able to make decisions on partial and actual information
– should support arbitrarily complex control structures
Often a complex problem can be solved in a more simple way using nature inspired models
Learning Package Overview
Workflow, workflow management
Problem and requirement analysis
– enactment in large-scale heterogeneous environments
A chemical metaphor for workflow enactment
A coordination model for workflow enactment formalized in
the -calculus
Further challenges
This talk is about a chemical metaphor but…
It is just an example
– Chemical reactions are reasonably similar to workflow enactment
– It has a well established formalism
Generally: nature inspiration for solving complex problems
– Simple, primitive parts behave “intelligently” as a whole
– Ants, termites, cells, molecules, etc.
A chemical metaphor
Reactions are
– autonomous
– distributed
– concurrent
– depending on local conditions
– depending on actual conditions
– not following any a priori pattern
– evolving in time
A chemical metaphor
Reactions are
autonomous
distributed
concurrent
depending on local conditions
depending on actual conditions
not following any a priori pattern
evolving in time
Workflow enactment should
provide a higher level of autonomy
be able to adapt to changing conditions
be distributed
be able to make decisions on partial and actual information
support arbitrarily complex control structures
A vision of chemical enactment
Resources
A vision of chemical enactment
Resources
Activities
A vision of chemical enactment
Resources
Activities
Control
A vision of chemical enactment
React
Learning Package Overview
Workflow, workflow management
Problem and requirement analysis
– enactment in large-scale heterogeneous environments
A chemical metaphor for workflow enactment
A coordination model for workflow enactment formalized
in the -calculus
Further challenges
From a vision to a model
Materialize information: resource quantums
Define an abstract chemical coordination model
– independent from actual technical realizations
– provide a high-level abstract framework for refinement
– formalism: -calculus
Advance, refine: find chemical examples for
– Complex control
– Fault tolerance
– Resource management
– Optimization
– etc
Resource quantums
The usual solutions
P1 P2
P3
P4
P5
P6
• stored information may be time sensitive
P6
Resource quantums
The usual solutions
P1
P2
P3
P4
P5
Resource quantums (« materialization of information »)
resources are represented as tickets
P1 P2
P3
P4
P5
P6
• tickets represent a guaranteed service
Resource quantums (« materialization of information »)
P2
P5
P1
P3
P4
P6
The -calculus
a declarative, functional formalism
inherently concurrent model of computation
basic data structure: multiset (chemical solution)
– passive molecules: booleans, integers, tuples, naming molecules
– active molecules: -abstraction
reaction: active molecules capture other molecules
– x.M, N → M[x:=N]
execution: perform reactions until a stable (inert) chemical
solution is resulted
The -calculus: -abstraction
Active molecules: -abstraction: PC.M
– P is a pattern that selects elements for the reaction
– C is a condition; the reaction takes place if C is true
– M is the action
Example: i:x, j:y i≤j, x>y. (j+1):x, j:y
Semantics: capture i:x and j:y and replace them by (j+1):x,
j:y if i≤j, x>y
-abstraction is one-shot
Universal matching symbol:
The -calculus
-terms are
– Commutative: M1,M2≡M2,M1
– Associative: (M1,M2),M3≡M1,(M2,M3)
– Realize Brownian motion
Reactions
– Locality: if M1→M2, then M,M1→M,M2
– Solution: if M1→M2, then <M1>→<M2>
Conditional reactions
– xC.M
Atomic capture – x1,x2,…xn.M
Resources
a resource quantum is modeled as a sub-solution
– recall: chemical solutions can be nested
elements in the solution are inert attribute:value pairs
there are mandatory attributes otherwise, the format is felxible
<id:R1, type:comp, proc:16, OS:Linux, …>
<id:N1, type:net, bandwidth:23, …>
<id:R3, type:comp, proc:1, installed:equsolver, network:N1 …>
Elementary activities
the chemical model does not execute an activity, just enacts it – execution is external
exit from the chemical world:
– execute activity on resource using parameter
– a symbolic notation that can be realised in many ways
return to the chemical world
– the execution produces some result or error put back in form of a solution (control molecule)
– <ActivityID:result>
– <ActivityID:result, error:errorcode, …>
– <ActivityID:result, executed:R1, …>
execute A on R→ <A:result…>,<id:R,…>
Resource dependency
activity A needs a resource
<id:r, type:comp, proc:1, >. execute A on r
<id:r, type:comp, proc:1, >.
execute A on r
< id:R1, type:comp,
proc:16, OS:Linux, …>
< id:R2, type:comp, proc:1,…>
< id:R3, type:comp,
proc:1, OS:SunOS, …>
< id:R4, type:comp,
proc:1, memory:23, …> < id:R5, type:comp,
proc:1, disk:12, …>
Resource dependency
<id:r, type:comp, proc:1, >. execute A on r
Captured a matching resource
< id:R1, type:comp,
proc:16, OS:Linux, …>
< id:R2, type:comp, proc:1,…>
< id:R4, type:comp,
proc:1, memory:23, …> < id:R5, type:comp,
proc:1, disk:12, …>
execute A on R3
Resource dependency
Both the resource and the activity are replaced by a control
molecule
< id:R1, type:comp,
proc:16, OS:Linux, …>
< id:R2, type:comp, proc:1,…>
< id:R4, type:comp,
proc:1, memory:23, …> < id:R5, type:comp,
proc:1, disk:12, …>
<A:12, error:0, executed:R3>
< id:R3, type:comp,
proc:1, OS:SunOS, …>
Data and control dependency
activity A waits a result from activity B
<B:x, >. execute A using x
<B:x, >. execute A using x <B:18>
<C:0, error:5>
<D:42>
<H:apple>
Data and control dependency
<B:x, >. execute A using x
Captured a corresponding control molecule
execute A using 18
<C:0, error:5>
<D:42>
<H:apple>
Data and control dependency
Replaced by the result
<A:2, error:0,...>
<C:0, error:5>
<D:42>
<H:apple>
Complex dependencies
An activity needs both input from another activity and
resources
– <id:r, 1>, <B:x, 2>. execute A on r using x
Dependencies can be arbitrarily combined
– find a resource r1 for activity A and a resource r2 for B so that r1 and r2
have the same operating system
– <id:r1, OS:x, 1>, <id:r2, OS:x, 2> .
execute A on r1, execute B on r2
Conditions can be added as well
– <id:r, disk:x, memory:y, > x>30, y>12.
execute A on r
Workflow patterns
Sequence
Conditional
Split
Synchronizing merge
P-split
– p-out-of-n activities follow A
P-merge
– p-out-of-n results trigger activity A
Loop
Learning Package Overview
Workflow, workflow management
Problem and requirement analysis
– enactment in large-scale heterogeneous environments
A chemical metaphor for workflow enactment
A coordination model for workflow enactment formalized in
the -calculus
Further challenges
Challenges
What we have so far is a framework
– Principles of a chemical coordination model
– It is just the beginning!
Let’s explore and advance
– Solve further aspects of workflow enactment: fault tolerance,
optimization, resource control, complex workflow structures,
constraints, co-allocation, compensation, etc.
– Find further nature analogon within the chemical model:
temperature, weight, magnetic properties, size, gravity, catalysts,
membranes, etc.
Challenges (examples)
Resource control by chemical agents
– withdraw, modify, block, accept, reject, group, etc.
Assert a neutralization agent: <id:r1, >.
Replace a molecule:
<id:r1, proc:1, OS:x, >.<id:r1, proc:1, OS:Linux, >
Combine two molecules:
(<id:r1, proc:1, >, <id:r1, proc:1, >).
<id:r1, proc:2, >
Challenges (examples)
Fault-tolerance by chemical agents
– error molecules, retry, rollback, redundancy, compensation, etc.
Assert an error handling molecule that reactivates in case of
error
(<Ai:x, >, <Aj:y, >, <id:r, R, >).
(execute A on r using x y,
(<A:errork, >,<id:r’, R, >).execute A on r’ using x y))
Challenges (examples)
Complex resource allocation and co-llocation scenarios by
molecules
– e.g. reserve r1 and r2 so that there is a network link between them
– (<id:r1, proc:1, network:n, >,
<id:r2, proc:4, network:n, >,
<id:n, type:net, >).
(execute A on r1, (<A:x, >.execute B on r2 using x))
Conclusion
The chemical metaphor: autonomous, adapting, distributed,
actual
The nature inspired coordination model
– activities, resources and control are modeled using the same
formalism
– -calculus is not just description but defines execution semantics as
well
– workflow structure, resources and control can be
added/withdrawn/modified in a fully dynamic and adaptive way
Connections to other teaching units
Foundations
– The Chemical Computing model and HOCL Programming
Applications
– Dynamic Adaptation with the Chemical Model
© S-Cube – 50/<Max>
References
Zsolt Németh, Christian Pérez, Thierry Priol: Distributed workflow coordination: molecules and reactions.
IPDPS 2006
Zsolt Németh, Christian Pérez, Thierry Priol: Workflow Enactment Based on a Chemical Metaphor. SEFM
2005: 127-136
Manuel Caeiro, Zsolt Németh, Thierry Priol: A Chemical Model for Dynamic Workflow Coordination. PDP 2011:
215-222
Acknowledgements
The research leading to these results has
received funding from the European
Community’s Seventh Framework
Programme [FP7/2007-2013] under grant
agreement 215483 (S-Cube).