Post on 28-Mar-2015
Flexible Service Composition
Adam BarkerSchool of InformaticsUniversity of Edinburgh
Robert G. MannInstitute for AstronomyUniversity of Edinburgh
Introduction
• Service Oriented Architectures– Distributed computing platform targeted at the web– Define a standard way to perform program to program
interaction– Can tie together any OS, application, data store,
programming language, device etc.– Defined using: XML, SOAP, WSDL etc.
• A Service Oriented World– Wide spread adoption and interest in simple, vanilla web
service standards– This standardised architecture is unlikely to change in
the forcible future• Technologies need to fit around and consume these
existing standards
Scientific Workflow
• Most workflow engines focus on Business Process Modelling
• A Scientific Workflow captures a series of analytical steps which describe a computational experiment– In order to verify a hypothesis etc.
• Scientific Workflow has an extra set of requirements:– Rapid prototyping of experiments– User Interaction with the scientist– Reliability and Fault Tolerant execution– Transparent access to resources– Repeatability, Smart re-runs and parameterisation– Provenance information, presentation of the results– Control Flow vs. Data Flow
Motivation
• Few languages which deal with the flexible, knowledge acquisition and discovery processes found in the sciences– myGrid, Kepler, ICENI frameworks allow
statically defined, pre-designed/pre-planned workflows to be executed by a centralised workflow engine
• The following science use-case serves as a counter example of coordination which is difficult to achieve by existing systems– Large Synoptic Survey Telescope (LSST)– Time-Domain astronomy
Science Use Case Background
• Current observatories are able to look very deeply at very small sections of sky, not likely to catch transient events:– Observatories always looking somewhere else– Small field of vision means that an impractically
large number of separate observations are required to map the entire night sky
– Observing facilities are scarce
• Observations of change in the universe are difficult to obtain
Science Use Case Background
• Automated Classification– For known classes of object– LSST is a first attempt at time-domain astronomy
• Likely to discover new (previously undetected) classes of object– Always data which the automated stage cannot classify
• Typically most of these will be junk, however this may only be revealed by comparisons with other detections made on the same night
Science Use Case
Reasoning
Web Service
Web Service
Web Service
Web Service
WorkFlow Requirements
• Complex coordination behaviour– Data is inherently distributed– Resources are scarce
• Requires selection and negotiation– Large quantities of data
• Requires Autonomous, Intelligent software• Human in the loop only when needed, e.g. specialist
– Workflow specification cannot be fully predicted at design-time
• Difficult to classify data type or quantity in advance• Component choice, e.g. web service, database etc.
– (semi) Flexible, Dynamic, runtime composition• In a decentralised, peer-to-peer environment
A convergence of Interests
Flexible
Autonomous
Cooperation, Negotiation
Run-Time Decisions
Uncertain Environments
Reliable
Secure
Middleware Support
Working Systems
Scaleable
Age
nts
SO
A
Workflow based on Interaction Protocols
• Interaction Protocols– Collection of conventions that allow cooperation
between agents in an open MultiAgent System– A recipe for communication– Defines ‘if and when’ agents communicate– Order and kind of messages relating to a certain domain
• Multi Agent Service Composition (MASC) – Agent-Based Workflow language and framework based
on Interaction Protocols– Decentralised, peer-to-peer architecture– Aimed at Scientific Workflow composition
• In order to solve the motivating science use-case– Discuss the language and framework in the following
slides
MASC: Scenes
• Scene– Bounded space in which a group of
agents interact on a single shared task– Divide a large, complex protocol into
manageable chunks– Scenes cannot begin execution until all
agents have reference to the protocolS := scene(ids, {ip}, {op},
{R}, {A})ip := inport(ids:idpin, T)
op := outport(ids:idpout,T )
MASC: Roles• Role
– A role type allows an engineer to specify a pattern of behaviour which an agent can adopt
• Many agents can adopt the same role type • Roles are defined as a set of methods {M}
– Methods are constructed from operations and actions
A := agent(ida, idr, Φ(k) )R := {idr, config(k) , {M}}M := method idm(Φ(k)) = op
Φ := v:T, _ , c:TT := XML Data Types
MASC: Operation Set
op := action
| op1 then op2
| op1 or op2
| op1 par op2
| waitfor op1 timeout op2
| invoke idm(Φ(k))
THEN OR
PAR RECURSION
MASC: Action Set
action := empty (empty)
| p(Φ(k)) → agent( ida, idr) (send)
| p(Φ(k)) → multicast(idr) (multicast) | p(Φ(k)) → user() (user send) | p(Φ(k)) ← agent(ida, idr) (receive) | p(Φ(k)) ← user() (user receive) | Φ(k) = portread(idpin) (port read)
| portwrite(idpout, Φ(k)) (port write)| Φ(k) = p(Φ(l)) fault Φ(m) (decision)
| Φ(k) = service(ws, Φ(l)) fault Φ(m) (service)
MASC: Decision Procedures
• Decision Procedures– Connect the protocol code (describing the interaction
model) to an agent’s internal reasoning model – Each agent interacting within a scene references a set
of decision procedures• Implemented as a set of methods inside a reasoning web
service• Throughout protocol execution agents can invoke methods
on its reasoning web service
• Does not sacrifice the autonomy of the agent• Each agent can subscribe to their own reasoning model
(BDI etc.)• Up to the agent engineer to specify these methods• Allowing a personalised strategy within the interaction
protocol
MASC: Service Enactment
• External Web Services– Agents consume external web services, acting
as a proxy to their execution• Extra level of abstraction
– Allows agents to consume the passive Service Oriented Architecture found in Internet and Grid Systems
– Can be hard coded in the interaction protocol or determined at runtime• e.g. from a broker or message exchange
MASC: Execution
P
P
P
P
R
R
R
R
W W
W
W
Web Service
MASC: Dataflow
P := protocol(idp,{S}, link(L)*)
L := source → sink+
• source/sink:– Web Service– Application– User interaction– File reading/writing
• High level experiment composition
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
MASC: Layers of Abstraction
Agents
Scientist: Has a Scientific Hypothesis
Engineer: Has an Engineering Aim
Reasoning Model
Dataflow MASC
Scene Definition
Port Definitions
Existing SOA
DataFlow Engineer
Interaction Engineer
Agent Engineer
The MASC Framework
• Implementation:– Full Java implementation
• Built using the Sun Java Web Services Development Pack (JWSDP) 2.0
– Agents can be executed locally: as a closely coupled system or as distributed processes
– SOAP messaging– XML Language specification– https://sourceforge.net/projects/multiagent
• Application– Requirements based on Real Science use cases– Applied to the UK e-Science project, AstroGrid
Advantages
• Standard MAS Arguments– Inherently a Distributed Peer-to-Peer System
• No centralised server• No central point of failure• Allows engineering in an open environment
• Interactions that are too complex, or simply cannot be specified at design-time– Agents act as a proxy to the services which are
being coordinated, extra level of abstraction– Reactive agents, runtime decision and
coordination
Advantages
• Inter-operability– Infrastructure independent: Interaction model
always remains a layer above any specific middleware or OS
– The web services being invoked require no modification before taking part in the interaction
– Low engineering requirement • Layer of s/w to translate protocol steps
• Reasoning Models– Protocols do not sacrifice the autonomy of the
agents– Each agent can subscribe to their own reasoning
model
Conclusions
• Internet and Grid Systems are filled with passive objects (services)
• Agency paradigm offers a way of programming autonomous, social and active components which consume this SOA
• Language provides a way of applying the principles and well understood concepts of agency to the web service composition problem
• Framework fits in with existing standards• AstroGrid provides a live science test bed
Questions
Thanks for listening