Transactional Workflow Chapter 9. © Jim Gray, Andreas Reuter Transaction Processing - Concepts and...
-
Upload
william-dalton -
Category
Documents
-
view
220 -
download
0
Transcript of Transactional Workflow Chapter 9. © Jim Gray, Andreas Reuter Transaction Processing - Concepts and...
Aug. 2 Aug. 3 Aug. 4 Aug. 5 Aug. 6 9:00 Intro &
terminologyTP mons& ORBs
Logging &res. Mgr.
Files &Buffer Mgr.
Structuredfiles
11:00 Reliability Lockingtheory
Res. Mgr. &Trans. Mgr.
COM+ Access paths
13:30 Faulttolerance
Lockingtechniques
CICS & TP& Internet
CORBA/EJB + TP
Groupware
15:30 Transactionmodels
Queueing AdvancedTrans. Mgr.
Replication Performance& TPC
18:00 Reception Workflow Cyberbricks Party FREE
Transactional Workflow
Chapter 9
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 2
What Is the Problem ?
“A workflow management system is an active system that manages the flow of business processes performed by multiple persons in multiple steps. It gets the right data to the right people with the right tools at the right time.”
(This definition omits a umber of aspects: roles, events, errors, cooperation, ...)
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 3
In More Technical Terms: What Is Workflow ?
WF is a long-lived execution involving a potentially large number of autonomous agents such as programs, databases, sensors, actors, humans.
Control flow and data flow are (partially) pre-defined and may evolve over time.
There are numerous interdependent consistency criteria.
A WF must be kept alive by “the system” under all circumstances.
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 4
Source
Step
case
case
fork
loop
compensationstep
Components of a Workflow
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 5
Steps ...
execute application logic of any kind, can interact with human operators, access shared data in databases, depend on events and can create their own
events, have a short duration and (should) behave like
classical transactions, are invoked depending on the execution history.
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 6
The Script ...
specifies the control flow and the workflow, defines the event and data conditions under which a
step is to be executed, defines the synchronization criteria for accessing
shared data, maintains the local execution context of a workflow
instance, handles resource conflicts, in particular on shared
data, represents a persistent execution.
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 7
What Is Transactional Workflow ?
There are different interpretations: Extended transaction models adapted to the
needs of workflow: Sagas, Flex transactions, etc.
Application of some transactional properties such as isolation and durability to workflows.
Using classical distributed transactions to implement the control flow machinery of a workflow system.
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 8
What Happens to the Transactional Properties ?
A: Atomicity does not apply to an entire workflow
C: Consistency must be redefined including the temporal dimension.
I: Isolation must be limited in time; cooperation must be allowed.
D: Rather than the effects of transaction, the execution itself must be durable.
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 9
Correctness
Transactional correctness guarantees a consistent overall state if each individual step is executed correctly (or not at all) and there was a consistent initial state.
In long-lived executions, this definition cannnot be used, because strict isolation is not feasible and rollback is not option at the workflow level.
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 10
Rollback vs. Compensation
Rollback is based on the assumption that an erroneous state can be reverted to the previous (correct) state without affecting anybody => Isolation.
Compensation tries to modify an erroneous state such that all the consistency constraints work as though the faulty operation was never executed => Formal definition of consistency.
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 11
Correctness For Long-Lived Activities
Transactions can be executed iff the possibility of rollback is maintained.
A step in a workflow can be executed iff the individual step can be rolled back and its commitment does not block any of the previously
executed steps from being compensated if needed.
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 12
What Are Invariants For?
Compensation must be guaranteed for completed steps => certain predicates on the shared and local state must
be maintained. The requirements for a state to be executable
are formalized as combined event / state predicates called “invariants”.
Invariants are alos useful to describe correctness criteria for forward execution.
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 13
Types of Invariants
Entry invariants guard the execution of a step. If an entry invariant is violated, there are different
options: give up (compensate), negotiate, resolve conflict.
Exit invariants formalize the new consistent state. Its protection can be strict (must), moderate (want), lose (hope).
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 14
Invariants: Virtual Objects
Invariants may contain expressions like: obj_1 + obj_2 rel_op value. The objects are not necessarily managed by the
same RM. To support such invariants, they are established
as virtual objects, which: have a special name, have a “value” method, are stored at each participating RM, are evaluated locally whenever possible.
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 15
Dynamic Aspects of Invariants
p1
pa2 pa3 pa4
pb2 pb3 pb4
Individual invariants established by each step
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 16
p1
p1&pa2 p1&pa2&pa3 p1&pa2&pa3&pa4
p1&pb2 p1&pb2&pb3 p1&pb2&pb3&pb4
Accumulated invariants for a “case of” control flow
Dynamic Aspects of Invariants
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 17
Dynamic Aspects of Invariants
Invariants can be deleted if the step with the corresponding entry invariant will never be executed. This implies: All invariants become obsolete at the end of a
workflow. Dead code must be detected dynamically. We need special loop invariants.
If a step´s compensation step is dynamically modified, this may cause problems.
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 18
Supporting Invariants
In order to support invariants, database systems must give up some their autonomy. In particular, they have to: provide notification about lock conflicts, implement recoverable locks, implement semantic locks (e.g. escrow), implement existence locks at the tuple and at the
schema level.
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 19
Using Transactions in a Workflow-System
Application-level transactions for grouping multiple steps
Transaction 1
Transaction 2
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 20
B C
System-level transactions for transferring control from one step to the next
Using Transactions in a Workflow-System
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 21
B C
System-level transactions for transferring control from one step to the next
Using Transactions in a Workflow-System
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 22
C
System-level transactions for transferring control from one step to the next
B
CM
transfercontrol
receiverequest
notifyCM
notifyCM
System transaction A
Using Transactions in a Workflow-System
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 23
B C
Input queuecommit
localprocessing
post
Version 1: Synchronous transfer
Workflow and Transactional Queues
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 24
B C
Input queuecommit
localprocessing
Version 2: Asynchronous transfer
outputqueue
Commit-Post-Transaction
Transfer Transaction
Workflow and Transactional Queues
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 25
C
Input queue Output queue
Local code
1. dequeue
2. local execution
3. post
Completelocal transactionincl. transferof control
Queue-Driven Step Processing
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 26
C
Normal processing
dequeuelocalprocessing
Transaction abort
localrollbackpost
Queues And Rollback
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 27
Problems Related to Rollback
Why did the local transaction fail (system abort or application-initiated rollback?)
In which cases should the TA be re-posted (and how often)?
Who gets notified about an abort (source or CM)? Who gets notified about the eventual failure to restart a
transaction? Should application-level TAs be treated as distributed or
nested transactions at the system level? Which programming level should handle these issues
(step or script)?
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 28
Summing It Up - 1
Transactional concepts can be carried over to workflow management in multiple ways and at differet levels.
The most obvious application of transaction technology is at the level of the workflow engine, where transactions provide persistent execution of a script, local recovery in case of partial failures, reliable state transitions, recoverable events, and consistent context.
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 29
Summing It Up - 2
Transactions at the system level need a number of extensions: nesting, chaining, leave-resume, transfer.
Participating resource managers need to be able to support an open distributed two-phase-commit protocol.
Persistent storage managers need a number of functional extensions such as recoverable locks.
© Jim Gray, Andreas Reuter Transaction Processing - Concepts and Techniques WICS August 2 - 6, 1999 30
Summing It Up - 3
At the step level, transactions provide atomicity for short-term related computations.
The TM must be able to support dependen-cies among transcations.
At the script level, the concepts of atomicity and consistency have to be translated into more abstract notions (compensation, invariants).