Post on 01-Apr-2015
ADBIS 2007; Varna, Bulgaria; 03.10.2007
Towards Self-Optimization of MessageTransformation Processes
Matthias Böhm1,2,3 *, Dirk Habich2, Uwe Wloka3, Jürgen Bittner1, and Wolfgang Lehner2
1 SQL GmbH Dresden, Germany2 Dresden University of Technology, Database Technology Group
3 University of Applied Sciences Dresden, Database Group
2 / 28
Outline
• Introduction
• Integration Platform TransConnect®
• Process Optimization Techniques
• Summary and Conclusion
3 / 28
Outline
• Introduction
• Integration Platform TransConnect®
• Process Optimization Techniques
• Summary and Conclusion
4 / 28
Introduction
• starting point– integration of heterogenous information systems– horizontal service integration by message-based communication
using the Message Transformation Model (MTM)
• motivation / problem description– suboptimal modeled processes – dynamic workload characteristics– total costs of ownership
• contribution towards self-optimization– first rule-based optimization techniques– first workload-based optimization techniques– prototypical implementation within TransConnect
5 / 28
Introduction
• Message Transformation Model (MTM)
6 / 28
Introduction
• Message Tansformation Model (MTM)– Message Model
– Process Model (reconsidered)• Interaction-oriented activ.• Control-flow-oriented activ.• Data-flow-oriented activ.
Base model "Directed Graph"
Hierarchical message structure
7 / 28
Introduction
• Message Tansformation Model (MTM)– Example Process
8 / 28
Outline
• Introduction
• Integration Platform TransConnect®
• Process Optimization Techniques
• Summary and Conclusion
9 / 28
Integration Platform TransConnect
• TransConnect– message based
application integration– inbound adapters– outbound adapters– process engine
• TransConnect 1.3.6overall architecture
10 / 28
Integration Platform TransConnect
• TransConnect 1.3.6 Server architecture
11 / 28
Integration Platform TransConnect
• Component ProcessParser
12 / 28
Integration Platform TransConnect
• External Layer:
WSBPEL 2.0 process
<process xmlns=""> <!--declarations--> <!– process description--></process>
13 / 28
Integration Platform TransConnect
• Conceptual Layer:
MTM process type
14 / 28
Integration Platform TransConnect
• Internal Layer: JAVA process plan
public class es_process1 extends ProcessPlan
{
private InternalMessage msg1 = null;
private InternalMessage msg2 = null;
@Override
protected InternalMessage executeNode(InternalMessage input)
throws MTMException
{
try
{
Invoke node1 = new Invoke("sap_mq","DEQUEUE",AService.OTYPE_RECEIVE);
node1.setIDs(getPTID(), getPID(), getNID());
msg1 = node1.execute( msg3 );
}
catch( MTMSignalException mse )
{
/*signal handling*/
}
/*...*/
}
}
15 / 28
Integration Platform TransConnect
• Component SystemMonitor– interval monitoring / continuous monitoring– determination of suboptimal process plans– recompilation of process plans
– Self-Optimization according to IBM MAPE concept
(Monitor,
Analyse,
Plan,
Execute)
adaptive optimization
strategies
16 / 28
Integration Platform TransConnect
• Component SystemMonitor– Inbound monitor events: performance measurement
-- average process type execution time -- (not normalized!)
SELECT AVG(EndTime - StartTime) FROM ProcessingPerformance WHERE NID = -1 AND -- node type process PID IN ( SELECT PID FROM Process WHERE PTID = (SELECT PTID FROM ProcessType WHERE Name=‘es_process1‘))
17 / 28
Outline
• Introduction
• Integration Platform TransConnect®
• Process Optimization Techniques
• Summary and Conclusion
18 / 28
Process Optimization Techniques
• influencing-factors– optimization aim: throughput / execution time– execution knowledge: statistics / ad-hoc– optimization techniques: rule-based / workload-based
• technique classification– Rule-based process optimization
• Control flow optimization• Data flow optimization
– Workload-based process optimization• Message indexing• Control flow optimization• Data flow optimization
19 / 28
Process Optimization Techniques
• Rule-based process optimization– Control flow optimization
• Redundant control flow elimination
• Unreachable subgraph elimination• Preventing local subprocess invocation
20 / 28
Process Optimization Techniques
• Rule-based process optimization– Data flow optimization
• Double Variable Assignments• Unnecessary Variable Assignments• Unnecessary Variable Declarations• Two sibling Tanslation operators• Unnecessary Switch-paths• Two sibling validations
Basically these techniques are adopted from imperative programming language compilers
21 / 28
Process Optimization Techniques
• Workload-based process optimization– Message indexing
22 / 28
Process Optimization Techniques
• Workload-based process optimization– Message indexing
– Control flow optimization• query scrambling techniques
(external systems delay,
network delay elimination, and
disk I/O delay)
• parallel flow management
23 / 28
Process Optimization Techniques
• Workload-based process optimization– Data flow optimization
•Switch operator optimization
24 / 28
Process Optimization Techniques
• Evaluation Experiment - "Complex Integration Process"
rule-based and
workload-based process plan
rewriting
25 / 28
Process Optimization Techniques
• Evaluation Experiment - "Complex Integration Process"– average inbound message size: 7KB
26 / 28
Outline
• Introduction
• Integration Platform TransConnect®
• Process Optimization Techniques
• Summary and Conclusion
27 / 28
Summary and Conclusion
• Summary– optimization techniques were illustrated– implementation and evaluation prove the high optimization potential– lots of further research items along
• Conclusion– research of optimization techniques will be displaced from the
grounding systems to the integration process
• Future work– DIPBench (Data-Intensive Integration Process Benchmark)– GCIP (Model-Driven Generation and Optimization of Complex
Integration Processes)– MIX (Message Indexing for Document-Oriented Integration
Processes)– Adaptive Enterprise Integration Platform
ADBIS 2007; Varna, Bulgaria; 03.10.2007
Towards Self-Optimization of MessageTransformation Processes
Matthias Böhm1,2,3 *, Dirk Habich2, Uwe Wloka3, Jürgen Bittner1, and Wolfgang Lehner2
1 SQL GmbH Dresden, Germany2 Dresden University of Technology, Database Technology Group
3 University of Applied Sciences Dresden, Database Group