ADBIS 2007; Varna, Bulgaria; 03.10.2007

Towards Self-Optimization of MessageTransformation Processes

Matthias Böhm1,2,3 *, Dirk Habich2, Uwe Wloka3, Jürgen Bittner1, and Wolfgang Lehner2

1 SQL GmbH Dresden, Germany2 Dresden University of Technology, Database Technology Group

3 University of Applied Sciences Dresden, Database Group

2 / 28

Outline

• Introduction

• Integration Platform TransConnect®

• Process Optimization Techniques

• Summary and Conclusion

3 / 28

Outline

• Introduction

• Integration Platform TransConnect®

• Process Optimization Techniques

• Summary and Conclusion

4 / 28

Introduction

• starting point– integration of heterogenous information systems– horizontal service integration by message-based communication

using the Message Transformation Model (MTM)

• motivation / problem description– suboptimal modeled processes – dynamic workload characteristics– total costs of ownership

• contribution towards self-optimization– first rule-based optimization techniques– first workload-based optimization techniques– prototypical implementation within TransConnect

5 / 28

Introduction

• Message Transformation Model (MTM)

6 / 28

Introduction

• Message Tansformation Model (MTM)– Message Model

– Process Model (reconsidered)• Interaction-oriented activ.• Control-flow-oriented activ.• Data-flow-oriented activ.

Base model "Directed Graph"

Hierarchical message structure

7 / 28

Introduction

• Message Tansformation Model (MTM)– Example Process

8 / 28

Outline

• Introduction

• Integration Platform TransConnect®

• Process Optimization Techniques

• Summary and Conclusion

9 / 28

Integration Platform TransConnect

• TransConnect– message based

application integration– inbound adapters– outbound adapters– process engine

• TransConnect 1.3.6overall architecture

10 / 28

Integration Platform TransConnect

• TransConnect 1.3.6 Server architecture

11 / 28

Integration Platform TransConnect

• Component ProcessParser

12 / 28

Integration Platform TransConnect

• External Layer:

WSBPEL 2.0 process

<process xmlns=""> <!--declarations--> <!– process description--></process>

13 / 28

Integration Platform TransConnect

• Conceptual Layer:

MTM process type

14 / 28

Integration Platform TransConnect

• Internal Layer: JAVA process plan

public class es_process1 extends ProcessPlan

{

private InternalMessage msg1 = null;

private InternalMessage msg2 = null;

@Override

protected InternalMessage executeNode(InternalMessage input)

throws MTMException

{

try

{

Invoke node1 = new Invoke("sap_mq","DEQUEUE",AService.OTYPE_RECEIVE);

node1.setIDs(getPTID(), getPID(), getNID());

msg1 = node1.execute( msg3 );

}

catch( MTMSignalException mse )

{

/*signal handling*/

}

/*...*/

}

}

15 / 28

Integration Platform TransConnect

• Component SystemMonitor– interval monitoring / continuous monitoring– determination of suboptimal process plans– recompilation of process plans

– Self-Optimization according to IBM MAPE concept

(Monitor,

Analyse,

Plan,

Execute)

adaptive optimization

strategies

16 / 28

Integration Platform TransConnect

• Component SystemMonitor– Inbound monitor events: performance measurement

-- average process type execution time -- (not normalized!)

SELECT AVG(EndTime - StartTime) FROM ProcessingPerformance WHERE NID = -1 AND -- node type process PID IN ( SELECT PID FROM Process WHERE PTID = (SELECT PTID FROM ProcessType WHERE Name=‘es_process1‘))

17 / 28

Outline

• Introduction

• Integration Platform TransConnect®

• Process Optimization Techniques

• Summary and Conclusion

18 / 28

Process Optimization Techniques

• influencing-factors– optimization aim: throughput / execution time– execution knowledge: statistics / ad-hoc– optimization techniques: rule-based / workload-based

• technique classification– Rule-based process optimization

• Control flow optimization• Data flow optimization

– Workload-based process optimization• Message indexing• Control flow optimization• Data flow optimization

19 / 28

Process Optimization Techniques

• Rule-based process optimization– Control flow optimization

• Redundant control flow elimination

• Unreachable subgraph elimination• Preventing local subprocess invocation

20 / 28

Process Optimization Techniques

• Rule-based process optimization– Data flow optimization

• Double Variable Assignments• Unnecessary Variable Assignments• Unnecessary Variable Declarations• Two sibling Tanslation operators• Unnecessary Switch-paths• Two sibling validations

Basically these techniques are adopted from imperative programming language compilers

21 / 28

Process Optimization Techniques

• Workload-based process optimization– Message indexing

22 / 28

Process Optimization Techniques

• Workload-based process optimization– Message indexing

– Control flow optimization• query scrambling techniques

(external systems delay,

network delay elimination, and

disk I/O delay)

• parallel flow management

23 / 28

Process Optimization Techniques

• Workload-based process optimization– Data flow optimization

•Switch operator optimization

24 / 28

Process Optimization Techniques

• Evaluation Experiment - "Complex Integration Process"

rule-based and

workload-based process plan

rewriting

25 / 28

Process Optimization Techniques

• Evaluation Experiment - "Complex Integration Process"– average inbound message size: 7KB

26 / 28

Outline

• Introduction

• Integration Platform TransConnect®

• Process Optimization Techniques

• Summary and Conclusion

27 / 28

Summary and Conclusion

• Summary– optimization techniques were illustrated– implementation and evaluation prove the high optimization potential– lots of further research items along

• Conclusion– research of optimization techniques will be displaced from the

grounding systems to the integration process

• Future work– DIPBench (Data-Intensive Integration Process Benchmark)– GCIP (Model-Driven Generation and Optimization of Complex

Integration Processes)– MIX (Message Indexing for Document-Oriented Integration

Processes)– Adaptive Enterprise Integration Platform

ADBIS 2007; Varna, Bulgaria; 03.10.2007

Towards Self-Optimization of MessageTransformation Processes

Matthias Böhm1,2,3 *, Dirk Habich2, Uwe Wloka3, Jürgen Bittner1, and Wolfgang Lehner2

1 SQL GmbH Dresden, Germany2 Dresden University of Technology, Database Technology Group

3 University of Applied Sciences Dresden, Database Group