A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek...
-
Upload
jamir-saltonstall -
Category
Documents
-
view
214 -
download
0
Transcript of A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek...
A Model Driven Tool for Automated System Level
Testing of MiddlewareTurker KeskinpalaAbhishek DubeySteve Nordstrom
Ted BaptySandeep Neema
Institute for Software Integrated SystemsVanderbilt University
Nashville, TN USA
STV'06 WorkshopMarch 30, 2006
Outline
Motivation and Challenges Model Driven Approach Tool Structure Test Series Definition Modeling
Language Example Test Series Definition Conclusions and Ongoing Work
STV'06 WorkshopMarch 30, 2006
Motivation and Challenges Many middleware framework configurations are
generally represented by XML Configuring functional operation of components Component deployment in the cluster
Frameworks are complicated and need to be tested Application topologies Parameter variations
Need for many test configurations
STV'06 WorkshopMarch 30, 2006
Motivation and Challenges
Hand writing test configurations Cumbersome Error prone Inefficient
Big configuration space of control and deployment parameters of applications
Response to changes in hardware resources or other criteria Difficult to scale up manually written XML
configurations
Usual outcome: Under tested software
STV'06 WorkshopMarch 30, 2006
Goals
Testing large sets of system configurations Simple powerful specifications Tool to take specifications to instances
Configure series of tests from the specifications
Help finding “corner cases” Potential functional errors Potential performance problems Unexpected system interactions
STV'06 WorkshopMarch 30, 2006
Model Driven Approach
Domain Specific Modeling Use domain specific models to encapsulate relevant
parts of the system properties
Provide multiple view points Decouple types, resources, dataflow and
deployment aspects
Generative modeling Vary range of behavioral and structural system
parameters
System generation Automatically generate configurations from higher
level models
STV'06 WorkshopMarch 30, 2006
Tool Structure
Test Series Definition Models
STV'06 WorkshopMarch 30, 2006
Test Series Definition Define tests using a high level modeling
language called Test Series Definition Modeling Language (TSDML) Define application types Form a template test case from those types Define dataflow association between applications
by connection rules Define deployment associations for applications Define the variance on the system size and
structure Define variance on the application parameters Define available resources
Automatically create series of test configuration models by replicating and adapting the template test case
STV'06 WorkshopMarch 30, 2006
Tool Structure
Test Series Definition Models
Generator (1-*)
Test Configuration Models
STV'06 WorkshopMarch 30, 2006
Test Configuration Models
Form the configuration layer Each test configuration model capture:
Specific dataflow scenario Specific deployment scenario Configuration Schema
Represents single system representation to be tested
Automatically generate single configuration file per configuration model
Test driver scripts can also be generated
STV'06 WorkshopMarch 30, 2006
Tool Structure
Test Series Definition Models
Generator (1-*)
Test Configuration Models
XML Configs Test Scripts
Generator (1)
Application Runtime (System Under Test)
Test E
xecutio
n &
V
alida
tion
En
gin
eApplications
STV'06 WorkshopMarch 30, 2006
System Under Test
XDAQ Framework High performance distributed middleware
framework developed at CERN Will be used for Compact Muon Selenoid
(CMS) experiment Run-time support for clustered applications Predefined functions to configure and
control the applications and their parameters
XML is used for configuration of XDAQ and its applications
STV'06 WorkshopMarch 30, 2006
Test Series Definition Modeling Language (TSDML)
Parameterized model Varying system structure
(size) Iterators: e.g. i,j,k = {start,
step, stop} Replicators: e.g. r1=n*i,
r2=m*j Connectors: Boolean
function of iterator, and replicator
If src=(i,r1) and dst=(j,r2) C=f(src,dst) C=2*src==3*dst
i
r1
App1
j
App2
r2
Connector
STV'06 WorkshopMarch 30, 2006
Iterator-Replicator Example
Assume i = 1:1:2, j= 1:1:4, r1 = 2, r2 = 4
r2
i
r1
App1
j
App2
Test Case i j #App1 #App2
1 1 1 2 4
2 1 2 2 8
3 1 3 2 12
4 1 4 2 16
5 2 1 4 4
6 2 2 4 8
7 2 3 4 12
8 2 4 4 16
STV'06 WorkshopMarch 30, 2006
Connector Examplei = 1:1:2, j= 1:1:4,
r1 = 2, r2 = 3CR = src=floor((dst-1)/(j*r2/i*r1)) + 1
i
r1
App1
i
App2
r2
Test Case i j #App1 #App2
1 1 1 2 4
2 1 2 2 8
3 1 3 2 12
4 1 4 2 16
5 2 1 4 4
6 2 2 4 8
7 2 3 4 12
8 2 4 4 16Connector
STV'06 WorkshopMarch 30, 2006
Connector Example (ctd…)
App1
App2
App1
App2
App2
App2
(1)
(2)
(1)
d
s
t
s
r
c
Rule:
src=floor((dst-1)/(j*r2/i*r1)) + 1
(2)
(3)
(4)
1=floor(0/2) + 1 => TRUE1=floor(1/2) + 1 => TRUE1=floor(2/2) + 1 => FALSE1=floor(3/2) + 1 => FALSE
STV'06 WorkshopMarch 30, 2006
Test Series Definition Modeling Language (TSDML)
Varying application parameters with parameter generator Has a function attribute Function of iterator Function attribute may be:
Internal function e.g. result = src * 256 where src = i
Matlab script e.g. lut = [10 20 55 88 400 800
1025 4490 7342]; result = lut[src]; Internal lookup table
STV'06 WorkshopMarch 30, 2006
Example TSDML
STV'06 WorkshopMarch 30, 2006
Example TSDML
2*i
1:1:8 1:1:1
1:1:1
2*i16*
i
1*i
STV'06 WorkshopMarch 30, 2006
Conclusion
Domain specific modeling for capturing relevant system parts
Vary system structure and application parameters from high level model
Generate series of test configurations from a single high level model
Generate the configuration representation
Mitigate the complexity of creating many system test configurations
STV'06 WorkshopMarch 30, 2006
Ongoing Work
Add another layer on top of TSDML System model
Application behavior Target framework
Analyze the models Assess important parts of the system
to exercise and test
STV'06 WorkshopMarch 30, 2006
Thank you!
A Model Driven Tool for Automated System Level
Testing of MiddlewareTurker KeskinpalaAbhishek DubeySteve Nordstrom
Ted BaptySandeep Neema
Institute for Software Integrated SystemsVanderbilt University
Nashville, TN USA