Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference...
-
Upload
edward-held -
Category
Documents
-
view
212 -
download
0
Transcript of Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference...
![Page 1: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/1.jpg)
Rational Israel Users Conference
Automated Model Driven Automated Model Driven Software TestingSoftware Testing
Rational Users Conference
Tel Aviv
June 7 2004
Alan Hartman
IBM Israel – Haifa Research Laboratory
![Page 2: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/2.jpg)
Rational Israel Users Conference
AgendaAgenda
• AGEDIS Project Overview• Architecture• Methodology• Interfaces• Tools• Future Plans
![Page 3: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/3.jpg)
Rational Israel Users Conference
AGEDIS OverviewAGEDIS Overview
• Automated model-based test Generation and Execution for DIStributed systems
• Methodology and tools for model-based testing• Open interfaces• Mixture of academic and industrial partners• Three phase timetable of experiment and
development• November 2001- February 2004• Partially funded by the European Commission
![Page 4: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/4.jpg)
Rational Israel Users Conference
Consortium PartnersConsortium Partners
• IBM Haifa Research Lab• Oxford University• VERIMAG/IRISA• Imbus• France Telecom• IBM UK• Intrasoft International
![Page 5: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/5.jpg)
Rational Israel Users Conference
Methodology & Architecture Methodology & Architecture
• UML based component testing
• Compliant with RUP• High degree of
automation• No silver bullet
![Page 6: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/6.jpg)
Rational Israel Users Conference
Automated Testing MethodologyAutomated Testing Methodology
![Page 7: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/7.jpg)
Rational Israel Users Conference
Automated Testing ArchitectureAutomated Testing ArchitectureG
UI
& P
rodu
ctiv
ity
Aid
s Model
GenerationDirectives
ExecutionDirectives
Compiler IntermediateFormat
Simulator
Generator
AbstractTest Suite
Suite Execution
Trace
Analyzers
VisualizerEditor
Execution
![Page 8: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/8.jpg)
Rational Israel Users Conference
BenefitsBenefits• Starting from specification
– Involves testers early in the development process – Teams testers with developers– Forces testability into product design
• Building behavioural model and test interface– Finds design and specification bugs - before code exists– The model is the test plan - and is easily maintained
• Automated test suite generation– Coverage is guaranteed - increases testing thoroughness– Matches coverage goals to testing budget– Zero test suite maintenance costs
• Automated test suite execution– Finds code and interface bugs– Includes a framework for the testing of distributed applications– Reduces test execution costs
![Page 9: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/9.jpg)
Rational Israel Users Conference
InterfacesInterfaces
• UML Profile for AGEDIS• Test Generation Directives• Test Execution Directives• IF Model Execution Interface• Abstract Test Suite• Suite Execution Trace
UML
TGD
TED
IF
ATS
SET
![Page 10: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/10.jpg)
Rational Israel Users Conference
User Modeling InterfaceUser Modeling Interface
• The AGEDIS Modeling Language is a profile for UML 1.4:– UML Class diagrams - structure– UML Object diagrams - snapshots– UML State diagrams – behaviour & test
purposes
• Annotated with an action language – IF
![Page 11: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/11.jpg)
Rational Israel Users Conference
Test Generation DirectivesTest Generation Directives
• Test Purposes – stereotyped system level state diagrams
• Coverage Directives – stereotyped system level state diagrams
• Coverage without tears – 4 levels – parameters to the test generation algorithm
![Page 12: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/12.jpg)
Rational Israel Users Conference
Test Execution DirectivesTest Execution Directives
• XML Schema– Model translation from model abstractions to
SUT concrete implementations– Test architecture
• Host addresses
• Delays, timeouts
• Prologues, epilogues
![Page 13: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/13.jpg)
Rational Israel Users Conference
IF - Model Execution InterfaceIF - Model Execution Interface
• IF Encoding of the model– Classes– Objects– State machines
![Page 14: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/14.jpg)
Rational Israel Users Conference
Test Suite and Trace InterfaceTest Suite and Trace Interface
• XML schema – for test execution and tracing
• Model description– classes : constants, types, control & observable
signatures– a special class is defined for the tester– object identities
• Test Suite - set of test cases• Test Trace – record of executed test cases
![Page 15: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/15.jpg)
Rational Israel Users Conference
Test Case and Test TraceTest Case and Test Trace
• Interaction graphs - between the tester and the SUT
• Associated verdicts (Pass, Fail, Inconclusive)
• Alternate paths for non-determinism in test cases
• Expected results in test suite• Observed results and verdicts in trace
![Page 16: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/16.jpg)
Rational Israel Users Conference
Abstract Test SuiteAbstract Test Suite
![Page 17: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/17.jpg)
Rational Israel Users Conference
ToolsTools
• User Interface• Modeler & Model Compiler• Model Simulator• Test Generator• Test Execution Engine• Test Suite/Trace Viewer/Editor• Feedback & Analysis• Bug Reporter• Report Generator
UI &
RG
CSG
FA
VEEE
![Page 18: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/18.jpg)
Rational Israel Users Conference
GUIGUI
![Page 19: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/19.jpg)
Rational Israel Users Conference
ModelingModeling Tool & CompilerTool & Compiler
• Objecteering UML modeling tool
• Tool profile to convert to XML
• General purpose XML to IF compiler– Written in Java, with
XMI in mind as a future input format
![Page 20: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/20.jpg)
Rational Israel Users Conference
Model SimulatorModel Simulator
![Page 21: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/21.jpg)
Rational Israel Users Conference
Test GeneratorTest Generator
• Based on GOTCHA and TGV
• GOTCHA – uses Murphi specification language
– explicit traversal of state space
– extensive coverage criteria
• TGV– language independent simulator
– focus on distributed applications
– explicit test purposes as sequences of interactions
![Page 22: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/22.jpg)
Rational Israel Users Conference
Test Execution EngineTest Execution Engine– Input: Abstract Test Suite and
Test Execution Directives– Output: Suite Execution Trace– Multiple platform (e.g. Linux,
Windows) support– Multiple external interface
(Java,C++,C) support– Test distribution + centralized
logging– Interactive test case execution– GUI to view execution progress,
inputs, and outputs
![Page 23: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/23.jpg)
Rational Israel Users Conference
Execution Engine ArchitectureExecution Engine Architecture
N e t w o r k
Test SuiteDriver
I/O
HostManager
ProcessManager
1 1 on each host 1..N on each host
. . .
Object
Object
![Page 24: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/24.jpg)
Rational Israel Users Conference
Feedback ToolsFeedback Tools• Coverage analysis
– Detect uncovered areas of the model in either test suite or test trace
– Create test purposes to reach them– Invokes a functional coverage tool from
www.alphaworks.com
• Defect analysis– Clustering of defects– Feature extraction from clusters– Create test purposes to reproduce the bug
![Page 25: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/25.jpg)
Rational Israel Users Conference
AGEDIS Future PlansAGEDIS Future Plans
![Page 26: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/26.jpg)
Rational Israel Users Conference
Future PlansFuture Plans
• Exploitation Activity– Available under license for non-commercial use– Available as a service
• Incorporation in wider Model Driven SE Effort– Integrate methodology with RUP– Migrate to Eclipse, XDE– Focus on business integration domain
![Page 27: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/27.jpg)
Rational Israel Users Conference
ModelWareModelWare• New European Project• Model driven software engineering• Eclipse based tools for UML modeling, and the
full MDA lifecycle– Model driven architecture
– Platform independent and platform specific models
– Model simulation
– Model transformation
![Page 28: Rational Israel Users Conference Automated Model Driven Software Testing Rational Users Conference Tel Aviv June 7 2004 Alan Hartman IBM Israel – Haifa.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c925503460f9494d4d5/html5/thumbnails/28.jpg)
Rational Israel Users Conference
Thanks to:Thanks to:• A. Ramfos., S. Liapis, N.
Giannelos, A. Hondouridakis, M. Sardis, K. Bechrakis, V. Akousi-Krivki
• K. Dussa-Zieger, J. Trost, B. Nossem, T. Linz, B. Mattern, J. Hofer, H. Raessler, T. Rossner,
• I. Craggs, I. Griffiths• Y.-M. Quemener, D. Vincent,
T. Heuillard, N. Moteau• A.Bertolino, A. Wills, S. Reid
• K. Nagin, O. Edelstein, A. Kirshin, S. Olvovsky, M. Berg, L. Raskin, T. Shiran, C. Sacharen, M. Barshay, D. Neimer
• L. Mounier, M. Bozga, Y. Lakhnech
• T. Jeron, E. Demairy, V. Tschaen
• J. Davies, A. Cavarra, C. Crichton, J. Woodcock, M. Field