Pilot Model Based Testing
-
Upload
ewald-roodenrijs -
Category
Business
-
view
579 -
download
1
description
Transcript of Pilot Model Based Testing
Pilot Model Based Testing
using Smartesting Test DesignerTM
3/12/08 - 5/12/08
Agenda
• Background
• Objective of the pilot
• Findings
• Conclusions & Recommendations
Background (1/2)
Expectations of model based testing...• Increased speed in test case specification• Excellent first step towards automated test
execution• Transferable and reusable testware
> opportunity in combination with India route?
• Improved involvement in requirements review> Shift up in the V-model
a possible new service offering to customers and/or in testlines
Background (2/2)
• Activities> Modelling by Smartesting> Turn around time: 3 days
• FEBO> Front end application for ....> Requirements used : release 2.1 (2.2
currently being built)> .NET application, back end simulated by
stubs> Vista based
Objective of the pilot (1/2)
In general1. To gain hands on experience with
Model Based Testing (MBT)
Specifically2. To assess the learning curve of the
UML models used> Class diagram> State diagram> Object diagram> OCL
Objective of the pilot (2/2)
3. To assess maintenance effort testware
4. To assess alignment with Test Strategy decisions
> How can test design techniques be utilized;> What coverage can be achieved;> How can coverage be influenced.
5. To compare effort and outcome with ‘normal’ test specification
> Time spent on preparation > Time spent on creating test cases> Number of test cases created
Findings (1/5)
1. and 2. Hands on experience;Learning curve> Due to reduced scope and turn around time:
no real result
3. Maintenance> Requirement change incorporated in model> Quick change
Findings (2/5)
4. Alignment with Test Strategy> Only possible to tailor (automated)
coverage with test depth in PCT> Test design techniques that can be used
are: PCT, UCT, DCoT, SEM, SYN, DTT, ECT
> To combine MBT with more test design techniques the maintenance is under pressure
Two separate activities
Findings (3/5)
5. Comparison to manual test specification
Time (hours)
Test level Use Case Manual MBT
Preparation C005 3 3
Preparation B001 3 1 (part)
Specification C005 8 5
Specification B001 24 3 (part)
Change B001 28 3
Findings (4/5)
5. Comparison to manual test specification
Test cases
Use Case Manual MBT
C005 45 21
B001 120 4 (part)
Change B001 - 18
Findings (5/5)
6. Extra: Infrastructure> Windows Vista not supported by
Smartesting Test Designer cannot be installed on Windows Vista
Sogeti> Need license for IBM Rational Software
Modeler> Export to HP Quality Center: OK> Export to IBM Raitional Quality Manager:
not yet available
Conclusions & Recommendations (1/5)
Overall:• Positive introduction• to small scale pilot: no statistical
metrics possible• Good and clear interaction with
analyst/designer> Same level of knowledge> Increases results tester> Increases acceptance designer
• Find defects in requirements
• Test Design Techniques> PCT: very suitable (adjustable test depth)> DCoT/SEM/SYN: auditable with adjustments
in test data> More analyses/experience is needed to
optimize the use of test design techniques with MBT
• Manual execution of model bases test cases is easy> Recommendation to use meaningful variables> focus not only on automated test execution
but also manuallyExtra service testers Sogeti?
Conclusions & Recommendations (2/5)
• Current infrastructure not suitable for Test Designer> Windows Vista Sogeti> Support Windows Vista> No export to Rational Quality Manager
• OCL looks a lot like programming> Not every tester will have the required skills
• Extra skill required/extra role• Suitable for outsourcing/offshoring
> Create models internally> Create and execute test cases externally
Conclusions & Recommendations (3/5)
• No discussion about who creates model> In test model there are items not used in
design modelBooleansTest data
> Customizing model takes longer> Creating model filters defects in design
Fully independent
Conclusions & Recommendations (4/5)
Conclusions & Recommendations (5/5)
• Prolong the pilot by course in MBT> Creating models and test cases> Teach this knowledge
• Use this knowledge in a real testline> Small project
• Add to services Software Control> Integrate in STaaS
TestManagement
tool
Nummer Testdoelen
1 Toon aan dat de huidige functionaliteit 1-op-1 is gemigreerd naar het nieuwe systeem
1.1 Toon aan dat de huidige functies uit “Handboek Balie” goed werken 1.2 Alle gangbare betaalvormen zijn mogelijk 1.3 De bron van kaartverkoop is traceerbaar
2 Lever inzicht in het gemak waarmee de gebruikers het nieuwe systeem kunnen gebruiken
2.1 Kaartjes kunnen makkelijk en snel worden geprint 2.2 Prijzen kunnen snel worden opgezocht 2.3 De front office applicatie van het call centre werkt 3 3.1 …
TestbasisTestgevallen
Models (UML)
TestRepository
Review
Test data
Test Cases
Review
Risks
Req’sKenmerk - deelobject
RK Opmerkingen
Functionaliteit - deelsysteem 1 A … - deelsysteem 2 C … - interface B … Gebruiksvriendelijkheid - online B …
TestAutomation