Introduction to Test AutomationIntroduction to Test Automation

Jerzy.Nawrocki@put.poznan.plwww.cs.put.poznan.pl/jnawrocki/models/

Models and Analysis of SoftwareLecture 8

Copyright, 2003 Jerzy R. Nawrocki

What is test automation?What is test automation?

That’s simple!

1. Buy a test execution tool.

2. Record the manual tests.

3. Play the tests back whenever you want to.

„Just as there is more to software design than knowing a programming language, there is more to automating testing than knowing a testing tool.”

-- M. Fewster & D. Graham

Quality attributes of a test caseQuality attributes of a test case

• How effective in detecting defects?

• How exemplary? (the more exemplary, the less test cases needed)

• How economic?

• How evolvable? (maintenance effort)

Test automation effectsTest automation effects

Effective

Exemplary

EvolvableEconomic

Test activitiesTest activities

Identify test conditions (‘what’ to test) and prioritize

Design test cases (‘how’ to test)

Build test cases (scripts, data etc.)

Execute test cases

Compare test outcomes to expected outcomes

Example test conditionsExample test conditions

• Order created for a single item (VB10)

• Order quantity of 100 (VB23)

• Order cancelled (V8)

Item Price [zl]

Quantity

Rose 2 5

Bag 3 1

An example test caseAn example test casePrerequisites:• logged into the purchase order system as a data entry clerk;• database system must contain the standard Data Set;• no other new purchase order activity on the system.Step Input Exp.out Cond.

1 Create a new order for any one standard order, setting quantity to 100

Order confirmation message displayed

VB10VB23

2 Confirm the order Purchase order printed VB10

3 Print a new orders report New orders report printed showing just this one new order

VB10VB23

4 Cancel the order Purchase order cancellation notice printed

V8

The limitations of automating software testingThe limitations of automating software testing

• Does not replace manual testing

• Manual tests find more defects than automated tests

James Bach: automated tests= 15%, manual= 85%

• Test automation does not improve effectiveness

• Test automation may limit software evolution

• Tools have no imagination

BibliographyBibliography

M. Fewster, D. Graham, Software M. Fewster, D. Graham, Software Test Automation, Addison-Wesley, Test Automation, Addison-Wesley, Harlow, 1999.Harlow, 1999.

QuestionsQuestions??