Chapter 14 Software Testing Techniques - Aalborg...
Transcript of Chapter 14 Software Testing Techniques - Aalborg...
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 1
Software Engineering: A PractitionerSoftware Engineering: A Practitioner’’s Approach, 6/es Approach, 6/e
Chapter 14Chapter 14Software Testing TechniquesSoftware Testing Techniques
copyright © 1996, 2001, 2005R.S. Pressman & Associates, Inc.
For University Use OnlyMay be reproduced ONLY for student use at the university level
when used in conjunction with Software Engineering: A Practitioner's Approach.Any other reproduction or use is expressly prohibited.
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 2
TestabilityTestability OperabilityOperability——it operates cleanlyit operates cleanly ObservabilityObservability——the the results of each test case are readilyresults of each test case are readily
observedobserved ControllabilityControllability——the degree to which testing can bethe degree to which testing can be
automated and optimizedautomated and optimized DecomposabilityDecomposability——testing can be targetedtesting can be targeted SimplicitySimplicity——reduce complex architecture and logic toreduce complex architecture and logic to
simplify testssimplify tests StabilityStability——few changes are requested during testingfew changes are requested during testing UnderstandabilityUnderstandability——of the designof the design
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 3
What is a What is a ““GoodGood”” Test? Test?
A good test has a high probability of findingA good test has a high probability of findingan erroran error
A good test is not redundant.A good test is not redundant. A good test should be A good test should be ““best of breedbest of breed”” A good test should be neither too simple norA good test should be neither too simple nor
too complextoo complex
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 4
Test Case DesignTest Case Design
"Bugs lurk in corners "Bugs lurk in corners and congregate at and congregate at boundaries ..."boundaries ..."
BorisBoris Beizer Beizer
OBJECTIVEOBJECTIVE
CRITERIACRITERIA
CONSTRAINTCONSTRAINT
to uncover errorsto uncover errors
in a complete mannerin a complete manner
with a minimum of effort and timewith a minimum of effort and time
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 5
Exhaustive TestingExhaustive Testing
loop < 20 Xloop < 20 X
There are 10 possible paths! If we execute oneThere are 10 possible paths! If we execute onetest per millisecond, it would take 3,170 years totest per millisecond, it would take 3,170 years totest this program!!test this program!!
1414
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 6
Selective TestingSelective Testing
loop < 20 Xloop < 20 X
Selected pathSelected path
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 7
Software TestingSoftware Testing
Methods
Strategies
white-boxmethods
black-box methods
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 8
White-Box TestingWhite-Box Testing
... our goal is to ensure that all ... our goal is to ensure that all statements and conditions have statements and conditions have been executed at least once ...been executed at least once ...
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 9
Why Cover?Why Cover?
logic errors and incorrect assumptions logic errors and incorrect assumptions are inversely proportional to a path's are inversely proportional to a path's execution probabilityexecution probability
we often we often believebelieve that a path is not that a path is not likely to be executed; in fact, reality is likely to be executed; in fact, reality is often counter intuitiveoften counter intuitive
typographical errors are random; it's typographical errors are random; it's likely that untested paths will contain likely that untested paths will contain some some
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 10
Basis Path TestingBasis Path TestingFirst, we compute the cyclomatic complexity:
number of simple decisions + 1
or
number of enclosed areas + 1
In this case, V(G) = 4
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 11
Cyclomatic ComplexityCyclomatic ComplexityA number of industry studies have indicated A number of industry studies have indicated that the higher V(G), the higher the probability that the higher V(G), the higher the probability or errors.or errors.
V(G)V(G)
modulesmodules
modules in this range are modules in this range are more error pronemore error prone
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 12
Basis Path TestingBasis Path TestingNext, we derive the Next, we derive the independent paths:independent paths:
Since V(G) = 4,Since V(G) = 4,there are four pathsthere are four paths
Path 1: 1,2,3,6,7,8Path 1: 1,2,3,6,7,8Path 2: 1,2,3,5,7,8Path 2: 1,2,3,5,7,8Path 3: 1,2,4,7,8Path 3: 1,2,4,7,8Path 4: 1,2,4,7,2,4,...7,8Path 4: 1,2,4,7,2,4,...7,8
Finally, we derive testFinally, we derive testcases to exercise these cases to exercise these paths.paths.
11
22
3344
55 66
77
88
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 13
Basis Path Testing NotesBasis Path Testing Notes
you don't need a flow chart, you don't need a flow chart, but the picture will help when but the picture will help when you trace program pathsyou trace program paths
count each simple logical test, count each simple logical test, compound tests count as 2 or compound tests count as 2 or moremore
basis path testing should be basis path testing should be applied to critical modulesapplied to critical modules
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 14
Graph MatricesGraph Matrices A graph matrix is a square matrix whose sizeA graph matrix is a square matrix whose size
(i.e., number of rows and columns) is equal to(i.e., number of rows and columns) is equal tothe number of nodes on a flow graphthe number of nodes on a flow graph
Each row and column corresponds to anEach row and column corresponds to anidentified node, and matrix entries correspondidentified node, and matrix entries correspondto connections (an edge) between nodes.to connections (an edge) between nodes.
By adding a By adding a link weightlink weight to each matrix entry, the to each matrix entry, thegraph matrix can become a powerful tool forgraph matrix can become a powerful tool forevaluating program control structure duringevaluating program control structure duringtestingtesting
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 15
Control Structure TestingControl Structure Testing
Condition testingCondition testing —— a test case design method that a test case design method thatexercises the logical conditions contained in a programexercises the logical conditions contained in a programmodulemodule
Data flow testingData flow testing —— selects test paths of a program selects test paths of a programaccording to the locations of definitions and uses ofaccording to the locations of definitions and uses ofvariables in the programvariables in the program
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 16
Loop TestingLoop Testing
Nested Nested LoopsLoops
ConcatenatedConcatenated Loops Loops Unstructured Unstructured
LoopsLoops
Simple Simple looploop
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 17
Loop Testing: Simple LoopsLoop Testing: Simple Loops
Minimum conditionsMinimum conditions——Simple LoopsSimple Loops1. skip the loop entirely1. skip the loop entirely2. only one pass through the loop2. only one pass through the loop3. two passes through the loop3. two passes through the loop4. m passes through the loop m < n4. m passes through the loop m < n5. (n-1), n, and (n+1) passes through 5. (n-1), n, and (n+1) passes through the loopthe loopwhere n is the maximum number where n is the maximum number of allowable passesof allowable passes
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 18
Loop Testing: Nested LoopsLoop Testing: Nested Loops
Start at the innermost loop. Set all outer loops to their Start at the innermost loop. Set all outer loops to their minimum iteration parameter values.minimum iteration parameter values.Test the min+1, typical, max-1 and max for the Test the min+1, typical, max-1 and max for the innermost loop, while holding the outer loops at their innermost loop, while holding the outer loops at their minimum values.minimum values.Move out one loop and set it up as in step 2, holding all Move out one loop and set it up as in step 2, holding all other loops at typical values. Continue this step until other loops at typical values. Continue this step until the outermost loop has been tested.the outermost loop has been tested.
If the loops are independent of one another If the loops are independent of one another then treat each as a simple loop then treat each as a simple loop else* treat as nested loops else* treat as nested loopsendifendif* * for example, the final loop counter value of loop 1 is for example, the final loop counter value of loop 1 is used to initialize loop 2.used to initialize loop 2.
Nested LoopsNested Loops
Concatenated LoopsConcatenated Loops
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 19
Black-Box TestingBlack-Box Testing
requirementsrequirements
eventseventsinputinput
outputoutput
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 20
Black-Box TestingBlack-Box Testing How is functional validity tested?How is functional validity tested? How is system behavior and performance tested?How is system behavior and performance tested? What classes of input will make good test cases?What classes of input will make good test cases? Is the system particularly sensitive to certain inputIs the system particularly sensitive to certain input
values?values? How are the boundaries of a data class isolated?How are the boundaries of a data class isolated? What data rates and data volume can the systemWhat data rates and data volume can the system
tolerate?tolerate? What effect will specific combinations of data haveWhat effect will specific combinations of data have
on system operation?on system operation?
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 21
Graph-Based MethodsGraph-Based Methods
newfile
menu select generates(generation time < 1.0 sec)
documentwindow
documenttext
is represented ascontains
Attributes:background color: whitetext color: default color
or preferences
(b)
object#1
Directed link(link weight)
object#2
object#3
Undirected link
Parallel links
Node weight(value)
(a)
allows editing of
To understand theTo understand theobjects that areobjects that aremodeled in softwaremodeled in softwareand the relationshipsand the relationshipsthat connect thesethat connect theseobjectsobjects
In this context, we considerIn this context, we considerthe term the term ““objectsobjects”” in the in thebroadest possible context. Itbroadest possible context. Itencompasses data objects,encompasses data objects,traditional componentstraditional components(modules), and object-(modules), and object-oriented elements oforiented elements ofcomputer software.computer software.
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 22
Equivalence PartitioningEquivalence Partitioning
useruserqueriesqueries mousemouse
pickspicks
outputoutputformatsformats
promptsprompts
FKFKinputinput
datadata
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 23
Sample Equivalence ClassesSample Equivalence Classes
user supplied commandsuser supplied commandsresponses to system promptsresponses to system promptsfile namesfile namescomputational datacomputational data physical parameters physical parameters bounding values bounding values initiation values initiation valuesoutput data formattingoutput data formattingresponses to error messagesresponses to error messagesgraphical data (e.g., mouse picks)graphical data (e.g., mouse picks)
data outside bounds of the program data outside bounds of the program physically impossible dataphysically impossible dataproper value supplied in wrong placeproper value supplied in wrong place
Valid dataValid data
Invalid dataInvalid data
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 24
Boundary Value AnalysisBoundary Value Analysis
useruserqueriesqueries mousemouse
pickspicks
outputoutputformatsformats
promptsprompts
FKFKinputinput
datadata
outputoutputdomaindomaininput domaininput domain
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 25
Comparison TestingComparison Testing
Used only in situations in which the reliability ofUsed only in situations in which the reliability ofsoftware is absolutely critical (e.g., human-ratedsoftware is absolutely critical (e.g., human-ratedsystems)systems) Separate software engineering teams develop independentSeparate software engineering teams develop independent
versions of an application using the same specificationversions of an application using the same specification Each version can be tested with the same test data to ensure Each version can be tested with the same test data to ensure
that all provide identical outputthat all provide identical output Then all versions are executed in parallel with real-timeThen all versions are executed in parallel with real-time
comparison of results to ensure consistencycomparison of results to ensure consistency
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 26
Orthogonal Array TestingOrthogonal Array Testing Used when the number of input parameters is small and
the values that each of the parameters may take areclearly bounded
One input item at a time L9 orthogonal array
XY
Z
XY
Z
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 27
Object-Oriented TestingObject-Oriented Testing
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 28
OOTOOT——Test Case DesignTest Case DesignBerard Berard [BER93] proposes the following approach:[BER93] proposes the following approach:
1.1. Each test case should be uniquely identified and should be explicitlyEach test case should be uniquely identified and should be explicitlyassociated with the class to be tested,associated with the class to be tested,2.2. The purpose of the test should be stated,The purpose of the test should be stated,3.3. A list of testing steps should be developed for each test and shouldA list of testing steps should be developed for each test and shouldcontain [BER94]:contain [BER94]:
a.a. a list of specified states for the object that is to be testeda list of specified states for the object that is to be testedb.b. a list of messages and operations that will be exercised as a list of messages and operations that will be exercised as
a consequence of the testa consequence of the testc.c. a list of exceptions that may occur as the object is testeda list of exceptions that may occur as the object is testedd.d. a list of external conditions (i.e., changes in the environment externala list of external conditions (i.e., changes in the environment external
to the software that must exist in order to properly conduct the test)to the software that must exist in order to properly conduct the test)e.e. supplementary information that will aid in understanding or supplementary information that will aid in understanding or
implementing the test.implementing the test.
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 29
Testing MethodsTesting Methods Fault-based testingFault-based testing
The tester looks for plausible faults (i.e., aspects of the implementation of the The tester looks for plausible faults (i.e., aspects of the implementation of thesystem that may result in defects). To determine whether these faults exist, testsystem that may result in defects). To determine whether these faults exist, testcases are designed to exercise the design or code.cases are designed to exercise the design or code.
Class Testing and the Class HierarchyClass Testing and the Class Hierarchy Inheritance does not obviate the need for thorough testing of all derived classes.Inheritance does not obviate the need for thorough testing of all derived classes.
In fact, it can actually complicate the testing process.In fact, it can actually complicate the testing process. Scenario-Based Test DesignScenario-Based Test Design
Scenario-based testing concentrates on what the user does, not what the productScenario-based testing concentrates on what the user does, not what the productdoes. This means capturing the tasks (via use-cases) that the user has to perform,does. This means capturing the tasks (via use-cases) that the user has to perform,then applying them and their variants as tests.then applying them and their variants as tests.
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 30
OOT Methods: Random TestingOOT Methods: Random Testing
Random testingRandom testing identify operations applicable to a classidentify operations applicable to a class define constraints on their usedefine constraints on their use identify aidentify a minimum test sequenceminimum test sequence
an operation sequence that defines the minimum lifean operation sequence that defines the minimum lifehistory of the class (object)history of the class (object)
generate a variety of random (but valid) test sequencesgenerate a variety of random (but valid) test sequences exercise other (more complex) class instance life historiesexercise other (more complex) class instance life histories
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 31
OOT Methods: Partition TestingOOT Methods: Partition Testing Partition TestingPartition Testing
reduces the number of test cases required to test a class inreduces the number of test cases required to test a class inmuch the same way as equivalence partitioning formuch the same way as equivalence partitioning forconventional softwareconventional software
state-based partitioningstate-based partitioning categorize and test operations based on their ability to changecategorize and test operations based on their ability to change
the state of a classthe state of a class attribute-based partitioningattribute-based partitioning
categorize and test operations based on the attributes that theycategorize and test operations based on the attributes that theyuseuse
category-based partitioningcategory-based partitioning categorize and test operations based on the generic functioncategorize and test operations based on the generic function
each performseach performs
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 32
OOT Methods: Inter-Class TestingOOT Methods: Inter-Class Testing Inter-class testingInter-class testing
For each client class, use the list of class operators toFor each client class, use the list of class operators togenerate a series of random test sequences. The operatorsgenerate a series of random test sequences. The operatorswill send messages to other server classes.will send messages to other server classes.
For each message that is generated, determine theFor each message that is generated, determine thecollaborator class and the corresponding operator in thecollaborator class and the corresponding operator in theserver object.server object.
For each operator in the server object (that has been invokedFor each operator in the server object (that has been invokedby messages sent from the client object), determine theby messages sent from the client object), determine themessages that it transmits.messages that it transmits.
For each of the messages, determine the next level ofFor each of the messages, determine the next level ofoperators that are invoked and incorporate these into the testoperators that are invoked and incorporate these into the testsequencesequence
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 33
OOT Methods: Behavior TestingOOT Methods: Behavior Testingempty
acctopen setup Accnt
set up
acct
deposit(initial)
working
acct
withdrawal(final)
deadacct close
nonworkingacct
deposit
withdrawbalance
credit
accntInfo
Figure 14.3 State diagram for Account class (adapted from [KIR94])
The tests to beThe tests to bedesigned shoulddesigned shouldachieve all stateachieve all statecoveragecoverage[KIR94]. That is,[KIR94]. That is,the operationthe operationsequencessequencesshould cause theshould cause theAccount class toAccount class tomake transitionmake transitionthrough allthrough allallowable statesallowable states
These coursew are materials are to be used in conjunction w ith Soft ware Engineering: A Pract it ioner’s Approach,6/e and are provided w ith permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 34
Testing PatternsTesting PatternsPattern name:Pattern name: pair testing pair testing
Abstract: Abstract: A process-oriented pattern, pair testing describes a technique that A process-oriented pattern, pair testing describes a technique thatis analogous to pair programming (Chapter 4) in which two testers workis analogous to pair programming (Chapter 4) in which two testers worktogether to design and execute a series of tests that can be applied to unit,together to design and execute a series of tests that can be applied to unit,integration or validation testing activities.integration or validation testing activities.
Pattern name: Pattern name: separate test interface separate test interfaceAbstract: Abstract: There is a need to test every class in an object-oriented system, There is a need to test every class in an object-oriented system,including including ““internal classesinternal classes”” (i.e., classes that do not expose any interface (i.e., classes that do not expose any interfaceoutside of the component that used them). The separate test interfaceoutside of the component that used them). The separate test interfacepattern describes how to create pattern describes how to create ““a test interface that can be used toa test interface that can be used todescribe specific tests on classes that are visible only internally to adescribe specific tests on classes that are visible only internally to acomponent.component.”” [LAN01] [LAN01]
Pattern name: Pattern name: scenario testing scenario testingAbstract: Abstract: Once unit and integration tests have been conducted, there is aOnce unit and integration tests have been conducted, there is aneed to determine whether the software will perform in a manner thatneed to determine whether the software will perform in a manner thatsatisfies users. The scenario testing pattern describes a technique forsatisfies users. The scenario testing pattern describes a technique forexercising the software from the userexercising the software from the user’’s point of view. A failure at this levels point of view. A failure at this levelindicates that the software has failed to meet a user visible requirement.indicates that the software has failed to meet a user visible requirement.[KAN01][KAN01]
Åben Uddannelse
Software Engineering, Efteråret 2002 1
Test i XP
Automatisk test
Test i XP 2
Software testing
“ Testing is the process ofexecuting a program with the
intent of finding errors.”
Glen Myers
Åben Uddannelse
Software Engineering, Efteråret 2002 2
Test i XP 3
Testing You couldn’t possibly write all those tests? It
would take too much time. ”Programmerswon’t write tests!”
Unless The design is as simple as it can be You are programming with a partner, so if you
couldn’t think of another test your partner can You feel good when you see the tests all running Your customer feels good about the system when
they see all of their tests running
Test i XP 4
Test-First Development Tests are specified before software is written
Users specify inputs and outputs as part of test “Outside-in” perspective improves the software design Ensures software delivers expected results Writing tests is part of s/w design process
Detailed test knowledge captured in tests Functional Tests specify requirements in detail Unit Tests developed as soon as component/class identified;
defines software interface Tests drive the development process
Software is only written because tests are failing. Ensures all software is tested Ensures all requirements are captured as tests
Åben Uddannelse
Software Engineering, Efteråret 2002 3
Test i XP 5
Timelines Acceptance need to be available in the same
iteration as the story is scheduled The only way to know that the stories are done is
that the acceptance tests are passed Programmers have the right to know what is
needed Therefore insist on automated functional tests
Customers have the right to see progress in arunning system
Proven to to work by automated tests that you specify
Test i XP 6
Functional Tests Specified by the user Implemented by users, developers,
and/or test team Automated Run at least daily Part of the specification
Åben Uddannelse
Software Engineering, Efteråret 2002 4
Test i XP 7
Test Metrics
Tests failed
Tests passed
1 2 3 4 5 6 7
Key:
Test i XP 8
Unit Tests Testing bit by bit
Tests made by the programmers forthe programmers
Åben Uddannelse
Software Engineering, Efteråret 2002 5
Test i XP 9
Unit Tests
A unit test is written BEFORE THE CODE ISWRITTEN for each piece of the system.
When you create your tests first, before the code, you will find it mucheasier and faster to create your code.
The combined time it takes to create a unit test and create some codeto make it pass is about the same as just coding it up straight away.
But, if you already have the unit tests you don't need to create themafter the code saving you some time now and lots later.
Eases regression testing JUnit (for Java), SUnit (for Squeak) allow for running all unit tests.
When a bug is found, a new unit test is written to catch for that bug in the future
Test i XP 10
Unit Tests Written by developers Always run at 100% Support design, coding, refactoring,
and quality.
Åben Uddannelse
Software Engineering, Efteråret 2002 6
Test i XP 11
Testing steps Create a test class
Name it like ClassNameTestCase Code a setup Write a test Run the test Modify the model code and repeat steps four
and five until the test runs successfully Release the code and the test case
12
Automated Testing Continuous Automated Testing throughout system lifecycle
during development & integration before delivery, after installation Tests are rerun many times every day
Low cost regression testing since most tests are automated Higher initial cost recouped before initial delivery; free
thereafter “one button” testing, anytime, anywhere
Improves Developer Morale Developers enjoy testing, because it is coding Removes developers’ fear of breaking something Reduces maintenance cost significantly
Åben Uddannelse
Software Engineering, Efteråret 2002 7
Test i XP 13
What is JUnit? A small testing framework written in Java. A series of extensible classes that do much
of the testing grunt work for us. (i.e. countingand reporting errors and failed tests, runningtests in batch, etc.)
Very handy for Extreme Testing… Developed by
Kent Beck, [email protected] Erich Gamma, [email protected]
http://www.junit.org/
Test i XP 14
Essential Classes junit.framework.TestCase
Allows running multiple tests Does all counting and reporting of errors
junit.framework.Assert A set of assert methods Test fails if the assert condition does not hold If a test fails it is counted and reported
junit.framework.TestSuite A collection of tests Uses Java introspection to find all the methods start with "test“ and
have void parameters. TestSuite's run method executes all the tests
Åben Uddannelse
Software Engineering, Efteråret 2002 8
Test i XP 15
Test Runners
Classes that run tests for a test case Can be Text or UI based Text Based:
Accepts the name of a TestCase class as an argument If this class defines a static suite method it will be invoked
and the returned test is run. Otherwise all the methodsstarting with "test" having no arguments are run.
AWT UI Based: A handy tool for running test cases Runs test cases and reports failures and errors
Test i XP 16
Class diagram
Åben Uddannelse
Software Engineering, Efteråret 2002 9
Test i XP 17
JUnit How-To (1) Define a test case (of course) Create a Class which extends TestClass Define the fixture for the test class, these are all
variables required to carry out the tests Override the setUp and tearDown methods of
TestCase to initialize members or free upresources, respectively
Test i XP 18
JUnit How-To (2) Define the test methods required for the test case.
Additional methods may be added at any time Define a suite method which creates a TestSuite
from the methods of the test class Define a main method which uses some test runner
to run the tests in the TestSuite Simply run the test case class
Åben Uddannelse
Software Engineering, Efteråret 2002 10
Test i XP 19
JUnit How-To (3) Multiple test cases my be run by creating a class
which contains a suite of TestCase subclasses These suites can be run in batch or individually by
simply running the whole class containing the suiteor by calling run with the name of the test case youwant to run. Each test case constructor can accepta string which is the name of the test case
Test i XP 20
Junit User Interface
Åben Uddannelse
Software Engineering, Efteråret 2002 11
Test i XP 21
A Quick Example (1) Class Person Person contains a first and a last name Also contains the method hashCode methods for computing hashcodes
for first and last
public class Person {public Person() { first = ""; last = ""; }...public int hashCode() { int result = 0;
result += first.hashCode();result += last.hashCode();return result; }
...}
Test i XP 22
A Quick Example (2) Class PersonTest extends the
Junit class TestCase The fields first and last make up
the fixture PersonTest contains a test
which test if the hashCodemethod returns the proper values
The method suite is inheritedfrom TestCase
Åben Uddannelse
Software Engineering, Efteråret 2002 12
Test i XP 23
A Quick Example (3)
The PersonTest Constructor
public PersonTest(String name) {
super(name);
}
Accepts the name of the test case and passes it to the superclass
Test i XP 24
A Quick Example (4)
SetUp of one PersonTest
private static final Person personThree =
new Person("First3", "Last3");
Åben Uddannelse
Software Engineering, Efteråret 2002 13
Test i XP 25
A Quick Example (5)The PersonTest test method testHashCode
public void testHashCode() {int code = personThree.hashCode();
Person test = new Person("First3","Last3");assertEquals(code,test.hashCode());
test = new Person( "First1", "Last1");assertTrue((code == test.hashCode()) == false);……}
Test i XP 26
A Quick Example (6)The PersonTest suite method
public static Test suite ( ) {
return new TestSuite (PersonTest.class));
}
Creates a new TestSuite containing the tests definedin the PersonTest class
Åben Uddannelse
Software Engineering, Efteråret 2002 14
Test i XP 27
A Quick Example (7)Run multiple test suites (class AllTests) Define new suite - TestSuite Add suites to TestSuite
public static Test suite ( ) {
TestSuite suite= new TestSuite("All JUnit Tests");
suite.addTest(new TestSuite(PersonTest.class));suite.addTest(new TestSuite(LinkedListTest.class));
return suite;
}
Test i XP 28
A Quick Example (8)The public class AllTests
public static void main (String[] args) {
/** junit.textui.TestRunner.run (suite());*/
junit.swingui.TestRunner.run (AllTests.class); }
Creates a new TestSuite that runs all the tests definedin the PersonTest and LinkedListTest classes