Lecture 12 testing.ppt
-
Upload
softwarecentral -
Category
Documents
-
view
7.823 -
download
3
Transcript of Lecture 12 testing.ppt
University of Sunderland
CSE100: Introductory Software Engineering
Lecture 12: Testing
Requirements
Analysis
Design
Development
Testing
Evaluation
Software Development Processes/Lifecycle
Sp
ecification
Development
Val
idat
ion
Evolution
Also, for SDLC see http://uk.youtube.com/watch?v=OfgfnZZdMlI
We are here
We are also here
Requirements
Analysis
Design
Development
Testing
Evaluation
Overlapping/In Parallel?
Testing, Debugging, Verification & Validation:
Explanations & clarifications!
Testing is not debugging!
• Testing is– Seeing if a given set of inputs causes an acceptable or unacceptable behaviour in a program
• Debugging is– Seeing why a given set of inputs causes an unacceptable behaviour and what must be changed to cause the behaviour to be acceptable
Verification
Testing is one (widely used) part of verification
Verification also includes:• Inspections/reviews (formal)
– Small teams inspecting the documents and work products in stages throughout the development lifecycle
• Walkthroughs (informal)– As above but less formal/no documentation
What is verified?– Requirements specification– functional design– component design– Code
You saw this in the SimSE game!
Requirements
Analysis
Design
Development
Testing
Evaluation
Software Development Processes/Lifecycle
Sp
ecification
Development
Val
idat
ion
Evolution
But the interplay of verification & validation willdepend on your chosen development methodology
Example verification/inspection stages
• E.g. in the requirements specification– Is there ambiguity? Is it complete?
• E.g. in the component/module– Are there errors in the code logic? (error of commission)
– Are some parts of the spec ignored in the code? (error of omission)
• E.g. boundary value errors– (see later examples)
‘V’ Model
Requirements
Design
Code
Designmatches
Requirements
Structural
Functional
Usability
Codematches Design
Verification – build it the right way
Verification – build it the right way (fault-free)
(fault-free)
Valid
atio
n -
build
the
right
thin
g
Valid
atio
n -
build
the
right
thin
g
Or, in more detail….
Question
• When I ‘tested’ the bridges you built in the 2nd week, was I doing verification or validation?
• What would be appropriate methods of verifying and validating your bridge builds?
Discuss with neighbour
Software Validation
• To ensure that the software:– Conforms to the specification– Meets the needs of the user– Performs effectively and efficiently
– Performs safely
Validation
• Done after verification stages, towards or at end of coding– depending on methodology e.g. waterfall, e.g. agile or XP or test-driven, this might be at end of development or throughout
In easy to understand terms, validation is, kind of, where ‘testing’ meets ‘evaluation’
Purpose of Testing?
• 2 different purposes:1. Test the Structure of the software2. Test the Function of the software
1. Structural (White Box or Glass Box testing)– Looks at the code and its structure– Test cases examine decision points in the code– Includes unit testing, integration testing
2. Functional testing (Black Box testing)– Doesn’t look at the code– Examines whether a program accepts the correct
inputs and produces correct outputs– Includes UAT, Alpha & Beta Testing
Structural Testing
Types of structural testing
1. Unit testing
2. Integration (system) testing – Do the units coordinate properly? Are there
any problems with component interfaces?
Remember, problem solving = breaking into chunks and working on the chunks, what passes between them, and how they combine into one large chunk (unit and integration)
Structural 1) Unit Testing
• Verifies that a module works correctly in isolation– Driver component makes method calls on the tested component
– Any methods used are simulated as stubs – e.g. prints ‘hello’ to tell you it’s been successfully tested
Unit testing
Driver Component
Stub
Stub
Stub
Drivers and Stubs Example
Book Hand Over
Read BookBar Code
Update Book Details
Return Book
Module Under TestStub (supplies BarCode Value)
Driver: Calls Stub then module under test, finally outputs Date of Return
BarCode
BarCode
DateofReturn
Structural 2) Integration Testing
• Big Bang– Bring everything together then test– Bad idea
• Improved Big Bang– Bring everything together having done unit testing first
– Better idea than Big Bang• Incremental
– Better approach– Faults are localised
Structural Testing: General Description
• Test all paths correctly and fully– This can be difficult if code is complex
• Follow a process to create test data– What is ‘test data’?
• Can be done by the code writer or programmers may swap modules, – or in teams e.g. Quality Assurance teams, adversaries, bounty hunters
Process for Structural Testing
1. Draw up a Test Run Chart (rectangles & circles) to identify all paths through the module
2. Identify the type of test for a particular path– The type of path determines the number
of tests needed for each path
3. Draw up a Test Case Table showing • all tests and • expected results• actual results
3 stages
Step 1) Test Run Chart
• Shows logical flow of a program/module• Identifies all paths to be tested
The rectangle indicates normal processing
The circle indicates a location in the module where the processingpath splits.
Path 1 (No) Path 2 (Yes)
SwitchOff?
Calculatetemperature
Testing Each Path
There are five There are five paths in the main, left loop: in the main, left loop:
1 2 3 4 5
Conditions: • If heater not on (1,2,3,4) or is on (5);• If office hours (1,2) or not (3,4)
• If temperature < 65F (2) or not (1)• If temperature < 45F (4) or not (3)
DecisionAction
How exhaustively do you test?
• Exhaustive – means what?• It means: every possible scenario
• What are the implications of this?– Let’s look at an example…
Exhaustive Testing
0 loop < 20
How many possible paths are there through this program?
Discuss this with your neighbour
Exhaustive Testing
0 loop < 20
How many possible paths are there through this program?
2,432,902,008,176,640,000
Exhaustive vs Effective Testing
• Exhaustive testing = unrealistic• In the example just given:
– loop executed 20 times• but also every possible variation in the loop • And every possible sequence of variations
When combined gives millions of tests!
• So, have to run a set of tests that is Effective not Exhaustive– E.g. 99.99% correct (exhaustive) and 99.5% correct (effective)
Step 2) Identify Type of Test
Conditional Tests: Individual Values
• Make sure TRUE and FALSE sides of each condition are tested
• Testing individual values:for every valid value test for an invalid value
E.g. test for gender (‘m’ or ‘f’)– Would test for (‘m’ , ‘f’, + 2 invalids) = 4 tests
• E.g. expect ‘m’ and ‘m’ is entered OR expect ‘m’ and ‘ ’ is entered
– Assumes another letter has not been coded as valid, would not be easy to find other than:• Every character in the character set is tested (exhaustive testing)
Conditional Tests: Value Ranges
• Requires Boundary Testing:• Every bound should be tested with 2 tests:
– E.g. If the range of values is from 0 to 10:•Test 0 (valid) and -1 (invalid)•Test 10 (valid) and 11 (invalid)•Test a value randomly between 1 and 9
– 5 tests in this example– Exhaustive would be -99 to +99 (inclusive) i.e. every possible 2-digit integer
Loop Testing
• Focus exclusively on the validity of loops
• The following tests are applied to simple loops where n is the maximum number of allowable passes through the loop, try to: 1. skip the loop entirely
2. only pass once through the loop
3. m passes through the loop where m < n
4. attempt n - 1, n, n + 1 passes through the loop
Identifying the Tests
• Good idea to number the tests
• Example test for value Range– 0 to 10 gives 5 tests (boundary)
• If this test is test path 2 in the module then number tests as follows:
Test Id.
Value Pass/
Fail
2/1 -1 F
2/2 0 P
2/3 4 P
2/4 10 P
2/5 11 F
Test Data for Multiple Tests
• The same data can be used for multiple tests.
• Take an example of a record with 2 values: A & B, both are in the range 0-5 to be valid, B must be less than A.
• One test data record can carry out 3 tests for 3 different paths
• Tests 3/1, 4/1 and 5/1. Use the same data
Test id
Values Pass Fail
3/1 Test A=3
Pass
4/1 Test B=5
Pass
5/1 B < A Fail
Value A Value B
3 5
Test Cases
A set of test inputs, execution conditions, & expected results for a particular objective:1. Guarantee all independent paths in a
program have been exercised at least once2. Exercise True & False for all logical
decisions 3. Execute all loops and conditions at their
boundaries and within their operational bounds
4. Exercise internal data structures (e.g. arrays, record structures) to ensure their validity
Test Cases
• Derived during all phases of the development cycle
• Determine what are your expected results before you run a test-case– Then, running a test case provides the actual results
• Never throw away test-cases and results
• Test cases are presented as a table.
Step 3) Test Case Table
Description Input ExpectedResults
ActualResults
Pass/Fail
Test id.(path/no.)
Functional Testing
Types of Functional Testing
UAT, Alpha Testing, Beta Testing1. User-acceptance testing (UAT) – before final
delivery:– Product is handed over to user to test it in a real
time scenario– See if it works according to the system specifications
and satisfies all the user requirements. – As the user uses the software, some unknown bugs might
be found – these are communicated to the developers to be fixed, and this helps improve the final product.
When testing is done by paid testers, or the developers – it’s done in 2 stages:
2. Alpha-testing – in development environment3. Beta-testing – in end-users’ environment (by
developer and/or end user)
User Acceptance Testing
• Define acceptance criteria with client during phases of SDLC
• Well defined acceptance plan helps development by identifying users’ needs during software development
• The UAT must be created or reviewed by client & Software Engineer together
• Identifies:– interim and final products for acceptance, – acceptance criteria (and who will do the acceptance
activities) – Schedule (including time for users to do the activities)
• When are you ready to do UAT? – System Testing is completed and defects identified are either
fixed or documented– Acceptance plan is prepared and resources have been identified– Test environment for the acceptance testing is available
• When is it finished?– Acceptance decision is made for the software – In case of any caveats, development team is notified
Beta-Testing
• Undertaken after UAT• Timed ahead of the official release date
• System released to a small (select) number of customers
• Customers use the system as though already released and are ‘encouraged’ to find bugs
Functional Testing: General Description
• Tests the specification (or data created from the spec.) not the code– ‘what system does’ not ‘how it does it’
• i.e. tests data produced by the designer not the programmer
Functional Testing
• Black box attempts to derive sets of inputs that will fully exercise all the functional requirements of a system.
• Aims to find errors in the following categories: • interface errors• errors in data structures or internal
database access• performance errors• initialization and termination errors
An Example Black Box Test
BookHanded Over
UpdateBook Details
ReadBook Record
Bar Code
Book Code
Date of Return
MembershipId
After testing these 3 modules individually, all 3 modules are combined in a test. For the 3-module together test, a single argument the – ‘MembershipId’ producing a single reply ‘Date of Return’. Book Barcode isRead as part of the processing.
Book Code
Date ofReturn
• Earlier we said functional testing uses a ‘black box’ approach i.e. looks at input and output but ignores ‘inside workings’
• So – how to decide on your test input?
• We can use Equivalence Partitioning to guide black-box testing
Deciding on functionaltest input data
Equivalence Partitioning
Input domain
12
3
4
Input domain partitioned into four sub-domains.
Too manytest inputs.
Four test inputs, one selected from each sub-domainor equivalence class.
How to partition
• Inputs to a program provide clues to
partitioning:– If a program takes an input X, where X
is an integer
– Where X<0 the program must perform one
task and another task where X>=0
– The input domain is way too big because
X can be a huge number of values
…how to partition
In partition testing our goal is to cover all equivalence classes
– We expect the program to behave the same way for every input where X < 0
– We expect the program to perform the same way for all values of X >= 0
– An equivalence class is considered covered when at least one test has been selected from it.
Two sub-domains
One test case:X = -3
Another test case:X = 15
X<0 X>=0Equivalence class
Equivalence class
Non-overlapping (disjoint) partitions
• In the previous example, the two equivalence classes are non-overlapping. In other words the two sub-domains are disjoint
• In this case we can pick just one test input from each equivalence class to test the program
Overlapping partitions
• Suppose a program takes three integers X, Y and Z, and it is known that:
•X<Y •Z>Y
Overlapping partitions
X<Y
X>=Y
Z>Y Z<=Y
X<Y, Z>YX=3, Y=4, Z=7
X<Y, Z<=YX=2, Y=3, Z=1
X>=Y, Z>YX=15, Y=4, Z=7
X>=Y, Z<=YX=15, Y=4, Z=1
Guidelines for defining equivalence classes for black
box testing: If an input condition specifies a range (e.g. 1-
12), one valid and two invalid equivalence classes are defined (e.g. 5, 0 and 13).
If an input condition requires a specific value (e.g. ‘m’) then one valid and two invalid equivalence classes are defined (e.g. ‘m’, ‘f’, and ‘_’).
If an input condition specifies a member of a set (e.g. ‘change pin’, ‘check balance’, ‘withdraw cash’), then one valid and one invalid equivalence classes are defined.
If an input condition is Boolean (may or may not be present), then one valid and one invalid class is defined.
Example equivalence class (set of valid or invalid states
for input conditions)
Example: • a function which accept input parameter "month" of a
date. • valid range for month is 1(January)to 12(December).
This valid range is called a partition. • In this example there are two further partitions of
invalid ranges. – The first invalid partition would be <= 0 – the second invalid partition would be >= 13. – You can also have invalid partition for non-digit input as well
.... -2 -1 0 1 .............. 12 13 14 15 ..... --------------|-------------------|--------------------- invalid partition 1 valid partition invalid partition 2
Final comments on testing
Testing good practice:1. Begins at component level & works out
towards integration of whole system2. Different testing techniques are
appropriate at different times3. Testing is done by the developers and (in
large projects) an independent test group4. Testing & debugging aren’t the same – but
any test strategy has debugging in it
• The last part of the course will look at:– user evaluation strategies and at – evolution of software projects
Before tomorrow, you must:
• Look over the course notes so far• Tomorrow you will get the assignment
– There’ll be time to do it in class in the next 2 weeks, with tutor support
– There’ll be an interim marking next Tuesday
• The assignment tests ALL the module learning outcomes
• This means we’ll be asking you to give a full account of what you’ve learned– All aspects over the full SDLC