Introduction To Software Testing - Aalborg...
Transcript of Introduction To Software Testing - Aalborg...
![Page 1: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/1.jpg)
Introduction ToSoftware Testing
Brian Nielsen [email protected] of Embedded Software Systems
Aalborg University, Denmark
C SS10101110110101011011010101110111
![Page 2: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/2.jpg)
CISS
Software development cycle ☺1. Programmer produces code he believes is bug-free.2. Product is tested. 20 bugs are found. 3. Programmer fixes 10 of the bugs and explains to the testing department that the other
10 aren't really bugs.4. Testing department finds that five of the fixes didn't work and discovers 15 new bugs.5. Repeat three times steps 3 and 4. 6. Due to marketing pressure and an extremely premature product announcement based
on overly-optimistic programming schedule, the product is released.7. Users find 137 new bugs.8. Original programmer, having cashed his royalty check, is nowhere to be found.9. Newly-assembled programming team fixes almost all of the 137 bugs, but introduce
456 new ones.10. Original programmer sends underpaid testing department a postcard from Fiji. Entire
testing department quits.11. Company is bought in a hostile takeover by competitor using profits from their latest
release, which had 783 bugs. 12. New CEO is brought in by board of directors. He hires a programmer to redo program
from scratch.13. Programmer produces code he believes is bug-free.
![Page 3: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/3.jpg)
CISS
Autonomous Systems
![Page 4: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/4.jpg)
CISS
Extreme environmental conditions
![Page 5: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/5.jpg)
CISS
Safety Critical Systems
Rotterdam Storm Surge Barrier
![Page 6: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/6.jpg)
CISS
Sensor-networks
![Page 7: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/7.jpg)
CISS
Consumer Electronics
![Page 8: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/8.jpg)
CISS
System Characteristics• Mass production• Limited Hardware (speed, power,memory)• Safety critical• Autonomous• Distributed• One-shot development• Control of physical environment• Mixed signal I/O• Extreme environments• State-based behavior• Real-time constraints (hard and soft)• Technical / scientific algorithms• …
![Page 9: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/9.jpg)
CISS
Spectacular Software Bugs
• ARIANE-5 • http://www.cs.auc.dk/~ask/Undervisning/MVP/ariane5rep.html
• INTEL Pentium II floating-point division 470 Mill US $
• Baggage handling system, Denver 1.1 Mill US $/day for 9 months
• Mars Pathfinder • http://catless.ncl.ac.uk/Risks/19.49.html#subj1
• Radiation theraphy, Therac-25• http://www.cs.auc.dk/~ask/Undervisning/MVP/therac25.ps
• …….
![Page 10: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/10.jpg)
CISS
Ariane 5 crash• An overrun exception stopped both replicated
position calculation computers (SRI’s) at time H0+36.7. Fail-stop was a wrong design decision
• Overrun caused by higher velocity in Ariane 5 compared to Ariane 4
• Overrun occured in a module which was irrelevant after take-off
• Simulations were not run with the new velocity (would have caught the failure)
![Page 11: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/11.jpg)
CISS
Software Qualities
MaintainerGood Documentation
Readable CodeGood Design
Customer
Low CostPortability Increased
productivity
ReliabilityCorrectnessEfficiency
User
FunctionalityEase of useEase of learning
![Page 12: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/12.jpg)
CISS
Costs of Poor Quality• Increased time to find and fix problems• Increased cost to distribute
modifications• Increased customer support• Product liability• Failure in the market place
![Page 13: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/13.jpg)
CISS
Software Quality Assurance• Quality Control
• Process definition and standards • Formal technical review• Test planning and review
• Quality Assurance• Analysis and reporting• Measurement
![Page 14: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/14.jpg)
CISS
Verification vs. Validation:
• Verification: • “are we building the product right?”• The software should conform to its specification
• Validation:• “are we building the right product?”• The software should do what the user really
requires
![Page 15: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/15.jpg)
CISS
Verification vs. Validation:
• Verification: • “are we building the product right?”• The software should conform to its specification
• Validation:• “are we building the right product?”• The software should do what the user really
requires
Assuring that a software system meets a user’s needs
![Page 16: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/16.jpg)
CISS
What is testing?
![Page 17: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/17.jpg)
CISS
A Self-Assessment Test [Myers]
![Page 18: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/18.jpg)
CISS
A Self-Assessment Test [Myers]
• “A program reads three integer values. The three values are interpreted as representing the lengths of the sides of a triangle. The program prints a message that states whether the triangle is scalene, isosceles, or equilateral.”
![Page 19: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/19.jpg)
CISS
A Self-Assessment Test [Myers]
• “A program reads three integer values. The three values are interpreted as representing the lengths of the sides of a triangle. The program prints a message that states whether the triangle is scalene, isosceles, or equilateral.”
• Write a set of test cases to test this program
![Page 20: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/20.jpg)
CISS
Test cases for:
A Self-Assessment Test [Myers]
![Page 21: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/21.jpg)
CISS
Test cases for:
A Self-Assessment Test [Myers]
1. valid scalene triangle ?
![Page 22: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/22.jpg)
CISS
Test cases for:
A Self-Assessment Test [Myers]
1. valid scalene triangle ?2. valid equilateral triangle ?
![Page 23: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/23.jpg)
CISS
Test cases for:
A Self-Assessment Test [Myers]
1. valid scalene triangle ?2. valid equilateral triangle ?3. valid isosceles triangle ?
![Page 24: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/24.jpg)
CISS
Test cases for:
A Self-Assessment Test [Myers]
1. valid scalene triangle ?2. valid equilateral triangle ?3. valid isosceles triangle ?4. 3 permutations of previous ?
![Page 25: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/25.jpg)
CISS
Test cases for:
A Self-Assessment Test [Myers]
1. valid scalene triangle ?2. valid equilateral triangle ?3. valid isosceles triangle ?4. 3 permutations of previous ?5. side = 0 ?
![Page 26: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/26.jpg)
CISS
Test cases for:
A Self-Assessment Test [Myers]
1. valid scalene triangle ?2. valid equilateral triangle ?3. valid isosceles triangle ?4. 3 permutations of previous ?5. side = 0 ?6. negative side ?
![Page 27: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/27.jpg)
CISS
Test cases for:
A Self-Assessment Test [Myers]
1. valid scalene triangle ?2. valid equilateral triangle ?3. valid isosceles triangle ?4. 3 permutations of previous ?5. side = 0 ?6. negative side ?7. one side is sum of others ?
![Page 28: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/28.jpg)
CISS
Test cases for:
A Self-Assessment Test [Myers]
1. valid scalene triangle ?2. valid equilateral triangle ?3. valid isosceles triangle ?4. 3 permutations of previous ?5. side = 0 ?6. negative side ?7. one side is sum of others ?8. 3 permutations of previous ?
![Page 29: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/29.jpg)
CISS
Test cases for:
A Self-Assessment Test [Myers]
1. valid scalene triangle ?2. valid equilateral triangle ?3. valid isosceles triangle ?4. 3 permutations of previous ?5. side = 0 ?6. negative side ?7. one side is sum of others ?8. 3 permutations of previous ?
9. one side larger than sum ofothers ?
![Page 30: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/30.jpg)
CISS
Test cases for:
A Self-Assessment Test [Myers]
1. valid scalene triangle ?2. valid equilateral triangle ?3. valid isosceles triangle ?4. 3 permutations of previous ?5. side = 0 ?6. negative side ?7. one side is sum of others ?8. 3 permutations of previous ?
9. one side larger than sum ofothers ?
10. 3 permutations of previous ?
![Page 31: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/31.jpg)
CISS
Test cases for:
A Self-Assessment Test [Myers]
1. valid scalene triangle ?2. valid equilateral triangle ?3. valid isosceles triangle ?4. 3 permutations of previous ?5. side = 0 ?6. negative side ?7. one side is sum of others ?8. 3 permutations of previous ?
9. one side larger than sum ofothers ?
10. 3 permutations of previous ?11. all sides = 0 ?
![Page 32: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/32.jpg)
CISS
Test cases for:
A Self-Assessment Test [Myers]
1. valid scalene triangle ?2. valid equilateral triangle ?3. valid isosceles triangle ?4. 3 permutations of previous ?5. side = 0 ?6. negative side ?7. one side is sum of others ?8. 3 permutations of previous ?
9. one side larger than sum ofothers ?
10. 3 permutations of previous ?11. all sides = 0 ?12. non-integer input ?
![Page 33: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/33.jpg)
CISS
Test cases for:
A Self-Assessment Test [Myers]
1. valid scalene triangle ?2. valid equilateral triangle ?3. valid isosceles triangle ?4. 3 permutations of previous ?5. side = 0 ?6. negative side ?7. one side is sum of others ?8. 3 permutations of previous ?
9. one side larger than sum ofothers ?
10. 3 permutations of previous ?11. all sides = 0 ?12. non-integer input ?13. wrong number of values ?
![Page 34: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/34.jpg)
CISS
Test cases for:
A Self-Assessment Test [Myers]
1. valid scalene triangle ?2. valid equilateral triangle ?3. valid isosceles triangle ?4. 3 permutations of previous ?5. side = 0 ?6. negative side ?7. one side is sum of others ?8. 3 permutations of previous ?
9. one side larger than sum ofothers ?
10. 3 permutations of previous ?11. all sides = 0 ?12. non-integer input ?13. wrong number of values ?14. for each test case: is
expected output specified ?
![Page 35: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/35.jpg)
CISS
Test cases for:
A Self-Assessment Test [Myers]
1. valid scalene triangle ?2. valid equilateral triangle ?3. valid isosceles triangle ?4. 3 permutations of previous ?5. side = 0 ?6. negative side ?7. one side is sum of others ?8. 3 permutations of previous ?
9. one side larger than sum ofothers ?
10. 3 permutations of previous ?11. all sides = 0 ?12. non-integer input ?13. wrong number of values ?14. for each test case: is
expected output specified ?15. check behaviour after
output was produced ?
![Page 36: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/36.jpg)
CISS
TestingTesting:• to check the quality (functionality, reliability,
performance, …) of an (software) object
-by performing experiments-in a controlled way
![Page 37: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/37.jpg)
CISS
TestingTesting:• to check the quality (functionality, reliability,
performance, …) of an (software) object
-by performing experiments-in a controlled way
![Page 38: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/38.jpg)
CISS
TestingTesting:• to check the quality (functionality, reliability,
performance, …) of an (software) object
-by performing experiments-in a controlled way
• In avg. 10-20 errors per 1000 LOC• 30-50 % of development time and cost in embedded software
![Page 39: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/39.jpg)
CISS
TestingTesting:• to check the quality (functionality, reliability,
performance, …) of an (software) object
-by performing experiments-in a controlled way
• In avg. 10-20 errors per 1000 LOC• 30-50 % of development time and cost in embedded software
![Page 40: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/40.jpg)
What is a Test?
Software under Test
![Page 41: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/41.jpg)
What is a Test?
Software under Test
Test Data
![Page 42: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/42.jpg)
What is a Test?
Software under Test
Test Data Output
![Page 43: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/43.jpg)
What is a Test?
Software under Test
Test Data Output
Correct result?
![Page 44: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/44.jpg)
What is a Test?
Software under Test
Test Data Output
Correct result?
Oracle
![Page 45: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/45.jpg)
What is a Test?
Software under Test
Test Data Output
Correct result?
Oracle
Test Cases
![Page 46: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/46.jpg)
CISS
Types of Testing
unit
integration
system
performancerobustness
functional behaviour
white box black box
Level
Accessibility
Aspect
usability
reliability
![Page 47: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/47.jpg)
CISS
Types of Testing
unit
integration
system
performancerobustness
functional behaviour
white box black box
Level
Accessibility
Aspect
usability
reliability
![Page 48: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/48.jpg)
CISS
Quality-Characteristics (ISO-9126)• Functionality
• Suitability, accuracy, security, compliance, interoperability
• Reliability• maturity, fault tolerance, recoverability
• Usability• understandability, learnability, operability
• Efficiency• time behaviour, resource utilization
• Maintainability• Analysability, changeability, stability, testability
• Portability• Adaptability, intallability, conformance, replaceability
![Page 49: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/49.jpg)
CISS
Quality-Characteristics (ISO-9126)• Functionality
• Suitability, accuracy, security, compliance, interoperability
• Reliability• maturity, fault tolerance, recoverability
• Usability• understandability, learnability, operability
• Efficiency• time behaviour, resource utilization
• Maintainability• Analysability, changeability, stability, testability
• Portability• Adaptability, intallability, conformance, replaceability
⇒ functional testing
⇒ reliability testing
⇒ usability testing
⇒ performance testing
⇒maintainability testing ??
⇒ portability testing ?
![Page 50: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/50.jpg)
CISS
Module testtest driver
Module under test
test stub test stub
![Page 51: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/51.jpg)
CISS
Whitebox Exampleint invoice (int x, int y) {int d1, d2, s;if (x<=30) d2=100;else d2=90;s=5*x + 10 *y;if (s<=200) d1=100;else if (s<=1000) d1 = 95;
else d1 = 80;return (s*d1*d2/10000);
}
Test Cases
![Page 52: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/52.jpg)
CISS
Whitebox Exampleint invoice (int x, int y) {int d1, d2, s;if (x<=30) d2=100;else d2=90;s=5*x + 10 *y;if (s<=200) d1=100;else if (s<=1000) d1 = 95;
else d1 = 80;return (s*d1*d2/10000);
}
Test Cases
ifd2=100 d2=90
ifd1=100
if
d1=95
d1=80
return
x>30
s>200
s>1000
![Page 53: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/53.jpg)
CISS
Whitebox Exampleint invoice (int x, int y) {int d1, d2, s;if (x<=30) d2=100;else d2=90;s=5*x + 10 *y;if (s<=200) d1=100;else if (s<=1000) d1 = 95;
else d1 = 80;return (s*d1*d2/10000);
}
Test Cases
ifd2=100 d2=90
ifd1=100
if
d1=95
d1=80
return
x>30
s>200
s>1000
Test Data Expected Output
X=5 Y=5 75
X=31 Y=10 229.5
X=30 Y=100 977.5
![Page 54: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/54.jpg)
CISS
Black box testing
SUT
requirements
input
events
output
y
x
domain testing
![Page 55: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/55.jpg)
CISS
Module IntegrationM1
M6M5
M10
M2 M3 M4
M11 M12
M9M7 M8
”calls””uses”
”interacts”
![Page 56: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/56.jpg)
CISS
Embedded system
Generic Embedded System
Spe
cific
inte
rface
User interface
Interfaces to other systems
Pow
er s
uppl
y
A/D
+ D
/AD
igita
l I/O
Processing Unit
OSI/O-d
river
s Application Software
Pla
ntSystem environment
Act
uato
rsS
enso
rs
![Page 57: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/57.jpg)
CISS
System test
• 2*CRTG (4 channels) 2 * 200 k€
![Page 58: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/58.jpg)
CISS
Test Equipment
• Complete Type Approval Test System (3 M€)
![Page 59: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/59.jpg)
CISS
Who Should Test?
![Page 60: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/60.jpg)
CISS
Who Should Test?
• Developer• Understands the system• But, will test gently• And, is driven by
deadlines
![Page 61: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/61.jpg)
CISS
Who Should Test?
• Developer• Understands the system• But, will test gently• And, is driven by
deadlines
![Page 62: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/62.jpg)
CISS
Who Should Test?
• Developer• Understands the system• But, will test gently• And, is driven by
deadlines
• Independent tester• Must learn system• But, will attempt to break it• And, is driven by “quality”
![Page 63: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/63.jpg)
CISS
Who Should Test?
• Developer• Understands the system• But, will test gently• And, is driven by
deadlines
• Independent tester• Must learn system• But, will attempt to break it• And, is driven by “quality”
![Page 64: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/64.jpg)
CISS
Testing Objectives• To identify as many errors as possible• To bring the software to an acceptable level of
quality• To determine risk of release• Perform testing efficiently, effectively and within
budget and time constraints• Compile a record of software errors for future
error prevention and correction (learning organization).
![Page 65: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/65.jpg)
CISS
Risk Based Test Strategy• Make best possible use of resources by
identifying and prioritizing quality aspects and subsystems• Higher risk ⇒ more testing• No risk ⇒ no testing
• Risk = chance of failure × damage
Use frequencyChance of error being present
ComplexityNew tools/techniquesInexperienced developers
Cost of repairLoss of market shareLegal claim
![Page 66: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/66.jpg)
CISS
Testability = how easy can a program be tested
• operability - it operates cleanly• observability
• the results are easy to see• distinct output is generated for each input• incorrect output is easily identified
• controllability• processing can be controlled• tests can be automated & reproduced
• decomposability - software modules can be tested independently
• simplicity - no complex architecture and logic• stability - few changes are requested during testing• understandability - program is easy to understand
![Page 67: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/67.jpg)
CISS
Testing• Quality assurance activities by running code –
examines functionality in the form used by customers
• Software testing is a formal process carried out by a specialized testing team in which a software unit, several integrated software units or an entire software package are examined by running programs on a computer. All the associated tests are performed according to approved test procedures on approved test cases.
![Page 68: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/68.jpg)
CISS
Testing• Dynamic testing is the process of executing a
program or system with the intent of finding error (Glenford Meyers’ definition)
• Static testing is any activity that aims at finding defects by inspecting, reviewing, walking through, and analyzing any static component of the software (code, documents, and models)
• Debugging is an ad hoc activity performed by individual developers to find and remove bugs from a program.
• Testing is a planned activity
![Page 69: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/69.jpg)
CISS
There is More than Testing
static analysisstatic analysis
reviewingreviewinginspectioninspection
walk-throughwalk-throughdebuggingdebugging
certificationcertification
CMMCMM
quality controlquality controlrequirement managementrequirement managementsoftware processsoftware process
![Page 70: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/70.jpg)
CISS
Lifecycles & testing
![Page 71: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/71.jpg)
CISSimplementation
code
detailed design
requirements
specification
acceptancetest
systemtest
moduletest
unit-test
V - Model
integrationtest
architecturespec
acceptance test spec
system test spec
integration test spec
module test spec
unit test spec
![Page 72: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/72.jpg)
CISS
Introducing, Detecting andRepairing Errors Liggesmeyer 98
![Page 73: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/73.jpg)
CISS
Life Cycle : Testing as a Process
plan
ning
& co
ntro
l
prep
arat
ion
test
gene
ratio
n
test
exec
utio
n
com
plet
ion
T
L
OI
![Page 74: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/74.jpg)
CISS
Life Cycle : Testing as a Process
plan
ning
& co
ntro
l
prep
arat
ion
test
gene
ratio
n
test
exec
utio
n
com
plet
ion
T
L
OI
• Testing as phase of development process vs.
![Page 75: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/75.jpg)
CISS
Life Cycle : Testing as a Process
plan
ning
& co
ntro
l
prep
arat
ion
test
gene
ratio
n
test
exec
utio
n
com
plet
ion
T
L
OI
• Testing as phase of development process vs.
• Testing as a process itself• with its own phases
• in parallel with development process
![Page 76: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/76.jpg)
CISS
Life Cycle : Testing as a Process
plan
ning
& co
ntro
l
prep
arat
ion
test
gene
ratio
n
test
exec
utio
n
com
plet
ion
T
L
OI
• Testing as phase of development process vs.
• Testing as a process itself• with its own phases
• in parallel with development process
quality report
implementationspecificationrequirements
![Page 77: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/77.jpg)
CISS
Challenges of Testing
![Page 78: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/78.jpg)
CISS
Challenges of Testing• New embedded systems
• more functionality• increasingly advanced• faster time-to-market• higher quality
![Page 79: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/79.jpg)
CISS
Challenges of Testing• New embedded systems
• more functionality• increasingly advanced• faster time-to-market• higher quality
• Testing • more to be tested• more complicated• in less time• more thorough
![Page 80: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/80.jpg)
CISS
Challenges of Testing• New embedded systems
• more functionality• increasingly advanced• faster time-to-market• higher quality
• Testing • more to be tested• more complicated• in less time• more thorough
• skilled developers and testers• advanced testing tools and techniques• well organized• using solid development method
![Page 81: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/81.jpg)
CISS
Summary
![Page 82: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/82.jpg)
CISS
Some Testing Principles
![Page 83: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/83.jpg)
CISS
Some Testing Principles• Testing starts during the requirements phase
![Page 84: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/84.jpg)
CISS
Some Testing Principles• Testing starts during the requirements phase• The programmer shall not be the tester
![Page 85: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/85.jpg)
CISS
Some Testing Principles• Testing starts during the requirements phase• The programmer shall not be the tester• A test case specifies the test inputs
and the expected outputs
![Page 86: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/86.jpg)
CISS
Some Testing Principles• Testing starts during the requirements phase• The programmer shall not be the tester• A test case specifies the test inputs
and the expected outputs• Test cases shall also cover invalid and unexpected inputs
![Page 87: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/87.jpg)
CISS
Some Testing Principles• Testing starts during the requirements phase• The programmer shall not be the tester• A test case specifies the test inputs
and the expected outputs• Test cases shall also cover invalid and unexpected inputs• Test cases shall test that the program does what it should do
and that it does not do what it should not do
![Page 88: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/88.jpg)
CISS
Some Testing Principles• Testing starts during the requirements phase• The programmer shall not be the tester• A test case specifies the test inputs
and the expected outputs• Test cases shall also cover invalid and unexpected inputs• Test cases shall test that the program does what it should do
and that it does not do what it should not do• Test cases shall be recorded for reuse
![Page 89: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/89.jpg)
CISS
Some Testing Principles• Testing starts during the requirements phase• The programmer shall not be the tester• A test case specifies the test inputs
and the expected outputs• Test cases shall also cover invalid and unexpected inputs• Test cases shall test that the program does what it should do
and that it does not do what it should not do• Test cases shall be recorded for reuse• A test is successful when it detects an error
( but the project manager thinks differently ! )
![Page 90: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/90.jpg)
CISS
Some Testing Principles• Testing starts during the requirements phase• The programmer shall not be the tester• A test case specifies the test inputs
and the expected outputs• Test cases shall also cover invalid and unexpected inputs• Test cases shall test that the program does what it should do
and that it does not do what it should not do• Test cases shall be recorded for reuse• A test is successful when it detects an error
( but the project manager thinks differently ! )• No risk, no test
![Page 91: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/91.jpg)
CISS
Lessons• Testing starts when project starts• Have a development approach
• How to develop (waterfall, cyclic, etc) : roles and activities• Clear expectations between testers and developers• Clarity about phases and tasks• Easier planning
• Test execution takes 3 steps• Test, fix, re-test
• Let the programmers find bugs• 80 % of bugs can be found by programmers (but they don’t)
• Every morning discuss the issues found• Testers create test report at end of the day• Next morning discuss which issues have to be solved• Programmer and tester fix issue together
![Page 92: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/92.jpg)
CISS
Non-Structured testing• Start too late• Don’t organize and manage your test process• No test plan (V&V plan)• Unclear responsibilities• Conflicting interests• No fixed or stable specifications• Don’t learn
• from a defect• from a test• from statistics
• Use test tools in a chaotic environment• No configuration management and change control• No standard test procedures• Don’t educate• Do not provide career options
![Page 93: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/93.jpg)
CISS
Model Based Development
![Page 94: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/94.jpg)
CISS
Traditional Software Development
The Waterfall Model
Analyse
Design
Implementation
Testing♦Costly in time-to-market and money♦ Errors are detected late or never♦ Application of FM’s as early as possible
ProblemArea
Runn
ing
Syste
m
REVI
EWS
REVI
EWS
![Page 95: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/95.jpg)
CISS
Models• A model is a simplified representation of the
real world.• Engineers use models to gain confidence in
the adequacy and validity of a proposed design.
• Focus on one or more aspects of interest:• Safety• Liveness• Peak time• Performance
![Page 96: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/96.jpg)
CISS
Modelbased Validation
Design Model SpecificationVerification & Refusal
AnalysisValidation
FORMAL METHODS
ImplementationTesting
UML
Monitoring
![Page 97: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/97.jpg)
CISS
Modelbased Validation
Design Model SpecificationVerification & Refusal
AnalysisValidation
FORMAL METHODS
ImplementationTesting
UML
Monitoring
AutomaticCode generation
![Page 98: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/98.jpg)
CISS
Modelbased Validation
Design Model SpecificationVerification & Refusal
AnalysisValidation
FORMAL METHODS
ImplementationTesting
UML
Monitoring
AutomaticCode generation
AutomaticTest generation
![Page 99: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/99.jpg)
CISS
Modelbased Validation
Design Model SpecificationVerification & Refusal
AnalysisValidation
FORMAL METHODS
ImplementationTesting
UML
Monitoring
AutomaticCode generation
AutomaticTest generation
AutomaticMonitoringgeneration
![Page 100: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/100.jpg)
CISS
Multiple V-model
• Different objects, objectives, techniques in different stages
• Complexity of ES: functional decomposition and stepwise integration, but tight coordination (e.g independent development of HW and SW)
• Cost: Detect and correct errors earlier => Test earlier
Model Prototype(s)Development Iteration(s) Final Product
design
build
test design
build
test
design
build
test
![Page 101: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/101.jpg)
CISS
END
![Page 102: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/102.jpg)
CISS
Control Systems
![Page 103: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/103.jpg)
CISS
Procedure P; begin while A do begin X; if B then if C then Y else Z else beginV; W end end end
![Page 104: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/104.jpg)
CISS
Testing: A Definition
![Page 105: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/105.jpg)
CISS
Testing: A DefinitionTesting is not:• static analysis• reviewing or inspection• walk-through• debugging• bug fixing• auditing• quality control• validation of the
specification
![Page 106: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/106.jpg)
CISS
Testing: A DefinitionTesting is not:• static analysis• reviewing or inspection• walk-through• debugging• bug fixing• auditing• quality control• validation of the
specification
But testing:• is one of the
instrumentsfor measuring quality
• increases value of productby establishing confidencein its quality
• helps in assessing risk of putting product into operation
![Page 107: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/107.jpg)
CISS
Verification & Validation
![Page 108: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/108.jpg)
CISS
Verification & Validation• Verification :
![Page 109: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/109.jpg)
CISS
Verification & Validation• Verification :
building the product right
• concerns development process
• intermediate products
• e.g. checking the testability of the specification
![Page 110: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/110.jpg)
CISS
Verification & Validation• Verification :
building the product right
• concerns development process
• intermediate products
• e.g. checking the testability of the specification
• Validation :
![Page 111: Introduction To Software Testing - Aalborg Universitetpeople.cs.aau.dk/~bnielsen/MBTV07/material/intro.pdfIntroduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of](https://reader036.fdocuments.us/reader036/viewer/2022062602/5ed7a58d48b98015c2020e68/html5/thumbnails/111.jpg)
CISS
Verification & Validation• Verification :
building the product right
• concerns development process
• intermediate products
• e.g. checking the testability of the specification
• Validation :
building the right product
• whether final product meets requirements and needs
• e.g. by reviewing the specification