Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it...
Transcript of Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it...
![Page 2: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/2.jpg)
”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979
“Testing is measuring quality” - William Hetzel, 1988
Testing reduces our risks
2008-10-07 2
o
![Page 3: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/3.jpg)
”Measure” quality (”Testing is to knowing everything”)
Release the right product in timeNOT to lose moneyStay in the market
2008-10-07 3
o
![Page 4: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/4.jpg)
Enhance, and help the development of, the productVerifiy requirements/functionalityMeasure the Quality of the Product
2008-10-07 4
![Page 5: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/5.jpg)
2008-10-07 5
Many defects found Few defects found
Few defects found Few defects found
High
HighLow
Low
Test Quality
SW Quality
You think you are here
..but you are here
o
![Page 6: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/6.jpg)
”Testing is expensive” ◦ Manhours for test◦ Test environmentCompared to:◦ What does it cost to recall a release?◦ What does it cost to get unhappy customers?◦ What does it cost to lose market volume because
the quality is too poor? ◦ etc...
2008-10-07 6
o
![Page 7: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/7.jpg)
Economical
Communication
Ethical
2008-10-07 7
![Page 8: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/8.jpg)
Exhaustive-testning
What is Exhaustive-testning?◦ We test all combinations of in parameters, prerequisites and
environments.
How long time does it take? (Yes/No)◦ Infinitely long time◦ Short time◦ Impractically long time
2008-10-07 8
o
![Page 9: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/9.jpg)
It depends of the risk of:◦ Miss serious defects◦ The defects we will find are expensive to correct.◦ Need to recall a release.◦ Releasing bad code◦ Losing market volume◦ Not be first on the market◦ Execute effective testing
2008-10-07 9
How is risk normally defined?Probability * ImpactRisk = S * K
o
![Page 10: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/10.jpg)
2008-10-07 10
Prioritise the testing so that whenever you stop testing you have done the best possible testing on the given time.
o
![Page 11: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/11.jpg)
ISO/IEC 12119◦ Information technology - Software packages -
Quality requirements and testingISO/IEC 14598◦ Information technology - Software product
evaluationIEEE Std 829◦ Standard for test documentationSIGIST◦ Specialist Interest Group in Software TestingISTQB◦ International Software Testing Qualifications Board
2008-10-07 11
o
![Page 12: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/12.jpg)
VerificationValidationTesting
BugError, Defect, FailureReliability
2008-10-07 12
o
![Page 13: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/13.jpg)
”The process of evaluating a system or component to determin whether the products of the given development phase satisfy the conditions imposed at the start of the phase”[ BS 7925-1]
2008-10-07 13
That the system is correct according to the written requirements.
o
![Page 14: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/14.jpg)
”Determination of the correctnesss of the products of software development with respect to the user needs and requirements”[ BS 7925-1]
2008-10-07 14
That the system is correct to the real requirements.
o
![Page 15: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/15.jpg)
”The process of exercising software to verify that it satisfies specifed requirements and to detect defects”[ BS 7925-1]
2008-10-07 15
o
![Page 16: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/16.jpg)
2008-10-07 16
o
![Page 17: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/17.jpg)
2008-10-07 17
A person does aprogramming error.ERROR
The Error createsa defect in the code.DEFECT
The defect can create aunwanted reaction.FAILURE
![Page 18: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/18.jpg)
2008-10-07 18
Test closure activities
Evaluating exit criteria and reporting
Implementation and execution
Analisys and Design
Planning and control
Delivery
START
o
![Page 19: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/19.jpg)
Unit test: The lowest level of testing, often executed by a developer.Functionality test: Test of the functionality of a controlled part of the system.System test: Tests the system.Acceptance test: Customer/user/supplier/3:e partAlpha- & Beta-test: executed by user/possible customer to obtain feedback and for information.Test in the maintenance level, mostly executed when changes are made to the product.
2008-10-07 19
![Page 20: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/20.jpg)
2008-10-07 20
Static testing
Formal / informal review Staticanalysis
o
![Page 21: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/21.jpg)
2008-10-07 21
Effective -Finds defects
Well constructed -represents other TC
Maintanable -easy to maintain
Economical -easy to use
o
![Page 22: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/22.jpg)
System with 2 input fields◦ Choose a method for black-box testing◦ How many test cases would you write?
2008-10-07 22
o
Age
Shoe size
![Page 23: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/23.jpg)
Statical (does not execute any code)Whitebox (glassbox)BlackboxOther (ex. experience based)
2008-10-07 23
Goto 10for i=1 to 100
o
![Page 24: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/24.jpg)
Black-box methods:◦ Equivalence partitioning (EP)◦ Boundary value analysis (BVA)◦ Random input◦ etc...
White box methods◦ Statements, decisions, test path coverage, dataflow,
branch coverage etc...
Experience based◦ Error guessing◦ Exploratory testing
![Page 25: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/25.jpg)
System with 2 input fields◦ Choose a method for black-box testing◦ How many test cases would you write?
2008-10-07 25
o
Age
Shoe size
![Page 26: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/26.jpg)
2008-10-07 26
UC1-1UC2-1....
Requirement spec.
TEST CASE 1UC1-1
TEST CASE2UC2-1
Test spec..
TestCase1 UC1-1 OK
TestCase2 UC2-1NOK
Test Protocol
UC2-1 NOKTestcase2, Step X
Incident report
o
![Page 27: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/27.jpg)
2008-10-07 27
![Page 28: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/28.jpg)
Examples of risk criterias:◦ Test where the defect has the most impact◦ Test where the defects are most visible◦ Test where defects are most probable◦ Ask the customer to prioritise◦ Test where many changes have been done◦ Test where many defects have been found◦ Test where most complexity or complicated functions exist
2008-10-07 28
![Page 29: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/29.jpg)
Evil messsenger, ”your baby is ugly”
29
![Page 30: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/30.jpg)
A creative workLearn more and deeper than most peopleIn the middle of the actionWide technical skillsDeep technical skillsHard, a challengeMay sometimes pretend that he is a customer
30
![Page 31: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/31.jpg)
ISTQB – www.istqb.org -- downloadsTMap & TMap NextTPI (Test Process Improvement)
www.stickyminds.com
2008-10-07 31
![Page 32: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/32.jpg)
Creating a test plan
![Page 33: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/33.jpg)
Management doc not a techincal docWhat (tested items, deliverables)Why? – give reasoning for decisionsHow? – give indications for methodsWho? – how many people, what kind ofpeople?When? – at what stage, when to stopWhere? – which environment, technicalrequirements etc.
![Page 34: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/34.jpg)
Define the purpose of testingDefine test items◦ Funcional◦ Non-functional
PerformanceResponse-timeScalabilitySecurity and authenticationRobustness, handling errors, wrong intput etc.
Tested or not testedGive reasoning for each!
![Page 35: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/35.jpg)
Approach – white-box, black-boxAcceptance criteriaDeliverables – test spec, instructions, protocols, reports, scripts, programsEnvirionmental needs – OS, browser etc.Testing tasks – writing spec, install, test datacreation, execution, bug reports, reportingRoles/responsibilities - creator-reviewerStaff & trainingScheduleRisks
![Page 36: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/36.jpg)
1. Write a test plan with your group1. Purpose & exit criteria2. Test items3. Environment needs4. Tasks5. Risks (project/product risks not testing risks!)
2. Write a review of your neighbours test plan3. Submit when done◦ Put your names on!!!
4. ASK QUESTIONS!
![Page 37: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/37.jpg)
An application ordocumentation?
![Page 38: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/38.jpg)
![Page 39: Kaspar Loog kaspar.loog@knowit · ”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979 ` “Testing is measuring quality” - William](https://reader033.fdocuments.us/reader033/viewer/2022050517/5fa177dd9d4ab974d003dd83/html5/thumbnails/39.jpg)
http://www.cs.gordon.edu/courses/cs211/AddressBookExample/Executable.html