schach5-chap06-14[1]
![download schach5-chap06-14[1]](https://fdocuments.us/public/t1/desktop/images/details/download-thumbnail.png)
of 37
-
Upload
hsiangliu4806 -
Category
Documents
-
view
212 -
download
0
Transcript of schach5-chap06-14[1]
-
8/14/2019 schach5-chap06-14[1]
1/37
-
8/14/2019 schach5-chap06-14[1]
2/37
Slide 6.2
The McGraw-Hill Companies, 2002
CHAPTER 6
TESTING
-
8/14/2019 schach5-chap06-14[1]
3/37
Slide 6.3
The McGraw-Hill Companies, 2002
Overview
q Quality issues
q Nonexecution-based testing
q Execution-based testing
q What should be tested?
q Testing versus correctness proofs
q Who should perform execution-based testing?
q When testing stops
-
8/14/2019 schach5-chap06-14[1]
4/37
Slide 6.4
The McGraw-Hill Companies, 2002
Testing
q Two types of testing Execution-based testing
Nonexecution-based testing
-
8/14/2019 schach5-chap06-14[1]
5/37
Slide 6.5
The McGraw-Hill Companies, 2002
Testing (contd)
q V & V Verification
Determine if the phase was completed correctly
Validation Determine if the product as a whole satisfies its requirements
-
8/14/2019 schach5-chap06-14[1]
6/37
-
8/14/2019 schach5-chap06-14[1]
7/37
Slide 6.7
The McGraw-Hill Companies, 2002
Software Quality
q Not excellence
q Extent to which software satisfies its specifications
q Software Quality Assurance (SQA) Goes far beyond V & V
Managerial independence development group
SQA group
-
8/14/2019 schach5-chap06-14[1]
8/37
Slide 6.8
The McGraw-Hill Companies, 2002
Nonexecution-Based Testing
q Underlying principles We should not review our own work
Group synergy
-
8/14/2019 schach5-chap06-14[1]
9/37
Slide 6.9
The McGraw-Hill Companies, 2002
Walkthroughs
q 46 members, chaired by SQA
q Preparationlists of items
q Inspection Up to 2 hours
Detect, dont correct Document-driven, not participant-driven
Verbalization leads to fault finding
Performance appraisal
-
8/14/2019 schach5-chap06-14[1]
10/37
Slide 6.10
The McGraw-Hill Companies, 2002
Inspections
q Five-stage process Overview
Preparation, aided by statistics of fault types
Inspection
Rework Follow-up
-
8/14/2019 schach5-chap06-14[1]
11/37
Slide 6.11
The McGraw-Hill Companies, 2002
Fault statistics
q Recorded by severity and fault type
q Compare with previous products
q What if there are a disproportionate numberof faults in a module?
q Carry forward fault statistics to the next phase
-
8/14/2019 schach5-chap06-14[1]
12/37
Slide 6.12
The McGraw-Hill Companies, 2002
Statistics on Inspections
q 82% of all detected faults (IBM, 1976)q 70% of all detected faults (IBM, 1978)q 93% of all detected faults (IBM, 1986)q 90% decrease in cost of detecting fault
(Switching system, 1986)q 4 major faults, 14 minor faults per 2 hours
(JPL, 1990). Savings of $25,000perinspection
q Number of faults decreased exponentiallyby phase (JPL, 1992)
q Warning Fault statistics and performance appraisal
-
8/14/2019 schach5-chap06-14[1]
13/37
Slide 6.13
The McGraw-Hill Companies, 2002
Metrics for Inspections
q Fault density (e.g., faults per KLOC)
q Fault detection rate (e.g., faults detected per hour)
q By severity (major/minor), by phase
q What does a 50% increase in the fault detection
rate mean?
-
8/14/2019 schach5-chap06-14[1]
14/37
Slide 6.14
The McGraw-Hill Companies, 2002
Execution-Based Testing
q Definitions Failure (incorrect behavior)
Fault (NOT bug)
Error (mistake made by programmer)
q Nonsensical statement Testing is demonstration that faults are not present
-
8/14/2019 schach5-chap06-14[1]
15/37
Slide 6.15
The McGraw-Hill Companies, 2002
What is execution-based tested?
q The process of inferring certain behavioralproperties of product based, in part, on resultsof executing product in known environment withselected inputs.
Inference Known environment
Selected inputs
q But what should be tested?
-
8/14/2019 schach5-chap06-14[1]
16/37
Slide 6.16
The McGraw-Hill Companies, 2002
Utility
q Does it meet users needs? Ease of use
Useful functions
Cost-effectiveness
-
8/14/2019 schach5-chap06-14[1]
17/37
Slide 6.17
The McGraw-Hill Companies, 2002
Reliability
q Frequency and criticality of failure Mean time between failures
Mean time to repair
Mean time, cost to repairresults of failure
-
8/14/2019 schach5-chap06-14[1]
18/37
Slide 6.18
The McGraw-Hill Companies, 2002
Robustness
q Range of operating conditionsq Possibility of unacceptable results with valid inputq Effect of invalid input
-
8/14/2019 schach5-chap06-14[1]
19/37
Slide 6.19
The McGraw-Hill Companies, 2002
Performance
q Extent to which space and time constraints are met
q Real-time software
-
8/14/2019 schach5-chap06-14[1]
20/37
Slide 6.20
The McGraw-Hill Companies, 2002
Correctness of specifications
q Incorrect specification for a sort
q
Function trickSort which satisfies this specification:
-
8/14/2019 schach5-chap06-14[1]
21/37
Slide 6.21
The McGraw-Hill Companies, 2002
Correctness of specifications (coptd)
q Incorrect specification for a sort:
q Corrected specification for the sort:
-
8/14/2019 schach5-chap06-14[1]
22/37
Slide 6.22
The McGraw-Hill Companies, 2002
Correctness
q NOT necessary
q NOT sufficient
-
8/14/2019 schach5-chap06-14[1]
23/37
Slide 6.23
The McGraw-Hill Companies, 2002
Correctness Proofs
q Alternative to execution-based testing
-
8/14/2019 schach5-chap06-14[1]
24/37
Slide 6.24
The McGraw-Hill Companies, 2002
Example of Correctness Proof
q Code segment to be proven correct
-
8/14/2019 schach5-chap06-14[1]
25/37
Slide 6.25
The McGraw-Hill Companies, 2002
Example of Correctness Proof (contd)
q Flowchart of
code segment
-
8/14/2019 schach5-chap06-14[1]
26/37
Slide 6.26
The McGraw-Hill Companies, 2002
Example of Correctness Proof
-
8/14/2019 schach5-chap06-14[1]
27/37
Slide 6.27
The McGraw-Hill Companies, 2002
Correctness Proof Case Study
q Never prove a program correct without testing it
as well
-
8/14/2019 schach5-chap06-14[1]
28/37
Slide 6.28
The McGraw-Hill Companies, 2002
Episode 1
q 1969 Naur Paper
q Naur text-processing problem
Given a text consisting of words separated byblankor by(new line) characters, convert it to line-by-line form in
accordance with following rules:(1) line breaks must be made only where given texthas blankornl ;(2) each line is filled as far as possible, as long as
(3) no line will contain more than maxpos charactersq Naur constructed a procedure (25 lines of Algol 60)
and informally proved its correctness
-
8/14/2019 schach5-chap06-14[1]
29/37
Slide 6.29
The McGraw-Hill Companies, 2002
Episode 2
q 1970 Reviewer in Computing Reviews The first word of the first line is preceded by blank unless
the first word is exactly maxpos characters long
-
8/14/2019 schach5-chap06-14[1]
30/37
Slide 6.30
The McGraw-Hill Companies, 2002
Episode 3
q 1971 London finds 3 more faults
q Including: The procedure does not terminate unless a word
longer than maxpos characters is encountered
-
8/14/2019 schach5-chap06-14[1]
31/37
Slide 6.31
The McGraw-Hill Companies, 2002
Episode 4
q 1975 Goodenough and Gerhart
find three further faultsq Including:
The last word will not be output unless it
is followed byblank
ornl
-
8/14/2019 schach5-chap06-14[1]
32/37
Slide 6.32
The McGraw-Hill Companies, 2002
Proofs and Software Engineering
q Lesson:
q Even if product is proved correct,it must STILL be tested.
-
8/14/2019 schach5-chap06-14[1]
33/37
Slide 6.33
The McGraw-Hill Companies, 2002
Three Myths
q Software engineers do not have enough
math for proofsq Proving is too expensive to be practical
q Proving is too hard
f S f ( )
-
8/14/2019 schach5-chap06-14[1]
34/37
Slide 6.34
The McGraw-Hill Companies, 2002
Proofs and Software Engineering (contd)
q Can we trust a theorem prover ?
q How to find inputoutput specifications,loop invariants
q What if the specifications are wrong?
q Can never be sure that specificationsor a verification system are correct
P f d S ft E i i ( td)
-
8/14/2019 schach5-chap06-14[1]
35/37
Slide 6.35
The McGraw-Hill Companies, 2002
Proofs and Software Engineering (contd)
q Correctness proofs are a vital software engineering
tool, WHERE APPROPRIATE.q If
Human lives are at stake
Indicated by cost/benefit analysis Risk of not proving is too great
q Also, informal proofs can improve the quality ofthe product Assertstatement
Wh P f E ti B d T ti ?
-
8/14/2019 schach5-chap06-14[1]
36/37
Slide 6.36
The McGraw-Hill Companies, 2002
Who Performs Execution-Based Testing?
q Testing is destructive
A successful test finds a fault
q Solution 1. The programmer does informal testing
2. SQA does systematic testing 3. The programmer debugs the module
q All test cases must be Planned beforehand, including expected output
Retained afterwards
Wh T ti C St
-
8/14/2019 schach5-chap06-14[1]
37/37
Slide 6.37When Testing Can Stop
q Only when the product has been irrevocably retired
