Test vs Inspection
-
Upload
gaspardo123 -
Category
Documents
-
view
232 -
download
0
Transcript of Test vs Inspection
-
7/30/2019 Test vs Inspection
1/40
Test vs. inspection
Tor Stlhane
-
7/30/2019 Test vs Inspection
2/40
What we will cover
Part 1
Introduction
Inspection processes Testing processes
Part 2
Tests and inspections some data Inspection as a social process two experiments
and some conclusions
-
7/30/2019 Test vs Inspection
3/40
Adams data - 1
Mean Time to Problem Occurrence
yearsProduct 1.6 5 16 50 160 500 1600 5000
1 0.7 1.2 2.1 5.0 10.3 17.8 28.8 34.22 0.7 1.5 3.2 4.3 9.7 18.2 28.0 34.33 0.4 1.4 2.8 6.5 8.7 18.0 28.5 33.74 0.1 0.3 2.0 4.4 11.9 18.7 28.5 34.25 0.7 1.4 2.9 4.4 9.4 18.4 28.5 34.26 0.3 0.8 2.1 5.0 11.5 20.1 28.2 32.07 0.6 1.4 2.7 4.5 9.9 18.5 28.5 34.08 1.1 1.4 2.7 6.5 11.1 18.4 27.1 31.99 0.0 0.5 1.9 5.6 12.8 20.4 27.6 31.2
-
7/30/2019 Test vs Inspection
4/40
Adams data 2
The main information that you get from the
table on the previous slide is that
Some defects are important because they will
happen quite often.
Most defects are not important since they will
happen seldom.
How can we tell the difference?
-
7/30/2019 Test vs Inspection
5/40
Testing and inspection the V model
-
7/30/2019 Test vs Inspection
6/40
Testing and inspection 1
The important message here is that testingcannot always be done.
In the first, important phases, we have nothing
to execute and will thus always have to dosome type of inspection.
This might be considered one of the weaknesses
of traditional software engineering over Agiledevelopment.
-
7/30/2019 Test vs Inspection
7/40
Testing and inspection 2
In order to understand the main differences
between testing and inspection, we should
consider Fits list.
Based on this, we will give a short discussion of
the relative merits of testing and inspection.
-
7/30/2019 Test vs Inspection
8/40
Area of
competence
Man Machine
Understanding Good at handling variations in
written material
Bad at handling variations in
written materialObserve General observations,
multifunctional
Specialized, good at observing
quantitative data, bad at
pattern recognition
Reasoning Inductive, slow, imprecise but
good at error correction
Deductive, fast, precise but
bad error correction
Memory Innovative, several access
mechanisms
Copying, formal access
Information
handling
Single channel, less than 10
bits per second
Multi channel, several
Megabits per second
Consistency Unreliable, get tired, depends
on learning
Consistent repetition of several
actions
Power Low level, maximum ca. 150
watt
High level over long periods
of time
Speed Slow seconds Fast
-
7/30/2019 Test vs Inspection
9/40
Man vs. machine 1
Good when we need the ability to Handle variation
Be innovative and inductive
Recognize and handle patterns
Not so good when we need the ability to
Do the same things over and over again in aconsistent manner
Handle large amount of data
-
7/30/2019 Test vs Inspection
10/40
Man vs. machine 2
In order to do the best job possible we needprocesses where we let each part
Do what they are best at:
Man is innovative Machine handles large amounts of data
Support the other with their specialties.
Machine supports man by making large amountsof information available
Man support machine by providing it with
innovative input
-
7/30/2019 Test vs Inspection
11/40
General considerations - documents
Architecture, system, sub-system and
component design plus pseudo code. Here we
can only use inspections.
Man will use experience and knowledge to
identify possible problems
Machine can support by identifying information
e.g. find all occurrences of a string.
-
7/30/2019 Test vs Inspection
12/40
General considerations code (1)
For executable code, we can use inspection,testing or a combination of both.
The size and complexity degree of dynamism
of the code will, to a large degree, decide ourchoice.
Other important factors are the degree of
experience with The programming language
The algorithms used
-
7/30/2019 Test vs Inspection
13/40
General considerations code (2)
Simple code Start with inspection all code
Design and run tests
Complex code Start with inspection focus on algorithm and
logic
Decide test completeness criteria we cannottest everything
Design and run tests
-
7/30/2019 Test vs Inspection
14/40
Inspection processes
-
7/30/2019 Test vs Inspection
15/40
Inspections 1
The term inspection is often used in a rather
imprecise manner. We will look at three typesof inspection:
Walkthrough
Informal inspection also called informalreview
Formal inspection also called formal review
or just inspectionThe first two types are usually project internal
while the last one is used as a final acceptanceactivity for a document.
-
7/30/2019 Test vs Inspection
16/40
Inspections 2
For all types of inspections:
The quality of the results depends on the
experience and knowledge of the participants.
Garbage in Garbage out
It might be a good idea to involve customer
representatives.
-
7/30/2019 Test vs Inspection
17/40
The walkthrough process
Walkthrough is a simple process mostly used
for early decisions for an activity. The
document owner:
1. Makes a rough sketch of the solution
architecture, algorithm etc.
2. Presents explain the sketch to whoever
shows up.
3. Registers feedback improvements.
-
7/30/2019 Test vs Inspection
18/40
Walkthrough pros and cons
Pros: Easy and inexpensive. Needs no extra
preparation.
Collect ideas at an early stage of development.Cons:
No commitment from the participants
May collect many loose or irrelevant ideas
-
7/30/2019 Test vs Inspection
19/40
The informal inspection process
Individual
checking
Planning
Productdocument
Changerequests
Rules,checklists,
procedures
Logging
meeting
-
7/30/2019 Test vs Inspection
20/40
Informal inspections pros and cons
Pros: Is simple and inexpensive to perform.
Can be used at all stages of development
Usually has a good cost / benefit ratio
Needs a minimum of planning
Cons:
No participant commitment
No process improvement
-
7/30/2019 Test vs Inspection
21/40
The formal inspection process
The formal inspection process described below
is with small variations the most
commonly used. The version shown on the
following slides stem from T. Gilb and D.Graham.
We recommend this process as the final
acceptance process for all importantdocuments
-
7/30/2019 Test vs Inspection
22/40
Formal inspection process overview
Walk-
through
Kick-off Individual
checking
Edit
and
follow-
up
Planning
Process
improvements
Product
document
Changerequests
Rules,
checklists,
procedures
Logging
meeting
-
7/30/2019 Test vs Inspection
23/40
Distribution of resources
Activity Range %Typical
value %
Planning 3 5 4
Kick-off 4 7 6
Individual checking 20 30 25
Logging 20 30 25
Editing 15 30 20
Process brainstorming 15 30 16
Leader overhead, follow up, entry,
exit
3 5 4
-
7/30/2019 Test vs Inspection
24/40
Initiating the inspection process
The inspection process starts with a request
for inspection from the author to the QA
responsible.
The QA responsible appoints an inspection
leader.
First step is always to check that the
document is fit for inspection.
-
7/30/2019 Test vs Inspection
25/40
Planning
Important planning points are:
Who should participate in the inspections
Who is interested?
Who have time available for preparation and
meetings?
Who has the necessary knowledge concerning
application, language, tools, methods?
-
7/30/2019 Test vs Inspection
26/40
Kick-off
Important activities here are:
Distribution of necessary documents:
Documents that shall be inspected
Requirements
Applicable standards and checklists
Assignment of roles and jobs
Setting targets for resources, deadlines etc.
-
7/30/2019 Test vs Inspection
27/40
Individual checking
This is the main activity of the inspection. Each
participant read the document to look for
Potential errors - inconsistencies with
requirements or common application
experience
Lack of adherence to company standards or
good workmanship
-
7/30/2019 Test vs Inspection
28/40
-
7/30/2019 Test vs Inspection
29/40
Improve the product - 1
The author receives the log from the inspection
meeting. All items - issues - in the log are
categorised as one of the following:
Errors in the authors document.
Errors in someone elses document.
Misunderstandings in the inspection team.
-
7/30/2019 Test vs Inspection
30/40
Improve the product - 2
Errors in own document:
Make appropriate corrections
Errors in someone elses documents:
Inform the owner of this document.
Misunderstandings in the inspection team:
Improve document to avoid further
misunderstandings.
-
7/30/2019 Test vs Inspection
31/40
Checking the changes
This is the responsibility of the inspectionleader. He must assure that all issues raised
in the log are disposed of in a satisfactory
manner: The documents that have been inspected
Related documents - including standards
and checklists Suggested process improvements
-
7/30/2019 Test vs Inspection
32/40
Formal inspection pros and cons
Pros:
Can be used to formally accept documents
Includes process improvement
Cons:
Is time consuming and expensive
Needs extensive planning in order to succeed
-
7/30/2019 Test vs Inspection
33/40
Testing processes
-
7/30/2019 Test vs Inspection
34/40
Testing
We will look at three types of testing:
Unit testing does the code behave as
intended. Usually done by the developer
Function verification testing also calledsystems test. Does the component or system
provide the required functionality?
System verification testing also called
acceptance test. Does the hardware and
software work together to give the user the
intended functionality?
-
7/30/2019 Test vs Inspection
35/40
The unit testing process
Unit testing is done by the developer one or
more times during development. It is a rather
informal process which mostly run as follows:
1. Implement (part of) a component.
2. Define one or more tests to activate the code
3. Check the results against expectations and
current understanding of the component
-
7/30/2019 Test vs Inspection
36/40
Unit testing pros and cons
Pros:
Simple way to check that the code works.
Can be used together with coding in an
iterative manner.
Cons:
Will only test the developers understandingof the spec.
May need stubs or drivers in order to test
-
7/30/2019 Test vs Inspection
37/40
The system test process
A systems test has the following steps:
1. Based on the requirements, identify
Test for each requirement, including error handling
Initial state, expected result and final state
2. Identify dependencies between tests
3. Identify acceptance criteria for test suite
4. Run tests and check results against
Acceptance criteria for each test Acceptance criteria for the test suite
-
7/30/2019 Test vs Inspection
38/40
Systems test pros and cons
Pros:
Tests systems behavior against customer
requirements.
Cons:
It is a black box test. If we find an error, the
systems test must be followed by extensive
debugging
-
7/30/2019 Test vs Inspection
39/40
The acceptance test process
The acceptance test usually has three actvities
both involving the customer or his
representatives:
Rerun the systems test at the customers site.
Use the system to solve a set of real-world
tasks.
Try to break the system by stressing it or by
feeding it large amounts of illegal input
-
7/30/2019 Test vs Inspection
40/40
Acceptance test pros and cons
Pros:
Creates confidence that the system will beuseful for the customer
Shows the systems ability to operate in thecustomers environment
Cons:
Might force the system to handle input that itwas not designed for, thus creating anunfavorable impression.