How to make the step from testing to QA
description
Transcript of How to make the step from testing to QA
How to make the step from testing to quality assurance
Jan Jaap CannegieterVice President SYSQA B.V. (The Netherlands)
jcannegieter @sysqa.nl or @jjcannegieter
© S
YS
QA
Alm
ere
Questions that will be answered
• What kind of QA fits a tester best?• Why is everybody else not asking it by themselves?• How to convince your manager to implement QA?• How can I show upfront and afterwards that QA pays?• How much does it pay?• How to deal with agile and outsourcing?
© S
YS
QA
Alm
ere
What is quality assurance?
3
The planned and systematic activities implemented in a quality system so that quality requirements for a product or service will be fulfilled
Doc.
Doc.
Code
Data
Code
Doc. Process
Process
Process
Result
© S
YS
QA
Alm
ere
Product QA
4
Doc.
Doc.
Code
Data
Code
Doc. Process
Process
Process
Result
© S
YS
QA
Alm
ere
Process QA
5
Doc.
Doc.
Code
Data
Code
Doc. Process
Process
Process
Result
© S
YS
QA
Alm
ere
Project QA
6
Doc.
Doc.
Code
Data
Code
Doc. Process
Process
Process
Result
© S
YS
QA
Alm
ere
7
Essence of product QA
Inspections/reviews/audits
Development Testing#
defects
Quality
© S
YS
QA
Alm
ere
9
MANAGEMENT:YEAH, NICE.
BUT WHEN IS TESTING READY?
AND WHAT DOES IT COST?
Problem 1: How do I convince management
© S
YS
QA
Alm
ere
What are important arguments for a manager
• Improve quality• Improve certainty • Improve continuity• Save time• Save money• Ease / comfort• Safety• Status
✔✔✔✖✔✔✖✖
Line
mng
r
✖✖✖✔✔✖✖✔
Proj.
mngr
✔✔✖✖✖✔✔✖
Engi
neer
Problem 2: Where do I find undisputed,
organizational specific data that shows upfront that QA saves
money and time?
© S
YS
QA
Alm
ere
We have the key!
13
Defects
© S
YS
QA
Alm
ere
Calculating the advantage of product QA
Barry Boehm
Capers Jones
SYSQA
14
=
Savings
32
16
8
4
2
1
-
Phase
Requirements
Functional design
Technical design
Code
Unit test
Functional test
Production
© S
YS
QA
Alm
ere
The case
• Big telecom organization• Testdepartment with approximately 200 testers, test
managers etc.• Projects are exceeding time and budget• Quality not good (enough)
15
© S
YS
QA
Alm
ere
16
Phase # defects Factor Pot. savings
Requirements 6 32 192 hours
Functional design 5 16 80 hours
Technical design - 8
Build 7 4 28 hours
Unit test - 2
Totaal 18 300 hours
Calculating costs of poor quality - I
Average € 80,= per hour: ‘avoidable failure costs’: € 24.000,-With review effectiveness of 50% actual savings: € 12.000,-
© S
YS
QA
Alm
ere
17
Phase # defects Factor Pot. savings
Requirements 65 32 2080 hours
Functional design 225 16 3600 hours
Technical design 60 8 480 hours
Build 105 4 420 hours
Unit test - 2 -
Totaal 457 6580 hours
Calculating costs of poor quality - II
Average € 80,= per hour: ‘avoidable failure costs’: € 526.400 ,-With review effectiveness of 50% actual savings: € 263.200,-
© S
YS
QA
Alm
ere
18
YOU HAVE A GO
MANAGEMENTTEST DEPARTMENT:
Problem 3: How do you measure the benefits?
© S
YS
QA
Alm
ere
Benefits per defect found in reviews
22
Savings
32
16
8
4
2
1
-
Phase
Requirements
Functional design
Technical design
Code
Unit test
Functional test
Production
© S
YS
QA
Alm
ere
Benefits
23
First six mounths
# Inspections 124
Solved majors 1385
Time spend on inspections in hours 2476
Saved hours 25354
Return on investment 10.24
Saved money in 6 mouths 1.601.460,-€ 3.2 mln per year
© S
YS
QA
Alm
ere
Different cases
Case 1: shared service center• ROI: 7,1• Saved: € 377.600 / 16%
Case 2: system dev. outsourced• ROI: 11,9• Saved: € 51.200
Case 3: insurance company• ROI: 6,8• Saved: € 1.824.000
Problem 4: How do you expand your focus?
© S
YS
QA
Alm
ere
Optimize and expand QA
Quality of a phase =
26
# defects removed in phase X
(# defects phase X-1) + (# defects injected phase X)Requirements is
your problem
© S
YS
QA
Alm
ere
So, how do I make the step from testing to QA?
• Use the arguments that fits the personal interest of the decision maker
• Analyse your defects: determine moment of injection• Quantify the cost of poor quality / avoidable failure costs
“We could have saved this, would you like to save it in the future.”
• Measure and communicate the benefits• Keep improvingAnd• Develop you political skills• Develop your persuasion skills / sales skills• Develop your change management skills
27
Congratulations,
you’re A QA-officer now
© S
YS
QA
Alm
ere
Everybody happy?
28
This is not the end of the story …
“We are going to implement scrum so we don’t need QA
anymore”
© S
YS
QA
Alm
ere
30
Product backlog
User story 4User story 5
User requirements
User story 1User story 2User story 3
Systemreq., design,
build, test
Systemreq., design,
build, test
Sprints
€
€
Quality Assurance
Quality Assurance in agile projecten
Productvision
Business requirements
“We are going to outsource everything, so we don’t need the QA anymore”
© S
YS
QA
Alm
ere
Outsourcing and quality assurance
Outsourced process• Design• Development • System test
Pro
ject
def
init
ion
•R
equi
rem
ents
•S
uppl
ier
sele
ctio
n•
Con
trac
ts
Acc
epta
nce
© S
YS
QA
Alm
ere
Outsourcing and quality assurance
Outsourced process• Design• Development • System test
Pro
ject
def
init
ion
•R
equi
rem
ents
•S
uppl
ier
sele
ctio
n•
Con
trac
ts
Acc
epta
nce
Quality Assurance
© S
YS
QA
Alm
ere
The answers
• What kind of QA fits a tester best?Product QA
• Why is everybody else not asking it by themselves?Because I used the wrong arguments
• How to convince your manager to implement QA?By showing that QA saves time and money
• How can I show upfront and afterwards that QA pays?Determining the moment of injection of defects and using the calculation model
• How much does it pay?An awful lot of money
• How to deal with agile and outsourcing?In agile and with outsourcing you also need QA