Getting Business Value out of (Test) Metrics Richard Terry (UK) [email protected] Rob Baarda...
-
Upload
loreen-paul -
Category
Documents
-
view
213 -
download
0
Transcript of Getting Business Value out of (Test) Metrics Richard Terry (UK) [email protected] Rob Baarda...
![Page 2: Getting Business Value out of (Test) Metrics Richard Terry (UK) richard.terry@sogeti.com Rob Baarda (NL)](https://reader036.fdocuments.us/reader036/viewer/2022082817/56649dc75503460f94abc68f/html5/thumbnails/2.jpg)
What we will do / Structure
• Subjects:>Test Metric goals / stakeholders>Which metrics; some demo>Process around metrics>How to apply metrics / pittfalls>Wrap-up
• For each subject: intro, discussion• Discussion
>hand raise>hand raise with comments>open
![Page 3: Getting Business Value out of (Test) Metrics Richard Terry (UK) richard.terry@sogeti.com Rob Baarda (NL)](https://reader036.fdocuments.us/reader036/viewer/2022082817/56649dc75503460f94abc68f/html5/thumbnails/3.jpg)
Metric Goals / Stakeholders
• Hand raise only1. Who is using project metrics?2. Who is using test metrics?3. Who would like to use test metrics?
• Some opinions please!• Goals / Stakeholders for test
metrics? OPEN!• Your votes please
![Page 4: Getting Business Value out of (Test) Metrics Richard Terry (UK) richard.terry@sogeti.com Rob Baarda (NL)](https://reader036.fdocuments.us/reader036/viewer/2022082817/56649dc75503460f94abc68f/html5/thumbnails/4.jpg)
Which test metrics?
Test basis
Test object
Test Execution Defects
Repair
Production
Specifying test
cases / scripts Test cases/
scripts
Test Process
Size test basis
Size test object # defects in
test object# defects in
production
For each process:# hours effortlead time
# test cases
# = number of
# defects in test basis
# repair rounds
![Page 5: Getting Business Value out of (Test) Metrics Richard Terry (UK) richard.terry@sogeti.com Rob Baarda (NL)](https://reader036.fdocuments.us/reader036/viewer/2022082817/56649dc75503460f94abc68f/html5/thumbnails/5.jpg)
Deductible metrics
• Effort = Size /# hours• Productivity = # test cases / # hours• Efficiency =
# defects / (# hours or # test cases)> Specification> Test execution> Retest of repaired defects
• DDP Defect Detection Percentage (Europe)DRE Defect Removal Efficiency (USA)
• Defect injection rate for rework• Damage prevented?• More metrics:
> Availability of test environments> OPEN
![Page 6: Getting Business Value out of (Test) Metrics Richard Terry (UK) richard.terry@sogeti.com Rob Baarda (NL)](https://reader036.fdocuments.us/reader036/viewer/2022082817/56649dc75503460f94abc68f/html5/thumbnails/6.jpg)
Dutch test metrics experiences
• Dutch initiative to gather test metrics
• Parties involved>NESMA
Netherlands Software Metrics Association
>Testnet, including SogetiDutch Testing community
>LaQuSOLaboratory for Software QualityUniversities Eindhoven & Nijmegen
Some slides with data from the database will follow in the presentation, not in the hand-out, due to a lack of statistical evidence
![Page 7: Getting Business Value out of (Test) Metrics Richard Terry (UK) richard.terry@sogeti.com Rob Baarda (NL)](https://reader036.fdocuments.us/reader036/viewer/2022082817/56649dc75503460f94abc68f/html5/thumbnails/7.jpg)
Processes around metrics
• Collection in a project> Embedded in daily work> Weekly summarisation> Sanity checks> Cost: about 2% project budget
• Distribution• For a benchmark on the level of:
> Project releases> Organisation> Country> International: ISBSG
International Software Benchmarking Standards Group
![Page 8: Getting Business Value out of (Test) Metrics Richard Terry (UK) richard.terry@sogeti.com Rob Baarda (NL)](https://reader036.fdocuments.us/reader036/viewer/2022082817/56649dc75503460f94abc68f/html5/thumbnails/8.jpg)
To what extent?
• OPEN Discussion about>Accuracy of definitions>Number of types of defects>Is a batch test case the same as an
online test case?>Only testing of functionality or also
security, performance, usability>How to include regression testing?>Predictive value
average (mean), median, standard deviation, correlations with?Prediction model needed?
![Page 9: Getting Business Value out of (Test) Metrics Richard Terry (UK) richard.terry@sogeti.com Rob Baarda (NL)](https://reader036.fdocuments.us/reader036/viewer/2022082817/56649dc75503460f94abc68f/html5/thumbnails/9.jpg)
To apply test metrics
• Project>Estimating Testing New development>Estimating Testing Maintenance release>Controlling the test project
• Benchmark>Measure the effectiveness of the testing,
Improve inside / outside the test process?
• PITTFALLS?
![Page 10: Getting Business Value out of (Test) Metrics Richard Terry (UK) richard.terry@sogeti.com Rob Baarda (NL)](https://reader036.fdocuments.us/reader036/viewer/2022082817/56649dc75503460f94abc68f/html5/thumbnails/10.jpg)
Make a judgement based with metrics?
• Of person production/ quality of work>OPEN Discussion>Some Dutch effects
• Of Service Level Agreement>What to measure and how to judge
![Page 11: Getting Business Value out of (Test) Metrics Richard Terry (UK) richard.terry@sogeti.com Rob Baarda (NL)](https://reader036.fdocuments.us/reader036/viewer/2022082817/56649dc75503460f94abc68f/html5/thumbnails/11.jpg)
Wrap-up
• Business value out of metricsHand raise> Who would like to use test metrics?
• Changed opinions> Comments please (- + ,+ -)
• Reading: TMap Next> Chapter 13 Goal-Question-Metrics for test> Chapter 11 for data
• End of 2008: www.isbsg.orgParticipate? Leave your business card