Risk-based testing - a common language for project stakeholders

© 2003 Insight Consulting Ltd. Version 1.01

Risk-based testing - a commonlanguage for project stakeholders

Mike RussellInsight Consulting Ltd.

e-mail: [email protected]: http://www.insight.ie

Insight Consulting Ltd.114 GranitefieldDun LaoghaireCo. DublinIreland


Learning objectives

To identify perception and communication problems betweentesting and other project stakeholders

To discuss how risk-based testing can help solve these problemsand improve the (perceived) added value of testing

To outline how to implement improvements such as risk-basedtesting in a practical way


Agenda

Key challenges facing testingWhat is risk-based testing?How does it address the challenges?Getting started


Perceptions of testing

Testing

Software

System

Quality

Software

System

(Sign-off )


Perceptions of testing - 2

Project Stakeholders include customers, users, seniormanagement, project management, development, etc.

Squeeze on testingSkills required?The ‘sign-off problem’...

….lack of appreciation of ‘added value’ of software testingmorale and staff turnoverdifficult in justifying testing resourcesless effective testing


Origins of these perceptions

A lack of a clear understanding of the objectives and role oftestingA lack of involvement in influencing testingPoor visibility of testingNot understanding or being able to interpret the results of testingLinking testing to a narrow definition of quality that is notaligned with the primary objective of the project

Underlying theme = Poor communication


Agenda



Risk Management

Risk = Impact x Likelihood SEI model:

• identify• analyse• plan• track• control• communicate

Likelihood

Impact

Hi

Lo

Lo Hi

!


Three types of software risk

Project Riskresource constraints, external

interfaces, supplierrelationships, contract

restrictions

Process Riskvariances in planning and

estimation, shortfalls instaffing, failure to trackprogress, lack of quality

assurance and configurationmanagement

Primarily a managementresponsibility

Planning and the development processare the main issues here.

Product Riskbusiness critical features,

complexity, design and codequality, non-functional issues,

requirements instability

Requirements risks are the most significant risksreported in risk assessments.

Testers are mainlyconcerned withProduct Risk


Risk-based testing - key elements

Risk identification and analysisRisk-based testing strategy (test planning)Design, prioritisation and review of test cases (logicaldesign part of test specification - ‘what’)Prioritised execution of test cases on the basis of risk(execution)Risk-based tracking and controlRisk-based test reportingRisk-based decision on release


Risk-based reporting*

Progress through the test plan

today Plannedend

residual risksof releasing

TODAY

Res

idua

l Ris

ks

start

all risks‘open’ atthe start

* From ‘Risk-based reporting’ by Paul Gerrard


Benefit & objectives based test reporting

Open

Closed

Ris

ks

Open

Open

Closed

Closed

Open

Obj

ectiv

e

Obj

ectiv

e

Obj

ectiv

e

Obj

ectiv

e

Bene

fit

Bene

fit

Bene

fit

Bene

fit

Bene

fit

Benefits available for releaseO

bjec

tive

Bene

fit

Closed


Agenda



Improving the communication

Risk Management well established practice withmanagement and customersRisks can be quantifiedLanguage of risk is easily understood and embraced

risk is sexy (relatively!)Easier to solicit input from stakeholdersStatus and reporting easily understoodTesting becomes aligned with core project objectives,addressing risks and informing as benefits becomeavailable


Addressing the challenges

Role of testingto provide good clear test evidence on risk tostakeholders

Moment of involvementImproved visibility and influence

through number of interfaces and interactions andmeaningful (understandable) reporting


Addressing the challenges -2

… and better testing by using more knowledge to focustesting with available resources

All facilitated by improved communication through use of thecommon language of risk


Stakeholder benefitsCustomer/users

opportunity to influencebetter focus on their concerns

Project Managerbetter use of testing resourcesimproved collaboration between teamsuseful information for tracking and release decision

Developmentrisks useful input to improve design and developmentclearer understanding of development test responsibilities


Stakeholder benefits - 2

Testearly involvementmanage the squeeze on testingthe ‘sign-off problem’ disappears‘added value’ of testing more obvious (even quantifiable)and easier to communicate


Agenda



RBT sample improvement actionsHold a risk identification workshop with customer, projectmanagement, development and testPiggy-back on existing risk management process

strengthen link between product risks and testingPrioritise test cases on basis of known risks

get developers and users to reviewDevelop a risk-based testing strategy for system testEnhance weekly test reporting on basis on risks addressedand risks remainingRevise test schedule to run high risk tests first


Improvement actions - how?Simple action plan with prioritised actions, estimated effort, who andwhenPilotDefine the process step in 1 page (what to do) with 1-2 page proceduraldetail only if required (how to do it) and supporting templates/checklists

training should not be an objectiveobjective is communication/consensus and reminding people what todo and how to do it

Train/coach/facilitate as requiredTrack and measure resultsRe-plan next set of actions - iterate


Summary

The language of risk and risk-based testing provides not only for more effective testing but for improved communication and collaboration between testing and other project stakeholders. Justifying the ‘added value’ of testing becomes less of an issue.


References

www.insight.ie - Our website containing testing news, links, opinions, etc.www.evolutif.co.uk - Systeme Evolutif’s website containing a number ofarticles on risk-based E-Business testing

‘Risk-based E-Business Testing - Paul Gerrard and Neil Thompson,Artech House, ISBN: 1580533140; 1st edition (August 2002) - see alsowww.riskbasedtesting.com

www.stickyminds.com - SQE website containing a number of relatedarticles‘Making Process Improvement Work: A Concise Action Guide forSoftware Managers and Practitioners’, Neil S. Potter, Mary E. Sakry,Addison-Wesley Pub Co; ISBN: 0201775778, March 2002. [see alsowww.processgroup.com for articles and templates on Practical SPI].

Risk-based testing - a common language for project stakeholders

Documents

Transcript of Risk-based testing - a common language for project stakeholders