Slide 1

Test Assurance – Ensuring Stakeholders get What They Want Paul GerrardGerrard ConsultingPO Box 347MaidenheadBerkshireSL6 2GU UK

e: paul@gerrardconsulting.comw: http://gerrardconsulting.comt: 01628 639173

Slide 2

Why Test Assurance? Most projects need to be kept ‘honest’ Sponsors and business- Need someone ‘on the inside’ on their side- Need translations from technical, evasive

spin to lay persons’ language Testing is rarely done well - project

needs support from someone capable and bias-free

Risks arising need someone to prod the project into action (who else will do this?)

Slide 3

Objectives of the role Primary objective: Support- To provide advice, leadership and direction to the

project team to improve the effectiveness of testing and in particular test reporting

Secondary objective: Assurance- To provide independent assurance to the

management board on the thoroughness, completeness and quality of testing

Optional Excluded from scope is any responsibility for the

delivery of testing - specifically excluded to avoid any conflicts of interest

Slide 4

Support Information Provision- Including specific points of expertise or experience

Decision Support- To support the decision making process, where points of

technique, documentation, reporting or appropriate (not always 'best') practice may be suggested to the team

Advisory- To advise on process for traceability, coverage, sign-off

for assurance purposes Risk Awareness- To identify risks that must be addressed by test phases

Review support- Critical reviews of coverage, documentation, plans,

results while under development.

Slide 5

Assurance Test Audit- Independent audit of test records and

process to include checks for traceability, coverage, consistency, accuracy etc.

Test Phase report- Phase report to document the audit,

identify areas for correction, interpretation Sign-Off- See later.

Slide 6

Assessment of Testing as a whole Can the system can be used in a realistic business

environment, processing realistic data? Does the system meets performance, availability

and reliability objectives? Can the system can be supported by operations

staff so that exceptions can be handled? Can the system can be installed, started, stopped,

backed up and restored from a range of failure scenarios?

Have all outstanding product risks of concern have been addressed

Have each of the test phases achieved their target level of coverage so that the project can be confident in their assessment?

Slide 7

Process and Coverage Assurance Is the process sound; will it achieve the

test phase objectives? Was the process followed, exit criteria met,

waived appropriately? Did the appropriate, responsibility people

perform sign-offs? What process was used to design tests;

were coverage targets set? Were the targets met in test planning and

execution?

Slide 8

Product Risk Assurance What risks drove the test planning,

prioritisation? Were risks addressed through the

testing? Are there risks, not tested that are

outstanding?

Slide 10

Interventions - Assurance

Test Assurance Notes: Where anomalies or uncertainties in the planning, scope or approach to testing are detected, or where a new risk to be addressed by testing appears, an Assurance Note is a challenge to the project to clarify the purpose of that aspect of the project.

Test Audit: Independent audit of test records and process to include checks for traceability, coverage, consistency, accuracy etc.

Test Phase report: Phase report to document the audit, identify areas for correction, interpretation.

Sign-Off: see slides that define precisely what Assurance Sign-off means (and doesn't mean).

Slide 12

Sign-Off by stakeholders and other participants Indicates that the signatories:- Have read and understood it what they have signed- Have approved a deliverable, outcome or process- Made their decision in a transparent fashion- Commit to a defined course of action – usually to ‘accept’

and/or implement- Have reached a consensus view- Have based their decision on their (or delegated)

judgements- Agree that their views have been taken into consideration

Signatories must be recognised authorities Sign-off is usually irreversible – once signed, cannot

be undone Not granted lightly.

Slide 13

Test Assurance Sign-Off (is different) The test approach, plan, specifications, scripts,

logs, incident reports have been reviewed and test lead has been interviewed/consulted

The following have been ‘assured’:- Objectives and acceptance/exit criteria- The test design approach enables objectives to be met- Coverage target(s) ensure that enough information will

be gathered to make the acceptance/exit decision- The exit criteria have been met or…

The remaining anomalies have been waived by the business And… the right people have been involved

- The test records indicate that the phase process has been followed and are a true reflection.

Slide 14

The politics of assurance Who will pay for the assurance function?- Would your department employ a critic?

Can the Assurance Manager stay independent?- Or will he go native with the development group?

Can the Assurance Manager speak the language of business?- Are you a techy at heart – unable to see beyond code, bugs,

incident reports? If you made a recommendation, would the

development organisation attack you, or the problem? Assurance Managers sign-off – do they take

responsibility for the release? Do Test Assurance Management skills exist yet? If you ‘did’ assurance, would it be a career limiting

move?