Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

42
QINETIQ PROPRIETARY QINETIQ PROPRIETARY The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016 Use of a Risk Maturity Model to Benchmark Project Health An exploration of the practical application of a risk audit model to benchmark project (and business) health and improve forecast cost and schedule out-turn 1 Mark Lee Principal P3M Consultant 26 May 2016

Transcript of Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

Page 1: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Use of a Risk Maturity Model to Benchmark Project Health An exploration of the practical application of a risk audit model to benchmark project (and business) health and improve forecast cost and schedule out-turn

1

Mark Lee

Principal P3M Consultant

26 May 2016

Page 2: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Agenda

• Introduce the principles of risk maturity assessment – Why a mature approach is important to project and programme cost and schedule control

• Explore and explain the QinetiQ Risk Maturity Model (QRMM) – Context and history: development of the model

– Construct and scope

– Why this particular model offers advantage to projects/programmes and businesses

– How it is applied in practice to benchmark projects/programmes and organisations

• Demonstrate the value of the QRMM in application – Summary case examples from Defence and Oil & Gas

2

Page 3: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Introducing QinetiQ

• Formed in 2001 from Defence Evaluation and Research Agency (DERA)

• FTSE 250 - £1.35bn market capitalisation

• 6,250 people worldwide, with over 1,000 specialists in weapons and testing

• Founding Member of The 5% Club – investing in graduates and apprentices

• 37 sites across the UK – from Cape Wrath to Shoeburyness

• 25 year Long Term Partnering Agreement (LTPA) with MOD, signed in 2006

• Empire Test Pilots’ School (ETPS) – training flight test professionals for over 70 years

• More than 1,500 patents granted

3

Page 4: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Advisory Services – P3 Consultancy

• We are QinetiQ’s leading Cost, Risk and P3 capability

• We provide independent, impartial advice to governments and industry

• We work in Land, Air, Maritime and Joint Domains

• We deliver innovation, supporting complex and often leading edge research and delivery programmes for international Defence and Commercial clients

• Our products and services include – Our QinetiQ proprietary P3 tools: QRMM, CEHC and FACET

– Project Management Office services and support, risk management, cost management, financial forecasting, resource management, stakeholder mapping and engagement, benefits mapping, business and P3 processes and governance

4

We enable our clients to make better decisions, based on good evidence and robust analysis, using a variety of proven tools and techniques

Page 5: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Risk Maturity Principles Why a mature risk management approach is important to schedule and cost control

5

Page 6: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Why is P3 Risk Maturity Important? Examples of ‘Failed’ Commercial Projects

Thermae Bath Spa

Forecast: £13m and 2002 opening

Actual: £45m and 2006 opening

Scottish Parliament

Forecast: £10-40m and 2001 opening

Actual: £414m and 2004 opening

6

Source: Wikimedia.org

Page 7: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Control of Risk Management Maturity Schedule Impact – QinetiQ Analysis of Historical NAO Data

Current Schedule Performance vs Original Forecast of MOD Top 20 Major Projects

41%

46%

114%

47%

32%

51%

63%

54% 53%

42%

75%

8%10%

0% 0% 0%

11%

0%0%

20%

40%

60%

80%

100%

120%

Typhoon (

Nov-8

7)

Sting R

ay (

May-9

5)

Nim

rod M

RA

4 (

Jul-96)

Astu

te C

lass S

ub (

Mar-

97)

A400M

(M

ay-0

0)

Type 4

5 D

estr

oyer

(Jul-

00)

Support

Vehic

le (

Nov-0

1)

NG

LA

AW

(M

ay-0

2)

Terr

ier

(Jul-02)

Naval E

HF

/SH

F S

at

Com

ms (

Aug-0

3)

Sooth

sayer

(Aug-0

3)

MT

AD

S (

Sep-0

4)

Watc

hkeeper

(Jul-05)

Falc

on (

Mar-

06)

Merlin

(M

ar-

06)

Futu

re L

ynx (

Jun-0

6)

Advanced J

et

Tra

iner

(Aug-0

6)

Typhoon F

utu

re

Capabili

ty (

Jan-0

7)

Sc

he

du

le o

verr

un

as %

of

ori

gin

al fo

reca

st

Risk Maturity Uncontrolled Risk Maturity @ Level 3+

Forecast schedule

overrun calculated from

the summary of post-

Main Gate projects in

NAO Major Projects

Reports

Many factors affect

projects, but those with

risk maturity applied at

all CADMID stages are

more aware of issues and

have mitigations in place

to respond to those risks

Major projects from the

NAO reports with risk

maturity applied are

statistically less likely to

experience schedule

overruns

Schedule Performance vs Original Forecast of MOD Major Projects

Source data: NAO Major Projects Reports

Average schedule over-run = 56%

Average schedule over-run = 4%

7

Page 8: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Control of Risk Management Maturity Cost Impact – QinetiQ Analysis of Historical NAO Data

Forecast cost overrun

calculated from the

summary of post-Main

Gate projects in NAO

Major Projects Reports

Projects with Risk

Maturity applied

experience less budget

volatility (overspend or

underspend), compared

with projects whose level

of risk maturity is

uncontrolled

Current Schedule Performance vs Original Forecast of MOD Top 20 Major Projects

-21%

28%

48%

0%

29%

-7%

-18%

6%

-26%

42%

-7%

-1%

-5%

-1%

1%

-5%

-2%

-30%

-20%

-10%

0%

10%

20%

30%

40%

50%

Typhoon (

Nov-8

7)

Stin

g R

ay (

Ma

y-9

5)

Nim

rod M

RA

4 (

Jul-96)

Astu

te C

lass S

ub (

Ma

r-

97)

A400M

(M

ay-0

0)

Type 4

5 D

estr

oyer

(Jul-

00)

Support

Vehic

le (

Nov-0

1)

NG

LA

AW

(M

ay-0

2)

Te

rrie

r (J

ul-02)

Naval E

HF

/SH

F S

at

Com

ms (

Aug-0

3)

Sooth

sayer

(Aug-0

3)

MT

AD

S (

Sep-0

4)

Watc

hkeeper

(Jul-05)

Fa

lcon (

Ma

r-06)

Me

rlin

(M

ar-

06)

Fu

ture

Lynx (

Jun-0

6)

Advanced J

et

Tra

iner

(Aug-0

6)

Typhoon F

utu

re

Capabili

ty (

Jan-0

7)

Cu

rre

nt

ov

ers

pe

nd

as %

of

ori

gin

al fo

reca

st

Risk Maturity Uncontrolled Risk Maturity @ Level 3+

Budget Performance vs Original Forecast of MOD Major Projects

Source data: NAO Major Projects Reports

Average cost overspend = 8%

Average cost underspend = 3%

8

Page 9: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Why is P3 Risk Maturity Important? Audit Evidence from Defence

9

Source data: NAO Major Projects Report 2013

700

600

500

400

300

200

100

0

£m

Ships Combat Air Submarines Helicopters Air Support Information Land ISTAR Weapons Systems Support Equipment

0 0

Defence – average cost growth across all 73 projects in the ships, combat air and submarine sectors

Page 10: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Why is P3 Risk and Cost Maturity Important? Avoiding the News Headlines!

10

“BBC Chief Technology Officer sacked over failed £100m Digital Media Initiative”

“A major £12m scheme to digitise patient records has so far failed to deliver and is now

expected to cost the Health Department double its original budget, according to the Public

Accounts Committee”

“Canadian provincial health officials fire prime contractor after the firm missed 3

years of deadlines and failed to deliver the province’s flagship online medical registry”

“Sacked bursar loses claim for unfair dismissal over school’s £2.5m expansion project”

“Environment Minister demoted after bungled $2.4bn home insulation fiasco”

Source: Pressgazette.co.uk

Page 11: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Why is P3 Risk and Cost Maturity Important? Avoiding the News Headlines!

11

Page 12: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Why is P3 Risk and Cost Maturity Important? Primary Causes of Project Failure

12

“Opportunities and risks were not defined” 31%

Page 13: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Why Does This Happen? Dealing with Uncertainty

All projects and programmes have uncertainties and changing variables, arising from …

• Budget changes

• Schedule changes

• Requirement changes

• Omissions and errors

• Failure to tackle risk at source

• Things that “just go wrong”

“…… because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns - the ones we don't know we

don't know” Donald Rumsfeld

Source: Wikimedia.org 13

Page 14: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Why Does This Happen? Dealing with Uncertainty – Defence Reform and QDC

Page 15: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Why Does This Happen? Dealing with Uncertainty – Defence Reform and QDC

Page 16: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Confidence … Driven by Uncertainty

Confidence is a major factor in human behaviour … and decision-making

• Sub-prime mortgage crisis – Missouri house prices crashed by 90%

• Global recession - US companies began storing cash

• Scottish Referendum – investors sold Sterling

• ‘Brexit’ – value of Sterling and FTSE depressed(?), investments on hold

Wouldn’t it be useful if we could have better confidence in our ability to manage uncertainty…?

Source: Edinburghnews.com 16

Source: Dilbert.com

Page 17: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

The QRMM Construct, context, scope, benefits and application of the QinetiQ Risk Maturity Model

17

Page 18: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QinetiQ Risk Maturity Model (QRMM) A Brief History …

• Developed by QinetiQ (1999) to improve confidence by objectively assessing risk management maturity

• Referenced to and compliant with

― MOD’s Acquisition System Guidance (ASG)

― Project risk management best practice – APM Project Risk Analysis and Management (PRAM) Guide

― Combined Code (‘Turnbull Guidance’) for UK Corporate Governance – Financial Reporting Council

• Proven capability and value in application over 15 years

― Circa £100bn of Defence projects/programmes (across all domains) assessed … and counting

― Also deployed in Oil & Gas (FTSE 100 multi-national), Rail and Manufacturing

• QinetiQ analysis of NAO Major Projects Reports has indicated that RMM can

― Increase confidence in project success through improved cost/schedule adherence

― Deliver forecast improvement in schedule and cost out-turn on major projects and programmes

18

Page 19: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM as an Enabler to Better P3 Delivery

• Assesses and benchmarks the quality and consistency of risk management implementation

• Improves confidence in the ability to predict and deliver against schedule and cost

• Establishes an independent, objective and evidence-based baseline measure of risk maturity

• Identifies strengths and weaknesses in risk management process and its enablers

• Supports formulation of a prioritised ‘roadmap’ of improvement actions against the baseline

• Supports identification of common issues across projects/programmes, to help tackle risk at source

• Facilitates sharing of good practice within and across business units

• Builds confidence in the quality of underpinning data (e.g. for Business Cases)

• Scalable: applicable at all levels, at all points in the project/business lifecycle

• Can be used to support supplier assessment

19

Page 20: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Comparison of Risk Management Maturity Models QinetiQ Analysis (2013) - Summary

20

Defence

Heritage

Risk-Specific

Maturity

Model

Implementation

Guidance

Available

Implementable

Risk Maturity

Model Available

Question and

Anwer Set

Developed and

Available

Questions

Mapped to

Maturity

Levels

Analysis and

Reporting

Tools

Available

Improvement

Roadmap

Guidance and/or

Tool Available

Number of

Questions

Implementable

Without

Investment

Management of

Risk (MOR®)

Maturity Model

0

HM Treasury Risk

Management

Assessment

Framework

(RMAF)

38

Cabinet Office

Portfolio,

Programme and

Project

Management

Maturity Model

(P3M3)

9

QinetiQ Risk

Maturity Model

(QRMM)

50

Page 21: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM Construct – Scope and Scoring

• A maturity framework covering 6 risk management perspectives

— Risk Identification; Risk Analysis; Risk Mitigation

— Project Management; Stakeholders; Culture

• Each perspective is scored within the algorithm, at one of 4 levels

— Level 1 = Naive (process design or application flawed and probably not adding value)

— Level 2 = Novice (some value-add, but weakness in process design or implementation)

— Level 3 = Normalised (formalised process, implemented systematically and adding value)

— Level 4 = Natural (applied at strategic level in driving objectives and optimising outcomes)

• Level 3 is an acceptable score (complex programmes/projects may aspire to Level 4)

21

Page 22: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM Construct – Top Level Output

Risk Maturity Assessment

Stakeholders Risk

Identification

Risk Analysis Risk

Mitigation

Project

Management

Culture

Current

Maturity

Potential

Improvement

Level 4 -

Natural

Level 3 -

Normalised

Level 2 -

Novice

Level 1 -

Naive

“An organisation is only as strong as its weakest element”

22

Page 23: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM – Example Characteristics of Maturity Levels

23

Level Descriptor Formal Definition Example Characteristics

1 Naïve

Although a risk management process

may have been initiated, its design or

application is fundamentally flawed

At this level, it is likely that the

process does not add value

• Low level of understanding of risk management principles and application

• No formalised process for managing risk, or ad-hoc and/or poorly applied

• Fundamental flaw in the design or application of the risk process

• Critical risk process components are likely to have lapsed into disuse

2 Novice

The risk management process

influences decisions taken by the

organisation in a way that is likely to

lead to improvements in performance

as measured against its objectives

However, whilst the process may add

value, weaknesses with either the

process design or its implementation

result in significant benefits being

unrealised

• At least a ‘light’ application of the risk process

• The process itself is standardised and followed with robustness

• A formal risk management process that follows best practice recently initiated

• Larger organisation where process application may be an issue

• Larger organisation where issues of process design may be difficult to correct

Page 24: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM – Example Characteristics of Maturity Levels

24

Level Descriptor Formal Definition Example Characteristics

3 Normalised

The risk management process is

formalised and implemented

systematically

Value is added by implementing

effective management responses to

significant sources of uncertainty that

could affect the achievement of

business objectives

• Disciplined risk process application across the whole organisation

• Risk register underpins routine reviews of the implications of risk

• The effectiveness and implementation of responses is designed to manage risk

• Risks clarify all relevant and significant sources of uncertainty

• The risk register contains the right risks (and they continue to be the right risks)

• Risks are managed by the right risk owners, with engagement of stakeholders

• Appropriate and sound methods are used to select and prioritise risks for review

• Risk management is being used to support achievement of objectives

4 Natural

The risk management process leads

to the selection of risk-efficient

strategic choices when setting

business objectives and choosing

between options for solutions or

delivery

Sources of uncertainty that could

affect the achievement of objectives

are managed systematically within

the context of an organisational

culture that is conducive to optimising

business outcomes

• Risk is managed from a strategic (not just tactical) perspective

• Risk responses are likely to be executed from Sponsor level

• Risk management is implicit, with over-reliance on the Probability Impact Matrix

and an integrated risk register and Monte-Carlo simulation toolset avoided

Page 25: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM in Application Empirical Assessment Process

Review Documentary

Evidence Audit

Conduct RMA Workshop

Analyse and Report Results

Implement Improvement Plan

RMA Benchmark

(Level 1 to 4)

Periodic re-assessment against current benchmark

25

Page 26: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM in Application Risk Maturity Assessment Workshop – Use of Electronic Voting

• Well-established method of group decision support

• Used to elicit opinion

— Primarily interested in the reasons for the votes

— Votes are anonymous

• Provides a framework to consider arguments before expressing opinion

• Discussion is limited to clarification before voting

— Understand the question and supporting narrative, in relation to risk maturity

— Understand how the question and context relates to the project under assessment

• Divergence in votes may provide additional insight: an opportunity for discussion

Source: Wikipedia.org

26

Page 27: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM in Application Risk Maturity Assessment Workshop – Use of Electronic Voting

• Question is posed

— Consider the question and context ... and vote

• Facilitated discussion

— Voting results presented

— Salient points recorded for analysis/reporting

— Record the consensus view (score, any narrative)

• Re-vote (as necessary)

• Why use Delphi Technique?

— Decisions from a structured group facilitation are more accurate

— In this scenario, exploring voting rationale can aid interpretation and understanding

Source: Powervote.com

27

Page 28: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM in Application Example Analysis Outputs

28

Page 29: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM in Application Example Analysis Outputs

29

Page 30: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM in Application Example Analysis Outputs

30

Page 31: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM in Application Example Analysis Outputs

31

Page 32: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM in Application Improvement Roadmap Implementation – Example Status Check

32

Recommendation Title Theme Value Complexity Status Comments

R3.1 Risk Descriptions Process Alignment High Low Ongoing Will be satisfied through RMP

R3.3 Risk Roles and Responsibilities in RMP Process Alignment High Low Ongoing Will be satisfied through RMP

R3.4 Briefing of Risk Roles and Responsibilities Stakeholder Engagement High Low Ongoing Briefed through RMP and risk training

R4.4 Use of Risk Reduction Strategies Process Alignment High Low Ongoing Will be satisfied through RMP

R5.2 DOR Matrix Roles and Responsibilities High Low Ongoing Will be satisfied through RMP

R1.2 Escalation Criteria Process Alignment High Medium Ongoing Will be satisfied through RMP

R3.10 Secondary Risk Effects in RMP Process Alignment High Medium Ongoing Will be satisfied through RMP

R3.11 Training on Secondary Risks Stakeholder Engagement High Medium Not started To be covered in risk training course

R4.10 Risk Response Metric Audit and Assurance Medium Low Complete Already being implemented and ought to be reflected in RMP

R4.11 XXXXX Risk Board TORs Audit and Assurance Medium Low Complete TORs have been drafted and will be included in RMP

R4.2 Review of Risks for Fallback Decision Points Audit and Assurance Medium Medium Not started Suggest this is part of risk audit, planned for March 2016

R2.12 Risk Identification Metric Process Alignment Low Low Deleted Not required - risk status checked pre-Risk Board

Recommendation Title Theme Value Complexity Status Comments

R2.3 Top-Down Risk Identification in RMP Process Alignment High Low Not started Suggest this is written into RMP

R2.14 Issue Management Process Alignment High Low Not started Suggest this is written into RMP

R1.13 Portfolio Office Risk Support to DLODs Resourcing High Medium Deleted XXXXX Programme has determined this isn't needed

R2.1 Assumptions Risk Identification High Medium Not started Consider bringing MDAL and 3OA under Programme control

R2.2 Top-Down Risk Identification Risk Identification High Medium Not started Risk identification workshop scheduled for 4 November 2015

R2.13 Milestone Schedule Reviews Process Alignment High Medium Not started

R4.6 Use of Risk Tolerance Threshold Process Alignment High Medium Not started

CATEGORY 1 RECOMMENDATIONS

CATEGORY 2 RECOMMENDATIONS

Page 33: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Case Examples Examples of the QRMM in application to benchmark and improve project and programme performance in Defence and Oil & Gas

33

Page 34: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM in Application Case Example 1 MOD Project Team – Analysis Summary and Plan

34

Perspective Number of Projects at Each Level – Feb ‘14

Level 2 Level 3 Level 4

Stakeholders √ √ √ √

Risk Identification √ √ √ √

Risk Analysis √ √ √ √

Risk Responses √ √ √ √

Project Management √ √ √ √

Culture √ √ √ √

• February 2014 forward improvement plans, focused to achieve

— Project A: from high L2 (almost L3) to weak L3 in 3 months, consolidating to a firm L3

— Project B: from high L3 to a weak L4, consolidating to a firm L4 through secondary actions

— Project C: from weak L3 (with risk of slipping back to L2) to a firm L3

— Project D: from weak L4 (risk of slipping back to L3) to a firm L4

A good example of where focused MOD effort, and periodic RMA, can enhance risk execution

Page 35: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM in Application Case Example 2 MOD 1* Portfolio – Analysis Summary

Current Schedule Performance vs Original Forecast of MOD Top 20 Major Projects

-21%

28%

48%

0%

29%

-7%

-18%

6%

-26%

42%

-7%

-1%

-5%

-1%

1%

-5%

-2%

-30%

-20%

-10%

0%

10%

20%

30%

40%

50%

Typhoon (

Nov-8

7)

Stin

g R

ay (

Ma

y-9

5)

Nim

rod M

RA

4 (

Jul-96)

Astu

te C

lass S

ub (

Ma

r-

97)

A400M

(M

ay-0

0)

Type 4

5 D

estr

oyer

(Jul-

00)

Support

Vehic

le (

Nov-0

1)

NG

LA

AW

(M

ay-0

2)

Te

rrie

r (J

ul-02)

Naval E

HF

/SH

F S

at

Com

ms (

Aug-0

3)

Sooth

sayer

(Aug-0

3)

MT

AD

S (

Sep-0

4)

Watc

hkeeper

(Jul-05)

Fa

lcon (

Ma

r-06)

Me

rlin

(M

ar-

06)

Fu

ture

Lynx (

Jun-0

6)

Advanced J

et

Tra

iner

(Aug-0

6)

Typhoon F

utu

re

Capabili

ty (

Jan-0

7)

Cu

rren

t o

vers

pen

d a

s %

of

ori

gin

al fo

recast

Risk Maturity Uncontrolled Risk Maturity @ Level 3+

Stakeholders

Risk Identif ication

Risk Analysis

Risk Responses

Project Management

Culture

Category 1 Category 2

Level 1 Level 2 Level 3 Level 4

Level of maturity after

• Level 1 across all 6 perspectives – the worst ever RMA score recorded by QinetiQ!

• Risk improvement roadmap established to target

— Level 2 in 3 months (22 actions)

— Level 3 in a further 9 months (16 actions)

Source: MoD

35

Page 36: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM in Application Case Example 2 MOD 1* Portfolio – Outcomes

Source: MoD • Improvements were not implemented, due to

— Lack of capacity within MOD to implement the plan

— Conflicting demands and changing priorities

— Ongoing organisational uncertainty

— Realisation that implementation of improvements at 1* level would be insufficient

• QinetiQ was then requested to

— Formulate a transformation programme covering the 2* group (4 x 1* units)

— Develop and roll-out improvements to enhance risk and cost management and control

• What happened next?

• MOD secured stakeholder buy-in to implement the ~18 month transformation programme

• In-FY underspend was secured to fund the initial phase

• Other areas developed an interest in improving their risk and cost maturity

A good example of where a localised RMA can uncover a need for wider risk/cost improvement

36

Page 37: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM in Application Case Example 2 Example Transformation Programme – Benefits Realisation

Forecast Risk Maturity

(left-hand scale)

Phase 1 Phase 3 Phase 2

Potential for Schedule/Cost Overrun

(right-hand scale)

Naïve (Level 1)

Risk process flawed No real value-add

Novice (Level 2)

Risk process influencing decisions Risk process adding value Improving performance against objectives Some process/implementation weaknesses Potential for significant unrealised benefits

Normalised (Level 3) Risk process formalised Process implemented systematically Effective risk responses executed Sources of uncertainty under control Significant value-add

Natural (Level 4) Risk process informing

strategic choices Sources of uncertainty

managed systematically Risk culture conducive to

maximising outcomes

Level 1

Level 4

Level 3

Level 2

Ris

k M

atu

rity

Leve

l

0-10%

100%+

60-99%

11-60%

Sch

ed

ule

/Co

st O

verr

un

(a

s a

% o

f b

ase

line

sch

ed

ule

/bu

dge

t)

37

Page 38: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM in Application Case Example 3 MOD HQ Programme - Results

38

• A high performing programme with excellent structural and cultural characteristics

• Key shortfalls noted in risk analysis due to personnel failing to record this in the risk tool

• Risk improvement roadmap established to target

— Clear consolidation at Level 3 (49 actions), a significant number of which were RMP shortfalls

— Longer-term improvement to Level 4 (45 actions)

Page 39: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM in Application Case Example 3 MOD HQ Programme - Outcomes

39

• What happened next?

— The programme enhanced its internal capability through the recruitment of additional P3M resource

— QinetiQ’s contract was extended to upskill the new P3M team and advise on improvement execution

— The programme targeted a further calibration of its risk management maturity in circa 6 months

A good example where even a risk-robust programme can benefit from targeted risk improvement

• QinetiQ was advised of another programme (under the same SRO) which required calibration

− QinetiQ briefed the benefits of conducting a formal risk maturity calibration to the other PMO

− The (new) PMO Head of Risk is introducing changes, having recognised some shortfalls

− QinetiQ will be contracted to undertake a risk maturity assessment in circa 3-6 months

Page 40: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

QRMM in Application Case Example 4 Oil & Gas FTSE 100 Multinational – Analysis and Outcomes

40

• FTSE 100 Oil & Gas multi-national approached QinetiQ to pilot an RMA on a UK project

• QinetiQ amended the RMA framework Q&A set to reflect O&G-specific language

− Underlying model and algorithms were unchanged

• The pilot was conducted on the UK project

− Identified that lack of risk disclosure from the JV partner was a significant threat

− RMA was extremely well received by the client organisation

− Actions to address the shortfalls were apparently not progressed

− There were serious repercussions for both JV partners

A good example where failure to address risk maturity shortfalls can impact business health

• What happened next?

− QinetiQ undertook a further RMA on an operation in Asia, on completion of the UK pilot

− The Asia RMA identified significant pockets of good practice to share across the company

− QinetiQ was requested to develop a new corporate Cost & Schedule Risk Analysis standard

Source: Wikipedia.org

Page 41: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

QINETIQ PROPRIETARY

QINETIQ PROPRIETARY

The document and information contained herein is proprietary information of QinetiQ Limited and shall not be disclosed or reproduced without the prior authorisation of QinetiQ Limited. © QinetiQ Limited 2016

Summary

1. Introduce the principles and importance of risk maturity assessment

− There is inherent uncertainty in all projects, programmes and businesses

− Formalised risk management help us understand and respond to uncertainty

− Control of risk maturity is a key enabler to good project and programme management

2. Explore QinetiQ Risk Maturity Model (QRMM)

− Audits and benchmarks project and programme health and focuses P3M improvement initiatives

− Enhances confidence in the likelihood of an out-turn to schedule and within budget

− Enables us to more confidently establish our risk appetite and inform strategic choice

3. Demonstrate value of QRMM in application

− Case 1 – focused improvement and periodic re-assessment enhanced project control

− Case 2 – localised RMA can trigger wider imperative to enhance risk management

− Case 3 – even projects/programmes which appear mature can benefit from targeted intervention

− Case 4 – failure to address risk maturity improvement can impact business health

Source: Pressgazette.co.uk

Page 42: Use of Risk Maturity Model to Benchmark Project Health, 26 May 2016

www.QinetiQ.com

FOLLOW US ON:

linkedin.com /company/qinetiq_2

youtube.com /user/QinetiQGroup

twitter.com /QinetiQ

facebook.com /qinetiqgroup