RISK-2510

(RISK-2510) Modelling Realistic Outcomes using

Integrated Cost and Schedule Risk Analysis

Colin H. Cropley

RISK-2510

PLEASE USE MICROPHONE FOR ALL QUESTIONS AND COMMENTS!

2

RISK-2510

BIO of Colin Cropley

• Colin Cropley is Managing Director of Risk Integration Management Pty Ltd (RIMPL), an Australian company focused on large project quantitative project risk analysis

• A chemical engineer with over 40 years’ experience in project management, controls & risk management

• He has conducted risk management processes, schedule and cost risk analyses and training for many major companies since 2003

• He was Chairman of his state Primavera Users Group from 1997 to 2009 and has guest lectured in post-graduate project management courses since 1992

• He is a member of AACE International, the Australian Cost Engineering & Risk Engineering Societies and PMI

• Colin is convinced by the overwhelming scientific evidence that climate change is accelerating and that we have to change our behaviour urgently to avert the most serious consequences

3

RISK-2510

OUTLINE OF PRESENTATION PURPOSES OF THE PAPER

4

RISK-2510

Presentation Outline

• Purposes of the paper • Nature of this Integrated Cost & Schedule Risk Analysis

methodology (IRA) • When it is most effective • Extend IRA to IRRA to model operation of project assets • Example IRRA model: Generic Tollway PPP • Strengths and weaknesses of IRA/IRRA methodologies • How well IRA methodology addresses RP40R-08 criteria

for quantifying project risk and contingencies

5

RISK-2510

Purposes of the Paper

• To describe this Integrated cost & schedule Risk Analysis methodology (“IRA”)

• To relate it to the requirements of this project risk quantification methodologies conference track : – Referencing AACE® Recommended Practices – Contexts in which IRA has greatest advantages – Strengths and weaknesses and how well it addresses

RP 40R-08 Contingency Estimating – General Principles • To extend the IRA methodology beyond project startup to

cover the risks and uncertainties of asset operation • To illustrate the extended methodology modelling a Build Own

and Operate tollway project and using probabilistic NPVs and IRRs, forecast investment resilience

6

RISK-2510

WHAT IS THIS VERSION OF INTEGRATED COST & SCHEDULE RISK ANALYSIS

(IRA)? - COMPONENTS AND PROCESS

7

RISK-2510

IRA Components

• The IRA Methodology can be defined as “the simultaneous Monte Carlo Simulation (MCS) analysis of cost and duration uncertainty using a critical path schedule representing the project scope and strategy overlaid with the project cost estimate and with all the known significant project cost and duration impact risks mapped into the model.”

• As per the above definition, there are three components: – Critical path schedule representing project scope & strategy – Project Estimate, split into fixed (time independent) and variable

(time dependent) costs – The most significant known risks with time and cost impacts:

systemic risks and discrete and risk factor project risks • All uncertain parameters (durations, costs and risk impacts)

have probability distributions applied

8

RISK-2510

IRA Process – Flow Diagram

• The IRA Process follows the following flow diagram:

9

Receive / Critique Project Schedules,

Estimate & Risk Registers

Conduct Risk Assessment / Identification Workshops or

Interviews

Incorporate Required Schedule Changes Identified as Part of

above Steps

Incorporate Duration Uncertainties &

Duration Correlation Model

Incorporate Weather Model (if applicable)

Map Materials & Labour Costs to

Schedule Including Cost Correlation

Model

Map Risk Events & Risk Factors to

Appropriate Schedule & Cost Model

Elements

Analyse Model; Validation Checks

and Initial Reporting

Incorporate Project Team Feedback, Re-Analysis & Reporting

RISK-2510

IRA Process – Receive & Assess Model Inputs

• Receive or build CPM schedule to model project – Detailed Integrated Master Control Schedule, or – Summarised schedule representing logic & strategy – Technically well constructed

• Receive project estimate – Sufficient detail to represent differing risk profiles – Sharing same assumptions as schedule – Able to split accurately into fixed and variable costs

• Review Project Risk Register – Risks expressed for QRA (single cause, metalanguage) – All significant risks with time and cost impacts – Ensure systemic and discrete risks and risk factors covered

10

RISK-2510

IRA Process – Uncertainty/Risk Assessments

• Systemic risk due to organisational limitations leading to flawed project delivery must be assessed based on past relevant project performance: – Preferably by the organisation, or – By similar organisations in same sector and similar scale,

allowing for capabilities specific to this project • Duration and cost ranges and correlation models to be

assessed by Subject Matter Experts (SMEs) – Representing project uncertainties and – Any systemic risks not covered by above empirical data

• Discrete risk events from project risk register and above schedule and cost ranging assessed by SMEs – Assessed for suitability for Quantitative Risk Analysis

11

RISK-2510

IRA Process – Build the model

• Overlay the estimate on the schedule: – Use hammocks linked to, but separate from, project

tasks describing scope of work covered by the costs – Separate the fixed and variable costs, but both to be

linked to applicable tasks • Apply duration and cost ranges from SME assessments • Apply probabilistic weather calendars (if applicable) • Apply schedule and cost correlation models • Apply Systemic risk schedule and cost impacts to model • Map significant discrete treated project risks to model,

correlate as applicable

12

RISK-2510

IRA Process – Analyse and Report

• Treated risk-loaded IRA model validated to ensure it represents project, then Monte Carlo Simulation applied

• Results checked to ensure they make sense before being distributed to stakeholders for review; should include: – Date & Cost Histograms and summaries – Cruciality and cost tornado diagrams (what is driving)

• Review results with stakeholders; agree any changes • Re-analyse and re-review; repeat until acceptable • Complete with Quantitative Exclusion Analysis (QEA):

– Highest sensitivity drivers excluded, re-analysed; time and cost differences at agreed probabilities reported

• QEA allows for further treatment and risk optimisation • Full reporting plus discussion of results, conclusions and any

recommendations then follow 13

RISK-2510

WHEN IS IRA APPROPRIATE, MOST EFFECTIVE?

14

RISK-2510

When to use IRA

• IRA can be used when a project estimate, a project plan and a project risk register exist and are aligned

• This can be early in the development of a project, even at pre-Concept or Select Stage (see Risk-2592), where cost and time-to-market differences can be compared

• The comparative advantage of the methodology may be best just prior to Financial Investment Decision (FID): – When inputs most mature and should be fully aligned – When opportunities to optimise risk are high

• IRA can also be used during project execution to verify project on track or identify where changes needed

15

RISK-2510

QUANTIFYING CONTINGENCY FAILS TO ASSESS SIGNIFICANT CAUSES OF PROJECT FAILURES

- EXTEND IRA TO MODEL PROJECT ASSETS

16

RISK-2510

Many projects fail due to impacts on assets

• Frequent causes of loss of viability of a project are not threats to the project but to assets produced by the project. Examples include: – Demand and thus prices for most minerals and metals have

fallen away, bringing to an end, from around 2013, the mining boom projects that started from around 2003

– Huge increases in production of unconventional oil & gas in America halved prices of crude oil and natural gas and consequently, traded LNG, from 2014, causing cancellation of all projects not irrevocably committed

– Australian tollways failed due to unrealistic traffic forecasts – IT projects failed due to products overtaken by technology:

E.g., Minicomputers, Nokia & BlackBerry cell phones 17

RISK-2510

Need for modelling of Assets as well as Projects

• Clear need for ability to model not just the project but the operation of the produced assets of the project

• This would enable the resilience of the project plus assets to be tested by risks to: – Asset operating revenues, – Operating costs, as well as – The project itself.

• IRA methodology has been extended beyond project startup to cover asset operations as well, in the one model, to bridge the identified gaps in modelling

18

RISK-2510

Extension of IRA to IRRA

• IRA methodology, built on CPM integration of time and cost, can produce probabilistic cash flows

• Extending the modelling beyond startup of the project to incorporate operation of the assets for their economic life allows modelling of: – Revenue uncertainties and risk events – Operating costs uncertainties and risk events

• This is in addition to the project risks and uncertainties already being modelled

• Including revenue and costs enables probabilistic Net Present Values (NPVs) and Internal Rates of Return (IRRs) to be produced and scenarios to be evaluated

• This is “Integrated capital & operating costs, schedule & Revenue Risk Analysis” (IRRA), used in the following example

19

RISK-2510

EXAMPLE IRA / IRRA MODEL: THE GENERIC TOLLWAY PROJECT (“GTP”)

- A BUILD, OWN & OPERATE PUBLIC PRIVATE PARTNERSHIP PROJECT

20

RISK-2510

Objectives of GTP Model

• The GTP CPM-based model of the design, construction, operation and possible expansion of a tollway was created to demonstrate several capabilities of IRA/IRRA: 1. Capacity of CPM-based project risk quantification to assess

contingency and incorporate systemic risk; 2. Capacity of CPM-based modelling to respond to the

occurrence of risk events through the application of risk treatments including logic to alter responses if circumstances change during the simulation; and

3. Model viability of a project and the assets created by the project in alignment with AACE’s Total Cost Management (TCM) Framework

21

RISK-2510

Generic Tollway Project Description

• Generic Tollway nominally 10km in length running east-west from the western edge of city’s Central Business District (CBD) to a major arterial road at its western end

• Westernmost 2km of tollway consists of two tunnels: for three inbound lanes and for three outbound lanes

• Other 8km of tollway runs east from tunnels to western edge of the CBD, crossing over two major roads

22

Western EasternArterial CBD

Generic Tollway Project SchematicSection 1Tunnels

Section 2Major road A Major road B

2 km 8 km

RISK-2510

Generic Tollway Project Plan

• High level tollway project schedule created based on past actual projects. Summary barchart shown below:

23

RISK-2510

Generic Tollway Operations

• Schedule was extended past commencement of tolling to cover 25 years of operation

• Allowance made for growth in use of tollway leading to threat of increasing congestion and travel times

• Treatment is to add extra inbound and outbound lanes – Extra Lanes Project (ELP)

• Decision on applying to add extra lanes depends on volume of traffic per day and toll revenue growth – Could be after just 5 years operation, or not in 25 years

• Also a threat that Govt. may install commuter rail line parallel to GT, reducing commuter traffic and deferring ELP beyond 25 years

24

RISK-2510

Generic Tollway Operations Plan (portion)

• First nine of 25 years operation shown summarised below, including the possible extra lanes expansion:

25

RISK-2510

Extra Lanes Project (ELP)

• Included in Operations phase of schedule are ELP Construct, Commission & Operate (for 10 years) tasks that only exist if called by a macro that runs in each iteration

• The macro only initiates the ELP if tollway revenue in a year reaches a threshold specified as a modelling input

• Depending on the threshold, the ELP could be initiated in any tolling year from Year 1 to Year 16, or not at all in that iteration

• If it is initiated the ELP takes 3 years to be approved and built and operates for 10 years

• A growth formula for revenue is included and a high degree of uncertainty is provided around the revenue and operating costs

• The following slide shows the ELP Project and Operations below Base Operations 26

RISK-2510

ELP Project & Operations Plan

27

RISK-2510

Threat that Metro Rail Line may be built

• In the Schedule Cost Structure is a milestone, at the start of Base Operations Year 6, that the Government is considering building a rail line along the GT route

• This has probabilistic branching to two alternative tasks: – a decision after ~ 1 year not to proceed with the rail line – a five year activity to design and construct the rail line,

then operating tasks from Base Operations Years 11 to 25 • The probability of proceeding can be set at 0% to 100%;

not proceeding is [100%-Proceeding Probability] • If the rail line proceeds, the macro for the ELP stops the

ELP from being constructed at any time in that iteration

28

RISK-2510

Possible Rail Line Plan

• The following screen dump shows the possible rail line design & construction, then operation from Year 11 :

29

RISK-2510

Risks in GTP IRRA Model

• Three kinds of risk were included in GTP IRRA model: 1. Systemic Risk that project delivery organisation and team may

perform less than optimally delivering project; 2. Estimating uncertainty in time and cost - ranges assigned to line

items and task durations rather than single values; 3. Project specific risk events identified by project team as applying to

this project and operation of this asset. • Systemic Risk has been modelled based on the parametric

method advocated by Hollmann3, adapted for the transport sector, for use where previous performance is unavailable

• The Hollmann model is based on data asserted to cover all uncertainty other than scope changes and severe risk events, a major part of which covers organisational project delivery flaws

• Many (most?) organisations do not recognise systemic project delivery risk

30

3 Hollmann, JK, “Project Risk Quantification”: Chs 11, 15 & Appendix A

RISK-2510

Systemic Risk Modelling • Values have been substituted into a Spreadsheet

Template, made available with Hollmann’s book, for cost growth and for schedule slip, as shown below (by permission)

• It calculates mean slip and three point distribution values

31

RISK-2510

Adjustment to Hollmann Hybrid Method

• Hollmann recommends applying the parametric output costs and duration distributions to a single systemic risk activity of P50 duration and cost at the end of a CPM-based ICSRA schedule, using the example in RP 57R-094

• This simple approach also involves removing all ranging of cost and schedule uncertainty from the CPM model

• But it is not workable where the P10 duration of the systemic risk task is negative, as it is in this case

• Problem is overcome by applying a duration risk factor across all project tasks using three point distributions from the parametric modelling. This models time-dependent cost contingency due to time-cost inter-dependency of IRA model

• Time independent cost is modelled by a cost hammock over the project of fixed cost contingency distribution from parametric modelling. This is correlated 100% to duration risk factor

32

4 “Integrated Cost and Schedule Risk Analysis using MCS of a CPM Model”, AACE International®

RISK-2510

Project and Asset Specific Risk Events

• Treated risk events from the project risk register were mapped to the project schedule

• The aggregate Expected Value of these risks is not a significant threat to the project outcome

• But the major threats affect operations of the asset rather than the construction of the tollway, as noted earlier: – Threat that a commuter rail line may be built alongside – Threat of congestion and slower travel requiring Extra

Lanes (ELP treatment of risk) • These threats and treatments are intelligently applied in

the IRRA model, as described earlier

33

RISK-2510

Register of Treated Risks mapped to model

34

RISK-2510

Modelling Results: Project Duration • The Model produced a pessimistic forecast of

schedule contingency due to: – Probabilistic weather calendar not included deterministically – Pessimistic duration risk factor from systemic risk modelling

35

Data

Duration of:

PC - Tollway Project

Analysis

Iterations: 1000

Statistics

Minimum: 1287

Maximum: 1631

Mean: 1457

Std Deviation: 67.03

Bar Width: week

Highlighters

Deterministic (1274) <1%

10% 1369

50% 1456

90% 1548

1300 1400 1500 1600

Distribution (start of interval)

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0

45.0

50.0

55.0

Hits

0% 1287

5% 1351

10% 1369

15% 1384

20% 1395

25% 1405

30% 1415

35% 1424

40% 1439

45% 1449

50% 1456

55% 1465

60% 1474

65% 1484

70% 1494

75% 1504

80% 1517

85% 1529

90% 1548

95% 1569

100% 1631

Cum

ulat

ive

Freq

uenc

y

GTP Rev10.1 $180m Extra Lanes Rev, 20% Rail Threat, Systemic & Treated RisksPC - Tollway Project : Duration

RISK-2510

Modelling Results: Project Capex • The Model also produced a pessimistic forecast of cost contingency

due to similar drivers as for schedule distribution • Costs are negative in cost & revenue model & $0 must be shown in negative range:

36

Data

Cost of:

PZ$.CC - GTP Capital Costs

Analysis

Iterations: 1000

Statistics

Minimum: ($3,651,400,869)

Maximum: ($2,708,894,444)

Mean: ($3,188,103,811)

Std Deviation: 176,512,583

Bar Width: $50,000,000

Highlighters

Deterministic (($2,625,000,0... <1%

10% ($2,959,561,948)

50% ($3,184,982,818)

90% ($3,418,499,211)

($3,000,000,000) ($2,000,000,000) ($1,000,000,000) $0

Distribution (start of interval)

0

10

20

30

40

50

60

70

80

90

100

110

Hits

0% ($2,708,894,444)

5% ($2,907,714,603)

10% ($2,959,561,948)

15% ($3,002,705,674)

20% ($3,030,880,728)

25% ($3,060,361,973)

30% ($3,089,056,949)

35% ($3,111,496,334)

40% ($3,138,824,232)

45% ($3,158,844,612)

50% ($3,184,982,818)

55% ($3,210,787,139)

60% ($3,235,425,842)

65% ($3,262,924,914)

70% ($3,282,514,255)

75% ($3,314,533,159)

80% ($3,342,230,158)

85% ($3,375,796,552)

90% ($3,418,499,211)

95% ($3,483,879,723)

100% ($3,651,400,869)

Cum

ulat

ive

Freq

uenc

y

GTP Rev10.1 $180m Extra Lanes Rev, 20% Rail Threat, Systemic & Treated RisksPZ$.CC - GTP Capital Costs : Cost

RISK-2510

Modelling Results: Drivers of Project Cost • Drivers of project duration and cost distributions

have been determined by Quantitative Exclusion Analysis:

37

RISK-2510

IRRA Scenario Analyses - Inputs

• Two scenarios were analysed to demonstrate the power of the IRRA methodology to compare different asset operating environments and their profitability

• Scenario 1: Low threshold revenue to add extra lanes, low probability of Rail Link being built: – $180 million annual revenue threshold input to macro to

trigger earlier decisions to add extra lanes to the tollway – 20% probability placed on the likelihood that the

government would build an adjacent commuter rail line • Scenario 2: High threshold revenue to add extra lanes,

high probability of Rail Link being built: – $400 million revenue input to the macro to delay decisions

to add extra lanes to the tollway – 80% probability set of an adjacent commuter rail line

38

RISK-2510

IRRA Scenario 1 Analysis – Probabilistic NPV

• Probabilistic NPV for optimistic Scenario 1 shown below (probability of NPV >= $x-value shown, 7% Discount rate)

• ~63% chance of the investment being profitable (positive NPV)

39

RISK-2510

IRRA Scenario Analyses –Compare Profitability

• Probabilistic NPVs for the two scenarios are compared below (probability of NPV < $x-value shown):

40

RISK-2510

STRENGTHS AND WEAKNESSES OF THE IRA AND IRRA METHODOLOGIES

41

RISK-2510

Key Strengths of IRA / IRRA Methodologies

1. Integrates estimating, planning, risk management and economic analysis (IRRA) to optimise time and cost outcomes of the project through simultaneous analysis of time and cost outcomes of the project and the assets (IRRA) and through evaluating the drivers that produce those outcomes.

2. Can enable project risk to be optimised through the integrated ranking of all drivers of project cost and their progressive optimisation.

3. Can evaluate major sources of project failure through examination of the viability of the assets created by the project and the effects on them of risks and uncertainties to project revenue as well as operating conditions, regulations and costs (IRRA).

4. Can improve schedule quality through exposing defects, gaps in scope and misalignment with the estimate.

5. Can facilitate cross-discipline communications and improve project management through collaboration

6. Quantifies effects of risks on the project and the asset (IRRA), focusing the PM team on the need for effective risk management

42

RISK-2510

Weaknesses of IRA / IRRA Methodologies

1. Requires estimate and schedule – thus not useful for early stages of project development, unless schedules have been drafted.

2. Requires a relatively good quality schedule to obtain useful results. 3. Requires significant levels of skills and experience to produce good

quality results. 4. No accepted standards defining procedures, processes or features

and functionality of tools required; depends on educated clients. 5. The methodologies are not empirically based, although historical

data has been referenced to assign repeat activities and previous similar analogous project logic and estimates (excludes hybrid approach incorporating systemic risk described here)

6. Hybrid approach used here assumes all uncertainty incorporated in systemic risk, but part of value of IRA is identifying different areas and magnitudes of uncertainty to reveal hidden risky critical paths.

43

RISK-2510

HOW WELL DOES IRA METHODOLOGY ADDRESS RP 40R-08 CRITERIA FOR

QUANTIFYING PROJECT RISK?

44

RISK-2510

How well IRA addresses RP 40R-08 (1 of 2)

45

RP 40R-08 Principle How well IRA Methodology meets Principle Meets client objectives, expectations and requirements

Over 30 projects analysed; 6 clients multiple users for up to 6 years confirming methodology meets client expectations and requirements. Similar methodologies by others confirm approach.

Part of decision and risk management process

IRA similar to NASA’s JCL 70 Process, part of NASA’s project budget development process.

Fit-for-use Refined since 2008 to be fit-for-purpose; now adapted to use parametric estimation of systemic risks

Starts with identifying risk drivers

Always involves collection of schedule and cost uncertainty information and risk factors where applicable then risk events, as inputs to analysis.

RISK-2510

How well IRA addresses RP 40R-08 (2 of 2)

46

RP 40R-08 Principle How well IRA Methodology meets Principle Links risk drivers & cost/schedule outcomes

Maps risk events, risk factors and probabilistic weather calendars to specific tasks. Also uses QEA to measure impact on schedule and cost of major sources of uncertainty in model.

Avoids iatrogenic risks

Documented IRA processes are repeatable and auditable, minimising chances of self-inflicted risks.

Employs empiricism

IRA methodology now incorporates a Systemic Risk Parametric method (refined by Hollmann) in IRA as a hybrid method. Further work required on this.

Uses experience / competency

Requires significant experience and competency to perform well – indicating power of methodology.

Provides probabilistic estimating results

Cost and schedule simulation output directly and simultaneously, supported by QEA ranking of cost and schedule drivers.

RISK-2510

CONCLUSION

47

RISK-2510

Conclusion

• IRA with parametric assessment of systemic risk complies with the principles of RP 40R-08

• Further development required to retain valuable area/ phase-specific ranging of duration and costs uncertainties to reveal “hidden critical paths”, yet incorporate parametrically derived systemic risk; clients to be educated to accept systemic risk

• CPM-based MCS methodology can incorporate decision-making during iterations to represent realistic treatment of risks. This requires use of macros, which no currently available CPM-based MCS software able to perform ICSRA provides – a market gap

• The Tollway project and asset modelling demonstrates the ability of IRRA methodology to optimise project and asset investment proposals and improve their resilience against asset threats not modelled in contingency–focused analysis

• The methodologies require sophisticated skills and experience, desirable attributes wherever high risk decisions are to be taken

48

RISK-2510

QUESTIONS/COMMENTS? (PLEASE USE MICROPHONE)

49