Cost-Benefit Analysis of indicative high level concept designs -BMT-Co… · ©DAS Ltd 2013...

©DAS Ltd 2013

Cost-Benefit Analysis of indicative high level concept designs -

SCAF Annual Conference

17 September 2013

c:\...\DAS Data\CJM Papers\SACF - MPSC Sep 13 I1 1

©DAS Ltd 2013

Introduction


The theme for this conference is “cost matters” with a commentary that “establishing realistic financial budgets at a start of a project, during the approval process, will initiate projects that are ultimately successful”.

This paper on cost-benefit analysis is intended to show how diverse skills and experiences, across a portfolio of topics of interest to SCAF members, can be applied successfully to assist in the establishment of realistic expectations for defence capability delivery within a constrained cost budget (the best we can afford).

This paper is therefore a joint effort by:

• Dr Dave Exelby – business modelling, strategy and decision sciences

• Chris Maughan – marine engineer and risk management

• Andy Jones – naval architect

©DAS Ltd 2013

Maritime Projection and Sustainment Capability


The New Zealand Defence Force's underway replenishment tanker, HMNZS Endeavour, will soon reach its end of life.

The Maritime Projection and Sustainment Capability (MPSC) project is developing options to replace this maritime capability with an appropriate ship. The project is considering a more versatile vessel which could incorporate some supplementary sea lift capacity, including amphibious and aviation capabilities.

http://www.google.co.uk/url?sa=i&rct=j&q=&source=images&cd=&cad=rja&docid=ikb-zdPF1qtw5M&tbnid=sVsAFSOEnfQOPM:&ved=0CAUQjRw&url=http://117.55.225.121/Publication:Navy_Annual_2007/Up_the_creek_with_a_paddle&ei=loq1UZPtN4af0QWWxYGgBA&bvm=bv.47534661,d.ZG4&psig=AFQjCNFmgFjRDsh-DKiJV35GyvAKfbhmqA&ust=1370938301183976

©DAS Ltd 2013

Acknowledgements


This presentation has been prepared with the co-operation and approval of the New Zealand Defence Force.

Elements of the following slides have been edited to protect $ budgets and sensitive areas of defence capability.

Please note: “The NZDF has not taken any decisions on whether to invest in this capability. Conference participants should not infer that the New Zealand Defence Force or Government of New Zealand is committed to any investment in a ship of this type at this time.”

©DAS Ltd 2013

Background and analysis requirement


The NZDF contracted BMT to undertake an Independent Conceptual Capability Study (ICCS) on the expected requirements of the HMNZS Endeavour replacement, in order to support a business case for the MPSC project.

• BMT undertook a variety of work packages, culminating in the development of indicative high level concept designs.

• Affordability and achievability are critical success factors for the MPSC. Cost for a capability of this nature, whether new build or conversion, is dependent on a number of interrelated factors.

• The challenge for decision makers is the ability to rapidly and confidently down select from a large number of initial, and alternative, concepts to arrive at a set to take forward for further development.

But, which options constitute the best overall portfolio of solutions so as to generate the most effective warship for the available budget?

The answer requires considering trade-offs between solution options, but some solutions meet objectives of capability and cost better than others, so we used a form of multi-criteria decision analysis (MCDA) to enable those solution options to be traded-off through the exercise of military judgement.

©DAS Ltd 2013

Content

• Introduction

• MPSC and acknowledgements

• Background and requirement

• Constraints and inputs

• Our approach and methodology

• The cost-benefit analysis model

• The approach to costing

• Analysis outputs

• Ship design validation modelling

• Conclusions

• Discussion and Questions


©DAS Ltd 2013

Constraints and inputs


The top level requirement for MPSC is defined as being to, renew capability that has a fuel and logistics replenishment ability (including organic aviation) and enhance to conduct independent logistic operations in an amphibious environment through integration of sealift and aviation capabilities.

The MPSC platform needs to be able to safely and effectively:

1. Conduct maritime force logistic support.

2. Maintain deployable bulk fuel reserves.

3. Provide an effective and appropriate maritime platform.

4. Conduct land force logistic support.

5. Support amphibious sealift operations.

6. Provide logistics support to deployed Air Force elements.

7. Deliver support to Other Government Agencies.

The MPSC is to be delivered within the acquisition budget as given in the Defence Capital Plan.

©DAS Ltd 2013

Our approach and methodology


The approach adopted to undertake the cost benefit analysis is one of Multi-Criteria Decision Analysis, or MCDA. It allows for the overall ordering of options preference choices.

It recognises:

• Need for structured and auditable framework to disaggregate decision making

• Accommodation of judgement and hard data

• A mixture of monetary and non-monetary objectives

• Sensitivity analysis to test robustness of decisions and outputs

• Consensus building between decision makers

Our approach was to develop a model that represents the various capability options that are available to deliver the total MPSC solution.

©DAS Ltd 2013

The cost benefit analysis work draws on the earlier ICCS study work packages ensuring a coherent set of outputs

• Various work packages involved the identification of key design drivers and cost impacts of the various options for how the MPSC capability could be delivered. Under 19 capability areas, 49 individual options were identified.

• For each individual option, the impact of an option on the whole ship was identified (principal dimensions, GA, equipment, systems)

• These formed the key inputs to the conduct of the cost-capability analysis


• A structured cost benefit analysis (CBA) of the overall concept ship designs in order to down select likely affordable solutions to take forward to the DBC

• This work included

–Cost estimation and research

–Development, population and analysis of a MCDA model using Equity

–Undertaking military judgement-based workshops

©DAS Ltd 2013

Cost Benefit Analysis (CBA) has adopted multi-criteria portfolio analysis (MCPA) using Equity and a structured methodology

• The workshops focussed on:

– Reviewing model structure

– Populating benefit scores

– Populating weighting

– Critically reviewing emerging results

– Scenario analysis


Establish the Context

Identify the Areas and Options

Identify Criteria

Score the Options against the Criteria

Weight the Areas and Criteria

Examine the Results

Sensitivity Analysis

Establish objectives to be achieved - the model requirements

Identify key decision areas that will be considered and within these options that are available

Identify set of criteria that will determine the benefit of options

Score the options in turn within the area against the criteria. Benchmark to best

Assign importance weight of areas to each criteria Assign weight across the criteria

Assess the resulting ranked list of portfolios of options producing all possible solutions

Explore the robustness of weighting and option score variation between workshop attendees

Workshop 1

Workshop 2

©DAS Ltd 2013

MPSC ship design areas and associated options scored within the CBA – creating a potential solution space


This grid produces 1,679,616 possible solutions



Identify Criteria

Score the Options against the

Criteria


Examine the Results


One of 19 design areas

©DAS Ltd 2013

The benefit criteria were based on the high level requirements (Operational Needs) identified in the IBC

• The top level requirement for MPSC is to:

– Renew capability that has a fuel and logistics replenishment ability (including organic aviation) and enhance to conduct independent logistic operations in an amphibious environment through integration of sealift and aviation capabilities

Renew and Enhance represent the opposite ends of the capability spectrum

• The MPSC platform needs to be able to safely and effectively:

1. Conduct maritime force logistic support

2. Maintain deployable bulk fuel reserves

3. Provide an effective and appropriate maritime platform

4. Conduct land force logistic support

5. Support amphibious sealift operations

6. Provide logistics support to deployed Air Force elements

7. Support to Other Government Agencies

• Each capability option scored for each Operational Need




Identify Criteria

Score the Options against the

Criteria


Examine the Results


©DAS Ltd 2013

Workshop attendees provided coverage and depth to contribute to the benefit scoring and reaching consensus

Workshop attendees were from the full spectrum of defence, including:

• MPSC Project Director

• MPSC Project Manager

• MoD Acquisition

• MoD Policy and Planning

• RNZN – Tanker and MRV operators

• NZDF Capability, covering:

–Maritime operations

–Amphibious Warfare (Army)

–Maritime aviation

–Projection and Sustainment Working Group

• HQ JFNZ

• ICCS Naval Architect

There were also a number of observers from central government agencies to provide oversight of the process and methodology adopted.


©DAS Ltd 2013

• In workshop 1, each design area and its options were scored for each Operational Need

– Considered where it contributed within the Operational Need

– Considered how the options degraded effect from best

– Once complete, these were reviewed as part of workshop 2

• In workshop 2, the relative importance of the effect from the design area was considered for each Operational Need and used to discount the scores

• These scores were used within the MCPA model using Equity

• The final remaining step was to determine a relative weighting of each Operational Need – initial scoring at workshop 1 with review and minor changes during workshop 2

The benefit scoring of the options took place at the two workshops


Determine Design Area importance within Operational Need

Score and review within Design Area options

Populate Equity model

©DAS Ltd 2013

The cost element of the Equity Model consists of ….

Estimated Unit Acquisition Cost (UAC);

• Baseline Costs

– Ship –The cost of acquiring the basic ship platform excluding capability options

– Crown–Internal NZDF/MoD costs (Project Management, ILS and Introduction into Service)

• Capability Option Costs

DAS

• GFE – Government Furnished Equipment linked to capability options.

BMT

• Ship Modification Cost – Modifying the ship structure /principal dimensions

• Ship System Cost – Modifying or adding ship systems

• Equipment Cost – Specific equipment associated with the capability option, i.e. cranes

Validation by comparison

• Prices paid by other users for similar ships or equipment

• ‘Should cost’ outturn costs , using DAS parametric estimating model, for naval auxiliary vessels providing the capability intended for MPSC


©DAS Ltd 2013

Capability Option Costs….

16 c:\...\DAS Data\CJM Papers\SACF - MPSC Sep 13 I1

Capability Option Costs

• Ship Modification Cost

• Baseline design created in Paramarine, 3D naval architecture software

• Each individual capability option modelled and the design balanced for;

– Area, volume, weight, stability, speed, power, cargo capacity etc….

• Principal particulars used with BMT’s Cost Model to generate a Modification Cost

• Ship System Cost/ Equipment Costs

• Cost estimated using parametric scaling from an equipment database captured from previous projects

©DAS Ltd 2013

Option costs have been based on parametric outturn cost models recognising explicitly potential pricing positions by shipyards

• Costs estimated for Unit Acquisition Cost (UAC)

• Outturn-based

• Variation between Commercial and Naval procurement for auxiliary source data captured

• Military shipyard price positioning of military vs. commercial introduces high costs

— Increased level of supervision by customer

— Increased level of complexity of vessel

— Increased level of rework

— Increased level of change

— Increased perception of risk/ profit by the yard

— Increased equipment costs due to performance required/ level of supervision/ documentation etc


©DAS Ltd 2013

Gap analysis Capability trading

Scenario modelling

Approach to analysis of cost benefit results

• Key questions to answer

– Do the aspirations for ship capability and associated concepts align in terms of cost and benefit?

– How much capability can be delivered within the capital budget allocation

– What is the impact of the shipyards’ price positioning to the capability required (commercial versus military ship price positions)


3. Scenarios – shipyards’ pricing positions

1. Top-down aspirational

solutions based on OCD/IBC

2. Optimal concepts

within budget

©DAS Ltd 2013

Four ship concepts were developed as proposed packages to assess relative to the cost benefit scores

• Workshop attendees developed a set of ship concepts based on careful review of the requirements

• Focus was on a concept that provides the minimum perceived capability to meet the operational needs (cost of options was not discussed or revealed to the participants)

• Discussion identified two concept solutions - with and without LCMs


Commercial Renew Commercial Enhance Commercial Enhance with LCM

Commercial Enhance+

Jackstays for RAS Use Endeavour RAS As New + Crane As New + Crane As New + Crane

Low Flashpoint Fuel Jettisonable stowage Jettisonable stowage Jettisonable stowage Jettisonable stowage

Aviation Organic Seasprite As Option2 + CH47 As Option2 + CH47 As Option2 + CH47

Vehicle Stowage LIMS Nil LIMS Vehicle Deck LIMS Vehicle Deck LIMS Vehicle Deck LIMS

Containers 8 X 20 foot ISO 8 X 20 foot ISO 8 X 20 foot ISO 8 X 20 foot ISO

Landing Craft No landing craft No landing craft 2 x LCM 2 x LCM

Materials Handling 25t Crane 65t Cranes 65t Cranes 65t Cranes

Manoeuvre Single Shaft+ Tunnel 2Shaft,Tunnel+Thrust 2Shaft,Tunnel+Thrust 2Shaft,Tunnel+Thrust

Power & Propulsion Mech. Propulsion Hybrid Propulsion Hybrid Propulsion Hybrid Propulsion

Stability Commercial Military Stability Military Stability Military Stability

Self Defence HMG Positions Mini Typhoon GS Mini Typhoon GS CIWS

Shock No Protection No Protection No Protection No Protection

Maximum Speed 16kts 18kts 18kts 20kts

Dangerous Cargo Cont. on Weatherdeck Cont. on Weatherdeck Cont. on Weatherdeck Deep Magazine

Accom. Standards Existing Standard Existing Standard Existing Standard Existing Standard

No. of Berths 70 Berths 120 Berths 120 Berths 120 Berths

Opr. Environment No Ice Capability No Ice Capability No Ice Capability Ice 1A

Command and Control Ships Ops Space Ships Ops & CPR Ships Ops & CPR Ships Ops & CPR

Medical Care Role 1 Role 2+ Surgical Role 2+ Surgical Role 2+ Surgical

Red text marks changes in options across successive proposed ship concepts 18kts

©DAS Ltd 2013

Aligning these aspirational concepts on the cost benefit envelope showed these were well aligned to the value frontier

• Enhance option lies on the value frontier representing a best value for money option

• The Enhance plus LCM is only marginally sub –optimal. This is due to Ice Class 1A position ahead in the order of priority


Order of Priority

Option Traded In

0 Renew

1 Military Stability

2 2x Shaft and Tunnel

3 Ships Ops & CPR

4 As New + Crane

5 2Shaft,Tunnel+Thrust

6 Role 2+ Surgical

7 Weatherdeck LIMS

8 70+50(austere)

9 Mini Typhoon GS

10 65t Cranes

11 Organic NH90

12 18kts

13 Hybrid Propulsion

14 120 Berths

15 Vehicle Deck LIMS

16 As Option2 + CH47

17 Ice 1A

18 Deep Magazine

19 2 x LCM

20 CIWS

21 20kts

Renew

Enhance+

Enhance

Value envelope for commercial build positioning (normalised benefit and costs UAC NZ$M 2012 e.c.)

©DAS Ltd 2013

A shipyard’s military price positioning will modify design area option costs


% Impact on total option price if procurement considered military project by shipyards (cost base includes GFE but excludes Crown costs)

• Basic Enhance platform estimated to increase in cost by 38%. Note this analysis includes GFE but excludes Crown costs

• Large GFE components within options will discount some impacts e.g. CIWS

Design Area Option 1 Addl. % Option 2 Addl. % Option 3 Addl. %

Basic Ship Basic Ship

Jackstays for RAS Use Endeavour RAS As New + Crane 8% 1 Station P&S, New 8%

Low Flashpoint Fuel Jettisonable stowage Canisters to 2000L 18% TEU Container 18%

Aviation Organic Seasprite Organic NH90 2% As Option2 + CH47 1%

Vehicle Stowage LIMS Nil LIMS Weatherdeck LIMS 65% Vehicle Deck LIMS 60%

Containers 8 X 20 foot ISO

Landing Craft No landing craft 2 x LCM 4%

Materials Handling 25t Crane 65t Cranes 0%

Manoeuvre Single Shaft+ Tunnel 2x Shaft and Tunnel 57% 2Shaft,Tunnel+Thrust 65%

Power & Propulsion Mech. Propulsion Hybrid Propulsion 24%

Stability Commercial Military Stability 40%

Self Defence HMG Positions Mini Typhoon GS 0% CIWS 1%

Shock No Protection

Maximum Speed 16kts 18kts 32% 20kts 26%

Dangerous Cargo Cont. on Weatherdeck Deep Magazine 40%

Accom. Standards Existing Standard

No. of Berths 70 Berths 70+50(austere) 23% 120 Berths 18%

Opr. Environment No Ice Capability Ice 1A 42%

Command and Control Ships Ops Space Ships Ops & CPR 23%

Medical Care Role 1 Role 2+ Surgical 10%

40%

©DAS Ltd 2013

The change in costs will shift the incremental value scores and the order of priority does change slightly


• Enhance platform is not a frontier option although it remains very close – simply value promotion of Ice Class 1A

• Manoeuvre and Vehicle Stowage impacted by large cost growth

Order of Priority

Commercial Military

0 Renew Renew

1 Military Stability As New + Crane

2 2x Shaft and Tunnel Military Stability

3 Ships Ops & CPR 2x Shaft and Tunnel

4 As New + Crane Role 2+ Surgical

5 2Shaft,Tunnel+Thrust Ships Ops & AOR

6 Role 2+ Surgical Mini Typhoon GS

7 Weatherdeck LIMS 2Shaft,Tunnel+Thrust

8 70+50(austere) 70+50(austere)

9 Mini Typhoon GS 65t Cranes

10 65t Cranes Weatherdeck LIMS

11 Organic NH90 Organic NH90

12 18kts 18kts

13 Hybrid Propulsion As Option2 + CH47

14 120 Berths 120 Berths

15 Vehicle Deck LIMS Hybrid Propulsion

16 As Option2 + CH47 Ice 1A

17 Ice 1A Vehicle Deck LIMS

18 Deep Magazine Deep Magazine

19 2 x LCM 2 x LCM

20 CIWS CIWS

21 20kts 20kts

Value envelope for naval auxiliary build positioning (normalised benefit and costs UAC NZ$M 2012 e.c.)

Options in Red forced into package, Green options removed to achieve Enhance

©DAS Ltd 2013

Combining the value frontiers for commercial and military pricing positions allows comparisons to the Defence Capital Plan


Value frontiers for commercial and naval auxiliary build positioning (normalised benefit and UAC NZ$M 2012 e.c.)

• It appears that if commercial price positioning by the shipyard can be achieved, then the Enhance option falls within the endorsed budget

• Achieving this will be an important aspect of the procurement strategy

©DAS Ltd 2013

A budget cap of NZ$M xyz would severely limit the capability of MSPC should shipyards adopt military price positioning


Commercial Positioning Commercial Enhance Military Enhance Military Optimal Trade Off

Benefit/Cost basis Frontier Package Frontier

Cost $NZ (2012 e.c.) x y z

Ship Design Areas

Jackstays for RAS As New + Crane As New + Crane As New + Crane

Low Flashpoint Fuel Jettisonable stowage Jettisonable stowage Jettisonable stowage

Aviation As Option2 + CH47 As Option2 + CH47 Organic Seasprite

Vehicle Stowage LIMS Vehicle Deck LIMS Vehicle Deck LIMS Nil LIMS

Containers 8 X 20 foot ISO 8 X 20 foot ISO 8 X 20 foot ISO

Landing Craft No landing craft No landing craft No landing craft

Materials Handling 65t Cranes 65t Cranes 25t Crane

Manoeuvre 2Shaft,Tunnel+Thrust 2Shaft,Tunnel+Thrust 2x Shaft and Tunnel

Power & Propulsion Hybrid Propulsion Hybrid Propulsion Mech. Propulsion

Stability Military Stability Military Stability Military Stability

Self Defence Mini Typhoon GS Mini Typhoon GS HMG Positions

Shock No Protection No Protection No Protection

Maximum Speed 18kts 18kts 16kts

Dangerous Cargo Cont. on Weatherdeck Cont. on Weatherdeck Cont. on Weatherdeck

Accom. Standards Existing Standard Existing Standard Existing Standard

No. of Berths 120 Berths 120 Berths 70 Berths

Opr. Environment No Ice Capability No Ice Capability No Ice Capability

Command and Control Ships Ops & CPR Ships Ops & CPR Ships Ops Space

Medical Care Role 2+ Surgical Role 2+ Surgical Role 1

• The cost of delivering the Enhance concept increases by 32%

• Concept no longer on the Frontier of best value

• Massive loss of capability to maintain budget

• Unable to provide any enhanced role

Drop in capability compared to Enhance option

As New

+ Crane Options in green better than Renew option

©DAS Ltd 2013

The MCPA analysis does not explicitly capture the system of systems aspects of the overall platform – revisiting the proposed concepts

• Validation has been considered through two lenses:

– Engineering modelling (using Paramarine) -

• Dimensions may be driven by area, volume, weight, powering, or stability

• Layout changes e.g. combined options will drive ship dimensions to be greater or smaller than summation of individual options (system of systems)

• Stability changes e.g. addition of vehicle deck may increase principle dimensions reducing the impact on stability of another option

– Acquisition price -

• Price drivers (based on principal dimensions, weight and power)

• Four combined option designs modelled:

— Points of interest identified and investigated

— Feasibility of designs confirmed

— Starting assumptions confirmed

— General arrangements created to capture second order effects


©DAS Ltd 2013

Ship design validation modelling


• To consider second order effects of combining options:

– Principal ship dimensions may be driven by area, Volume, Weight, Powering, or Stability or a combination thereof

– Addition of a capability may increase principle dimensions reducing the impact on stability of another option

– Addition of a combination of small changes may require a change to principal dimensions not individually required, thus increasing the impact of the options.

– The combination of options may drive ship dimensions to be greater or smaller than summation of individual options.

• ‘Bookend’ options investigated to cover the range of capability:

– Area of capability interest identified

– Greater definition of capability now possible (e.g. effect of 1 LCM)

– All options were assessed as ‘fitted’. Investigation of FFBNW would require benefit scores would to be reviewed

©DAS Ltd 2013

• Principal particulars used to create revised cost estimates

The indicative general layout for the Enhance without LCM concept illustrates incorporation of the proposed design options


Main Dimensions Enhance without LCMs Reference Commercial Vessel

Length Waterline (m) 169 167*

Beam Waterline (m) 27.4 26

Draught (m) 8.8 9.2

* Length between perpendiculars

Prices Enhance (NZ$M 2012) Enhance + LCMs (NZ$M 2012)

Commercial Price Positioning

CBA Combined Option Price

Validation Price

Percentage 96% 97%

Naval Price Positioning

CBA Combined Option Price

Validation Price

Percentage 96% 95%

Enhance ship concepts – to show accuracy of approach, Option price versus Validation price

©DAS Ltd 2013

Conclusions and recommendations

For MPSC:

• Alignment of ship concepts to best value for money:

– MCPA has provided an independent bottom up approach to assessing best value for money solutions for a given budget

– The results show that the Enhance options are the best value for money solutions, and affordable within the endorsed Defence Capital Plan

• Shipyard commercial and military price positioning is a key driver of affordable capability

• Crown and GFE costs are a significant driver to the total price

• Team left with a MCPA model that can be rapidly updated as source data is further validated

For the CBA approach:

• Successfully broke down a complex problem to a level that allowed real choices to be made in a structured and auditable manner

• Diverse military stakeholders quickly came to a consensus on the desired capability attributes for MPSC. There were no capability ‘sacred cows’ – ship speed and Army logistics being examples.

• Speed of analysis – not a significant demand on stakeholders time


©DAS Ltd 2013

Any questions


Dr Dave Exelby Decision Analysis Services Ltd [email protected] +44 1256 338384 +44 7867 801362 www.das-ltd.co.uk

Chris Maughan CEng Decision Analysis Services Ltd [email protected] +44 1256 338385 +44 7770 362835 www.das-ltd.co.uk

Mr Andrew Jones BMT Defence Services Ltd [email protected] +44 1225 473656 +44 7973 820071 www.bmtdsl.co.uk

mailto:[email protected]















http://www.bmtdsl.co.uk/

Cost-Benefit Analysis of indicative high level concept designs -BMT-Co… · ©DAS Ltd 2013...

Documents

Transcript of Cost-Benefit Analysis of indicative high level concept designs -BMT-Co… · ©DAS Ltd 2013...