A Trade-off Analysis - Defense Acquisition University · Web view2012/12/12 · Finally, there is...

LOG 211 Supportability Analysis Student Guide

Lesson 11: A Trade-off Analysis

Content

Slide 11-1. Lesson 11: A Trade-off Analysis

Welcome to Lesson 11: A Trade-off Analysis.

January 2013Final v1.3

1 of 72


Topic 1: Introduction

Content

Slide 11-2. Topic 1: Introduction

The first topic orients Life Cycle Logisticians (LCLs) to their current location in the Life Cycle Management Framework, informs them of where they have influence in the process, and lists the lesson objectives.

2 of 72 January 2013Final v1.3


Content

Slide 11-3. The Life Cycle Management Framework – Where Are You?

What Influence Do You Have?

Where are you?

While there are trade-off analyses done throughout the Life Cycle Management Framework (e.g., the Analysis of Alternatives (AoA) conducted as part of Pre-Milestone A Phase activities), the focus is usually on those done as part of the Supportability analyses during the Technology Development (TD)Maturation and Risk Reduction (TMRR) and Engineering & Manufacturing Development (EMD) phases. For purposes of this lesson, we will illustrate a trade-off occurring in the EMD phase. Trade-off analyses are conducted to address specific areas within the overall Supportability Analysis process to determine the most affordable system design that meets user needs over the life cycle.

What influence do you have?

Beginning in the TD TMRR phase of the Life Cycle Management Framework, the LCL influences the system design from a Supportability perspective. As the phase progresses and transitions to the EMD phase, the LCL influences the design and acquisition of the system’s support resources, and ultimately in the late stages of EMD, the LCL influences requirements to support the design.


3 of 72

Technology Maturation & Risk Reduction

PDallosta, 02/24/14,

Slide 11-3 page 11-3 update to TMRR


Updated TMRR


Updated TMRR


Content

Slide 11-4. Topics and Objectives



Topic 2: A Trade-off Analysis Overview

Content

Slide 11-5. Topic 2: A Trade-off Analysis Overview

This topic provides an general overview of the trade-off analysis process with regard to the following:

Purpose Concepts The Life Cycle Management Framework Inputs and outputs Documentation The LCL’s role


5 of 72


Content

Slide 11-6. What Is the Purpose of a Trade-off Analysis?

Trade-off analyses enable the LCL to compare system components, costs, maintenance concepts, etc., to determine the best total Life Cycle Cost for a system. The Affordability System Operational Effectiveness (ASOE) Model provides a high-level illustration of what primary and secondary trade-offs are performed. Primary trade-offs compare Mission Effectiveness and Life Cycle Cost and Total Ownership Cost, whereas secondary trades make comparisons between Mission Effectiveness, Process Efficiency and Design Effectiveness. Corresponding third-tier trades within Design Effectiveness compare Technical Performance and Supportability, while Process Efficiency compares the Integrated Product Support (IPS) Elements and Operations. All trades are conducted to ensure the meeting of the Availability Key Performance Parameter (KPP) and its Key System Attributes (KSAs).

The LCL uses trade-off analysis results to determine the following:

Design that balances user requirements and the lowest affordable Life Cycle Cost

Preferred support system alternative for each new system alternative Support, design, and operational approach that best satisfies the

requirements Optimum level of Reliability and Maintainability to achieve an

Availability requirement Preferred design approach



Content

Additionally, trade-offs:

Ensure Supportability Analysis results in an affordable design and product support package

Provide an initial assessment of requirement Affordability during the Technology Development Maturation and Risk Reduction (TMRR) phase

Enable the LCL to determine the product support arrangement during the EMD phase

Support the LCL’s examination of alternatives to control Sustainment costs or achieve Materiel Availability (Am) at a lower cost

While the ASOE Model is fresh in your minds, turn your attention to different trade-off opportunities pertaining to:

Technical Performance vs. Maintainability vs. Cost Technical Performance vs. Process Efficiency vs. Cost Reliability vs. Maintenance vs. Cost Operations Tempo (OPTEMPO) vs. Maintenance vs. Cost


7 of 72


Updated TMRR


Content

Slide 11-7. Trade-off Opportunities

A trade-off analysis is about achieving balance among competing priorities to get the best outcome possible. Four of the distinct intersections, highlighted in the ASOE Model with different shapes, represent program performance and Sustainment objectives. Balancing these priorities requires a series of trade-off analyses.

The following visuals illustrate the relationship of the ASOE Model to trade-off analyses.



Content

Slide 11-8. Trade-off Opportunity: Technical Performance vs. Maintainability vs. Cost

Technical Performance (i.e., requirements) versus Maintainability versus Cost:

This trade-off assesses Technical Performance requirements (i.e., Capability Development Document (CDD)) versus Supportability requirements, Maintainability versus Life Cycle Cost).

This ASOE connection investigates the benefit versus the cost of achieving KPPs and Maintainability KSAs through accessibility, modularity, and testability.

This trade-off opportunity answers two questions:o What maintainability requirements are best incorporated into

design?o What are the priorities based upon a maximum Return On

Investment (ROI)?


9 of 72


Content

Slide 11-9. Trade-off Opportunity: Technical Performance vs. Process Efficiency vs. Cost

Technical Performance (i.e., requirements) versus Process Efficiency versus Cost:

This trade-off assesses requirements (i.e., CDD) versus supply chain efficiency.

This ASOE connection investigates the benefit versus the cost of achieving KPPs and KSAs around Mean Down Time (MDT) through effective implementation of Supportability analyses such as Reliability Centered Maintenance (RCM) Analysis, Maintenance Task Analysis (MTA) and Level of Repair Analysis (LORA).



Content

Slide 11-10. Trade-off Opportunity: Reliability vs. Maintenance vs. Cost

Reliability versus Maintenance versus Cost:

This trade-off assesses Reliability versus Maintenance Efficiency versus Cost.

This ASOE connection investigates the benefit versus the cost of the Reliability of a design on Total Maintenance Cost.

This trade-off is part of a LORA.o A LORA can conduct a ‘what-if’ sensitivity analysis holding all

variables constant except Reliability.


11 of 72


Content

Slide 11-11. Trade-off Opportunity: OPTEMPO vs. Maintenance vs. Cost

OPTEMPO versus Maintenance versus Cost:

This trade-off assesses an increase in Operations Tempo (OPTEMPO) in terms of maintenance efficiency and cost. The result provides rationale for updating the initial Operations and Maintenance (O&M) requirements in the CDD/Capability Production Document (CPD) program documents.

This ASOE connection investigates the benefit versus the cost of increased operating hours per year.

This trade-off may also be done as part of a LORA.o A LORA can conduct a ‘what-if’ sensitivity analysis holding all

variables constant except operating hours.

Turn your attention to an example illustrating a trade-off between the Operational Availability (AO) and Cost criteria.



Content

Slide 11-12. Trade-off Ao vs. Cost Criteria

This visual illustrates the relationship of the AO versus Cost criteria specific to a trade-off. Notice the steep slope of the line from 0 – 80%, which indicates a good return on investment. Also notice how the curve flattens out after 85% (i.e., the knee of the curve). At this point, the cost of buying a single point of Availability significantly increases.


13 of 72


Content

Slide 11-13. When Do Trade-off Analyses Occur?

The acquisition phase and system complexity dictate the nature of the trade-off, the techniques used to perform it, and the analysis scope. As the visual illustrates, trade-off analyses occur throughout the life cycle process. Supportability trades pertain to the following (at a minimum):

Supportability Cost (e.g., research and development, engineering development,

production, and Operation & Support (O&S)) Readiness drivers of the new system Technological opportunities Supportability objectives Thresholds (Reliability & Maintainability (R&M), Reliability Centered

Maintenance (RCM), Level of Repair Analysis (LORA)) Constraints (R&M) Functional requirements (R&M) New system design Operational and support concepts under consideration Risk




Content

Trade-offs early in the program are generally interdisciplinary and broad in scope. They are typically made between requirements and cost. As development progresses and the process matures, trade-offs:

Are progressively more refined Have more specific inputs Result in outputs influencing a smaller number of related parameters

Trade-off analyses between design, operational, and support alternatives are an inherent part of system development. Benefits are realized when these analyses are conducted by considering all system factors (e.g., cost, schedule, performance, technology, and Supportability) before the system is finalized.

Trade-offs between the support alternatives identified for the new system drive the support approach that best satisfies requirements. In a general sense, these trade-offs use a model or manual procedure that relates design, operation, and logistics support factors of alternatives to system Supportability requirements.

Trades become more specific as programs progress through the life cycle. They are repeatable and gain precision with the maturity of associated Logistics Product Data and related Supportability analyses.

Lesson 5 describes how the R&M analysis provides the framework to establish realistic trades between subsystems and major assemblies to meet Reliability objectives. This is where the LCL has the most influence in designing in Supportability.

Lessons 8 – 10 discuss how the RCM Analysis, MTA, and LORA work together to advance Maintainability and Availability as a cost trade-off.

In this lesson, we investigate how to apply all of these trade-offs using additional criteria derived from Lesson 3: Measures of Effectiveness.

Trade-off analyses are not one-time tasks but are refined throughout all major milestones and Supportability artifacts (e.g., prior to the selection of a Product Support Strategy and when contracts are in place). At Milestone C the Product Support Management (PSM) IPT updates the Life-Cycle Sustainment Plan (LCSP) and Business Case Analysis (BCA) as necessary with the trade-off outcomes. These outcomes are prominent parts of a decision matrix for preferred Performance Based Agreement determinations and become legal contracts (inorganic) and Memorandum of Agreements (organic). After contracts are in place, trade-offs occur at a minimum of every five years as part of the Sustainment BCA, or sooner if:

User requirements change significantly The product support strategy undergoes a change Technology advances dictate a system change

Now that you know what trade-offs are, turn your attention to inputs and outputs.


15 of 72


Content

Content

Slide 11-14. Trade-off Analysis Inputs and Outputs

This diagram provides a high-level view of the inputs, process, and outputs of a trade-off analysis. Trade-off analysis inputs and outputs are specific to a point-in-time on the program schedule. Trade-off analyses inputs may include:

Documentso Analysis of Alternatives (AoA)o Initial Capabilities Document (ICD)o Capability Development Document (CDD)o Capability Production Document (CPD)o Program schedule

Datao Logistics Product Datao Cumulative trade studies from previous Supportability analyses

which pertain to justifying design decisions, to include: Reliability & Maintainability (R&M) Modeling Failure Mode, Effects and Criticality Analysis (FMECA) Fault Tree Analysis (FTA) LORA/MTA Post-fielding



Content

Trade-off analyses outputs lead to:

Preferred design/product support alternatives Improved Life Cycle Cost estimates/Better Buying Power (BBP) Refined Logistics Product Data Refined/Improved LCSP and BCA


17 of 72


Content

Slide 11-15. Trade-off Analysis and User and Program Documentation

In this visual, the ASOE Model highlights trade-offs relative to the Life Cycle Management Framework. Pre-Milestone A, trade-off analyses occur between requirements defined in the ICD and Life Cycle Costs.

Post-Milestone A, trade-offs are performed between the requirements defined in the CDD, Supportability requirements, and Life Cycle Costs. The results of these trades are captured in the LCSP and BCA program documents.

The earlier the trade-off is conducted, the greater the LCL’s influence is in designing a supportable system. It is important to emphasize this LCL design influence. Specifically, the LCL’s role is to advocate supportable designs to maximize the opportunity to save support resources. In Lesson 9, a trade-off example yielded significant return on investment, where the LCL doubled unit cost (buying greater Reliability), returning a 24-fold Life Cycle Cost savings. Being a Supportability advocate is the most significant role an LCL performs throughout the acquisition.

Next, you will learn about the LCL’s role within the IPT structure to influence trade-offs.




Content

Slide 11-16. The LCL’s Role in Trade-off Analysis

The Integrated Product Team (IPT)/Working Integrated Product Team (WIPT) participates in trade-off analyses to ensure all aspects of the design and the different perspectives are represented and considered in the overall acquisition process. The support element that is part of a given trade-off corresponds with the IPT(s) to which the LCL reports. As part of an IPT, the LCL assures that the trade-off is relevant and targeted to the appropriate outcome. The LCL also plays a role in setting up the trade, especially if it is in the LCL’s specialty area.

During a trade-off analysis, the LCL:

Participates in the life cycle acquisition process as a member of the IPT Balances user requirements and the lowest affordable Life Cycle Cost Ensures the logistics costs are included in the total Life Cycle Cost Reviews all previous trades Ensures trades and trade results get into the database (i.e., confirm and

add new trades) Documents trade results and shares them with the appropriate IPT(s)

As illustrated here, each trade-off may involve one or more Integrated Product Support (IPS) Elements, and accordingly, a line of communication to one or more IPTs.


19 of 72


Content

Slide 11-17. LCL’s Role in a Trade-off Analysis – Example

In post-fielding, the Sustaining Engineering IPS Element addresses whether or not to go with the major upgrade/modification/replacement of a component or system-level upgrade. Keep in mind that trade-offs do not end with the initial acquisition of a system or component; they can also influence the subsequent production run during post-fielding.

The LCL should consider the following:

What are all the options and what criteria should be used to determine the best option?

In terms of BBP, does the change favor a more affordable and effective operational outcome for the investment?

Is the best option to make the modification now or continue monitoring for a better opportunity (e.g., batch many components rather than modifying one at a time)?

Or is the best option to take no design upgrade action at all, and instead emphasize training and better repair procedures?



Content

In this example, the LCL’s primary line of communication is to the Requirements Management IPT, to which the LCL reports the opportunity to fix the component during the next production run. The secondary line of communication for an LCL regarding the Maintenance Planning and Management Element extends to the Configuration Management IPT.

After the analysis is complete, the LCL shares the findings with the appropriate IPTs and agrees upon updates to apply to the LCSP.

Now that you finished Topic 2: A Trade-off Analysis Overview, turn your attention to Affordability and Should-Costs.


21 of 72


Topic 3: Affordability and Should-Cost

Content

Slide 11-18. Topic 3: Affordability and Should-Cost

This topic introduces the concepts of Affordability and Should-Cost.

The following references will be helpful during this topic:

USD(AT&L) Memorandum, Should-Cost and Affordability dated August 24, 2011 (http://www.acq.osd.mil/docs/Should-cost%20and%20Affordability.pdf)

USD(AT&L) memorandum Implementation of Will-Cost and Should-Cost Management memorandum dated April 22, 2011 (http://www.acq.osd.mil/docs/USD(ATL)_Memorandum_on_Implementation_of_Will-Cost_and_Should-Cost_Management_042211.pdf)


http://www.acq.osd.mil/docs/USD(ATL)_Memorandum_on_Implementation_of_Will-Cost_and_Should-Cost_Management_042211.pdf)

http://www.acq.osd.mil/docs/USD(ATL)_Memorandum_on_Implementation_of_Will-Cost_and_Should-Cost_Management_042211.pdf)

http://www.acq.osd.mil/docs/Should-cost%20and%20Affordability.pdf

http://www.acq.osd.mil/docs/Should-cost%20and%20Affordability.pdf


Content

Content

Slide 11-19. Affordability

Affordability means conducting a program at a cost constrained by the maximum resources the Department can allocate for the capability. Affordability targets, from a requirements perspective, ensure that qualified goals for Unit Production Cost and Sustainment costs are established and driven by financial resources.

While this graphic suggests that Reliability and Sustainment cycle time are independent variables, they are not. The purpose of this graphic is to show that costs of the two variables are inversely related, i.e., if more money is spent on Reliability in the design phase, then generally speaking, Sustainment costs are decreased. This graphic also emphasizes the need for balance between the cost of Reliability and the cost of Sustainment.

The Target Area is the point where equilibrium between Reliability and Sustainment cycle time is achieved. As illustrated by the inflection point in the Life Cycle Cost curve, the impact of the investment (increased spending) on Reliability has a corresponding improvement (decreased cost) in Sustainment cycle time, which reflects the optimized of Life Cycle Cost.


23 of 72


Content

As addressed in Lesson 2: Product Supportability Across the Life Cycle, the BBP Affordability initiative assigns each program a BBP Affordability target and tracks progress toward that goal on a routine basis using the Quad chart. Lesson 3: Measures of Effectiveness for Supportability discussed how these cost goals are translated into KPPs and KSAs. These goals are set early and drive design trades and choices about affordable priorities which are best described as a balance between alternatives. Affordability analyses:

Are based on the budgets that are expected to finance a product over its entire life cycle

Provide design constraints on the product which is to be built, procured, and sustained



Content

Slide 11-20. Affordability and the ASOE Model

This graphic illustrates the relationship between the ASOE Model and the trade-off analysis graph. Affordability, as depicted in the ASOE Model, relates to the optimization of all facets of the program. When the correct trades are performed between Supportability and process efficiency using a target budget, an affordable system (i.e., the target area) is achievable. The trade-off analysis graphic depicts the inverse relationship of Reliability and Sustainment relative to cost.

In the trade-off analysis conducted at the end of this lesson, the Affordability opportunity intersects the following ASOE diagram nodes (i.e., brown circles): capabilities (CDD technical performance requirements), Supportability (R&M applied to prognostics and diagnostics), and Life Cycle Cost. The trade-off for the simulation at the end of this lesson will:

Answer the question, ‘what is the most affordable product design and support solution?’

Investigate the best Operational Availability (AO) at the lowest cost with all relevant/quantifiable Product Support Elements considered.


25 of 72


Content

Slide 11-21. Should-Cost

“What are the life cycle product support cost implications in choosing one product support strategy over another?”

“What are the savings over the life cycle by investing in Reliability, Maintainability, and Operational Availability now?”

To answer these questions, the trade-off can leverage specific analytical techniques found in a LORA. This lesson includes factors LCLs should consider that may completely overshadow their trade-off result.

The relationship of Should-Cost and Will-Cost below is paraphrased from the draft Operating and Support Cost Management Guidebook:

“After the 14 September 2010 introduction by Dr. Ashton Carter, USD AT&L of the Will-Should Initiative to the acquisition workforce, many follow on memorandums and articles expanded and clarified affordability initiatives… The inextricable relationship of Will- and Should-Costs contained in the Better Buying Power memorandums drives at a budgetary survival requirement of cultural shifts in management approaches and philosophies. These managing duties must emphasize costs and value within the context of the enterprise’s budgetary constraints.”

Note: This relationship description continues on the next page.



Content

“The AT&L definition of Will-Cost is the “reasonable extrapolations from historical experience.” Simply put, Will-Cost estimates are independent cost estimates created by the CAPE or other independent cost analysis agencies.”

“Going forward, the utility of the Will-Cost estimate is expected to shift. Historical perspectives and accounts indicate that ICEs established a self-fulfilling prophecy that actual costs gravitate towards, as well as a floor upon which cost overruns would be added to above expectations. The new cultural paradigm of the Will-Cost uses this estimate as a ceiling of costs that programs simply cannot exceed. Without the Will-Cost/ICE as the targeted cost, that role is shifted to the Should-Cost.”

“A Should-Cost is a management plan for a program that is developed by the PM and the program‘s complete team of technical experts that identifies, documents, measures, and realizes optimization opportunities within the program that reduce costs without sacrificing performance requirements. This management forum may involve many techniques from simply applying lessons learned, to critical thinking of techniques “outside the box” that generate more efficient ways of conducting business. Fundamentally, PMs are responsible for delivering the required product to the warfighter through the most efficient and cost effective means possible. Those techniques, procedures, and processes that PMs control within the execution of their program are the sources for identifying improvement and efficiency gains.”

“The PM and PSM can develop a Should-Cost estimate by identifying initiatives to realize potential cost optimization opportunities. Each Should-Cost initiative identified within a program must include a specific description of what actions are being done differently from the Will-Cost baseline estimate, and include a full assessment of the resultant targeted cost savings.”

Should-Cost estimates optimize opportunities within the program that reduce costs without sacrificing performance requirements.


27 of 72


Content

Slide 11-22. Affordability and Should-Cost

Affordability and Should-Cost are sometimes difficult to quantify early in programs, particularly Pre-Milestone B. At this point, the Affordability requirement may not be defined based on expected budgets. This is also the time when investigating opportunities to reduce future costs can have the biggest payoff.

As a result, during the early stages of product development, LCLs should establish Affordability constraints and work to achieve the constraints in the ultimate design.

In the early phases of programs, Should-Cost is used to:

Control program overhead Control unproductive expenses Reduce contracted development costs

Note: Should-Cost should be a significant driver for the LCL’s investigations and priorities in forming recommendations for sound investments in product Affordability.



Content

Preliminary Supportability analyses (e.g., R&M analysis) address opportunities for increasing system Reliability. These opportunities are documented in the RAM-C Report that is an Office of the Secretary of Defense mandate that focuses on Reliability requirements and trade-offs early in the design process influence design and cost trade-offs. The focus of the RAM-C Report is to quantify the KPP/KSAs and to establish the trade space between the Threshold and Objective values.

You have just completed Topic 3: Affordability and Should-Cost. Next, you will learn how to complete a trade-off analysis.


29 of 72


Topic 4: Performing Trade-off Analyses

Content

Slide 11-23. Topic 4: Performing Trade-off Analyses

This topic walks you through the trade-off analysis process.



Content

Slide 11-24. Trade-off Analysis Approach

This topic is structured to walk through a trade-off analysis in three phases: Set Up, Analyze, and Report Findings. Each phase illustrates general steps to complete which are appropriate to most trade-offs.


31 of 72


Content

Slide 11-25. Trade-off Analysis Set Up

The trade-off analysis Set Up phase involves capturing and reviewing feedback, building a plan for the trade-off analysis, identifying/selecting a tool, and collecting/validating data. This process helps ensure the trade-off analysis is complete and accurate.



Content

Slide 11-26. Set Up – Feedback Trigger

An IPT receives feedback based on a business rule or other notification that a trade-off analysis between competing priorities is needed. There are many different sources from which an IPT receives feedback:

A logistics or design IPT Formal Technical Reviews, including

o Preliminary Design Review (PDR)o Critical Design Review (CDR)

Results from previous analyses New Market Survey information A Test & Evaluation event

Examples of the feedback types that may indicate a trade-off need include:

A component’s Reliability does not meet threshold Life Cycle Cost projections are high The impact of Technical Data Rights assertions on maintenance,

training, and cost

The ASOE diagram trades Reliability and Sustainment down time and calculates the cost.


33 of 72


Content

Slide 11-27. Set Up – Build Plan

To build a plan, the LCL needs to visualize/consider the entire span of the program. Building a plan involves the first four steps highlighted on the slide:

1. Define trade-off opportunities:o Technical Performance vs. Maintainability vs. Costo Technical Performance vs. Process Efficiency vs. Costo Reliability vs. Maintenance Costo OPTEMPO vs. Maintenance Cost

2. Establish milestones by answering:o What life cycle phase are you in?o What are the requirements you are working toward?

3. Identify stakeholderso It is important for the appropriate IPT(s) to establish:

Roles and responsibilities The analysis goal A schedule/timeline WIPT credentials



Content4. Identify required inputs to the trade-off

o User requirements*o BBP Affordability target*

Contractor labor rateso R&M Modeling and Analysis results-Predictions & Actuals-Mean

Time Between Failure (MTBF), Mean Time To Repair (MTTR)o RCM Analysis Results

Added tasks Updates to existing tasks and their frequency

o Maintenance Task Analysis Special Equipment cost Labor cost Maintenance Replacement Rate

o LORA resultso Sensor, diagnostic, or preventative replacement decisionso Cumulative design trade-off decisionso Risk (cost) of technical data rights uncertainty

An* indicates the most important Supportability-related inputs to a Trade-off Analysis.

Note: The second part of Step 4, Identify Criteria, is addressed on the next slide.

Keep in mind that some Technical Data costs remain unknown if the program does not create an option to purchase them or access them, or does not appropriately counter the Original Equipment Manufacturer’s (OEM’s) Technical Data Rights assertions. In this situation, the program may not have budgeted appropriately for the Technical Data which means the Life Cycle Cost estimate does not account for the Technical Data Rights. Such an oversight presents a real risk to programs that did not address this issue early in the acquisition process. The Government must protect its data rights to assure life cycle support strategies are affordable and effective.

DOD Instruction 5000.02, Enclosure 1, Table 2,2 -- Systems EngineeringProgram Management (https://acc.dau.mil/CommunityBrowser.aspx?id=332558) requires programs to develop an Data ManagementIntellectual Property (IP) Strategy (DMS) and new inputs on Technical DataIntellectual Property Rights. Specifically, Program Managers for ACAT I and II programs, regardless of planned Sustainment approach shall assess the long-term Technical DataIntellectual Property (IP) needs of their systems and reflect that assessment in an Intellectual Property (IP) StrategyDMS.


35 of 72


Updated IP


Content

The Intellectual Property (IP) Strategy DMS shall:

Be integrated with other life cycle Sustainment planning and included in the Acquisition Strategy;

Assess the data required to design, manufacture, and sustain the system, as well as to support re-competition for production, Sustainment, or upgrades; and

Address the merits of including a priced contract option for the future delivery of Technical Data and intellectual property rights not acquired upon initial contract award and shall consider the contractor's responsibility to verify any assertion of restricted use and release of data.

The Intellectual Property (IP) Strategy DMS shall be approved in the context of the Acquisition Strategy prior to issuing a contract solicitation.

Per the Better Buying Power: Guidance for Obtaining Greater Efficiency and Productivity in Defense Spending memorandum from the Office of the Under Secretary of Defense dated September 14, 2010:

“At Milestone B, I (Secretary Ashton Carter) will require that a business case analysis be conducted in concert with the engineering trade analysis that would outline an approach for using open systems architectures and acquiring technical data rights to ensure sustained consideration of competition in the acquisition of weapons systems.”

“A successful example of the strategic use of open architecture and buying of appropriate Technical Data Rights is the Navy’s Virginia-class SSN program. The Virginia program uses a Modular Open Systems Architecture (MOSA) approach and selective subcomponent technical data rights procurement that promotes a robust competition at the component supplier level, while still supporting continual and effective block upgrades to the existing systems that reduces the overall Life Cycle Cost of the system.”



Updated


updated


Content

Slide 11-28. Set Up – Build Plan, Identify Criteria

Criteria are used for judging alternatives (e.g., AO, Reliability investment cost, efficiency, effectiveness, etc.) and are instrumental when making comparisons in a decision matrix. They are the basis for measuring the achievement of planning objectives and levels of important attributes. Effective criteria are:

Directional—have a distinct requirement for the direction optimizing them (i.e., minimize or maximize outcome)

Concise—provide a distinct definition for addressing the analysis goal Complete—cover all aspects of success so that no significant impact

goes unmeasured Clear—define parameters mathematically and identify data and their

sources


37 of 72


Content

Slide 11-29. Set Up – Build Plan, Identify Criteria (cont.)

There are four types of criteria:

Qualitative—a judgment without a numerical measurement outcome (e.g., yes or no, sustainable or unsustainable, acceptable or unacceptable)

Quantitative—a matter of degree (e.g., good, better, or best) or a numerical benefit (e.g., a dollar amount)

Screening—a tactic which qualifies plans for or disqualifies plans from further consideration

Decision—measures that reflect achievement of critical planning objectives and other important plan attributes (e.g., KPPs, KSAs)



Content

Slide 11-30. Set Up – Define Methodology

After the plan is built, a methodology must be defined for the analysis (Step 5 in the graphic). The recommended methodology for performing a trade-off analysis involves:

Creating a decision matrix Transforming the decision matrix data so it is measured on the same

scale; in other words, developing a common basis/unit of measure for comparison

Performing a range transformation so ALL values in the decision matrix range from 0 to 1

Note: This methodology is addressed during the Analyze phase discussion.


39 of 72


Content

Slide 11-31. Set Up – Select Tool

Select the appropriate tool to use for the analysis (Step 6 in the graphic). Examples include:

Systems Planning and Requirements Software (SYSPARS) Cost Analysis Strategy Assessment (CASA) Logistics Cost Estimating Tool (LCET) Acquisition Force Total Ownership (AFTOC) Hypatia COMPASS Wiebull++ LMI Reliability Investment Model

The Trade-off Analysis Tools table provides a list of tools typically used when performing a trade-off analysis.

Table 1. Trade-off Analysis Tools

Tools Description

Systems Planning and Requirements Software (SYSPARS)

Leads the logistician through a series of questions designed to:o Establish program management and Supportability

strategyo Develop the associated tailored program planning

documentation



Tools Description

Cost Analysis Strategy Assessment (CASA)

Is a Life Cycle Cost decision support tool Estimates total cost of system ownership based on the

logistician's input across all phases

Logistics Cost Estimating Tool (LCET)

Estimates the logistics costs for a weapon system Establishes a logistics cost baseline and quantifies cost savings

resulting from improvements and changes to the weapon system and the way it is supported

Evaluates a weapon system's logistics costs associated with different proposals in a source selection

Uses operating hours and Mean Time between Failures (MTBFs) to calculate some of the logistics costs (these inputs may not be appropriate for some weapon systems)

Acquisition Force Total Ownership (AFTOC)

Tracks asset consumption for Cost per Flying Hour (CPFH) Significantly reduces the need for analysts and DoD staff to

acquire, normalize, aggregate, allocate and organize financial and logistics data on Acquisition Force (AF) systems and infrastructure and to provide analytical capability

Provides a single source of authoritative, processed financial and logistics data organized by system or infrastructure

Hypatia Is an end-to-end Life Cycle Cost estimator Serves as a repository for missing essential data elements Generates Should-Cost estimates Assures PM formal process is being followed

COMPASS Determines where items should be removed/replaced and repaired/discarded

Selects the least cost maintenance policy Enables analyst to evaluate costs and Availability associated

with user-defined maintenance policy, over-writing economic recommendation with mandatory policy constraint

Enables trade-off between AO and Life Cycle Costs Provides manpower cost estimations Provides special tool cost estimations Provides item demand consumption costs Enables contractor vs. organic repair cost vs. Availability

evaluations for all repair levels or any specific level

Wiebull++ Characterizes underlying distribution of failures for parts Performs life cycle data analysis using multiple lifetime

distributions Designs Reliability tests Generates Monte Carlo data Estimates parameters of non-linear equations Includes tools for related Reliability analyses (e.g., warranty

data analysis, degradation data analysis, non-parametric data analysis, and recurrent event data analysis


41 of 72


Tools Description

LMI Reliability Investment Model

Provides quantitative estimates for the time and cost required to achieve a given level of Reliability and provides the ability to make minimum-time or minimum-cost plans for coordinated Reliability growth and qualification testing

Develops a preliminary relationship between investment in Reliability and achieved Reliability improvement



Content

Slide 11-32. Set Up – Gather and Import/Validate Data

The last part of the Set Up phase pertains to gathering and importing data (Steps 7 and 8 in the graphic). Data may originate from the:

Logistics Product Database Output from a previous analysis (e.g., LORA) Specifications sheet

After gathering the requisite data, the logistician imports the data into the selected tool and then validates it using a Front End Analysis report to ensure a complete and accurate import. If there are errors, the LCL should coordinate the necessary updates.

A best practice is to repurpose existing models, templates, and data (as appropriate) to streamline the analysis process. In some situations, the Set Up may simply be a revision to an existing model.

Now that you have completed the Set Up phase, continue with the Analysis phase.


43 of 72


Content

Slide 11-33. Trade-off Analysis

The following list includes examples of the types of questions LCLs will answer as part of their analysis:

How will this affect training (e.g., will manuals need to be updated)? Does the new unit require more or less power? What is the new unit’s physical footprint? Does the new unit have storage requirements? What are the life cycle labor requirements? What are the values between the new results and the baseline?

o Rank and balance the alternatives and determine the sensitivity of the results to changes regarding: Parameters/Requirements (e.g., Does the alternative affect

the design? If yes, by how much?) Operational effects (e.g., OPTEMPO) Product support effects Production and O&S costs

Next, you will learn about the decision matrix and how it helps to make an informed decision for selecting the best alternative.



Content

Slide 11-34. Analysis – Construct Matrix/Framework

The overarching goal and the LCL’s desired outcome of a Supportability trade-off analysis is to balance user requirements with the lowest affordable Life Cycle Cost. To that end, it is important to better understand this Analysis phase. To do so, assume that an existing program is presented with four possible alternatives resulting from a trade-off analysis and five criteria. Next, walk through several ways of transforming the data using a decision matrix so that the best alternative becomes clear.

Initial opportunities and problem areas and their alternatives may not be well-defined. The decision matrix often provides a viable way of structuring and presenting the problem and its alternatives to decision makers as well as stakeholders.

A decision matrix:

Summarizes the performance of each alternative for each criteriono Alternatives are listed in the rows (e.g., 1, 2)o Criteria comprise the columns (e.g., Reliability Investment Cost,

Investment Break Even Point) Consists of alternatives from which the best one will be selected Contains values which quantify the performance of each alternative

relative to its criteria


45 of 72


Content

Slide 11-35. Analysis – Run Calculation

Once the decision matrix includes the alternatives and criteria, the next step is to populate it with data. The values entered in the matrix express the performance of each alternative relative to the criteria.

The LCL uses the tool and data identified during the Set Up phase (e.g., COMPASS) to run the necessary calculations for capturing the criteria values for each alternative listed in the decision matrix.



Content

Slide 11-36. Analysis – Analyze Alternatives

The LCL should use the following guidance when analyzing alternatives:

Eliminate any alternative that does not conform to major policies/requirements (e.g., the CDD).o For this example, Alternative 3 is eliminated because its AO is not ≥

80%, per the requirement. If there is a clear/dominant alternative, then re-check the calculations

to ensure they are accurate. If they are, the analysis is complete. If no clear cut winner is apparent, transform calculations into another

scale.

What makes an alternative a clear cut winner?

If all criteria are in the same relative scale and direction, this would represent the final matrix and be suitable for recommending a best alternative.

Cost, years, and availability are all quantitative numeric values, while Obsolescence Risk is degree (e.g., no change, to modest increase). Two decision makers may reach very different conclusions because the matrix components cannot be easily aggregated and definitively compared – some subjectivity is needed to reach a conclusion.

To move the matrix to a more objective form, some transformations of criteria are needed to bring them into similar direction and range.

Next, you will transform the obsolescence risks into numerical values.


47 of 72


Content

Content

Slide 11-37. Analysis – Analyze Alternatives: Revised Alternatives

Since none of the alternatives stand out as a best option, ensure all information in the decision matrix have a numerical value. In this example, the first data to be transformed are the Obsolescence Risk ratings which are now numerical rank values for each alternative.



Content

Slide 11-38. Analysis – Analyze Alternatives: Transform Data

Next, it is necessary to transform the remaining criteria’s numerical values in the decision matrix to values which are measured on the same scale. The goal is to take a series of measurements for a given criterion and convert it into a series of normalized values between zero and one, which permits the values to be summed to determine the best alternative.

This slide illustrates how to change the criteria direction when necessary. For AO, bigger is better and for Cost, bigger is worse. To transform the data:

1. Take the reciprocal of each value in the first three columns to change the criteria directiono Alternative 1 - $8M cost becomes .125, Alternative 2 - $4M cost

becomes .25, and Alternative 4 – $3M becomes .333Now cost is moving in the right direction: Alternative 4 (.333) > Alternative 1 (.125)

o It is now more obvious that Alternative 4 cost is better than Alternative 1 cost – the data is mapped in the right direction


49 of 72


Content

Slide 11-39. Analysis – Analyze Alternatives: Range Transformation

As the Transformed Decision Matrix – By Reciprocal table (top table) on the slide illustrates, there is still no alternative which is obviously better than the other. In this situation, the LCL performs a range transformation so all values range from 0 – 1 as illustrated in the Transformed Decision Matrix – By Range table:

1. Calculate the range: Subtract the smallest value from the largest in each columno In the top table, row four contains the range values

Calculate the transformation: For each cell in the top table, subtract the smallest value in a column and then divide by the range value

o E.g., Alternative 2: (.25-.125)/.208 = .6

The bottom table illustrates the Range Transformation for each column. Now the LCL is able to compare and contrast the alternatives.

Next, you will determine the best alternative.



Content

Slide 11-40. Analysis – Select Best Alternative

Notice the decision matrix now has a sixth column—Sum. Each value in the Sum column is the summation of the values across the corresponding row. Finally, there is a clear, quantifiable alternative to select.

Based upon this cumulative analysis, Alternative 4 is the best option for the program.

Now that the Analysis phase is complete, turn your attention to the Report Findings phase.


51 of 72


Content

Slide 11-41. Trade-off Analysis Report Findings

Report Findings for a trade-off analysis involves updating the trade-off report, briefing the appropriate IPTs, and making appropriate updates.

Trade-off analysis outputs include, but are not limited to, the following:

Understanding that the design meets user requirements and clarifying what has been done to that point

Freezing the design which meets all other specifications Establishing a preferred design Establishing and refining a preferred support concept Refining specification values Revising Life Cycle Cost estimates



Content

Slide 11-42. Report Findings

The logistician is responsible for capturing the analysis findings in the trade-off report. Given the findings of a Life Cycle Cost advantage for one component versus the other, the LCL must provide this information to the IPTs to enable inclusion of this important information in their decision-making and planning processes. Upon receipt of the LCL’s findings, the IPT:

Determines who owns any problems identified Makes recommendations Determines outputs hand-off

Based upon the team’s agreed upon actions, an IPT member(s) updates the following as appropriate:

BCA Logistics Product Database LCSP Section 10: Supportability Analysis

o Trade name and completion dateo Lead IPT/IPPDo Options analyzedo Criteria used to evaluate costs and benefitso Resultso Impact on the weapon system design and/or product support

strategy and package


53 of 72


Content

Slide 11-43. Trade-off Analysis Report Structure

An IPT’s report structure and complexity is driven by the type of trade-off analysis performed. LCLs may report their findings to multiple IPTs; in the illustration above, the analysis results went to the Product Support Management and Risk Management IPTs.



Content

Slide 11-44. Trade-offs and the Strike Talon Case Study

Trade-offs occur and are refined throughout the Life Cycle Management Framework. After a program is fully funded at Milestone C, trade-offs may occur in connection with evolving product support strategy updates. As previously discussed, a Sustainment BCA is required every five years and/or if technology product support or user requirements change significantly.

The Strike Talon Case Study Trade-offs table (page 57) highlights several trade-offs which have occurred prior to this lesson and relates them to the ASOE Model for a contextual point of reference:

Analysis of Alternatives (AoA)—narrow thoughts to a materiel solution and Maintenance Concept

Technology Readiness Level (TRL)Assessment—provide assessment of technological maturity of “must haves” versus “nice-to-haves”

Reliability and Maintainability (R&M) Allocation, Modeling, Prediction, and Analysis—provide a means of differentiating between alternatives

Reliability-Centered Maintenance (RCM) Analysis—provide RCM alternatives based upon acceptable risk

FINAL Level of Repair Analysis (LORA)—finalize design per Maintenance Concept


55 of 72


Content

Slide 11-45. Strike Talon and the ASOE Model

Review the Strike Talon Case Study Trade-offs table on the following page for a review of the five types of trade-offs which occurred as part of the case study. This slide provides a visual indicator of what each trade-off was comparing relative to the ASOE Model:

1. Technical Performance vs. Cost2. Technical Performance vs. Supportability3. Supportability vs. Cost4. Supportability vs. Process Efficiency5. Mission Effectiveness vs. Cost

You have just completed Topic 4: Performing a Trade-off Analysis. Next, you will apply what you have learned throughout this lesson in Topic 5: Exercise & Simulation.



Table 2. Strike Talon Trade-offs

ASOE Model Trade-Off Analytical Method Inputs Outputs

1. Technical Performance vs. Cost

Analysis of Alternatives (AoA) (Pre-Milestone A)

Joint Capabilities Integration & Development System (JCIDS) analysis

Capability-based Assessment

Concept of Operations (CONOPs)

Analysis of Alternatives Study Plan

AoA Report Initial Availability

Assessment Initial Life Cycle Cost

estimate Materiel solution

2. Technical Performance vs. Supportability

Technology Readiness Level (TRL) Assessment

Materiel solution Technology maturity Impact of immature

technology on Reliability, Supportability and Cost

System specification Business Case Analysis

(BCA) KPP/KSA

Thresholds/Objectives

3. Supportability vs. Cost

Reliability and Maintainability (R&M) Allocation, Modeling, and Analysis

System specification Engineering design/data Cost data

Quantification of Operational Availability (AO), Reliability, Cost of Reliability

4. Supportability vs. Process Efficiency

Reliability-Centered Maintenance Analysis (RCM-A)

Decision risk assessment of the interaction between Reliability and maintenance requirements

Maintenance Optimized based on risk

Updated Maintenance Task Analysis (MTA)/maintenance procedures

Maintenance Plan (LCSP)

5. Mission Effectiveness vs. Cost

FINAL Level of Repair Analysis (LORA)

Maintenance Plan (LCSP) Engineering design/data Cost data

Quantification of the design’s Availability and Life Cycle Cost Assignment of maintenance actions at appropriate maintenance level

Identification of support resources and manpower requirements


57 of 72


Topic 5: Exercise & Simulation

Content

Slide 11-46. Topic 5: Exercise & Simulation

Topic 5 provides logisticians an opportunity to apply what they learned in this lesson as well as use the existing LORA results to complete a trade-off analysis specific to the Strike Talon.



Content

Slide 11-47. Exercise & Simulation Scenario

In the Lesson 11: A Trade-off Analysis exercise and simulation, you will perform a Should-Cost analysis of competing Strike Talon radios to determine whether the current radio (Radio #1) or the newly proposed radio (Radio #2) assembly is the best performing choice using relevant decision criteria.

This exercise and simulation leverages the analysis performed on the Strike Talon COM/NAV subsystem in Lesson 10: LORA. This analysis confirmed that the LORA’s AO and Total Maintenance Policy Cost (i.e., Life Cycle Support Cost) outputs for Radio #1 were consistent with the CDD Maintenance Concept requirements (i.e., field removal of the LRU and depot repair of the SRU).


59 of 72


Content

Slide 11-48. Exercise & Simulation Roadmap

As illustrated in the Exercise & Simulation Roadmap above, the analysis performed for this exercise mirrors the analytical steps presented in the lesson: Set Up, Analyze, and Report Findings.

The Exercise corresponds to Set Up and the Simulation to Analyze and Report Findings.



Content

Slide 11-49. Exercise & Simulation Key Question

A best practice when performing a trade-off analysis is to repurpose existing models, templates, and data (as appropriate) to build in efficiencies and streamline the analysis process. To that end, the LORA model developed in Lesson 10: LORA for Radio #1 will be the starting point for this exercise and simulation.


61 of 72


Content

Slide 11-50. Exercise



Content

Slide 11-51. Roadmap: Exercise Set Up


63 of 72


Content

Slide 11-52. Part 1 of Exercise

READ the following key attributes for both Radio #1 (below) and Radio #2 (following page).

Known key attribute for Radio #1:

Radio #1 meets all CDD requirements. Also, Strike Talon, as currently configured, shares Sustainment assets among 20 other UAV programs.

Note: Attributes for Radio #2 are on the following page.



Content

Known key attributes for Radio #2:

Meets all form, fit, and functional requirements inherent in Radio #1o Power requirementso Environmental considerations (e.g., heating/cooling)o Physical dimensions meet UAV requirements

Has 20% better Reliability but has a production cost that is 10% more than Radio #1o Cost does not reflect purchasing technical data

May have transportation costs (e.g., Do a number of them need to be stored for batch retrograde, or are they returned one at a time as they fail?)

Has uncertain technical data rightso Cost of these pending data rights claims is not knowno LCL considers worst case—this legal matter will not be resolvedo Contractor depot repair is available for use as trade-off evaluation

criteria Contractor repair costs were determined from similarly competed

contracts and from Department of Labor labor categories in conformance with Cost Estimation best practices

Note: Technical data is critical to enable organic repair at the Depot level. In this scenario, Radio #2 is more reliable but does not include the appropriate data package to conduct organic Depot-level repair. The Government is constrained to use contractor repair. For this scenario, assume the Government does not purchase technical data for Radio #2.


65 of 72


Content

Slide 11-53. Part 2 of Exercise



Content

Slide 11-54. Roadmap: Simulation – Analyze & Report Findings


67 of 72


Content

Slide 11-55. Simulation

For this simulation, using the existing LORA model significantly accelerates the trade-off process because the set up is simply a revision to the existing model, versus building a whole new one.



Content

Slide 11-56. Debrief


69 of 72


Topic 6: Summary

Content

Slide 11-57. Topic 6: Summary



Content

Slide 11-58. Takeaways

Slide 11-59. Summary

Congratulations! You have completed Lesson 11: A Trade-off Analysis.


71 of 72

A Trade-off Analysis - Defense Acquisition University · Web view2012/12/12 · Finally, there is...

Documents

Transcript of A Trade-off Analysis - Defense Acquisition University · Web view2012/12/12 · Finally, there is...