The integrated master plan and integrated master schedule

The Integrated Master Plan And Integrated Master Schedule

The Integrated Master Plan and Integrated Master Schedule are one of the six elements of a Credible Performance Measurement Baseline (PMB). The PMB is the source of data for the Program Manager. Some of this data is used in the Integrated Program Measurement Report (IPMR). Some is used in the assessment of the Program’s risk. Some define the deliverables and their Technical Performance Measures.

1

5

V8.5

The IMP tells us where is the program going?

The Plan describes where we are going, the various paths we can take to reach our desLnaLon, and the progress or performance assessment points along the way to assure we are on the right path. These assessment points measures the “maturity” of the product or service against the planned maturity. This is the only real measure of progress – not the passage of Lme or consumpLon of money.

The Integrated Master Plan (IMP) Is A Strategy For The Successful Comple=on Of The Project

2

5.0 Start with the IMP

3


Quick View to IMP/IMS

!  VerLcal traceability defines the increasing maturity of key deliverables

!  Horizontal traceability defines the work acLviLes needed to produce this increasing maturity

!  Both are needed, but the verLcal traceability is the starLng point

!  Program Events, Significant Accomplishments, and Accomplishment Criteria must be defined before the horizontal work acLviLes can be idenLfied

!  For all IMP elements, Key Risks must be idenLfied and assigned from Day One, even without miLgaLons

4


The IMP/IMS is Needed on Both Sides of the Contract

!  Ver%cal traceability defines the increasing maturity of the program’s deliverables measures in EffecLveness (MoE) and Performance (MoP) for the Government.

!  Horizontal traceability defines the progress to plan for MoE’s and MoP’s with tangible evidence for both the Government and the Contractor.

!  Ver%cal traceability provides the Government with insight into the progress of the MoE’s and MoP’s and Technical Performance Measures.

!  Horizontal traceability provides insight into Cost and Schedule performance for the Contractor, reportable through the IPMR to the Government.

5


MisconcepLons of the IMP / IMS

Why we don’t need/want an IMP/IMS !  Only required for ACAT I programs !  Too big and burdensome for our small dollar value program !  Contractor spends B&P and program budget generaLng and

maintaining the IMP, without measurable benefit !  Doesn’t apply on a services contractors !  Management tool, not a technical tool !  Doesn’t apply to technology programs !  Doesn’t apply to R&D efforts !  Doesn’t apply to the government !  Help me get a waiver so I don’t have to use an IMP/IMS

6


Aeributes of IMP

!  Traceability –  Expands and complies with the SOO, Performance Requirements, CWBS, and CSOW

–  Based on the customers WBS –  Is the basis of the IMS, cost reports, and award fees

!  Implements a measurable and trackable program –  Accomplishes integrated product development –  Integrates the funcLonal acLviLes of the program –  Incorporates funcLonal, lower level and S/C IMPs

!  Provides for evaluaLon of Program Maturity –  Provides insight into the overall effort –  Level of detail is consistent with risk and complexity per §L –  Decomposes events into a logical series of accomplishments –  Measurable criteria demonstrate compleLon / quality of accomplishments

7


Aeributes of the IMS

!  Integrated, networked, mulL-‐layered schedule of efforts required to achieve each IMP accomplishment – Detailed tasks and work to be completed –  Calendar schedule shows work compleLon dates – Network schedule shows interrelaLonships and criLcal path

–  Expanded granularity, frequency, and depth of risk areas

!  Resource loading !  Correlates IMS work with IMP events

8


The Importance of the IMP

!  The IMP/IMS is the single most important document to a program’s success –  It clearly demonstrates the providers understanding of the program requirements and the soundness of the approach a represented by the plan

!  The program uses the IMP/IMS to provide: –  Up Front Planning and commitment from all parLcipants –  A balanced design discipline with risk miLgaLon acLviLes –  Integrated requirements including producLon and support – Management with an incremental verificaLon for informed program decisions

9


Just A Reminder, before moving on

10

Page 47, Defense AcquisiLon Guide, January 10, 2012

Page 317, Defense AcquisiLon Guide, January 10, 2012


Our Goal is simple …

How can we recognize the Reality of the Program’s current status and its future performance?

The top spins conLnuous while in a dream – stops spinning in the real world – Cobb’s totem, Incep=on

11


Principles of Building a Credible Integrated Master Plan

Building the IMP is a Systems Engineering ac<vity. The Integrated Master Plan is the Program Architecture in the same way the hardware and soBware are the Product Architecture. Poor, a weak, or unstructured Programma<c Architecture reduces visibility to the Product Architecture’s performance measures of cost and schedule connected with Technical Performance Measures.

1

6

V8.5

Quick View of Building the IMP

!  Start with each Program Event and define the Significant Accomplishments their entry and exit criteria to assess the needed maturity of the key deliverables

!  Arrange the Significant Accomplishments in the proper dependency order

!  Segregate these Significant Accomplishments into swim lanes for IPTs

!  Define the dependencies between each SA 2

6.0 Build IMP

A Cri<cal Understanding of the IMP

3

The IMP defines the connec<ons between the Product maturity – Ver<cal – and the implementa<on of this Product maturity through the Func<onal ac<vi<es – the Horizontal

6.0 Build IMP

Benefits of this Formality

4

Objec&ve Implementa&on Event Driven Plan versus Schedule Driven Plan is based on comple<on of tasks not passage of <me

Separate the plan (IMP) from the schedule (IMS) but link elements with numbering system

Condensed, easy to read “Plan” showing the “Events” rather than the work effort Indentured, outline format – not text

Pre-‐defined entry and exit criteria for major Program Events

Significant Accomplishments for each key Program Event

Objec<ve measures of progress and comple<on for each Accomplishment

Pre-‐defined Accomplishment Criteria (AC) for each Significant Accomplishment (SA)

Stable, contracture plan, flexible enough to portray program status

IMP part of the contract, IMS is a data item

Capture essence of the func<onal progress without manda<ng a par<cular process for performing the work

Split IMP into Product and Process

6.0 Build IMP

Risk Management

Building the IMP Starts at the RFP

5

SOW SOO

ConOps WBS Techncial and Opera<onal

Requirements

CWBS & CWBS Dic<onary

Integrated Master Plan (IMP)

Integrated Master Schedule (IMS)

Earned Value Management System

Objec<ve Status and Essen<al Views to support the proac<ve management processes needed to keep the program GREEN

Performance Measurement Baseline

Measures of Effec<veness

Measures of Performance

Measures of Progress

JROC Key Performance Parameters

Program Specific Key Performance Parameters

Technical Performance Measures

1.0 Introduc<on

The IMP / IMS Structure

6

IMS

IMP Describes how program capabilities will be delivered and how these capabilities will be recognized as ready for delivery

Supplemental Schedules

Work Packages and Tasks

Criteria

Accomplishment

Events or

Milestones

1

6.0 Build IMP

The IMP/IMS provides Horizontal and Ver<cal Traceability of progress to plan

!  Ver<cal traceability AC " SA " PE !  Horizontal traceability WP " WP " AC

7

Program Events Define the maturity of a Capability at a point in time.

Significant Accomplishments Represent requirements that enable Capabilities.

Accomplishment Criteria Exit Criteria for the Work Packages that fulfill Requirements.

Work Package

Work Package

Work Package

Work Package

Work Package

Work Package

Work package

6.0 Build IMP

The IMP’s role during Execu<on

8

Program Execution PMB for IBR Proposal Submittal DRFP & RFP

Performance Measurement Baseline

Tasks (T)

BOE

% Complete

Statement of Work

Program Deliverables

IMP

Accomplishments (A)

Criteria (C)

EVMS

Events (E)

Budget Spreads by CA & WP CAIV

Capabilities Based Requirements

X BCWS =

Probabilistic Risk Analysis

=

Time keeping and ODC =

Technical Performance Measure

BCWP

ACWP

Cost & Schedule Risk Model

BCWS

Decreasing technical and programmatic risk using Risk Management Methods

IMS

Physical % Complete

Continuity and consistency from DRFP through Program Execution

WBS

6.0 Build IMP

1st, Principle – IMP Building is a Full Contact Sport

9

6.0 Build IMP

Our First Approach to the IMP / IMS Paradigm

!  The 1st approach defines Program Events (PE), Significant Accomplishments (SA), and Accomplishment Criteria (AC), derived from the Work Breakdown Structure or the SOW

!  This 1st approach can be done in 6 easy steps

10

6.0 Build IMP

1.  Iden<fy the Program Events (PE) (as the ACQ Guide tells us)

2.  Iden<fy the Significant Accomplishments (SA) from the WBS deliverables

3.  Iden<fy the Accomplishment Criteria (AC) needed to produce these deliverables

4.  Iden<fy the Work Packages for each Accomplishment Criteria

5.  Sequence the Work Packages 6.  Assemble the IMP/IMS

This approach doesn’t give us visibility into what “done” looks like

!  We must measure increasing product maturity in units meaningful to the decision makers

!  We must see the risks before they arrive so we can take correc<ve ac<on

11

6.0 Build IMP

First Look at a Significant Accomplishment (SA)

!  SAs are interim and final steps to define, design, develop, verify, produce, and deploy the product or system.

!  SAs must occur in a manner that ensures a logical path is maintained throughout the development effort.

!  SAs are event related and not just <me coincidental. !  SAs should have one or more of the following

characteris<cs: –  Consists of a discrete step in the process of planned development that must be complete prior to an event

–  Produces a desired result at a specified event that indicates a level of design maturity (or progress) directly related to each product and process

–  Defines interrela<onships, interdependencies, or “hand-‐off” points of different func<onal disciplines applied to the program

12

6.0 Build IMP

First look at an Accomplishment Criteria (AC)

!  ACs are defini<ve measures directly suppor<ng successful comple<on of a significant accomplishment.

!  ACs show objec<ve evidence of work progress (maturity of a product); i.e., be seen, read, demonstrated, or quan<fied. These results are usually incorporated into a report or document as evidence of accomplishment.

!  ACs are prerequisites for comple<on of an SA (i.e., exit criteria).

!  The ques<ons that need to be repeatedly asked when developing ACs are: –  How do I know when an accomplishment has been completed? –  Is the criteria directly related to the accomplishment? –  Is it proof? –  What is the work product?”

13

6.0 Build IMP

The IMP speaks to Measures of Effec<veness (MoE) and Measures of Performance (MoP)

14

6.0 Build IMP

!  This is where TPMs are connected with the MoE’s and MoP’s

!  For each deliverable from the program, all the “measures” must be defined in units meaningful to the decision makers.

! Here’s some “real” examples.

1.  Provide Precision Approach for a 200 FT/0.5 NM DH

2.  Provide bearing and range to AC plaporm

3.  Provide AC surveillance to GRND plaporm

Measures of Effec<veness (MoE)

1. Net Ready 2. Guidance Quality 3.  Land Interoperability 4. Manpower 5. Availability

JROC Key Performance Parameters (KPP)

1. Net Ready !  IPv4/6 compliance ! 1Gb Ethernet

2. Guidance quality ! Accuracy threshold p70 @ 6M

!  Integrity threshold 4M @ 10-‐6 /approach

3.  Land interoperability ! Processing capability meets LB growth matrix

4. Manpower ! MTBC >1000 hrs ! MCM < 2 hrs

5. Availability ! Clear threshold >99% !  Jam threshold >90%

Measures of Performance (MoP)

1. Net Ready ! Standard message packets

2. Guidance Quality ! Mul<path alloca<on budget ! Mul<path bias protec<on

3.  Land Interoperability ! MOSA compliant ! Civil compliant

4. Manpower ! Opera<ng elapsed <me meters

! Standby elapsed <me indicators

5. Availability ! Phase center varia<ons

Technical Performance Measures (TPM)

Mission Capabili<es and Opera<onal Need

Technical Insight – Risk adjusted performance to plan

The 1st Problem with the Ini<al IMP/IMS Paradigm

!  There is no single source of guidance for construc<ng a credible IMP and IMS – A DOD Guidebook, but s<ll in Version 0.9 – A few DOD service pamphlets – A commercial guidebook – Many contractor guidelines

!  No defini<ve guidance from DoD on what makes a good IMP

!  No defini<ve DID requiring an IMP on specific programs

15

6.0 Build IMP

The IMP is a good start, but we’re really aBer the IMP Narra<ves

!  The IMP Narra<ves start with the Statement of Objec<ves (SOO) or Concept of Opera<ons (ConOps) – These iden<fy the top level program objec<ves – They define the big picture and provide pre-‐award trade space

– They provide the framework for the Contractor to develop the proposal through the IMP

! With the Government’s Execu<ve summary, provides the contractor an understanding of what is needed and what is important

16

6.0 Build IMP

The IMP Process Narra<ve

The IMP Process Narra<ve describes how the technical and business elements of the program will be conducted, monitored, and controlled. Narra<ves provide the customer visibility into the contractor’s key func<onal processes and procedures, the rela<onship of these processes and procedures, and an overview of the effort required to implement them.

17

6.0 Build IMP

IMP Narra<ve for PDR Program Event Event Descrip&on PE Maturity Assessment

shown in SAs

PDR PDR establishes the “design-‐ to” allocated baseline to the subsystem level, assures this design meets the func<onal baseline, assures system requirements have been properly allocated to the proper subsystem. PDR establishes the feasibility of the design approach to meet the technical requirements and provide acceptable interface rela<onships between the hardware and other interfacing items. Any changes to the requirements that have occurred since the System Requirements Review (SRR2) will be verified at the PDR. PDR assures the design is verifiable, does not pose major IMS or Cost risk, and is mature enough to advance to the detailed design phase – CDR

!  Subsystem level opera<onal concepts defined

!  System level interfaces baselined

!  Supportability plans established

!  SoBware requirements finalized

!  Subsystem requirements finalized & allocated

!  System verifica<on, valida<on & cer<fica<on plans updated

!  PDR subsystem design completed

18

6.0 Build IMP

What the IMP Narra<ve Tells Us?

!  States the objec<ve of the processes used to build the products

!  Provides the governing documents, compliance, and reference for the process ac<vi<es

!  Explains the process approach –  Portrays the key ac<vi<es of the approach –  Illustrates the processes tailored for the specific program

19

Inputs

Ac<vity 3

Ac<vity 5

Ac<vity 4

Tools

Ac<vity 1

Ac<vity 2

Outputs

Metrics

6.0 Build IMP

Some Guidance

20

6.0 Build IMP

The Defense Acquisi<on Guide (DAG) Tells Us About the IMP

21

6.0 Build IMP

The DAG Also Tells Us

22

6.0 Build IMP

5000.01 Part 2.B.3 Acquisi<on Strategies, Exit Criteria, and Risk Management

“Event driven acquisi<on strategies and program plans must be based on rigorous, objec<ve assessments of a program’s status and the plans for managing risk during the next phase and the remainder of the program. The acquisi<on strategy and associated contrac<ng ac<vi<es must explicitly link milestone decision reviews to events and demonstrated accomplishments in development, tes<ng, and ini<al produc<on. The acquisi<on strategy must reflect the interrela<onships and schedule of acquisi<on phases and events based on logical sequence of demonstrated accomplishments not on fiscal or calendar expediency.”

23

6.0 Build IMP

What makes a good Program Event (PE)?

!  Events are the conclusion of an interval of major program accomplishments (SA) with their criteria (AC)

!  IMP events represent key decision transi<on points between major ac<vi<es distributed over the contract period –  The IMP is a Mini Authoriza;on’s to Proceed (ATP) to the next Program Event (PE)

!  Some guidance for establishing program/product events: –  Customer Given Events. –  Key Decisions Needed –  Risk Mi<ga<on Event –  DOD Systems Engineering Technology Review (SETR) Guidance

24

6.0 Build IMP

What makes a good Significant Accomplishment (SA)?

!  IPT can manage it at a working level !  Shows comple<on and result s of discrete steps in the

development process !  Indicates maturity of the product through MoEs and MoPs !  Its “significance” measures program event status !  Relevant and logically linked to the proper PE

–  Just because the work occurs during the <me-‐frame for PE A doesn’t mean it’s logically linked to PE A

!  Uses consistent language, style, format through verb dic<onary, for example: –  Segment build 4 detailed design completed –  Analysis of structural integrity completed –  Structural integrity verified

25

6.0 Build IMP

IMP Significant Accomplishments

!  SAs are NOT just a list of “things” to do before the Program Event (PE)

!  They are sequenced accomplishments, each of which leads to the PE, e.g. Cri<cal Design Review (CDR) –  SA # 1 = CDR mee<ng conducted –  SA # 2 = CDR ac<on item work-‐off plan established –  SA # 3 = 85% drawings completed –  SA # 4 = CDR CDRLs delivered –  SA # 5 = Development environment opera<onal –  SA # 6 = Cri<cal methods analyses completed –  SA # 7 = RVTM approved

26

6.0 Build IMP

What makes a good Accomplishment Criteria (AC)?

!  Provides objec<ve, measureable and, explicit proof of comple<on and closure of the work ac<vi<es

!  Defines condi<ons for closing the Significant Accomplishment (SA)

!  Answers the ques<on how do we know when a Significant Accomplishment has been completed?

27

6.0 Build IMP

!  Not Significant –  Too small to significantly contribute to successful event comple<on –  Would lead to trivial tasks (e.g., 1 day dura<on)

!  Ambiguous –  Read can’t tell what Done looks like

!  Wrong verb –  Uses a verb that’s not on the list (Dic<onary) –  Uses a listed verb incorrectly –  Doesn’t have a verb at all

!  Not measurable –  Can’t tell when we’re done

!  Too Many SAs or ACs –  Confuses the reader and confuses the execu<on process of the

program –  Dilutes the MoE and MoP –  Reduces visibility into increasing maturity

28

What makes a Not So Good SA or AC?

6.0 Build IMP

!  Program Event (PE) – A PE assess the readiness or comple<on as a measure of progress

–  First Flight Complete !  Significant Accomplishment (SA)

–  The desired result(s) prior to or at comple<on of an event demonstrate the level of the program’s progress

–  Flight Test Readiness Review Complete !  Accomplishment Criteria (AC)

– Defini<ve evidence (measures or indicators) that verify a specific accomplishment has been completed

–  SEEK EAGLE Flight Clearance Obtained 29

F-‐22 Example 6.0 Build IMP

!  PE’s are easy, they are in the SETR and Integrated Logis<cs Lifecycle Management System

!  The SA’s can be defined from the program’s deliverables – these may not be obvious but can be discovered with a Product Development Kaizen process (more later)

!  It’s the AC’s that are hard part – the ACs must represent the exit criteria for the series of Work Packages that do the work to produce the product

30

Now the Hard Part 6.0 Build IMP

6 Steps to IMP Development

Let’s start to build the IMP. This step-‐by-‐step process needs to be followed carefully. The IMP is constructed one Program Event at a Cme – LeE to Right in Cme. To do otherwise allows confusion and disconnecCon between Program Events to occur and dilutes our focus on defining what Done looks like for each Program Event.

1

7

V8.5

2

7.0 6 Steps

Quick View of Step-‐By-‐Step IMP

IdenCfy Program Events IdenCfy Significant Accomplishments IdenCfy Accomplishment Criteria IdenCfy Work Packages needed to complete the Accomplishment Criteria Sequence the Work Packages (WP), Planning Packages (PP), Summary Level Planning Packages (SLPP) in a logical network. Adjust the sequence of WPs, PPs, & SLPPs to miCgate major risks.

3

7.0 6 Steps

1

2

3

4

5

6

IdenCfy the Program Events

Actors Processes Outcomes

Systems Engineer

Define the process flow for product producCon from contract award to end of contract

!  Confirm Program Events represent the logical process flow for program maturity

Program Manager Confirm customer is willing to accept the process flows developed by the IMP

!  Engage with contracts and customer for PE definiCon

Project Engineer IdenCfy interdependencies between program event work streams

!  IdenCfy Value Stream components at the PE level before flowing them down to the SA level

IMP/IMS Architect Capture Program Event contents for each IPT or work stream

!  Establish foundaCon for a structure to support the descripCon of the increasing mature as well as the flow to needed work.

Copyright © 2012, Glen B. Alleman, Niwot Ridge, LLC 4

1

7.0 6 Steps

Outcomes of Step

!  Confirm the end to end descripCon of the increasing maturity of the program’s deliverables

!  Establish of RFP or Contract target dates for each Event.

!  Socialize the language of speaking in “Events” rather than Cme and efforts

5

1

7.0 6 Steps

Events Define the Assessment of the Program’s Maturity

6

! Program Events are maturity assessment points in the program

! They define what levels of maturity for the products and services are needed before proceeding to the next maturity assessment point

! The entry criteria for each Event defines the units of measure for the successful compleCon of the Event

! The example below is typical of the purpose of a Program Event

The Cri+cal Design Review (CDR) is a mul+-‐disciplined product and process assessment to ensure that the system under review can proceed into system fabrica+on, demonstra+on, and test, and can meet the stated performance requirements within cost (program budget), schedule (program schedule), risk, and other system constraints.

1

7.0 6 Steps

IdenCfy the Significant Accomplishments (SA) for Each Program Event (PE)

7


System Engineer IdenCfy Integrated Product Teams (IPT) responsible for the SA’s

! Define the boundaries of these programmaCc interfaces

! Define technical and process risk categories and their bounds

Technical Lead Confirm the sequence of SA’s has the proper dependency relaConships

! Define the product development flow process improves maturity

! Define technical risk drivers

Project Engineer Confirm logic of SA’s for project sequence integrity

! Define the program flows improves maturity

Control Account Manager

Validate SA outcomes in support of PE entry condiCons

!  Confirm budget and resources adequate for defined work effort

IMP/IMS Architect

Assure the assessment points provide a logical flow of maturity at the proper intervals for the program

! Maintain the integrity of the IMP, WBS, and IMS

2

7.0 6 Steps

Outcomes of Step

!  The Significant Accomplishments are the “road map” to the increasing maturity of the program

!  The “Value Stream Map” resulCng from the flow of SA’s describes how the products or services move through the maturaCon process while reducing risk

!  The SA map is the path to “done”

8

2

7.0 6 Steps

SAs define the entry criteria for each Program Event

9 Preliminary Design Review Complete

3

7.0 6 Steps

IdenCfy Accomplishment Criteria (AC) for each Significant Accomplishment (SA)


CAM

Define and sequence the contents of each Work Package and select the EV criteria for each Task needed to roll up the BCWP measurement

!  Establish ownership for the content of each Work Package and the Exit Criteria – the Accomplishment Criteria (AC)

Project Engineer

IdenCfy the logical process flow of the Work Package to assure the least effort, maximum value and lowest risk path to the Program Event

!  Establish ownership for the process flow of the product or service

Technical Lead Assure all technical processes are covered in each Work Package

!  Establish ownership for the technical outcome of each Work Package

IMP/IMS Architect

Confirm the process flow of the ACs can follow the DID 81650 structuring and Risk Assessment processes

! Guide the development of outcomes for each Work Package to assure increasing maturity of the program 10

3

7.0 6 Steps

Outcomes of Step

!  The definiCon of “done” emerges in the form of deliverables rather than measures of cost and passage of Cme.

!  At each Program Event, the increasing maturity of the deliverables is defined through the Measures of EffecCveness (MoE) and Measures of Performance (MoP)

11

3

7.0 6 Steps

ACs are higher fidelity descripCons of “Done” than SAs

12 Cri9cal Design Review Complete

4

7.0 6 Steps

IdenCfy the Work for Each Accomplishment Criteria in Work Packages



IdenCfy or confirm the work acCviCes in the Work Package represent the allocated work

! Define bounded work effort defined “inside” each Work Package

Technical Lead Confirm this work covers the SOW and CDRLs

! Define all work effort for 100% compleCon of deliverable visible in a single locaCon – the Work Package

!  Confirm risk drivers and duraCon variances

IMP/IMS Architect Assist in the sequencing the work efforts in a logical manner

! Develop foundaCon of the maturity flow starCng to emerge from the contents of the Work Packages

Earned Value Analyst

Assign iniCal BCWS from BOE to Work Package

!  ConfirmaCon of work effort against BOEs

! Define EVT for measures progress to plan 13

4

7.0 6 Steps

Outcomes of Step

!  The work idenCfied that produces a measurable outcome.

!  This work defined in each Work Package !  The Accomplishment Criteria (AC) state explicitly what “done” looks like for this effort

! With “done” stated, measures of Performance and measures of EffecCveness can be defined

14

4

7.0 6 Steps

Work is done in “packages” that produce measureable outcomes

15 Launch Readiness Review Complete

5

7.0 6 Steps

Sequence Work Packages (ACs) for each Significant Accomplishment (SA)

16



Define the order of the Work Packages needed to meet the Significant Accomplishments for each Program Event

! Define the process flow of work and the resulCng accomplishments.

! Assure value is being produced at each SA and the AC’s that drive them

IMP/IMS Architect

Assure that the sequence of Work Packages adheres to the guidance provided by DCMA and the EVMS System descripCon

!  Begin the structuring of the IMS for compliance and loading into the cost system

Program Controls Staff

Baseline the sequence of Work Packages using Earned Value Techniques (EVT) with measures of Physical Percent Complete

! Develop insight to progress to plan with measures of physical progress for each Work Packages (EVT)

5

7.0 6 Steps

Outcomes of Step

! Work Packages parCCon work efforts into “bounded” scope

!  Interdependencies constrained to Work Package boundaries prevents “spaghek code” style schedule flow

!  Visibility of the Increasing Flow of Maturity starCng to emerge from the flow of Accomplishment Criteria (AC)

17

5

7.0 6 Steps

Sequence Work Packages (AC’s) into an IMS for each Program Event

18

6

7.0 6 Steps

Assemble Final IMP/IMS Actors Processes Outcomes

IMP/IMS Architect

StarCng with the AC’s under each SA’s connect Work Packages in the proper order for each Program Event

!  Establish the Performance Measurement Baseline framework.

!  IdenCfy MoE and MoP points in the IMP

Program Manager Confirm the work efforts represent the commiled acCviCes for the contract

!  Review and approval of the IMS – ready for baseline.

!  Review and approve risk drivers and duraCon variance models

Project Engineer Assess the product development flow for opCmizaCons

!  Review and approval of the IMS – ready for baseline.

!  IdenCfy risk drivers and their miCgaCons

Systems Engineer Confirm the work process flows result in the proper products being built in the right order

!  Confirm risk drivers and duraCon variances.

!  Review and approval of the IMS – ready for baseline 19

6

7.0 6 Steps

Outcomes of Step

!  Both the maturity assessment criteria and the work needed to reach that level of maturity are described in a single locaCon

!  Risks are integrated with the IMP and IMS at their appropriate levels –  Risks to EffecCveness – risk to JROC KPPs –  Risks to Performance – risk to program KPPs and TPMs

!  Leading and Lagging indicator data provide through each measure to forecast future performance

20

6

7.0 6 Steps

The Previous 6 Steps Result In A Credible IMP/IMS

21

!  The IMP is the “Outer Mold Line”, the Framework, the “Going Forward” Strategy for the Program.

!  The IMP describes the path to increasing maturity and the Events measuring that maturity.

!  The IMP tells us “How” the program will flow with the least risk, the maximum value, and the clearest visibility to progress.

!  The IMS tells us what work is needed to produce the product or service at the Work Package level.

Our Plan Tells Us “How” We are Going to Proceed The Schedule Tells Us

“What” Work is Needed to Proceed

7.0 6 Steps

Significant Accomplishments for an actual Program Event Flight Test Ascent Aerodynamics Confirmed

22

7.0 6 Steps

Horizontal and VerCcal Traceability of the IMP/IMS

Integrated Master Schedule Work sequenced to produce outcomes

for each WP.

!  VerCcal traceability AC " SA " PE !  Horizontal traceability WP " WP "AC

Program Events Define the maturity of a Capability at a point in time.

Significant Accomplishments Represent requirements that enable Capabilities.

Accomplishment Criteria Exit Criteria for the Work Packages that fulfill Requirements.

Work Package

Work Package

Work Package

Work Package

Work Package

Work package

Work Package

Work Package

23

7.0 6 Steps

The IMP’s connecCon to the WBS !  Start with the Significant Accomplishments and sequence

them to the maturity flow for each Program Event !  The WBS connecCons then become orthogonal to this flow

24

7.0 6 Steps

Program Event

SRR SDR PDR CDR TRR ATLO Work Breakdown Structure

4.920-‐SDAI A01, A02 B01 C01, C02 D01 E01 F01

4.200-‐Sys Test A05 B03, B04 D02, D03 E02 F02

4.300-‐Radar A03 B02 C03 E03

4.330-‐O&C Sys A06, A07 B05 C04 D04 E04 F03, F04

4.400-‐ I&T A08 C05 E05, E06 F05

4.500-‐Support A09 D05 E07 F06, F07

25

7.0 6 Steps

26

7.0 6 Steps

27

7.0 6 Steps

28

7.0 6 Steps

29

7.0 6 Steps

30

7.0 6 Steps

Nuances Of These 6 Steps

Building the Program Event, to Significant Accomplishment, to Accomplishment decomposi?on is straight forward. For each Program Event, simply iden?fy what are the needed Significant Accomplishment for the entry and exit criteria, and the Accomplishment Criteria for the Work Packages that produce the AC.

Yea Right, no problem

1

8

V8.5

2

8.0 Nuances

Quick View of the Nuances

!  Unfortunately building a credible IMP/IMS is a nuanced process, subject to many opportuni?es for diversions, blind alleys, and false starts

!  It is slightly counter intui?ve from the tradi?onal scheduling approach to start with the ver?cal integra?on – but it is cri?cal to start ver?cally

!  Success requires the full par?cipa?on of Systems Engineering, CAMs, and the Program Manager

!  Success requires everyone to understand the nuances of the IMP building efforts

3

8.0 Nuances

The 1st Nuance We need to change the Planning Paradigm

Beginner Intermediate Advanced

! Take the program events and use the WBS to define the SAs and ACs.

!  Iden?fy the tasks to produce the ACs and SAs

! Collect them into Program Events

! Organize the tasks by Work Package

! Sequence the work packages (AC’s)

! Examine the exit criteria for the Program Events – what does Done look like?

! Ask what the entry criteria are for each Program Event

! Build the AC’s to support these entry criteria

! Pull these together under each SA

! Determine the Technical and Programma?c maturity for each Program Event from the Concept of Opera?ons

! Assess the SA’s for each Integrated Product Team in terms of their streams maturity at that point in the program

! Sequence the SA’s for each PE and assess the units of measure of “maturity”

! Build the AC’s to support each SA’s level of maturity


8.0 Nuances

The 2nd Nuance We need to speak about Increasing Maturity


! The sequence of work closely matches the horizontal schedules of the past

! Align this work with the WBS elements

! Focus on the WBS Dic?onary of the delivered products or services

! No explicit TPM, MOE, and MOP elements

! The sequence of work is related to the Program Events, but essen?ally “hangs” from the PE to the SA’s and then the AC’s

! All deliverables are visible but their TPMs and other system measures are not stated in the IMP or its narra?ve.

! There is a narra?ve in the form of SA’s and AC’s that describes how the program moves from lef to right alone its maturity path

! Risk buy down and re?rement are visible

!  Intermediate Technical Performance Measures, Measures of Effec?veness, and Measures of Performance are visible in the IMP


8.0 Nuances

The 3rd Nuance Everything Foot and Ties to the IMP & IMS


! The IMS contains all the proper fields in columns and is horizontally linked

! The WBS elements can be found for all work elements

! CDRL’s are visible and their mul?ple delivery dates connected to each Program Event

! WBS is structured in a product manner or possibly a func?onal manner with some deliverables defined in the terminal nodes

! The WBS is properly formed inside each AC with incremental deliverables

! WBS numbers form a “well structured” tree, but s?ll is not “pure” in the sense of deliverables only, no func?onal

! Each column and each field can be “pivoted” to form a proper “tree” of value flow.

! The WBS is a “pure” Product Breakdown Structure (PBS) and the services needed to produce those products

! The WBS defines the structure of the delivered product or service

! The Ver?cal trace of the IPM describes the flow of increasing maturity of these products or services

! The Horizontal trace of the IMP describes to work to be done to produce this maturity 6

8.0 Nuances

The 4th Nuance IMP/IMS is Programma?c Architecture


! The IMP is built from the WBS for each Program Event.

! The IMP is seen as a compliance document that lists the Program Events and a “bunch of stuff” underneath.

! The IMP is structured around separate Program Events, but below the SA’s looks like a “shop floor” schedule with lijle ver?cal connec?vity.

! The IMP is built as a “value stream” flow for the program but the Systems Engineers

! This programma?c architecture is built in the same way the technical system architecture is built

!  It is derived from the ConOps and Tier 1 System Requirements

! The IMP shows explicitly how these are supported in the flow of the SA’s

7

8.0 Nuances

The 5th Nuance The IMP/IMS connects all the dots

8

8.0 Nuances

Measure of Effec?veness

Measure of Performance

Technical Performance Measure

Risk

Aleatory Uncertainty

Epistemic Uncertainty

Reference Classes

Past Performance

SME

Past Performance

System Architecture

AHP

Connec&ng The IMP To Program Performance Measures

Assembling the IMS from the IMP appears to be a straight forward process – details the tasks that support the Accomplishment Criteria. But there are some cri&cal steps that must be done in the right order to end up with a risk tolerant IMS.

Let’s do this for TSAS

1

9

V8.5

The Primary Role for the IMP is to describe what done looks like in MoE’s and MoP’s

2 19 October 1899 Robert Goddard decided that he wanted to "fly without wings" to Moon.

9.0 Framework

Quick View to IMP/IMS Framework

! Measures of increasing maturity for the key deliverables is the founda&on for increasing the Probability of Program Success

! Measures of Effec&veness (MoE) and Measures of Performance (MoP) are defined in the Integrated Master Plan (IMP) Narra&ve

!  Key Performance Parameters (KPP) – both JROC and program specific are needed

!  Technical Performance Measures (TPM) are needed for all key deliverables

3

9.0 Framework

Star&ng out on the Right Foot …

!  Most IMP/IMS literature states how to build an IMP from the RFP and contractual elements, in simple and maybe simple minded terms –  Decompose the events into SAs, ACs, and their Tasks – sounds easy

!  This approach fails to provide advice for several things: –  How to minimize the topological connec&ons between Events –  How to increase the concurrency between IPTs –  How to increase the tolerance of the IMS to disrup&ve events

•  Known and knowable risk •  Unknown and possibly unknowable risk

!  The construc&on of the IMS needs to take place in what seems to be a reverse order –  Build the IMP as a Value Stream Map describing the increasing maturity – and

therefore the increasing VALUE of the deliverables to the customer. –  It’s the delivery of Customer Value that inverts the management process, and

focuses on Keeping the Program GREEN as planned that maximizes value to the customer (the Government)

4

9.0 Framework

SETR Program Events

5 hdps://acc.dau.mil/docs/technicalreviews/dod_tech_reviews.htm

9.0 Framework

Build PEs Leg to Right Start with SRR (or something on the leA) and completely define its compleDon, before moving to the next PE

!  Define the SAs for an Event and construct a work flow of the ac&vi&es needed to sa&sfy the SA –  These ac&vi&es are yet tasks, so don’t commit too soon to defining the detailed work

–  Isolate the SAs by event first – only work on one event at a &me

!  Iden&fy the par&cipants in the work – What IPTs par&cipate in this work? – What swim lanes are needed to isolate the IPTs?

!  Define the elements –  Ac&vi&es performed to sa&sfy the SA –  Deliverables that result from these ac&vi&es

!  There are s&ll not Accomplishment Criteria (AC), but that comes next

6

9.0 Framework

The Accomplishment Criteria (AC)

!  A defini&ve measure or indicator that verifies comple&on of work for the accomplishment – Completed work effort

•  Manufacturing Plan Completed – Confirma&on of performance compliance

•  Flight Test Report Approved –  Incremental verifica&on

•  Maintenance Demonstra&on Completed – Completed cri&cal process ac&vi&es

•  Risk Management Plan Approved

7

9.0 Framework

The Accomplishment Criteria (AC)

!  Defines the measure by which an Accomplishment (SA) is considered “done”

!  Terms like complete, delivered, closed have no “units of measure” in the context of a Significant Accomplishment (SA) and are open to interpreta&on

!  Terms like … –  Measures of comple&on – 80% of drawings approved for release –  Counts of available items – 75% of pin-‐outs assigned voltage –  Fidelity of a design – outer mold line defined within 90% of target

–  Error bounds – spacecraA mass known to ±20% –  Performance parameters – disconnect force within allowed limits

–  Maturity parameters – flight arDcle successful in last 3 tests … are used to define the “exit criteria”

8

9.0 Framework

2 Types of Accomplishment Criteria: Entry and Exit

!  Entry Criteria – Substan&ates readiness for the review

!  Exit Criteria – Substan&ates successful comple&on of the review

!  Cri&cal Design Review (CDR) example – Are we ready for the Flight Test Readiness Review?

– How do we know the FTRR a success? – What did we learn from the FTRR that increases the maturity of the program’s deliverables.

9

9.0 Framework

The IMP Focuses us on Measures of Effec&veness and Performance

10

MoE

KPP

MoP TPM Mission Need

Acquirer Defines the Needs and Capabili&es in terms of Opera&onal Scenarios

Supplier Defines Physical Solu&ons that meet the needs of the Stakeholders

Opera3onal measures of success related to the achievement of the mission or opera3onal objec3ve being evaluated.

Measures that characterize physical or func3onal aAributes rela3ng to the system opera3on.

Measures used to assess design progress, compliance to performance requirements, and technical risks.

9.0 Framework

Measure of Effec&veness (MoE)

! Measures of Effec&veness … !  Are stated in units meaningful to the buyer, !  Focus on capabili&es independent of any technical implementa&on,

!  Are connected to the mission success.

The opera&onal measures of success that are closely related to the achievements of the mission or opera&onal objec&ves evaluated in the opera&onal environment, under a specific set of condi&ons.

“Technical Measurement,” INCOSE–TP–2003–020–01

MoE’s Belong to the End User

11

9.0 Framework

Measure of Performance (MoP)

! Measures of Performance are … !  Adributes that assure the system has the capability to perform,

!  Assessment of the system to assure it meets design requirements to sa&sfy the MoE.

Measures that characterize physical or func&onal adributes rela&ng to the system opera&on, measured or es&mated

under specific condi&ons.


MoP’s belong to the Program – Developed by the Systems Engineer, Measured By CAMs, and Analyzed by PP&C

12

9.0 Framework

Key Performance Parameters (KPP) Both JROC and Program Specific

!  Key Performance Parameters … !  Have a threshold or objec&ve value, !  Characterize the major drivers of performance,

!  Are considered Cri&cal to Customer (CTC).

Represent the capabili&es and characteris&cs so significant that failure to meet them can be cause for

reevalua&on, reassessing, or termina&on of the program


The acquirer defines the KPPs during the opera&onal concept development – KPPs say what DONE looks like

13

9.0 Framework

Technical Performance Measures (TPM) for key deliverables

!  Technical Performance Measures … !  Assess design progress, !  Define compliance to performance requirements,

!  Iden&fy technical risk, !  Are limited to cri&cal thresholds, !  Include projected performance.


Adributes that determine how well a system or system element is sa&sfying or expected to sa&sfy a technical

requirement or goal

14

9.0 Framework

What are Technical Performance Measures Really?

!  TPMs are measures of the system technical performance that have been chosen because they are indicators of system success. They are based on the driving requirements or technical parameters of high risk or significance -‐ e.g., mass, power or data rate.

!  TPMs are analogous to the programma&c measures of expected total cost or es&mated &me-‐to-‐comple&on. There is a required performance, a current best es&mate, and a trend line.

!  Actual versus planned progress of TPMs are tracked so the systems engineer or project manager can assess progress and the risk associated with each TPM.

!  The final, delivered system value can be es&mated by extending the TPM trend line and using the recommended con&ngency values for each project phase.

!  The project life trend-‐to-‐date, current value, and forecast of all TPMs are reviewed periodically (typically monthly) and at all major milestone reviews.

15

9.0 Framework

!  Tracking TPMs and comparing them to the resource growth provides an early warning system to detect deficiencies or excesses

!  Reserve alloca&ons narrow as design proceeds !  TPMs that violate reserve alloca&ons or have trends that

do not meet the final performance trigger correc&ve ac&ons

16

Tracking the Technical Performance Measures

9.0 Framework

No&onal TPM Design Margin Guidelines

17

9.0 Framework

Sample IMP: A Flight Avionics System (Con&nued)

Hardware PDR – Purpose !  Ensure system hardware ini&al design has been updated, and meets

func&onal and allocated performance requirements within program constraints.

!  Opera&onal security concept assessed. !  Ensure training requirements have been analyzed and their objec&ves

have been defined for training missions. !  Confirma&on training objec&ves and MTC design and integra&on conform

to the Air Force syllabus and Ready Aircrew Program (RAP). !  Training plan will be updated. !  Hardware PDR – Expecta&ons !  Team agrees system hardware ini&al design has been updated and can

proceed to the detailed design phase. !  Team agrees training plans and objec&ves correlate with the Air Force

syllabus, RAP and training planning, development can con&nue. !  System Specifica&on and TTL requirements are traceable to the allocated

hardware design. 18

9.0 Framework


!  Hardware PDR – Entry Criteria !  Func&onal Baseline Authen&cated (FBA) !  Ini&ate system hardware ini&al design and allocate func&ons to the appropriate Configura&on Items

!  All specifica&ons updated and required documenta&on is made available including an&cipated lower level design documenta&on

!  All SRR/SFR ac&on items closed or disposi&oned

19

9.0 Framework


Hardware PDR – Accomplishments !  System hardware ini&al design complete.

–  Func&ons allocated to one or more hardware configura&on items and are traceable to the MTC SSS and TSSC SSS.

–  Human, safety, R&M, EMI, opera&onal security, instructor and operator interfaces, etc, design factors have been reviewed.

!  Drag instructor and operator manuals reviewed !  Program risks updated, assessed, and reviewed.

–  Mi&ga&on plans in place. !  Program schedule and constraints updated and reviewed.

–  Cri&cal schedule path drivers reviewed. !  Design criteria for the simula&on and database development

reviewed and updated. !  Program processes and metrics reviewed. !  Test Planning ac&vi&es and relevant documenta&on reviewed by

test team. 20

9.0 Framework

Sample IMP: A Flight Avionics System (Concluded)

Hardware PDR – Exit Criteria !  Hardware (ownership, visual, IOS, brief/debrief) design reviewed, allocated to a

hardware configura&on item and updated to include instructor and operators interfaces, malfunc&on and control requirements, etc.

!  RTM updated, MTC SSS, TSSC SSS and TTL traceable to allocated hardware design to include ESOH requirements.

!  Human, safety, R&M, EMI, opera&onal security, instructor and operator interfaces, etc, design factors reviewed.

!  MTC and TSSC allocated baselines established and controlled by appropriate level documenta&on for PDR.

!  Drag instructor and operator manuals reviewed with user concurrence and incorporate sa&sfactory human factor design factors into the operator interfaces.

!  Risk management and mi&ga&on plans updated, in place, addresses ESOH plans and risks, and within program constraints.

!  Risks assesses, understood, documented, accepted and understood by team. !  Program schedule reviewed

–  Cri&cal path drivers iden&fied –  IMS Updated and reflects cri&cal paths

21

9.0 Framework

One More IMP Sample

!  (SA) System & Segment Requirements Updated & Allocated –  (AC) SRR / SDR Update Review Conducted

–  (AC) Preliminary System Specifica&on Documents (A011) Baselined

–  (AC) Preliminary Spacecrag Segment Specifica&on Baselined

–  (AC) Preliminary Ground Segment Specifica&on Baselined

–  (AC) Preliminary Specifica&on Tree Baselined

!  (SA) Preliminary ICDs Baselined For Customer Review

–  (AC) Preliminary Space-‐Ground ICD Baselined

!  (SA) PDR System Design Completed –  (AC) Top Level System Architecture Updated

–  (AC) PDR Level System Analyses Completed

–  (AC) PDR Level Reliability / Availability Analysis Completed

–  (AC) Preliminary System Level Risk Assessment Completed

–  (AC) System Level Plans Updated For PDR

–  (AC) Flight Long Lead Review Conducted

22

Preliminary Design Review

9.0 Framework

The “But” for this Guidance

! With these samples and the SETR guidance we’ve just started

!  The program needs to define program specific events to assure the actual maturity measures are captured – The IMP provides sufficient definiDon to track the step-‐by-‐step compleDon of the required accomplishments for each event and to demonstrate saDsfacDon of the compleDon criteria for each accomplishment. [AFMC PAMPHLET 63-‐5] 23

9.0 Framework

IMP Verbs for Significant Accomplishments

24

Integrated Master Plan Allowable Verbs

Allocated: Segment requirement is flowed down from the System Specifica&on

Released: Approved item for delivery for intended customer or supplier; all internal distribu&on and sign offs complete. An electronic version is made accessible on the IDE

Completed: The subject item, data, document, or process is prepared or concluded, and reviewed and accepted by the responsible IPT. Suppor&ng documenta&on is available through IDE

Reviewed: The subject item, data, document, or process is prepared or concluded, and documented for comple&on. Suppor&ng documenta&on is available through IDE

Conducted: The subject mee&ng or review has been held with all required par&cipants. The charts or minutes are available through the IDE

Updated: The subject process, data, or document has been reevaluated using later informa&on, and adjustments incorporated.

Defined: The subject configura&on items, data, or document was submided to the customer

Validated: Requirements are validated, received contractor approvals, were distributed, and are available through the IDE

Established: The subject items is created and set in place in a manner consistent with its intended use ager review and accepted by the IPT

Verified: Requirements are verified or processed in accordance with established prac&ce.

9.0 Framework

The IMP Process Narra&ve

!  ObjecFve: a brief statement explaining why this process set is applied for this program

!  Governing DocumentaFon: lists of the guidance or compliance documents; e.g., specifica&ons, manuals, and procedures including company, government, and industry references

!  Approach: concise descrip&on as to who owns each process; what are the roles and responsibili&es; and the overall process including a process flow diagram.

25

9.0 Framework

Generic IMP Evalua&on Criteria !  Do the Program Events and Accomplishments reflect the logical evolu&on and progress of the overall Program? –  Do program events and their defini&ons clearly demonstrate the maturity of the program over its life?

–  Do the selected Accomplishments and associated Criteria iden&fy meaningful and measurable progress toward the key goals of the Program?

!  Do the Accomplishments for each event demonstrate a meaningful understanding of the program requirements or are they tasks that anyone could do for any contract? –  Do they reflect your SOW requirements?

!  Does the IMP structure readily map to the IPT structure such that each IPT can easily visualize the scope of their responsibility?

!  When awarded, could the contractor use the Accomplishments as discrete ac&vity cost accounts in their earned value system? Or are they level of effort in nature?

!  Is sufficient visibility provided to iden&fy and track the Program Risk Plan and associated risk mi&ga&on accomplishments and/or con&ngencies?

–  Does criteria suppor&ng the accomplishments include key performance requirements?

!  Are IPT cross-‐dependencies and dependencies external to the Program appropriately reflected if they reflect poten&al schedule or performance risks to the success of the Program?

!  Is the submided "Contract IMP" (the Product IMP that is to be included as part of the Program Contract) defined to the appropriate level: –  Do the Accomplishments and associated Criteria go down to a level sufficient to provide visibility into key subcontractor ac&vi&es upon which the success of the Program may be dependent?

–  Are the Events and Accomplishments included in the IMP at such a level as to make the maintenance of the 'Contractual IMP' prac&cal or does it include an unnecessary level of detail?

26

9.0 Framework

Program Management Levels Program Levels" IMP/IMS Elements" CWBS"

Tier 1!Program Manager!Technical Leads!IPT Manager!Technical performance goals!

Major Program Events (PE)!!!!Significant Accomplishments (SA)!

Level 1 & 2!Links to CLINs!Level 3 & 4!Control Packages!Link to PBS!Integrated EVMS!

Tier 2!Control Account Managers !Product Work Plan!Responsible organization elements!!

Accomplishment Criteria (AC)!!!!!Tasks (NA)!

Level 5!Cost Account Package!Cost Collection Level!Links to WBS by OBS!Resource summaries!Early warning EVMS analysis!

Tier 3!Work Package Manager!

Detailed plans! Work package!Earned value calculations!

27

9.0 Framework

Connec&ng the Components of the IMP/IMS

!  The assembled IMP/IMS links all work ac&vi&es ver&cally to the ACs, SAs, and PEs

28

IMS

Customer Requirements IMP

ORG

IPTs

ORG

IPTs

Performance

Analysis/ Management

Review

Events (E) Events (E)

Accomplishments Process

Narratives Criteria



Control Account Control Account

Work Package Work Package

Work Package Tasks Work Package Tasks

WBS

Program Performance Management

System

Supplemental Schedules

Risk and opportunity Risk and

opportunity

9.0 Framework

The integrated master plan and integrated master schedule

Technology

Transcript of The integrated master plan and integrated master schedule