Technical Performance Measures

CPM–500 EVM/SE/TPM Integration

Rights Reserved

CPM–500–B/C/F : Integrating Systems

Engineering with Earned Value ManagementLesson 3 CPM–500F: Technical Performance Measures

June 2010

Naples, Florida

Professional Education Program (Training Track) presented by

PMI–College of Performance Management faculty

Glen B. AllemanLewis & Fowler

[email protected]

(303) 241 9633

1/63


Rights Reserved

The Purpose Of This Series

This is the last “connected” lesson in a 3–part series that explores the integration of System Engineering (SE) and Earned Value Management (EVM).

You don’t need to attend all three lessons – but it helps in Connecting The Dots

Why have this series? – Often, project control specialists see the planning of the cost and

schedule baselines as an isolated activity disconnected from the technical or engineering aspects.

– This type of thinking significantly impacts development of the baseline which results in poor maintenance and eventual irrelevance of the baseline

– We claim “victory” if after any one of these lessons, you can recognize the importance of integrating these disciplines such that he/she will advocate greater cooperation between the engineering and project control elements of his/her respective organization

2/63


Rights Reserved

Lesson 3 Objectives

In this lesson we will: Understand how to establish credibility for

Technical Performance Measures by using Earned Value and Systems Engineering to measure progress

Establish the processes of defining and measuring technical performance

Establish the role of the technical baseline in program management

Learn how TPM’s are integrated into planning and execution at the control account level through examples and a hands on exercise

3/63


Rights Reserved

Can Earned Value Alone Get Us

To Our Destination?

How do we increase visibility into the program’s performance? How do we reduce cycle time to deliver the product? How do we foster accountability? How do we reduce risk? How do we start this journey to success?

Increasing the Probability of Success means we have to Connect The Dots to Reach Our Destination 4/63


Rights Reserved

To Achieve Success …

©gapingvoid ltd www.gapingvoidgallery.com 5/63

We Need to …


Rights Reserved

Learnings from Lesson 1 and 2

How cost and schedule (EV) are connected to the Performance

Measurement Baseline

6/63


Rights Reserved

Increasing the Probability of

Program Success Means …

Risk

SOW

Cost

WBS

IMP/IMS

TPM

PMB

Building A Credible Performance Measurement Baseline

This is actually harder than it looks!7/63


Rights Reserved

Doing This Starts With Some Guidance

Systems engineering uses technical performancemeasurements to balance cost, schedule, and performance throughout the life cycle. Technical performance measurements compare actual versus planned technical development and design. They also report the degree to which system requirements are met in terms of performance, cost, schedule, andprogress in implementing risk handling. Performance metrics are traceable to user–defined capabilities.

― Defense Acquisition Guide (https://dag.dau.mil/Pages/Default.aspx)

In The End ― It’s All About Systems Engineering

8/63


Rights Reserved

Guidance for Measures of

Effectiveness, Performance, and

Technical Performance

Our starting point is not EVM, it’s Systems Engineering

MOE’s are an essential part of Systems Engineering, guided by IEEE 1220 and EIA 632.

System’s Engineers drive the content of all measurement items, customer or supplier.

9/63


Rights Reserved

Just A Reminder Of The …

Primary Elements of Earned Value

Cost

TechnicalPerformance

Schedule

Funding margin for under

performance

Schedule margin for over target baseline

(OTB)

Schedule margin for underperformance or

schedule extension

Over cost or under

performance

Over cost or over

schedule

Over schedule or

under performing

10/63


Rights Reserved

Previous Approaches Using EV

Are Mostly Unsuccessful In

Connecting These

Traditional approaches to program management are retrospective

– Cost and schedule of Earned Value

– Risk Management

– Systems Engineering

Reporting past performance

– Sometimes 30 to 60 days old

– Variances are reporting beyond the widow of opportunity for correction

11/63


Rights Reserved

It’s All Been Said Before.

We Just Weren’t Listening…

… the basic tenets of the process are the need for seamless management tools, that support an integrated approach … and “proactive identification and management of risk” for critical cost, schedule, and technical performance parameters.

― Secretary of Defense, Perry memo, May 1995

Why Is This Hard To Understand?

We seem to be focused on EV reporting, not the use of EV to manage the program.

Getting the CPR out the door is the end of Program Planning and Control’s efforts, not the beginning.

12/63


Rights Reserved

The Gap Seems To Start With A

Common Problem

Many Times, The Information from Cost, Schedule, Techncial Performance, and Risk Management Gets Mixed Up When We

Try to Put Them Together

13/63


Rights Reserved

When We Put The Cart Before

The Horse, We Discover …

EVM really doesn’t do its job effectively

Most of the time EV has no measure of quality or compliance with technical requirements.

EV measures progress to plan in units of “money,” not tangible value to the customer

Most EV System Descriptions fail to connect the dots between cost, schedule, and technical performance – even though instructed to do so in the official guidance

14/63


Rights Reserved

The NDIA EVM Intent Guide Says

Notice the inclusion of Technical along with

Cost and Schedule

That’s the next step is generating Value from Earned Value

EV MUST include the Technical Performance Measures


Rights Reserved

Back To Our Technical

Performance Measures

Technical Performance Measures do what they say,

Measure the Technical Performance

of the product or service produced by the program

16/63


Rights Reserved

What’s Our Motivation for

“Connecting the Dots?”

Technical Performance Measures … Provide program management with information

to make better decisions Increase the probability of delivering a solution

that meets both the requirements and mission need

TPMs are a set of measures that provide the supplier and acquirer with insight into progress to plan of the technical solution, the associated risks, and emerging issues.

17/63

We’ve been talking about this since as early as 1984, in Technical Performance Measurement Handbook, Defense Systems Management College, Fort Belvoir, VA 22060


Rights Reserved

Measure of Effectiveness (MoE)

Measures of Effectiveness … Are stated by the buyer in units meaningful to

the buyer Focus on capabilities independent of any

technical implementation

The operational measures of success that are closely related to the achievements of the mission or operational objectives evaluated in the operational environment, under a specific set of condition

“Technical Measurement,” INCOSE–TP–2003–020–01

MoE Belong to the End User18/63


Rights Reserved

Measure of Performance (MoP)

Measures of Performance are … Attributes that assure the system has the

capability to perform Assessment of system to assure it meets design

requirements necessary to satisfy the MOE

Measures that characterize physical or functional attributes relating to the system operation, measured or estimated under specific conditions


MoP’s belong to the Program – Developed by the Systems Engineer, Measured By CAMs, and Analyzed by PP&C

19/63


Rights Reserved

Key Performance Parameters (KPP)

Key Performance Parameters … have a threshold or objective value Characterize the major drivers of performance Are considered Critical to Customer (CTC)

Represent the capabilities and characteristics so significant that failure to meet them can be cause for reevaluation, reassessing, or termination of the program


The acquired defines the KPPs during the operational concept development – KPPs say what DONE looks like

20/63


Rights Reserved

Technical Performance Measures (TPM)


Technical Performance Measures …

Assess design progress

Define compliance to performance requirements

Indentify technical risk

Are limited to critical thresholds

Include projected performance

Attributes that determine how well a system or system element is satisfying or expected to satisfy a technical requirement or goal

21/63


Rights Reserved

Dependencies Between Measures

MoE

KPP

MoP TPMMission

Need

“Coming to Grips with Measures of Effectiveness,” N. Sproles, Systems Engineering, Volume 3, Number 1, pp. 50–58 22/63

Stakeholders Define Needs and Capabilities in terms of Operational Scenarios

Suppliers Define Physical Solutions that meet the needs of the Stakeholders

Operational measures of success related to the achievement of the mission or operational objective being evaluated.

Measures that characterize physical or functional attributes relating to the system operation

Measures used to assess design progress, compliance to performance requirements, and technical risks


Rights Reserved

“Candidates” for Technical Measures

INCOSE Systems Engineering Handbook

Concept Description

Physical Size and StabilityUseful LifeWeightVolumetric capacity

Functional CorrectnessAccuracyPower performance

All the “ilities”

SupportabilityMaintainabilityDependabilityReliability = Mean Time Failure

EfficiencyUtilizationResponse timeThroughput

Suitability for Purpose Readiness23/63

Reliability


Rights Reserved

“Measures” of Technical Measures

INCOSE Systems Engineering Handbook

Attribute Description

Achieved to DateMeasured technical progress or estimate of progress

Current EstimateValue of a technical parameter that is predicted to be achieved

MilestonePoint in time when an evaluation of a measure is accomplished

Planned Value Predicted value of the technical parameter

Planned Performance Profile

Profile representing the project time phased demonstration of a technical parameter

Tolerance Band Management alert limits

Threshold Limiting acceptable value of a technical parameter

VariancesDemonstrated technical variancePredicted technical variance 24/63


Rights Reserved

A Familiar Graphic of TPMs

Variance

Planned Value

Planned Profile

Current Estimate

Milestones

Threshold

Upper Limit

Lower Limit

Mea

n T

o B

etw

een

Fai

lure

Time = Program Maturity

Achieved to Date

25/63

TPM


Rights Reserved

A Simple Method of Assembling the TPMs

Select Technical Performance Parameters

Define the planned

progress for each TPM

Assess the impact on Risk

from this progress

Weight XXXX

Speed XXXX

MTBF XXXX

Loiter Time XXXX

Parameters Progress Risk

MOE / MOP KPP / TPM Risks

26/63


Rights Reserved

TPMs from an Actual Program

James Webb Space Telescope

27/63


Rights Reserved

TPMs from an Actual Program

Chandra X–Ray Telescope

28/63


Rights Reserved

What Does A Real Technical

Performance Measure Look Like?

Not that bagels are not interesting in Lesson 1 and 2, but let’s get ready to look at a flying machine.

29/63


Rights Reserved

1.1 Air Vehicle

1.1.1 Sensor Platform

1.1.2 Airframe

1.1.3 Propulsion

1.1.4 On Board Comm

1.1.5 Auxiliary Equipment

1.1.6 Survivability Modules

1.1.7 Electronic Warfare Module

1.1.8 On Board Application & System SW

1.3 Mission Control / Ground Station SW

1.3.1 Signal Processing SW

1.3.2 Station Display

1.3.3 Operating System

1.3.4 ROE Simulations

1.3.5 Mission Commands

TPMs Start With The WBSThe WBS for a UAV

1.1.2 Airframe

30/63


Rights Reserved

What Do We Need To Know About

This Program Through TPMs

What WBS elements represent the TPMs?

What Work Packages produce these WBS elements?

Where do these Work Packages live in the IMS?

What are the Earned Value baseline values for these Work Packages?

How are going to measure all these variables?

What does the curve look like for these measurements?

31/63


Rights Reserved

Let’s Connect The Dots

Risk

SOW

Cost

WBS

IMP/IMS

TPM

PMBNamed Deliverables defined in the WBS

BCWS at the Work Package, rolled to the Control Account

TPMs attached to each critical deliverables in the WBS and identified in each Work Package in the IMS, used to assess maturity in the IMP

The Products and Processes that produce them in a “well structured” decomposition in the WBS

IMS contains all the Work Packages, BCWS, Risk mitigation plans, and rolls to the Integrated Master Plan to measure increasing maturity

Technical and Programmatic Risks Connected to the WBS and IMS

32/63


Rights Reserved

Verifying Each TPM

Evidence that we’re in compliance

CADo we know what we promised to deliver, now that we’ve won?

With our submitted ROM what are the values we need to get through Integrated Baseline Review «how do we measure weight for each program event»

SFRCan we proceed into preliminary design?

The contributors to the vehicle weight are confirmed and the upper limits defined in the product architecture and requirements flow down database (DOORS) into a model

SRRCan we proceed into the System Development and Demonstration (SDD) phase

Do we know all drivers of vehicle weight? Can we bound their upper limits? Can the subsystem owners be successful within these constraints uses a high fidelity model?

PDR

Can we start detailed design, and meet the stated performance requirementswithin cost, schedule,risk, and other constraints?

Does each subsystem designer have the target component weight target and have some confidence they can stay below the upper bound? Can this be verified in some tangible way? Either through prior examples or a lab model?

CDR

Can the system proceed to fabrication, demonstration, and test, with the within cost, schedule, risk, and other system constraints.

Do we know all we need to know to start the fabrication of the first articles of the flight vehicle. Some type of example, maybe a prototype is used to verify we’re inside the lines

TRRCan the system ready toproceed into formal test?

Does the assembled vehicle fall within the weight range limits for 1st flight – will this thing get off the ground? 33/63


Rights Reserved

25kg

23kg

28kg

TPM Trends & Responses

Dr. Falk Chart – modified

EV Taken, planned values met, tolerances kept, etc.

26kg

PDRSRRSFRCA TRRCDR

ROM in Proposal

Design Model

Bench Scale Model Measurement

Detailed Design Model

Prototype Measurement

Flight 1st Article

34/63

Tech

nic

al P

erfo

rman

ce M

easu

re

Veh

icle

Wei

ght


Rights Reserved

The Assessment Of Weight As A

Function Of Time

At Contract Award there is a Proposal grade estimate of vehicle weight

At System Functional Review, the Concept of Operations is validated for the weight

At System Requirements Review the weight targets are flowed down to the subsystems components

At PDR the CAD model starts the verification process At CDR actual measurements are needed to verify all

models At Test Readiness Review we need to know how much

fuel to put on board for the 1st flight test

35/63


Rights Reserved

1.1 Air Vehicle

1.1.1 Sensor Platform

1.1.2 Airframe

Airframe Weight TPMThe WBS for a UAV

1.1.2 Airframe

CA SFR SRR PDR CDR TRR

Planned Value 28.0kg 27.0kg 26.0kg 25.0kg 24.0kg 23.0kg

Actual Value 30.4kg 29.0kg 27.5kg 25.5kg

Assessed Risk to TRR

Moderate >2.0kg off

target

Low 1–2 kg off

target

Low 1–2 kg off

target

Very Low (less than 1.0 kg off target)

Planned Method

“Similar to” Estimate

ROMProgram–

unique design model

Program–unique design

model with validated data

Actual measurement of bench–test components

Actual measurement of prototype

airframe

Actual Method

“Similar to” Estimate

ROM ROM ROM

The planned weight is 25kg. The actual weight is 25.5kg.

Close to plan! So we are doing okay, right?

36/63

Here’s the Problem


Rights Reserved

Is This A Problem?

You Bet’ya It’s A Problem!

The measurement is close to the planned value,

But the planned method of measurement is a program unique design model with validated data,

But the actual method of measurement is a rough order of magnitude estimate,

No improvement in fidelity since the System Functionality Review (SFR), and

The TPM provides no new information – so we’re probably late and don’t know it yet.

37/63


Rights Reserved

Raison d'etre for Technical

Performance Measures

The real purpose of Technical Performance Measures is to reduce Programmatic and Technical RISK

38/63

Risk

SOW

Cost

WBS

IMP/IMS

TPM

PMB


Rights Reserved

Buying Down Risk with TPMs

“Buying down” risk is planned in the IMS.

MoE, MoP, and KPP defined in the work package for the critical measure – weight.

If we can’t verify we’ve succeeded, then the risk did not get reduced.

The risk may have gotten worse.

39/63

Risk: CEV-037 - Loss of Critical Functions During Descent

Planned Risk Level Planned (Solid=Linked, Hollow =Unlinked, Filled=Complete)

Ris

k S

core

24

22

20

18

16

14

12

10

8

6

4

2

0

Conduct Force and Moment Wind

Develop analytical model to de

Conduct focus splinter review

Conduct Block 1 w ind tunnel te

Correlate the analytical model

Conduct w ind tunnel testing of

Conduct w ind tunnel testing of

Flight Application of Spacecra

CEV block 5 w ind tunnel testin

In-Flight development tests of

Damaged TPS flight test

31.M

ar.

05

5.O

ct.05

3.A

pr.

06

3.J

ul.0

6

15.S

ep.0

6

1.J

un.0

7

1.A

pr.

08

1.A

ug.0

8

1.A

pr.

09

1.J

an.1

0

16.D

ec.1

0

1.J

ul.1

1

Weight risk

reduced from

RED to Yellow

Weight confirmed

ready to fly – it’s

GREEN at this point


Rights Reserved


Success with Risk Management

Going outside the TPM limits always means cost and schedule impacts

“Coloring Inside the Lines” means knowing the how to keep the program GREEN, or at least stay close to GREEN

40/63

So much for our strategy of winning through technical dominance


Rights Reserved

Connecting the EV Variables

41/63

Integrating Cost, Schedulele, and Technical PerformanceAssures Program Management has the needed performance information to deliver

on‒time, on‒budget, and on‒specification

Technical Performance Measures

Cost Schedule

Conventional Earned Value

+=

Master Schedule used to derive Basis of Estimate (BOE) not the other way around.

Probabilistic cost estimating uses past performance and cost risk modeling.

Labor, Materiel, and other direct costs accounted for in Work Packages

Risk adjustments for all elements of cost.

Cost Baseline

Earned Value is diluted by missing technical performance.

Earned Value is diluted by postponed features.

Earned Value is diluted by non compliant quality.

All these dilutions require adjustments to the Estimate at Complete (EAC) and the To Complete Performance Index (TCPI).


Requirements are decomposed into physical deliverables.

Deliverables are produced through Work Packages.

Work Packages are assigned to accountable manager.

Work Packages are sequenced to form the highest value stream with the lowest technical and programmatic risk.

Schedule Baseline


Rights Reserved

TPM Checklist

MoE MoP TPMTraceable to needs, goals, objectives, and risks

Traceable to applicable MOEs, KPPs, system level performance requirements, and risks

Traceable to applicable MoPs, system element performance, requirements, objectives, risks, and WBS elements

Defined with associated KPPs

Focused on technical risks and supports tradesbetween alternative solutions

Further decomposed, budgeted, and allocated to lower level system elements in the WBS and IMS

Each MoE independent from others

Provided insight into system performance

Assigned an owner, the CAM and Work Package Manager

Each MoE independent of technical any solution

Decomposed, budgetedand allocated to system elements

Sources of measure identifiedand processes for generating the measures defined.

Address the required KPPs

Assigned an “owner,” the CAM and TechnicalManager

Integrated into the program’s IMS as part of the exit criteria for the Work Package 42/63


Rights Reserved


Program Success Means …

Risk

SOW

Cost

WBS

IMP/IMS

TPM

PMB

Building A Credible Performance Measurement Baseline

Using the Check List – “Connect the Dots”43/63


Rights Reserved 44/63


Rights Reserved

Backup Materials

Knowledge is of two kinds. We know a subject ourselves, or we know where

we can find information on it— Samuel Johnson

45/63


Rights Reserved

OMB Circular A–11, Section 300 Interim Defense Acquisition Guidebook (DAG) 6/15/09

GAO Report 06–250 Systems Engineering Plan (SEP) Preparation Guide 4/08

DoDI 5000.02, Operation of the Defense Acquisition System (POL) 12/08

WBS Handbook, Mil–HDBK–881A (WBS) 7/30/05

Integrated Master Plan (IMP) & Integrated Master Schedule Preparation & Use Guide (IMS) 10/21/05

Guide for Integrating SE into DOD Acquisition Contracts 12/06

Defense Acquisition Program Support Methodology (DAPS) V2.0 3/20/09

Guide to the Project Management Institute Body of Knowledge (PMBOK Guide®), 4th Edition

Standard for Application and Management of the SE Process (IEEE 1220)

Capability Maturity Model Integration (CMMI®)

IEEE 1220: 6.8.1.5 Processes for Engineering a System (ANSI/EIA–632)

NASA EVM Guide NPG 9501.3

Many of Sources for Connecting the Dots

46/63


Rights Reserved

Office of Management and

Budget

Circular No. A–11, Section 300

Planning, Budgeting, Acquisition and Management of Capital Assets

Section 300–5– Performance–based acquisition management

– Based on EVMS standard

– Measure progress towards milestones• Cost

• Capability to meet specified requirements

• Timeliness

• Quality

47/63


Rights Reserved

Need: Accurate Performance

Measurement

GAO Report 06–250Findings and

Recommendations

Information Technology:Improve the Accuracy and Reliability of Investment Information

2. If EVM is not implemented effectively, decisions based on inaccurate and potentially misleading information

3. Agencies not measuring actual versus expected performance in meeting IT performance goals.

48/63


Rights Reserved

DOD Guides:


Department of Defense Guidelines for Technical Performance Measures

DoDI 5000.02, Operation of the Defense Acquisition System (POL) 12/08

Interim Defense Acquisition Guidebook (DAG) 6/15/09

Systems Engineering Plan (SEP) Preparation Guide 4/08

WBS Handbook, Mil–HDBK–881A (WBS) 7/30/05

Integrated Master Plan (IMP) & Integrated Master Schedule Preparation & Use Guide (IMS) 10/21/05

Guide for Integrating SE into DOD Acquisition Contracts (Integ SE) 12/06

Defense Acquisition Program Support Methodology (DAPS) V2.0 3/20/09

49/63


Rights Reserved

DoD: TPMs in Technical Baselines and Reviews

DoD Policy or Guide PO

L

DA

G

SEP

WB

S

IMP/

IMS

Inte

grat

ed

Sy

ste

ms

Engi

ne

eri

ng

DA

PS

Technical Baselines:IMP/IMSFunctional (SFR)Allocated (PDR)Product (CDR)

Event driven timing

Success criteria of technical review

Entry and exit criteria for technical reviews

Assess technical maturity

50/63


Rights Reserved

DoD: TPMs in Integrated Plans

DoD Policy or Guide PO

L

DA

G

SEP

WB

S

IMP/

IMS

Inte

grat

ed

Sy

ste

ms

Engi

ne

eri

ng

DA

PS

Integrated SEP with:IMP/IMSTPMsEVM

Integrated WBS withRequirement SpecificationStatement of WorkIMP/IMS/EVMS

Link risk management, technical reviews, TPMs, EVM, WBS, IMS

51/63


Rights Reserved

Guidance in Standards, Models,

and Defense Acquisition Guide

Processes for Engineering a System (ANSI/EIA–632)

Standard for Application and Management of the SE Process (IEEE 1220)

Capability Maturity Model Integration (CMMI®) – CMMI for Development, Version 1.2

– CMMI for Acquisition, Version 1.2

– Using CMMI to Improve Earned Value Management, 2002

Guide to the Project Management Institute Body of Knowledge (PMBOK Guide®), 4th Edition

52/63


Rights Reserved


Measures (TPM)

More Sources

IEEE 1220: 6.8.1.5, Performance–based

progress measurement

EIA–632: Glossary CMMI for Development

Requirements

Development

TPMs are key to

progressively assess

technical progress

Predict future value of key

technical parameters of

the end system based on

current assessments

Specific Practice (SP) 3.3,

Analyze Requirements

Typical work product:

TPMs

Establish dates for

– Checking progress

– Meeting full

conformance to

requirements

Planned value profile is

time–phased achievement

projected

• Achievement to date

• Technical milestone

where TPM evaluation is

reported

Subpractice:

Identify TPMs that will be

tracked during

development

53/63


Rights Reserved

PMBOK®

Guide

10.5.1.1 Project Management Plan

Performance Measurement Baseline:

– Typically integrates scope, schedule, and cost parameters of a project

– May also include technical and quality parameters

54/63


Rights Reserved

PMBOK®

Guide

8.3.5.4 Work Performance Measurements

Used to produce project activity metrics

Evaluate actual progress as compared to planned progress

Include, but are not limited to:

– Planned vs. actual technical performance

– Planned vs. actual schedule performance, and

– Planned vs. actual cost performance.

55/63


Rights Reserved

TPMs in DAG and DAPS

Defense Acquisition Guide Performance measurement of WBS elements, using

objective measures:– Essential for EVM and Technical Assessment activities

Use TPMs and Critical Technical Parameters (CTP) to report progress in achieving milestones

DAPS Use TPMs to determine whether % completion

metrics accurately reflect quantitative technical progress and quality toward meeting Key Performance Parameters (KPP) and Critical Technical Parameters

56/63


Rights Reserved

TPMs in DAG

Compare the actual versus planned technical development and design

Report progress in the degree to which system performance requirements are met.

Plan is defined in terms of:

– Expected performance at specific points

• Defined in the WBS and IMS

– Methods of measurement at those points

– Variation limits for corrective action.

57/63


Rights Reserved

PMBOK®

Guide

11.6.2.4 Technical Performance Measurement Compares technical accomplishments… to … project

management plan’s schedule of technical achievement

Requires definition of objective quantifiable measures of technical performance which can be used to compare actual results against targets.

Might include weight, transaction times, number of delivered defects, storage capacity etc.

Deviation, such as demonstrating more or less functionality than planned at a milestone…forecast degree of success in achieving the project’s scope.

58/63


Rights Reserved

CMMI–ACQ

Acquisition Technical Management

SP 1.3 Conduct Technical Reviews

Typical supplier deliverables

Progress reports and process, product, and service level measurements

TPMs

59/63


Rights Reserved

SMS Shall:

Monitor Progress Against the Plan

4.2.12.2 Monitoring

Contractor SHALL monitor progress against plan to validate, approve, and maintain each baseline and functional architecture

4.2.12.2.2 Required Product Attributes

Each documented assessment includes:

TPMs, metrics

Metrics and technical parameters for tracking that are critical indicators of technical progress and achievement

60/63


Rights Reserved

NASA EVM Guide:


• NASA EVM Guide NPG 9501.3– 4.5 Technical Performance Requirements (TPR): When

TPRs are used, – appropriate and relevant metrics…– must be defined in the solicitation– Appendix A.7, 14.1 TPR• Compares:

• Expected performance and• Physical characteristics• With contractually specified values.

• Basis for reporting established milestones• Progress toward meeting technical requirements

61/63


Rights Reserved

See next chart for linkage of technical baselines to technical reviews

Document, Baseline,

IMS, EVM Parameter

IMP, Functional Baseline Measures Of Effectiveness (MOE)

IMP, WBS, Functional Baseline Measures Of Performance (MOP)

IMP, Allocated Baseline Technical Performance Measure

IMSTPM Milestones And Planned

Values

Work Packages TPM% Complete Criteria

Derivation and Flow

Down of TPMs

62/63


Rights Reserved

Interesting Attributes of TPMs

Achieved to Date (sounds like EV)

Current Estimate (sounds like EAC/ETC)

Milestone

Planned (target) value (sounds like PV)

Planned performance profile (sounds like a PMB)

Tolerance band (sounds like reporting thresholds)

Threshold (yep, just what we thought)

Variance (sounds like variance!)

63/63

Technical Performance Measures

Technology

Transcript of Technical Performance Measures