Woods.edwards.pm challenge bpr presentation 2012 v1

Overview of the Overview of the Baseline Performance Baseline Performance

Review (BPR)Review (BPR)

1

Office of the Chief EngineerOffice of the Chief Engineer

Program Management Challenge Program Management Challenge

Orlando, FLOrlando, FL

February 22-23 , 2012February 22-23 , 2012

2February 2012 For NASA Internal Use Only

Presentation PurposePresentation Purpose

• Describe NASA’s Baseline Performance Review (BPR) to the greater NASA community at PM Challenge 2012. Characterize the BPR objectives, scope, process, and products and

recent updates Provide insight into other complementary initiatives Summarize NASA’s technique for monitoring Agency performance

against its baseline plans

2


ContentsContents

• BPR History and Context• BPR Objectives• BPR Scope, Participants• BPR Process• BPR Process Improvements• Sample BPR Meeting Agenda• Sample BPR Analysis Content• Summary


BPR Background and ContextBPR Background and Context

• In the spring of 2006, the Office of the Chief Engineer (OCE) was given the action by the Associate Administrator to develop a State of the Agency (SoA) process that provides the Agency Project Management Council (APMC) an independent top level view of the status of the Agency’s programs and projects.

• The first SoA presentation, to the Agency PMC, was by the Science Mission Directorate (SMD) in August of 2006. The process has matured and continued to meet the objectives outlined by the Associate Administrator, the Associate Deputy Administrator, the Chief Engineer (CE) and the various stakeholders. The SoA evolved to a distinct monthly meeting in December 2007 and was officially entitled the Baseline Performance Review (BPR).


BPR Background and ContextBPR Background and Context(continued)(continued)

• The BPR instituted a modified “balanced score card stop-light chart” process based on metrics encompassed in a core set of functional area questions and supporting raw data.

• Participants from the projects, programs, Centers, Mission Directorates (MDs) and Mission Support Offices respond monthly to each core area question based on provided thresholds and key metrics for use in internal and external reporting.

• All Mission Directorates are reporting at the BPR meeting the findings of the independent process.

Office of Management and Budget (OMB) recognized the Agency BPR as a “best practice”

in the Federal Government.


BPR ObjectivesBPR Objectives

• Provide NASA Senior Leadership comprehensive, integrated, and objective information that describes the “performance-to-plan” of the Agency’s programs, projects, and institutional capabilities. A single place to see the full execution context of the Agency

• Assure open cross-functional communications among NASA’s organizations to enhance Agency performance.

• Identify and analyze performance trends and cross-cutting or systemic issues and risks

• Serve the Agency’s decision-making Councils as their ”execution arm” Within NASA Governance, the BPR is distinct from the decision-

making Executive Committee, Program Management, and Mission Support Council

BPR is “action-oriented” to improve performance and inform Agency decision authorities of issues needing attention.


BPR Scope and ParticipantsBPR Scope and Participants

• The BPR is co-chaired by the NASA Associate Administrator (Chris Scolese)and the Mission Support Directorate Associate Administrator (Woodrow Whitlow).

• Membership includes the heads of NASA Staff Offices, Mission Directorates, Mission Support Offices, and its ten Field Centers.

• Mission Support functions (e.g., finance, workforce, acquisition, infrastructure, IT, etc.) report monthly and Small Business and Diversity and Equal opportunity report quarterly.

• Mission Directorates (covering all program areas) report monthly with one (and sometimes 2) “highlighted” in greater detail.

• The Office of the Chief Engineer (OCE) leads the process and a team of “independent assessors” from OCE, Office of the Chief Financial Officer (OCFO) Strategic Investment Division (SID), Office of Safety Mission Assurance analyze programs’ and projects’ performance. “Green/Yellow/Red” criteria were established to assess of technical, cost,

schedule, and programmatic performance are provided monthly for all Mission Directorates.

MDs

Issue charts

OCE sends out notifications to users who answer questions

Mission Directorates (MDs)

Mission Support Offices

NASA Centers

Projects

Programs

1

2

3

4

5

OCE consolidates input from stakeholders1- 5

(OCE, IPCE, OSMA, and OCFO) Assessor Teams

OCE reps. for

MDs

OCFO reports

OCE generates

BPR package

“Stoplight” ratings for projects and programs from the Assessor Teams

TechnicalCross-Cutting, PCA Status

Non-TechnicalCross-Cutting

Package briefing to Chief Engineer for approval

BPR Package is available for Read-only viewing from site

Package presented at the BPR

OCE feedback and Action items on BPR process

OCE Provides Executive Summary

BPR Detailed Process FlowBPR Detailed Process Flow

3

For NASA Internal Use OnlyFor NASA Internal Use Only 88February 2012February 2012

9

• Cross-cutting issues Across missions, faulty material was discovered resulting in a mitigation process for

the materials use saving time and resources.

• Organizational communication issues Turned a conflict between projects on thermal protection systems into a collaboration

between the major development projects who previously were competing for shared resources.

• Mission support issues Workforce planning resolution was expedited when a project was confronted with

changes resulting in 500 people at a Center needing reassignment because the project was delayed 2 years.

• Senior leaders as an “Execution Board of Directors” Charlie Bolden utilized the BPR to successfully highlight and resolve the delays in

Freedom of Information Act (FOIA) responses.

• Mission specific issues Timely communication resulted in a Center director de-conflicting a vendors schedule

between 2 NASA projects from different Centers.

February 2012 For NASA Internal Use Only

Example Outcomes/ResultsExample Outcomes/Results


BPR Process Updates - 2011BPR Process Updates - 2011

BPR is a dynamic process – stakeholder driven

• AA in July ’10 directed call for inputs for BPR improvements

• Online survey conducted in Aug ‘10– wide agency participation to gather recommendations for BPR process and meeting improvements

• Town Hall in Sep ‘10 to review and refine comments• Core Team developed recommendations

– Core Team (assessors) - OSMA, OCFO, IPCE and OCE

• Brief to senior agency leadership in early Jan ‘11 • Results promulgated to stakeholders with BPR process

changes implemented throughout 2011


BPR Process Update HighlightsBPR Process Update Highlights

Presentation Content determination and Frequency –• Workforce & FOIA – from monthly to as required• OCIO – monthly • New BPR topic - addition of semi-annual brief by Physical Security• Deletion of individual briefs by ASM/ASP and External Reporting

(presentation materials will still be included in BPR package)• OCT to present two different technology areas on a quarterly basis

MSD/MSO – institutional assessment is still a pending issue• To report to BPR on periodic basis• Form of assessment and reporting to be determined at a later date

Document Compliance, Life Cycle Review, MD Rollups, MD –• Monthly Highlighted MD: assessors and MD designee will submit and

present their own inputs• Non-highlighted MD: assessors only will prepare and submit their

inputs for inclusion in the BPR package, but not be required to brief


Notional BPR Presentation ScheduleNotional BPR Presentation Schedule


Example BPR ReportsExample BPR Reports


Total of 36 Questions for ProjectsTechnical/Programmatic/Schedule/Cost

BPR Project QuestionsBPR Project Questions

15

Typical Monthly BPR AgendaTypical Monthly BPR Agenda

*Specific topic is rotated


10:00 a.m. – 4:30 p.m. (EST)NASA Headquarters, Room 8R42

10:00 Opening Remarks and Action Items Associate Administrator, BPR Manager10:15 Mission Support Reports

Finance Chief Financial Officer Workforce Human Capital Management ASP/ASM Mission Support Division Acquisition Procurement Small Business Small Business Programs Infrastructure Infrastructure Information Technology Chief Information Officer

12:15 Lunch 12:30 Agency Aggregate Assessments

OSMA Safety - NASA Mishap Posture NASA Safety Center Freedom of Information Act (FOIA) Status Public Affairs Office External Reporting Status Chief Financial Officer/Strategic Investment (SID) Cross Cutting Non-Technical Issues Chief Financial Officer/SID Cross Cutting Technical Issues Office of Chief Engineer Documents Compliance Office of Chief Engineer Life Cycle Milestone Schedule Office of Chief Engineer

1:40 Center Summaries Center Representatives 2:40 Roll-up of Mission Directorates Program/Project Office of Chief Engineer3:00 Highlighted: Science Mission Directorate (SMD)

SMD Assessment Science Mission DirectorateJWST Status/Recovery Plan* JWST Program

4:15 Actions Summary BPR Executive Secretary4:30 Adjourn Associate Administrator


Example BPR ReportsExample BPR Reports

• Finance• Acquisition• Small Business• Infrastructure• Safety and Mishaps• Crosscutting Issues• Field Center Summaries• Independent Assessments of Programs and Projects

(“Stoplight” ratings)


Finance ExampleFinance Example

Budget Execution

18

Undefinitized Contract

Action (UCA) Status


Procurement ExampleProcurement Example


$0 $5,000 $10,000 $15,000 $20,000 $25,000

In Million $

Department of Energy

Health and Human Services

National Aeronautics & SpaceAdministration

Department of Veterans Affairs

Department of HomelandSecurity

Department of Justice

General ServicesAdministration

Total Dollars Small Business Dollars

Federal Agency

Prime Small Business Dollars

Overview

Small Business ExampleSmall Business Example


Infrastructure ExampleInfrastructure Example

ARC DFRC GRC GSFC JPL JSC KSC LaRC MSFC SSC Remarks

Design & ConstructionG G G G G Y G G G G

JSC damage from Ike could impact JWST Project.. Chamber A MOU and Management Plan not complete. Concern about communication between Eng Div and Facil Div at JSC

Facility MaintenanceY Y Y Y Y R Y Y Y Y

Facility repair backlog is 8.6% of facility value with increasing trend.JSC damage from Ike could impact SOMD Operations

Planning/Real EstateG Y G Y G Y Y G Y Y

DFRC, KSC, and SSC were changed to yellow since they have not submitted revised Master Plans. GSFC presented new Master Plan, but has not submitted document for formal approval.

Supply

G G Y G G G G Y G Y

Delay in replacement of critical civil service personnel at GRC and KSC. Also at KSC, a concern existed as retiring civil servants positions were not backfilled. Situation improved by increased use of Support Services Contractors. Sinlge point failure for multiple logistics functions at LaRC and SSC. Anticipated hire at SSC currently on hold.

Equipment

G G Y G NA G G Y G Y

Delay in replacement of critical civil service personnel at GRC and KSC. GRC expects to fill vacancy by end of 2008. At KSC, a concern existed as retiring civil servants positions were not backfilled. Situation improved by increased use of Support Services Contractors. Single point failure for multiple log functions at LaRC and SSC. Anticipated hire at SSC currently on hold. GSFC filled HQ SEMO position.

Contract Property

Y Y Y Y Y Y Y Y Y Y

CFO management control findings related to property management are being addressed, however center IPO staffing concerns to support transition workload and FAR update present an on-going staffing issue. JSC in process of filling Contract Property Administrator position. JSC also submitted request to fill the following: 2 positions for Property Disposal, 1 position for Shuttle Clerical (Property) and 1 position for Packing/Shipping in Transportation.

Transportation

G G G Y G G G Y Y Y

Agency risk for not meeting EO 13423 (alternate fuel): discussions with MSFC to resolve center contribution ongoing. Single point failure for multiple log functions at SSC, anticipated hire currently on hold. GSFC loss of CNG capability for approx. 13 vehicles. KSC added additional contractor to assist with transportation functions. At LaRC, TO/SEMO is one person, he will be out in Jan. (medical leave) no replacement appointed.

Property DisposalG G G Y G Y Y G Y Y

Increase in excess property workload due to Shuttle and Hubble disposition at GSFC, MSFC, KSC, SSC & JSC. Single point failure for multiple log functions at SSC.

Logistics

Facilities

21

Jan 2008 – Mar 2008 Jan 2009 - Mar 2009 Jan 2008 – Mar 200 Jan 2009 - Mar 2009 Jan 2008 - Mar 2008 Jan 2009 - Mar 2009

March Mishaps:

• No Type A Mishaps

• One Type B Mishap: DRC- (Deluge System Release in Shuttle Hangar - 3/13/09)

• One Potential Type B Mishap: DRC- (Global Hawk Electronics Failure 3/12/09) – currently a Type D

Data does not include 4/2/09 Type B Mishap: Water Drill mishap resulting in multiple amputations

Close Call Incidents Show An Increase From Previous Reporting Period (Jan – Mar 2008).

March Mishaps:

• No Type A Mishaps

• One Type B Mishap: DRC- (Deluge System Release in Shuttle Hangar - 3/13/09)

• One Potential Type B Mishap: DRC- (Global Hawk Electronics Failure 3/12/09) – currently a Type D

Data does not include 4/2/09 Type B Mishap: Water Drill mishap resulting in multiple amputations

Close Call Incidents Show An Increase From Previous Reporting Period (Jan – Mar 2008).

February Mishap

OCO

February Mishap

OCO


NASA and Contractor

Safety Statistics

Dashboards

Mishap Reports ExampleMishap Reports Example

22

Technical Cross Cutting Technical Cross Cutting ExampleExample


Capability of Communications NetworksCapability of Communications Networks

23

Agency ProgramsAgency Programs


PCA Status ExamplePCA Status Example


Agency Schedule ExampleAgency Schedule Example

25

Technical Authority/Mission Support/InstitutionalTechnical Authority/Mission Support/InstitutionalInstitutional Risk to Mission

Budget/Finance:Budget/Finance: Grants reconciliation and transition: The ORR for the Phase 2 transition to NSSC is scheduled for November 30th; will complete GSFC transition activities.

Workforce:Workforce:

Early Career Hiring: FY09 - 66 on-board; FY10 – 10 with an additional 10 offers accepted and negotiating dates.

Acquisition:Acquisition: Contractor Support: Consolidated Agency contracting approach being pursued. GSFC had restricted authorization to use the existing GSFC contract for critical tasks ending in December. “Authorization” to seek services on another contract granted but is not expected to save resources and requires unnecessary actions.

XYZ contract protest activities continue. ZYX contract has been extended through Feb 2011. The delayed transition from ZYX to XYZ continues to impact the contractor’s ability to retain experienced employees to perform the work.• Agency-level protests denied on August 11th and the Corrective Action Plan (CAP) implementation was initiated on August 17th. • A protest was submitted on August 21st to GAO with regard to the CAP. CAP implementation permitted to continue. GAO decision due NLT November 30th. • Congressional request for information with regard to the CAP was submitted October 22nd.

For NASA Internal Use OnlyFebruary 2012

Center Summary ExampleCenter Summary Example


Assessment Aggregate Rollup ExampleAssessment Aggregate Rollup Example

Science Mission Directorate

27

Program Status Roll Up ExampleProgram Status Roll Up Example


ARMD

28

Cost & Schedule Rollup ExampleCost & Schedule Rollup Example


29

Assessment Quad Chart ExampleAssessment Quad Chart Example



BPR Actions and Status ExampleBPR Actions and Status Example

Subj: Cross Cutting Technical Issues – Review and Update

Action Statement: • The Mission Directorate Chief Engineers (MDCE’s) will meet and do a

comprehensive review of the technical cross cutting process and issues to include discussion about what new issues should be added to the list.

Action for: OCE MDCEs Date Assigned: Dec 15th

Action Assigned: • Present candidate Tech Cross Cutting Issues at Feb BPR

.

2011-12-047


Assessment Color CriteriaAssessment Color Criteria


Green CriteriaGreen Criteria

A Category for a Project is Green if:There are minimal programmatic issues and adequate margin (budget reserve,

schedule slack, technical margin), or capability (for operational missions), to complete the Project on plan.

A Project is Green overall if:All 4 categories are Green, orOne or more are Yellow and the issues are collectively not a threat/impact to the

project by virtue of their being solvable within the project's planned resources. (Margin in one category may provide capability to resolve a lack of margin in another category.)

A Program is Green if:The constituent projects are Green overall, or are Yellow with realistic, defined plans

to return to green and all other program elements are on budget and schedule.


Yellow CriteriaYellow Criteria

A Category for a Project is Yellow if:The remaining margin within the category appears inadequate to meet Project requirements or external commitments, however, mitigation plans are in place or in development to recover margin and achieve commitments within the available resources of the Project. (For operational projects in the technical category "margin" may be interpreted as operational capabilities.)

A Project is Yellow overall if:Any one category is Red, the overall is at least Yellow, orOne or more categories are Yellow and the issues are collectively a threat to the project's ability to meet their commitments within the project's planned resources.

A Program is Yellow if:Any of its constituent projects are Red overall, or if one or more are Yellow and the issues are collectively a threat to the Program's ability to meet its commitments within planned resources.


Red CriteriaRed Criteria

A Category for a Project is Red if:There is minimal margin or negative margin within the category such that the Project is estimated not to meet its requirements or commitments without significant impact to other categories. There is high risk that the Project will require external resources or relief from performance or programmatic requirements.

A Project is Red overall if:One or more categories are Red, and it is estimated that the Project will not have sufficient resources to meet its commitments.

A Program is Red if:Most of its constituent projects are Red overall, or if one or more are Red and the issues are collectively a threat to the Program's ability to meet its commitments.

35

Assessment RubricAssessment Rubric for Space Flight Projects and Multi-Project Programsfor Space Flight Projects and Multi-Project Programs

Cost Schedule Technical Programmatic

Cost estimates remain consistent with cost commitments:-- Formulation: Current estimated formulation costs are consistent with the program FAD. -- Development: Current estimated LCC at completion is at or below the LCC commitment. For multi-project programs, the current cost estimate by year for the program’s projects, including reserve, is within the 5 year cost run-out in the program plan. The program is consistently executing its projects within their cost cap or commitment.-- Operations: Annual operational costs are at or below planned costs for these operationsand there are minimal issues and adequate cost margin (budget reserve) to complete the Program or Project on plan: -Lifecycle cost reserves are consistent with plan and are being drawn down at a rate consistent with plan and emergent cost risks/threats. For multi-project programs, reserves are being managed consistent with the annual reserves planned for each project. -- The project/program will not require additional funds to complete the current fiscal year with required carryover. -- Phasing of cost reserves are consistent with emergent risks or threats. -- Current year budget level and timing of funds distribution are consistent with plan or commitments are not threatened as a result of any or delay in current year funding. -- External partners are providing key resources as planned.

Schedule progress remains consistent with schedule commitments: -- Formulation: The current estimated authority-to-proceed to implementation date is at or before the FAD.-- Development: Current estimated operational readiness date (ORD) is at or before the ORD commitment. For multi-project programs, the current estimated schedule for planned start dates for future projects is consistent with the program plan. -- Operations: Planned operations, including maintenance, are proceeding on schedule. and there are minimal issues and adequate schedule margin to complete the Program or Project on plan:--Funded schedule reserves & phasing are consistent with plan and are being drawn down at a rate consistent with plan and emergent schedule risks/threats. For multi-project programs, any faster-than-planned project draw-down of schedule reserve is not impinging on the ability of the program to meet its overall schedule commitments. --Key interim milestones (KDP as well as WBS milestones) being met. -- For multi-project programs, any delay in project schedule is not jeopardizing inputs required for another project or resulting in an increased risk to the program’s schedule commitments.-- External partners are providing key deliverables on schedule.

Technical progress remains consistent with technical commitments: -- Formulation: The project/program is making planned progress in developing technical design, reliability, safety, and risk mitigation requirements necessary to proceed to implementation as planned. -- Development: Technical requirements are stable and progress and risk management is consistent with plan while maintaining safety. For multi-project programs, any lack of planned technical progress at the project level is not resulting in an increased risk to the program’s technical commitments and objectives. -- Operations: Operational systems are delivering the planned level of services and reliability, with a general trend towards improved safety, and efficiency. and there are minimal issues and adequate technical margin to complete the Program or Project on plan:-- Technical reserves are being drawn down are consistent with plan; technical difficulties/risks encountered are typical for this type of project/program and are being mitigated or controlled as required to make planned technical progress. -- Are consistent with the requirements of safety, reliability & quality assurance, and incorporate good systems engineering & integration practice.-- External partners are making planned technical progress and external events (e.g., import approvals) are not altering technical requirements. -- For operational projects, the ground and in-flight hardware is operating satisfactorily on prime or backup units and mission requirements are being met.

Programmatic performance remains consistent with program/project requirements: -- Formulation: An approved FAD is in place and planned progress is being made in completing programmatic arrangements necessary to enter development as planned. -- Development: Top-level objectives and programmatic requirements in the plan are stable and programmatic capabilities are being developed/ managed as planned. Plans are consistent with 7120.5 or necessary waivers are in place. A multi-project program is effectively managing its projects, with new projects beginning per the Program Plan.-- Operations. Operational requirements are stable consistent with plan, and operations are being managed as planned. Retirement/ transition plans are being developed/ implemented consistent with the timing for these events. and there are minimal issues and adequate programmatic flexibility to complete the Program or Project on plan:--Key contract actions are being undertaken and completed as planned. The multi-project program’s acquisition strategy (both contracted and in-house) is resulting in the timely start of projects and delivery of project elements. --Key critical workforce, access to key facilities, EVM, and management systems are in place consistent with plan. --Key partnerships are in place and operating as planned; external requirements are stable; and there are no other factors external to the project/program (e.g., new Agency requirements) which are significantly disrupting planned project/program management.



Assessment Rubric forAssessment Rubric forSpace Flight Projects and Multi-Project Programs (continued)Space Flight Projects and Multi-Project Programs (continued)


In general, the program or project follows the green assessment guidelines, but cost commitments are now threatened.

Remaining cost margin appears inadequate to meet requirements due to one or more of the following deviations from plan, however, mitigation plans are in place or in development to recover margin and achieve commitments within the planned resources of the Program or Project:

-- Lifecycle cost reserves (including those held at the program level) are being drawn down faster than planned or required for emergent cost risks/threats.

-- Phasing of cost reserves are inconsistent with emergent risks or threats.

-- Current year cost growth is exceeding planned current year cost reserves and will require a re-phasing of the annual budget in order to complete the fiscal year with required carryover.

-- A reduction in current year budget or slow- down in funds distribution will require the project/program to rephrase future year annual costs.

-- External partner changes or factors external to the project/program are placing additional pressure on project/program costs.

-- For operating missions, in flight anomalies are draining cost reserves.

In general, the program or project follows the green assessment guidelines, but schedule commitments are now threatened.

Remaining schedule margin appears

inadequate to meet requirements due to one or more of the following deviations from plan, however, mitigation plans are in place or in development to recover margin and achieve commitments within the planned resources:

-- Schedule reserves are being drawn down faster than planned or as required for emergent schedule risks/threats.

-- Key interim milestones (KDP and/or WBS milestones) have slipped.

-- For multi-project programs, delay in project schedules is resulting in an increased risk to the program’s schedule commitments.

-- External partners are delayed in providing key deliverables or external factors are delaying completion of a key milestone.

In general, the program or project follows the green assessment guidelines, but technical commitments are now threatened:

Remaining technical margins appear

inadequate to meet requirements due to one or more of the following technical deviations from plan however, mitigation plans are in place or in development to recover margin and achieve commitments within the planned resources:

-- Technical reserves are being drawn down at a rate inconsistent with plan; or technical difficulties/risks encountered exceed those which are typical for this type of project/program or are not being mitigated or controlled as required to make planned technical progress.

-- Are not consistent in some key areas with the requirements of safety, reliability & quality assurance, and the incorporation of good systems engineering & integration practice.

-- External partners are not making planned technical progress or external events (e.g., import approvals) are altering technical requirements.

-- For operational projects, ground or flight capabilities have been curtailed, but plans are in work to recover the capability or develop a workaround such that mission requirements will still be met.

In general, programmatic performance remains consistent with program/project requirements for a green assessment, but remaining programmatic flexibility may be

inadequate to meet requirements due to one or more of the following programmatic deviations from plan however, mitigation plans are in place or in development to recover margin and achieve commitments within the planned resources:

-- Key contract actions are not being undertaken or completed as planned.

-- A multi-project program’s acquisition strategy (both contracted and in-house) is not resulting in the timely start of projects and delivery of project elements.

-- Key critical workforce, access to key facilities, EVM, and management systems are not consistent with plan..

-- Key partnerships are not in place or operating as planned; external requirements are not stable; or there are other factors external to the project/program (e.g., new Agency requirements) which are significantly disrupting planned project/program management.

37February 2012 For NASA Internal Use Only 37

Assessment Rubric forAssessment Rubric forSpace Flight Projects and Multi-Project Programs (continued)Space Flight Projects and Multi-Project Programs (continued)


The Program or Project is estimated not to meet its cost commitments without significant impact to other categories.

Depending on phase, the project's formulation costs, or development costs, or annual operating costs are exceeding the planned budget beyond all planned reserves.

For multi-project programs, total annual costs for the program’s projects exceed the program’s 5 year planned run-out, including reserves.

There is minimal or negative cost margin such that there is high risk that the Program or Project will require additional funding or relief from performance or programmatic requirements due to one or more of the following deviations from plan for a realistic recovery strategy has not been identified:

-- Lifecycle cost reserves (including those held at the program level) are being drawn down at an unsustainable rate and remaining reserves are insufficient given emergent risks/threats.

-- Phasing of cost reserves are inconsistent with emergent risks or threats.

-- Current year cost growth exceeds planned costs and cannot be accommodated by rephrasing annual costs.

-- A reduction or delay in current year funding is inconsistent with maintaining the overall LCC commitment.

-- A loss of key external partners or resources is inconsistent with keeping the project or program within planned costs.

-- For operational projects, in-flight anomalies have drained all cost reserves and the project will need more funds to complete the prime mission.

The Program or Project is estimated not to meet its schedule commitments without significant impact to other categories. The key end-of-phase KDP milestones are unlikely to be met.

For multi-project programs, the planned start-up of future projects is delayed beyond program commitments or objectives.

For operations missions, the current operations schedule is insufficient to meet project commitments or the mission failed prior to the planned end-of-operations date. For multi-project programs: The current operations schedule is insufficient to meet program commitments or the failure of one or missions prior to the planned end date has resulted in the project not being able to meet its commitments or objectives.

There is minimal or negative schedule margin is such that there is high risk that the Program or Project will require additional schedule or relief from performance or programmatic requirements due to one or more of the following deviations from plan for which a realistic strategy for recovery has not been identified:

-- Schedule reserves are being drawn down at an unsustainable rate and remaining reserves are insufficient given emergent risks/threats.

-- A slip of one or more key interim milestones (KDP and/or WBS milestones) is inconsistent with the planned ORD. For programs, delay in project schedules is inconsistent with program’s schedule commitments.

-- An external partner delays or delays caused by other external factors are inconsistent with the planned ORD. For multi-project programs, delays caused by external factors are inconsistent with the program’s schedule commitments.

The Program or Project is estimated not to meet its technical requirements without significant impact to other categories:.

A project in formulation is not making the technical progress required to enter development.

For a project in development, technical failures or delays preclude the project from being able to meet its technical requirements and commitments.

There is minimal or negative technical margin such that there is high risk that the Program or Project will require additional technical resources or relief from performance or programmatic requirements due to one or more of the following technical deviations from plan for which the project or program has not identified a realistic strategy for recovery:

-- Technical reserves are being drawn down at an unsustainable rate or emergent technical difficulties, risks, or threats are not being resolved as required to meet the technical commitments.

-- Are not consistent with the requirements of safety, reliability & quality assurance, and the incorporation of good systems engineering & integration practice.

-- Lack of technical progress by external partners or external events (e.g., import approvals) are inconsistent with meeting the project/program technical requirements, commitments, or objectives, or additional/different external cannot be met within plan.

-- For operational missions, in-flight anomalies have used up redundancy and made it unlikely that the mission will achieve its mission requirements.

The Program or Project is estimated not to be able to meet its requirements due to programmatic issues affecting the other categories.

A formulation project has failed to make the programmatic progress required to be confirmed for development.

A development project lacks an approved plan or has failed to implement that plan consistent with meeting its commitments. The multi-project program lacks an approved PCA or project plan, or has failed to implement the projects require to meet its commitments.

There is limited or no programmatic flexibility such that there is high risk that the Program or Project will require relief from performance or programmatic requirements due to one or more of the following deviations from plan for which the project/ program has not identified a realistic strategy for recovery:

--Key contract actions are not being undertaken or completed as required. The multi-project program’s acquisition strategy (both contracted and in-house) is not resulting in the timely start of projects and delivery of project elements.

--Key critical workforce, availability of key facilities, EVM, and management systems are missing or inconsistent with meeting the project’s commitments.

--Key planned partnerships did not materialize, quit or not to delivering consistent with project/program commitments; unstable or newly emergent external requirements are not implementable; or other external developments are inconsistent with the project/program commitments.


SummarySummary

• NASA Senior Management monitors Agency-Wide performance monthly through its baseline Performance Review (BPR) Process Mission Directorate detailed assessments are reported quarterly

on a rotating basis

• BPR continues to evolve and improve to reflect the Agency priorities and plans

Self Assessment – “There is a luxury in self-reproach. When we blame ourselves we feel that no one else has a right to blame us.” Oscar Wilde

Woods.edwards.pm challenge bpr presentation 2012 v1

Technology

Transcript of Woods.edwards.pm challenge bpr presentation 2012 v1