NASA Ames Research Center OCIOAligning project prioritization to the

organizational roadmap

Vonnie Simonsen, PMPFebruary, 2011

Just How Important is Your Project?

Outline

Abstract Dilemma NASA Ames Research Center (ARC) OCIO Project Challenges Addressing the Gaps Selection & Prioritization High-level Process Requirements Leveraging GAO Best Practices Selecting the Appropriate Projects Select Process Overview Prioritization Evaluation Scorecard

2

Abstract

This presentation will discuss the development of the project prioritization approach, how it aligns to overall organizational goals and objectives, and the resulting benefits to the organization. It will also provide an overview of the governance structure.

3

Abstract

The NASA Ames Research Center OCIO has developed and implemented an agnostic approach to prioritizing projects within the directorate

The process allows for defining which projects are deemed most important by the organization and increases the effectiveness of work being accomplished that aligns to organizational and agency goals

The visual representation of the project prioritization on the OCIO dashboard provides the entire organization a clear understanding of the efforts deemed most important by the senior leadership

4

Dilemma

5

According to the Government Accountability Office (GAO) Accounting and Information Management Division:

“Poor investments, those that are inadequately justified or whose costs, risks, and benefits are poorly managed, can hinder and even restrict an organization’s performance.”1

1 Assessing Risks and Returns: A Guide for Evaluating Federal Agencies’ IT Investment Decision-making, GAO/AIMD-10.1.13

6

o Controlled cost, risk, and value (benefit) within the context of IT strategic mission

o Captured accurate estimates regarding scope, schedule, and cost early in the conceptual phase of lifecycle

o Captured over taxation of resources

o Discerned opportunity cost impacts across all proposed projects

o Allocated the proper resources and capabilities to the right efforts

o Ensured consistent and reliable results through objective, quantifiable criteria

o Communicated prioritizes throughout the organization

NASA ARC OCIO Project Challenges

• Project costs not controlled

• Competing resources across several projects

• Resources not managed efficiently

NASA ARC OCIO recognized consistent challenges in the ability to “on-board” proposed projects in a way that.…

Main challenges

7

NASA ARC OCIO Project Challenges (cont.)

Lack of project prioritization lead to:

Confusion as to which projects should be given attention by competing resources

Lack of consistent standards and messaging regarding importance of particular projects

Misallocation of resources to IT projects that did not best support mission and organizational needs

If everything is a priority then nothing is really a priority!

8

In order to address the lack of project prioritization:

Major Directorate selection and prioritization requirement gaps were identified

Industry and Government best practices were evaluated

GAO high level concepts for prioritization guidelines were adopted

Prioritization Scorecard was developed

Addressing the Gaps

Selection & Prioritization High-Level Process Requirements

High-Level Requirements:

• Align with NASA ARC Code I Project Management Office (PMO) NPR 7120.7 based framework

• Provide a governance framework for decision-making determinations on all proposed projects

• Capture high-level business needs

• Allow for screening and selection of proposed projects

• Ensure alignment of proposed projects with mission objectives

• Provide visibility into proposed project pipeline to better align project resources and determine priorities

9

Major requirements to address gaps were identified for the selection and prioritization process:

Leveraging GAO Best Practices

10

GAO best practice recommends that a successful IT investment management process have three fundamental phases:

» Select – How do you know you have selected the best projects?

o Project “on-boarding” process

» Control – How are you ensuring that the projects will deliver the promised benefits?

o PMO framework using 7120.7

» Evaluate – Are the solutions delivering what you expected?

o Project Completion Reviews (PCRs)

In addressing the needed requirements for development of the selection and prioritization process, NASA ARC OCIO leveraged GAO guidance on IT investment decision-making (GAO/AIMD-10.1.13)

How do you Select the appropriate projects?

We will focus exclusively on the select phase by outlining key steps incorporated into NASA ARC OCIO “on-boarding” process

Select phase represents the first stage in the “on-boarding” process

3 steps comprise the select phase:

11

1) Screen• Establish initial acceptance requirements all

proposed projects must meet• Ensure all proposed projects are being reviewed at

the most appropriate organizational level

2) Rank• All proposed projects meeting initial acceptance

requirements are analyzed based using GAO recommended minimum criteria of value (benefit), cost, and risk

• Evaluation score results in overall ranking

3) Determine• OCIO makes final project portfolio and prioritization

IT investment determination

12

• Establish initial acceptance requirements all proposed projects must meet

• Ensure proposed projects are being reviewed at the most appropriate organizational level

• All proposed projects meeting initial acceptance requirements are analyzed using NASA ARC OCIO criteria of value (benefit), cost, risk, and intangibles

• Evaluation score results in overall ranking

• ARC OCIO makes final project portfolio and prioritization IT investment determination

Screen

Rank

Determine

• Allow for screening and selection of proposed projects

• Capture high-level business needs

• Provide visibility into proposed project pipeline to better align project resources and determine priorities

• Ensure alignment of proposed projects with mission objectives

Business NeedQualification

Business NeedRefinement

Select PhaseNASA ARC OCIO

Process RequirementNASA ARC OCIO

Process Step

1

Select Process Overview

Proposal3

2

Selection Process Alignment with PMO Framework At a Glance

13

Phase AConcept

Development

Phase BPreliminary

Design

Acquisition, Development & Implementation

KDP - CKDP - B

Customer Decision Pt

Phase C, D, EFinal Design & Build

Assembly, Integration, TestDeployment & Operations

Alignment to PMO FrameworkInitial Request Evaluation

OCIODecision Pt

Selection Control

Eva

luat

ion

Business CaseDevelopment

Business NeedQualification

Business NeedRefinement

Proposal

PreliminaryDesign

“On-boarding” selection steps aligned to NASA ARC OCIO Project Management Office Framework

1 2 3

A B

OCIO Decision Pt

C

Screen Rank Determine

What special considerations must be

considered?

14

Evaluation criteria used for the select phase include factors for cost, value (benefit), risk, and intangibles

Basis of PrioritizationEvaluation Criteria

15

Prioritization Evaluation Scorecard

NASA ARC OCIO has developed evaluation scorecard containing specific questions and point values for all evaluation criteria

*Scorecard model adopted from the Value Measuring Methodology (VMM), CIO Council Best Practices Committee, October 2002

• DME• Ops. & Maint.• Resources

• Direct Users• Organization/Strategic• Mission

• Schedule• Technical• Programmatic• Security

• Special ConsiderationsScorecard

Cost

100 Pts Total

Value

Risk

Intangibles

30 Pts Total

30 Pts Total

20 Pts Total

20 Pts Total

16

Several approaches considered in devising suitable point values for each criteria

Higher point values considered for proposed projects that indicate/show the following (by factor):» Cost

• Realistic preliminary estimates for labor (FTE, WYE) and procurement (hardware/software) • Sustainability to the overall organization via Operations & Maintenance (O&M) efficiencies

» Risk• Low risk to the organization in the area of schedule, technical, programmatic, and security • Availability and utilization of “in-house” resources

» Value (Benefit)• Support a mission directorate, and/or align to OCIO strategic mission objectives • Show a strong EA alignment

Evaluation Criteria

17

Prioritization Ranking Example

Proposed Project Name

Estimated Project Costs/Year

Value (Benefit)(30 pts)

Cost(30 pts)

Risk(20 pts)

Intangibles (20 pts)

Total Score (100 pts)

Project 01 $375K 30 20 20 10 80

Project 02 $250K 30 20 20 7 77

Project 03 $500K 20 25 20 10 75

Project 04 $675K 15 25 20 5 65

Project 05 $150K 20 20 20 0 60

Project 06 $275K 10 25 20 5 60

Project 07 $300K 20 15 15 0 50

Project 08 $1,000K 5 10 20 10 45

Project 09 $100K 10 15 10 5 40

Project 10 $500K 15 10 5 0 30

Ap

pro

ved

PrioritizationEvaluation Scorecard

18

19

Basis of PrioritizationEvaluation Criteria Details

• DME (Development, Modification, Enhancement)

• O&M (Operations & Maintenance)

• Human Resources

• Direct Users• Organization/Strategic• Mission

• Schedule• Technical• Programmatic• Security

• Executive Management special considerations

What special considerations must be

considered?

Sample: NASA ARC OCIO Prioritization Scorecard*

20*Scorecard model adopted from Value Measuring Methodology (VMM) how-to-guide

Prioritization Scorecard:Understanding Cost Criteria

21

• DME• O&M• Human Resources

How MuchWill It Cost?

22

Prioritization Scorecard:Understanding Value (Benefit) Criteria

• Direct Users• Organization/Strategic• Mission

What Benefits Will It Provide?

23

• Schedule• Technical• Programmatic• Security

Prioritization Scorecard:Understanding Risk Criteria

What Could Make Costs Go Up or

Performance Slip From Projected

Levels?

Prioritization Scorecard:Understanding Cost Criteria

24

What special considerations

must be considered?

• Special Considerations

25

Selection process ensures that the NASA ARC OCIO

Matures its capabilities via a repeatable and efficient methodology for choosing IT projects that will best support mission needs and benefit

Makes smarter IT investment decisions

Prioritizes projects to better align resources to meet current and future project needs

Maintains a consistent set of standards and messaging around the importance of priority projects

In Conclusion. . .