1 June 18, 2008

Data for Costing & Cost Uncertainty

Mort AnvariDirector, Acquisition Costing

(SAFM-CEA)

June 18, 2008 Cost Management Panel

PRICE Customer Symposium

2 June 18, 2008

Key Points

We have Several Data Quality Issues and Potential for Type I (α) and II (β) Errors in our Cost Estimating.

Opportunity for Cost Analysis Enterprise Resource Planning (CA-ERP) and Data for Costing Architecture.

Need for Range Cost Estimating and Cost Confidence Analysis

Need Data for Costing:Future Technologies; Processes; Performances; Products; Services; Personnel; and Schedule.

3 June 18, 2008

Army Cost Data Sources

DCARC CSDR Repository Automated Cost Databases (ACDB) Cost Analysis Requirements Description (CARD) Capability Knowledge Base (CKB) Operation and Support Management Information Systems (OSMIS) Contractor Logistic Support (CLS) Data Joint Integrated Analysis Tool (JIAT) Software Data Collection Initiatives

Army Cost Positions (ACP) Database ACE-IT Sessions, Periodical SAR, DAES, WSR, APB, P-Form, Contracts

Summaries

+

4 June 18, 2008

Data for Cost Estimating

Cost Models Historical Data: Cost and (Physical, Technical, Performance, Capability, Schedule).

Subject System Data: Physical, Technical, Performance, Capability, Schedule.

Information KnowledgeData

Performance

Cost Schedule

Estimates are only as good as the data that they are based on.

5 June 18, 2008

Data Issues

Current Data Reporting and Usage Issues (CSDR CCDR, SRDR, and CPR).

Family or System of Systems (Commonality Issue) LSI, JV, Prime, Sub (Contracting Issue) CPIF, FFP,. . Reporting Requirements (Contract Type Issue) Manufacturing, Services, SW, and ERP (Activity Type Issue) Mod, ECP, and Incremental Contracts (Completeness Issue) Competition and Business Base Issues (Quantity issue) Physical, Technical, TRL, Performance (Technology Issue)

KPP and Capability Data Issue

Allocation Issue: Mapping to Standard ReportsLabor – Material

Direct – Indirect

Recurring – Non-Recurring

6 June 18, 2008

Data Quality Challenges

Lack of data. Physical, technical, performance, capability. S/W ERP FFP Contract. ACAT II and below programs.

Policies and procedures. DoDI 5000-2 dated May 2003.

Requires SRDR (w/o Cost) but not CCDR for some programs. Vague in some areas; CSDR “not required for procurement of commercial systems, or for non-commercial systems bought under competitively awarded, firm fixed-price contracts, as long as competitive conditions continue to exist”.

Army regulations do not require additional cost reporting.

Data collection process. Existing data is not readily available to the Cost Analysts; sometimes they are not even aware that data exists. (Data in Jail) Contractors required to map CSDR reports IAW DoD MIL-HDBK-881.

7 June 18, 2008

Data Collection Process

Contractor

1

Contractor

1

Contractor 2

Contractor 2

Contractor3

Contractor3

Contractor Accounting Systems:

Contractor Accounting Systems:

S1

S3

S4

S2S5S6

S7

S8

S10

S9

Contractor Data / Reports:

Contractor Data / Reports:

R1

R3

R4

R2

R5

R6

R7R8

R10

R9

R11

R12

R13

R14

R15

R16

R17

R18

Labor vs. MaterialDirect vs. IndirectRec. vs. Non Rec.

Unique vs. CommonContractor allocation

Contracting

Labor vs. MaterialDirect vs. IndirectRec. vs. Non Rec.

Unique vs. CommonContractor allocation

ContractingG1

G2

G3

Co

ntr

ac

tor

All

oc

ati

on

Data Quality?

Process affects data quality. Increases probability for data errors. Results in mapping / allocation inconsistencies.

Process vs. Quality.

COST AND SOFTWARE DATA REPORTING PLAN Form Approved OMB No. 0704-0188

1a. PROGRAM 2a. WEAPON SYSTEM TYPE 3. SUBMISSION TYPE 4. DATE AS OF (MM/DD/YY) 5. REPORT DATE (MM/DD/YY)

X INITIAL SUBMISSION

1b. MILESTONE A B X C: LRIP C: PROD CHANGE

6. POINT OF CONTACT (POC) INFORMATION 7. W BS 8. PREPARING ORGANIZATIONa. POC AND ADDRESS (Include ZIP Code) X PROGRAM

CONTRACT 9. REVIEW AND REFERENCE NUMBER

12. 13.CONTRACTOR CONTRACT d. DD 2630

a. PROGRAM b. CONTRACT (DUNS Code) NUMBER REQUIRED

DD FORM 2794, Oct 2003 PREVIOUS EDITION IS OBSOLETE

6d. E-MAIL ADDRESS:

REQUIREDWBS

REPORTING ELEMENTS

14. REPORT FREQUENCYa. DD 1921 b. DD 1921-1 (Part 1) c. DD 1921-1 (Part 2)

Public reporting burden for this collection of information is estimated to average 15 hours per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing the burden to Department of Defense, Washington Headquarters Service, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204 Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provisions of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.

REQUIRED

6b. TELEPHONE NUMBER

6c. FAX NUMBER

10.WBS ELEMENT CODE

11.

REQUIRED

CSDR

8 June 18, 2008

Cost Data Flow

Need for Data Warehousing. Repository of electronically stored data.

Several databases typically supply data to the warehouse.

Improved ease of data retrieval and data quality.

Contractor Cost Accounting Systems

Government InformationalSystems / Databases

Contractor

Sys1

Contractor

Sys2

Contractor

Sys3

Contractor

Sys4

ACDB OSMIS CLS

Current Information Access Flow.

Cost Analysts

9 June 18, 2008

Data Warehousing

Raw Data loaded directly from contractor accounting system into data warehouse.

Metadata: ‘Data about Data’.

Data visualization / data warehousing / intelligent data mining.

Global definitions that can be referenced by many different databases.

InformationInformationDelivery SystemDelivery System

Government InformationalSystems / Databases

DataWarehouse

ExternalData

ExternalUsers

Proposed Information Access Flow.

Contractor Cost Accounting Systems

Data Marts

10 June 18, 2008

Cost Analysis ERP

Potential for Enterprise Resource Planning (ERP) of Cost Analysis Data and Models across the DoD.

Enable data and information sharing. Allow automation and integration of business processes.

Cost Analysts need to be more involved in the process.

Data for Costing – Vision: Collect raw data with more flexibility and focus on metadata development – XML. Data warehousing of raw and current data. Acquiring data mining and search tools to assist data analysis.

Data ERP for Costing? Check out http://www.anvari.net/E_Proc_Model.htm

11 June 18, 2008

Cost Risk Analysis

Risk analysis reduces the uncertainty between requirements and funding

12 June 18, 2008

Con

fiden

ce

100%

75%

50%

0%

25%

Cost $

Causes of Cost Variations

Cost Confidence Curve or “S” Curve

Technical & Schedule Risk (Included)

Cost Estimating Methodology Uncertainty (to-be Included)

– Engineering– Analogy– Parametric

• Physical Parameters• Technical Parameters• Performance Parameters• Capability Parameters

– Expert Opinion

Programmatic Changes (Mod) (Excluded)– Restructuring– Reprogramming – Rescheduling– Resourcing

13 June 18, 2008

Current:• Point Estimate Costing and Reporting, with

Additional Cost Range Modeling Option at Technical & Schedule Risk Items.

• Narrow Band Confidence. • Utilities are Risk Management and Execution.

Proposed:• Range Estimate Costing and Reporting, with Point

Estimate Reporting Option at Different Confidence Level.

• Wide Band Confidence. • Utilities are Planning and Programming.

Current & Proposed Estimating Methods

14 June 18, 2008

Cumulative Distribution Functions

Increasing cost confidence from 50% to 75%

Con

fiden

ce L

evel

100%

75%

50%

0%

25%

Cost $

Tec

hnic

al R

isk

All Risks and Uncertainties

Path Forward: Increase cost estimating confidence and implement rigorous risk analysis

15 June 18, 2008

Backups

Backups

16 June 18, 2008

Current Cost Data Collection Efforts

Automated Cost Databases (ACDB) Primary source of data is CSDRs, CPRs, etc. Includes programmatic and technical information.

Cost Analysis Requirements Description (CARD) Redesigned to reduce variation (IAW Lean Six Sigma process).

On-line living document. Standardized format for documents. Controlled author access to CARD sections. Milestone feeder documents (draft) provided early.

New process allows variation to be controlled (man-hour avoidance).

Capability Knowledge Base (CKB) Performance/capability-based relationships. Uses capabilities and costs of current systems to provide rough order of magnitude estimates for capability-filling solutions. Cost support for earlier acquisition decisions.

Missiles / MunitionsAircraft

Com / Electronics

Surface Vehicles

17 June 18, 2008

Current Cost Data Collection Efforts

Operating and Support Management Information System (OSMIS) Used by over 1,400 DoD employees (government and contract). Over 25,000 queries generated annually. Data used to generate OPTEMPO Weapon System Factors; direct link to PPBE process. Historical archive of O&S data for period covering 1993 to 2007. Prior year data is available by 1 April of the following year; Monthly activity reports by serial number level are available by 25th of the following month.

Contractor Logistic Support (CLS) Data CLS accounting for more of the total OPTEMPO requirement. Established a standardized process for CLS data collection. Requirement place on contract in conjunction with CSDR requirements. Cost data will enable the Army to forecast more reliable cost estimates for supply support.

18 June 18, 2008

Current Cost Data Collection Efforts

Joint Integrated Analysis Tool (JIAT) Cost estimating, engineering design, requirements, capability and performance analysis linked. Ready access to cost and technical data in each commodity area: Missiles, Aircraft, Vehicles and Communications and Electronics Systems. Supports cost as an independent variable (CAIV) analysis, capabilities costing and trade-off analysis. Near real-time cost estimating capability to the acquisition and requirements communities.

Software (SW) Data Collection Initiatives Collecting SW development actuals on past programs. Emphasis on improving SW maintenance estimates . . . Working with Army SW Engineering Centers collecting SW maintenance actuals. Partnering with Air Force & Navy . . . Collect & share data on Enterprise Resource Planning (ERP) Projects and SW intensive programs.