Data QualityData QualityThe Logistics Imperative The Logistics Imperative

Elaine S. Chapman

Defense Logistics Information Service

Chief, Data Integrity Branch

October 26, 2006

2

DNA of the DOD Supply Chain

What meets the requirement? How many do we have and where? or, Where/how can we obtain? How must it be handled?

Define NewRequirements Design Build Test Deploy Sustain Retire

Weapon System Lifecycle Management

Maintenance & ConfigurationAcquisition Management-Contract-Provision-Purchase

MaterialsManagement

& Warehousing

Distribution &TransportationManagement

Disposal

Material Supply and Services Management

OngoingRequirements

& Demand Management

Quality

Finance

Reporting

Retail

Who is the customer?What is needed?

How many are needed?Where is it needed?

DLIS

SupplierSupplierSupplier

SupplierSupplierSupplier

SupplierSupplierSupplier

SupplierSupplierSupplier

“Data is the DNA of supply chain management”• Acquisition• Financial management• Hazardous material• Freight & packaging• Maintenance• Sustainability• Disposal• Demilitarization

3

From a logistics perspective . . . supporting an F-15 is about 171,000 parts flying. . . and a Bradley is

about 14,000 parts rolling in close formation

DATA INTEGRITY . . . It’s About Parts

In this case $125,000,000

The price of a Mars Climate Orbiter

Mars OrbitInsertion Burn

M/D/Y HH:MM:SS PDT(Earth Receive Time, 10min. 49 sec. Delay)

Distance (miles) Speed(miles/hr)

Force(Pounds)

Begin 9/23/99 02:01:00 121,900,000 12,300 143.878End 9/23/99 02:17:23 9,840

Mars OrbitInsertion Burn

YYYYMMDD EDT(Earth Receive Time, 10min. 49 sec. Delay)

Distance (km) Speed(km/sec)

Force(Newtons)

Start 19990923 05:01:00 196,200,000 5.5 640

Finish 19990923 05:17:23 4.4

How Expensive Can Bad Quality Data Be?

5

Root Causes of Poor Data Quality

Shared Data Problems

Interface

Interface Disconnects

Novaces, LLC

6

Benefits

• Saving money right from the start– $1 to correct an error at data entry– $10 to correct a number of errors after the fact with batch

processing– $100 cost of not correcting an error

• Benefits– Eliminates time to reconcile data– Alleviates customer dissatisfaction– Prevents loss of system credibility– Eliminates system downtime– Prevents some revenue loss– Assists with compliance issues

7

Master Data Management

• Authoritative sources • Data Standards• Meta Data

BSM

NAVY ARMY

…No New Stovepipes Air Force

8

Vendor Master Example

Data Input

Public Web Search

Data Output

3rd Party Data Validation/Certification

CCR ToolsCCR Tools

On-lineDUNS

Validation

USPS

XML TransactionsXML Transactions

Federal Reserve

CAGESBA

On-lineD&B Parent

LinkageDaily

Extracts

ERPDLA Business Systems

Modernization, Army Logistics

Modernization Program

NOC NO ANNUAL UPDATE = INACTIVE

> 500 BUSINESS RULES> 500 EDITS

CAGE INTERNATIONAL

IRS

9

• Knowledge exchanges with the experts – Universities, Gartner, others

• Plan addresses: People-Process-Technology

− Management priority / visibility

− Program managers: overall responsibility

− Data stewards: analyze, measure, report and support PMs

− Elaborate, fact-based methodology / measures

− Edits, profiling tools and system checks

DLIS Data Quality Process

Action PlanDefine – Identify Data IssuesMeasure – Apply appropriate metrics.Improvements – Address needed enhancements.Implement – Initiate approved changes/corrections.Monitor – Re-measure for effectiveness.Report – Document status improvements and cost savings.

Action PlanDefine – Identify Data IssuesMeasure – Apply appropriate metrics.Improvements – Address needed enhancements.Implement – Initiate approved changes/corrections.Monitor – Re-measure for effectiveness.Report – Document status improvements and cost savings.

The Process

Data Quality Methodology

PeopleProcessTechnology

86%

DCB Recommendations:Begin checking additional DRNs

100% 92.7%

PM/DS: J6B/Wendy Ball/Roy Marko/Lori RowleyParticipants: Wendy Ball/Lori Rowley 25 Jan 05

Overall J6B quality assessment of FLIS on DLA Mgd NIINs/DRNs where FLIS or BSM is the authoritative source

DQ ISSUES A CR CMOverall

Grading Scale

100%

DQ ISSUES A CR CMOverall

100% NM 100% 100%

100% NM 100% 100%

100%100%NM100%

Process Step – Measure/Baseline

CN CN

85%

NM

NM

NM

NM 100% NM 100% 100%

100% 100% 100% 100% 100%

NM 100% NM 100% 100%

100% NM 100% 63.1% 87.7%

NM 100% NM 100% 100%

NM 100% NM 100% 100%

90-100% A Green80-89% B Yellow70-79% C Orange60-69% D Pink59%-0% E Red

Not Established - White

Issues/Concerns:

A – Accuracy CN – Consistency CR – Currency CM- Completeness NM – Not Measured

1. Shelf Life Code

2. Jump to Code

3. Order of Use Code

5. Precious Metal Indicator Code

6. Quantity PerAssembly

7. Federal Stock Class

8. Reference Numbers BSM Data Cleansing (BR2) PID project

4. Demil Code 9. Reference NumberCategory Code

10. Reference NumberVariation Code

AFTER: FLIS Status as of25 Jan 05

System/Product DQ Baseline

JTAV First Look Status

PM/DS: Mary Faber, Brad Williams / Lori RowleyParticipants: DLIS-SIQ/SXS

System/Product System/Product Benchmark

Example of Baselines, Benchmarks, Trends, Gaps and Quarterly Changes

0

20

40

60

80

100

Data Quality Issues

Pe

rce

nt

or

Gra

de

ACCURACY

CONSISTENCY

CURRENCY

COMPLETENESS

BENCHMARK

Revision:Date:

Root Cause AnalysisProblemIdentified

NoNo NoNoPolicyProblem?

TrainingProblem?

InterfaceProblem?

UnassignedError

YesYesYesYes

Adequate/Current?

DoesProcedureExist?

No

Yes

Establish/Administer adequatetraining, Policy, Procedure Edits, or Interface required

Procedure Problem?

Internal System Problem?

Does Interface

Exist?

Does EditExist?

Does PolicyExist?

Does TrainingExist?

System/Program Approval/Assistance

Methodology

Accuracy

Currency

G

G

DCB Recommendations:

Completeness G

Consistency

G

PM/DS: Mary Faber Brad Williams, DLIS-SXS Participants: SXS/SIQOverall Grade Date Briefed:

Process: Describe the process

Problems/Errors: Describe the problems or deficiencies found

Measurable observations: Annotatethe findings

Desired Improvement/need: Statethe desired improvement or need

Complete extract all NSNsComplete extractall NSNs

Complete extractall NSNs

Complete extractall NSNs

Characteristic Grade Percentage

68%

82%

95%

75%

Target Population: Example: FLIS Issues/Needs/Concerns: Address anyknown conflicts regarding suggested improvement; any methods or tools required;and overall concerns.

The Results

Accuracy

Consistency

Currency

Completeness

11

DQ Applied

• Identified top five queries for program• Worked with Program Manager to prioritize data elements• Broke them into small pieces that can be measured• Worked with contractor• Began building metrics• Get downstream systems involved in reviewing/implementing

solutions

12

NM

DCB Recommendations:

45% NA

PM/DS: Teresa Lindauer / Rich HansenParticipants: Date Briefed: 5 Jun 06

Quality Assessment Oct 06

DQ ISSUES A CR CM Overall

Grading ScaleWe have identified 5 of 117 public queries. We have researched 3 of 105 data elements of the 5 queries

NM

DQ ISSUES A CR CM Overall

NM NA

49% 49%

NM

Process Step – Measure/Baseline

CN CN

NM 49%

NM30% NM

NM

NM

NA4%NM

NM48%

NM

NM 0%NMNM NM

NM NM NM

NM NM NM

Baseline Grade%

90-100% A Green 80-89% B Yellow 70-79% C Orange 60-69% D Pink 59%-0% E RedNot Established - White

Issues/Concerns:

A – Accuracy CN – Consistency CR – Currency CM- Completeness NM-Not Measured

NM NA

NM NA

48%

24%

98%NM NM

NM

NM

52%

0%

Data Date: 1 Jun 06

11 CharacterIn RF ITV

Wrong VanOwner Data

No VanOwner Data

Leading Zeros

Cross DockOperations

AV DODAAC

NM

NM

49% 49%

A0 VLIPS flow toAV

AS VLIPS flow toAV

AE VLIPS flow toAV

DR VLIPS flow toAV

13

Root Cause Analysis

Doc IDError Types 4 - Interface5 - Procedure

Proposed solution

System/Product:

Asset Visibility/ In-Process-Doc ID

Revision: 1 Date: 15 May 06

Training problem?

Analysis step (DQ indicators 1., 2.)

Procedure problem?

Policy problem?

The Doc ID are not flowing from DMARS/SOMA

to AV

Document resolution and close problem

Monitor improvement

via metric

Interface problem?

Procedure problem?

The DOC ID A4x and ACx are not in the

AV ODS

Analysis step(DQ indicator

3. and 4.

Other Error

Proposed solution

No

No

No

No

No

Yes

Yes

Yes

14

Template for Analysis of System Potential IssuesAnalyze and Improve

DQ Indicator Root Cause Current Status Proposed Solution

1. Doc IDs not flowing from DMARS/ SOMA to AV

AV’s “in Process” functionality is primarily designed to answer questions that support the “requisitioning organization”.

AV processes AE1, AE2 and AE3 Doc IDs which provides the requisitioning organization status by document number. The Doc IDs AE6, AE8 and AE9 provide depot level status and do not flow to AV.

Because we were unable to evaluate SOMA raw data at entry to AV we performed the next best available comparison: AV to WEBVLIPS. WEBVLIPS and DMARS/SOMA are both DAASC products. Comparisons were made based on equivalent queries to both AV and WEBVLIPS. AV DS briefed DQ indicator results to (1.) to AV PMO.

Short term - Keep the status quo since AV’s “in Process” functionality is primarily designed to answer questions that support the “requisitioning organization” and it meets that requirement.

Long term – Combine the WEBVLIPS an AV capability into AV and Eliminate WEBVLIPS

15

Using Standards to Ensure Quality

• The NATO Codification System is the foundation of an international standard for product and service descriptions

• The eOTD is an open standard for encoding product data through the life cycle of a product – from design through disposal

16

ISO 8000**in development

• Labeling: Each data element must be tagged using a globally unique identifier that can be resolved to its terminology through a free (anonymous) internet interface

• Originating and cataloging organizations *

The originating and cataloging organizations for each data element must be identified using globally unique identifiers that can be resolved to contact information through a free (anonymous) internet interface

• Origination and cataloging date*

The origination and cataloging date of each data element must be specified

17

Requirements of ISO 8000

Information quality at the level of the organization

• Assessment of the level (grade) of information management capabilities

18

What can you do?

• Insist on access to quality information

• Participate in the development of Standard Identification Guides to define your data requirements

• Promote alignment and interoperability among standards efforts by insisting on standard data labeling (internally and externally)

• Encourage your data providers to prepare for ISO 8000

19

Data Quality

20

Questions?