| Los Alamos National Laboratory |

April 2013 | UNCLASSIFIED | 1Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA

UNCLASSIFIEDUNCLASSIFIED

Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA

Monitoring the Effectiveness of the LANL Operating Experience Program

Robert B Stuewe

Operating Experience Program Coordinator

Los Alamos National Laboratory

May 14, 2013

| Los Alamos National Laboratory |

April 2013 | UNCLASSIFIED | 2Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA

UNCLASSIFIED

Scope of the LANL OPEX ProgramServes 11,000 people including the Los Alamos Field Office

OPEX Program Stats- 700-800 items processed per year

- Multiple pre-planned sources- 50% from DOE (all of you)- All program/function areas

- >100 internal institutional per year- 1500-3000 unique users/month- >5000 in searchable archive- >12,000 doc accesses/month- >200 searches/month (up to 400) Customer

Portfolio

NNSA Weapons Programs

NNSA Nonproliferation

NNSA Safeguards & Security

DOE Environmental Management

DOE Energy & Other Programs

DOE Office of Science

Work for Others ( incl. Nat Security)

| Los Alamos National Laboratory |

April 2013 | UNCLASSIFIED | 3Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA

UNCLASSIFIED

OPEX Monitoring and EvaluationUse Assessments, Measures, Analysis, Stakeholder Feedback

Annual- Assess OPEX Program Implementation and

Effectiveness (part of Annual CAS Assurance Process)

- Lessons Learned Improvement Workshop (includes Field Office)

Semi-Annually (or more frequently)

- Lessons Learned Effectiveness Monitoring (often includes Field Office)

Quarterly- ORPS/Sub-ORPS Trending and Analysis

Monthly- LANL Dashboard

Daily/Weekly- Lessons Learned Archive Analytics

As Occur- Other Internal and External Assessments (3-5 per year)

Lessons Learned Images: Adult Mexican Spotted Owl recorded on LANL property in June 2011. In compliance with the LANL

Habitat Management Plan, work with heavy equipment must stop in areas of sensitive

habit beginning March 1.

| Los Alamos National Laboratory |

April 2013 | UNCLASSIFIED | 4Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA

UNCLASSIFIED

OPEX Implementation Processes are designed to reliably fulfill requirements, and are used by functional, programmatic , and organizational areas important to the site’s mission

OPEX Effectiveness“There is objective evidence that experience from operational events is being tracked and used to drive continuous improvement” NNSA NAP-21 Element 2 Effectiveness Criteria

Annual CAS Management Self-Assessment

Part of LANS Board of Governors Approved Contractor Assurance System

DOE O210, O 226, and Prime Contract H-4 Clause set primary requirements

Uses feedback from interviews, measures, issues management system, LLEM, and other assessments

Results used by the Annual ISM Verification

Lessons Learned in Images: A subcontractor employee at LANL suffered the loss of a fingertip on 2/9/12, when he

caught a finger in the bolt hole shown above during the repair of a bucket on a skid-steer

loader.

| Los Alamos National Laboratory |

April 2013 | UNCLASSIFIED | 5Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA

UNCLASSIFIED

Monitoring and Evaluation ResultsSummarizing results from the last two years

The LANL OPEX Program is compliant, implemented, and is demonstrating effectiveness, but opportunities to improve effectiveness still exist

The use of lessons learned for work planning could be improved

Variability is evident in the lessons learned processes used by sub-units

The rate of sharing lessons learned across the Laboratory is at an acceptable level but is trending unfavorably

These opportunities for improvement are likely evident across the DOE.

| Los Alamos National Laboratory |

April 2013 | UNCLASSIFIED | 6Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA

UNCLASSIFIED

Many Lessons Learned are not easy to use for work planning

- Too general or truisms (e.g., Always follow procedures and you won’t have problems)

- Too site specific (e.g., Procedure GG-XY-0010-1.1 needs to be changed)

- Do not include actions that others could use to prevent the adverse event - Does not describe the activity or equipment that was associated with the lesson

in a manner that enables quick relevance determination during work planning

Redundancy and confusion in DOE LL directives and processes (When to use which PM lessons learned method, DOE-EM lessons learned process)

DOE-Wide Contributing Factors

Need to reduce cost-to-share while also creating higher-quality lessons for our activities?

| Los Alamos National Laboratory |

April 2013 | UNCLASSIFIED | 7Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA

UNCLASSIFIED

Additional Details

| Los Alamos National Laboratory |

April 2013 | UNCLASSIFIED | 8Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA

UNCLASSIFIED

Quarterly Lessons Learned Effectiveness Monitoring (LLEM)

Lessons Learned in Images: On 8/9/10 at TA-54 Area L, a 4-liter glass container of nitric acid began to vent a red vapor from

inside a 14-gallon poly drum. The drum was closed and the area evacuated. No workers

were exposed. HAZMAT responded and handled the cleanup.

1. Trending/analysis suggests that an accelerated rate of learning is needed

2. OPEX Program Coordinator selects the focus area, approves the data collection plan

3. OPEX Program personnel go into field to collect data (Field Office Counterpart can and does shadow)

Glovebox Safety Lessons Learned EffectivenessUse of LL in Training Development and Delivery R&D Safety Improvement Effectiveness Evaluation Project Management Lessons Learned ProcedureUse of LL in Maintenance Work PlanningUse of LL in Maintenance Pre-Job BriefsCrane Inspection Card Maintenance LL Follow-up

| Los Alamos National Laboratory |

April 2013 | UNCLASSIFIED | 9Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA

UNCLASSIFIED

OPEX LANL Dashboard MeasuresUpdated Monthly, Action Plans for Red and Yellow levels, accessible by Field Office to support oversight

Processing Time - Urgent within 24 hours- Caution within 7 days

Knowledgebase Inventory Internal LL Generation Rate Document Access Rate Search Rate

Staying Green requires monitoring knowledgebase analytics on daily and weekly bases, and taking immediate action before

monthly measures suffer!

Lessons Learned Images: Plumber’s kit containing propane bottle discovered 3/6/12

at the LANL CMR facility during the performance of an extent of condition

immediate action following a TSR violation that had resulted from the discovery of

another propane bottle on 2/15/12.