NPR 8621.1C Status

National Aeronautics and Space Administration

www.nasa.gov

NPR 8621.1C Status

NASA Safety & Health Managers’ Meeting

March 27, 2013

Steve Lilley

NASA Safety Center

1


www.nasa.gov 2


www.nasa.gov

Your handout is a preview to the draft

3

When 8621.1C is published, it will be more extensive


www.nasa.gov

NPR 8621.1 Revision Process History• August 2011 – Kickoff and Step 1: Justify requirements as a set

- Charter the committee - Cost/ benefit analysis

• September 2011 – March 2012: Step 2: scrub all 461 requirements X 9 separate tests-Restack Chapter 1 requirements -Necessary? Achievable? Verifiable?

• March 2012 -- July 2012: Step 3: Outline and First Draft - “Gang of Five” detailed scrub and streamlining - Revisit teams for V & V

• July 2012 -- August 2012 – Technical Review by Mishap Program Working GroupGoals: find and fix showstoppers. Verify the streamlined organization is more

usable.

• August-November 2012 –Step 4: “Gang of Five” statuses ALL 836 MPWG comments

• Traceable on NSCKN - Project schedule drove G of 5 to finish by Nov 30

• November 2012-March 2013: Step 4 at OSMA: independent V&V of second draft

• Second Directives Control Board

• April 2013-Draft into NODIS

• OSMA tech writer formatting for NODIS entry

4


www.nasa.gov

Thanks to the team

Marv Bunting

Marilyn Chambers

Dan Clem

Chris Counts

Izzy Davila

Beth Dickey

Manny Dominguez

Kristen Easton

Lou Ann Fikes

Jennifer Franzo

Wayne Frazier

Kristie French

Laurel Gaab

Mark George

5

Isi Greenfeld

Don Hall

Norma Horton

John LaPointe

Lynne Lee

Dave Loyd

Jimmy McLaughlin

Jon Mullin

Rick Parker

Debbie Ripley

Rebecca Selvage

Dan Thomas

Casey Tull


www.nasa.gov

New graphics in the draft

6

We split the Figure 2R&R table, in two:

New

New


www.nasa.gov

New graphics in the draft

7

Replaced table and lots of text with one

self- explanatory table

Old New


www.nasa.gov 8

New graphics in the draft-Mishap Report Outbrief Process-1


www.nasa.gov 9

New graphics in the draft- Mishap Report Outbrief

Process-2


www.nasa.gov

New graphics in the draft-current IRIS reporting requirements

10

Current guidance on everything required for IRIS entry.

Several data elements are called out, but overall guidance is open to much interpretation.


www.nasa.gov

New graphics in the draft-Incident reporting 1

11

Ensures 29 CFR 1904 and NPR 8621.1C investigation requirements are met.

Ensures that, regardless of the system used, an agency-wide data set is published and traceable.

Provides much-improved ability to understand causal scenarios behind high-risk incidents of any classification.

Supports modern safety management systems in use today by NASA (VPP, ANSI Z-10, ICAO SMS)


www.nasa.gov

New graphics in the draft-Incident reporting 2

12

All elements were agreed upon at the MPWG Technical Review in August 2012.

Who-What-When-Where-Why justification.

This concludes the 8621.1C update. Thanks for your time!

Are there any questions?


www.nasa.gov

Backup

13


www.nasa.gov

Requirements, Coming and Going

• Revision process began August 11, 2011 at MIWG annual meeting.• By July 6, 2012, the following had been accomplished:

• Teams of investigative experts and senior staff had reviewed all 461 requirements and surrounding policy language in detail to establish necessity, attainability and verifiability.

• 115 requirements were identified for removal, most as redundant.

• 29 new requirements were identified for inclusion based on inputs from each Center.

• All remaining requirements were re-stacked in functional order. Roles and responsibilities were collected and called out for each step in the process.

• 193 requirements-59 redundant = 134 requirements in the current Chapter 1 overview were flowed into each process step and Chapter 1 was eliminated.

14


www.nasa.gov

Chapter-by-Chapter Highlights

• Current Chapter 1 Approximate Requirements flow:

15

Ch 1 – Readiness to Conduct Investigations Ch 2 – Initial Response to a Mishap or Close Call

Ch 3 – Select the Investigating Authority and Support Ch 4 – Mishap Investigation Process

CH 5 – Mishap Report Ch 6 – Post-Investigation Activities


www.nasa.gov

Readiness to Conduct Investigations

• Current Chapter 2 will be the new Chapter 1.

• Highlights:• Mishap Preparedness Contingency Plan section becomes one

list of plan topics, with Center vs. Program guidance broken out.• Readiness Roles and Responsibilities all together in one

section.• All training is consolidated into this chapter, recognizing

differences between knowledge and skills needed in the field versus administratively.

• 34 redundant requirements removed, 2 new requirements added.

16





www.nasa.gov

Initial Response to a Mishap or Close Call

• Current Chapter 3 will be the new Chapter 2.• Highlights:

• NASA mishap classification criteria changed to match DoD.

• NTSB notification scenarios for aircraft increased from 11 to 16.

• IRT Roles and Responsibilities:• IRT Lead R&R added

• Contractors may collect non-privileged witness statements

• New guidance for Human Test Subject incidents.• Coordination with OCHMO and Institutional Review Board process per 45 CFR

46.103

• 30 redundant requirements removed.

17





www.nasa.gov

Select the Investigating Authority and Support


• Simplified table to identify the cognizant Appointing Official per mishap classification and involved personnel/property.

• Re-organized and clarified selection criteria for Investigating Authority and members.


18





www.nasa.gov

Mishap Investigation Process


• Evidence Collection: instituted defined concept of ‘evidence’ instead of ‘data’• Simplified or removed much non-requirement advice regarding evidence collection

(now implemented in formal training)

• Determine Findings: added analytic rigor to outcome requirements, yet maintained flexibility on analytic methods.

• MWAR procedures added.

• 30-day Report requirements specified.

• Recommendations: should assess and rate for mishap-preventive effectiveness to assist AO post-mishap.


19





www.nasa.gov

Mishap Report


• Streamlined A/B/HV endorsement process at the Outbrief.

• Public Release Review procedures codified.

• Mishap Summary format and review process established.

• IRIS Common Data Elements identified.


20





www.nasa.gov

Post-Investigation Activities


• Rationale for requirements better-documented.


21





www.nasa.gov

Definitions

17 new definitions:• Control (per NASA RCA training)

• Evidence (physical, demonstrable, testimonial, documentary)

• Hazard (per NASA-STD for SMA terms)

• Human Error (per NASA RCA training)

• Initiating Event (per NASA RCA training)

• Intent for flight (per DoD)

• Launch (per NASA-STD for SMA terms)

• NASA Aircraft (per NPR 7900.3)

• Probable cause (per Center input)

• Procedure (per NASA-STD for SMA terms)

• Process (per NASA-STD for SMA terms)

• Range (per NASA-STD for SMA terms)

• Risk (per NASA-STD for SMA terms)

• Spacecraft (per NASA-STD for SMA terms)

• Test (per NASA-STD for SMA terms)

• Timeline (per NASA RCA training)

• Undesired Outcome (per NASA RCA training)

22


www.nasa.gov

Definitions continued

4 Re-worded definitions:• Aircraft mishap >>Aircraft flight or ground mishap• Cause (includes precedence, necessity, relational tests)• Executive Summary (expanded)• Human Factors (expanded per NASA RCA training)

23


www.nasa.gov

Methodology & Assumptions

In support of the review and revision of NPR 8621.1, this cost – benefit analysis is developed as part of the “Requirements Review & Evaluation Process” to characterize overall effort, necessity, and effectiveness of the NASA Mishap Investigation process. •Each mishap investigation requires dedicated performance from defined resources over a defined period of time. An additional 50% margin is added to this estimate to account for consultation, endorsement and critique. Investigation hours are averaged across a 5-year history.

•Estimates of mishap consequences are based on National Safety Council (NSC) estimates of disabling injuries (including permanent disability, partial disability, or lost time) costing approximately $50,000. Property damage costs are directly recorded.•A 2008 RAND study estimates that a comprehensive safety and health standard results in 20-40% reduction in workplace injuries beyond common compliance. A similar study by OSHA, and NASA’s agency-wide incident rate both confirm 50%-60% mishap avoidance below comparable industry.

•It is assumed that the mishap investigation process is responsible for a portion of the mishap prevention (or subsequent mishap recurrence) benefit derived from the RAND, OSHA and NASA estimates.


www.nasa.gov

Investigation Costs

NASA Mishap Investigation Cost 2006 - 2010

TOTAL Count 06-10 Ave /Yr Hours/MI * Cost/Hour Cost/MI

TOTAL Cost 06-10 Ave Cost/Yr.

Type A 10 2.0 4500 $75 $337,500 $3,375,000 $675,000

Type B 71 14.2 2700 $75 $202,500 $14,377,500 $2,875,500

Type C 1457 291.4 32 $75 $2,400 $3,496,800 $699,360

Type D 2305 461.0 8 $75 $600 $1,383,000 $276,600

Close Call 5887 1177.4 4 $75 $300 $1,766,100 $353,220

TOTAL 9730 1946.0 $24,398,400 $4,879,680

* Hours expended in A & B mishap investigations are augmented an additional 50% to account for consultation, endorsement, and critique activities.


www.nasa.gov

Cost of Mishap Consequences 06-10

2006 2007 2008 2009 2010 # Mishaps Loss/Mishap*TOTAL Losses

Injury 610 597 619 584 581 2991 $50,000 $149,550,000

Mission 0 2 2 1 0 5 $228,000,000

Property 184 181 159 186 172 882 $41,816 $36,881,712

TOTAL 794 780 780 771 753 3878 $106,867 $414,431,712

* Based on 2009 NSC estimate for "disabling injury". Property damage estimate is based on NASA captured costs.

Consequence Costs


www.nasa.gov

Benefits• Based on the total cost of mishaps consequences during the 2006 – 2010 period

($414 million), NASA loses approximately $83 million each year to mishaps. • Based on RAND study, it is estimated that comprehensive safety and health program

activities can be credited with preventing 20-40% of mishaps beyond regulatory compliance. NASA’s performance is over 60% lower than comparable industry mishap rates. A 50% mishap prevention improvement is used for this analysis.

• As much as $83 million in additional mishap costs could be incurred by the agency each year (where $166 million represents theoretical un-mitigated mishap cost).

Safety & Health Program Phase FY10 BSL Funding Model *

Investment & Avoidance Proportion

Annual Cost Avoidance**

Design/planning phase $11.1 million 24% $19.0 million

Operational phase $25.8 million 54% $44.8 million

Post-accident phase $10.7 million 22% $18.2 million

* $47.6 million total agency institutional safety cost estimate.** Proportion of total $83 million in estimated annual cost avoidance.


www.nasa.gov

Conclusions• Based on this analysis, the mishap investigation process costs NASA approximately $5

million annually in direct investigation resources. The mishaps themselves have averaged an estimated $83 million annually in debilitating injuries and property damage. Together this represents $88 million annually in direct mishap costs. There are significant additional costs in administration, corrective action, and mission impact.

• Based on RAND and OSHA estimates and NASA performance, the highest performing safety and health programs realize as much as 40-60% improvement in mishap prevention performance over those organizations choosing to rely on regulatory compliance alone. With this in mind, it is estimated that a sound investigation process can contribute approximately 22% to the full cost avoidance estimate, or approximately $18.2 million annually.

• The close call proportion of the mishap investigation process represents a very small relative investigation investment – 7% of investigation costs – to learn from the largest overall proportion of opportunities – 61% of all mishap events – in the 2006 to 2010 period. Theoretically, the close call proportion of the investigation process could be responsible for as much as $11.1 million of the estimated annual cost avoidance.

NPR 8621.1C Status

Documents

Transcript of NPR 8621.1C Status