National Aeronautics and Space Administration
www.nasa.gov
NPR 8621.1C Status
NASA Safety & Health Managers’ Meeting
March 27, 2013
Steve Lilley
NASA Safety Center
1
National Aeronautics and Space Administration
www.nasa.gov 2
National Aeronautics and Space Administration
www.nasa.gov
Your handout is a preview to the draft
3
When 8621.1C is published, it will be more extensive
National Aeronautics and Space Administration
www.nasa.gov
NPR 8621.1 Revision Process History• August 2011 – Kickoff and Step 1: Justify requirements as a set
- Charter the committee - Cost/ benefit analysis
• September 2011 – March 2012: Step 2: scrub all 461 requirements X 9 separate tests-Restack Chapter 1 requirements -Necessary? Achievable? Verifiable?
• March 2012 -- July 2012: Step 3: Outline and First Draft - “Gang of Five” detailed scrub and streamlining - Revisit teams for V & V
• July 2012 -- August 2012 – Technical Review by Mishap Program Working GroupGoals: find and fix showstoppers. Verify the streamlined organization is more
usable.
• August-November 2012 –Step 4: “Gang of Five” statuses ALL 836 MPWG comments
• Traceable on NSCKN - Project schedule drove G of 5 to finish by Nov 30
• November 2012-March 2013: Step 4 at OSMA: independent V&V of second draft
• Second Directives Control Board
• April 2013-Draft into NODIS
• OSMA tech writer formatting for NODIS entry
4
National Aeronautics and Space Administration
www.nasa.gov
Thanks to the team
Marv Bunting
Marilyn Chambers
Dan Clem
Chris Counts
Izzy Davila
Beth Dickey
Manny Dominguez
Kristen Easton
Lou Ann Fikes
Jennifer Franzo
Wayne Frazier
Kristie French
Laurel Gaab
Mark George
5
Isi Greenfeld
Don Hall
Norma Horton
John LaPointe
Lynne Lee
Dave Loyd
Jimmy McLaughlin
Jon Mullin
Rick Parker
Debbie Ripley
Rebecca Selvage
Dan Thomas
Casey Tull
National Aeronautics and Space Administration
www.nasa.gov
New graphics in the draft
6
We split the Figure 2R&R table, in two:
New
New
National Aeronautics and Space Administration
www.nasa.gov
New graphics in the draft
7
Replaced table and lots of text with one
self- explanatory table
Old New
National Aeronautics and Space Administration
www.nasa.gov 8
New graphics in the draft-Mishap Report Outbrief Process-1
National Aeronautics and Space Administration
www.nasa.gov 9
New graphics in the draft- Mishap Report Outbrief
Process-2
National Aeronautics and Space Administration
www.nasa.gov
New graphics in the draft-current IRIS reporting requirements
10
Current guidance on everything required for IRIS entry.
Several data elements are called out, but overall guidance is open to much interpretation.
National Aeronautics and Space Administration
www.nasa.gov
New graphics in the draft-Incident reporting 1
11
Ensures 29 CFR 1904 and NPR 8621.1C investigation requirements are met.
Ensures that, regardless of the system used, an agency-wide data set is published and traceable.
Provides much-improved ability to understand causal scenarios behind high-risk incidents of any classification.
Supports modern safety management systems in use today by NASA (VPP, ANSI Z-10, ICAO SMS)
National Aeronautics and Space Administration
www.nasa.gov
New graphics in the draft-Incident reporting 2
12
All elements were agreed upon at the MPWG Technical Review in August 2012.
Who-What-When-Where-Why justification.
This concludes the 8621.1C update. Thanks for your time!
Are there any questions?
National Aeronautics and Space Administration
www.nasa.gov
Backup
13
National Aeronautics and Space Administration
www.nasa.gov
Requirements, Coming and Going
• Revision process began August 11, 2011 at MIWG annual meeting.• By July 6, 2012, the following had been accomplished:
• Teams of investigative experts and senior staff had reviewed all 461 requirements and surrounding policy language in detail to establish necessity, attainability and verifiability.
• 115 requirements were identified for removal, most as redundant.
• 29 new requirements were identified for inclusion based on inputs from each Center.
• All remaining requirements were re-stacked in functional order. Roles and responsibilities were collected and called out for each step in the process.
• 193 requirements-59 redundant = 134 requirements in the current Chapter 1 overview were flowed into each process step and Chapter 1 was eliminated.
14
National Aeronautics and Space Administration
www.nasa.gov
Chapter-by-Chapter Highlights
• Current Chapter 1 Approximate Requirements flow:
15
Ch 1 – Readiness to Conduct Investigations Ch 2 – Initial Response to a Mishap or Close Call
Ch 3 – Select the Investigating Authority and Support Ch 4 – Mishap Investigation Process
CH 5 – Mishap Report Ch 6 – Post-Investigation Activities
National Aeronautics and Space Administration
www.nasa.gov
Readiness to Conduct Investigations
• Current Chapter 2 will be the new Chapter 1.
• Highlights:• Mishap Preparedness Contingency Plan section becomes one
list of plan topics, with Center vs. Program guidance broken out.• Readiness Roles and Responsibilities all together in one
section.• All training is consolidated into this chapter, recognizing
differences between knowledge and skills needed in the field versus administratively.
• 34 redundant requirements removed, 2 new requirements added.
16
Ch 1 – Readiness to Conduct Investigations Ch 2 – Initial Response to a Mishap or Close Call
Ch 3 – Select the Investigating Authority and Support Ch 4 – Mishap Investigation Process
CH 5 – Mishap Report Ch 6 – Post-Investigation Activities
National Aeronautics and Space Administration
www.nasa.gov
Initial Response to a Mishap or Close Call
• Current Chapter 3 will be the new Chapter 2.• Highlights:
• NASA mishap classification criteria changed to match DoD.
• NTSB notification scenarios for aircraft increased from 11 to 16.
• IRT Roles and Responsibilities:• IRT Lead R&R added
• Contractors may collect non-privileged witness statements
• New guidance for Human Test Subject incidents.• Coordination with OCHMO and Institutional Review Board process per 45 CFR
46.103
• 30 redundant requirements removed.
17
Ch 1 – Readiness to Conduct Investigations Ch 2 – Initial Response to a Mishap or Close Call
Ch 3 – Select the Investigating Authority and Support Ch 4 – Mishap Investigation Process
CH 5 – Mishap Report Ch 6 – Post-Investigation Activities
National Aeronautics and Space Administration
www.nasa.gov
Select the Investigating Authority and Support
• Current Chapter 4 will be the new Chapter 3.• Highlights:
• Simplified table to identify the cognizant Appointing Official per mishap classification and involved personnel/property.
• Re-organized and clarified selection criteria for Investigating Authority and members.
• 14 redundant requirements removed, 9 new requirements added.
18
Ch 1 – Readiness to Conduct Investigations Ch 2 – Initial Response to a Mishap or Close Call
Ch 3 – Select the Investigating Authority and Support Ch 4 – Mishap Investigation Process
CH 5 – Mishap Report Ch 6 – Post-Investigation Activities
National Aeronautics and Space Administration
www.nasa.gov
Mishap Investigation Process
• Current Chapter 5 will be the new Chapter 4.• Highlights:
• Evidence Collection: instituted defined concept of ‘evidence’ instead of ‘data’• Simplified or removed much non-requirement advice regarding evidence collection
(now implemented in formal training)
• Determine Findings: added analytic rigor to outcome requirements, yet maintained flexibility on analytic methods.
• MWAR procedures added.
• 30-day Report requirements specified.
• Recommendations: should assess and rate for mishap-preventive effectiveness to assist AO post-mishap.
• 14 redundant requirements removed, 9 new requirements added.
19
Ch 1 – Readiness to Conduct Investigations Ch 2 – Initial Response to a Mishap or Close Call
Ch 3 – Select the Investigating Authority and Support Ch 4 – Mishap Investigation Process
CH 5 – Mishap Report Ch 6 – Post-Investigation Activities
National Aeronautics and Space Administration
www.nasa.gov
Mishap Report
• Current Chapter 6 will be the new Chapter 5.• Highlights:
• Streamlined A/B/HV endorsement process at the Outbrief.
• Public Release Review procedures codified.
• Mishap Summary format and review process established.
• IRIS Common Data Elements identified.
• 18 redundant requirements removed, 7 new requirements added.
20
Ch 1 – Readiness to Conduct Investigations Ch 2 – Initial Response to a Mishap or Close Call
Ch 3 – Select the Investigating Authority and Support Ch 4 – Mishap Investigation Process
CH 5 – Mishap Report Ch 6 – Post-Investigation Activities
National Aeronautics and Space Administration
www.nasa.gov
Post-Investigation Activities
• Current Chapter 7 will be the new Chapter 6.• Highlights:
• Rationale for requirements better-documented.
• 5 redundant requirements removed, 2 new requirements added.
21
Ch 1 – Readiness to Conduct Investigations Ch 2 – Initial Response to a Mishap or Close Call
Ch 3 – Select the Investigating Authority and Support Ch 4 – Mishap Investigation Process
CH 5 – Mishap Report Ch 6 – Post-Investigation Activities
National Aeronautics and Space Administration
www.nasa.gov
Definitions
17 new definitions:• Control (per NASA RCA training)
• Evidence (physical, demonstrable, testimonial, documentary)
• Hazard (per NASA-STD for SMA terms)
• Human Error (per NASA RCA training)
• Initiating Event (per NASA RCA training)
• Intent for flight (per DoD)
• Launch (per NASA-STD for SMA terms)
• NASA Aircraft (per NPR 7900.3)
• Probable cause (per Center input)
• Procedure (per NASA-STD for SMA terms)
• Process (per NASA-STD for SMA terms)
• Range (per NASA-STD for SMA terms)
• Risk (per NASA-STD for SMA terms)
• Spacecraft (per NASA-STD for SMA terms)
• Test (per NASA-STD for SMA terms)
• Timeline (per NASA RCA training)
• Undesired Outcome (per NASA RCA training)
22
National Aeronautics and Space Administration
www.nasa.gov
Definitions continued
4 Re-worded definitions:• Aircraft mishap >>Aircraft flight or ground mishap• Cause (includes precedence, necessity, relational tests)• Executive Summary (expanded)• Human Factors (expanded per NASA RCA training)
23
National Aeronautics and Space Administration
www.nasa.gov
Methodology & Assumptions
In support of the review and revision of NPR 8621.1, this cost – benefit analysis is developed as part of the “Requirements Review & Evaluation Process” to characterize overall effort, necessity, and effectiveness of the NASA Mishap Investigation process. •Each mishap investigation requires dedicated performance from defined resources over a defined period of time. An additional 50% margin is added to this estimate to account for consultation, endorsement and critique. Investigation hours are averaged across a 5-year history.
•Estimates of mishap consequences are based on National Safety Council (NSC) estimates of disabling injuries (including permanent disability, partial disability, or lost time) costing approximately $50,000. Property damage costs are directly recorded.•A 2008 RAND study estimates that a comprehensive safety and health standard results in 20-40% reduction in workplace injuries beyond common compliance. A similar study by OSHA, and NASA’s agency-wide incident rate both confirm 50%-60% mishap avoidance below comparable industry.
•It is assumed that the mishap investigation process is responsible for a portion of the mishap prevention (or subsequent mishap recurrence) benefit derived from the RAND, OSHA and NASA estimates.
National Aeronautics and Space Administration
www.nasa.gov
Investigation Costs
NASA Mishap Investigation Cost 2006 - 2010
TOTAL Count 06-10 Ave /Yr Hours/MI * Cost/Hour Cost/MI
TOTAL Cost 06-10 Ave Cost/Yr.
Type A 10 2.0 4500 $75 $337,500 $3,375,000 $675,000
Type B 71 14.2 2700 $75 $202,500 $14,377,500 $2,875,500
Type C 1457 291.4 32 $75 $2,400 $3,496,800 $699,360
Type D 2305 461.0 8 $75 $600 $1,383,000 $276,600
Close Call 5887 1177.4 4 $75 $300 $1,766,100 $353,220
TOTAL 9730 1946.0 $24,398,400 $4,879,680
* Hours expended in A & B mishap investigations are augmented an additional 50% to account for consultation, endorsement, and critique activities.
National Aeronautics and Space Administration
www.nasa.gov
Cost of Mishap Consequences 06-10
2006 2007 2008 2009 2010 # Mishaps Loss/Mishap*TOTAL Losses
Injury 610 597 619 584 581 2991 $50,000 $149,550,000
Mission 0 2 2 1 0 5 $228,000,000
Property 184 181 159 186 172 882 $41,816 $36,881,712
TOTAL 794 780 780 771 753 3878 $106,867 $414,431,712
* Based on 2009 NSC estimate for "disabling injury". Property damage estimate is based on NASA captured costs.
Consequence Costs
National Aeronautics and Space Administration
www.nasa.gov
Benefits• Based on the total cost of mishaps consequences during the 2006 – 2010 period
($414 million), NASA loses approximately $83 million each year to mishaps. • Based on RAND study, it is estimated that comprehensive safety and health program
activities can be credited with preventing 20-40% of mishaps beyond regulatory compliance. NASA’s performance is over 60% lower than comparable industry mishap rates. A 50% mishap prevention improvement is used for this analysis.
• As much as $83 million in additional mishap costs could be incurred by the agency each year (where $166 million represents theoretical un-mitigated mishap cost).
Safety & Health Program Phase FY10 BSL Funding Model *
Investment & Avoidance Proportion
Annual Cost Avoidance**
Design/planning phase $11.1 million 24% $19.0 million
Operational phase $25.8 million 54% $44.8 million
Post-accident phase $10.7 million 22% $18.2 million
* $47.6 million total agency institutional safety cost estimate.** Proportion of total $83 million in estimated annual cost avoidance.
National Aeronautics and Space Administration
www.nasa.gov
Conclusions• Based on this analysis, the mishap investigation process costs NASA approximately $5
million annually in direct investigation resources. The mishaps themselves have averaged an estimated $83 million annually in debilitating injuries and property damage. Together this represents $88 million annually in direct mishap costs. There are significant additional costs in administration, corrective action, and mission impact.
• Based on RAND and OSHA estimates and NASA performance, the highest performing safety and health programs realize as much as 40-60% improvement in mishap prevention performance over those organizations choosing to rely on regulatory compliance alone. With this in mind, it is estimated that a sound investigation process can contribute approximately 22% to the full cost avoidance estimate, or approximately $18.2 million annually.
• The close call proportion of the mishap investigation process represents a very small relative investigation investment – 7% of investigation costs – to learn from the largest overall proportion of opportunities – 61% of all mishap events – in the 2006 to 2010 period. Theoretically, the close call proportion of the investigation process could be responsible for as much as $11.1 million of the estimated annual cost avoidance.
Top Related