Federal Aviation Administration Normal Operations Data: Air Traffic Facility Evaluations and NOSS...

Federal AviationAdministration

Normal Operations Data:Air Traffic Facility Evaluations and NOSS

Presented to: Second ICAO Global Symposium on TEM/NOSS in Air Traffic Control

By: Karen Burcham

Date: February 8, 2007

Safety Evaluations and NOSS2Federal Aviation

AdministrationFebruary 8, 2007

Briefing Objectives

• FAA Safety Data Sources

• Purpose, objectives, and philosophy of Safety Evaluations and Audits

• Internal Evaluation and Audit processes

• NOSS role and status



Heinrich’s Triangle and Safety

Lea

din

g In

dic

ato

rs

La

gg

ing

In

dic

ato

rs1Aircraft

Accident

~30Incidents

~300HazardousConditions

~1,000Unreported

“Unsafe Acts”

(OE, OD, NMAC, PD)(OE, OD, NMAC, PD)

(Non-Compliance, Bad(Non-Compliance, Bad Documentation, bad phraseology, etc.)Documentation, bad phraseology, etc.)

(Non-Reported OE’s/OD’s, Missed Readbacks, etc.)(Non-Reported OE’s/OD’s, Missed Readbacks, etc.)



Safety Data Reporting RequirementsFAA Orders and Processes Related to Safety Data Reporting

Type of Data Overview References Mandatory Reporting Data

Air traffic incidents

This order mandates that data concerning air traffic incidents is collected and analyzed.

Order 7210.56, Air Traffic Quality Assurance

Aircraft accident/ incident

This order contains reporting requirements regarding safety issues, concerns, incidents, and accidents.

Order 8020.16, Aircraft Accident And Incident Notification, Investigation, and Reporting

System outages This order mandates outage reports and contributes to the daily system performance and incident reporting.

Order 6040.15, National Airspace Performance Reporting System

Significant system events

This order mandates the reporting of significant events and contributes to the daily system performance and incident reporting.

Order 6030.41, Notification Plan for Unscheduled Facility and Service Interruptions and Other Significant Events

Unsatisfactory condition

This order provides Agency employees a means of advising management of unsatisfactory conditions.

Order 1800.6, Unsatisfactory Condition Report

Oceanic Altitude/Nav Errors

This order establishes procedures for processing reports and for collecting system data for analysis.

Order 7110.82. Monitoring of Navigation, Longitudinal Separation, and Altitude Keeping Performance in Oceanic Airspace



Safety Data SourcesFAA Orders and Processes Related to Safety Data Reporting

Type of Data Overview Mandatory Reporting Data

National Aviation Information Management System (NAIMS) and Air Traffic Quality Assurance (ATQA)

A database where safety events are collected and categorized.

Runway Safety Directorate within the Safety Service Unit

The Runway Safety Directorate categorizes surface events that include surface operational errors, operational deviations, pilot, vehicle, and pedestrian deviations.

Accident/Incident Reporting Data System (AIDS) The FAA AIDS database contains accident and incident data records for all categories of civil aviation.

NTSB Accident/Incident Database The NTSB accident/incident database is the official repository of aviation accident data and causal factors. In this database, events are categorized as accidents or incidents.

Operational Error/Deviation System (OEDS) Used by FAA to determine if actions of a controller resulted in a loss of separation or an aircraft landing or departing on a closed runway.

Pilot Deviation System (PDS) Used by FAA to determine if actions of a pilot violated regulations.

Hazard Tracking System, FAST Toolset, and the Acquisition Management System

Designed to identify, eliminate, or resolve determined or assigned risk, estimate a likelihood of occurrence, and track hazards throughout the life cycle of a program.



FAA Safety Data Sources (cont.)Voluntary Reporting Data

Aviation Safety Reporting System (ASRS) and Aviation Safety Reporting Program (ASRP)

ASRS and ASRP are voluntary programs designed to encourage the identification and reporting of deficiencies and discrepancies in the airspace system.

Advisory Circular 00-46, Aviation Safety Reporting Program

Aviation Safety Action Program (ASAP)

Voluntary reporting of safety issues and events that come to the attention of employees of certain certificate holders. To encourage an employee to voluntarily report safety issues even though it may involve an alleged violation of Title 14 of the Code of Federal Regulations (14 CFR), enforcement-related incentives have been designed into the program.

Order 1110.129, Aviation Safety Action Program Aviation Rulemaking Committee and Advisory Circular 120-66, Aviation Safety Action Program (ASAP)

Automatic Reporting Data Incident and maintenance reporting in Maintenance Management System (MMS)

The MMS contains general maintenance logging, which contributes to the daily system performance and incident reporting.

Order 6000.48, General Maintenance Handbook for Automated Logging



.

So We Went Looking For The Big Picture

• Some data only shows the tip of the problem.

• Multiple databases may provide a lot of information but what does it all mean?

• What is needed to identify better precursors?

– Photo courtesy of Claudia VIRLAN, ANSS Directorate, Romanian CAA



Sources of Leading Indicators

• Data Collected During Normal Operations—not as the result of an Incident/Accident– Revamped Air Traffic Facility Evaluations/Audit

Process– NOSS



Safety Evaluations Serve the Assurance Component of SMS

Supporting Requirements

ATO must continue to meet its performance targets for safety

Reduce operational errors

Reduce runway incursions

To date, ATO collects the number, rate, and severity of these incidents (“lagging” safety indicators)

ATO does not consistently collect data that may identify precursors to these incidents (“leading” safety indicators)

Continual tracking and analysis of lagging and leading indicators will drive toward the identification of mitigation strategies that have a greater impact on the achievement of safety performance targets

ATO must continue to meet its performance targets for safety

Reduce operational errors

Reduce runway incursions

To date, ATO collects the number, rate, and severity of these incidents (“lagging” safety indicators)

ATO does not consistently collect data that may identify precursors to these incidents (“leading” safety indicators)

Continual tracking and analysis of lagging and leading indicators will drive toward the identification of mitigation strategies that have a greater impact on the achievement of safety performance targets

Assurance Component of SMSAssurance Component of SMS

Safety assurance and evaluations1

2 Safety data tracking and analysis

FAASafety Management

System

Policy Architecture

Assurance

Safety Promotion



Safety Evaluations Purpose/Mission

• Conduct audits, program assessments, operational assessments, and facilitate evaluations of ATC facilities to promote safety, quality, and efficiency of the ATC system.

• Measure, through the evaluation and audit processes, ATC system compliance with established policies, procedures, and requirements.

• Identify leading indicators of operational errors and deviations (OE/Ds), near mid-air collisions, controlled flight into terrain, and other safety hazards.

• Monitor and report trends of system compliance with established policies, procedures, and requirements.



Goals of New Evaluations Process

• Knowledge sharing through sharing of mitigation plans across facilities/service areas

• Ability to compare facility performance on key safety indicators

• Data collection/analysis capability that results in:– Identification of safety hazards – Identification of leading indicators – Communication of Evaluations/Audit findings

throughout the ATO



Safety Evaluations Data Sharing

Safety Evaluations

Field Customers-- Service Areas-- Hubs-- Facilities

Headquarters Customers-- OIG / GAO-- Administrator-- COO & Executive Council-- Safety Oversight



Facility Internal Evaluations

• Purpose is to identify compliance issues and safety hazards at a facility by evaluating and rating a set of checklist items.

• Conducted by every Facility in the NAS except Flight Service and NFCTs (approximately 520 facilities)

• Completed at least once per year• Due August 1st each year (beginning

August 1, 2006)



Safety Audits

• An audit is a method of assessing an air traffic facility’s performance and compliance with FAA directives and procedures.

• Audit is Independent Oversight• Audit is NOT a check of the facility Internal Evaluation• All AT Facilities audited at least every 3 years. Federal

Contract Towers (FCT’s) audited every other year.• Conducted by Safety Employees. Support from Service

Areas/Hubs.• Audit sites determined through application of criteria• Audits are little or no notice to the Facility.



Facility Safety Assessment System (FSAS)• Web-based database tool that supports Safety

Evaluations and Audits– Facilitates conduct of facility internal evaluations and

audits by serving as a central source of information on requirements as contained in current directives.

– Serves as a central collection point of the evaluation/audit data as it is completed.

– Serves to facilitate coordination and approval throughout the process.

• Plans to expand FSAS to support other safety processes



Where NOSS Fits

• NOSS provides data about normal situations– Identify situations that resulted in safe and efficient operations– Profile effective system performance

• A new technique for the Evaluations office toolbox • More in-depth look at a facility or portion of a

facility than is provided by current audit process• Provides structured look at operations• Assess precursors to operational errors (leading

indicators)• Use results to help target safety improvements



FAA NOSS Status

• The ATO-P Human Factors Research and Engineering Group focused first on the science of NOSS

• Conducted a study of NOSS observer inter-rater reliability at WJHTC (8/06) to assess the consistency in how operational experts make the same interpretations of data– Partial training of 4 Evaluations specialists on the

TEM model and NOSS observation approach• Plan to conduct a field trial at an En Route

facility in the April/May timeframe



Summary

• FAA ATO-Safety Evaluations focuses on Safety Assurance portion of SMS (leading indicators)

• Most data currently collected on incidents (lagging indicators)

• NOSS could provide a more in-depth tool for collecting leading indicators

Federal Aviation Administration Normal Operations Data: Air Traffic Facility Evaluations and NOSS...

Documents

Transcript of Federal Aviation Administration Normal Operations Data: Air Traffic Facility Evaluations and NOSS...