Normal Operations Safety Survey (NOSS) Chris Henry The University of Texas Human Factors Research...
-
Upload
jenna-emery -
Category
Documents
-
view
214 -
download
0
Transcript of Normal Operations Safety Survey (NOSS) Chris Henry The University of Texas Human Factors Research...
Normal Operations Safety Survey (NOSS)
Chris HenryThe University of Texas Human Factors Research Project
The University of Texas at Austin
First ICAO TEM & NOSS Week
Luxembourg – 10 November 2005
Aviation Safety EnvelopeAviation Safety Envelope
Safety
Incidents
Accidents
Normal Operations
Safety Data CoverageSafety Data CoverageAccidents
Incidents
Normal Operations
Voluntary Incident Reports
NOSS
Accident InvestigationMandatory Incident Reports
Aviation Safety EnvelopeAviation Safety EnvelopeAccidents
Incidents
Normal Operations
NOSS Rationale Proactive snapshot of system / controller performance strengths and weaknesses
in normal operations (just like a health checkup)
NOSS Success FactorsNOSS Success Factors NOSS success is dependent upon methodology and
execution
Low controller trust = Low quality data because there will be no differentiation between NOSS and proficiency checks
AngelPerformance
NaturalPerformance
Formal Check NobodyRegulator
NOSS Observer
- NOSS value +- +
- Controller Trust +- +
NOSS: Gaining Controller TrustNOSS: Gaining Controller Trust
1. Over-the-shoulder observations during normal shifts
2. Joint management / association sponsorship
3. Voluntary Participation
4. Trusted and trained observers
5. Anonymous, confidential, and non-punitive data collection
6. Trusted and secure data collection site
7. Systematic data collection instrument
8. Data verification process
9. Data-derived targets for safety enhancement
10. Feedback results to controllers
NOSS Operating Characteristics NOSS Operating Characteristics
1. Over-the-shoulder observations during normal shifts No observations of controllers who are undergoing training No observations while checks are being conducted
2. Joint management / association sponsorship Letter signed by management & association representatives sent to
all controllers
3. Voluntary controller participation Controllers have a right to decline a NOSS observation Denial rates in NOSS trials were low – 1 per 50 observations
NOSS Operating CharacteristicsNOSS Operating Characteristics
4. Trusted and trained observers Trust and Credibility
Selection process – management / association approval of candidates Most observers should be line controllers – but diversity of interested
parties is good.
Training Classroom training, test observations, recalibration session Curriculum
TEM principles and exercises NOSS observation protocols Narrative guidelines
NOSS Operating CharacteristicsNOSS Operating Characteristics
5. Anonymous, confidential, and non-punitive data collection No names, employee numbers, dates, experience, or other
identifying information Data used for safety purposes only – no punitive actions Observers identity is known only by the third party facilitator
6. Trusted and secure data collection site Third party or controller association gatekeeper Controllers and observers must be comfortable with the data
collection site
7. Systematic data collection instrument No judgments – observers record data based on TEM events
NOSS Operating CharacteristicsNOSS Operating Characteristics
8. Data verification process TEM data checked to ensure coding accuracy and consistency with
SOP Data analysis does not begin until verification has been completed
9. Data-derived targets for safety enhancement Serve as benchmarks for safety change Initial NOSS, safety change process, follow-up NOSS
10. Feedback results to controllers Results summarized for controllers Information on how organization intends to respond to the data
NOSS DefinedNOSS Defined
The 10 characteristics that differentiate NOSS (LOSA) from other methodologies have been endorsed by ICAO IATA IFATCA (6) IFALPA US ALPA UT
NOSS must have all ten characteristics
Methodological ConsiderationsMethodological Considerations
At which level were the NOSS trial observations focused? Groups Groups Individual ControllersIndividual Controllers PositionsPositions
Interviews Threats, errors, and undesired states must be observableThreats, errors, and undesired states must be observable
DemographicsDemographics
General NarrativeGeneral Narrative
BriefingsBriefings
Threat Management WorksheetThreat Management Worksheet
Error Management WorksheetError Management Worksheet
TEM CountermeasuresTEM Countermeasures
Threat Codebook - IThreat Codebook - I
Threat Codebook - IIThreat Codebook - II
Error CodebookError Codebook
US CodebookUS Codebook
NOSS Report and Raw Data Most frequent and mismanaged threats
Most frequent and mismanaged errors
Strengths and weaknesses of Threat and Error Countermeasures
Facility to facility or sector to sector differences
Comparison to similar airspace of other air traffic service providers
NOSS Report
NOSS Raw Data
Text from all narratives
Listing and coding of every threat, error, and undesired state observed
Sample Final Report ChartsSample Final Report ChartsErro r Typ es(To ta l = 2 2 5 N OSS obs erv ations )
Error Prevalence (Perc entage of NOSS obs erv ations w ith an error)
1 0 % 2 0 % 3 0 % ` 4 0 % 5 0 % 6 0 % 7 0 % 8 0 % 9 0 % 1 0 0 %
All Errors
Inform ation Dis playT he data pres ented in this table is fic titious -
intended for us e as an ex am ple only
BriefingsT he data pres ented in this table is fic titious -
intended for us e as an ex am ple only
Phras eologyT he data pres ented in this table is fic titious -
intended for us e as an ex am ple only
81%
34%
8%
10%
Error Types(T otal = 225 NOSS Ob se rva tio n s)
Error M ismanagement (Perc entage of NOSS o b se rva tio n s w ith a misma n a g e d e rro r)
1 0 % 2 0 % 3 0 % 4 0 % 5 0 % 6 0 % 7 0 % 8 0 % 9 0 % 1 0 0 %
All ErrorsT he data pres ented in this table is fic titious -
intended for us e as an ex am ple only
Inform ation Dis playT he data pres ented in this table is fic titious -
intended for us e as an ex am ple only
BriefingT he data pres ented in this table is fic titious -
intended for us e as an ex am ple only
Phras eologyT he data pres ented in this table is fic titious -
intended for us e as an ex am ple only5%
27%
9%
42%
Raw Data: Narratives - HandoversRaw Data: Narratives - Handovers
NOSS Observation #17 Handover/Takeover #1 Time Stamp 14:36 XYZ Radar Sector
Positions Staffed: Radar Controller Position Relieved: Radar Controller
The oncoming controller was at the start of their shift, when they arrived they walked straight up to the console and
started a conversation with the XYZ Controller about non operational matters, this distracted the controller whilst
aircraft were calling resulting in a number of missed calls from aircraft being made to the XYZ Controller. The
briefing commenced with the oncoming controller not reading any of the requisite briefing material. The controller
giving the HO/TO did it from memory, the checklist provided was not referred to and not all the pertinent information
was covered. The controller giving the HO/TO did not mention the RWY configuration in XXX or the fact that they
were on ILS approaches. At the end of the HO/TO, the relieving controller plugged in and did not voice log on.
NOSS Raw Data - NarrativesNOSS Raw Data - Narratives
ABC47 crossed into the XYZ sector but did not contact the controller. At time 19:42, BB TMA called the XYZ sector controller requesting a non-standard level of FL350 for a flight, which was approved and properly noted. During this conversation, ABC47 called but was not heard due to the ongoing conversation with BB TMA. At this point the A/C was already 40 nm into the sector (this was a very late call). A minute later, ABC47 called again and this time the controller heard the call. At this point it was discovered that ABC47 was east of track by 3nm without prior authorization leading to a potential conflict with EDF54 who was in the vicinity climbing to FL160. ABC47 given a right turn to manage the situation and both A/C were given traffic information. Additionally, CVR310 was descending through the level of ABC47 while it was not on the appropriate frequency (inside the XYZ sector).
NOSS Observation #: 27 Threat # 3 XYZ Radar Sector Time Stamp: 17:42
Threat DescriptionAAA4103,04,4003, and BBB405 were operating in the XYZ airspace at the same time with similar callsigns.
Threat Management DescriptionThe XYZ controller made a general broadcast to the aircraft advising of the potential for confusion and was particularly careful to ensure the correct read-back and callsign with instructions. All calls where made to the appropriate a/c and the correct a/c responded to the calls each time.
Threat Type Airborne
Threat Code Similar Call Signs
Threat Outcome: Inconsequential
Raw Data: ThreatsRaw Data: Threats
NOSS SummaryNOSS Summary Primary features
Purely diagnostic – no solutions – “operations health check”
Captures system safety and controller performance issues prior to the occurrence
of incidents
Identifies system safety and controller performance strengths and weaknesses in
normal operations using observations
Analysis
Aggregated TEM data provides a safety snapshot of operations on a typical day
Organizational change
Measure the effectiveness safety solutions with a follow-up NOSS
NOSS – Future UT ActivitiesNOSS – Future UT Activities
NOSS & LOSA: What can each add to the other? NAV CANADA radar trial Airservices Australia tower trial Multi-stage reliability studies in conjunction with the FAA &
NOSS trial partners European trials?