Analysis of SNET performance based on monitoring data · Analysis of SNET performance based on...

Analysis of SNET performance based on monitoring data 28-04-2009 PASS/WA1/WP5/64/D Version 1.3

EUROCONTROL HQ ATC Domain – DSNA, Deep Blue, Egis Avia & QinetiQ – PASS Project /59

Work Area 1 / Work Package 1 Report: Analysis of SNET performance based on monitoring data

Performance and safety Aspects of Short-term Conflict Alert – full Study

PASS Project



This document has been produced under contract for EUROCONTROL.

EUROCONTROL ALDA reference: 09/04/27/11

Disclaimer © 2009 The European Organisation for the Safety of Air Navigation (EUROCONTROL). This document is published by EUROCONTROL for information purposes. It may be copied in whole or in part, provided that EUROCONTROL is mentioned as the source and to the extent justified by the non-commercial use (not for sale). The information in this document may not be modified without prior written permission from EUROCONTROL. EUROCONTROL makes no warranty, either implied or express, for the information contained in this document. Neither does it assume any legal liability or responsibility for the accuracy, completeness or usefulness of this information. EUROCONTROL 96 Rue de la Fusée B-1130 Bruxelles Belgium

Email: [email protected]

mailto:[email protected]



RECORD OF CHANGES

Issue Date Detail of changes

0.1 04-11-2008 Proposed outline of the report

0.2 27-01-2009 First draft, without the final section and appendices

1.0 06-03-2009 Initial Eurocontrol comments addressed, final section drafted, appendices added

1.1 24-03-2009 Eurocontrol comments addressed, final section completed.

1.2 03-04-2009 Title changed, disclaimer added, additional references included, list of occurrences analysed in one study added in an appendix, miscellaneous editorials

1.3 28-04-2009 EUROCONTROL proofreading

IMPORTANT NOTE: EACH NEW VERSION SUPERSEDES THE PRECEDING VERSION, WHICH MUST BE DESTROYED OR CLEARLY MARKED OBSOLETE VERSION ON THE FRONT PAGE.

Report drafted by: Christian Aveneau

Authorised by: Thierry Arino on 03-04-2009



LIST OF DEFINITIONS

ACAS Airborne Collision Avoidance System – a system standardised in the ICAO SARPs which uses transponder replies from other aircraft to warn the pilot of a risk of impending collision

Hereafter, ACAS always refers to ACAS II – a system generating traffic advisories (TAs) and resolution advisories (RAs) in the vertical dimension, and whose carriage and operation is mandatory for many aircraft in Europe.

TCAS Traffic alert and Collision Avoidance System – aircraft equipment implementation of an ACAS

Hereafter, TCAS refers to TCAS II – the only equipment thus far compliant with the ACAS II standards.

RA

Resolution Advisory – an ACAS alert that indicates to a pilot how to adjust the vertical rate of the aircraft so as to avoid a mid-air collision

Closest approach Minimum physical distance between two aircraft (slant range) involved in an encounter.

This distance is used by ACAS for the determination of its alerts.

STCA Short-Term Conflict Alert – a ground-based safety net intended to assist the controller in preventing collision between aircraft by generating, in a timely manner, an alert of a potential or actual infringement of separation minima

Minimum separation

Horizontal and vertical distances between two aircraft involved in an encounter at the minimum ‘propinquity’.

The propinquity value measures the horizontal and vertical distances between the aircraft in accordance with the respective separation minima applicable by ATC.

This value is commonly used in ANSP monitoring activities because it allows comparison of situations, possibly involving very different horizontal and vertical distances, using a single figure, and because it readily indicates a loss of separation where this is lower than 1.

Encounter A traffic situation involving two (or more) aircraft in which STCA and/or ACAS issued an alert



Level-off encounter

A traffic situation involving two aircraft which are ultimately vertically separated by 1,000 feet (or 2,000 feet) following the level-off of at least one of the aircraft above or below the flight level occupied by the other aircraft

More precisely, a distinction can be made between: - a ‘single level-off encounter’ in which only one aircraft levels off above or below the level of the other aircraft; and - a ‘double level-off encounter’ in which the two aircraft level off at adjacent flight levels

Safety-net related occurrence

An ATM occurrence involving two (or more) aircraft in which the ground-based safety-net, i.e. STCA, or the airborne safety-net, i.e. ACAS, issued an alert

STCA occurrence An occurrence in which the STCA system triggered an alert

RA occurrence An occurrence in which the TCAS triggered an RA in at least one of the aircraft involved

More precisely, a distinction can be made between: - a ‘single TCAS RA occurrence’ in which only one of the aircraft involved experienced a TCAS RA on board; and - a ‘coordinated TCAS RA occurrence’ in which TCAS RAs are triggered on board both aircraft, i.e. coordinated RAs

STCA-only occurrence

An occurrence in which an alert was triggered by the STCA system but not by TCAS in any of the aircraft involved

RA-only occurrence

An occurrence in which an alert was triggered by TCAS in at least one of the aircraft involved but not by the STCA system

Elementary events

STCA and/or ACAS-related events occurring during a safety-net-related occurrence (e.g. the alerts themselves, pilot and controller radio communications prompted by these alerts, aircraft manoeuvres in response thereto, etc.)

Avoiding instruction

A controller instruction designed to prevent loss of separation or to mitigate the effects of a loss of separation which has already occurred. It may or may not be effective.

For PASS purposes, an instruction was defined as ‘avoiding’ if:

• it occurs after the STCA was triggered; or

• it uses the avoiding instruction phraseology; or

• it occurs after the separation has been lost.



LIST OF ACRONYMS

ACAS Airborne Collision Avoidance System ACC Area Control Centre AI Avoiding Instruction ANSP Air Navigation Service Provider ASR Air Safety Report ATC Air Traffic Control ATCO Air Traffic Controller ATM Air Traffic Management CFL Clear Flight Level DFS Deutsche Flugsicherung DSNA Direction des Services de la Navigation Aérienne EHQ EUROCONTROL Headquarters ESARR EUROCONTROL Safety Regulatory Requirement FARADS Feasibility of RA Downlink Study FL Flight Level ICAO International Civil Aviation Organization IFR Instrument Flight Rules

LoS Loss of Separation

PASS Performance and safety Aspects of Short-term Conflict Alert – full Study

RA Resolution Advisory RT RadioTelephony STCA Short Term Conflict Alert SNET Safety Nets SPIN Safety nets Performance Improvement Network TI Traffic Information TCAS Traffic Alert and Collision Avoidance System TMA Terminal Manoeuvring Area VFR Visual Flight Rules VSL Vertical Speed Limit WA Work Area



TABLE OF CONTENTS

1. INTRODUCTION ....................................................................................................10

1.1. CONTEXT AND BACKGROUND...........................................................................10

1.2. SCOPE AND OBJECTIVES OF THE DOCUMENT...............................................10

1.3. REPORT STRUCTURE..........................................................................................11

1.4. NOTE TO THE READER........................................................................................11

2. OVERVIEW OF THE PASS MONITORING ACTIVITY..........................................12

2.1. RATIONALE FOR A MONITORING ACTIVITY .....................................................12

2.2. USE OF MONITORING RESULTS IN OTHER PASS ACTIVITIES.......................13

2.3. OCCURRENCES OF INTEREST ...........................................................................14

2.4. ANSP INVOLVEMENT ...........................................................................................15

2.5. DATA COLLECTION AND PROCESSING............................................................17

2.6. CONSTRAINTS AND THEIR IMPLICATIONS FOR THE PROJECT....................19

2.7. DESCRIPTIVE ANALYSIS OF SNET OCCURRENCES .......................................20

2.8. SPECIFIC ANALYSIS OF INFLUENCING FACTORS ..........................................21

2.9. SPECIFIC ANALYSIS OF RA DOWNLINK ...........................................................21

3. RESULTS OF PASS MONITORING ACTIVITY.....................................................23

3.1. GENERAL DESCRIPTION OF SNET OCCURRENCES INVESTIGATED............23

3.2. ADEQUACY OF ALERTS......................................................................................24

3.3. CHARACTERISTICS OF CONTROLLER REACTIONS........................................32

3.4. CHARACTERISTICS OF PILOT REACTIONS......................................................36

3.5. CHARACTERISTICS OF OCCURRENCES IDENTIFIED THROUGH RA DOWNLINK............................................................................................................................43

4. CONCLUSIONS .....................................................................................................49

4.1. SYNTHESIS OF RESULTS....................................................................................49

4.2. SETTING UP THE MODEL-BASED PERFORMANCE EVALUATION.................50

5. REFERENCES .......................................................................................................53



LIST OF FIGURES

Figure 1: Scope of the monitoring activity ..............................................................................13 Figure 2: Contribution of the monitoring activity to work area 2 .............................................13 Figure 3: Contribution of the monitoring activity to work area 3 .............................................14 Figure 4: Contribution of the monitoring activity to work area 4 .............................................14 Figure 5: Sources of data .......................................................................................................17 Figure 6: Operations performed on the data collected ...........................................................17 Figure 7: Distribution of SNET occurrences vs. type of alert..................................................23 Figure 8: Distribution of aircraft involved in STCA occurrences vs. time between STCA start and actual loss of separation..................................................................................................24 Figure 9: Horizontal and vertical views of an occurrence with average observed STCA warning time ...........................................................................................................................25 Figure 10: Horizontal and vertical views of an occurrence with late STCA triggering ............26 Figure 11: Horizontal and vertical views of an occurrence with CFL use...............................27 Figure 12: Distribution of time between initial RA and actual time of closest approach .........28 Figure 13: Horizontal and vertical views of an occurrence with average RA observed warning time.........................................................................................................................................29 Figure 14: Horizontal and vertical views of an occurrence with late RA observed warning time ...............................................................................................................................................29 Figure 15: Distribution of aircraft involved in STCA + RA occurrences vs. time between STCA start and time of initial RA ......................................................................................................30 Figure 16: Horizontal and vertical views of an occurrence with STCA triggered sufficiently early before RA ......................................................................................................................31 Figure 17: Horizontal and vertical views of an occurrence with STCA triggered close to the RA ..........................................................................................................................................32 Figure 18: Distribution of SNET occurrences vs. controller intervention method ...................32 Figure 19: Distribution of SNET occurrences vs. number of aircraft issued with an avoiding instruction ...............................................................................................................................33 Figure 20: Distribution of SNET occurrences vs. time between first avoiding instructions given to each aircraft ..............................................................................................................34 Figure 21: Distribution of aircraft vs. time between first avoiding instruction and first STCA start ........................................................................................................................................34 Figure 22: Distribution of aircraft vs. horizontal geometry and direction of the avoiding instruction ...............................................................................................................................35 Figure 23: Distribution of aircraft vs. vertical geometry and direction of the avoiding instruction ...............................................................................................................................36 Figure 24: Distribution of (84) STCA-only occurrences with AI vs. response to AI ................37 Figure 25: Distribution of (16) RA-only occurrences vs. response to RA ...............................37 Figure 26: Distribution of (56) STCA + RA occurrences vs. response to AI and RA..............37



Figure 27: Distribution of aircraft vs. time between first controller avoiding instruction and start of pilot’s response ..........................................................................................................38 Figure 28: Distribution of aircraft vs. type of vertical avoiding instruction and pilot’s degree of compliance .............................................................................................................................39 Figure 29: Distribution of aircraft vs. type of horizontal avoiding instruction and pilot’s degree of compliance .........................................................................................................................40 Figure 30: Distribution of aircraft involved in RA occurrences vs. time between initial TCAS RA and start of aircraft manoeuvre ........................................................................................41 Figure 31: Distribution of aircraft vs. type of RA and pilot’s degree of compliance ................42 Figure 32: Radar coverage of the RA downlink recording......................................................43 Figure 33: Intruder equipment ................................................................................................44 Figure 34: Intentionality of occurrences .................................................................................45 Figure 35: Altitude distribution of occurrences .......................................................................46 Figure 36: Hourly distribution of unintentional occurrences ...................................................46 Figure 37: Duration of the RA sequences ..............................................................................47 Figure 38: Most frequent sequences of RAs ..........................................................................47 Figure 39: Framework for the development of STCA performance requirements..................50

LIST OF TABLES

Table 1: Descriptive model of safety-related occurrences of interest.....................................15 Table 2: List of descriptive elements for a safety-net-related occurrence ..............................19 Table 3: Order of priority for the extraction of descriptive elements.......................................19



1. Introduction

1.1. Context and background

1.1.1. In the context of STCA standardisation in Europe, EUROCONTROL has launched the PASS project (Performance and safety Aspects of Short-term Conflict Alert – full Study).

1.1.2. The study was undertaken based on the recommendations of the RA downlink study (FARADS project) and the ACAS-STCA workshop held on 27 and 28 March 2007 in Zurich. It falls within the scope of the SPIN (Safety nets Performance Improvement Network) and is intended to help establish quantified performance requirements for STCA, and to define a consistent overall concept for ground-based and airborne safety nets.

1.1.3. The project is divided into three main phases, as follows:

• Phase 1: Monitoring & understanding of current situation;

• Phase 2: European STCA environment modelling & safety and performance analysis; and

• Phase 3: Enhanced modelling and analysis, synthesis and guidelines.

1.1.4. Within the PASS project, Work Area (WA) 1 is part of Phase 1 and is intended to provide a better understanding of the typical sequence of elementary events in encounters in which STCA and/or ACAS played a role and of the factors that have a major influence on the features of this sequence.

1.1.5. The means for achieving this goal was a monitoring activity to capture a significant number of ATC occurrences where an ACAS alert and/or an STCA alert were triggered. This monitoring covered as wide an airspace as possible in order to reflect all types of ATC operations.

1.1.6. This was a key activity because it determines the realism of the tools used to derive performance-related elements of standardisation for STCA during Phases 2 and 3. Indeed, the quantitative and qualitative results derived from the monitoring will be used in developing various models for the Model-based Performance Assessment of STCA (WA2), will help to build safety-net-related scenarios for the Operational Safety Assessment (WA4), and will contribute to the set-up of the (optional) real-time experiment (WA3).

1.2. Scope and objectives of the document

1.2.1. This document summarises the PASS monitoring activity. It reports on the sources and tools used for the data collection, explains how the various analyses were performed, provides the results of the analyses and outlines how those results will be used in the next PASS activity: the modelling phase.

1.2.1.1. It should be stressed that the purpose of the monitoring activity is not to compare the operational practices and STCA performance of the contributing ANSPs. The present report addresses only general patterns in the airspace monitored as a whole, as contributed by all ANSPs, with all sensitive data de-identified.



1.3. Report structure

1.3.1. Section 1 introduces the reader to the report. It gives an overview of the PASS project, its context, and the need for a monitoring activity.

1.3.2. Section 2 is an overview of the PASS monitoring activity. It explains that a monitoring activity had to be carried out in order to improve understanding of the performance of actual safety nets, and that the results will be used to design simplified models of controller and pilot reactions to safety nets, as well as to produce realistic scenarios for simulations and trials. It presents the methodology of the activity: type and amount of data sources used, the ANSPs that provided them, types of occurrences investigated and information extracted from the occurrences. The constraints on the project are then set out. Lastly, the section describes the two studies based on several data sources and the specific study based only on RA downlink data.

1.3.3. Section 3 provides the main results of the monitoring activity. The results of the studies based on several data sources are described under the following themes: general description, adequacy of the alerts, characteristics of the controller’s reactions and characteristics of the pilot’s reactions. The results consist in:

• graphs showing the different times between two events related to safety nets (e.g. between an alert and a reaction to the alert) in terms of the amount of occurrences (or aircraft);

• graphs showing the proportion of occurrences (or aircraft) having some characteristic related to safety nets (e.g. kind of reaction to an alert);

• descriptions of factors influencing the times and characteristics of the events related to safety nets.

At the end of the section, the occurrences captured by RA downlink are described in statistical terms.

1.3.4. Section 4 is the conclusion to the report. It summarises the results and sets out a number of design decisions necessitated by the results for the forthcoming modelling activity.

1.3.5. Appendix A provides the list of metrics used for the one of the studies based on several data sources (i.e. the descriptive analysis). Some of those metrics are presented in graph form in section 3.

1.3.6. Appendix B gives a list of the occurrences analysed in depth in the other study based on several data sources (i.e. the study on influencing factors) as well as a synthetic view of the analysis of an occurrence for illustration purposes. The influencing factors revealed by those analyses are presented in section 3.

1.4. Note to the reader

1.4.1. The safety-net-related occurrences analysed during the monitoring activity were biased in favour of more serious incidents in order to ensure that as wide a range of occurrence types as possible was captured. If readers wish to extrapolate the statistics presented here to normal STCA performance, they should exercise both caution and expert judgment.



2. Overview of the PASS monitoring activity

2.1. Rationale for a monitoring activity

2.1.1. The aim of the PASS project is to provide guidelines for an overall concept of STCA/ACAS operations and to draft performance-oriented standards for STCA. Such guidelines and standards cannot be based purely on expert judgment. They must be based on a thorough understanding of the current operational environment.

2.1.2. STCA and ACAS are technically independent safety nets, using different surveillance data and different alerting logic. They interact on an operational level because there is an unavoidable overlap of the warning times of both systems, which means, for example, that STCAs may occur just before ACAS alerts. This can lead to conflicting manoeuvre demands from the controller and ACAS.

2.1.3. The pilot may thus not manoeuvre at all or may perform a wrong, insufficient or excessive manoeuvre, which has an effect on the final severity of the occurrence. Interactions between STCA and ACAS must be better understood to determine how to better integrate these safety nets into the ATM system. The timeline of elementary events is particularly decisive in this endeavour (e.g. which alert occurs first, do pilots report RAs before a controller issues an avoiding instruction?).

2.1.4. Because STCAs are currently in operation, the assessment of the minimum performance that an STCA should comply with will depend on current STCA performance (effective warning times, undesirable alerts, etc.) and on how they are affected by:

• airspace factors (e.g. airspace and traffic characteristics such as traffic patterns, mix of IFR/VFR traffic, proximity of military traffic, holding patterns);

• procedural factors (e.g. ATC procedures such as letters of agreement, management of control positions, contingency procedures)

• technical factors (e.g. STCA features and parameters, surveillance data quality); and

• human factors (e.g. controller working practices, level of training and knowledge of STCA).

2.1.5. Capturing current interactions between safety nets, as well as STCA performance and the determining factors influencing that performance, requires the analysis of current ATC incidents where either STCA or ACAS was involved (safety-net-related occurrences). Such an analysis should provide a better understanding of the typical sequence of elementary events in encounters in which STCA and/or ACAS played a role and of the factors having a major influence on the features of this sequence. This is the purpose of the PASS monitoring and investigation of real ATC incidents (WA1), as illustrated in Figure 1.



WA1:Monitoring activity

Operational datafrom real ATC incidents

Statistics on thesequence of events(timeline, actions,…)

Influence ofenvironmental, technicaland human factors


Operational datafrom real ATC incidents

Statistics on thesequence of events(timeline, actions,…)

Influence ofenvironmental, technicaland human factors

Figure 1: Scope of the monitoring activity

2.2. Use of monitoring results in other PASS activities

2.2.1. The guidelines and performance standards will not be directly derived from the analysis of safety-net-related occurrences. Instead, they will be developed from performance, operational and safety requirements derived from three other PASS activities.

2.2.2. A performance evaluation and requirement determination (WA2):

2.2.2.1.

2.2.2.2.

This activity will first develop operationally realistic models of safety-net-related occurrences, of safety net behaviour and of human reactions to safety-net alerts. It will then use these models to conduct fast-time simulations of the behaviour of the whole system (safety nets plus human actions) in a large number of reconstructed safety-net-related occurrences. Performance metrics and their sensitivity to variations of certain model parameters will thus be measured in several operational scenarios.

The PASS monitoring activity will contribute to the performance evaluation and requirement determination by providing operational data to build a controller response model to STCA alerts and a pilot response model to ACAS alerts. In addition, the better understanding of the current situation achieved through the monitoring activity will support the definition of operationally realistic scenarios for simulations.

WA2: Performance evaluation andrequirement determination


Modelling of pilotreaction to ACAS alerts

Modelling of controllerreaction to STCA alerts

Definition of operationalscenarios for the simulations

Performance requirements


WA2: Performance evaluation andrequirement determination


Modelling of pilotreaction to ACAS alerts

Modelling of controllerreaction to STCA alerts

Definition of operationalscenarios for the simulations



Figure 2: Contribution of the monitoring activity to work area 2



2.2.3. An optional real-time experiment (WA3):

2.2.3.1.

2.2.3.2.

This activity will further investigate controller / pilot behaviour and performance in relevant STCA & ACAS scenarios derived from actual occurrences identified during the PASS monitoring activity.

The scenarios for the real-time experiment may include hazard-causing conditions such as system failures, pilot and controller errors, STCA / ACAS interactions and other operational problems, reproduced with the collaboration of the simulation staff. The results will ultimately help to define requirements related to automation, procedures and training, for the overall operational concept of joint STCA / ACAS operations.

WA3: real - time experimentation WA1:

Monitoring activity

Definition of operational scenarios for the simulations

Operational requirements Operational

requirements

WA3: real - time experimentation WA1:

Monitoring activity

Definition of operational scenarios for the experiment

Operational requirements Operational

requirements


2.2.4. An operational safety assessment (WA4):

2.2.4.1. This activity will identify hazards associated with the safety net operations, will assess their operational consequences and will identify appropriate mitigation means and safety requirements. Ultimately, it will provide an estimate of the level of safety to be expected from joint STCA and ACAS operations. The PASS monitoring activity will contribute to the operational safety assessment by allowing the identification of hazards in certain specific scenarios of safety-related occurrences.

WA4: Operational safety assessment WA1:

Monitoring activity

Identification of operational hazards

Safety requirements

Safety requirements

WA4: Operational safety assessment WA1: Monitoring activity

Identification of operational hazards

Safety requirements

Safety and operational

requirements


2.3. Occurrences of interest

2.3.1. The monitoring activity looked for occurrences where at least one safety net was triggered. Occurrences where no safety net was triggered, although this would have been desirable, were not identified.

2.3.2. As it was not possible to take into account all possible occurrences involving safety nets, the monitoring activity selected 180 meaningful occurrences, covering the whole range of operational scenarios. The following table provides a descriptive model of safety-net-related occurrences of interest.



STCA alerts only Combined ACAS/STCA alerts ACAS alerts only

No loss of separation as a result of ATCO intervention, before or after the alert

Late STCA in specific conflict geometries

Non-operative STCA

Loss of separation without ACAS RA

Lack of or late ATC intervention in response to STCA

ACAS threat non-eligible for STCA (e.g. military, VFR)

Conflicting avoiding instruction initiated by ATCO

Non-operative ACAS

Lack of or late pilot response to ATC intervention

Specific conflict geometries (e.g. 1,000 feet level-off, slow converging tracks)

Table 1: Descriptive model of safety-related occurrences of interest

2.3.3. The safety-net-related occurrences with combined ACAS/STCA alerts were of particular interest because they potentially featured the most crucial interactions between ACAS and STCA.

2.3.4. The interest of a given safety-net-related occurrence was assessed by an operational expert within the PASS team. All the safety-net-related occurrences of interest were analysed quantitatively to obtain descriptive knowledge of the occurrences and of the way in which they occur in general.

2.3.5. However, this quantitative description may fail to highlight a number of determining elements. There was a need to conduct more detailed analyses, but due to resource constraints this could not be done for all occurrences. Therefore a subset of occurrences had to be selected, with the following criteria: it should represent a wide range of operational situations, sufficient information should be available for each occurrence, and each occurrence should be relevant in terms of human factors or other influencing factors. Ultimately, 12 safety-net-related occurrences were selected for further (qualitative) analysis.

2.4. ANSP involvement

2.4.1. The need for ANSP involvement

2.4.1.1.

2.4.1.2.

The overall concept of joint ACAS / STCA operations, as well as the performance-related standards, are expected to become applicable to all European airspace. Whereas ACAS has already been standardised and only one implementation of it exists (i.e. TCAS), STCA algorithms have various levels of sophistication and various underlying operating principles throughout Europe. Furthermore, even within one State, an STCA implementation generally distinguishes between several airspace volumes where aircraft operations require different alerting parameters and filters.

The PASS project needed to gather safety-net-related occurrences from several States, where the STCA implementations differ. Within each State, where



possible, the monitoring activity covered all possible operations either in approach or en-route airspace: parallel approaches, holding patterns, crossing between departures and arrivals, crossing or merging at a waypoint, etc. Therefore, the participation of several ANSPs was essential to ensure that the project findings were representative of the European situation.

2.4.2. ANSP contribution to the monitoring activity

2.4.2.1.

2.4.2.2.

2.4.2.3.

2.4.2.4.

2.4.2.5.

2.4.2.6.

2.4.2.7.

As DSNA is a subcontractor in the PASS project, the members of the PASS team had access to its data-recording infrastructure and to French ATM operational experts. The involvement of DSNA in the monitoring activity extended over seven months, from September 2007 to March 2008, and covered the Aix, Paris and Reims en-route control centres as well as Paris approach.

Other ANSPs which offered to participate were asked to provide data about safety-net-related occurrences over a recent time period. These data could have come from ATC units where the occurrence occurred or from a centralised cell set up for occurrence gathering / analysis.

As each ANSP did not have the same amount of resources to allocate to a monitoring effort in coordination with the PASS team, they had various levels of involvement:

• skyguide’s contribution to PASS monitoring extended over three months, from May 2008 to July 2008, and covered the Geneva and Zurich control areas of responsibility.

• the DFS contribution to PASS monitoring extended over one month, July 2008, and covered the Dusseldorf, Frankfurt and Munich control areas.

Each contributing ANSP was provided with feedback on the monitoring data it provided. This took the form of a restricted report, provided only to the contributing ANSP and focusing on its airspace.

Additional data was extracted from significant occurrences reported in public sources from the following countries: the United Kingdom, Denmark, the Czech Republic, Ireland and Estonia. These occurrences are described in [WEB].

Although limited in time and scope, the monitoring activity highlighted the benefits (from an ANSP point of view) that could be expected from the monitoring and analysis of recorded safety-net alerts.

The monitoring activity also pointed out the need for appropriate filtering and classification of recorded alerts before drawing definitive conclusions on their operational relevance.



2.5. Data collection and processing

2.5.1.1. Data of interest included: radar data, RA downlink data and STCA data. These data were used by the PASS team for the selection of the 180 safety-net-related occurrences of interest and the analysis of their elementary events.

Radar Surveillance & RA downlink

Radar Surveillance & RA downlink

Approach & ACC units

Approach & ACC units

ATC recordings (radar situation,

STCA alerts, RA downlink,

etc)

ATC recordings (radar situation,

STCA alerts, RA downlink,

etc)

Airborne recordingsAirborne recordings

ATCO incident reportsATCO incident reportsATCO incident reports

Air Safety Reports (by pilots)



Mode S Ground StationMode S Ground Station

Figure 5: Sources of data

2.5.1.2. For the 12 safety-net-related occurrences with significant interest in terms of influencing factors (see 2.3.5), additional relevant information about the occurrence was requested: any existing incident report (ASR, controller report, occurrence analysis report) and if possible the recording or transcription of the communications between ATC and the aircraft involved.

2.5.2. Figure 6 provides a summary of the processing of the data collected.

Synchronisation

ACASsimulations

Extraction ofdescriptive elements

Data set ofdescriptiveelements

Operationalrecordingsand reports

Synchronisation

ACASsimulations

Extraction ofdescriptive elements

Data set ofdescriptiveelements

Operationalrecordingsand reports

Figure 6: Operations performed on the data collected

2.5.3. The diversity of the data sources which supported the monitoring activity should be stressed. The data may have been collected at different times and there may have been different ways of labelling the same safety-net-related occurrence; therefore it was necessary to correlate the data. The data had different time references and refresh rates; therefore they needed synchronisation.



2.5.4. Sometimes, the data did not contain everything required for the reconstruction of the sequence of elementary events and it was necessary to approximate it through simulations. Finally, the data had different accuracies and reliability, therefore the best source was chosen if several were available providing the same element.

2.5.5. The final step of the data processing consisted in extracting a number of elements describing each safety-net-related occurrence: information relating to the occurrence in general; to the aircraft involved; to their flights; and to the sequence of elementary events. These elements were the essential input to the descriptive analysis (see section 2.6) and made a major contribution to the qualitative analysis (see section 2.8). The extraction was performed manually by an operational expert.

2.5.6. The elements extracted are summarised in Table 2.

Date

Position

Geometry

Time and separation at closest approach

Time and separation at minimum separation

Time and separation at loss of separation

General information about the safety-net-related occurrence

Severity

Mode S address

ACAS equipment status

Aircraft registration

Information about each aircraft involved

Aircraft type

Flight type

Flight ID

Flight rules

Vertical trend

Mode A code

ATC sector

ATC unit

Cleared flight level

Closing speeds at RA and STCA start

Information about the flight of each aircraft involved

Distances at RA and STCA start



STCA alerts

ACAS alerts

Pilot requests

Pilot contact reports (visual and TCAS)

Pilot ACAS reports

Controller instructions

Controller traffic information

Controller avoiding instructions

Pilot response to ATC instructions

Timing and nature of elementary events

Pilot response to RAs

Table 2: List of descriptive elements for a safety-net-related occurrence

2.5.7. Due to the diversity of the sources and their varying level of completeness, sometimes only the timing or the nature of an elementary event could be extracted. On rare occasions, an elementary event was estimated as likely by the operational experts, while not recorded in the data collected. It was then retained for extraction but marked as uncertain.

2.5.8. Where more than one data source contained the same information, an order of priority was given to decide what would be the origin of the recorded value. This priority depended on the accuracy and reliability of the data source. In particular, the order of preference for extracting information about ACAS and STCAs from available sources is indicated in Table 3.

ACAS alerts STCA alerts

Priority 1 Airborne-recorded data STCA recordings

Priority 2 RA downlink data Incident reports

Priority 3 ACAS simulation results

Table 3: Order of priority for the extraction of descriptive elements

2.6. Constraints and their implications for the project

2.6.1. The following constraints have been identified for the monitoring activity, regarding the studies based on several sources of data (i.e. not the study based on RA downlink):

− Occurrences have been selected mainly through human reporting because other sources were rarer or less complete regarding the context of the occurrence. Reported occurrences are in general the more severe ones, because they often involve a loss of separation. Therefore the set of selected occurrences was not representative of all SNET occurrences.



− Sufficient information for the computation of the indicators was not always available, because of less detailed reports or of the absence of a data source. Therefore, indicators were often computed from a reduced set of occurrences. This set was further reduced when divided into categories of occurrences, in order to study specific behaviours. Thus, the deeper the analysis, the less statistically significant the specific values. For some indicators, only the general trend is meaningful.

− Regarding the timing of elementary events, each source had a different accuracy. Airborne-recorded data had the best accuracy (step of 1 second). RA downlink data had an accuracy which depended on the radar source rotation period but which could be improved by simulating the RA logic on the radar trajectory. Timing of events found in incident reports had the worst accuracy, because sometimes a single time stamp was followed by several actions or messages, which means that the operational expert had to approximate the time of events not occurring immediately after the time stamp.

2.6.2. These constraints were unavoidable due to the nature of the data collected. Their implications will be taken into account when using the results of the monitoring activity in Phase 2 of the project.

2.7. Descriptive analysis of SNET occurrences

2.7.1. The descriptive analysis (fully reported on in [STATS]) used the descriptive elements of safety-net-related occurrences listed in section 2.5 to provide a general picture of where (e.g. approach, en-route, flight phases, in between ATC units or sectors, etc.) and when safety-net-related occurrences occur and how they develop (e.g. sequence of elementary events).

2.7.2. A set of attributes and performance metrics used to characterise these safety-net related occurrences was computed from the set of 180 safety-net-related occurrences. The full list of metrics can be found in Appendix A. They are grouped under the following categories:

• general statistics;

• adequacy of alerts;

• characteristics of controller reactions;

• characteristics of pilot reactions; and

• consequences of alerts.

2.7.3. Where practicable, the metrics for safety-net-related occurrences were computed separately for STCA-only occurrences, ACAS-only occurrences and combined STCA/ACAS occurrences and on average for all occurrences. Not all the metrics were computable for a given safety-net-related occurrence, when referring to an elementary event which had not occurred (e.g. no actual separation infringement, no RA or STCA, etc.).



2.7.4. The resulting trends and numerical values are necessary for the development of models in WA2. For example, when designing the model of controller response to STCAs, the metrics will be instrumental in deciding the probability of action by the controller, the probability of choosing an avoiding instruction in the horizontal sense, the average communication delay, and so on.

2.8. Specific analysis of influencing factors

2.8.1. The specific analysis of influencing factors (fully reported on in [INF]) was based on the detailed analysis of specific safety-net-related occurrences. This analysis was designed to highlight the factors which most likely influenced the sequence of events in each occurrence, as well as the operational consequences of this sequence of events.

2.8.2. The sample of occurrences analysed in detail is intended to illustrate the wide range of situations observed during the WA1 monitoring activities. For that purpose, 12 specific safety-net-related events were selected from among the 180 occurrences of the descriptive analysis.

2.8.3. It is important to note that the purpose of this analysis was not to perform an incident investigation of each occurrence as such investigation would be outside the scope of the PASS study.

2.8.4. In each detailed analysis of safety-net-related event, the influencing factors were identified and described in detail for the following items:

• disparity in STCA triggering time,

• controller’s reaction,

• pilot’s reaction to ATC instructions,

• efficacy of ATC instructions,

• TCAS RA follow-up by pilots.

A synthetic view of one of these analyses can be found in Appendix B.

2.8.5. These analyses are necessary for the identification of operational hazards in the operational safety assessment of WA4. They might also highlight the need to include a specific influencing factor in the design of the models of WA2.

2.9. Specific analysis of RA downlink

2.9.1. This study (fully reported on in [RADL]) focussed on the analysis of RA downlink data. The objectives were to present quantified results about the quality and reliability of RA downlink data; to provide a set of statistical figures dealing with the operational aspect in addition to the technical aspect and to make technical recommendations for the display of RA downlink on the controller working position.

2.9.2. The RA downlink data used for this study are Mode S RA reports and correspond to the extraction by Mode S ground radar of a Comm-B message containing data about the RA. They originated from six Mode S radars with a coverage including most of the European core area and were collected from September 2007 to March 2008. This represents 1,333,000 flight hours.



2.9.3. Radar data files were processed off-line in order to extract RA downlink messages. For each sequence of RA downlink messages sent by one aircraft, an extract of radar data is built in order to obtain all the data corresponding to this aircraft and to the traffic around this aircraft.

2.9.4. All RA downlink messages and radar data extracts corresponding to the same occurrence were associated. So, when two aircraft involved in the same occurrence both sent RA downlink messages, the data corresponding to both aircraft are associated.

2.9.5. A specific tool was developed in order to perform TCAS simulations directly from the multi-radar data. The tool performs fast-time simulations in order to obtain simulated RAs matching as closely as possible with the RA downlink messages.

2.9.6. The TCAS simulation was validated when the sequence of simulated RAs matched the sequence of RAs from the RA downlink. In this case, the TCAS simulation was used as a reference to check the other contents of the RA downlink message, such as the threat identity. Otherwise, a case-by-case analysis was performed.

2.9.7. Data from the RA downlink messages were cross-checked against other data downlinked from the aircraft (e.g. contents of the BDS10 register, altitude). This allowed the consistency between data to be checked. For example, the aircraft type obtained from the ICAO address made it possible to check that the reported ACAS equipage was consistent with the ACAS mandate.

2.9.8. All the occurrence data were stored in a spreadsheet table allowing the derivation of statistical results. These data included: extracted data (decoded RA downlink messages, ICAO address, Mode A, aircraft identification, altitude), data from simulations, data derived from other data (aircraft type, military or civil aircraft) and the results of the analyses performed by a TCAS expert.



3. Results of PASS monitoring activity

3.1. General description of SNET occurrences investigated

3.1.1. Figure 7 gives the distribution of the occurrences by type of alert. Most of the occurrences feature an STCA (91%), while RAs occur in a large part of the occurrences (41%).

STCA only 105

RA only 16

STCA + RA 59

Figure 7: Distribution of SNET occurrences vs. type of alert

3.1.2. The altitude distribution of the occurrences reflects the density of traffic. Two peaks can be observed: one at FL130/140, which is characteristic of arrival/departure vertical crossings, and another around FL350, for aircraft in cruise, mainly in level flight. The peak around FL350 is less pronounced for RA-only occurrences, which is due to the reduced sensitivity of ACAS to level-off situations at high altitudes.

3.1.3. STCA occurrences are mainly distributed between 08h00 UTC and 14h00 UTC with a peak at 12h00 UTC. RA occurrences are quite evenly distributed from 06h00 UTC to 22h00 UTC.

3.1.4. In a non-negligible number of occurrences (20%), two control sectors are involved, one for each aircraft. This happens more frequently than expected, which could be due to occurrences of this type being reported more often, because a problem with coordination between sectors may seem of particular concern. This issue would be worthy of further investigation during Phase 3 of the PASS project.

3.1.5. Standard separation was almost always lost before minimum separation for STCA-only occurrences. This reflects the focus on occurrences with loss of separation resulting from the use of reported occurrences. A small proportion of the occurrences involving RAs have aircraft separated at minimum separation. There are two reasons for this: RAs can be triggered without separation loss, and a reaction to an RA can prevent a predicted loss of vertical separation or restore an actual loss of vertical separation.



3.2. Adequacy of alerts

3.2.1. STCA warning time

3.2.1.1. Figure 8 shows the distribution of STCA time around the time where the loss of separation (LoS) was observed. The difference between the time of observed LoS and the time of alert is an approximation of the real warning time given by STCA to the controller: the real warning time is the time between the alert and the predicted time of LoS. The LoS prediction assumes that the aircraft remain on their course. If, between the start of the alert and the predicted time of LoS, any aircraft performs a manoeuvre prompted by the controller or by TCAS, the observed time of LoS will be different from the predicted time of LoS. The observed STCA warning time plotted below includes the results of any aircraft manoeuvres.

0

5

10

15

20

25

30

> 120

s

119-1

10 s

109-1

00 s

99-90

s

89-80

s

79-70

s

69-60

s

59-50

s

49-40

s

39-30

s

29-20

s

19-10

s9-1

s

simult

aneo

us 1-9 s

10-19

s

20-29

s

30-39

s

40-49

s

50-59

s

60-69

s

70-79

s

80-89

s

90-99

s

100-1

09 s

110-1

19 s

> 120

s

STCA alert before LoS STCA alert after LoS

Num

ber o

f airc

raft

In approachIn en-route

Figure 8: Distribution of aircraft involved in STCA occurrences vs. time between STCA start and actual loss of separation

3.2.1.2.

3.2.1.3.

In most cases, the STCA is triggered sufficiently early before the predicted loss of separation, allowing ATC to take effective corrective action to maintain separation. However, it was observed in at least one occurrence that too long a warning time may lead to no reaction by ATC, in particular if clearances (and read backs) are correct.

In many cases, a short STCA warning time was observed, which either increased the risk of interaction with TCAS or reduced significantly the efficacy of ATC corrective actions or even prevented the possibility of such action.



3.2.1.4.

3.2.1.5.

3.2.1.6.

Sometimes, there was an observed loss of separation but no STCA. One possible reason for this is that the aircraft were already diverging, i.e. the horizontal distance between them was increasing, when separation was lost. In that case the minimum distance for an alert is smaller than when aircraft are converging.

The shape of the distribution is similar for en-route and approach areas, with a shorter range of warning times for the approach area. Indeed, the warning time before observed LoS is shorter mainly due to reduced STCA-triggering thresholds for some ANSPs in the approach area. The average observed warning time is 26 seconds before observed LoS in en-route and 20 seconds in approach.

An example of an occurrence with average warning time is given in Figure 9. Two aircraft are on crossing tracks in en-route airspace. One is climbing (blue trajectory) and one is descending (red trajectory). The STCA is triggered 25 seconds before the observed (slight) LoS.

Legend: The dotted black line represents the STCA triggering time;; the solid black line shows the relative aircraft positions at the time of minimum separation; the solid green line shows the time of LoS.

Horizontal view Vertical profile

AC1 �

AC2 �

AC1 �

AC2 �

Figure 9: Horizontal and vertical views of an occurrence with average observed STCA warning time

3.2.1.7.

3.2.1.8.

The occurrences where the STCA starts after the actual LoS are of the following types:

• Interface between two centres with different separation minima;

• Slow convergence between aircraft;

• Start of vertical evolution towards the other aircraft when both aircraft are initially separated by 1,000 ft and close horizontally.

An example of a problem occurrence is given in Figure 10. Two aircraft are both climbing from the same airport. A separation of 3 NM is applied, which is correct for approach sectors but not for the subsequent en-route sectors, which require 5 NM. The STCA is triggered only when the aircraft leave the approach area (i.e. above FL130) because a filter prevents the STCA from being shown to the en-route controller while the aircraft are still within the approach volume.



Legend: The dotted black line represents the STCA triggering time; the solid black line shows the relative aircraft positions at the time of minimum separation; the solid green line shows the time of LoS.


AC1 �

AC2 �

AC1 �

AC2 �

Figure 10: Horizontal and vertical views of an occurrence with late STCA triggering

3.2.1.9.

3.2.1.10.

3.2.1.11.

As seen in Figure 8 for the approach and en-route areas, the set of parameters used in the airspace is clearly an important factor in the STCA triggering time. The detailed analysis of specific encounters also highlighted the following factors:

• the geometry of the encounter:

o head-on or crossing or catch-up

o rapid or sudden vertical evolution

o slow vertical rates, etc

• the possible use of CFL.

An example of an occurrence influenced by the use of CFL is given in Figure 11. A level aircraft (red trajectory) is controlled by ATC1. A descending aircraft, on crossing track (blue trajectory), controlled by ATC2 is to level off 1,000 ft above the first. The descending aircraft busts its FL and receives simultaneously a TCAS RA to climb and an avoiding instruction to climb back from ATC2.

The ATC1 STCA is triggered 42 seconds before LoS. The ATC2 STCA, which uses CFL, is triggered only 3 seconds before LoS. The RAs are received 6 seconds after LoS in both aircraft. In this particular case, the late alert of STCA2, due to the use of CFL, leaves little time for the controller to react.



Legend: BBBooottthhh SSSTTTCCCAAA ttt rrr iiiggggggeeerrr iiinnnggg ttt iiimmmeeesss aaarrreee dddeeepppiiicccttteeeddd iiinnn yyyeeelll lllooowww; the solid black line shows the relative aircraft positions at the time of minimum separation; the solid green line shows the time of LoS.


STC

A

AC2FL320

(with ATC2)

AC1FL310=

(with ATC1)

STC

A

AC2FL320

(with ATC2)

AC1FL310=

(with ATC1)

AC2FL320

AC1FL310=

AC2FL320

AC1FL310=

Figure 11: Horizontal and vertical views of an occurrence with CFL use

3.2.2. TCAS RA warning time

3.2.2.1.

3.2.2.2.

As STCA is designed to warn of an actual or predicted LoS, the measure for its warning time is based on the time of LoS. TCAS is designed to warn of a potential collision. Furthermore, the triggering of an RA depends on many factors, including the encounter geometry and closing speeds. That is why the observed TCAS warning time is measured not on the infringement of a given distance between aircraft, but on the time at which the aircraft reach their closest approach.

Figure 12 gives the distribution of the initial RA alert time around the time when the closest approach was observed. This is an approximation of the real warning time: as explained for the observed LoS, the observed closest approach differs from the predicted closest approach because it includes the results of any aircraft manoeuvres prompted by the controller or by TCAS.



0

2

4

6

8

10

12

14

16

18

> 51 s

50-46 s

45-41 s

40-36 s

35-31 s

30-26 s

25-21 s

20-16 s

15-11 s

10-6 s

5-1 s

simult

aneous 1-5

s6-10

s> 11 s

RA be fore close st a pproa ch RA a fte r close st a pproa ch

Num

ber o

f airc

raft


Figure 12: Distribution of time between initial RA and actual time of closest approach

3.2.2.3.

3.2.2.4.

3.2.2.5.

3.2.2.6.

In almost all cases, the TCAS RA is triggered with sufficient time before closest approach, as required by TCAS specifications, allowing the flight crew to ensure safe vertical separation at closest approach.

In a few cases, a shorter than nominal TCAS warning time was observed, reducing or negating the efficacy of the avoidance manoeuvre.

The observed warning time is reduced in the approach area because TCAS RA thresholds are reduced in the lower altitude layer, where the occurrence happens. The average observed delay is 28 seconds in en-route and 18 seconds in approach.

An example of an occurrence with average delay is given in Figure 13. Two aircraft are on crossing tracks. One is level (blue trajectory) and the other (red trajectory) climbing rapidly to the adjacent level, 1,000 ft below. The RA is triggered onboard the climbing aircraft 27 seconds before the observed closest approach. The pilot busts its cleared FL while following the RA. Despite the subsequent LoS, there is no STCA.



Legend: The red square shows the time of the initial RA onboard the red aircraft, the solid black line shows the relative aircraft positions at the time of minimum separation, the dotted black line shows the relative aircraft positions at the time of closest approach, the solid green line shows the time of LoS (here, simultaneous with the closest approach).


AC1FL120

AC2 FL130

TCAS RA

CPP

AC1FL

AC2 FL130

120

CPPAC2

AC1

FL120

TCAS RA

CPP

TCAS RA: « ADJUST »

CPPAC2

AC1

FL120

TCAS RA: « ADJUST »

Figure 13: Horizontal and vertical views of an occurrence with average RA observed warning time

3.2.2.7. An example of an occurrence with insufficient time for an effective reaction is given in Figure 14. An IFR aircraft (blue trajectory) is flying 500 ft above a VFR flight (red trajectory). An RA is received by the IFR almost at closest approach. The RA is late mainly due to the imprecise altitude keeping of both aircraft (plus or minus 100 ft).

Legend: The red square shows the time of the initial RA onboard the blue aircraft, the solid black line shows the relative aircraft positions at the time of closest approach.


AC2�1500 ft

AC11000 ft �

AC11000 ft �

AC2�1500 ft

ClimbRA Climb

RA

1500 ft

Figure 14: Horizontal and vertical views of an occurrence with late RA observed warning time



3.2.3. Compatibility between STCA and TCAS

3.2.3.1. Figure 12 gives the distribution of STCA time around the time where the initial RA is triggered. This distribution reflects two cases:

• the STCA gave the controller enough time but his/her reaction (or the pilot’s) was non-existent/too late/insufficient; or

• the STCA occurred too late for the controller’s intervention to avoid an RA.

0

1

2

3

4

5

6

7

8

9

> 61 s

60-56

s

55-51

s

50-46

s

45-41

s

40-36

s

35-31

s

30-26

s

25-21

s

20-16

s

15-11

s10

-6 s

5-1 s

simult

aneo

us 1-5 s

6-10 s

> 11 s

STCA before RA STCA after RA

Num

ber o

f airc

raft


Figure 15: Distribution of aircraft involved in STCA + RA occurrences vs. time between STCA start and time of initial RA

3.2.3.2.

3.2.3.3.

3.2.3.4.

In half of the cases, the STCA is triggered well before the TCAS RA. When less than 20 seconds remains before the TCAS RA, the controller’s reaction may potentially occur shortly before or shortly after the RA, possibly leading to pilot confusion.

When the STCA is triggered after the TCAS RA, the potential for adverse interaction is reduced, as the response to the RA may be over when the controller’s avoiding instruction is transmitted, or a pilot RA report might prevent the controller from issuing his own avoiding instruction.

An example of an occurrence with sufficient time to react is given in Figure 16. One aircraft (red trajectory) is climbing while another aircraft (blue trajectory) is descending on a crossing track. Both are to level off at 1,000 ft separation, however the high vertical closure rate triggers an STCA 44 seconds before the RA is received onboard the descending aircraft. This significant period should provide enough time to avoid the RA. Unfortunately, the controller’s reaction is late. Ultimately the RA is followed rather than the controller’s instruction.



Legend: The dotted black line represents the STCA triggering time; the red square shows the time of the initial RA onboard the blue aircraft; the solid black line shows the relative aircraft positions at the time of closest approach, the double black line shows the relative aircraft positions at the time of minimum separation.


ClimbRA

AC1�FL300

AC2�FL310

AC1�FL300

AC2�FL310

FL300

FL310

ClimbRA

FL300

FL310

Figure 16: Horizontal and vertical views of an occurrence with STCA triggered sufficiently early before RA

3.2.3.5. An example of an occurrence with insufficient time to react is given in Figure 17. A climbing aircraft (blue trajectory) is expected to level off 1,000 ft below a level aircraft (red trajectory). Instead, the climbing aircraft overshoots its cleared flight level. The STCA is triggered 10 seconds before the coordinated RA is received onboard both aircraft. Only traffic information was given to both aircraft as a result. The observed short STCA warning time is linked to approach airspace STCA parameters.

Legend: The dotted black line represents the STCA triggering time; the red squares show the time of the initial RA onboard both aircraft, the solid black line shows the relative aircraft positions at the time of closest approach, the double black line shows the relative aircraft positions at the time of minimum separation; the solid green line shows the time of LoS.


AC1FL110 �

AC2�FL100

AC1FL110 �

AC2�FL100

FL100

FL110



Figure 17: Horizontal and vertical views of an occurrence with STCA triggered close to the RA

3.3. Characteristics of controller reactions

3.3.1. Type of reaction

3.3.1.1.

3.3.1.2.

Figure 18 shows how frequently the controllers chose to intervene and by which method for each type of occurrence. The method might involve giving traffic information, shortened to TI, or providing avoiding instructions, shortened to AI. Avoiding instructions (alone or in combination) are of course more likely in STCA-only occurrences, while they do not appear at all in RA-only occurrences.

In a small proportion of occurrences (14%), no action at all is taken by the controller after the STCA is triggered. Two main reasons can be posited for this:

• controllers think that the separation standards will still be maintained, by a planned aircraft manoeuvre for example;

• too little time remains for controller action to be effective before the aircraft have crossed, due to late STCA triggering.

0

20

40

60

80

100

120

STCA only STCA + RA RA only

Type of occurrence

Num

ber o

f occ

urre

nces

No actionTI only

AI only

TI and AI

Figure 18: Distribution of SNET occurrences vs. controller intervention method

3.3.2. Number of aircraft acted upon

3.3.2.1. Avoiding instructions may be issued for both aircraft in the following cases:

• Two different sectors are involved and each acts on the aircraft which is its responsibility;



• The first aircraft neither reads back nor executes the avoiding instruction, forcing the controller to act on the other aircraft;

• The controller prefers to minimise route deviation by issuing a minor change instruction to two aircraft rather than a significant change instruction to one aircraft.

• The controller does not want to take any chances in a hazardous situation.

3.3.2.2. Figure 19 shows how frequently the controllers acted on only one or both aircraft when issuing avoiding instructions.

0

10

20

30

40

50

60

70

80

90

STCA only STCA + RA RA only

Type of occurrence

Num

ber o

f occ

urre

nces

AI to 2 a /c

AI to 1 a /c on ly

Figure 19: Distribution of SNET occurrences vs. number of aircraft issued with an avoiding instruction

3.3.2.3.

3.3.3.1.

Compared with STCA-only occurrences, STCA + RA occurrences are likely to be more serious, which explains why avoiding instructions are more often issued to both aircraft rather than to one.

3.3.3. Time between two avoiding instructions issued to different aircraft

Figure 20 gives the distribution of the time elapsed between the first avoiding instruction given to the first aircraft and the first avoiding instruction given to the second aircraft.



0

2

4

6

8

10

12

14

0-4 5-9 10-14 15-19 20-24 25-29 > 30

Tim e betw een first avoiding instructiongiven to each aircraft of an occurrence

Occ

urre

nces

Figure 20: Distribution of SNET occurrences vs. time between first avoiding instructions given to each aircraft

3.3.4. Time between STCA start and first controller avoiding instruction

3.3.4.1. Figure 21 gives the distribution of the time of the avoiding instruction around the time of the STCA. A distribution is shown for each aircraft involved in the occurrence. Of course, the second distribution is based on a smaller sample, as an AI was not always issued to both aircraft.

0

5

10

15

20

25

30

35

40

> 30 s

29-20 s

19-10 s

9-1 s

simult

aneous 1-9

s

10-19 s

20-29 s

30-39 s

40-49 s

50-59 s

60-69 s

70-79 s

80-89 s

90-99 s

> 100 s

AI be fore STCA AI afte r STCA

Num

ber o

f airc

raft

Firs t AI to a /c 1

Firs t AI to a /c 2

Figure 21: Distribution of aircraft vs. time between first avoiding instruction and first STCA start



3.3.4.2.

3.3.4.3.

3.3.5.1.

On average, the first avoiding instruction for the occurrence is issued 10 seconds after the STCA start, but there is a wide interval of values around that. When an avoiding instruction is issued to the other aircraft involved in the occurrence, it is issued on average 10 seconds after the first one, i.e. 20 seconds after the STCA start, but with the same kind of distribution.

Looking at specific occurrences, the time taken by the controller to react to the alert appears to be influenced by the following:

• controller perception of the conflict: a late controller avoiding instruction may result from the initial confidence of the controller that there will be no LoS;

• external intervention: an avoiding instruction issued late or not at all by the controller may be the result of intervention by a third party, e.g. a controller from an adjacent sector phoning about a situation giving rise to concern; and

• other human factors: a controller already under stress due to traffic pressure may react earlier when an STCA is triggered. On the other hand, a heavy workload (other tasks, jammed RT) or traffic complexity (other aircraft not on frequency) may delay the controller’s reaction to certain STCAs.

3.3.5. Direction of avoiding instructions

Figure 22 shows how frequently a horizontal or vertical sense is chosen for the avoiding instruction, depending on the horizontal geometry of the occurrence.

639 13

1

0 0

2448

133

0

5

2 8 1 0

0

1

0%10%20%30%40%50%60%70%80%90%

100%

High track

conve

rgence

Medium tra

ck co

nvergence

Slight t r

ack c

onve

rgence

Merging t

o same t

rack

Merging t

o paralle

l t rac

ks

Parallel

t rack

s

Horizontal geom etry

Airc

raft Both

Horizontal AI

Vertica l AI

Figure 22: Distribution of aircraft vs. horizontal geometry and direction of the avoiding instruction



3.3.5.2.

3.3.5.3.

Horizontal avoiding instructions are more often used for high angles of convergence. The tighter the angle of convergence, the more controllers seem to use vertical avoiding instructions. The lack of data on the parallel track geometries does not allow any conclusions to be drawn for these geometries.

Figure 30 shows how frequently a horizontal or vertical sense is chosen for the avoiding instruction, depending on the vertical geometry of the occurrence

3

5

10

330 8

2

14

1

1551 10

0 1 0 1 8 2

0%

20%

40%

60%

80%

100%

Both leve

l, at d

ifferent a

ltitude

s

Both leve

l, at s

ame a ltit

ude

1+ leve

l-of f a

t diffe

rent altit

udes

1+ leve

l-of f a

t sam

e a ltitude

1+ non-le

vel, c

onvergi

ng

Both non-le

vel, s

ame sense

Vertical geom etry

Airc

raft Both

Horizonta l AI

Vertical AI

Figure 23: Distribution of aircraft vs. vertical geometry and direction of the avoiding instruction

3.3.5.4.

3.4.1.1.

When aircraft are already at or are planned to be at the same altitude, horizontal avoiding instructions seem to be preferred. Situations involving classic level-offs at different altitudes are usually addressed by a vertical avoiding instruction (usually a “Maintain FL”). In the geometries where one or both aircraft are moving vertically throughout the occurrence, only a slight preference towards horizontal avoiding instructions is seen.

3.4. Characteristics of pilot reactions

3.4.1. Frequency of reactions

The following three figures show how frequently pilots (of at least one aircraft) respond to avoiding instructions and/or ACAS RAs. Figure 24 focuses on STCA-only occurrences, Figure 25 focuses on RA-only occurrences and Figure 26 focuses on STCA + RA occurrences. Note that when a response exists, it is not necessarily compliant with the avoiding manoeuvre requested (see 3.4.3 and 3.4.5).



3.4.1.2. The level of non-response grows in relation to the severity of the situation. It is very low when ACAS is not involved (4%). It is significant when an ACAS RA occurs alone (19%).

W ith AI only and at least 1 a/c

responds96%

W ith AI only and no a/c responds

4%

W ith RA only and no a/c responds

19%

W ith RA only and at least 1 a/c

responds81%

Figure 24: Distribution of (84) STCA-only occurrences with AI vs. response to AI

Figure 25: Distribution of (16) RA-only occurrences vs. response to RA

3.4.1.3. For the occurrences with both kinds of alerts, the rate of non-response is in-between, but still low (6%, combining non-responses to AI + RA and non-responses to RA when no AI was issued following STCA).

W ith A I + RA and response to both

36%

W ith RA only and no a/c responds

4%

W ith RA only and at least 1 a/c

responds40%

W ith A I + RA and at least one

response 18%

W ith A I + RA and no a/c responds

to either2%

Figure 26: Distribution of (56) STCA + RA occurrences vs. response to AI and RA



3.4.2. Time distribution of reactions to avoiding instructions

3.4.2.1.

3.4.2.2.

3.4.2.3.

Figure 27 gives the distribution of the time when the aircraft track shows the start of a manoeuvre after start of receipt of an avoiding instruction. The figure distinguishes between avoiding instructions which use the avoiding instruction phraseology and those which do not.

Reaction times of below ten seconds have been observed, which may appear too fast in view of the unavoidable delay caused by human decision and communication latencies. First, there is a margin of error in the measures as a result of uncertainties in the synchronisation between RT transcripts and radar tracks. In addition, some pilots may have had a quicker reaction because of increased awareness on account of previous traffic information, visual or electronic acquisition. Finally, some pilot actions were performed in response to almost simultaneous and compatible ACAS RA and AIs.

Two sets of reactions seem to exist: timely reactions distributed along a curve up to 25 seconds and, based on limited data, late reactions distributed uniformly after that. In the first set, the use of avoiding instruction phraseology allows an average gain of 3 seconds on the implementation of the manoeuvre, i.e. from 11.7 seconds to 8.6 seconds. This figure seems low and requires further investigation.

0

5

10

15

20

25

simult

aneous

1-5 s

6-10 s

11-15 s

16-20 s

21-25 s

26-30 s

31-35 s

36-40 s

41-45 s

46-50 s

51-55 s

> 56 s

Time betw een AI instruction andmanœuvre in response

Num

ber o

f airc

raft

With AI phraseologyWithout AI phraseology

Figure 27: Distribution of aircraft vs. time between first controller avoiding instruction and start of pilot’s response

3.4.2.4. The influence of the avoiding instruction phraseology was illustrated previously, but other influencing factors were found in the series of specific occurrences:

• the quality of controller/pilot communications: in one occurrence, a pilot did not understand the ATC instruction even though avoiding instruction phraseology was used. Possibly the controller was speaking too fast;



• the provision of traffic information by ATC: traffic information can help the pilots gain a better understanding of the situation and react more promptly to the ATC avoiding instruction;

• the compatibility of the ATC instruction with the expected flight path: in one occurrence, the pilot did not turn as requested by ATC as the turn was contrary to his flight plan;

• visual acquisition by the pilot: this can help the pilot understand the situation better, and apply more promptly the avoiding instruction required by ATC. However, it can cause the pilot not to execute the instruction. In addition, visual acquisition is not always correct; and

• the compatibility of the ATC instruction with TCAS RAs, if any: a compatible instruction may reinforce the RA and lead to an overreaction, while an incompatible instruction requires a choice by the pilot, and if the RA is followed, may lead to horizontal instructions being ignored.

3.4.3. Compliance of reactions to avoiding instructions

3.4.3.1.

3.4.3.2.

Figure 28 shows how frequently the reaction to a vertical avoiding instruction was correct, partially correct, non-existent or opposite, depending on the type of vertical avoiding instruction.

The only opposite response was an action to reduce the rate of descent instead of increasing it as requested by the controller. Later, the pilot received an RA to descend, which was not followed.

Expedite

Climb level

Maintain level

Maintain reaching

Descend level

Correct

Partial

None

Oppos

ite

0

2

4

6

8

10

12

14

16

Number of aircraft

Vertical avoiding action

Quality of response

Figure 28: Distribution of aircraft vs. type of vertical avoiding instruction and pilot’s degree of compliance



3.4.3.3.

3.4.3.4.

The values seem to indicate that vertical AIs to maintain level are the least followed, and that vertical AIs to climb are less followed than vertical AIs to descend. However, the low number of cases for each category means that only the general trend can be used with caution when designing the model of pilot response to controller AIs.

Figure 29 shows how frequently the reaction to a horizontal avoiding instruction was correct, partially correct, non-existent or opposite, depending on the type of horizontal avoiding instruction.

Turn le ftMaintain heading

Turn r ight

Increase speed

Reduce speed

Co rrect

Partial

None

Opposite

0

5

10

15

20

25

30

35

Num be r of a ircra ft

Horizonta l a voiding instruction

Qua lity of re sponse

Figure 29: Distribution of aircraft vs. type of horizontal avoiding instruction and pilot’s degree of compliance

3.4.3.5.

3.4.3.6.

Taking into account the size of the samples for each category, it would appear (as common sense suggests) that there is no difference in following horizontal AIs to the left or to the right. The few cases of speed change as an AI were used in the approach path, but are not numerous enough to be taken into account in the design of the model of the controller AI.

The efficacy (in terms of increase in the distance between aircraft) of avoiding instructions with a pilot response was studied in more depth for specific occurrences. Of 10 occurrences with a horizontal AI, only 3 AIs were effective Of 8 occurrences with a vertical AI, 5 AIs were effective. The samples are too small to support a definitive conclusion that vertical instructions are more often effective than horizontal ones, although such a conclusion would fit with the greater reactivity of aircraft in the vertical plane, in particular at high altitude.



3.4.4. Reaction times to resolution advisories

3.4.4.1. Figure 30 gives the distribution of the time when the aircraft track shows a start of manoeuvre in the vertical dimension after receipt of its initial TCAS RA.

0

5

10

15

20

25

30

35

simult

aneou

s1-3

s4-6

s7-9

s

10-12 s

> 13 s

Time between TCAS RA andmanœuvre in response

Num

ber o

f airc

raft

Figure 30: Distribution of aircraft involved in RA occurrences vs. time between initial TCAS RA and start of aircraft manoeuvre

3.4.4.2.

3.4.5.1.

With an average of 5 seconds and taking into account a margin of error of around 3 seconds for the time measurement, the pilot’s response time is as assumed by ACAS logic.

3.4.5. Compliance of reactions to resolution advisories

Figure 31 shows how frequently the reaction to the initial TCAS RA was excessive, correct, partially correct, non-existent or opposite, depending on the type of RA (Cl=Climb, Des=Descend, LC=Limit Climb, LD=Limit Descent.)



ClLD

LCDes

OverreactionCorrect Partial None Opposite

0

5

10

15

20

25

30

Number of aircraft

Resolution advisory

Quality of response

Figure 31: Distribution of aircraft vs. type of RA and pilot’s degree of compliance

3.4.5.2.

3.4.5.3.

3.4.5.4.

3.4.5.5.

Taking into account the size of the samples for each category, it would appear that the compliance with RAs to climb, descend or reduce vertical speed (i.e. LC and LD) is of the same order of magnitude.

Three opposite reactions were climb manoeuvres following an “Adjust vertical speed, adjust” RA requesting a reduction in climb rate, and even sometimes a level-off. The fourth opposite reaction was a climb manoeuvre following a “Descend” RA.

Regarding the interaction between responses to RAs and avoiding instructions, there were only five occurrences in the following sequence: RA, avoiding instruction, TCAS RA report. This sequence means that an ongoing RA which has not yet been announced to the controller may be disrupted by the controller’s intervention.

Three of these are with vertical avoiding instructions and two with horizontal avoiding instructions. For those with vertical avoiding instructions, the avoiding instruction phraseology was used in 2 cases, and this led to a delayed reaction to the RA (around 10 seconds after the RA). The other occurrences had normal delays. All RAs were followed in the right direction, despite one avoiding instruction being incompatible with the RA.



3.5. Characteristics of occurrences identified through RA downlink

3.5.1. These occurrences were captured using 6 mode S radars covering most of the European core area, as illustrated in Figure 32. Overlapping areas had to be processed to avoid duplicates, but also allowed a better sampling of the RA sequence, because RA downlink messages from a given aircraft were received more frequently in these areas.

Roissy (CD)La Dole (LD)Toulouse (TS)

Chaumont (CH)Nice (CA)Vitrolles (PV)

Roissy (CD)La Dole (LD)Toulouse (TS)

Chaumont (CH)Nice (CA)Vitrolles (PV)

Figure 32: Radar coverage of the RA downlink recording



3.5.2. Collection took place over a time span of 7 months (September 2007 to March 2008), recording both aircraft trajectories and RA downlink messages. The radar data collected amounts to 15,200 hours. The recording was not continuous as some radars were not available due to maintenance, upgrading or network failure. This corresponds to an estimated 1,333,000 flight hours.

3.5.3. 350,000 RA downlink messages were extracted overall from the 15,200 hours of Mode S radar data. As during a TCAS occurrence, several RA downlink messages can be sent by a given aircraft, the number of aircraft involved in message transmission was only 1,332.

3.5.4. However, only 12,476 messages came from real TCAS RAs. This corresponds to 1,029 aircraft, involved in 880 occurrences. On average, each aircraft has an RA sequence of 960 flight hours. The other messages are sent without an active TCAS RA, but are easy to filter out because the content of the message is not consistent with a TCAS RA.

3.5.5. In each occurrence, at least one aircraft had a TCAS RA. The equipment (transponder and TCAS) of the intruder and the existence of a coordinated RA is indicated in Figure 33.

Distribution of the 880 occurrences

Intruder with RA capable TCASand with coordinated RA

17%

Intruder with RA capableTCAS but without RA

33% Intruder with TCASin TA-Only Mode(without RA)

2%

Mode S intruder (no TCAS orinoperative TCAS)

12%

Mode A/C intruder 29%

Ghost intruder7%

Figure 33: Intruder equipment



3.5.6. Half (52%) of the occurrences took place between two aircraft equipped with an active TCAS. However, a sequence of RAs was reported onboard both aircraft only in a minority of cases. This lack of RA onboard the intruder is normal because it mostly originates from TCAS features used to reduce the number of unnecessary RAs, which are asymmetric. In the case of 1,000-foot level-off geometries, the level aircraft may not have an RA in order to avoid disturbing ATC. In the case of a sufficiently high predicted distance at closest approach, the prediction is highly sensitive to how the relative bearing changes, which may be seen slightly differently by both aircraft.

3.5.7. A ghost intruder appears in a small fraction of the occurrences (7%). Their origin is varied:

• garbled altitude; • self-tracking; • on-ground aircraft reporting in-flight status; • careless transponder tests on the ground; • threat simulation during a test flight.

3.5.8. The occurrences may have been provoked (military operations, flight tests, etc.) or be purely involuntary. The distribution of occurrences according to this factor is shown in Figure 34.

unintentionaloccurrences

70%

intentionaloccurrences

16%

unqualifiedoccurrences

7%

ghostoccurrences

7%

Figure 34: Intentionality of occurrences

3.5.9. The altitude distribution of the occurrences is shown in Figure 35. The peak of occurrences at low altitudes is mainly due to VFR/IFR occurrences. Unintentional occurrences happen also mainly in two altitude bands: FL90 to FL140 and FL210 to FL360.



0

10

20

30

40

50

60

70

80

0 50 100 150 200 250 300 350 400 450

FL

Num

ber of occurrences

unintentional occurrences Intentional occurrences

Figure 35: Altitude distribution of occurrences

3.5.10. The hourly distribution of the occurrences is shown in Figure 36.

unintentional encounter

01020304050607080

0:001:0

02:0

03:0

04:0

05:0

06:0

07:0

08:0

09:0

010

:0011

:0012

:0013

:0014

:0015

:0016

:0017

:0018

:0019

:0020

:0021

:0022

:0023

:00

Figure 36: Hourly distribution of unintentional occurrences

3.5.11. A large majority (95%) of active RA sequences lasted less than one minute. The RA sequence duration for those sequences is shown in Figure 37. Generally, the RA duration is linked to the encounter altitude because of the increase in the threshold values with altitude. The gap of around 20 seconds could not be explained.



0

30

60

90

120

150

180

0 10 20 30 40 50 60duration (s)

Num

ber o

f seq

uenc

es w

ith a

ctiv

e R

A

all encountersintentionalunintentionalunknown

Figure 37: Duration of the RA sequences

Sequences lasting more than one minute correspond mainly to intentional occurrences, and sometimes to unintentional occurrences featuring a very slow range divergence after overtaking.

3.5.12. The most frequent types of RA sequence are shown in Figure 38. The RA sequence is a single corrective vertical speed limit (VSL) RA in half of the occurrences (55%), a corrective RA (sometimes followed by a weakening VSL RA) in a quarter of the occurrences (23%), and a single preventive RA in a small proportion of the occurrences (10%). The other sequences (12%) are more complex.

0

50

100

150

200

250

climb,cl_adjust(ld)

descend,de_adjust(lc)

adjust (ld) adjust (lc) monitor (ld) monitor (lc) other

Figure 38: Most frequent sequences of RAs



4. Conclusions

4.1. Synthesis of results

4.1.1. 180 SNET occurrences covering a wide geographical area and the whole range of aircraft operations were provided either directly (DSNA, DFS and skyguide) or indirectly (United Kingdom, Denmark, Czech Republic, Estonia). Elementary events related to safety nets were extracted and used to compute performance indicators.

4.1.2. From the previous set, 12 SNET occurrences were selected for more detailed analysis, taking into account more contextual elements, in particular human factors, in order to highlight influencing factors of the occurrence.

4.1.3. As a result of using manually reported data, the set of occurrences is thought to be biased towards losses of separation. This must be kept in mind when exploiting the results as they characterise the more challenging situations and must be adjusted when nominal situations are to be addressed.

4.1.4. The observed warning time for STCAs is 26 seconds on average in en-route airspace and 20 seconds in approach. It is influenced by the STCA parameters (en-route or approach, whether CFL is used, etc.), the encounter geometry and the filtering strategy.

4.1.5. 14% of STCAs elicited no controller reaction. When the controller decides to issue an instruction, both aircraft are often acted upon, with an average 10-second time span between the instructions. The first instruction is issued 10 seconds after the STCA on average, but a small proportion occur before and another small proportion well after. The timing of the controller’s reaction appears to be influenced by the STCA warning time and by his perception of the conflict.

4.1.6. Concerning controller instructions, the horizontal direction seems to be chosen predominantly in situations of high or medium horizontal convergence as well as for occurrences with aircraft at the same altitude. The vertical sense seems to predominate in standard 1,000-ft level-off situations.

4.1.7. Pilots respond to nearly all avoiding instructions, with a slight delay most of the time. Oddly, the influence of the avoiding instruction phraseology is weak. The timing appears to be influenced by the quality of the communications, previous provision of traffic information, successful visual acquisition and the compatibility of the instruction with the expected path and with an ongoing TCAS RA.

4.1.8. 19% of RAs in RA-only occurrences elicited no pilot reaction. Where there is a reaction, the average delay is compliant with ACAS logic assumptions. No significant difference is observed regarding compliance to RAs involving different directions. In 3 STCA + RA occurrences with an RA followed immediately by a controller vertical avoiding instruction, 2 featured a delayed reaction to the RA, maybe due to the use of avoiding instruction phraseology.



4.1.9. In a specific study based on RA downlink data, 350,000 RA downlink messages were collected over seven months in most of the European core area. Almost all (97%) of the messages had to be filtered out (fortunately with no difficulty) as they did not correspond to actual RAs. For actual RAs, a significant part (16%) of these are in fact the result of a deliberate closure (e.g. military operations, flight tests). When the RA sequence duration exceeded 60 seconds, which was very rare (5%), this was mainly in intentional occurrences.

4.2. Setting up the model-based performance evaluation

4.2.1. Introduction

4.2.1.1. As part of Phase 2 of the PASS study, the modelling activities have just been commenced, with the development of various models to allow evaluation of STCA performance and safety benefits while taking into account the effect of ACAS operations. The simulation framework is illustrated in Figure 39.

Figure 39: Framework for the development of STCA performance requirements

4.2.1.2.

4.2.1.3.

This framework advocates the separate modelling of controller intervention in response to STCA and subsequent pilot reaction and a refined modelling of STCA behaviour, including the effect of optional STCA features and the quality of input surveillance data. This framework is intended to support the determination of the contribution of each safety net, both separately and in combination.

The STCA model, the model of controller response to STCA and the models of pilot response to avoiding instructions and to RAs can be better designed using the results of the monitoring activity. The following sections describe the anticipated implications for model design. Consideration of the implications with a marginal impact or excessive complexity will be delayed until Phase 3, where the models will be refined, taking into account the results of Phase 2.



4.2.2. STCA model

4.2.2.1.

4.2.2.2.

4.2.3.1.

4.2.3.2.

4.2.3.3.

4.2.3.4.

4.2.3.5.

The differences observed between STCA implementation for approach and en-route sectors must be taken into account. Different parameters for these areas will therefore be used in simulations to reflect the lower thresholds that are the hallmark of approach sectors. Additionally, for each area, the sensitivity of the STCA performance to the set of parameters will be assessed through incremental parameter variation.

A few (3%) occurrences involve two control sectors from different ANSPs. In those occurrences, each sector has a different STCA implementation. This can be modelled by using a different set of STCA parameters to simulate each STCA implementation. This refinement might be introduced in Phase 3, to address interoperability requirements for STCA at the boundaries of ATC units.

4.2.3. Controller model

The controller model will consider only avoiding instructions as controller responses to STCA. Traffic information will not be considered because when traffic information is issued by ATC for avoiding action by the pilot, the final separation depends much more on the efficacy of the pilot’s response than on the STCA performance. Non-response to STCA might be introduced only when refining the controller model, in Phase 3.

The time between the triggering of the STCA and the first avoiding instruction by a controller could be based on the distribution found in the monitoring activity. The average response time might have to be adjusted to model not only the challenging situations (as observed during the monitoring activity) but also more nominal controller reaction times.

The model may need to simulate the occasional delivery of an avoiding instruction to the other aircraft of the pair, if it is in the same sector. The time between the avoiding instruction to the first aircraft and to the second aircraft could be based on the distribution found in the monitoring activity.

The direction of the instruction will be chosen between the horizontal and vertical dimensions. Instructions in both dimensions and speed instructions will not be considered as they are too rare. There may be a need to take into account encounter geometries to issue realistic avoiding instructions.

Less than a quarter (20%) of the occurrences investigated involved two sectors. In those occurrences, a different controller is in charge of each aircraft of the pair. Each controller is likely to issue an avoiding instruction. Although both instructions are usually complementary, due to direct coordination or compliance with a letter of agreement, those occurrences have been observed to be more severe. Therefore, the possibility of having two (coordinated) controller decision processes in the model should be considered in Phase 3.



4.2.4. Pilot model of response to avoiding instructions

4.2.4.1.

4.2.4.2.

4.2.5.1.

4.2.5.2.

Non-response to avoiding instructions could be modelled in a very small proportion (4%) as indicated by the monitoring activity. This step will be considered in Phase 3, but for Phase 2 all pilots will respond to avoiding instructions.

The time between the avoiding instruction and the observable pilot response could be based on the distribution found in the monitoring activity. The average response time should be reduced to keep only the part due to communication delays and pilot decision, discarding the part due to aircraft manoeuvre, which will be simulated separately. The response time may take into account the existence of an ongoing RA and the compatibility of the instruction with an ongoing RA or with the planned flight path.

4.2.5. Pilot model of response to RAs

Non-response to RAs in the monitoring activity is within the range commonly used (10 to 20%) in the existing pilot model, although close to the extreme figure, which is probably due to the bias of the data collection.

The distribution of the time interval between the RA and the pilot response found in the monitoring activity does not support a change in the current pilot model, all the more so since the sample is small compared to the one used in previous studies on pilot response to TCAS (ACASA, ASARP). Only the possible delay introduced by a close subsequent avoiding instruction could be considered for addition.



5. References

[FRAM] Monitoring framework description, PASS, WA1/WP1/08/D, version 1.1, 05-02-2008

[INF] * Consolidated analysis of a set of events of interest, PASS, WA1/WP5/42W, version 1.1, 02-12-2008

[RADL] Analysis of RA downlink data, PASS, WA1/WP4/38/W, version 1.4, 02-02-2009

[STATS] * Descriptive analysis of observed SNET performance, PASS, WA1/WP5/90W, version 1.0, 20-01-2009

[WEB] * Web incidents monitoring report (slides), PASS, WA1/WP6/80/D, version 1.0, 02-12-2008

Note: The documents marked with an asterisk will be made available to qualified stakeholders on request to EUROCONTROL.



APPENDIX A

List of metrics used for the descriptive analysis

General statistics

• Altitude distribution of safety-net-related occurrences.

• Time distribution of safety-net-related occurrences.

• Distribution of observed separation at closest approach (including any controller intervention and pilot reaction to controller avoiding action or ACAS RA).

Adequacy of alerts

• Delay between STCA start and actual time of separation infringement (observed STCA warning time including any controller intervention).

• Delay between initial RA occurrence and actual time of closest point of approach (observed RA warning time including any pilot reaction).

• Delay between STCA start and initial RA occurrence.

Characteristics of controller reactions

• Number of safety-net-related occurrences with controller avoiding action (to maintain or restore separation).

• Number of safety-net-related occurrences with controller traffic information (with or without controller avoiding action).

• Number of safety-net-related occurrences where the initial RA was in the opposite direction of the controller avoiding action.

• Delay between last controller instruction (e.g. flight level clearance, radar vector, etc.) and STCA start.

• Delay between controller traffic information and STCA start.

• Delay between controller avoiding action (either before or after STCA start) and STCA start.

• Delay between controller traffic information and initial RA occurrence.

• Delay between controller avoiding action and initial RA occurrence.

• Distribution of controllers’ action (nature and strength) versus occurrence geometry.

• Delay between STCA end and subsequent controller instruction (for aircraft to proceed or resume navigation).



Characteristics of pilot reactions

• Number of safety-net-related occurrences with pilot reaction (in response to controller avoiding action or ACAS RA).

• Number of safety-net-related occurrences with pilot ACAS report (to ATC).

• Number of safety-net-related occurrences with pilot visual report (to ATC).

• Delay between controller avoiding action and pilot reaction (start of manoeuvre).

• Level of pilot compliance with controller avoiding action (none, adequate, underreaction, overreaction).

• Delay between first corrective RA occurrence and pilot reaction (start of manoeuvre).

• Level of pilot compliance with RA (none, adequate, underreaction, overreaction).

• Delay between initial RA occurrence and first pilot ACAS report.

Consequences of alerts

• Number of safety-net-related occurrences with observed separation infringement (e.g. 50%, 70%, 80%, etc. of separation minima).

• Distribution of severity, measured according to ESARR2 methodology.



APPENDIX B

Specific study on influencing factors

1) List of occurrences selected for in-depth analysis

Event #1: Conflict due to direct routing

Event #2: Level-bust

Event #3: Conflicting descent

Event #4: Conflict in approach

Event #5: Level bust due to TCAS

Event #6: Crew following TCAS RA rather than ATC

Event #7: Two aircraft following ATC rather than TCAS RA

Event #8: Event involving 3 aircraft

Event #9: Unauthorised descent

Event #10: Slow convergence

Event #11: No loss of separation due to STCA and ATC intervention

Event #12: Loss of separation and TCAS RA but no STCA



2) Synthetic view of the analysis of an occurrence

Conflict Conflict due to direct routing during climb at occupied FL. En-route, single ATC sector involved, crossing tracks, cleared level-off at same altitude.

Actors Single controller (ATC) and STCA system in operation 2 airline pilots (AC1, AC2) in contact with ATC, and 2 TCAS units (TCAS1, TCAS2) operated in TA/RA mode


AC1FL310

330AC2FL330=

STCAAC1

FL310330AC2

FL330=

STCA

STCA

AC2

AC1

FL320

CPP

FL330

STCA

AC2

AC1

FL320

CPP

FL330

Sequence of events

STCA triggered sufficiently far ahead (63s) of effective loss of separation and before (57s and ~60s) TCAS RAs onboard both aircraft. Controller’s instructions to both pilots between 42s and 57s after STCA had been triggered, in the horizontal direction, manoeuvred more or less rapidly but with no or little effect. Simultaneously with instructions, coordinated TCAS RAs triggered and followed onboard both aircraft with efficacy, the upper aircraft climbing excessively. Controller’s instructions compatible with TCAS RAs.

(continued on next page)



Diagram of interactions Alert

“Turn 40° Left”

“Turn 40° Right”

“Descend RA” “Adjust V/S” “Climb RA”

STCA ATC

AC1 AC2

TCAS 1 TCAS 2 Influencing factors

STCA:

simple en-route geometry

TCAS:

conflict geometry rather simple for TCAS, in operation in both aircraft

ATC:

controller’s knowledge of ATS procedures but possibly biased mental picture of the conflict

A/C1:

Urgency phraseology used by the controller

Likely positive traffic awareness (TCAS traffic display +TA + visual)

Possible stress due to difficult communications with ATC, and to the RA itself

A/C2:

Possible pilot misunderstanding due to low quality of controller R/T (quick diction, necessity to repeat the message)

Pilot possibly preferred to execute TCAS manoeuvre rather than ATC instruction (e.g. SOP, flight envelope)

Stress induced by the TCAS RA and simultaneous ATC avoiding instruction (to turn)

Pilot’s knowledge of TCAS procedures

Efficacy of RA follow-up by pilots:

coordinated RAs were received and followed in both aircraft. The upper aircraft climbed by about one thousand feet, although 200ft would have been sufficient



*** END OF DOCUMENT ***

Analysis of SNET performance based on monitoring data · Analysis of SNET performance based on...

Documents

Transcript of Analysis of SNET performance based on monitoring data · Analysis of SNET performance based on...