22 Asmgcs Final Results Conclusions

8/8/2019 22 Asmgcs Final Results Conclusions

1/189

EUROPEAN ORGANISATIONFOR THE SAFETY OF AIR NAVIGATION

EUROCONTROL

EUROPEAN AIR TRAFFIC MANAGEMENT PROGRAMME

A-SMGCS VIS2 - VIS3Transition Simulation Report

Edition Number : 1.0Edition Date : 31 January 2005Status : Released IssueIntended for : General Public


2/189


Page ii Released Issue Edition Number: 1.0

DOCUMENT CHARACTERISTICS

TITLE

A-SMGCS VIS2 - VIS3

Transition Simulation Report

EATMP Infocentre Reference:

Document Identifier Edition Number: 1.0

Edition Date: 31 January2005

Abstract

This document summarises the results from the A-SMGCS VIS2 VIS 3 transition simulation inToulouse, France

Keywords

Contact Person(s) Tel Unit

Paul ADAMSON +32 2 729 3161 DAP/APT

STATUS, AUDIENCE AND ACCESSIBILITY

Status Intended for Accessible via

Working Draft General Public Intranet Draft EATMP Stakeholders Extranet Proposed Issue Restricted Audience Internet (www.eurocontrol.int)

Released Issue Printed & electronic copies of the document can be obtained from

the EATMP Infocentre (see page iii)

ELECTRONIC SOURCE

Path: C:\Documents and Settings\dupwood\Desktop\P1 Docs On HBRUWA52A

Host System Software Size

Windows_NT Microsoft Word 10.0 6785 Kb


3/189


Edition Number: 1.0 Released Issue Page iii

EATMP InfocentreEUROCONTROL Headquarters96 Rue de la FuseB-1130 BRUSSELS

Tel: +32 (0)2 729 51 51Fax: +32 (0)2 729 99 84

E-mail: [email protected]

Open on 08:00 - 15:00 UTC from Monday to Thursday, incl.

DOCUMENT APPROVAL

The following table identifies all management authorities who have successively approvedthe present issue of this document.

AUTHORITY NAME AND SIGNATURE DATE

A-SMGCSProject Manager

Paul ADAMSON


4/189


Page iv Released Issue Edition Number: 1.0

DOCUMENT CHANGE RECORD

The following table records the complete history of the successive editions of the presentdocument.

EDITION

NUMBER

EDITION

DATE

INFOCENTRE

REFERENCEREASON FOR CHANGE

PAGES

AFFECTED

1.0 31/01/05 Documentation Creation All


5/189


Edition Number: 1.0 Released Issue Page v

CONTENTS

DOCUMENT CHARACTERISTICS ...........................................................................ii

DOCUMENT APPROVAL ........................................................................................iii

DOCUMENT CHANGE RECORD ............................................................................iv

EXECUTIVE SUMMARY...........................................................................................1

1. INTRODUCTION.................................................................................................51.1 Background .........................................................................................................................5

1.2 Scope of the document ........................................................................................................5

1.3 Methodology........................................................................................................................ 5

1.4 Structure of the document ....................................................................................................6

1.5 Reference documents..........................................................................................................6

1.6 Acronyms ............................................................................................................................7

1.7 Explanation of terms ............................................................................................................7

2. CONTEXT AND OBJECTIVES OF THE EXPERIMENT....................................102.1 Evaluation aims ................................................................................................................. 10

2.2 High-level objectives ..........................................................................................................12

2.3 Low-level objectives........................................................................................................... 12

2.4 Hypothesis, metrics and measures.....................................................................................12

3. EXPERIMENTAL DESIGN................................................................................14

3.1 Introduction........................................................................................................................ 14

3.2 Simulation platform............................................................................................................14

3.3 Facilities and Equipment....................................................................................................15

3.3.1 Facilities overview ......................................................................................................15

3.3.2 Pilots positions ...........................................................................................................16

3.3.3 Observer positions......................................................................................................16

3.3.4 Pseudo-controller position .......................................................................................... 16

3.3.5 Wizard of Oz Position.................................................................................................16

3.3.6 Supervision position ...................................................................................................16

3.4 Scenarios and Events ........................................................................................................17


6/189


Page vi Released Issue Edition Number: 1.0

3.5 Experimental plan ..............................................................................................................21

3.5.1 Organisation...............................................................................................................21

3.5.2 Session runs ..............................................................................................................22

3.5.3 Planning and timing of simulation session...................................................................223.5.4 Session participants ...................................................................................................23

3.6 Experimental variables.......................................................................................................24

3.6.1 Selected design.......................................................................................................... 24

3.6.2 Independent variables ................................................................................................24

3.6.3 Dependent variables................................................................................................... 25

3.6.4 Run Plans................................................................................................................... 25

3.7 Simulated environment limits.............................................................................................. 27

3.7.1 Visibility Calibration ....................................................................................................27

3.7.2 Pilot HMI .................................................................................................................... 27

3.7.3 Airport Realism........................................................................................................... 28

3.7.4 Pseudo-controller position .......................................................................................... 28

4. DATA COLLECTION, VERIFICATION AND ANALYSIS ..................................28

4.1 Data collection................................................................................................................... 28

4.1.1 Data measurable by automated system ......................................................................28

4.1.2 Data gathered through exercise observation...............................................................29

4.1.3 Data obtained from participants .................................................................................. 29

4.2 Data verification................................................................................................................. 30

4.3 Data analysis.....................................................................................................................30

4.3.1 Qualitative analysis method........................................................................................ 30

4.3.2 Quantitative analysis method......................................................................................30

5. SYNTHESIS OF RESULTS...............................................................................31

5.1 Introduction........................................................................................................................ 31

5.2 Safety ................................................................................................................................31

5.2.1 See and avoid when visibility condition change (HLO1-LL1)........................................ 31

5.2.2 Visibility threshold assessment from crews perspective (HLO1-LL2)........................... 32

5.3 Feasibility / Efficiency.........................................................................................................36

5.3.1 The traffic information delivery enhances pilot ability to see traffic when thevisibility changes (HLO2-LL1) ..................................................................................... 36


7/189


Edition Number: 1.0 Released Issue Page vii

5.3.2 The traffic information delivery enhances pilot ability to avoid traffic when thevisibility changes (HLO2-LL2) ..................................................................................... 37

5.3.3 When traffic information is missing (HLO2-LL2) ..........................................................38

5.4 Summary of results ............................................................................................................39

6. DETAILED RESULTS.......................................................................................42

6.1 Safety ................................................................................................................................42

6.1.1 Visibility condition perceived by crews ........................................................................42

6.1.2 See traffic : Intruder detection and visibility condition ................................................ 44

6.1.3 Avoid traffic ...........................................................................................................46

6.2 Feasibility / Efficiency.........................................................................................................47

6.2.1 Traffic information success rate VS event detection .................................................... 47

6.2.2 Detection distance VS traffic information delivery........................................................ 48

6.2.3 Safety distance VS traffic information delivery............................................................. 48

6.2.4 Traffic information quality as assessed by pilots VS visibility conditions....................... 48

6.2.5 Stress and traffic information quality ...........................................................................49

7. CONCLUSIONS................................................................................................50

ANNEX A: SCENARIOS MAPS ..............................................................................51

ANNEX B: SCENARIO SCRIPTS ...........................................................................55

ANNEX C: SCENARIO LOGS.................................................................................68

ANNEX D: FLIGHT BRIEFING..............................................................................108

ANNEX E: PRE-REGISTERED MOBILES TRAJECTORIES................................156

ANNEX F: QUESTIONNARY AND OBSERVATION SUPPORT...........................163

ANNEX G: SCHEMATIC PRESENTATION OF THE POINTS OBTAINEDTHROUGH THE SIMULATION .......................................................................174


8/189


Page viii Released Issue Edition Number: 1.0


9/189


Edition Number: 1.0 Released Issue Page 1

EXECUTIVE SUMMARY

With regards to the aerodrome traffic, the ATC provides to users:

1) Separations on the runway.

2) Traffic information over the aerodrome circuit and the manoeuvring area (an aerodromecontroller does not ensure separations on the manoeuvring area).

3) Clearances and, in addition to traffic information, any information aim at avoiding collisionand at ensuring a safe, ordered and fast routing for the air traffic.

Whenever visibility conditions are as degraded as the controller could no more exercise control overall traffic on the basis of the visual surveillance (i.e., from visibility condition 2 onwards), his externalvisual vision is supposed to be replaced by an external electronic vision provided by an A-SMGCSLevel 1 display.

Up to visibility condition 2, visibility is sufficient for pilots to taxi and to avoid collision with other trafficon taxiways and at intersections by visual reference. Then, in visibility conditions 1 and 2, the A-SMGCS Level 1 display, can still be used by controllers while maintaining current pilots andcontrollers respective responsibilities on the manoeuvring area. That is to say that up to visibilitycondition 2, pilots when receiving traffic information, with an associated clearance can still assumean interval estimated as sufficient with another mobile in order to maintain the aircraft safety.

That would no more be the case while transiting from visibility condition 2 (VIS2) to visibility condition3 (VIS3), because in that case, pilots could no more avoid other traffic and then - even when receivingan traffic information, with an associated clearance pilots could no more maintain aircraft safety.

From the moment ATC can no more visually monitored the manoeuvring area from the tower (in orderto maintain aircraft safety), the current Low Visibility Operations (LVO) forces to come back toprocedural control

1. It is the aim of A-SMGCS level I and II to replace the controller inability to visually

monitor by an ability to use a dedicated surveillance display, until the point (VIS 3) when pilots can nomore see and avoid.

Provided that presently, the visibility threshold at the VIS2-VIS3 transition is not completely defined,the present validation aimed at assessing from pilots perspective, the real minimum visibility at whicha pilot can still safely ensure its operation on runways and taxiways. With current LVO rules thismeans assessing the visibility threshold before transiting to procedural control, assuming that A-SMGCS level I and II release the need of procedural control in VIS2.Knowing that a controller only knows the Runway Visual Range (RVR), and not the real visibility onthe manoeuvring area, the present validation aimed at providing preliminary elements to assess themargin a controller has at his disposal in case a traffic information would be missing.

Finally, the validation aimed at evaluating the efficiency of traffic information under low visibilityconditions. Subsidiary, it aimed at assessing the visibility threshold below which the traffic informationdelivery becomes ineffective, because upon receiving the traffic information, pilots can no more seethe announced mobile so as to give way to it.

Experiments were carried out with four different crews from three different commercial airlines (AirBourbon, Air France and Star Airlines). Totally 8 pilots participated to the real-time simulations: in-operation instructors, captains and first officers, all qualified on Airbus (A320, A330 or A340), rangingfrom 1.5 to 33 years of total number of flight years, all applying their own procedures and, sharing theirexperience in low visibility taxiing.

1In procedural control, the aerodrome is structured in blocks and each aerodrome block is used by one and only one aircraft.


10/189


Page 2 Released Issue Edition Number: 1.0

Each crew of two pilots was involved in one of the four experiment sessions that were held on 21rst,22

nd, 27

thand 28

thof October 2004 in Toulouse with EPOPEE, the research A340 simulator of

AIRBUS France. Each crew run four simulations exercises, each exercise simulating a particularvisibility condition (ranging 100 m, 200 m, 300 m or 400 m). It is worth noting that, in the context of

these simulations, visibility is measured from pilots eyes and is not RVR nor met visibility. Twosimulation exercises were run under day conditions and the other two under night conditions. Eachexercise lasted 20-25 minutes and simulated a taxiing phase during which the crew would haveencountered 4 mobiles that were in the vicinity of their aircraft, triggering a potential risk situationdepending on the visibility conditions. The mobiles were either aircraft or vehicles crossing left at 90the crew aircraft, or crossing it right at 45, or facing it, or being in front of it allowing a catching up.Simulated mobiles were either on movement or still. Generally, the controller announced the mobilesto the crew, using traffic information. However, for some events, the traffic information wasintentionally missing.

In night conditions, the simulator introduced a bias because of insufficient marking and airport lightsand because of poor brightness of aircraft and vehicles lights. However, with regards to reality, the

halo effects around highly enlighten airport areas (e.g., in the vicinity of parking or fret areas) tonedown contrasts (i.e. white headlights on white halo) providing contrasts equivalent to the simulatedones.

This means that, although simulated night conditions are to be considered as worst observableclimatic cases, they are representative of realistic situations.

The main conclusion of the study is that the minimum visibility thresholdbelow which pilots areunable to comply with ATC instructions based on traffic information requiring him to see and avoidtraffic is somewhere between 200 and 300 meters.

For detection performances, from visibility condition 300 meters, pilots start activating functions thatthey did not feel necessary at visibility condition 400 meters. Detection performance is better forvisibility 300 meters than for 400 meters. The Yerkes & Dodsons human factor phenomenon is thenobserved: performance increases with the level of activation.

In other words, pilots perceive the visibility condition 400 meters as normal visibility, and theyconsequently had nominal behaviours, while their detection functions start being mobilised fromvisibility 300 meters onwards. From visibility 200 meters onwards, despite the mobilisation, there is aphysical limit to detection performance (condition felt as a major risk for 25% of involved pilots,degraded detection performances, uncomfortable situations).

As preliminary elements for assessing the margin a controller has at his disposal in case a trafficinformation would be missing, nearly catastrophic situations occurred at real visibility 200m, whentraffic information is missing: a buffer of approximately 4 secondsto prevent a major incident.

Note however that although pilots submitted, for simulation purpose, to conditions that sometimes theywould not have accepted in the real life, no major incident has been observed during the experiment,even in cases where pilots did not detect other traffic due to the low visibility conditions.

In low visibility, without surveillance, ATC and pilots are both blind. It appears from this simulation thatpilots highly appreciate traffic information in low visibility. This point is a positive benefit of A-SMGCSsince ATC can then provide accurate traffic information.

Traffic information becomes less effective from visibility 300 meters and below, reaching itsefficiency limit at visibility 100 meters. Indeed, whereas visibility 300 meters is not critical yet, itwas observed that some pilots adopted a taxiing strategy (e.g., stopping, reducing speed) that maymake some traffic information ineffective. This was reinforced by the fact that, with regards to thesimulator, several elements of the airport representation were not completely representative of an


11/189



airport as defined in Annex 14. Particularly, the lack of stops bars and holding points on the groundincited pilots to stop their aircraft where it was as soon as the traffic information was received. Suchpilots taxiing strategy is not known by the controller since it is not a standard practice in real life and could lead to safety problem, especially in LVO, before transiting to procedural control (VIS3). In a

real environment, this taxiing strategy would not normally happen since the LVO with proceduralcontrol in 300m visibility would have been applied, forcing to taxi only one aircraft at the same time.

With regards to traffic information efficiency results, visibility 300 meters proves to be the minimumacceptable threshold at which pilots can safely maintain their aircraft.

A visibility threshold ranging between 300m and 400m by day and above 400m by night is consideredacceptable and comfortable by all crews involved in the simulations. However, at visibility condition300m, pilots are still able to comply safely with ATC instructions based on traffic information.

We would like to express our thanks to all actors of the experiments: all participating pilots from AirBourbon, Air France and Star Airlines; the EYTM Team from Airbus France for EPOPEE supervision;the Sofravia Team for project co-ordination, the validation aspects and pilots supervision; David Reitzfor his ATC expertise in the pseudo-controller role; and finally, Eurocontrol as the client and asobservers.


12/189




13/189



1. INTRODUCTION

1.1 Background

In defining new procedures for A-SMGCS, it is apparent that there will be no change to the respectiveresponsibilities of controllers, pilots and drivers. That is to say, during those conditions when pilots areable to see and avoid (i.e. during visibility conditions 1 & 2) they will continue to be primarilyresponsible for avoiding collisions, whilst complying with traffic information and clearances given byATC.

When visibility conditions prevent pilots & drivers from seeing other traffic in sufficient time to avoid acollision (i.e. during visibility conditions 3 & 4), responsibility for separation is transferred to the AirTraffic Controller, who applies procedural control (in the form of Low Visibility Procedures).

Following discussion within the A-SMGCS Procedures Drafting Group, it has been agreed that the

transition threshold from visibility condition 2 to visibility condition 3 should be the same as thethreshold used to commence Low Visibility Procedures or LVPs (normally 550m). It is also recognisedthat the only way that this visibility can be practicably measured is by Runway Visual Range (RVR).However, visibility measured at the runway may not be the minimum visibility on the manoeuvring area(i.e. lower visibility conditions may be present elsewhere). To allow for this possibility, there should bea reasonable assurance that pilots are able to see and avoid in visibility conditions below 550m (i.e.there should be a buffer to allow for the possibility of visibility conditions below 550m beingexperienced by pilots & drivers elsewhere on the manoeuvring area).

To determine the safe visibility threshold at which it is possible to see and avoid, it is believednecessary to assess the pilots perspective of the transition and to assess his ability to see and avoidin visibility conditions of 550m or less, at all times of the day.

1.2 Scope of the document

The Validation Plan (Interim (D1) & Final (D2)) specified the real-time simulations foreseen to validatethe threshold when the pilot can no longer see other traffic, at VIS2 VIS3 transition or at transition toLVP application. The LVP threshold is an ICAO decision not involved as such in our study.

Real-time simulations of visibility conditions transition from VIS2 to VIS3 were carried out asdeliverable D3.

The Interim results and conclusions (D4) depicted the context and objectives of the experiment, theexperimental design, collected data, its verification and analysis. The present document D5complements the D4 document with Final results and conclusions. It is to be made clear that takinginto account the size and duration of the project, the value that will be presented in conclusion is not aglobally agreed value in the ATM community. Nevertheless, it gives an idea about the value to be

reached (this will be further evaluated during operational trials).

1.3 Methodology

The Validation process relies on the [MAEVA] methodology which has been especially designed forthis kind of exercise by the Master ATM European Validation Plan (MAEVA) project. This project wassponsored by the European Commissions Directorate General for Energy and Transport within itsFifth Framework Programme (5th FP) for research and development.

MAEVA establishes a uniform framework for the validation of ATM concepts. This methodology ishelpful to provide guidelines along the entire validation process. This methodology allows to ask thegood questions related to validation and presents concrete examples of applications of the


14/189



methodology. Its step-by-step approach helps the validation team to address the validation activity inan exhaustive way.

In the MAEVAs Validation Guideline Handbook (VGH) [MAEVA], it is proposed a five-step process for

conducting validation exercises. These steps are as follows: Step 1: Define validation aims, objectives and hypotheses.

Step 2: Prepare the validation plan and exercise runs.

Step 3: Execute the exercise runs and take measurements.

Step 4: Analyse results.

Step 5: Develop and disseminate conclusions.

The results and conclusions presented in this document are the result of the step 4 and 5 of the[MAEVA] methodology. These steps analyse the measurements taken during the exercise runs anddevelop conclusions and recommendations based on the analysis conducted and disseminate them ina readily understandable manner to the intended audience.

1.4 Structure of the document

Introduction

Chapter 1 describes the purpose of this document, its structure, the reference documents and givesan explanation of terms used throughout the document.

Context and objectives of the experiment

Chapter 2 recalls the validation aims, high-level and low-level objectives of the simulation, as well asderived hypothesis and measures.

Experimental design

Chapter 3 recalls the experimental conditions, scenarios, environment, variables, and limits.

Data collection, verification and analysis

Chapter 4 presents the data collected and analysed.

Synthesis of results

Chapter 5 presents the results with regards to simulation objectives.

Detailed results

Chapter 6 presents the detailed results.

Conclusion

Chapter 7 develops conclusions and recommendations based on the result analysis conducted.

Annexes

1.5 Reference documents

[ICAO-A-SMGCS] ICAO European Manual on Advanced Surface Movement Control and GuidanceSystems (A-SMGCS) AOPG, Final Draft, Nov 2001.

[ICAO-Annex14]

[EUROCAE-MASPS]

[MAEVA]

ICAO Annex 14, Volume I, Chapter 8.

EUROCAE WG-41, MASPS for A-SMGCS, Edition ED-87A, January 2001.

MAEVA Validation Guidelines Handbook.


15/189



1.6 Acronyms

ATC Air Traffic Control

ATCO Air Traffic Control Officer

A-SMGCS Advanced Surface Movement Guidance & Control System

ATM Air Traffic Management

CTOT Calculated Take Off Time

HMI Human Machine Interface

ICAO International Civil Aviation Organisation

LVO Low Visibility Operation

LVP Low Visibility Procedure

MAEVA Master ATM European VAlidation plan

PF Pilot Flying

PNF Pilot Not Flying

RVR Runway Visual Range

SMGCS Surface Movement Guidance and Control System

VISn Visibility condition n

1.7 Explanation of terms

This section provides the explanation of terms required for a correct understanding of the presentdocument. Most of the following explanations are drawn from the A-SMGCS manual [ICAO-A-

SMGCS], the ICAO Annex 14 [ICAO-Annex14] or the EUROCAE MASPS for A-SMGCS [EUROCAE-MASPS], in that case it is indicated in the definition. [ICAO-A-SMGCS] definitions are used as a firstoption. In general, other definitions are only used where there is no ICAO definition. If not, it isexplained why another definition is preferred to the ICAO one.

Advanced Surface Movement Guidance and Control Systems (A-SMGCS)

[ICAO-A-SMGCS] definition

Systems providing routing, guidance, surveillance and control to aircraft and affected vehicles in orderto maintain movement rates under all local weather conditions within the Aerodrome VisibilityOperational Level (AVOL) whilst maintaining the required level of safety.

Aerodrome

[ICAO-Annex14] and [ICAO-A-SMGCS] definition

A defined area on land or water (including any buildings, installations, and equipment) intended to beused either wholly or in part for arrival, departure and surface movement of aircraft.

Aerodrome movement

[ICAO-A-SMGCS] definition addresses only aircraft movement, we extended the definition to allmobiles.


16/189



The movement of a mobile (aircraft or vehicle) on the movement area.

Aerodrome Visibility Operational Level (AVOL)


The minimum visibility at or above which the declared movement rate can be sustained.

Apron


A defined area on a land aerodrome, intended to accommodate aircraft for purposes of loading orunloading passengers, mail or cargo, fuelling, parking or maintenance.

Conflict


A situation when there is a possibility of a collision between aircraft and/or vehicles.

Control


Application of measures to prevent collisions, runway incursions and to ensure safe, expeditious andefficient movement.

45

In the present document, the use of the term 45 degrees in sentences that define a scenario eventtype (e.g. Mobile crossing right 45) corresponds to 45 degree abaft the beam (nautical term).

EPOPEE

Research flight simulator of AIRBUS France, usually dedicated to studies of cockpit design andhuman/machine interfaces, with pilots involved early in the loop. For the present VIS2-VIS3 transitionsimulation, EPOPEE was used as a real-time simulator with pilots in the loop, Its A340 cockpitconfiguration was coupled to an external visual system with a large view size, so as to enable thesimulation of different visibility conditions.

Intruder

Any mobile which is detected in a specific airport area into which it is not allowed to enter.

Manoeuvring area


That part of an aerodrome to be used for the take-off, landing and taxiing of aircraft, excluding aprons.

Mobile

A mobile is either an aircraft or a vehicle.


17/189



Movement area


That part of an aerodrome to be used for the take-off, landing and taxiing of aircraft, consisting of themanoeuvring area and apron(s).

Normal Visibility

Visibility conditions sufficient for personnel of control units to exercise control over all traffic on thebasis of visual surveillance (correspond to visibility condition 1 defined by ICAO [ICAO-A-SMGCS]).

Reduced Visibility

Visibility conditions insufficient for personnel of control units to exercise control over all traffic on thebasis of visual surveillance (correspond to visibility conditions 2, 3, and 4 defined by ICAO [ICAO-A-

SMGCS]).

Traffic information

Information about the surrounding traffic given specifically to the crew.

(In the context of these simulations, each time EP123 appears in the scenario logs

in annex C of the present document).

Visibility condition 1

[ICAO-A-SMGCS] and [ASMGCS operating procedures, ed 1.0, 25/02/04]

Visibility sufficient for the pilot to taxi and to avoid collision with other traffic on taxiways and atintersections by visual reference, and for personnel of control units to exercise control over all traffic ofvisual surveillance.



Visibility sufficient for the pilot to taxi and to avoid collision with other traffic on taxiways and atintersections by visual reference, but insufficient for personnel of control units to exercise control overall traffic on the basis of visual surveillance.



Visibility sufficient for the pilot to taxi but insufficient for the pilot to avoid collision with other traffic ontaxiways and at intersections by visual reference, and insufficient for personnel of control units toexercise control over all traffic on the basis of visual surveillance. For taxiing this is normally taken asvisibilitys equivalent to a RVR of less than 400m but more than 75m.


18/189



2. CONTEXT AND OBJECTIVES OF THE EXPERIMENT

2.1 Evaluation aims

Whenever visibility conditions are as degraded as the controller could no more exercise control overall traffic on the basis of the visual surveillance (i.e., from visibility condition 2 onwards), his externalvisual vision is supposed to be replaced by an external electronic vision provided by an A-SMGCSLevel 1 display.

From the TWR, ATCO:

- Does not see outside

- Relies on an electronic

external vision

(A-SMGCS display)

Up to visibility condition 2, visibility is sufficient for the pilot to taxi and to avoid collision with othertraffic on taxiways and at intersections by visual reference. Then, in visibility conditions 1 and 2, the A-SMGCS Level 1 display, can still be used by controllers while maintaining present responsibilitysharing between pilots and controllers on the manoeuvring area. That is to say, while a pilot whenreceiving traffic information with or without a clearance can assume an interval estimated assufficient with another mobile in order to maintain the aircraft safety.

That would no longer be the case while transiting from visibility condition 2 to visibility condition 3,because in that case, pilots can no longer avoid other traffic and then - even when receiving


19/189



information, with or when receiving traffic without a clearance pilots can no longer maintain aircraftsafety.

For Pilots, Visibility is:

- still sufficient to taxi

- but at the border

between sufficient

(VIS2) and insufficient

(VIS3) to avoid other

traffic

Presently, the visibility threshold is not yet completely defined while crossing the border between avisibility condition sufficient for pilots to taxi and to avoid other traffic (VIS2) to a visibility conditioninsufficient for pilots to avoid other traffic (VIS3).

Regulatory sources define several different measured visibility threshold for the transition between

those two visibilitys (that correspond to LVP level) from RVR 400 m up to 550 m.Provided that the RVR may not be the minimum visibility on the manoeuvring area so called realminimum visibility -, there is a need to assess the pilot perspective of transition from visibility 2 tovisibility 3 through the assessment of the minimum real visibility.

Then, the validation aim for this simulation is to assess from pilots perspective, the minimumreal visibility measured from pilots eyes, at which a pilot can safely ensure his operations onrunways and taxiways.

A subsidiary question that would be answered by this experimentation is what is be room ofmanoeuvre available to pilots to see and avoid other traffic when visibility conditions are low. In otherterms, what buffer - in time or in distance is left to pilots before a major incident occurs. From thecontrollers perspective, the subsidiary related question that would be indirectly answered would be:what is visibility threshold below which traffic information delivery becomes inefficient.


20/189



However, it should be make it clear that in any case, the simulation did not intend to assess the risk ofcollision on low visibility. In other words, the fact that another traffic is not seen does not imply thatthere is a risk of collision.

The purpose of the following section is to convert the validation aims, into high-level and low-levelobjectives that can be measured in the real-time simulation exercises.

2.2 High-level objectives

The high-level objectives (HLO) can be grouped in two categories:

Safety

Feasibility / Efficiency

During the real-time simulation, the following high level objective has been checked for safety andFeasibility/Efficiency purpose:

- HLO1: assess the pilot perception of the minimum distance needed by a crew to receive,

interpret and assume the information given on the traffic, while following all the requiredprocedures and visually checking external conditions.

During the real-time simulation, the following high level objective has been checked forfeasibility/efficiency purpose:

- HLO2: Assess the success rate of traffic information (traffic information is consideredsuccessful, when after receiving a traffic information, the pilot can see the announcedmobile and give way to it).

2.3 Low-level objectives

Each high-level objective that is addressed in the real-time simulation is broken down into low-levelobjectives.

The low-level objectives associated to HLO1 are:

HLO1-LL1: To assess pilot ability to comply with ATC instructions based on trafficinformation requiring him to see and avoid traffic when the visibility conditionschange.

HLO1-LL2: To assess a visibility threshold below which the previous activities (inLL1) are no longer possible.

The low-level objectives associated to HLO2 are:

HLO2-LL1: To assess pilot ability, when traffic information is given, to see and avoidtraffic when the visibility conditions change.

HLO2-LL2: To assess pilot ability, when traffic information is missing, to see andavoid traffic when the visibility conditions change.

2.4 Hypothesis, metrics and measures

With regards to objectives, two quantitative measures are of interest:

The major measurement objective is Ddetection, the distance between EPOPEEand the mobile when the pilots of EPOPEE detect the latter.

A subsidiary measurement is Dsafety, the minimum distance between EPOPEEand the mobile during the event.


21/189



The following diagram illustrates these quantitative measures:

GMOBILE (T1)

GEPOPEE (T1)

GMOBILE (T0)

GEPOPEE

T0: Time at which the mobile trajectory is triggeredT1: Time at which the EPOPEEs pilots detect the mobile

DDetection: Minimum distance between

EPOPEE and Mobile at T1

DSafety: Minimum distance between

EPOPEE and Mobile

GMOBILE

GEPOPEE (T0)

T0 T1DDetection

DSafety

Recordings: T1, EPOPEE Position (T1),

Speed (T1), Breaks, ATC msg..

Recordings: T0, EPOPEE Position(T0), Speed(T0)

With regards to objectives, validation hypotheses and measures were as follow:

Objectives(low-level Objectives) Hypothesis

Safety(HLO1-LL1)

Below 200 meters of visibility, the pilot is unable to see andavoid other traffic.

Questionnaire / Debriefings.

Measure ofDdetection, the distance between EPOPEE andthe mobile when the pilot detects the latter.

Measure ofDsafety, the minimum distance betweenEPOPEE and the mobile during the event.

The relative configuration of a mobile with regards aircraftand the illumination conditions impacts the pilots ability tosee traffic (i.e., detect the mobile) and to avoid traffic (i.e., tokeep reasonable minimum distance between aircraft andmobile).


Ddetection, the distance between EPOPEE and the mobilewhen the pilot detects the latter.


22/189




Safety(HLO1-LL2)

The visibility threshold below which the pilot is unable toperform reliably any of the previous activities is 200 meters.



Dsafety, the minimum distance between EPOPEE and themobile during the event.

Feasibility / Efficiency(HOL2-LL1)

The traffic information delivery enhances pilot ability to seetraffic when the visibility changes.



Rate of successful detection of announced trafficwhenthe traffic information is given.

The traffic information delivery enhances pilot ability to avoidtraffic when the visibility changes.



Table 2-1: validation hypothesis and measures / validation objectives

3. EXPERIMENTAL DESIGN

3.1 Introduction

The validation objectives have been addressed by simulating situations where a pilot can see anothermobile that is a potential source of conflict while its aircraft is taxiing out in an airport. Those situationsare simulated at different visibility levels, for comparison purpose.

The airport environment used for the simulation is based on Toulouse Blagnac airport. However, itsrepresentation has some limitation of accuracy. Then, with regards to pilots involved in theexperiment, it has been presented as a new one, called EUROSOF airport, so as not to surprisethem by unusual path for Toulouse Airport.

The choice of this airport has been motivated by the fact that, despite its limitations, it had the bestrepresentation accuracy.

3.2 Simulation platform

EPOPEE is the research experimental flight simulator of AIRBUS France. It is dedicated to studies ofcockpit design and human and machine interfaces, with pilots involved early in the loop.

The basic configuration of the simulator is A340-300 configuration, with engine CFM56-5C4. Thissimulator is "fully simulated" and does not include any real calculator (all the systems are simulated).

The cockpit configuration is "A340-300" like, but is not completely representative of a real aircraftcockpit. The main equipment necessary for piloting the aircraft are available: FCU, side-sticks,


23/189



throttles, nose wheel steering, slats/flaps commands, rudder pedals. Other equipment is not fullyphysically representative: radio management panel, overhead panel

The configuration of the displays in the cabin is A340 configuration, even Pilot Flying Display (PFD)

and Navigation Display (ND) screens are 6"*8" size. The displays are EIS2 A340 displays.The simulator is coupled to an external visual system with a large size of view (220 * 45). The visualintegrates a ground database with a large area of Southwest of France, and main airports in the world.For the specific purpose of this experimentation, Toulouse Blagnac airport representation was used.

A sound-restitution system can be launched, in order to have aircrafts sounds and improve therepresentation of the simulation: engines, aerodynamic sounds, landing gear.

3.3 Facilities and Equipment

3.3.1 Facilities overview

Physically, EPOPEE is constituted of two adjacent rooms:- A monitoring room from which the simulator can be technically operated and supervised.

- The cockpit with its external view.

In the monitoring room, facilities encompass:

- 5 monitoring screens representing the cockpit external view that are:

- Available for DAY CONDITIONS ONLY.

- UNUSEABLE for NIGHT conditions.

- Monitoring screens displaying aircraft screens.

- Possibility to listen what is happening inside the cockpit from the monitoring room.

- Possibility to display what cameras are filming inside the cockpit.

The following figure illustrates facilities and positions used in the experiments and located in themonitoring room:

Monitoring screen for external view

Left view Front view Right view

ATCposition

Wizard

of OZposition

Super-visor

position

Cameradisplay

Audio

recorder

Data

recorder

Pilot displays

Figure 3-1: Monitoring Room Facilities and Positions


24/189



3.3.2 Pilots positions

The two pilot positions for both PF and PNF are located in the cockpit. They dispose of EPOPEEcockpit configuration as described in section 3.2.

Pilots performed ATC communications using a hand microphone.

3.3.3 Observer positions

In the cockpit, behind pilots seats, ordinary seats were installed for observers: one for the pilotsupervisor (SOF), one for the human factor expert. Some other seats were installed also for otherobservers (e.g. Eurocontrol).

In order not to disturb pilots, especially under night conditions, observers wore front lights to facilitatetheir notes taking.

3.3.4 Pseudo-controller position

One pseudo-controller position was present to provide ATC instruction and traffic information. He hadto provide the necessary ATC instruction and traffic information to EPOPEE pilots as well as to othermobiles (simulated ones).

The pseudo-controller position was located in the monitoring room. It is only a microphone, allowingthe pseudo-controller to communicate with pilots involved in the experiment and located in the cockpit.

So as to simulate controller communications with other aircraft, messages from virtual pilots werepre-registered on a laptop (See Play Lists in Annex). Pre-registered virtual pilot voices wereamplified using speakers placed close to the microphone.

The pseudo-controller communicated in life with pilots involved in the experiment and virtual pilots,triggering virtual pilot messages as necessary.

The pseudo-controller used paper support to follow the traffic (airport map and traffic). For day

scenarios, he used the monitoring screen to follow EPOPEE position, For night scenarios, herequested the Wizard of Oz support to know EPOPEE position. He relied on the Wizard of Oz supportto know whether the event planned in the scripts have been launched when he can start deliveringtraffic information.

3.3.5 Wizard of Oz Position

The Wizard of Oz (SOF) was in charge of supervising the simulation, ensuring that - through eventsynchronisation and visibility changes management each scenario script was properly run.

The Wizard of Oz synchronised mobile triggering with EPOPEE position using EPOPEE real-timelatitude and longitude displayed on the supervision position. This was especially necessary for nightscenarios where no external view is available in the monitoring room.

He assisted the human factor expert (SOF), in pilots observation so as to gather relevant qualitativedata from the real-time simulation and to follow-up the experiments.

He assisted the pseudo-controller for EPOPEE position.

The Wizard of Oz Position was located in the monitoring room, with the pseudo-controller position juston his left and the supervision position just on his right.

3.3.6 Supervision position

For overall technical aspects, an AIRBUS computer engineer was necessary to supervise and tooperate the simulator. In particular, this platform supervisor was in charge of varying the visibility leveland to launch other mobiles trajectories upon Wizard of Oz instruction.


25/189



The supervisor Position was located in the monitoring room.

3.4 Scenarios and Events

Eight different scenarios corresponding to different visibility conditions were described (e.g. day nightand visibility of 100, 200, 300 and 400 meters).

Visibility

Illumination

400 meters 300 meters 200 meters 100 meters

DAY Scenario A Scenario B Scenario C Scenario D

NIGHT Scenario E Scenario F Scenario G Scenario H

Each scenario included a set of 4 events, which had to take place during an exercise run, and lasted

approximately 20-25 minutes. These events represented mobiles that were in the vicinity of thesimulated aircraft (i.e, EPOPEE) triggering a potential risk situation depending on the visibilityconditions. Each event was described with the following set of variables:

Scenario event variables

Name Description Values

Event ID Event identification in a scenario. From event 1 to event 32.

Event type Relative configuration of a mobilewith regards to the simulatorsimulated aircraft (i.e., EPOPEE).

- Mobile crossing left 90

- Mobile crossing right 45abaft the beam

- Mobile face-to-face

- Mobile to catch up

Mobile type Type of mobile. - Aircraft

- Vehicle

Mobilemovement

Whether the mobile is moving orstopped.

- Moving

- Stopped

Mobiletrajectory

Taxi route followed by the mobile onthe surface manoeuvring area.

Many (see Annex A: scenario maps).

Mobile Speed Uniform speed of the mobile while

taxiing according to the mobiletrajectory.

- 0 kts for stopped mobiles

- 5 kts for mobile to catch up

- 12 kts otherwise

TrafficinformationDelivery

Whether the traffic information isdelivered or not.

- Given

- Missing

So as to implement these scenarios and events, the following data set was produced (See Annexes):

- Routes to be followed by the pilots using EPOPEE (See Annex A: scenario maps).

- Trajectories of the mobiles related to specific events and simulated to be in potential conflictwith EPOPEE.


26/189



- Script to be followed by the pseudo-controller.

Apart from mobiles simulated to be in potential conflict with EPOPEE, and fixed aircraft on the apron,no other traffic is simulated.

The mobiles, simulated to be in potential conflict with EPOPEE, are traffic for which the trajectory hasbeen pre-registered (including path, speed and traffic type aircraft or vehicle). Consequently, theyhave to be launched at the appropriate time to have the desired conflict (catching up, crossing or faceto face) - the Wizard of Oz ensured this.

Each scenario was run for 2 pilot teams, varying parameters for some events depending on the team(the detailed parameters for each crew are given in annexes). The following table summarises thescenarios with their detailed events.


27/189

A-SMGCSVIS2-VIS3

Transitio

nSimulationReport

E

ditionNumber:1.0

ReleasedIssue

Page19

Sce

nario

Day/Night

Condition

Threshold

Visibility

Events

Eventtype

MobileType

MobileMovementMobile

speed

Trafficinfodelivery

Sc

enarioA

day

400m

Event1

Mobilecrossingright45Aircraft

Moving

12knots

Given

400m

Event2

Mobilecrossingleft

90

Aircraft

Moving

12knots

Missing

orgiven

400m

Event3

Mobilefacetoface

Vehicleoraircraft

Stoppe

d

-

Given

400m

Event4

Mobilecatchingup

Aircraft

Moving

5knots

Given

Sc

enarioB

day

300m

Event5

Mobilecrossingleft

90

Vehicleoraircraft

Moving

12knots

Given

300m

Event6

Mobilefacetoface

Aircraft

Stoppe

d

-

Given

300m

Event7

Mobilecatchingup

Vehicleoraircraft

Stoppe

d

-

Given

300m

Event8


Moving

12knots

Missing

orgiven

Sc

enarioC

day

200m

Event9


Moving

12knots

Given

200m

Event10

Mobilecrossingleft

90

Aircraft

Moving

12knots

Missing

orgiven

200m

Event11

Mobilefacetoface

Vehicleoraircraft

Stoppe

d

-

Given

200m

Event12

Mobilecatchingup

Aircraft

Moving

5knots

Given

Sc

enarioD

day

100m

Event13

Mobilecrossingleft

90

Vehicleoraircraft

Moving

12knots

Given

100m

Event14

Mobilefacetoface

Aircraft

Stoppe

d

-

Given

100m

Event15

Mobilecatchingup

Vehicleoraircraft

Stoppe

d

-

Given

100m

Event16


Moving

12knots

Missing

orgiven

Sc

enarioE

night

400m

Event17


Moving

12knots

Given

400m

Event18

Mobilecrossingleft

90

Vehicleoraircraft

Moving

12knots

Given

400m

Event19

Mobilecatchingup

Aircraft

Stoppe

d

-

Given

400m

Event20

Mobilefacetoface

Aircraft

Stoppe

d

-

Missing

orgiven

Sc

enarioF

night

300m

Event21

Mobilecrossingleft

90

Aircraft

Moving

12knots

Given


28/189

A-SMGCSVIS2-VIS3

Transitio

nSimulationReport

P

age20

ReleasedIssue

Edition

Number:1.0

Sce

nario

Day/Night

Condition

Threshold

Visibility

Events

Eventtype

MobileType

MobileMovementMobile

speed

Trafficinfodelivery

300m

Event22

Mobilefacetoface

Vehicleoraircraft

Stoppe

d

-

Given

300m

Event23

Mobilecatchingup

Aircraft

Moving

5knots

Missing

orgiven

300m

Event24


Moving

12knots

Given

Sc

enarioG

night

200m

Event25


Moving

12knots

Given

200m

Event26

Mobilecrossingleft

90

Vehicleoraircraft

Moving

12knots

Given

200m

Event27

Mobilecatchingup

Aircraft

Stoppe

d

-

Given

200m

Event28

Mobilefacetoface

Aircraft

Stoppe

d

-

Missing

orgiven

Sc

enarioH

night

100m

Event29

Mobilecrossingleft

90

Aircraft

Moving

12knots

Given

100m

Event30

Mobilefacetoface

Vehicleoraircraft

Stoppe

d

-

Given

100m

Event31

Mobilecatchingup

Aircraft

Moving

5knots

Missing

orgiven

100m

Event32


Moving

12knots

Given

Table3-2:

scenarioandevent


29/189



3.5 Experimental plan

3.5.1 Organisation

The experimental plan included a pre-experimental phase and an experimental phase.

Pre-experimentation Experimentation

Nb of simulationsessions

1 session. 4 sessions, each one with adifferent pilot crew.

Session duration 2 days. Half days.

Session location Toulouse in A340 simulator(EPOPEE) on the Airbus site.

Toulouse in A340 simulator(EPOPEE) on the Airbus site.

Session dates 7-8/10/2004 Session 1: 21/10/2004

Session 2: 22/10/2004Session 3: 27/10/2004Session 4: 28/10/2004

Scenarios run persession

8 scenarios (from A to H). 4 scenarios run per session,comprising 2 day-scenarios and2 night-scenarios.Session 1: A, F, G, DSession 2: B, C, E, HSession 3: A, G, D, FSession 4: B, H, C, EEach exercise had a 25 minutesplanned duration that wasmodulated depending onexperiment hazards.

Team per session 1 Wizard of Oz (SOF),

1 ATC expert (provided by SOF),that is an operational towercontroller of the simulated platform,

1 Human Factor support (SOF)responsible for validatingqualitative material prepared forthe experimentation,

2 pilots (1 Eurocontrol + 1 SOF),the SOF pilot preparing hissupervision role for theexperimentation,

2 Computer Engineers (Airbus)responsible for platformmaintenance,

2 Eurocontrol observers.

1 Wizard of Oz (SOF),

1 ATC expert (provided bySOF), that is an operationaltower controller of thesimulated platform,

1 human factor expert(SOF), who observed pilotsand conducted shortdebriefings,

1 pilot supervisor (SOF),who cumulates humanfactor skills and whoparticipated to pilotobservations,

2 airline pilots (provided bySOF),

2 Computer Engineers(Airbus) responsible forplatform maintenance,

2 Eurocontrol observers.


30/189



3.5.2 Session runs

During the experimentation phase, each session run as described hereafter.

3.5.2.1 Briefing

Before the beginning of each simulation session, a plenary session gathering all simulation attendeeswas organised. This plenary session included a briefing to present:

- the experimental team,

- the project,

- the organisation of the day,

- the assessment methodology (e.g., the aim of this project is: the identification of taxiingbehaviour in variable conditions of visibility),

- an overview of EPOPEE.

3.5.2.2 EPOPEE Training

During the plenary session, the pilot supervisor introduced the difference between EPOPEE and a realA340 cockpit in order to avoid breaks or interruptions at the time of exercises.

Inside the cockpit, the pilot supervisor was in charge of pilots training for the specific aspects related tothe simulator, and for pilots briefings and debriefings before and after each simulation exercise.

3.5.2.3 Exercise runs

Before each exercise run, the same instructions were given to all the crews. These instructionsconcerned the taxiing objective and recalled the need and utility of spontaneous verbalisation.

From pilots perspective, this allowed them to feel confident with the presence of audio recording.From the observers perspective, the aim was:

- To collect data about pilots perception related to:

- stress,

- visibility condition,

- simulation realism,

- simulated events, and more particularly, the moment when pilots detect other traffic.

- To identify respective roles between PF and PNF when visibility conditions change.

During each exercise run, the pilot supervisor and the human factor expert ensured a role of observersin order to collect data significant for assessment (problems, difficulties or easiness, informal dialogue,communication strategies, actions, etc.). They also collected every comment and suggestion coming

from the controller.

After each exercise, a short debriefing was organised. A questionnaire was filled out withcrewmembers and their comments on the exercise were registered.

3.5.2.4 Debriefing

As a plenary session gathering all simulation attendees, a final debriefing took place with each crew atthe very end of the session in order to collect subjective data about different thematic.

3.5.3 Planning and timing of simulation session

Planning dates have been established as follows.


31/189



Day 1-Crew 1

21/10/04

Day 2- crew 2

22/10/04

Day 3-crew 3

27/10/04

Day 4- crew 4

28/10/04

13:30 Project

presentationproject, roles,constraints

08:30 Project


13:30 Project


08:30 Project


14:15 Epopee Training 09:15 Epopee Training 14:15 EpopeeTraining

09:15 Epopee Training

14:40 Instructions forthe session




14:45 Exercise 1 09:45 Exercise 1 14:45 Exercise 1 09:45 Exercise 1

15:10 Short debriefing 10:10 Short debriefing 15:10 Short debriefing 10:10 Short debriefing



15:55 Break 10:55 Break 15:55 Break 10:55 Break




17:00 Short Debriefing 12:05 Short Debriefing 17:00 Short Debriefing 12:05 Short Debriefing

17:10 Break 12:15 Break 17:10 Break 12:15 Break

17:20 Long debriefing 12:25 Long debriefing 17:20 Long debriefing 12:25 Long debriefing

18:30 End of Day 13:35 End of Day 18:30 End of Day 13:35 End of Day

Table 3-3: Planning and timing of simulation session

3.5.4 Session participants

Four pilot teams have been involved for the real-time simulations. Four experimentation sessions tookplace, over half-day for each one. Two pilots were involved per day of simulation, and 4 exerciseswere carried out for each pilot team, in different conditions.

Four crews were invited to take part in this experimentation, that is to say 8 pilots coming from thefollowing airlines:

2 Air Bourbon crews,

1 Air France crew,

1 Star Airlines crew.

All the pilots were qualified on Airbus (320, 330 or 340) and were used to perform medium or long haulflights. The average number of flight years on Airbus for all the participants was 2.15 years (min 0.7years; max 5 years), and the average total number of flight years was 15.81 years (min 1.5 years; max33 years).

This sample is considered as rather representative in this experimentation because of:

Different typologies according to the airlines to which they belong: the big and small airlinesdo not have the same procedures in low visibility conditions and the same behaviours in thecockpit.

The presence of qualified young people having few years of flight experience, as well aspilots having many flight years and officiating sometimes as instructors.


32/189



3.6 Experimental variables

3.6.1 Selected design

The experiment design consisted in comparing different visibility thresholds for a given group of flightcrews. There were four groups of flight crews from different aircraft companies. Each group tested thesame set of exercises but in different visibility conditions.

3.6.2 Independent variables

The independent variables are factors operated by the experimenter and that are supposed to giverise to expected effects. They are described and argued in the table below.

Variables Description Arguments

Vdn Variable Day/Night =variation of the illuminationconditions of the simulatorenvironment.

This variable has two levels :

Day,

Night.

Vvis Variable Visibility = variationof visibility threshold on theEPOPEE external viewdisplay.

This variable has four levels:

400 meters,

300 meters,

200 meters,

100 meters.

Vet Variable event type = rotationof the different events met byEPOPEE during one

exercise.

There are four event types met by EPOPEE :

A/C crossing 90,

A/C crossing 45,

A/C face to face,

A/C catching up.

Each event type corresponds to the meeting ofEPOPEE with a mobile under a particular condition.

Vit Variable mobiles type =variation of mobiles size metby EPOPEE during anexercise.

There are two type of size mobiles:

Aircraft (Boeing),

Vehicle.

Vim Variable mobile movement =

variation of the mobile speedfor one type event.

This variable has two conditions:

Stopped,

Moving.

For the face-to-face event type, the conditionMoving is not used.

Vtd Variable traffic info delivery =variation of an informationpresence given by the ATCofor one event type.

This variable has two conditions:

Given (ATCo delivers correct informationconcerning the mobile movement).

Missing (ATCo doesnt deliver any informationconcerning the mobile movement).


33/189



3.6.3 Dependent variables

The dependent variables are made up of the measured parameters that enable assessing the effectsof the visibility threshold deterioration on moving object (vehicle or aircraft) perception for the PF and

the PNF.This section presents the methods used to collect data during simulation exercises.

According to the MAEVA methodology, the data can be collected according the following methods: bythe simulator automated means, by humans or by a combination of both.

Moreover, in the scope of the Simulation Validation Plan, data collection is broken down further intothe following categories: data measurable by automated system, data gathered through observation ofthe exercise and opinion provided by the participants (those categories are presented in part 4.1).

At last, a distinction can be done between all the data collected. Indeed, the data collected can beobjective or subjective. The data collected using an objective data collection method is not affected bythe opinion or viewpoint of the person providing or recording the information, whereas the datacollected using a subjective way is influenced by the person providing or recording the information.

3.6.4 Run Plans

This section describes for each crew the contents of each exercise.

Simulation variable details for Crew 1 are as follows:

Scenario IlluminationCondition

Threshold

Visibility

Events Event type MobileType

Mobile

Movement

Traffic info

delivery

Scenario A day 400m Event 1 Mobile crossing right 45 Aircraft Moving Given

400m Event 2 Mobile crossing left 90 Aircraft Moving Given

400m Event 3 Mobile face to face Vehicle Stopped Given

400m Event 4 Mobile catching up Aircraft Moving Given

Scenario F night 300m Event 21 Mobile crossing left 90 Aircraft Moving Given

300m Event 22 Mobile face to face Aircraft Stopped Given

300m Event 23 Mobile catching up Aircraft Moving Missing

300m Event 24 Mobile crossing right 45 Aircraft Moving Given

Scenario G night 200m Event 25 Mobile crossing right 45 Aircraft Moving Given


200m Event 27 Mobile catching up Aircraft Stopped Given

200m Event 28 Mobile face to face Aircraft Stopped Missing

Scenario D day 100m Event 13 Mobile crossing left 90 Vehicle Moving Given




Table 3-4: Simulation variable details for Crew 1


34/189





Threshold

Visibility


Mobile

Movement

Traffic info

delivery

Scenario B day 300m Event 5 Mobile crossing left 90 Vehicle Moving Given


300m Event 7 Mobile catching up Vehicle Stopped Given


Scenario C day 200m Event 9 Mobile crossing right 45 Aircraft Moving Given

200m Event 10 Mobile crossing left 90 Aircraft Moving Missing



Scenario E night 400m Event 17 Mobile crossing right 45 Aircraft Moving Given

400m Event 18 Mobile crossing left 90 Aircraft Moving Given400m Event 19 Mobile catching up Aircraft Stopped Given

400m Event 20 Mobile face to face Aircraft Stopped Missing

Scenario H night 100m Event 29 Mobile crossing left 90 Aircraft Moving Given







ThresholdVisibility


MobileMovement

Traffic infodelivery

Scenario A day 400m Event 1 Mobile crossing right 45 Aircraft Moving Given

400m Event 2 Mobile crossing left 90 Aircraft Moving Missing



Scenario G night 200m Event 25 Mobile crossing right 45 Aircraft Moving Given




Scenario D day 100m Event 13 Mobile crossing left 90 Aircraft Moving Given


100m Event 15 Mobile catching up Vehicle Stopped Given

100m Event 16 Mobile crossing right 45 Aircraft Moving Missing

Scenario F night 300m Event 21 Mobile crossing left 90 Aircraft Moving Given






35/189





Threshold

Visibility


Mobile

Movement

Traffic info

delivery

Scenario B day 300m Event 5 Mobile crossing left 90 Aircraft Moving Given



300m Event 8 Mobile crossing right 45 Aircraft Moving Missing

Scenario H night 100m Event 29 Mobile crossing left 90 Aircraft Moving Given


100m Event 31 Mobile catching up Aircraft Moving Missing


Scenario C day 200m Event 9 Mobile crossing right 45 Aircraft Moving Given

200m Event 10 Mobile crossing left 90 Aircraft Moving Given200m Event 11 Mobile face to face Vehicle Stopped Given


Scenario E night 400m Event 17 Mobile crossing right 45 Aircraft Moving Given





3.7 Simulated environment limits

3.7.1 Visibility Calibration

The following visibility calibration was necessary while using EPOPEE:

For visibility < 200 meters, EPOPEE visualisation was calibrated with 8 meters inaddition to the value decided (e.g.: input 108 meters for 100 meters of visibility frompilots eyes).

For visibility 200 meters, keep the same value as desired (e.g., input 200m for 200mof visibility from pilots eyes, input 300m for 300m of visibility from pilots eyes, etc).

3.7.2 Pilot HMI

The cockpit configuration is "A340-300" like, but is not completely representative of a real aircraftcockpit. The main equipment necessary for piloting the aircraft are available: FCU, side-sticks,throttles, nose wheel steering, slats/flaps commands, rudder pedals. Other equipment is not fullyphysically representative: radio management panel, overhead panel The configuration of thedisplays in the cabin is A340 configuration, even Pilot Flying Display (PFD) and Navigation Display(ND) screens are 6"*8" size. The displays are EIS2 A340 displays.

A sound-restitution system can be launched, in order to have aircrafts sounds and improve therepresentation of the simulation: engines, aerodynamic sounds, landing gear.

Moreover, during night condition simulations, headlights couldnt be activated from the cockpit butfrom the simulation monitoring. This was announced to the pilot and didnt imply problems asheadlights were activated with other parameters at each exercise launching.


36/189



3.7.3 Airport Realism

Several elements of the representation were not completely representative of an airport as defined inannex 14:

No panels were present for the taxiway identification.

No stop bars were present at the runway entry.

This lack of visual information prevented pilots to have classical marks and forced them to refer moreto the provided airport map.

Other limitations were noticed for this simulator:

In night conditions, markings and airport lights were considered as not sufficient resulting inhesitation and few route confusions.

Lights for other traffic were not considered representative of real aircraft and vehicle lights(not bright enough).

This implies for the result exploitation that the night conditions are considered as the worst observable

climatic case.

3.7.4 Pseudo-controller position

The pseudo-controller position was simply a microphone allowing communication with pilots through asingle voice channel.

This limited ATC environment implied a great amount of work from the pseudo-controller andsynchronisation with the Wizard of Oz to be kept aware of EPOPEE position so as to deliver thedefined information at the appropriate moment.

Furthermore, to reinforce cockpit realism with partly-line communications, the ATC position has beenenforced providing him pre-registered communications with virtual pilots (the generated surroundingmobiles and other mobiles not seen from EPOPEE) that he defined for simulation purpose in the log

presented in annex.

This highly contributed to scenarios realism. However, in order to cope with the simulation scope, it isworth noting that the following two points lower scenarios realism:

In order to create a workload and as taxiing was a necessity, crew were put under a heavyR/T loading.

In low visibility, for simulation purpose, it was sometimes asked to the pilot to cross therunway, operation that they would have not necessarily accepted in the real life.

4. DATA COLLECTION, VERIFICATION AND ANALYSIS

4.1 Data collection

4.1.1 Data measurable by automated system

The objective of the quantitative data collection is to provide the experimenter with the requireddistances between EPOPEE and the mobile.

As defined in section 2.4, two distances are of interest for the simulation:

The major measurement objective is Ddetection, the distance between EPOPEEand the mobile when the pilot of EPOPEE detects the latter.


37/189



A subsidiary measurement is Dsafety, the minimum distance between EPOPEEand the mobile during the event.

In the scope of the simulation, the following data were recorded:

The instantaneous position (Lat / Long) of EPOPEE.

The instantaneous speed of EPOPEE.

The instantaneous braking action it is an analogue value (the interesting point isthe variation of this value).

The instantaneous throttles control (i.e. the variation of its angle of inclination).

Mobiles pre-registered trajectory.

Mobile launch time.

Mobile detection time (recorded through a binary marker).

From the mobile launch, the mobile detection time and mobile pre-registered trajectory, it wasretrieved the instantaneous mobile position at the moment pilots detect it. A first Ddetection

estimation was computed from relative instantaneous positions expressed in latitude and longitude.However, since EPOPEE and mobile instantaneous positions were given from EPOPEE virtualgravity point (different from real gravity point) to mobile gravity point, a correction was necessary totake account of simulated aircraft (A340) and mobiles sizes, depending on their relative positions. Thefollowing corrections were given:

Aircraft (Boeing 737) Vehicle

Crossing 90 26m 15 m

Crossing 45 40m -

Face-to-Face 25m 15m

Catching up 28m 15m

Table 4-1: Corrections made on Ddetection and Dsafety measurements

For Dsafety estimation, a first computation was performed in order to detect the point where thedistance was minimum between EPOPEE and the mobile gravity points. Then, depending on theposition of each mobile at the closest distance, a correction was also given. When EPOPEE and themobile were really close, a correction different than the one presented in the previous table wasapplied, taking into account the real position and aspect of each one at this time.

4.1.2 Data gathered through exercise observation

Both observations during the session simulation and the different video and audio recordings wereused to collect the following:

Events impact on the PF and the PNF behaviour.

The condition when the flight crew sees the mobile (who, what, when, where).

The taxiing strategies.

The avoidance strategies.

4.1.3 Data obtained from participants

Data can be obtained from participants either during the short debriefing (exercise debriefing) from theitems (answers & comments) or during the long debriefing (session debriefing). The objective of thesedebriefings is to recover general and specific data in order to check that the results obtained areinterpretable in the context.


38/189



The different thematic asked to the flight crew were about:

Remarkable events and issues.

The risk perceived (risk/event, risk area, criticism of events, classification).

Comments about the simulated ATC informationcompared to reality.

Personal data of the flight crew are also used.

In this study, the personal data can make it possible to explain differences in the results between theflight crews. Of course, in the result presentation, the crew having taken part will remain anonymous.

This data concern:

Status of each member of the flying crew (captain or co-pilot).

The number of years of flight.

The number of year flying with airbus aircraft.

The name of the company.

4.2 Data verification

Data have been checked, placing the different EPOPEE Latitude/longitude points on a map, using anExcel routine. This allowed us to ensure that the data (EPOPEE movements and pre-registeredmobile movements) were coherent.

4.3 Data analysis

During the simulation run, qualitative and quantitative data have been gathered. The following sectionsdescribed the methods used for analysing both data types.

4.3.1 Qualitative analysis method

The qualitative data were extracted from different sources of information:

Short Debriefing after each simulation exercise.

Long debriefing after each simulation session.

Observations during each simulation exercise.

In the scope of this real-time simulation, the qualitative data analysis method consisted in analysingthe various comments with regards to a set of topics, defined on purpose for the study aims.

4.3.2 Quantitative analysis method

The quantitative data were extracted from different sources of information:

Questionnaires given to the pilots after each simulation exercise (Subjective data).

Data measurable by automated system (Objective data).

The quantitative method in this study consisted in a statistical analysis limited to the description ofdifferent numerical results for each variable. The quantitative data extracted from the exercises areplotted in figures and tables.

In order to validate the hypotheses, impact effects between variables are analysed for both subjectiveand objective data, based on the statistical analysis. However, due to the limited sample, themathematical verification of variable dependencies was not carried out. In other words, the statisticalanalysis does not include the analysis of correlation between different variables.


39/189



5. SYNTHESIS OF RESULTS

5.1 IntroductionThe experiment high-level objective was twofold:

At first, it aimed at assessing the success rate of traffic information from bothquantitative and qualitative perspectives.

Then, it aimed at assessing the minimum visibility threshold at which a crew canreceive, interpret and take in charge a traffic information while following all the requiredprocedures.

Please refer to chapter 2 for low-level objectives and hypotheses.

The present chapter synthesises the results issued by the experiment conducted with the EPOPEEA340 experimental flight simulator. It is organised so to present results with regards to the experimentobjectives.

5.2 Safety

5.2.1 See and avoid when visibility condition change (HLO1-LL1)

5.2.1.1 See other traffic : intruder detection vs detection distance

From visibility 200 meters and below, pilots ability to see other traffic is degraded.

In average, crews detected 83 % of events that were presented to them. This detection rate isdeteriorated with the fall in visibility conditions and is emphasised with the illumination degradation.

Indeed, the detection rate is degraded from 200 m of visibility (even from 300m for one case) undernight conditions whereas this degradation occurs only at 100 m of visibility under day conditions.However, at 100m of visibility

22 Asmgcs Final Results Conclusions

Documents

Transcript of 22 Asmgcs Final Results Conclusions