CARE ASAS Action The Application of the Validation ... · [1] ASAS Activity 2 Report: Towards a...

CARE/ASAS/QINETIQ/02-035 ASAS in CARE

CARE ASAS Activity 2: Validation Framework Project-WP4-D1ASAS VF and the EMERALD RTD Plan Version 1.0 – 16-October 2002QINETIQ/KI/AMS/ASAS/VF/WP4

CARE ASAS ActionThe Application of the Validation

Framework to the EMERALD RTD PlanWP4 Deliverable 6 Part 1

EUROCONTROL Ref: CARE/ASAS/QINETIQ/02-035 (Part 1)


CARE ASAS Activity 2: Validation Framework Project-WP4-D1 page iiASAS VF and the EMERALD RTD Plan Version 1.0 – 16-October 2002QINETIQ/KI/AMS/ASAS/VF/WP4

This page intentionally left blank


CARE ASAS Activity 2: Validation Framework Project-WP4-D1 page iiiASAS VF and the EMERALD RTD Plan Version 1.0 – 16-October 2002QINETIQ/KI/AMS/ASAS/VF/WP4

DOCUMENT REVIEW

Version Date Description Modifications0.1 31/07/02 First draft for internal review

0.2 05/08/02 Reviewed draft for consortiumreview

0.3 09/08/02 Review by consortium leader

0.4 02/09/02 Review comments atteleconference

1.0 16/10/02 Release in CARE/ASAS format,post dissemination forum

Revised for content and format

DISTRIBUTION LIST

Consortium EUROCONTROL and CARE/ASAS Action Manager

Rosalind Eveleigh NATS Mick van Gool EUROCONTROL Agency

Jose Miguel de Pablo Aena Francis Casaux EUROCONTROL Agency

John Bennett QinetiQ Ulrich Borkenhagen EUROCONTROL Agency

Juan Alberto Herreria Isdefe

Brian Hilburn NLR


CARE ASAS Activity 2: Validation Framework Project-WP4-D1 page ivASAS VF and the EMERALD RTD Plan Version 1.0 – 16-October 2002QINETIQ/KI/AMS/ASAS/VF/WP4

This page intentionally left blank


CARE ASAS Activity 2: Validation Framework Project-WP4-D1 page 5ASAS VF and the EMERALD RTD Plan Version 1.0 – 16-October 2002QINETIQ/KI/AMS/ASAS/VF/WP4

EXECUTIVE SUMMARY

EUROCONTROL’s CARE-ASAS programme aims to consolidate previous work on ASASand to co-ordinate future EUROCONTROL sponsored research in this area. As part of thisprogramme, EUROCONTROL invited tenders for the development of a validation frameworkfor the assessment of proposed ASAS applications.

The aim of the current project is to specify a Validation Framework (VF) that provides forcomparability and consolidation of results across various ASAS research projects. The wide-range of potential operational concepts, and diverse techniques that may be used for theirvalidation, have led to the requirement for the framework to be generic.

This document analyses the ASAS Research and Technical Development (RTD) Plan thatwas developed in Work Package 5 of the European Commission project EMERALD. Thework is concerned with the Validation issues in the plan, and analyses their match to theASAS validation framework developed during the course of this project.

The issues from the Validation Phase of the RTD plan can be equated to the identification ofvalidation aims within MAEVA. These need to be developed into high-level and low-levelvalidation objectives to be translated into experimental design.

The findings are that the RTD plan maps well to the Validation approach outlined in theMAEVA documentation. A totally generic, single Validation for ASAS as a whole cannot bedefined. The operational procedures and/or airspace requirements will be different for eachapplication.

The EMERALD RTD Plan approach may result in long delays if the validation activities arecarried out to late in the process to address any issues that may effect the applicationdesign.

Safety assessments are key to the validation of ASAS applications. There is no one size fitsall approach.



REFERENCE LIST

[1] ASAS Activity 2 Report: Towards a validation framework for ASAS applicationsASAS Activity 2 ReportEdition 1.0 12 June 2001

[2] MAEVA Validation Guideline HandbookEC DG-TREN Transport Programme. Release 1.0 25/05/01

[3] EUROCONTROL , ATM Strategy for 2000+Edition January 2000

[4] PRC’s European ATM Performance Measurement System, EUROCONTROL, PRUreference document. Edition 1.7, 1 June 1999

[5] INTEGRA Safety Study reference TBD[6] CARE- ASAS Activity 2 Report: Initial Validation Framework and Scenario Report

Version 0.1 6 March 2002[7] Principles of Operation for the use of ASAS

FAA / EUROCONTROL Co-operative R&D. Version 7.1, 19 June 2001[8] Project EMERALD Technical Report, WP5 – Assessment of Emerging technologies:

the specific case of ADS-B/ASAS, Vol. 6 of 8, European Commission, Ref:AI-97-AM-1173, Version 2.0, October 1998

[9] CARE-ASAS Activity 2 – D2: WP2 Systems Performance Metrics,QINETIQ/KI/AMS/CR021122, Version 1.0, 7 June 2002.

[10] A Proposed Generic High Level Validation Methodology, DEVAM deliverable D2/D3,version 1.0, 2 December 1999.

[11] EATMS, Operational Concept Document (OCD), EUROCONTROL,FCO.ET1.ST07.DEL01, Proposed Issue, Edition 1.0, Brussels, March 1997.

[12] Project EMERALD Technical Report Volume 2, “WP1- Definition of BaselineScenario”, AI-97-AM-1173, October 1998.

[13] Project INTENT, WP2.2 Validation Plan, GRD1-2000-25326, Version 1.0, 31October 2001.

[14] CARE-ASAS Activity 2, “Toward a Validation Framework for ASAS Applications”,Version 1.0, 12 June 2001

[15] ‘A study of aircraft equipage and current capabilities of the aircraft. The futureexpectations of system designers, avionics manufacturers and operators’,EUROCONTROL, C1.184/CB/TRS124, version 1.0 May 2002.

[16] Project EMERTA, WP3.2, ‘Safety/separation modelling of a particular ASASapplication’, version 2.0, 5 March 2001.

[17] RTCA MASPS for ADS-B Draft 6.0 – August 1997.[18] ECAC APATSI Document on medium-term air traffic control procedures and

techniques – August 1995.[19] Airspace and ATM concepts and options for the single unified European CNS/ATM

system (ATAS study), June 1993.[20] CARE-ASAS Activity 2 Work Package 4.2 – ‘Case study of Validation Framework’,

August 2002.



TABLE OF CONTENTS

1. INTRODUCTION................................................................................................................. 81.1. Objectives of the project............................................................................................ 81.2. Objective of WP4....................................................................................................... 8

2. ABBREVIATIONS AND GLOSSARY.................................................................................. 83. THE EMERALD RTD PLAN .............................................................................................. 10

3.1. Background.............................................................................................................. 103.2. The RTD plan structure........................................................................................... 103.3. Referencing the RTD plan....................................................................................... 11

4. SUMMARY OF ATM VALIDATION................................................................................... 125. DESCRIPTION OF RTD PLAN ANALYSIS ...................................................................... 13

5.1. Scope of the work.................................................................................................... 135.2. Approach to the work............................................................................................... 14

6. HIGH LEVEL ANALYSIS OF THE RTD PLAN ................................................................. 157. VALIDATION ACTIVITIES................................................................................................. 21

7.1. The RTD plan and the MAEVA approach............................................................... 217.2. The types of validation exercises required to deliver the RTD plan ....................... 247.3. Real-time simulations .............................................................................................. 257.4. Flight-trials ............................................................................................................... 267.5. Fast-time simulations .............................................................................................. 267.6. Analytic techniques.................................................................................................. 267.7. Survey techniques ................................................................................................... 277.8. Translating the high-level aims into experiments ................................................... 277.9. The development of an ASAS application validation plan from the RTD plan....... 317.10. Developing the high-level validation aims further ................................................... 337.11. RTD issues that have been addressed................................................................... 33

8. CONCLUSIONS................................................................................................................. 34

Annex A EMERALD RTD Plan ..............................................................................................35Annex B Issues in the RTD plan requiring specific validation exercises ..............................41



1. INTRODUCTION1.1. Objectives of the project

The EUROCONTROL CARE-ASAS programme aims to consolidate previous work on ASASand to co-ordinate future EUROCONTROL sponsored research in this area. As part of thisprogramme, EUROCONTROL invited tenders for the development of a validation frameworkfor the assessment of proposed ASAS applications.

The aim of the current project is to specify a Validation Framework (VF) that provides forcomparability and consolidation of results across various ASAS research projects. The wide-range of potential operational concepts and diverse techniques that may be used for theirvalidation have led to the requirement for the framework to be generic.

This document forms part of the CARE ASAS Activity 2 work. It is part of Deliverable D6 ofWork Package (WP) 4.

1.2. Objective of WP4

The project is divided into one management and four technical work packages defined asfollows:

− WP0 - Management− WP1 - Identification of ASAS operational scenarios− WP2 - System performance metrics− WP3 - Human performance metrics− WP4 - Application of validation framework

Within WP4 “The Application of the Validation Framework” the Deliverable D6 will be a“Guideline for the Application of the ASAS Validation Framework”. This will be achieved by 3activities.

WP4.1 “Identification of links between ASAS VF and MAEVA validation framework”. Theapproaches of CARE/ASAS and MAEVA will be compared for consistency. The EMERALDRTD Plan will analysed against the ASAS VF to provide an indication of the types ofvalidation exercises necessary to deliver the RTD plan programme aims.

WP4.2 “Guidelines for Application of the ASAS VF”. Two specific case studies will beperformed on the CARE/ASAS activity 3 applications - ‘time-based sequencing in approach’and ‘airborne self-separation in en-route airspace’.

WP4.3 “Guideline Report” Presents the guidelines developed in a suitable format for use onother CARE/ASAS applications.

This work is the EMERALD RTD Plan work of WP4.1.

2. ABBREVIATIONS AND GLOSSARY

ACAS Airborne Collision Avoidance SystemADS-B Automatic Dependent Surveillance – BroadcastAP Acceptability PhaseASAS Airborne Separation Assurance SystemATC Air Traffic ControlATM Air Traffic Management



CAA Civil Aviation AuthorityCARE Co-operative Actions of Research and development in EurocontrolCNS Communication, Navigation and SurveillanceCP Concept PhaseEATMS European Air Traffic Management SystemEC European CommissionEMERALD Emerging RTD Activities of Relevance for ATM Concept DefinitionEMERTA Emerging Technologies Opportunities, Issues and Impact on ATMFFAS Free Flight Air SpaceFP Feasibility PhaseGNSS Global Navigation Satellite SystemHMI Human Machine InterfaceHUD Head Up DisplayICAO International Civil Aviation OrganisationIFR Instrument Flight RulesILS Instrument Landing SystemIMC Instrument Meteorological ConditionsMAEVA Master ATM European Validation PlanMAS Managed Air SpaceNATS National Air Traffic ServicesNLR National Aerospace LaboratoryOCD Operational Concept DocumentPP Prototyping PhaseRTCA Radio Technical Commission for AeronauticsRTD Research and Technical DevelopmentRVSM Reduced Vertical Separation MinimaSC Strategic Co-operativeSMGCS Surface Movement Guidance and Control SystemTC Tactical Co-operativeTMA Traffic Manoeuvring AreaTSA Traffic Situational AwarenessVDR Validation Data RepositoryVF Validation FrameworkVFR Visual Flight RulesVP Validation PhaseWI Work ItemWP Work Package



3. THE EMERALD RTD PLAN3.1. Background

Project EMERALD was initiated in response to the Air Transport Research Task 4.1.2/19 inthe second call for proposals by the European Commission (EC) Fourth FrameworkProgramme. EMERALD was designed to provide recommendations for future Research andTechnical Development (RTD) activities for CNS development in support of future ATMconcepts, covering the period 1997 to 2015.

Volume six of the eight volumes produced by Project EMERALD and was concerned withWork Package 5 (WP5) - ‘Assessment of emerging technologies: The specific case of ADS-B/ASAS’ [8].

This objectives of the work conducted for EMERALD WP5, were to:

- Perform an initial but domain comprehensive assessment of the ADS-Btechniques for Airborne Separation Assurance Systems (ASAS). Assessmentwill be made of their technical and operational capabilities to support flightoperation of aircraft, enhanced situation awareness and ASAS within the futureEATMS environment, wherever it is both feasible safety-wise and beneficialfrom an airspace capacity point of view.

- Take stock of the expected results achieved at the end of EMERALD and toproduce recommendations for future Research and Technical Development(RTD) activities, documented as draft RTD plans for consideration of decisionmakers both at the European level (i.e. the EC and EUROCONTROL) andnational level (i.e. the CAA and the EMERALD partners).

The main assumptions used were:

- The definition of the ASAS concept is that of the ICAO/SICAS Panel, the mainfeatures of which are that ASAS is intended to:- improve pilot’s traffic situation awareness;- increase the capacity of the ATM system;- provide better services for the airspace users.- ASAS and ACAS must remain as independent functions.- ASAS applications are developed in the context of European Airspace

(EATMS OCD[11]).

3.2. The RTD plan structure.The EMERALD ASAS/ADS-B Research and Technical Development Plan summarises theASAS/ADS-B issues that were identified in the work and classifies them in a systemdevelopment framework in the following high-level phases:

- User Requirement or Concept Phase (CP)- User Requirement Analysis or Feasibility Phase (FP)- Functional Requirement or Acceptability Phase (AP)- ASAS Development or Prototyping Phase (PP)- Experimentation and Validation Phase (VP)- Implementation Phase.



The implementation phase is not really part of Research and Technology Development. It isincluded for completeness to illustrate the requirements for the final steps to full ASASimplementation.

The elements of the RTD plan are further broken down into the following domains:-

A. Operational Concepts;B. Benefits and Constraints;C. Safety Assessment;D. ASAS Operations and Human FactorsE. ASAS Design and Airborne Functions (including HMI)F. Transition Issues

3.3. Referencing the RTD plan

The original RTD plan is shown in Appendix A of this document. The system used in thisdocument to reference each of the issues to the original plan is described here.

Each of the elements represents an issue identified as part of the EMERALD Work Package5. The references will consist of a prefix, denoting the RTD plan phase (e.g. “PP” for thePrototyping Phase), and a suffix which will be the two character identifier as it exists in thecurrent version of the RTD plan. There are six domains identified by a letter A to F asillustrated above. For example: FP.D3 refers to the third issue in the ASAS OperationsDomain (D) in the Feasibility Phase of the RTD plan, namely; “Study the Transition BetweenFFAS and MAS.”



4. SUMMARY OF ATM VALIDATION

The MAEVA work was designed to enable the validation of any new Air Traffic Management(ATM) Concept as it passes through each stage of its development lifecycle.

An overview of the Validation Process is illustrated in figure 1 taken from the MAEVAValidation Guideline Handbook [2]. This illustrates the solution to an ATM problem via anATM concept that requires a set of validation activities. This report is concerned with thevalidation framework and the validation concept required to deliver the EMERALD ASASRTD plan (see section 2).

Figure 1. Validation Process Overview

When a new ATM operational concept is developed, it passes through many stages before itreaches operational implementation. At the end of each stage a decision is made whether tocontinue or terminate development, based on an evaluation of the concept against its statedaims. When a go-ahead decision is taken, the design of the concept is refined (whennecessary) with the experience gained in the previous phase(s) and the outcome of theevaluation. Through validation, confidence is provided that an ATM concept addresses theATM problem for which it was designed and achieves its stated aims.

Up to now validation activities in Europe have usually been performed in an enabler-targetedway, taking a bottom-up approach. The EMERALD RTD plan, which was developed in 1998,prior to the conclusion of MAEVA activities, defined high level issues and a strategy thatneed to be addressed in order to realise ASAS implementation.

The MAEVA framework defined validation exercises in five basic steps:

1. Define the validation aims, objectives and hypotheses2. Prepare the validation plan and prepare for exercise runs3. Execute exercise runs and make measurements4. Analyse results5. Develop conclusions and evaluate.

The object of this project is to develop a Validation Framework that is applicable to the ASASconcept. This element of the work analyses the EMERALD ASAS RTD plan to inform thatFramework. The approach to this work is described in the next section. The applicability ofthe MAEVA approach to validation within the RTD plan is explored and the mismatches, ifany, identified.



5. DESCRIPTION OF RTD PLAN ANALYSIS5.1. Scope of the work

The EMERALD RTD plan was developed before the MAEVA Validation Plan. The Plan’svalidation activities and issues are explored further as a course of this work. A ValidationFramework that is applicable to ASAS is to be developed as a result of this project. Analysisof the RTD plan validation issues informs this development. This report focuses on theExperimentation and Validation Phase of the RTD plan, where the validation work is of mostsignificance.

The RTD Plan was designed to be applicable to all ASAS applications, and therefore to theASAS concept as a whole. It has high-level validation activities specified within it, particularlyin the Experimentation and Validation Phase of the Plan. These equate to the High-LevelValidation Aims specified in the MAEVA guidance material (Figure 2 and reference [2]).

Figure 2 summarises the processes for Step 1 in the MAEVA approach [2]. Step 1 is to“Define the validation aims, objectives and hypotheses”. This step identifies Validation Aims,initially at a high level. The Validation Aims in the RTD plan are similarly described at a highlevel. Subsequent steps in the MAEVA approach are concerned with Validation exercisespecific details, such as the methods of data analysis and reporting.

Figure 3. Process Diagram for MAEVA Step 1

This report will not develop the high-level objectives into low level ones, nor develop theexperiments described down to the identification of metrics. It would be impossible within thetime-scales and resources to do this, as it would have to be developed for all the individualASAS applications. This work is addressed in other parts of this project. The metrics issuesare described in reference [9] and two ASAS applications are developed further within aMAEVA framework by [20].



5.2. Approach to the work

A high-level analysis of validation issues in the RTD plan is discussed in Section 5 of thisreport. The RTD plan is analysed to identify the elements that require validation activities.(Section 5 and Appendix B).

In section 6 a more detailed analysis of the Validation issues occurs. Equating the issues inthe Validation and Experimentation Phase of the RTD plan to the Identification of ValidationAims in MAEVA, the work of this report will be to develop these further using the MAEVAapproach.

The Validation Aims will be developed to identify high-level objectives for each of the relevantissues.

The nature of the experiments required is discussed, together with the required inputs,outputs and interdependencies.

Projects that have addressed any issues in the RTD plan are reported in section 6.11.

A summary of the validation exercises required in order to deliver the RTD programme isgiven in Section 6.8.



6. HIGH LEVEL ANALYSIS OF THE RTD PLAN

The RTD plan follows a staged development lifecycle. Where validation activities arerequired a validation strategy or validation route map can be developed as defined in theMAEVA [2] and DEVAM [10] work. An example route on a Validation route map is illustratedin figure 2.

Figure 2. Example Route on Validation Route Map (from DEVAM)

The route map shown would apply to a particular ASAS application (e.g. Station Keeping onApproach) rather than the ASAS concept as a whole. The generic nature of the RTD plan asit stands makes the definition of validation experiments impossible. There needs to be adetailed specification of the ASAS application that is to be validated and from this thedefinition of the planned validation activities would follow. The individual ASAS applicationprocedures are initially defined in the Functional Requirement Phase of the RTD plan. Theseare further refined as a course of the Experimentation and Validation Phase of the plan.

The route map applies to specific validation tasks that relate to one of the high-levelvalidation aims described in the RTD plan. These aims are identified and described insections 5 and 6 of this report.

The route map shows a validation path from analytic modelling through a variety of potentialroutes to operation. This has some similarity to the approach in RTD plan. The analyticalwork in earlier phases of the RTD plan (particularly the User Requirement Analysis andFunctional Requirement phases) informs the validation exercises and experiments in theExperimentation and Validation Phase. The RTD plan uses the Validation Phase to informthe ICAO standards for ASAS applications, whereas the Route Map seems to go straight tooperations without mention of the standardisation process.

Comparing the EMERALD RTD process to the MAEVA first steps will identify any synergiesand mismatches between the two approaches.

The EMERALD RTD has already addressed the initial, high-level steps of the MAEVA step 1processes. These are all addressed directly by EMERALD WP5 [8] or in the case ofIdentification of Stakeholders in other work within the EMERALD project [12].



With regard to ASAS, the EMERALD approach addresses the following items in the MAEVAguidelines:

- Understanding the ATM Problem

- Understanding the Operational Concept

- Identification of Stakeholders

- Identification of Validation Aim.

The EMERALD RTD plan identifies the following validation issues that need to be addressedin order to realise ASAS applications:

- Analytical modelling of ASAS applications and airspace

- Fast-time and real-time simulations, that also address Human Factors issues

- Validation and Trials exercises

The MAEVA Framework can be applied from this stage onwards for individual ASASapplication validation exercises. Each scenario will be a specific ASAS application in aspecified airspace and set of traffic samples. A totally generic, single set of validationexercises and experiments cannot be defined for ASAS as a whole. The operationalprocedures and airspace requirements may be different for each application. In addition tothis ASAS applications will mature at different rates (it is possible that Enhanced VisualAcquisition will occur first and autonomous aircraft last).

The Validation Phase of the RTD plan is described as being essential by EMERALD. Itincludes validation through real-time simulations involving pilots and controllers and alsoflight trials.

The validation process should measure the potential benefits and constraints but also theassociated controller and pilot workloads. During the Validation Phase a more in depthcost/benefit analysis addressing partial and progressive equipage of the fleet is advocated bythe RTD plan.

Similarly, the safety of the ASAS applications needs to be proven. Through the RTD planphases the benefits and constraints issues occur from the User Requirements through to theValidation Phase, with no activity occurring in the Prototyping Phase. Safety assessmentoccurs from the Users Requirement Analysis Phase onwards. Validation in each of these keyareas is separate. It is only at the completion of the Validation Phase that standards arefinalised and implementation may begin.

EMERALD states after some iterations of the issues described in the Experimentation andValidation Phase, the ICAO standards and that the industry standards can be finalised. Amajor concern here is the length of time associated with this process and the delay thatmany iterations may take.



EMERALD proposes estimates of the time scales required for each phase of the RTD planas illustrated in the following table. There are estimates for each of the EMERALD categoriesof ASAS application. These are:-

- Traffic Situational Awareness (TSA) includes Enhanced Visual Acquisition.- Tactical Co-operative (TC). This category contains both shadowing and

distancing applications. This includes the ‘time-based sequencing in approach’application.

- Strategic Co-operative (SC). This includes the ‘airborne self-separation in en-route airspace’ application.

These map onto the PO-ASAS categories:- Air traffic Situational Awareness, AirborneSpacing, Airborne Separation, Airborne Self-Separation.

The ASAS application categories in the RTD plan were seen as a key factor for the durationof their development. Indeed, the Traffic Situation Awareness applications are the easiestapplications to address. The proposed values are considered as optimistic even if in someinstances the phases can overlap.

Schedule(in years)

TSAapplications

TCapplications

SC applications

User Requirement or Concept Phase - 1 1

User Requirement Analysis orFeasibility Phase 0.5 1 2

Functional Requirement orAcceptability Phase 0.5 2 3

ASAS Development orPrototyping Phase 1 3 5

Experimentation andValidation Phase 1 3 3

Total duration 3 10 14

The Validation Phase in the TC class applications can range from 3 to 6 years depending onthe application. For example ‘closely spaced parallel approach’ is perceived to be verydifficult to Validate. Traffic Situational Awareness (or Enhanced Visual Acquisition as it isnow called) is perceived to be relatively easy.



The ASAS categories are not comparable because they will lead to different benefits:

- the TSA applications aims to improve the level of safety in specific airspacebut cannot provide increased capacity or better flexibility on their own;

- the TC applications aims to improve capacity and provide better flexibility inMAS;

- the SC applications aims to provide better flexibility in FFAS.

There are other institutional issues that the aircraft industry reports as causing delays in theprocess of implementation [15]. They prefer to have the applications well defined before theycan confidently conduct benefits/constraint studies. The RTD plan shows the definitionoccurring late in the overall plan.

EMERALD identifies the need for a common approach to the issues by all of thestakeholders. The MAEVA approach may enable this.

The elements of the RTD plan that have been identified as requiring validation activities aresummarised in Appendix B. This initial selection was made on the basis of those activitiesthat contain:

- Simulations- Comparisons- Experiments- Validation.- System Safety/Capacity Studies

The focus of this work will be on the main validation exercises in the Validation andExperimentation Phases of the RTD plan.

In most cases the choices are clear. There are some issues that are not clear choices forinclusion. The following two candidate issues have been omitted:

- Cost Benefit Analyses- Impact of System Architecture on Safety

The EMERALD ASAS work identified 32 potential applications and classified them accordingto type of airspace, term of implementation and expected benefits as shown overleaf:



N° Application Name Type of airspace Term Expected benefits

Oceanic,en route

and remotenon-radar

Enroute

Terminal Airport Short Medium Long Safety Capacity

Efficiency

1 ASAS CROSSING PROCEDURE X X X X X

2 ALERT OTHER AIRCRAFT, ATC ANDFLIGHT SERVICE OF PILOTS WITHSPECIAL NEEDS

X X X X X X X

3 AUTOMATIC WEATHER REPORTING X X X X X X X X

4 AUTONOMOUS AIRCRAFT X X X X X

5 BAROMETRIC ALTITUDE AND GNSSHEIGHT REPORTING FOR RVSMCONFORMANCE MONITORING

X X X X X X

6 CLOSELY SPACED PARALLELAPPROACHES IN IMC

X X X X X

7 COLLABORATIVE RE-ROUTING X X X X X X X

8 COLLISION SITUATIONALAWARENESS

X X X X X X X

9 CONFLICT MANAGEMENT WHILEPERFORMING SELF-SEPARATIONFOR FREE FLIGHT

X X X X X X X X

10 CONFLICT SITUATIONALAWARENESS

X X X X X X X

11 DEPARTURE SPACING X X X X

12 ENHANCED COCKPIT SITUATIONALAWARENESS FOR CONVERGINGRUNWAY APPROACHES AND/ORLAND-AND-HOLD-SHORTAPPLICATIONS

X X X X X

13 ENHANCED IFR SURFACEOPERATIONS

X X X X X

14 ENHANCED VISUAL ACQUISITIONOF OTHER TRAFFIC FOR « SEE ANDAVOID »

X X X X X X X

15 ENHANCED VISUAL ACQUISITIONOF OTHER TRAFFIC IN THE VFRTRAFFIC PATTERN ATUNCONTROLLED AIRPORTS

X X X X X



16 ENHANCED VISUAL APPROACH X X X X X

17 ESTABLISH IN-TRAIL SPACINGINTERVAL

X X X X

18 FINAL APPROACH SPACING X X X X

19 IN-TRAIL CLIMB, IN-TRAIL DESCENT X X X X X

20 LATERAL PASSING MANEUVERS X X X X X

21 LEAD AIRCRAFT GLIDEPATHVISUALISATION

X X X

22 LEAD CLIMB AND DESCENT X X X X

23 LOW ALTITUDE STATION KEEPING X X X X

24 NAVIGATION GUIDANCE FOR LOSTOR DISORIENTED AIRCRAFT(2) X X X

25 PASSING MANEUVERS X X X X

26 PATROL FLIGHT(1)X X X

27 RUNWAY AND FINAL APPROACHOCCUPANCY AWARENESS

X X X

28 RUNWAY INCURSION MONITORINGWHILE CROSSING RUNWAYS ANDTAXIWAYS(2)

X X X X X

29 STATION KEEPING X X X X

30 SURFACE SITUATIONALAWARENESS

X X X X X

31 SURVEILLANCE ENHANCEMENTSFOR TCAS/ACAS

X X X X X X

32 TRAFFIC SITUATIONALAWARENESS

X X X X X X

The experiments required for the applications will be influenced by the nature of the airspacefor which they are required.

All applications must be validated for Safety. The applications table indicates the mainbenefits expected from the listed applications.



7. VALIDATION ACTIVITIES7.1. The RTD plan and the MAEVA approach

The first step in defining Validation activities to support the RTD plan is to create a clear anddocumented understanding of the requirement for the validation exercise and develop anexercise requirement specification with sufficient precision for detailed exercise planning tostart. These will arise as a result of the high-level Validation Aims stated as an issue in theRTD plan. It is likely that these will have to be single ASAS application specific. It is possiblethat a single experiment may address more than one Validation Aim.

The RTD plan issues where validation has been indicated as necessary are shown insections 6.1.11 and 6.1.12. A fuller analysis of the validation activities follows in section 6.2through to section 6.8. Projects that have already addressed some of the work are reportedin section 6.11.

The analysis draws on the System Performance Metrics work of this project [9], and theMAEVA Validation Framework [2].

Within the framework each step is decomposed into a further set of activities and sub-activities. Within Step 1, Activity 1.9 “Determine Operational and Statistical Significance”describes the relationship between the Size and Significance of the Exercise. This isparticularly relevant in terms of the EMERALD RTD plan. The level of significance, which isproportional to the operational significance and the size of the exercise, will be determined bythe phase within the RTD plan that the validation exercise refers to. For example the initialASAS safety study that was conducted as part of the work of project EMERTA [16] would notbe subject to the same level of significance as the full safety study required by the ValidationPhase.

Greater rigour and more significance will be applied to the Experimentation/ Validation Phaseissues of the RTD plan than to those issues in the Feasibility Phase. Several of the issuesidentified in the plan differ only in the phase in which they are addressed (e.g. themeasurement of benefits and constraints with suffix “.B1” and the safety studies with suffix“.C1”). This was a deliberate part of the design of the RTD plan.

Figure 4 is taken from the MAEVA Validation Guidelines and summarises Activity 1.9 ofStep1. This activity is aimed at determining the operational/experimental significance of theresults of any given Validation exercise. The significance of the Validation in the RTD planwill be proportional to the phase in which the validation activity is specified.

Figure 4. Relationship between Size and Significance of Exercise



Following on from step one, the next steps in the MAEVA Validation guidance material aretowards designing a suitable validation exercise. The step 2 processes illustrated in figure 5will need to be applied to each scenario and each high-level validation issue identified in theRTD plan. The RTD plan does not require any work in the implementation phase. It isincluded for completeness. The ultimate validation exercise beyond certification is flying theequipment.

Figure 5. Process diagram for MAEVA Step 2

The high-level validation issues identified from the RTD plan (see appendix B) are listed inthe following paragraphs. Common issues from different phases of the RTD plan have beengrouped together. Within the 5th FP, eight high-level strategic objectives were identified.Those that are addressed by this study are are:

- Safety

- Economics

- Capacity

- Environment

- National security and defence requirements

Each of the identified issues has been associated with the above high-level strategicobjectives to indicate the group of metrics that is most applicable. Metrics issues aredescribed in WP2 [9] of this project.

Key to ASAS validation is the complete specification of the applications, the operationalprocedures, airspace issues and the HMI. Along the path to this definition either linearly if



successful or iteratively, there are many exercise that require experiments. The focus of thework of this study is on the validation of the finalised ASAS applications.

The following issues have been identified from the RTD plan prior the Validation Phase.These issues may require validation activities and may act in support of ASAS applicationand final validation experiment development:

- Evaluate potential benefits through fast-time simulations (FP.B1)

- Evaluate potential benefits & constraints through fast-time simulations (AP.B1).The potential benefits are not specifically defined at this stage. They aredefined as operational benefits. These high-level aims are not identified in theRTD plan. It is assumed that they are Capacity, Safety and Economic. But theycould include environmental, security and uniformity issues.

- Conduct an initial safety study (FP.C1), Conduct a more comprehensive safetystudy (AP.C1). An initial ASAS safety study was conducted as part of theEuropean Commission funded project EMERTA. These issues relate to theSafety high-level aims.

- Two FHAs have to be conducted, one for the airborne segment and one for theATC segments (AP.C5).

- Select ASAS algorithms after validation through off-line simulations (AP.E2)and

- Evaluate the compatibility between ASAS and ACAS algorithms through off-line simulations (AP.E5).

- FHAs based on a top-down iterative approach in order to analyse systemfunctions (PP.C4).

The following issues are from the Experimentation and Validation Phase of the RTD plan andwill be explored further in section 6.8

- Measure potential benefits & constraints through real-time simulations (VP.B1)

- Conduct a full safety study taking into account human factors (VP.C1)

- Validate ASAS algorithms through real-time simulations (VP.E2).

- Test the compatibility between ASAS and ACAS through real-time simulations(VP.E5).

- Validation of the selected ASAS applications through real time simulations(VP. A1) and Experiment and validate selected ASAS applications throughflight trials (VP.A1)

- Validate the new airspace boundaries and organisation through simulations(VP.A4)

- Validate the new share of responsibility between pilots & controllers (VP.A3)

- SSA to demonstrate that the system and their elements, meet the safetyobjectives and requirements (VP.C6).

- Validate the ‘New ‘Rules of the Air’ and the airborne separation minimathrough simulations (VP.D6 & D7). Capacity issue.

- Measure controller & pilot workloads during real-time simulations (VP.D10)and Evaluate pilot workload during flight trials (VP.D10)



- Evaluate the compatibility between the ASAS and ACAS procedures duringflight simulations (VP.D9). Safety and Uniformity issues.

- Experiment and validate the ASAS HMI (VP.E20 to E23). Human involvementand commitment. Safety issues.

7.2. The types of validation exercises required to deliver the RTD plan

The stages of the MAEVA validation framework are summarised in the following flowdiagram. There are interdependencies between these stages and other issues, these areshown in more detail in figure 3, the flow diagram indicates the areas of the ASAS validationprogram that will be addressed here.

Subsequent Validation Issues:

High-level experimental design Identification of metrics Operational significance Platform design

Understanding the ATM problem

Understanding the operationalconcept

Identification of stakeholders

Identification of validation aim

Identification of high-levelobjectives

Identification of low-level objectives

Figure 5. Summary of the Step 1 in the MAEVA Validation Guidelines

The EMERALD RTD plan addressed the stages above the dotted line in the diagram, (shownabove in green), ending up with an expression of the high-level validation aims. The ATCproblem is defined as an overall ATM capacity issue, which may be mitigated via ASASsystems as part of the ICAO FANS 1/A and EUROCONTROL ATM 2000+ Strategies.



The Operational Concept understanding for the RTD plan was derived from the EATMSOperational Concept Document [11], the version of the ADS-B MASPS concurrent with theEMERALD project [17] as well as other operational concept work available in 1997 (e.g. [18]and [19]).

The customer/development team approach was used during EMERALD to develop the RTDplan and identify the ASAS validation aims. MAEVA recommends the appointment of aValidation Manager (VM) to oversee a validation programme to take these high-level aimsforward. One or more Validation Teams will then tackle each or all of the validation aimsthrough validation exercises. The VM co-ordinates all the validation activities.

The work of EMERALD identified the ATM problem and the operational concept with respectto ASAS/ADS-B. It identifies the stakeholders, and identifies the validation aims. These needto be translated into the identification of high-level validation objectives, experimental designand metrics. It is likely that experiments may need to be ASAS application specific.

The Validation (and Experimentation) Phase of the RTD plan contains the key Validationexercises that finally ensures that the ASAS concept, via the individual applications, achievesits stated benefits while maintaining and/or improving safety. At the end of this phase theICAO and industry standards can be finalised and the implementation of ASAS may begin.Several iterations may be required if Operational, Safety or benefits issues arise as a courseof the investigations. This phase also contains ASAS flight trials activities.

The Validation Phase, like the other phases in the RTD plan, is split into six domains. Thereare three Validation activities in the Operational Concepts domain. These relate to Validationof selected ASAS applications. The following sections outline the experiments required.Sections 6.3 through to section 6.7 address the different data collection techniques that maybe employed during validation exercises arising from the RTD plan. Section 6.8 develops thehigh-level validation aims from the RTD plan further.

7.3. Real-time simulations

The work in the Operational Concept Phase is closely tied to the issues in the ASASOperations and Human Factors domain, in that many of the issues will be addressed by real-time simulations. If correctly organised then all of the issues may be addressed by the samereal-time simulation experiments.

With good experimental design, many of the Validation aims in the RTD plan may beaddressed during the same real-time simulations of the air and ground components for anATM system incorporating ASAS applications. These Validation Exercises could includesome or all of the following:-

- the ASAS application (operational procedures)

- the new airspace boundaries

- the share of responsibility between pilots and controllers

- the New Rules of the Air and the airborne separation minima

- controller and pilot training programmes

- ASAS algorithms

- the ASAS HMI



- compatibility of ASAS and ACAS

One real-time experimental organisation may be able to address all, or most, of these issues.The design of the experiment the associated validation exercises must be carefully plannedto make the measurements required. The MAEVA Validation Guidance material states that“factors of interest to the validation can be investigated without any ambiguity”. There may beelements within the Experimentation/Validation exercises that are interdependent, e.g.Workload, Training and Algorithm design/optimisation.

In the real-time simulations the platform (which may include elements from the ground ATMSystem as well as the air) must represent the real-life situation as closely as is possible.

There are advantages in economies of scale in using large real-time experiments to addressthe multiple validation aims. However the complexity and inter-dependency of the selectedmetrics to cover all the aims may offset the potential gains. For example the traffic sampleorganisation in an exercise may have a greater effect on workload metrics that theintroduction of the new concept, in this case ASAS.

7.4. Flight-trials

The flight trial activities in the Validation Phase of the RTD Plan are a special case of thereal-time simulation. Simulations will have given confidence in the safety of the application. Inflight trials the introduction of one or more aircraft carries potential risk to human life andequipment. Where possible, it is probably preferable to

- Experiment and validate selected ASAS applications

- Evaluate pilot workload

- evaluate ASAS algorithms

- Measure the performance of the selected ADS-B architecture

7.5. Fast-time simulations

Other ASAS experimental measures and their associated Validation Exercises will bederived from fast-time simulations. These include:-

- measurement of benefits and constraints

- full safety study

In fast-time simulations many of the variables will need to be modelled and run in Monte-Carlo style in order to uncover a true representation of the real world scenario. Allassumptions (e.g. with regard to traffic sample sizes and organisation) will need to bedocumented.

7.6. Analytic techniques

Analytical techniques are more applicable to the earlier phases of the RTD plan, but withregard to similar issues. For example, in the Acceptability Phase an issue identified is to“Study compatibility between ASAS and ACAS procedures” (AP.D9). Here an analyticalapproach may be applied. In the corresponding issue in the Validation Phase the issue is to“Evaluate the compatibility between ASAS and ACAS procedures during flight simulations”(VP.D9). Here the technique has changed from analytical to real-time simulation.



7.7. Survey techniques

Surveys are primarily used to provide qualitative data to gather the views and synthesiseconclusions. These may be employed as part of the human factors issues in the AcceptabilityPhase to determine help refine the ASAS procedures. The qualitative data collected frompilots and controllers during this phase must be analysed using qualitative techniques thatenable key issues to be identified efficiently. The Validation Phase of the RTD plan is moreconcerned with the measurement of Human Factors issues, such as workload, once theprocedures have been well defined. It may be possible to incorporate similar surveys duringthe course of real-time simulations in order to capture any issues that arise, but this shouldnot be allowed to interfere with the prime data collection aims of the experiment.

7.8. Translating the high-level aims into experiments

This section develops the high-level validation aims further, towards experimentation. Eachof the high-level validation issues identified in section 6.1.11 are defined in more detail interms of their high-level validation objectives, the inputs and outputs of the experiments, andtheir dependency on other activities.

Measure potential benefits & constraints through real-time simulations (VP.B1)

Techniques Real-TimeValidation Exercise Aim The RTD plan does not specify the term

‘benefits’. These benefits are capacity andefficiency.

Inputs Final definition of ASAS applicationincluding HMIDefinitions of traffic samplesPilotsControllersRealistic experimental platform

Outputs (measurements ofmetrics)

Capacity metricsSafety metricsEfficiency metrics

Dependencies All previous work leading to ASASapplication definitions

Conduct a full safety study taking into account human factors (VP.C1)

Techniques Real-time, Fast-time, AnalyticalValidation Exercise Aim Measurement of level of system safety.

Looking for ‘hot-spots’ in the system safety.Inputs Final definition of ASAS application

including HMITo determine the level of safety of a givenASAS application. This experiment is alsodesigned to determine the key areas of


Safety metrics

Dependencies The definition of the procedures and HMIshould be closed prior to this study

Validate ASAS algorithms through real-time simulations (VP.E2).



Techniques Real-timeValidation Exercise Aim To examine all of the possible flows of

control and data issues with regard to thealgorithms to ensure safety in all possiblelogical states.

Inputs Final definition of ASAS applicationincluding HMISimulation using current encounters (asthey are and perturbed).An encounter model to generate all of thepotential encounters that ASAS may face.


Safety metricsTested algorithms

Dependencies The ASAS algorithms should be fully matureprior to this experiment.

Test the compatibility between ASAS and ACAS through real-time simulations (VP.E5).

Techniques Real-timeValidation Exercise Aim To explore all the possible scenarios and

interactions between ASAS and ACAS. Allthe possible flows of control should beexplored to ensure safety in all possiblelogical states.

Inputs Final definition of ASAS applicationincluding HMISimulation using current encounters (asthey are and perturbed).An encounter model to generate all of thepotential encounters that ASAS may face


Safety metrics

Dependencies All previous work leading to ASASapplication definitions.



Validation of the selected ASAS applications through real time simulations (VP. A1) andExperiment and validate selected ASAS applications through flight trials (VP.A1)

Techniques Real-time (and flight-trials)Validation Exercise Aim Final Validation of all the aspects (Human

Factors, HMI, Procedures, Safety)Inputs Final definition of ASAS application

including HMIOutputs (measurements ofmetrics)

Validated ASAS procedures

Dependencies All of the proceeding ASAS applicationdevelopment work and benefitsmeasurement will have contributed to thisstage.

Validate the new airspace boundaries and organisation through simulations (VP.A4)

Techniques Real-time and fast-timeValidation Exercise Aim To ensure that the ASAS system and new

airspace boundaries are compatible andoptimised for maximum safety and capacitygains.

Inputs Final definition of ASAS applicationincluding HMIDefinition of the new airspace boundaries


Safety metricsCapacity metrics

Dependencies New airspace definitions should be finalisedat this point.

Validate the new share of responsibility between pilots & controllers (VP.A3)

Techniques Real-time, analytical, surveyValidation Exercise Aim Find the pilot and controller workload with

current traffic and future traffic under allpossible ASAS situations

Inputs Final definition of ASAS applicationincluding HMIRealistic current and future traffic samplesTrained pilots and controllers


Workload metrics




SSA to demonstrate that the system and their elements, meet the safety objectives andrequirements (VP.C6).

Techniques Fast-timeValidation Exercise Aim Full ASAS system safety study, exploring all

possible outcomes.Inputs Final definition of ASAS application

including HMIRealistic current and future traffic samples


Safety metrics


Validate the ‘New ‘Rules of the Air’ and the airborne separation minima through simulations(VP.D6 & D7).

Techniques Real-timeValidation Exercise Aim Determine new safe separation minima and

rules of the air for the ASAS proceduresInputs Final definition of ASAS application

including HMIRealistic current and future traffic samplesTrained pilots and controllers


Capacity metricsSafety/separation metrics


Measure controller & pilot workloads during real-time simulations (VP.D10) and Evaluatepilot workload during flight trials (VP.D10).

Techniques Real-timeValidation Exercise Aim Find the pilot and controller workload with

current traffic and future traffic under allpossible ASAS situations

Inputs Final definition of ASAS applicationincluding HMIRealistic current and future traffic samplesTrained pilots and controllers


Workload




Evaluate the compatibility between the ASAS and ACAS procedures during flight simulations(VP.D9).

Techniques Real-time and fast-timeValidation Exercise Aim To determine the safety and human factors

issues in the interaction between ASAS andACAS. The interaction must guaranteesafety with unambiguous

Inputs Final definition of ASAS applicationincluding HMI


Safety metrics

Dependencies The ASAS algorithms should be fully matureprior to this experiment.

Experiment and validate the ASAS HMI (VP.E20 to E23).

Techniques Real-time, surveyValidation Exercise Aim Ensure that the HMI allows optimum and

safe operation of ASAS under all conditions(workload, airspace, etc).

Inputs Final definition of ASAS applicationincluding HMIRealistic current and future traffic samplesTrained pilots


Safety metricsWorkload metricsOpinion/survey

Dependencies Definition of HMI should be finalised

7.9. The development of an ASAS application validation plan from the RTDplan

As stated earlier the RTD plan is designed to be applicable to any ASAS application, but inpractice the process is likely to be application specific. There may be areas where thevalidation of a specific component or an issue from one validation activity may be applicableacross several applications, for example the validation of an ASAS HMI may be valid forseveral applications in the same type of airspace. It is not possible within the scope of WP4.1to comment on all of the potential ASAS applications, however application specificrequirements are discussed under the issues taken from the Validation Phase.

The following illustrates the starting point for expanding the RTD plan into a series ofindividual plans for each of the applications. This, together with the detail provided in section6,8, presents the overall validation strategy for the ASAS category. The comments in thissection relate to the example ASAS application ‘time-based sequencing on approach’.

1. ASAS application (operational procedures)The operational procedures for the procedure including communicationsprotocols and the effect on aborted landings need to be clearly defined.

2. New airspace boundariesAirspace boundaries are unlikely to effect the introduction of this procedure.



3. Share of responsibility between pilots and controllersThe share of responsibility will change with more responsibility for separationresting with the pilot during the execution of the procedure. In most of theASAS applications, with the exception of Enhanced Visual Acquisition, moreresponsibility rests with the pilot than was previously the case.

4. New Rules of the Air and the airborne separation minima

5. Controller and pilot training programmesThese are key to delivering the new application safely.

7. ASAS algorithms. These have been defined as a course of the proceedingRTD issues

8. ASAS HMIThe HMI definition and the amount of heads down time is crucial to safety

9. Compatibility of ASAS and ACASThe tuning and interaction between ASAS and ACAS is particularly crucial inthe TMA and lower levels where alert time are shorter and manoeuvres

10. Validate of the ASAS application (see section 6.8)

11. Pilot workloadThe pilot workload during the terminal and approach phases is a crucial factorin the acceptability of the ASAS application

12. Measure the performance of the selected ADS-B architecture measurement ofbenefits and constraints

13. Full safety study



7.10. Developing the high-level validation aims further

The experiment and platform issues need top be developed further for each experimentdefined in section 6.8. Figure 6 shows the link between high-level objectives and metrics.

Figure 6. Relationship between high-level validation objectives and metrics

7.11. RTD issues that have been addressed

The three main projects identified as part of this work that have addressed issues in the RTDplan are EMERTA, INTENT and IAPA.

Project EMERTA [16] performed an initial ASAS safety study. There were no Validationapplied as a course of this work.

Project INTENT [13] sought to identify capacity benefits related to the use of INTENTinformation, and to define the various options for aircraft intent. INTENT applied the MAEVAframework to its work, which was solely concerned with capacity metrics. The work containsboth real-time and fast-time simulations.

Project IAPA, yet to start, will address some of the ASAS/ACAS compatibility issuesidentified as part of the RTD plan.

Other issues that have been addressed are:

1. The definition of an operational concept on ‘Co-operative ATC for Europe’ bythe EUROCONTROL COOPATS initiative;

2. The EUROCONTROL ADS-B cost-benefit analysis starts to address some ofthe initial cost –benefit issues in the Feasibility Phase of the plan.

Neither of the above initiatives contained formal validation exercises.



8. CONCLUSIONS

This section concludes that the MAEVA Validation Framework maps well to the high-levelvalidation activities defined in the EMERALD RTD Plan. However:

Many of the issues in the RTD plan relate to scenario definitions, others that have beenspecified in this document relate to validation activities.

The degree of rigour, and the level of significance, of the plan must be determined for eachphase by the customer and stakeholders.

The feedback path, where validation activities may inform as necessary, the OperationalConcept activity needs to be clear. The process works at present in a relatively informal way.Validation actors in the ASAS domain are normally well connected, if not the identicalindividual/organisation, to the bodies that are responsible for the definition of procedures.

The EMERALD RTD Plan approach may result in long delays if the validation activities arecarried out to late in the process to address any issues that may effect the applicationdesign.

Safety assessments are key to the validation of ASAS applications. There is no one size fitsall approach.

Current ASAS related projects apply the MAEVA guidelines (e.g. INTENT [13])

There are many institutional issues in the RTD plan that are outside the Validation aactivities, e.g. “ Develop an international consensus on Co-operative ATC”. There are otherse.g. Establishing the new share of responsibility between pilots and controllers that will beinformed via validation activities.

For large scale real-time and flight trials cost is a key issue. It must not be a limiting factor invalidation activities.



Annex A. EMERALD RTD Plan

OperationalConcepts

Benefits &Constraints

Safety Assessment ASASOperations andHuman Factors

ASAS Design andAirborne HMI

Transition Issues

User Requirementor Concept Phase

- Define anoperational concepton ‘Co-operativeATC’ for Europe (A1)

- Investigate howaircraft performanceaffects the concept(A5)

- Identify potentialbenefits &constraints (B1)

- Identify the benefitsof early ASASequipage (B3)

- Assess thesuitability of theEATMS airspacestructure for ASASoperations (D1)



OperationalConcepts


Safety Assessment ASASOperations andHuman Factors

ASAS Design andAirborne functions

TransitionIssues

User RequirementAnalysis orFeasibility Phase

- Define and selectASAS applicationsfor Europe (A1)

- Establish the newshare ofresponsibilitybetween pilots &controllers (A3)

- Study the impact onthe airspaceboundaries andorganisation (A4)

- Develop aninternational (ICAO)consensus on ‘Co-operative ATC’ (A2)

- Investigate howaircraft performanceaffects the feasibilityof ASAS applications(A5)

- Evaluate potentialbenefits (capacityand flexibility) &constraints throughfast-time simulationsin various airspaces(B1)

- Conduct an initialcost/benefit analysis(B2)

- Conduct an initialsafety study (C1)

- Investigaterequirements for aground basedconformancemonitor (C2)

- Review ofcandidate systemarchitectures todetermine howsafety objectivesand requirementscan be met.Functions areallocated to systemelements and safetyobjectives andrequirements areapportionedaccordingly (C4)

- Define the airspacestructure for ASASoperations (D1)

- Define the role ofthe ground system inFFAS and theassociated ATMfunctions (D2)

- Study the transitionbetween FFAS andMAS (D3)

- Define the userrequirements for thenecessary tools forthe ground systemfor ASAS operationsin MAS (D4)

- Identify thelimitations on ASASlinked to ACASoperations (D9)

- Define operationalscenarios for theselected ASASapplications (A1)

- Conduct a study onthe airbornearchitecture for theintegration of ASASfunctions (E1 & E6)- Study ASASalgorithms (E2)

- Identify the limitationson ASAS linked toACAS algorithms (E5)

- Define the variousoptions for the aircraftintent (E3)

- Study ASASmonitoring and back-upto achieve requiredcriticality performance(E4)- Study and address theSTDMA issues (E12 toE15)- Study and address theMode S issues (E17 toE19)- Define the ASASrequirements for theADS-B media (E7 toE11)

- Define the userrequirements for ASASHMI (E20 to E23)

- Determine thefeasibility ofASASapplicationsduring thetransition phase(partial ASASequipage) (F1 &F2)



OperationalConcepts


Safety Assessment ASAS Operations andHuman Factors


TransitionIssues

FunctionalRequirement orAcceptabilityPhase

- DefineOperationalprocedures for theselected ASASapplications (A1)

- Identify if theoperationalprocedures reflectcorrectly the newshare ofresponsibility (A3)

- Draft ICAOoperationalstandards for theselected ASASapplications (A2)

- Identify thenecessary changeson the airspaceboundaries andorganisation (A4)

- Evaluatepotential benefits& constraintsthrough fast-timesimulations (B1)

- Conduct a morecomprehensivecost/benefitanalysis (B2)

- Conduct a morecomprehensivesafety study (C1)

- Two FHAs have tobe conducted, onefor the airbornesegment and onefor the ATCsegments (C5)

- Define the ASASprocedures andcontingency procedures(D5 & D8)

- Define the ‘New ‘Rulesof the Air’ and theairborne separationminima (D6 & D7))

- Evaluate controller &pilot workloads on atheoretical basis (D10)

- Study compatibilitybetween ASAS andACAS procedures (D9)

- Refine the operationalscenarios for the selectedASAS applications withthe knowledge of theASAS procedures (A1)

- Define the functionalrequirements for the toolsnecessary for the groundsystem for ASASoperations (D2 & D4)

- Define an airbornearchitecture for theintegration of ASAS whichachieves the requiredcriticality performance andmaintains theindependence of ACAS(E1, E4, E5 & E6)

- Select ASAS algorithmsafter validation through off-line simulations (E2)

- Evaluate the compatibilitybetween ASAS and ACASalgorithms through off-linesimulations (E5)

- Select an option for theaircraft intent (E3)

- Draft ICAO technicalstandards for ASAS (E2 &E3)

- Select an ADS-B mediaarchitecture (E7 to E19)

- Define the functionalrequirements for an ASASHMI (E20 to E23)

- Estimate thesuitable rate ofASAS equipagefor ASASoperationacceptability (F1)

- Determine theacceptability ofASASapplicationsduring thetransition phasein MAS (F2)



OperationalConcepts


Safety Assessment ASAS Operations andHuman Factors


TransitionIssues

ASASDevelopment orPrototypingPhase

- Evaluate the impact ofthe system architecture onsafety (C1)

- FHAs are based on atop-down iterativeapproach in order toanalyse system functionswith more and moredetails, the consequencesof their failures and thepotential mitigating means(C4)

- Develop initialcontroller and pilottraining programmes(D11)

- Develop the toolsnecessary for theground system for ASASoperations (D2 & D4)

- Design an ASASequipment or a suitableairborne architecture forthe integration of ASASwhich maintains ACASindependence (E1, E4,E5 & E6)

- Develop an ASASprototype andimplement ASASalgorithms (E1 & E2)

- Implement the selectedADS-B mediaarchitecture (E7 to E19)

- Design and implementan ASAS HMI (E20 toE23)

- Draft industrystandards for ASAS

- Determine howto provide ADS-B/ASASinformation toequipped aircraftabout non-equipped aircraft(F3)



OperationalConcepts


SafetyAssessment

ASAS Operations andHuman Factors

ASAS Design and Airbornefunctions

TransitionIssues

Experimentationand ValidationPhase

- Validation of theselected ASASapplicationsthrough real timesimulations (A1)

- Validate the newairspaceboundaries andorganisationthroughsimulations(A4)

- Experiment andvalidate selectedASAS applicationsthrough flight trials(A1)

- Validate the newshare ofresponsibilitybetween pilots &controllers (A3)

- Finalise ICAOstandards forselected ASASapplications (A2)

- Measure potentialbenefits &constraints throughreal-timesimulations (B1)

- Re-evaluate thecost/benefitanalysis (B2)

- Demonstrate thebenefits of earlyequipage (B3 &F4)

- Conduct a fullsafety studytaking intoaccount humanfactors (C1)

- SSA todemonstratethat thesystem andtheir elements,meet thesafetyobjectives andrequirements(C6)

- Evaluate the ASASprocedures and contingencyprocedures (D5 & D8)

- Validate the ‘New ‘Rules ofthe Air’ and the airborneseparation minima (D6 & D7))through simulations

- Evaluate the tools necessaryfor the ground system forASAS operations (D2 & D4)

- Measure controller & pilotworkloads during real-timesimulations (D10)

- Evaluate pilot workloadduring flight trials (D10)

- Validate and updatecontroller and pilot trainingprogrammes (D11)

- Evaluate the compatibilitybetween the ASAS and ACASprocedures during flightsimulations (D9)

- Validate ASAS algorithmsthrough real-time simulations(E2)

- Evaluate ASAS algorithmsduring flight trials (E2)

- Test the ASAS equipment orairborne architecture for theintegration of ASAS functions(E1 & E4)

- Test the compatibilitybetween ASAS and ACASthrough real-time simulations(E5)

- Measure the performance ofthe selected ADS-B mediaarchitecture through flight trials(E7 to E19)

- Experiment and validate theASAS HMI (E20 to E23)

- Finalise ICAO technicalstandards for ASAS (E2 & E3)

- Finalise industry standardsfor ASAS



OperationalConcepts


SafetyAssessment

ASAS Operations andHuman Factors


TransitionIssues

ImplementationPhase

- ICAOstandards forASASapplications areapproved (A2)

- The newairspaceboundaries andorganisation arein place (A4)

- Regulations onoperationalprocedures arein place inEurope (A1)

- The potentialbenefits shouldbecome real (B1)

- The constraintsshould be keptminimum (B1)

- Implementationcosts should be aspublicised (B2)

- The safety shouldbe maintained orincreased (C1)

- The ASAS proceduresand contingencyprocedures are clearlydefined (D5 & D8)

- The tools necessary forthe ground system forASAS operations areimplemented (D2 & D4)

- Controller & pilotworkloads are acceptable(D10)

- Controller and pilottraining programmes are inplace (D11)

- ACAS is still anindependent safety tool(D9)

- ICAO technical standardsfor ASAS are approved andcompatible with ACASstandards (E2, E3 & E5)

- Industry standardsapproved by theCertification Authorities

- ASAS equipment orairborne architecture for theintegration of ASASfunctions are suitable for awide range of aircraft (E1 &E4)

- The selected ADS-Bmedia architecture has thesuitable performances (E7to E19)

- Enoughaircraft arefitted withASASequipment(F1)

- ASASapplicationsare practicalin MAS (F2)



Annex B. Issues in the RTD plan requiringspecific validation exercises.

Reference RTD Issue Validation Activities

CP. A1 Define an operational concept on ‘Co-operative ATC’ for Europe COOPATS

CP. A5 Investigate how aircraft performance affects the conceptCP. B1 Identify potential benefits & constraints

CP. B3 Identify the benefits of early ASAS equipage EMERTACP. D1 Assess the suitability of the EATMS airspace structure for ASAS

operations

FP. A1 Define and select ASAS applications for EuropeFP. A3 Establish the new share of responsibility between pilots & controllersFP. A4 Study the impact on the airspace boundaries and organisationFP. A2 Develop an international (ICAO) consensus on ‘Co-operative ATC’FP. A5 Investigate how aircraft performance affects the feasibility of ASAS

applicationsFP. B1 Evaluate potential benefits (capacity and flexibility) & constraints

through fast-time simulations in various airspacesYes

FP. B2 Conduct an initial cost/benefit analysis E’CONTROL

FP. C1 Conduct an initial safety study Yes EMERTAFP. C2 Investigate requirements for a ground based conformance monitorFP. C4 Review of candidate system architectures to determine how safety

objectives and requirements can be met.FP. D1 Define the airspace structure for ASAS operationsFP. D2 Define the role of the ground system in FFAS and the associated

ATM functionsFP. D3 Study the transition between FFAS and MASFP. D4 Define the user requirements for the necessary tools for the ground

system for ASAS operations in MASFP. D9 Identify the limitations on ASAS linked to ACAS operations IAPAFP. A1 Define operational scenarios for the selected ASAS applicationsFP. E1 & E6 Conduct a study on the airborne architecture for the integration of

ASAS functionsFP. E2 Study ASAS algorithmsFP. E5 Identify the limitations on ASAS linked to ACAS algorithms IAPAFP. E3 Define the various options for the aircraft intent INTENTFP. E4 Study ASAS monitoring and back-up to achieve required criticality

performanceFP. E12-E15 Study and address the STDMA issuesFP. E17-E19 Study and address the Mode S issuesFP. E7-E11 Define the ASAS requirements for the ADS-B mediaFP. E20-E23 Define the user requirements for ASAS HMIFP. F1+F2 Determine the feasibility of ASAS applications during the transition

phaseEMERTA

AP. A1 Define Operational procedures for the selected ASAS applicationsAP. A3 Establish the new share of responsibility between pilots & controllersAP.A2 Draft ICAO operational standards for the selected ASAS applicationsAP. A4 Identify the necessary changes on the airspace boundaries and

organisationAP. B1 Evaluate potential benefits & constraints through fast-time simulations YesAP. B2 Conduct a more comprehensive cost/benefit analysis



Reference RTD Issue Validation ActivitiesAP. C1 Conduct a more comprehensive safety study YesAP. C5 Two FHAs have to be conducted, one for the airborne segment and

one for the ATC segmentsYes

AP. D5+D8 Define the ASAS procedures and contingency proceduresAP. D6+D7 Define the ‘New ‘Rules of the Air’ and the airborne separation minimaAP. D10 Evaluate controller & pilot workloads on a theoretical basisAP. D9 Study compatibility between ASAS and ACAS proceduresAP. A1 Refine the operational scenarios for the selected ASAS applications

with the knowledge of the ASAS proceduresAP. D2+D4 Define the functional requirements for the tools necessary for the

ground system for ASAS operationsAP. E1 Define an airborne architecture for the integration of ASAS which

achieves the required criticality performance and maintains theindependence of ACAS

AP. E2 Select ASAS algorithms after validation through off-line simulations YesAP. E5 Evaluate the compatibility between ASAS and ACAS algorithms

through off-line simulationsYes

AP. E3 Select an option for the aircraft intentAP. E2+E3 Draft ICAO technical standards for ASASAP. E7-E19 Select an ADS-B media architectureAP. E20-23 Define the functional requirements for an ASAS HMIAP. F1 Estimate the suitable rate of ASAS equipage for ASAS operation

acceptabilityAP. F2 Determine the acceptability of ASAS applications during the

transition phase in MAS

PP.C1 Evaluate the impact of the system architecture on safetyPP. C4 FHAs are based on a top-down iterative approach in order to analyse

system functionsYes

PP. D11 Develop initial controller and pilot training programmesPP. D2+D4 Develop the tools necessary for the ground system for ASAS

operationsPP. E1 Design an ASAS equipment or a suitable airborne architecture for the

integration of ASAS which maintains ACAS independencePP. E2 Develop an ASAS prototype and implement ASAS algorithmsPP. E7-E19 Implement the selected ADS-B media architecturePP.E20-E23 Design and implement an ASAS HMIPP. ** Draft industry standards for ASASPP. F3 Determine how to provide ADS-B/ASAS information to equipped

aircraft about non-equipped aircraft

VP. A1 Validation of the selected ASAS applications through real timesimulations

Yes

VP. A4 Validate the new airspace boundaries and organisation throughsimulations

Yes

YP. A1 Experiment and validate selected ASAS applications through flighttrials

Yes

VP. A3 Validate the new share of responsibility between pilots & controllers YesVP. A2 Finalise ICAO standards for selected ASAS applicationsVP. B1 Measure potential benefits & constraints through real-time

simulationsYes

VP. B2 Re-evaluate the cost/benefit analysisVP. B3+F4 Demonstrate the benefits of early equipageVP. C1 Conduct a full safety study taking into account human factors YesVP. C6 SSA to demonstrate that the system and their elements, meet the

safety objectives and requirementsYes

VP. D5+D8 Evaluate the ASAS procedures and contingency procedures



Reference RTD Issue Validation ActivitiesVP. D6+D7 Validate the ‘New ‘Rules of the Air’ and the airborne separation

minima through simulationsYes

VP. D2+D4 Evaluate the tools necessary for the ground system for ASASoperations

VP. D10 Measure controller & pilot workloads during real-time simulations YesVP. D10 Evaluate pilot workload during flight trials YesVP.D11 Validate and update controller and pilot training programmesVP. D9 Evaluate the compatibility between the ASAS and ACAS procedures

during flight simulationsYes

VP. E2 Validate ASAS algorithms through real-time simulations YesVP.E2 Evaluate ASAS algorithms during flight trialsVP. E1+E4 Test the ASAS equipment or airborne architecture for the integration

of ASAS functionsVP. E5 Test the compatibility between ASAS and ACAS through real-time

simulationsYes

VP. E7-E19 Measure the performance of the selected ADS-B media architecturethrough flight trials

VP.E20-E23 Experiment and validate the ASAS HMI YesVP. E2+E3 Finalise ICAO technical standards for ASAS

Finalise industry standards for ASAS

CARE ASAS Action The Application of the Validation ... · [1] ASAS Activity 2 Report: Towards a...

Documents

Transcript of CARE ASAS Action The Application of the Validation ... · [1] ASAS Activity 2 Report: Towards a...