Post on 06-Jul-2018
6 March 2014 Page 1
ICAO EUR Region Performance Framework
(ICAO EUR Doc 030)
World ATM Congress
Madrid, 5th March 2014
ICAO European and North Atlantic Office
6 March 2014 Page 2
- Present the context: the ICAO global framework and regional
initiatives
- Describe the ICAO EUR Region performance framework
- Raise awareness for non-regional stakeholders and provide
information to support implementation for regional
stakeholders
- Collect inputs and views from participants
2
Objectives
6 March 2014 Page 3
States in the ICAO EUR Region, responding to ICAO requirements, decided to
elaborate a Regional Performance Framework, to improve the performance of
the aviation system and indentify areas where improvements are possible
To prepare such framework a dedicated Task Force was created
The TF analysed what already in place in/outside the Region – to understand the
context, to make maximum use of what available and avoid duplications
The TF prepared a comprehensive framework with KPAs/KPIs and guidance
material to support implementation
We are at the beginning of the implementation phase
3
Introduction
6 March 2014 Page 4
ICAO Performance Based
Approach
ICAO European and North Atlantic Office
6 March 2014 Page 5
• Endorsement of the global ATM operational concept already at 11th Air Navigation Conference in 2003 and ICAO is requested to:
» develop ATM system requirements
» address interoperability and seamlessness
» define requirements for global AIM
» publish the operational concept
» amend the Global Plan
» develop a performance framework
• States and PIRGs consider the Global Air Navigation Plan for CNS/ATM Systems as a catalyst for change, providing a global safety and interoperability framework while allowing regional or local adaptation to efficiently meet regional and local needs
• States and PIRGs agree on a set of metrics related to KPAs, incorporate these metrics into the performance monitoring process and review their results on a regular basis (EANPG,RASG)
Air Navigation Conference 11 in 2003 How did we start ?
6 March 2014 Page 6
• The Conference unanimously endorsed the Aviation System Block Upgrades (ASBUs) framework introduced by ICAO to set goals in terms of operational improvements on a consensus-driven basis.
• The Conference also endorsed the draft 4th edition of the Global Air Navigation Plan (Doc 9750), as unified planning mechanism, which was approved by the 38th ICAO Assembly in September 2013
• ICAO is tasked to include the key policy principles in the GANP, develop financial policies, SARPS development plan, define a stable and efficient process to update the GANP
• ICAO (through Regional Offices) will provide guidance and assistance on ASBUs implementation to States, establish a group and mechanism for inter-regional cooperation
Air Navigation Conference 12 in 2012 How do we continue
6 March 2014 Page 7
GANP and GASP Advantages
• A globally harmonized and time-wise synchronized set of ATM improvements to direct the work of States, ANSPS, AOs, airports, ICAO and other international and standard making organizations
• A globally agreed ATM baseline through commitment to implement Block 0
• A systematic approach to ATM improvements, a packaged solution that includes standards/guidance, technology, procedures, regulatory framework, business case, performance measurements
6 March 2014 Page 8
• Recommendation 1/15 – Performance monitoring and measurement of air navigation systems
The Committee noted that the implementation of “One Sky” would require the use of a common “language” regarding performance monitoring and measurement. This implies a coordinated approach across States and stakeholders for data provision, collection, storage, protection, dissemination, indicator calculation and use of the results to support the various ATM improvement processes. It was agreed that there was a need to develop a global methodology, to identify metrics and indicators which could be used to allow States and regions to measure and evaluate the effectiveness of their ATM performance initiatives. In doing so, particular attention should be given to avoiding duplication of efforts and to use, to the maximum extent possible, existing arrangements and solutions
That ICAO: a) establish a set of common air navigation service performance metrics supported by guidance material, building on existing ICAO documentation (e.g. Manual on Global Performance of the Air Navigation System (Doc 9883) and the Manual on Air Navigation Services Economics (Doc 9161)); b) promote the development and use of “leading safety indicators” to complement existing “lagging safety indicators” as an integral and key component to drive improvement in performance and in the achieved management of risk; and; c) encourage the early and close involvement of the regulator and oversight bodies in the development, proving of concepts and implementation of the aviation system block upgrades and regional programmes.
AN-Conf/12 Recommendations
6 March 2014 Page 9
• A wider (more regional, if not even global vision) perspective is needed to make even greater gains in capacity and efficiency
• A far-reaching cooperation is necessary in order to support the implementation of facilities and services over larger geographical areas
• Enhanced planning perspectives and a global framework for performance measurement is required to support the implementation activities
• States and all aviation stakeholders should – work together towards a common vision (global ATM Operational Concept)
– use a common planning framework
• Global Air Navigation Plan (GANP), the regional air navigation plans and several other documents and tools (e.g. European ATM Masterplan)
• Global Aviation Safety Plan (GASP)
– utilize harmonised performance objectives (targets, metrics, indicators)
• Interoperability and harmonization are key factors in order to ensure the required improvements for the future
The wider ATM perspective
6 March 2014 Page 10
ICAO EANPG
• Ensure that air navigation system development plans and actions within the EUR Region remain coherent and compatible with the ICAO Strategic Objectives, with the ICAO global plans (especially GANP and ASBUs, parts of GASP), with those of the adjacent ICAO Regions and with other world-wide provisions.
• Manage the EUR Air Navigation Plan (Performance directives and associated requirements for
facilities and services established through regional air navigation agreements, in support of the global air navigation infrastructure) and facilitate the implementation of the international operational requirements contained therein.
• Promote and facilitate the harmonisation and co-ordination of the air navigation related programmes (e.g. SESAR, State ATM Corporation modernisation program, NextGen) of other international organisations such as the European Commission, the European Civil Aviation Conference (ECAC), Eurocontrol and the Interstate Aviation Committee of the Commonwealth of Independent States (IAC/CIS).
• Ensure the interoperability of the EUR CNS/ATM systems with those of adjacent regions and in line with the ICAO GANP.
• Assist States in their planning and implementation efforts, if and when required.
6 March 2014 Page 11
Performance Based Approach
• ICAO Performance Based Approach is based on the following three principles:
– Strong focus on desired/required results
– Informed decision making, driven by the desired/required results
– Reliance on facts and data for decision making
6 March 2014 Page 12
• Instead of prescribing solutions, desired/required performance is specified.
• Management attention is shifted from a resource and solution centric view (how will we do it) towards a primary focus on desired/required (performance) results (what is the outcome we are expected to achieve).
• This implies finding out – what the current performance situation is, – what the most appropriate results should be, – clarifying who is accountable for achieving those
results.
Focus on desired/required results
6 March 2014 Page 13 ICAO EUR/NAT Office COG PERF TF Workshop 13
• Performance objectives: Specific, Measurable, Achievable, Relevant & Timely (SMART) objectives defined to satisfy ATM community expectations.
• Performance indicators: to measure achievement of performance
objective • Performance targets: value of performance indicators that need to be
reached/exceeded to fully achieve performance objective. • Performance gaps: Performance indicators compared against performance
targets identifying gaps • Performance assessment metrics: determine which data needs to be
collected to calculate values of performance indicators
Performance terms
6 March 2014 Page 14 ICAO EUR/NAT Office COG PERF TF Workshop 14
Expectations
KPA
Objectives
Indicators Quantitative
Targets ... Objective is met when indicators meet or exceed targets
6 March 2014 Page 15 ICAO EUR/NAT Office COG PERF TF Workshop 15
• Assess current performance: establish baseline & track progress towards performance objectives
• Identification/diagnosis performance gaps: Performance gaps denote current/anticipated mismatch (unfavorable) between current/planned performance and performance target.
• Addressing performance gaps: Operational improvements in terms of changes to ATM system that are on transition path towards Global ATM Operational Concept and result in a direct performance enhancement.
• Developing list of options for operational improvements: Operational improvements made possible by technical systems, human factors, procedure and industrial enablers in terms of feasibility, timing, cost and impact on operational change. Global Air Navigation Plan (Doc 9750) is one of the sources, which can be used.
• Building/updating transition roadmaps: Performance gaps identify the affected performance objectives. Categorization of operational improvements according to performance objectives leads to development of appropriate shortlist of candidate solutions. If the solution is not included in the transition roadmap, then it should be selected on the basis of maturity or research, if no mature solution is available.
Performance assessment
6 March 2014 Page 16
• Understanding Performance is key for successful transition from past and current to the future
• Working under one common umbrella – Performance based transition
– Continued operational improvements
– Toward a common vision
– Using the ICAO framework
• Using the 3-step transition approach – Set objectives/metrics
– Identify and understand performance gaps
– Select and plan Operational Improvements
• Supporting the goal of a more seamless and performance based Global Air Navigation System
In summary
6 March 2014 Page 17
Existing processes in the Region
ICAO European and North Atlantic Office
6 March 2014 Page 18
SES II - Performance in RP2 (2015-2019) – Binding performance targets in 4 key
performance areas (Safety, Environment, Capacity and Cost-Efficiency)
– Mandatory implementation of a financial incentive scheme for the target achievement on the KPA Capacity
– Performance planning on FAB level
– 'gate-to-gate' approach covering the entire chain of air navigation services
• Target setting on terminal charges as of 2015 (Union-wide target setting foreseen as of 2017)
– Network Manager integral part of the scheme
– Clear link of new investments and major overhaul of systems to the EU ATM Master Plan
EU Performance scheme
6 March 2014 Page 19
Performance Review Report – analysis of the performance of the ATM systems in the KPA of Safety, Capacity, Environment and Cost-efficiency in Europe
US-Europe comparison reports – analysis of the operational and economic ATM performance
ATM Cost-Effectiveness Benchmarking Report – analysis of cost-effectiveness and productivity in 37 European ANSPs
- Comparison of Air Traffic Management-related Operational Performance 2008-2012
- Comparison of ANS cost-efficiency trends 2002-2011
- Joint publications influenced ICAO and CANSO work on global performance benchmarking
Performance reports
6 March 2014 Page 20
ICAO EUR Region Performance Framework
ICAO European and North Atlantic Office
6 March 2014 Page 21
• Based on clear principles: • Produce improvements to the aviation system • Avoid duplication of efforts and make maximum use of existing arrangements • As far as possible be applicable also in the NAT Region • Consider impact on resources • Follow a pragmatic and stepwise approach
WHY • To elaborate a Performance Framework for the EUR Region, to improve
the performance of the aviation system and indentify areas where improvements are possible
WHEN • Decision at COG 47 (Rostov on Don, June 2010)
HOW
Should also represent
• Regional input to ICAO HQ activities in developing metrics for global use
• Support the future developments of EU Performance Scheme
ICAO EUR Performance Framework WHO ?
6 March 2014 Page 22
Pragmatic approach in developing the proposal
Input ICAO framework (ANC 12, GA38, etc.) EU-ECTL (Single European Sky and Performance Scheme) FAA Russian Federation ICAO EUR Workshops (Rome 2011, Bishkek 2013)
To assess the Regional performance and to identify areas where improvements are possible
Output
Development of a comprehensive framework Identification of 6 KPAs out of 11 ICAO KPAs Definition of focus areas, objectives and KPIs Definition of Processes, Roles and Responsibilities Guidance material
Safety
Access and equity
Efficiency
Capacity
Environment
CostEffectiveness
Participation byATM
community
Global Interoperability
Flexibility
Security
Predictability
ICAO EUR Performance Framework WHAT ?
6 March 2014 Page 23
• ICAO EUR Region Performance Framework Document (ICAO EUR Doc 030) was approved by EANPG and COG in 2013 and published in English and Russian language.
• Performance Framework document describes:
INTRODUCTION
– BACKGROUND INFORMATION
– RELATIONSHIPS WITH ICAO GLOBAL DEVELOPMENTS AND REGIONAL INITIATIVES
PART A
– SCOPE
– ROLES AND RESPONSIBILITIES
– KPA/KPI/METRICS
– MONITORING, REPORTING AND REVISION PROCESS AT REGIONAL/NATIONAL LEVEL
PART B
– KPAs
– TEMPLATES FOR THE REGIONAL PERFORMANCE REVIEW REPORT
ANNEX 1: TERMINOLOGY
ANNEX 2: REFERENCE MATERIAL
Performance Framework Document
6 March 2014 Page 24
KPA OBJECTIVES FOCUS AREAS INDICATORS
SAFETY
Ensure safety continuousimprovement through reductionof ATM related safetyoccurrences and implementation of uniformsafety standards
Effectiveness of Safety Management (SafetyMaturity Questionnaire)
Level of State Safety/Just culture (SafetyCulture Questionnaire)
Adoption of an harmonized occurrences severity classification methodology
CAPACITY Capacity meets demand for en-route and at airports
En-route ATFM Delay Average en-route ATFM delay generated by airspace volume
Airport ATFM Delay Average ATFM delay per flight in the main airports (to be identified by States)
EFFICIENCY Ensure users may use most efficient routes
HorizontalFlight Efficiency
Average horizontal en route flight efficiency (length of the en route part of the actual trajectory/last flight planned route vs great circle)
ENVIRONMENT Contribute to the protection of environment (fuel/CO2 emissions reduction)
CO2 emissions related to inefficiencies in route extension
COST EFFECTIVENESS
Contribute to optimization of costs for ANS
ATCO Productivity IFR Flights (en-route) per ATCO hour duty
IFR flight hours per ATCO hour on duty
IFR movements per ATCO hour on duty
PARTICIPATION BY ATM COMMUNITY
Ensure States' participation toRegional planning and implementation activities
Level of participation to meetings
Level of responses to planning activities
Level of provision of performance results
KPAs-KPIs summary table
6 March 2014 Page 25
KPA Safety
Objective Ensure the continuous improvement of safety through the reduction of ATM related safety occurrences and the implementation of uniform safety standards.
Indicators - Effectiveness of Safety Management (EoSM)
- Level of State Just Culture (JC)
- Application of a common methodology for classification of occurrences in terms of risk severity
Safety KPA – Overview
6 March 2014 Page 26
• Effectiveness of Safety Management (EoSM) – Based on the annual completion of two questionnaires
(State level and ANSP level)
• Level of State Just Culture (JC) – Based on the annual completion of two questionnaires
(State level and ANSP level)
• Application of RAT methodology for Severity Classification – Objective is to motivate States to use a common
methodology for the severity assessment of safety occurrences
– Benefit: harmonisation of occurrence statistics
Safety KPA – Nature of indicators
6 March 2014 Page 27
Safety KPA – Conclusions
• Three Indicators – These are leading indicators
• measuring precursors to improved safety – And not lagging indicators
• which are directly related to safety outcome
• Application – Have already been used by many States since 2012 during SES
Reference Period 1 (RP1) and now reinforced for RP2 – Under ICAO umbrella geographical extension to remaining States of the
EUR Region • Complex subject, but detailed guidance material is available
– Fully described in EASA documentation – ICAO guidance material summarises and explains the EASA
documentation
6 March 2014 Page 28
KPA Capacity
Objective Ensure that Air Navigation Service capacity meets demand in en-route airspace and at airports
Indicators - Average ATFM delay per flight generated by the airspace volume (en-route)
- Average ATFM delay per flight in the main airports (to be identified by States in advance and based on the regional relevance)
Capacity KPA – Overview
6 March 2014 Page 29
• Re-use of ATFM delay indicators as defined in SES Performance Scheme Regulation – Measures the delay generating
entities (airspace and airports which are bottlenecks), not the penalised entities (flights)
• Covers majority of States where capacity is an issue – ATFCM Area
• Relevance in remainder of EUR Region to be determined – Capacity often not (yet) an issue
or only a local issue
ATFM regulation
requested by ACC
Arrival
Airport Departure
Airport
ACC anticipating
capacity shortfall
NM
Aircraft held at
dep. airport
Capacity KPA – Application
6 March 2014 Page 30
• Indicators measure – The location where the problem is, not where the delay is
taken (departure airport) – Performance of airspace volumes and airports, not flights
• Despite the expression as a value “per flight”
– Within the Capacity KPA • Demand/capacity imbalance • Not capacity itself
• Limitations • Not designed to measure excess capacity • No data if airspace or airport does not participate in a centralised
ATFCM process
Capacity KPA – Conclusions
6 March 2014 Page 31
KPA Efficiency
Objective Ensure that users [can?] use the most efficient routes – focussing on the horizontal flight-efficiency
Indicator Average horizontal en route flight efficiency, defined as the difference between the length of the en route part of the actual trajectory (where available) or last flight planned route and the great circle. Note: specificities shall be considered for flights longer than 1000 nm where the optimum
could differ from the great circle (wind optimal routes, etc.).
Efficiency KPA – Overview
6 March 2014 Page 32
• Re-use of indicator as defined in SES Performance Scheme Regulation – The average horizontal en route flight efficiency of [(a) the actual
trajectory, or (b) the last filed flight plan trajectory], defined as follows: • (i) the indicator is the comparison between the length of the en route part
of [(a) the actual trajectory derived from surveillance data, or (b) the last filed flight plan trajectory] and the corresponding portion of the great circle distance, summed over all IFR flights within or traversing the European airspace;
• (ii) ‘en route’ refers to the distance flown outside a circle of 40NM around the airports;
• (iii) where a flight departs from or arrives at a place outside the European airspace, only the part inside European airspace is considered.
Note: the SES indicator is expressed as a percentage: Sum of extra distance divided by the sum of corresponding portions of great circle distances
Efficiency KPA – Origin of indicator
6 March 2014 Page 33
Indicator option B
requires surveillance
data
Indicator option A
City pair distance (Great circle distance)
Shortest Route
Shortest Available Route
Last Filed FPL
Actual Trajectory
Business need: “Get from A to B” = direct route
Planning Operations Leng
th o
f Tra
ject
ory
Route Network Design
Route & Airspace Availability
Awareness and Choice
ATC Separation Fragmentation
Wind-optimum Cost-optimum
Difference due to planning limitations
Ref
eren
ce d
ista
nce
Extr
a di
stan
ce
Ref
eren
ce d
ista
nce
Extr
a di
stan
ce
Based on information known in advance
Tactical decisions based on updated information
Efficiency KPA – Nature of indicator
6 March 2014 Page 34
• Indicator – Measures actual (flight plan data or radar data), achieved,
and additional distance flown within an airspace volume (eg within a State)
– Can cope with overflights
– Additional distance can be computed separately for flights shorter and longer than 1000 nm
– Results are geographically aggregatable
– Represents the overall outcome of several influencing factors, for which different Stakeholders are accountable
Efficiency KPA – Conclusions
6 March 2014 Page 35
KPA Environment
Objective Contribute to the protection of the environment – focussing on fuel savings and CO2 emission reductions
Indicator CO2 emissions deriving from inefficiencies in flight efficiency (conversion of additional distance into CO2 emissions based on standard values formula) .
Environment KPA – Overview
6 March 2014 Page 36
• Reference to excess fuel consumption: Total additional distance flown in the airspace volume (i.e. State) multiplied by a standard fuel consumption factor (value State-specific)
• Reference to excess CO2 emission: Reference to excess fuel consumption multiplied by 3.15 (net carbon content of kerosene)
Trajectory
Actual distance
Achieved distance
Additional distance
Standard fuel consumption
per km
Net carbon content of kerosene (3.15)
Reference excess fuel consumption
Reference excess CO2 emission
Environment KPA – Definitions
6 March 2014 Page 37
• Indicator – Has low data requirements and is therefore easy to
implement • Because mostly based on “additional distance” already computed
under the Efficiency KPA
– Publishes an approximation of excess CO2 emission, resulting from horizontal flight inefficiency
• The optimum indicator value is not equal to zero
• Hence the absolute value of the indicator should not be interpreted as representing the CO2 emissions caused by ANS.
• Indicator can be calculated with IFSET or other tools and should be used for trend analysis only
Environment KPA – Conclusions
6 March 2014 Page 38
KPA Cost effectiveness
Objective Contribute to optimize the cost for air navigation services
Indicators - IFR flights (en-route) per ATCO hour on duty
- IFR flight hours (en-route) per ATCO hour on duty
- IFR movements (airport) per ATCO hour on duty
Cost Effectiveness KPA – Overview
6 March 2014 Page 39
• SES indicators not re-usable because dependent on States’ participation in EUROCONTROL Route Charges System
• Asking for extensive disclosure of economic and financial information not deemed realistic for the initial EUR Region performance framework
• Focus is on controller (ATCO) productivity:
– Staff cost represents the main cost for service provision – ATCO staff cost represents the most important share in
staff cost – ATCO productivity can be expressed in non-financial terms – Existing data flow covers many EUR Region States already
Cost Effectiveness KPA – Origin of indicator
6 March 2014 Page 40
• Indicators – Focus on an important component of ANSP costs
• However with the limitation that working hours per ATCO, employment cost per ATCO and support cost are not covered
– Re-use data reporting already in place for many States • Also reuse existing data definitions and terminology
– Have relatively simple reporting requirements • Only traffic volume and ATCO hours on duty
– Avoid entering into financial information disclosure issues for the initial implementation of the framework
Cost Effectiveness KPA – Conclusions
6 March 2014 Page 41
KPA PARTICIPATION BY ATM COMMUNITY
Objective Ensure States participation to regional planning and implementation activities so that air navigation system development plans and action within the EUR/NAT Region remain coherent and compatible with those of the adjacent ICAO Regions and with the ICAO global plan and world-wide provisions.
Indicators - Level of active participation by States and international organisations
- Level of responses to State Letters
- Level of provision of requested information or performance results from aviation stakeholders
ATM Community Participation KPA – Overview
6 March 2014 Page 42
• ICAO Manual on Global Performance of the Air Navigation System (Doc 9883) KPA - Participation by the ATM community There are currently no common indicators obtained and some stakeholders are attempting to use the following indicators:
– number of yearly meetings covering planning, implementation and operation, and covering a significant estimated proportion (e.g. 90 per cent) of the whole of the regional aviation activity
– number of yearly meetings for planning
– number of yearly meetings for implementation
– number of yearly meetings for operations
• The ATM community is defined in Appendix A of the Global Air Traffic Management Operational Concept (Doc 9854) : It comprises (in alphabetical order) the aerodrome community, the airspace providers, the airspace users, the ATM service providers, the ATM support industry, the International Civil Aviation Organization (ICAO), the regulatory authorities and States
ATM Community Participation KPA – Origin
6 March 2014 Page 43
ATM Community Participation KPA – Conclusions • Three Indicators will be monitored and evaluated by the ICAO
EUR/NAT Office • Level of participation of States and international organisations to planning and
implementation meetings (e.g. EANPG or its contributory groups with a focus on regional coordination/implementation aspects like Aeronautical Fixed Services Group, Air Traffic Management Group, Frequency Management Group, Meteorology Group, Route Development Group, Regional Performance Framework implementation activities)
• Level of responses to State Letters asking for information on planning and implementation aspects (e.g. Completed Air Navigation Report Forms ARNF, State Letters which assess the status of planning and implementation of key ASBU modules, State Letters concerning the amendment to Regional Supplementary Procedures, Reports on the environmental benefits that were derived from the implementation of operational improvements)
• Level of provision of performance results from States for the Regional Performance Review Report (RPRR)
• Crucial support mechanism for PIRGs (EANPG) to monitor planning and implementation activities in the ANP
• Important for Air Navigation Report Forms (ANRFs) and ASBU monitoring
6 March 2014 Page 44
Implementation and next steps
ICAO European and North Atlantic Office
6 March 2014 Page 45
Roles and Responsibilities in the functioning of the ICAO EUR Performance Framework – EANPG – COG and contributory Bodies – ICAO EUR Secretariat – States
Processes at Regional and National level
Roles and Responsibilities for implementation
6 March 2014 Page 46
Process (Regional – National)
6 March 2014 Page 47
• EANPG 55 (2013) concluded to launch the implementation phase – Collect data (2014)
– Monitor performance
– Fill the gaps in the system to improve
– Report performance results in 2015
TF is working to support States and Stakeholders in this implementation
– Circulation of a questionnaire
– Identification of focal points
– Awareness campaign - Meetings and workshops (especially in the Eastern part of the EUR Region)
Implementation of the Framework
6 March 2014 Page 48
Implementation of the Framework
All material is on the ICAO EUR/NAT Office, Paris website http://www.paris.icao.int/documents_open_meetings/subcategory.php?id=169
http://www.paris.icao.int/documents_open/files.php?subcategory_id=170
Next workshop in Baku (Azerbaijan) 10 - 11 April 2014 !!!
ICAO Regional Performance Framework Workshop
(State Letter invitation EUR/NAT 14-0109.TEC, from 10 February 2014)
6 March 2014 Page 49
Conclusions
• ICAO EUR Region has defined a performance framework aimed at monitoring/reporting performance
• Objective is to identify areas where improvements are possible
• Built on existing ICAO documents and regional initiatives
• Proposed as a good practice towards ICAO HQ and other Regions
• This presentation is available at: http://www.paris.icao.int/documents_open_meetings/subcategory.php?id=169
6 March 2014 Page 50
Thank You