Machine Learning; forecasting safety risk and performance · Machine learning NATS Private Learns...

16
Machine Learning; forecasting safety risk and performance Dr Claire Le Cras NATS Analytics NATS Private

Transcript of Machine Learning; forecasting safety risk and performance · Machine learning NATS Private Learns...

Page 1: Machine Learning; forecasting safety risk and performance · Machine learning NATS Private Learns from previous data and computations to predict decisions and results for new, unseen

Machine Learning; forecasting safety risk and performance

Dr Claire Le Cras

NATS Analytics

NATS Private

Page 2: Machine Learning; forecasting safety risk and performance · Machine learning NATS Private Learns from previous data and computations to predict decisions and results for new, unseen

Machine learning to predict safety events

Risk Analysis Tool (RAT) is a methodology used to classify safety related occurrences in the Air Traffic Management (ATM) domain. Events which involve a loss of standard separation between aircraft and for which an air navigation service provider (ANSP) had some contribution are those which score what is referred to as "RAT ATM ground points". ANSPs try to minimise the occurrence of all safety related occurrences, whether RAT-scoring or not.

There are a small number of events making the operation relatively safe but what can we do to improve/enhance the operation and make it more resilient?

Short term forecasting

Long term forecasting

NATS Private

Page 3: Machine Learning; forecasting safety risk and performance · Machine learning NATS Private Learns from previous data and computations to predict decisions and results for new, unseen

Part 1: Short term forecasts

Last year we introduced machine learning to help us predict the likelihood of potential safety events – such as aircraft level busts, or airspace infringements – in our London Terminal Control operation.

Complex issue with possible links to many variables such as:

• Traffic volume

• Airport runway direction

• Weather conditions

• And more!

NATS Private

Page 4: Machine Learning; forecasting safety risk and performance · Machine learning NATS Private Learns from previous data and computations to predict decisions and results for new, unseen

Machine learning

NATS Private

Learns from previous data and computations to predict decisions and results for new, unseen data.

Can produce models that can analyse bigger, more complex data and deliver faster (and often times more accurate) results – even on a very large scale.

https://cdn.infoq.com/statics_s2_20170411-0445/resource/articles/apache-spark-machine-learning/en/resources/fig1.jpg

Page 5: Machine Learning; forecasting safety risk and performance · Machine learning NATS Private Learns from previous data and computations to predict decisions and results for new, unseen

Random Forest

NATS Private 5

https://d2wh20haedxe3f.cloudfront.net/sites/default/files/random_forest_diagram_complete.png

Fits a number of decision tree classifiers on various sub-samples of data and uses averaging / majority vote to predict classification.

Improved accuracy vs. standard decision tree and helps control over-fitting.

Page 6: Machine Learning; forecasting safety risk and performance · Machine learning NATS Private Learns from previous data and computations to predict decisions and results for new, unseen

Weekly risk forecast

NATS Private

Date Risk

23/04/2018 Medium

24/04/2018 Low

25/04/2018 Medium

26/04/2018 Medium

27/04/2018 Low

28/04/2018 High

29/04/2018 Medium

Since March 2017, we’ve been using this data to present a weekly forecast to our colleagues at London Terminal Control that can alert them if our modelling suggests corrective action could avoid a potential issue days before it might occur.

Note: example output only

Page 7: Machine Learning; forecasting safety risk and performance · Machine learning NATS Private Learns from previous data and computations to predict decisions and results for new, unseen

NATS Private

Page 8: Machine Learning; forecasting safety risk and performance · Machine learning NATS Private Learns from previous data and computations to predict decisions and results for new, unseen

NATS Private

Aircraft level bust, airspace infringement of Air Traffic Controller overload.

Page 9: Machine Learning; forecasting safety risk and performance · Machine learning NATS Private Learns from previous data and computations to predict decisions and results for new, unseen

Black box

Aviation – extremely secure device designed to provide investigators with highly factual information about any anomalies that may have led to incidents during a flight.

Machine learning – data goes in and an answer comes out, what happens in the box is unknown.

NATS Private

?

Page 10: Machine Learning; forecasting safety risk and performance · Machine learning NATS Private Learns from previous data and computations to predict decisions and results for new, unseen

Opening the box

By understanding the decisions the algorithm is making and the factors that contribute to them we can be better informed in our own decision making.

Depending on the situation we could reduce risk by:

• Applying regulations on traffic flows

• Splitting bandboxed sectors

• Having an extra controller on hand

• Generally being aware of the situation

NATS Private

Page 11: Machine Learning; forecasting safety risk and performance · Machine learning NATS Private Learns from previous data and computations to predict decisions and results for new, unseen

Part 2: Long term forecasts

A holistic model to help us better understand the challenge of rising growth in aviation traffic and the impact it could have on our operation.

This holistic model sees us take our traffic predictions up to 2024 using bespoke algorithms and historic data from the operation. We apply machine learning to forecast the potential impact of the rise in traffic per sector on delays, safety and the environment if the current capacity and ways of working remained the same.

NATS Private

Page 12: Machine Learning; forecasting safety risk and performance · Machine learning NATS Private Learns from previous data and computations to predict decisions and results for new, unseen

Long term forecasts

NATS Private

Note: example output only

Better performance

Worse performance

Page 13: Machine Learning; forecasting safety risk and performance · Machine learning NATS Private Learns from previous data and computations to predict decisions and results for new, unseen

Benefit assessments

To adjust the forecast to take account of future planned projects we need an estimate of the percentage change on the safety performance of projects.

In early stages estimates are constructed with project teams but Air Traffic Controllers are felt best placed to relate the changes in tasks to their ability to control effectively so final assessment is from them.

Percentage changes are adjusted to take into account the area affected by the project as well as the frequency of relevant events.

NATS Private

Page 14: Machine Learning; forecasting safety risk and performance · Machine learning NATS Private Learns from previous data and computations to predict decisions and results for new, unseen

Long term forecasts

NATS Private

Note: example output only

Better performance

Worse performance

Page 15: Machine Learning; forecasting safety risk and performance · Machine learning NATS Private Learns from previous data and computations to predict decisions and results for new, unseen

Long term forecasts

• Helps give us the ‘big picture’ of what to expect

• Allows us to drill down and take a closer look at how our ongoing and upcoming transformation projects could have an individual effect on efficiency, safety and the environment.

• Able to model varying timings and scope for projects, and optimise decision-making to help meet the future regulatory targets.

• Could be used to make the case to key policy and decision makers in the aviation industry, and to our customers.

NATS Private

Page 16: Machine Learning; forecasting safety risk and performance · Machine learning NATS Private Learns from previous data and computations to predict decisions and results for new, unseen

Any Questions?

NATS Private


Introducing Machine Learning · 4 ntroducing Machine Learning How Machine Learning Works Machine learning uses two types of techniques: supervised learning, which trains a model on

Introducing Machine Learning · 4 ntroducing Machine Learning How Machine Learning Works Machine learning uses two types of techniques: supervised learning, which trains a model on

NATS Anytime

NATS Anytime

Machine Learning and Azure Machine Learning

Machine Learning and Azure Machine Learning

MathWorks - Introducing Machine Learning · 4 ntroducing Machine Learning How Machine Learning Works Machine learning uses two types of techniques: supervised learning, which trains

MathWorks - Introducing Machine Learning · 4 ntroducing Machine Learning How Machine Learning Works Machine learning uses two types of techniques: supervised learning, which trains

Machine Learning with MATLAB · 2 Agenda Machine Learning Overview Machine Learning with MATLAB –Unsupervised Learning Clustering –Supervised Learning Classification Regression

Machine Learning with MATLAB · 2 Agenda Machine Learning Overview Machine Learning with MATLAB –Unsupervised Learning Clustering –Supervised Learning Classification Regression

¿Qué es Machine Learning? Usos de Machine Learning.

¿Qué es Machine Learning? Usos de Machine Learning.

Learning with large datasets Machine Learning Large scale machine learning.

Learning with large datasets Machine Learning Large scale machine learning.

NATS & IoT

NATS & IoT

Masters of Craft: Programming - Machine Learning : Research · Masters of Craft: Programming - Machine Learning: Research Janet Choi. Machine Learning Machine Learning is a trend

Masters of Craft: Programming - Machine Learning : Research · Masters of Craft: Programming - Machine Learning: Research Janet Choi. Machine Learning Machine Learning is a trend

Machine Learning for NLP - Ethics and Machine Learning

Machine Learning for NLP - Ethics and Machine Learning

Machine Learning with MATLAB - … · 2 Agenda Machine Learning Overview Machine Learning with MATLAB –Unsupervised Learning Clustering –Supervised Learning Classification

Machine Learning with MATLAB - … · 2 Agenda Machine Learning Overview Machine Learning with MATLAB –Unsupervised Learning Clustering –Supervised Learning Classification

Machine Learning FabSpace March 2018 - irit.fr Learning FabSpace March 2018... · Machine Learning 5 Machine Learning • Supervised : Train / test • Predictiveanalysis Machine

Machine Learning FabSpace March 2018 - irit.fr Learning FabSpace March 2018... · Machine Learning 5 Machine Learning • Supervised : Train / test • Predictiveanalysis Machine

Machine learning for neuroimaging with scikit-learn · Machine learning for neuroimaging with scikit-learn. ... a Python machine learning library, ... Abraham et al. Machine learning

Machine learning for neuroimaging with scikit-learn · Machine learning for neuroimaging with scikit-learn. ... a Python machine learning library, ... Abraham et al. Machine learning

Gina & Nats

Gina & Nats

Fast Track Machine Learning Part 1 (Machine Learning Overview) - Types of Machine Learning Problems

Fast Track Machine Learning Part 1 (Machine Learning Overview) - Types of Machine Learning Problems

An Introduction to Machine Learning - GitHub Pages · Introduction Machine learning The machine learning process What’s machine learning History Supervised learning Non-supervised

An Introduction to Machine Learning - GitHub Pages · Introduction Machine learning The machine learning process What’s machine learning History Supervised learning Non-supervised

Bayesian Methods for Machine Learning - Machine Learning (Theory)

Bayesian Methods for Machine Learning - Machine Learning (Theory)

NATS Unclassified NATS overview. NATS Unclassified What do we do? Slide 2.

NATS Unclassified NATS overview. NATS Unclassified What do we do? Slide 2.

Methods MACHINE LEARNING FROM MACHINE LEARNING TO …

Methods MACHINE LEARNING FROM MACHINE LEARNING TO …

Tutorial: Deep Reinforcement Learning - Machine Learning ...hunch.net/~beygel/deep_rl_tutorial.pdfTutorial: Deep Reinforcement Learning - Machine Learning ...

Tutorial: Deep Reinforcement Learning - Machine Learning ...hunch.net/~beygel/deep_rl_tutorial.pdfTutorial: Deep Reinforcement Learning - Machine Learning ...