Quality Assessment of the National Convective Weather Forecast
Estimating Flow Rates in Convective Weather ... - ATM Seminar · Estimating Flow Rates in...
Transcript of Estimating Flow Rates in Convective Weather ... - ATM Seminar · Estimating Flow Rates in...
Estimating Flow Rates in Convective
Weather: A Simulation-Based Approach
20 June 2019
James Jones, Yan Glina
DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited.
ATM Reinforcement Learning - 2
JJ 6/20/2019
This material is based upon work supported by the National
Aeronautics and Space Administration under Air Force Contract No.
FA8702-15-D-0001. Any opinions, findings, conclusions or
recommendations expressed in this material are those of the
author(s) and do not necessarily reflect the views of the National
Aeronautics and Space Administration.
© 2019 Massachusetts Institute of Technology.
Delivered to the U.S. Government with Unlimited Rights, as defined
in DFARS Part 252.227-7013 or 7014 (Feb 2014). Notwithstanding any
copyright notice, U.S. Government rights in this work are defined by
DFARS 252.227-7013 or DFARS 252.227-7014 as detailed above. Use
of this work other than as specifically authorized by the U.S.
Government may violate any copyrights that exist in this work.
ATM Reinforcement Learning - 3
JJ 6/20/2019
• Collaborative Trajectory Options Program (CTOP) assigns delay and/or reroutes around one or more Flow Constrained Area-based airspace constraints in order to balance demand with available capacity
• NASA’s Integrated Demand Management (IDM) program is exploring ways to use CTOP to precondition demand into time-based metering programs at airports in the Northeast United States
• Estimates from strategic decision support systems (TFMS) may be inconsistent with delivery capability of tactical decision support systems (TBFM)
• Good estimates of airport/airspace capacity are needed to effectively control demand to the appropriate levels
• Proposed Approach
– Leverage reinforcement learning and integer programming to estimate airport and terminal airspace capacity
– Use fast-time simulation to evaluate performance of algorithms
Background and Approach
TFMS = Traffic Flow Management System TBFM= Time-Based Flow Management
ATM Reinforcement Learning - 4
JJ 6/20/2019
• If actual capacity exceeds planned arrival rate: may under-deliver
• Incur costs due to unused airspace capacity and unnecessary ground delays
Under-delivery
t
Capacity
UnnecessaryGround Delay
Cplanned
Cactual
ATM Reinforcement Learning - 5
JJ 6/20/2019
• If actual capacity is less than the planned arrival rate: may over-deliver
• May need tactical intervention (e.g. holding, miles-in-trail restrictions)
• Incur costs due to airborne delay, diversion, …
Over-delivery
t
Capacity
Cplanned
Cactual
Airborne Delay
ATM Reinforcement Learning - 6
JJ 6/20/2019
• When we over-deliver, the achieved throughput serves as a decent initial estimate for capacity (otherwise we would not be holding)
• This estimate is performed under operationally undesirable conditions, however, so it may not be sustainable
Baseline Capacity
t
Capacity
Cplanned
Cactual
Airborne Delay
ATM Reinforcement Learning - 7
JJ 6/20/2019
t
Capacity
• If we can adjust the Cplanned to produce comparable throughput while reducing holding to a manageable level, then the planned rate serves as a good estimate for the true capacity
Optimization Goal
Optimized Capacity Estimate
Cplanned
Cactual
ATM Reinforcement Learning - 8
JJ 6/20/2019
Example Traffic Management Initiative for this Paper: Airspace Flow Program (AFP)
• AFPs reduce demand by issuing ground delays for
flights planned to cross Flow Constrained Areas (FCAs)
– Acceptance rate = # aircraft that can cross FCA per hour
• AFP sets acceptance rates for each hour
• Flights wait based on their order in the schedule
• Example AFP:
• Large search space, determining “optimal” rates is very challenging
FCA1
FCA2
12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00
FCA1 59 59 59 57 58 63 61 62 60
FCA2 61 63 64 62 62 60 60 60 60
Can machine learning and integer programming help us identify the
appropriate solutions?
Acceptance rates
ATM Reinforcement Learning - 9
JJ 6/20/2019
Outline
• Background
• Modeling and simulation approach
• Optimization methods
• Results
• Summary
ATM Reinforcement Learning - 10
JJ 6/20/2019
Simulation Details
Inputs
CIWS/CoSPA Weather
Flight Schedule
Weather-Based
Capacity Constraints
Sector-Based
Constraints
Aircraft Separation
Constraints
Runway Separation
Constraints
NASPlay Simulation Outputs
Ground Delay
Airborne Delay
Number of
Arrivals
Airborne Holding
Diversions
Cancellations
ATM Reinforcement Learning - 11
JJ 6/20/2019
Simulation Details
NASPlay provides
high fidelity
simulation of national
airspace but it is
difficult to generate
good solutions due to
large search space
Can we use
optimization methods
to expedite the
process?
Inputs
CIWS/CoSPA Weather
Flight Schedule
Weather-Based
Capacity Constraints
Sector-Based
Constraints
Aircraft Separation
Constraints
Runway Separation
Constraints
NASPlay Simulation Outputs
Ground Delay
Airborne Delay
Number of
Arrivals
Airborne Holding
Diversions
Cancellations
ATM Reinforcement Learning - 12
JJ 6/20/2019
• May 15, 2018: Airspace capacity in
Northeast was compromised due
to weather
• Case study: Estimate airport
capacity at Newark Liberty
International Airport (EWR) from
11:00am to 00:45am GMT
• NASPlay Baseline: no control
strategy (“do nothing”)
– 388 aircraft landed in total (number
of Arrivals = 34-40 per hour)
– 161 aircraft accrued at least 15
minutes of holding before landing
• Goal: Find AFPs that achieve
comparable throughput but reduce
holding
Case Study
AFP = Airspace Flow Program
EWR
ATM Reinforcement Learning - 13
JJ 6/20/2019
• Hourly rates need to be specified for each of three FCAs
• Each solution has 30 dimensions: (3 FCAs) x (10 hr)
• Optimization covers a 30-dimensional space
Case Study Search Space
15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 0:00
FCA1 14 14 14 15 15 17 17 16 16 17
FCA2 10 10 10 10 10 8 8 9 9 10
FCA3 13 14 14 13 13 12 12 12 12 10
FCA1
FCA2
FCA3
FCA = Flow Constrained Area
ATM Reinforcement Learning - 14
JJ 6/20/2019
Search Space Description
• Choose rates for each FCA from a normal distribution: 𝒓𝒊𝒕~𝑵(𝝁𝒊, 𝝈𝒊
𝟐)
• Search space is constrained so that rates do not change drastically each hour
– Hourly AFP rates are correlated to the previous hour
– FCA hourly rates sum to a target rate for the airport R
Hour 1 2 3 4 5 6 7 8 9 T = 10
FCA1 r11 r12 r13 r14 r15 r16 r17 r18 r19 r110
FCA2 r21 r22 r23 r24 r25 r26 r27 r28 r29 r210
FCA3 r31 r32 r33 r34 r35 r36 r37 r38 r39 r310
FCA = Flow Constrained Area
AFP = Airspace Flow Program
𝑠. 𝑡.
𝑖
𝑟𝑖𝑡 = 𝑅 ∀𝑡
ATM Reinforcement Learning - 15
JJ 6/20/2019
Optimizing Airspace Flow Programs
• X = array of AFP rates
– X is 30-dimensional in above example
• y = Number of Arrivals
– Compute by running NASPlay simulation and calculating number of aircraft that landed per hour (# arrivals)
• Want to identify the AFP rate with maximum number of arrivals while achieving low holding levels
AFP Rates (X)
# o
f A
rriv
als
(y) Global
maximum
AFP = Airspace Flow Program
15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 0:00
FCA1 14 14 14 15 15 17 17 16 16 17
FCA2 10 10 10 10 10 8 8 9 9 10
FCA3 13 14 14 13 13 12 12 12 12 10
ATM Reinforcement Learning - 16
JJ 6/20/2019
Outline
• Background on demand / capacity balancing
• Modeling and simulation approach
• Optimization methods
– Reinforcement Learning (RL)
– Integer Programming (IP)
– Integer Programming with Random Exploration (IPRE)
• Results
• Summary
ATM Reinforcement Learning - 17
JJ 6/20/2019
Major Categories of Machine Learning
Supervised Learning Unsupervised Learning Reinforcement Learning
Approach• Train model given labeled
(truth) dataset
• Train model given unlabeled
dataset
• Train model based on trial and
error
Example
Applications
• Classification
• Pattern recognition
• Data fusion
• Clustering
• Anomaly detection
• Dimensionality reduction
• Game theory / strategy
• Optimal control
Example
Techniques
• Support Vector Machine
• Random Forest
• Convolutional Neural Network
• Principal Component
Analysis
• K-means clustering
• Dynamic programming
• ε-greedy algorithms
• Monte Carlo Tree Search
Air Traffic
Management
Example
Offshore Weather Prediction Trajectory Clustering ACAS X Collision Avoidance
-400 -300 -200 -100 0 100 200 300
-300
-200
-100
0
100
200
300
ATM Reinforcement Learning - 18
JJ 6/20/2019
Major Categories of Machine Learning
Supervised Learning Unsupervised Learning Reinforcement Learning
Approach• Train model given labeled
(truth) dataset
• Train model given unlabeled
dataset
• Train model based on trial and
error
Example
Applications
• Classification
• Pattern recognition
• Data fusion
• Clustering
• Anomaly detection
• Dimensionality reduction
• Game theory / strategy
• Optimal control
Example
Techniques
• Support Vector Machine
• Random Forest
• Convolutional Neural Network
• Principal Component
Analysis
• K-means clustering
• Dynamic programming
• ε-greedy algorithms
• Monte Carlo Tree Search
Air Traffic
Management
Example
Offshore Weather Prediction Trajectory Clustering ACAS X Collision Avoidance
-400 -300 -200 -100 0 100 200 300
-300
-200
-100
0
100
200
300
ATM Reinforcement Learning - 19
JJ 6/20/2019
e-Greedy Approach
Sampling entire space is intractable due to:
• Size of search space
• Time required to run each simulation
global maximum
Alternative is to search for optimum
using combination of two strategies:
ExplorationFind new point by random
sampling
ExploitationFind new point by fitting model f
to current sample points and
taking optimum
• Earlier iterations emphasize exploration
• Later iterations emphasize exploitationFor iteration 𝒊, 𝑷 𝒆𝒙𝒑𝒍𝒐𝒓𝒂𝒕𝒊𝒐𝒏 =
𝟓
𝟒 + 𝒊
X
Y
If sampling entire space were tractable, we
would simulate all possible points and
find global optimum
X
Y
y = f(x)
X
Y
AFP rate settings
# o
f A
rriv
als
ATM Reinforcement Learning - 20
JJ 6/20/2019
• Run 5 iterations with random sampling to initialize RL algorithm
• Run subsequent iterations with RL-selected samples
– RL algorithm selects exploration or exploitation
– If exploitation is selected:
• Surrogate model is generated using gradient tree boosting
• Surrogate model predicts number of arrivals for 100,000 randomly sampled points
• Choose sample point that maximizes predicted number of arrivals while limiting holding
Reinforcement Learning (RL) Details
15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 00:00
FCA1 14 15 15 16 16 16 15 16 17 16
FCA2 8 9 9 10 10 10 10 11 11 11
FCA3 14 13 13 12 12 12 13 10 9 10
• # Arrivals
• # Holdings
Set of acceptance rates
NASPlay simulation
Performance
metrics
ATM Reinforcement Learning - 21
JJ 6/20/2019
• Run 5 iterations with random sampling to initialize RL algorithm
• Run subsequent iterations with RL-selected samples
– RL algorithm selects exploration or exploitation
– If exploitation is selected:
• Surrogate model is generated using gradient tree boosting
• Surrogate model predicts number of arrivals for 100,000 randomly sampled points
• Choose sample point that maximizes predicted number of arrivals while limiting holding
Reinforcement Learning (RL) Details
• # Arrivals
• # Holdings
Set of acceptance rates
NASPlay simulation
Performance
metrics
15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 00:00
FCA1 14 15 15 16 16 16 15 16 17 16
FCA2 8 9 9 10 10 10 10 11 11 11
FCA3 14 13 13 12 12 12 13 10 9 10
ATM Reinforcement Learning - 22
JJ 6/20/2019
• Run 5 iterations with random sampling to initialize RL algorithm
• Run subsequent iterations with RL-selected samples
– RL algorithm selects exploration or exploitation
– If exploitation is selected:
• Surrogate model is generated using gradient tree boosting
• Surrogate model predicts number of arrivals for 100,000 randomly sampled points
• Choose sample point that maximizes predicted number of arrivals while limiting holding
Reinforcement Learning (RL) Details
• # Arrivals
• # Holdings
Set of acceptance rates
NASPlay simulation
Performance
metrics
15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 00:00
FCA1 14 15 15 16 16 16 15 16 17 16
FCA2 8 9 9 10 10 10 10 11 11 11
FCA3 14 13 13 12 12 12 13 10 9 10
ATM Reinforcement Learning - 23
JJ 6/20/2019
• Run 5 iterations with random sampling to initialize RL algorithm
• Run subsequent iterations with RL-selected samples
– RL algorithm selects exploration or exploitation
– If exploitation is selected:
• Surrogate model is generated using gradient tree boosting
• Surrogate model predicts number of arrivals for 100,000 randomly sampled points
• Choose sample point that maximizes predicted number of arrivals while limiting holding
Reinforcement Learning (RL) Details
• # Arrivals
• # Holdings
Set of acceptance rates
NASPlay simulation
Performance
metrics
15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 00:00
FCA1 14 15 15 16 16 16 15 16 17 16
FCA2 8 9 9 10 10 10 10 11 11 11
FCA3 14 13 13 12 12 12 13 10 9 10
ATM Reinforcement Learning - 24
JJ 6/20/2019
• Run 5 iterations with random sampling to initialize RL algorithm
• Run subsequent iterations with RL-selected samples
– RL algorithm selects exploration or exploitation
– If exploitation is selected:
• Surrogate model is generated using gradient tree boosting
• Surrogate model predicts number of arrivals for 100,000 randomly sampled points
• Choose sample point that maximizes predicted number of arrivals while limiting holding
Reinforcement Learning (RL) Details
• # Arrivals
• # Holdings
Reinforcement Learning
Set of acceptance rates
NASPlay simulation
Performance
metrics
15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 00:00
FCA1 14 15 15 16 16 16 15 16 17 16
FCA2 8 9 9 10 10 10 10 11 11 11
FCA3 14 13 13 12 12 12 13 10 9 10
ATM Reinforcement Learning - 25
JJ 6/20/2019
• Epsilon greedy (RL) approach is one way to derive good acceptance rates
– Model-free approach
– Initial solutions are likely to perform poorly
– May require many simulations to provide sufficient data to train the model
• Integer programming (IP) offers an alternative approach to identifying acceptance
rates
– Model-based approach
– Uses estimates of demand and capacity
– Initial solutions are likely to outperform e-greedy if the estimates for objective function and
constraints are accurate
• Two variants
– IP: Stochastic Integer Programming
– IPRE: Use Stochastic Integer Programming for baseline solution, then Random Exploration
(RE) to produce alternative acceptance rates
Alternative Method: Integer Programming
ATM Reinforcement Learning - 26
JJ 6/20/2019
Terminal AFP Planning Model for Integer Program
TMI = Traffic Management Initiative
15:00 16:00 17:00 18:00
FCA1 14 15 15 16
FCA2 8 9 9 10
FCA3 14 13 13 12
15:00 16:00 17:00 18:00
FCA1 15 13 15 14
FCA2 9 11 12 12
FCA3 14 13 10 12
Perturb
Randomly
Inputs:
• List of flights and scheduled times of arrival
• System cost of ground and air delay
Problem 1 (Estimation)
Quantify Model Uncertainty
• Estimate Airspace Capacity in
nearby en route airspace
• Generate Airport Demand based on
historical data
Inputs:
• Convective weather models
• Historical airspace throughput
Problem 2 (Optimization)
Objective: Minimize total expected cost of
ground and air delay
• All flights scheduled to take off must either
take off or be delayed on the ground
• All scheduled arrivals that have taken off
must land at their scheduled times or be
delayed in the air
• Number of arrivals cannot exceed the
assigned capacity (estimate based on
nearby airspace capacity)
• Decision Variable: Number of flights that
take off in a given time period
AFP rates
ATM Reinforcement Learning - 27
JJ 6/20/2019
Outline
• Background on demand / capacity balancing
• Modeling and simulation approach
• Optimization methods
– Reinforcement Learning (RL)
– Integer Programming (IP)
– Integer Programming with Random Exploration (IPRE)
• Results
• Summary
ATM Reinforcement Learning - 28
JJ 6/20/2019
• “Do Nothing” scenario resulted in 388
arrivals and 161 flights held for greater
than 15 minutes
Results
ATM Reinforcement Learning - 29
JJ 6/20/2019
• “Do Nothing” scenario resulted in 388
arrivals and 161 flights held for greater
than 15 minutes
• We’d like to achieve a comparable
throughput level with lower holding
Results
ATM Reinforcement Learning - 30
JJ 6/20/2019
• “Do Nothing” scenario resulted in 388
arrivals and 161 flights held for greater
than 15 minutes
• We’d like to achieve a comparable
throughput level with lower holding
Results
Ideal
performance
ATM Reinforcement Learning - 31
JJ 6/20/2019
• “Do Nothing” scenario resulted in 388
arrivals and 161 flights held for greater
than 15 minutes
• We’d like to achieve a comparable
throughput level with lower holding
• Typically, AFP programs identified
through random sampling will reduce
holding at the expense of the number
of arrivals
Results
AFP = Airspace Flow Program
ATM Reinforcement Learning - 32
JJ 6/20/2019
• “Do Nothing” scenario resulted in 388
arrivals and 161 flights held for greater
than 15 minutes
• We’d like to achieve a comparable
throughput level with lower holding
• Typically, AFP programs identified
through random sampling will reduce
holding at the expense of the number
of arrivals
Results
AFP = Airspace Flow Program
ATM Reinforcement Learning - 33
JJ 6/20/2019
• “Do Nothing” scenario resulted in 388
arrivals and 161 flights held for greater
than 15 minutes
• We’d like to achieve a comparable
throughput level with lower holding
• Typically, AFP programs identified
through random sampling will reduce
holding at the expense of the number
of arrivals
• Algorithm-based samples perform
considerably better than random
sampling
Results
AFP = Airspace Flow ProgramIP = Integer Programming
IPRE = IP with Random Exploration
ATM Reinforcement Learning - 34
JJ 6/20/2019
• “Do Nothing” scenario resulted in 388
arrivals and 161 flights held for greater
than 15 minutes
• We’d like to achieve a comparable
throughput level with lower holding
• Typically, AFP programs identified
through random sampling will reduce
holding at the expense of the number
of arrivals
• Algorithm-based samples perform
considerably better than random
sampling
Results
AFP = Airspace Flow ProgramIP = Integer Programming
IPRE = IP with Random Exploration
ATM Reinforcement Learning - 35
JJ 6/20/2019
• “Do Nothing” scenario resulted in 388
arrivals and 161 flights held for greater
than 15 minutes
• We’d like to achieve a comparable
throughput level with lower holding
• Typically, AFP programs identified
through random sampling will reduce
holding at the expense of the number
of arrivals
• Algorithm-based samples perform
considerably better than random
sampling
– The e-greedy algorithm outperforms other
candidates’ approaches
Results
AFP = Airspace Flow ProgramIP = Integer Programming
IPRE = IP with Random Exploration
ATM Reinforcement Learning - 36
JJ 6/20/2019
• “Do Nothing” scenario resulted in 388
arrivals and 161 flights held for greater
than 15 minutes
• We’d like to achieve a comparable
throughput level with lower holding
• Typically, AFP programs identified
through random sampling will reduce
holding at the expense of the number
of arrivals
• Algorithm-based samples perform
considerably better than random
sampling
– The e-greedy algorithm outperforms other
candidates’ approaches
Results
Best
selected
point
AFP = Airspace Flow ProgramIP = Integer Programming
IPRE = IP with Random Exploration
ATM Reinforcement Learning - 37
JJ 6/20/2019
• Fit distributions to number of arrivals and holdings
Variance in Results
Holding increases with target rate on e-greedy algorithm but
no clear relationship for IPRE.e-greedy algorithm outperforms IPRE when rates
are comparable. A target rate of 37 flights/hour
performs best for both methods.
IP = Integer Programming
IPRE = IP with Random Exploration
ATM Reinforcement Learning - 38
JJ 6/20/2019
• Reinforcement learning method demonstrates that effective balance of flight demand and
airport capacity is achievable
– Optimal strategy results in 98% of the original throughput with 76% reduction in holding
– Airport can sustain 33-41 arrivals/hr even when compromised with significant convective weather
Results Summary
Do-Nothing Best Selected
ATM Reinforcement Learning - 39
JJ 6/20/2019
Summary
• Approaches successfully applied to estimate flow rates in terminal airspace
– Demonstrated ability to efficiently find Airspace Flow Program (AFP) rates that provide high
throughput with low holding
– Resulting AFP provides a good proxy for airspace capacity at the arrival gate level
• Reinforcement learning is effective in optimizing over large search space when the
appropriate objective function and constraints cannot be captured in closed form
– Performance is significantly better than random selection
– Outperforms integer programming approaches
• Opportunities for future study
– Arrival/Departure balancing with traffic from neighboring airports
– Developing regional strategies to manage traffic through multiple airports
– Mapping resulting algorithmic solutions to more intuitive and interpretable control strategies
ATM Reinforcement Learning - 40
JJ 6/20/2019
Acknowledgements
• We would like to thank our sponsors William Chan, Paul Lee and Nancy Smith of NASA Ames for providing excellent technical oversight, discussions and instrumental feedback in the examined area.
Back-up
ATM Reinforcement Learning - 42
JJ 6/20/2019
Variance in Results
• e-greedy approach provides more consistent performance with respect to throughput
• IPRE outperforms IP when the target rate is not well aligned, but performance degrades slightly with good rate alignment
Selection
Method
Individual Throughput and Holding Performance
Number of
Arrivals
Mean (STD)
Number of
Holds Mean
(STD)
Maximum
Arrivals Case
Arrivals/Holds
IP
Solution
IPRE 35 354.1 (4.99) 35.0 (14.34) 362 / 19 357 / 89
IPRE 36 357.5 (4.66) 33.0 (15.16) 365 / 52 364 / 77
IPRE 37 365.2 (6.25) 32.3 (14.64) 371 / 45 372 / 44
e-greedy 35 365.4 (2.37) 20.3 (15.25) 371 / 15 N/A
e-greedy 36 365.8 (1.22) 18.2 (12.59) 368 / 51 N/A
e-greedy 37 371.9 (14.15) 42.8 (10.87) 381 / 39N/A
Selection method
Aggregate Throughput and Holding
Performance
Number of
Arrivals Mean
(STD)
Number of
Holds Mean
(STD)
Maximum
Arrivals Case
Arrivals/Holds
Random 356.27 (9.50) 62.75 (46.96) 374 / 145
IPRE 359.14 (6.80) 33.34 (14.74) 371 / 45
e-greedy 367.70 (4.90) 27.10 (16.92) 381 / 39
IP = Integer Programming
IPRE = IP with Random Exploration
ATM Reinforcement Learning - 43
JJ 6/20/2019
• Epsilon greedy (Reinforcement Learning) approach is one way to derive good
acceptance rates
– Model-free approach
– Initial solutions are likely to perform poorly
– May require many simulations to provide sufficient data to train the model
• Integer programming (IP) offers an alternative approach to identifying acceptance
rates
– Model-based approach
– Uses estimates of demand and capacity
– Initial solutions are likely to outperform e-greedy if the estimates for objective function and
constraints are accurate
Alternative Method: Integer Programming
ATM Reinforcement Learning - 44
JJ 6/20/2019
• Epsilon greedy (Reinforcement Learning) approach is one way to derive good
acceptance rates
– Model-free approach
– Initial solutions are likely to perform poorly
– May require many simulations to provide sufficient data to train the model
• Integer programming (IP) offers an alternative approach to identifying acceptance
rates
– Model-based approach
– Uses estimates of demand and capacity
– Initial solutions are likely to outperform e-greedy if the estimates for objective function and
constraints are accurate
Alternative Method: Integer Programming
Will the IP approach yield better solutions overall?
ATM Reinforcement Learning - 45
JJ 6/20/2019
Terminal AFP Planning Model for Integer Program
Inputs:
• List of flights and scheduled times of arrival
• List of historical days for TMI intervention
Inputs:
• Convective weather models
• Historical airspace throughput
Problem 1 (Machine Learning)
Quantify Model Uncertainty
• Estimate Airspace Capacity in
nearby en route airspace
• Generate Airport Demand based on
historical data
FCA capacity
estimates
and demand
scenarios
TMI = Traffic Management Initiative
ATM Reinforcement Learning - 46
JJ 6/20/2019
• Traffic Flow Impact* translates convective weather blockage forecast at a resource to a percentage known as permeability
• Idea: Use the permeability forecast at an upstream en route resource as a proxy for permeability through terminal FCAs
Initial Capacity Estimate for Integer Program
Traffic Flow Impact
* M. Matthews, M. Veillette, J. Venuti, R. DeLaura and J. Kuchar, "Heterogeneous Convective Weather Forecast
Translation into Airspace Permeability with Prediction Intervals.," Journal of Air Transportation, pp. 1-14, 2016.
ATM Reinforcement Learning - 47
JJ 6/20/2019
• Traffic Flow Impact* translates convective weather blockage forecast at a resource to a percentage known as permeability
• Idea: Use the permeability forecast at an upstream en route resource as a proxy for permeability through terminal FCAs
• Multiply en route permeability by target rate Cjt on terminal FCA and normalize values so that the flow rate through the 3 FCAs is equals target rate for airport C
Initial Capacity Estimate for Integer Program
Traffic Flow Impact
𝑅𝑗𝑡 = 𝑃𝑒𝑟𝑚𝑗𝑡 ∗ 𝐶𝑗𝑡 ∀𝑗, 𝑡
𝑠. 𝑡.
𝑗
𝐶𝑗𝑡 = 𝐶 ∀𝑡
* M. Matthews, M. Veillette, J. Venuti, R. DeLaura and J. Kuchar, "Heterogeneous Convective Weather Forecast
Translation into Airspace Permeability with Prediction Intervals.," Journal of Air Transportation, pp. 1-14, 2016.