Verification has been undertaken for the 3 month Summer period (30/05/12 – 06/09/12) using...
-
Upload
philomena-lester -
Category
Documents
-
view
216 -
download
2
Transcript of Verification has been undertaken for the 3 month Summer period (30/05/12 – 06/09/12) using...
![Page 1: Verification has been undertaken for the 3 month Summer period (30/05/12 – 06/09/12) using forecasts and observations at all 205 UK civil and defence aerodromes.](https://reader037.fdocuments.us/reader037/viewer/2022110401/56649de55503460f94add96f/html5/thumbnails/1.jpg)
Verification has been undertaken for the 3 month Summer period (30/05/12 – 06/09/12) using forecasts and observations at all 205 UK civil and defence aerodromes. Various different verification measures have been calculated to compare the models and explore their strengths and weaknesses.
CONTINGENCY TABLES AND RELATIVE OPERATING CHARACTERISTIC CURVE
RELIABILITY DIAGRAM COST-LOST ANALYSIS
The forecast products are verified using observations from the ATDnet (Arrival Time Difference) system; this is an automatic lightning location network that senses lightning flashes over a geographical area. By timing the arrival of unique very low frequency radio waves (‘Sferics’) that are generated by individual lightning strokes the location of every stroke can be calculated.A stroke count was processed to give the number of strokeswithin a 50km radius of each airport over a 6 hour period.
MOGREPS-GLOBAL UKV (post processed)
Probability of lightning index exceeding 10 from a 12 member ensemble:
0 → lightning is unlikely
10 → deep convectively unstable environment with lightning a possibility
Model resolution: 60km
Forecast lead times: T+24 and T+36
Model diagnostics: Precipitation, CAPE, convective cloud features (temperature, base height, depth), wet bulb temperature.
Risk of lightning within 50km radius of a location:
LR1 → lightning expected
LR5 → lightning not expected
Model resolution: 1.5km
Forecast lead times: T+21 and T+33
Model diagnostics: Lifted index, CAPE, lightning index, total precipitation rate.
Forecasts are all valid at 06Z and 18Z (±3hours)
© Crown copyright Met Office and the Met Office logo are registered trademarks
Met Office FitzRoy Road, Exeter, Devon, EX1 3PB United KingdomTel: 01392 885680 Fax: 01392 885681Email: [email protected]
The forecasting of hazardous weather conditions such as lightning is of huge importance to the Civil Aviation Authority. The accuracy and skill of lightning forecasts is key to the safety of all flights in the UK. This project looks into the verification of two different models’ lightning products to assess how well they forecast this rare event during the 2012 summer.
Forecasts Observations
Verification
Conclusions and Future Work
There were almost 60,000 recorded strikes across the UK within a 13 hour period (05Z to 18Z). Both models indicated lightning was expected with a focus on the East coast of England. The ATDnet observations for 1800Z indicate two strong bands of lightning in that region.
Hits
1485
Misses
48False alarms
11791
Correct rejections
25216
Hits
175
Misses
1359False alarms
931
Correct rejections
36076Hit =Non zero probability
Figure 3: MOGREPS-Global ROC curve. Assumption: zero probability of precipitable water, minimal cloud cover, minimal risk of lightning.
Figure 4: UKV ROC curve.Hit = LR1 and LR2
• UKV shows considerably higher hit rate (0.97), however the global model results have a much smaller false alarm rate (0.025).
• A ROC curve displays the forecasts’ ability to distinguish between an event and a non event.
• Both ROC curves have a very similar initial gradient showing similar skill where data is present.
• The limited number of points on the MOGREPS-Global curve is due to the small number of members in the ensemble (lowest threshold = 1/12).
•The rare nature of lightning focuses the points on the ROC curve closer to the origin.
Case Study 28th June 2012
Figure 6: Value plot of MOGREPS-Global and UKV. Perfect score = 1.0.
Figure 7: Forecasts for 2012/06/28 18Z: a) MOGREPS-Global T+36, b) UKV T+33, c) lightning stroke map for 15Z to 21Z on 28/06/12.
MOGREPS-Global
UKV
A reliability diagram displays how well the predicted probabilities correspond to their observed frequencies.
Figure 5: Reliability diagram of MOGREPS-Global and UKV.
A value plot shows the relative improvement in economic value between climatology and a perfect forecast.
The UKV curve fully encompasses the MOGREPS-Global curve, indicating UKV has greater value.
UKV shows high skill and value, greater than MOGREPS-G, however the specific risk and observed frequencies of LR 1&2 could be explored further. MOGREPS-G does show some skill which shows it has potential given the resolution of the model and the rare nature of lightning.
In the future MOGREPS-UK (resolution of 1.5km) will be verified to allow direct comparison of the two models for the same domain and resolution, to determine if an ensemble forecast would be better. The ensemble could also be expanded to use 24 members by combining two forecast runs to investigate the ROC curves of the two models further.
Figure 1: Hyperbola plot - the potential locations of the lightning source from every pair of stations, the intersection of all the hyperbolae is the lightning location.
Figure 2: Stroke counts within 50km radius of Heathrow and Gatwick airports.
Heathrow
Gatwick
• MOGREPS-Global shows good reliability for available probabilities.
• Due to the categorical nature of the UKV forecast a classic reliability diagram could not be produced. LR3,4,5 indicate minimal risk of lightning which correlates with the near zero observed frequency and LR2&1 indicate some risk of lightning correlating with observations.
a) b) c)
Verification of Met Office Lightning Forecasts for Aviation
Rebecca Stretton, Piers Buchanan, Stephen Moseley, Clare Bysouth