Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a...
Transcript of Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a...
![Page 1: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/1.jpg)
Calibration and Parameter Risk Analysis for Gas Storage Models
Greg Kiely (Gazprom)
Mark Cummins (Dublin City University)
Bernard Murphy (University of Limerick)
![Page 2: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/2.jpg)
New Abstract
![Page 3: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/3.jpg)
![Page 4: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/4.jpg)
Model Risk Management: Regulatory Context
u Fed OCC (2000): Risk Bulletin on Model Validation
u Fed OCC (2011): Supervisory Guidance on Model Risk Management
u Basel Committee on Banking Supervision
![Page 5: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/5.jpg)
Fed OCC: Risk Bulleting on Model Validation
u Independent Review
u Defined Responsibility
u Model Documentation
u Ongoing Validation
u Audit Oversight
![Page 6: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/6.jpg)
Fed OCC: Supervisory Guidance on Model Risk Management
u Model development, implementation and use
u Governance and control mechanisms
u Policies and procedures
u Controls and compliance
u Appropriate incentives and organisational structure
![Page 7: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/7.jpg)
Model Risk Definition
u Derman (1996) u Model inapplicability
u Incorrect model use
u Incorrect solution to a correct model
u Incorrect use of a correct model
u Use of poorly specified model approximations
u Software and hardware errors
u Unstable or poor quality data input
![Page 8: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/8.jpg)
Model Risk v Model Uncertainty
u Model Risk u Exposure to future possible outcomes but with a unique
defined probability measure
u Model Uncertainty u Exposure to future possible outcomes for which there is
no one unique defined probability measure
u Knight (1921)
![Page 9: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/9.jpg)
![Page 10: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/10.jpg)
Model Risk Measurement
u Artzner (1999) u Theory of coherent risk measures
u Follmer and Schied (2000) u Theory of convex risk measures
u Cont (2006) u Theory of model risk measurement
![Page 11: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/11.jpg)
![Page 12: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/12.jpg)
Bannor and Scherer (2013)
u Propose calibration risk functional concept u Incorporate calibration risk into bid-offer levels
u Theoretical framework consistent with convex risk measure concept
u Derive push-forward distributions for asset values based on calibration error from stochastic models
![Page 13: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/13.jpg)
Bannor et al (2015) u Examine parameter risk inherent in a real options model-
based approach to valuing power plant infrastructure
u Draw on innovative risk capturing approach of Bannor and Scherer (2013)
u Authors use sophisticated multi-factor setting to model emissions, gas and power prices
u Multidimensional search problem that poses considerable parameter risk
u Given unavailability of joint estimator’s distribution in closed form, parameter risks are separately studied
u Bid-ask spreads from AVaR risk capturing price functional show spike risk is the most important parametric risk!
![Page 14: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/14.jpg)
Motivation for Current Work
u Paucity of academic literature on model risk and model uncertainty
u Growing importance of industry practice of model risk management and model validation activity u Regulatory impetus
u Bannor et al (2015) pave the way for further research into model risk issues in energy markets u Perfect context given the range of OTC products and
structures u Extensive use of models for valuation and hedging activity
![Page 15: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/15.jpg)
Motivation for Work
u Gas storage capacity presents a prime candidate for model risk analysis given: u Ever increasing importance of gas storage capacity in
Europe, and globally
u Difficulty in deriving competitive prices for this capacity
u Growing secondary market for capacity … allowing market players to adjust seasonal flexibility to match portfolio growth
u Dependency on models for valuation and hedging
u No industry- or academic-consensus in circulation
![Page 16: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/16.jpg)
Contributions u Derive holistic approach to evaluating the uncertainty associated
with parameter calibration and estimation
u Calibration to market derivative instruments
u Estimation to historical data
u Consider an innovative suite of mean-reverting Levy-driven models that offer market consistency
u Developed in first two papers of three-paper series
u Suggest method for ranking models based on their robustness to calibration errors
u Relevance to trading, risk and regulatory stakeholders
![Page 17: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/17.jpg)
Henaff et al (2013)
u Examine historically estimated parameter risk associated with storage valuation
u Employ coherent model risk measure of Cont (2006) u But deviate from this by replacing the set of benchmark instruments …
u … with a test which determines whether a set of model parameters returns likelihood value “close” to the ML value
u Use two proposed spot price models with price spikes
u We differ from Henaff et al (2013) in two ways: u Unified approach to evaluating calibration and estimation risk
u Use of market consistent model framework that incorporates wider market information and is more in line with practical trading considerations
![Page 18: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/18.jpg)
Bannor and Scherer (2013): Risk capturing functionals
u Risk functional Γ … giving bid-ask spread
u Γ has certain desirable properties
![Page 19: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/19.jpg)
Bannor and Scherer (2013): Risk capturing functionals
u Q is one of a family of potential models Q generally unknown distribution R
u In many cases the estimators of the model Q will possess asymptotic normality
u We can exploit this to note that
![Page 20: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/20.jpg)
Bannor and Scherer (2013): Risk capturing functionals
u With this in place …
![Page 21: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/21.jpg)
Bannor and Scherer (2013): Risk capturing functionals
u Finally …
![Page 22: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/22.jpg)
Energy Model Risk Analysis u Wish to consider parameter calibration risk and estimation risk jointly. But why?
u Realistic models of natural gas forward curve cannot be calibrated to benchmark instruments alone
u Due to lack of liquid time-spread options market
u Correlation structure is typically estimated from historical data and then approximated by a suitable model specification
u Storage valuation models are particularly sensitive to the model implied correlation structure
u Hence exposed to parameter estimation risk!
u Overall level of volatility implied by model constrained to be calibrated to the market
u To obtain consistency with the products used to hedge volatility risk
u Hence exposed to calibration risk!
![Page 23: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/23.jpg)
Energy Model Risk Analysis
u We propose the following transformation function form
u Transformation function decomposed into
u Error term density conditional upon the historically estimated parameters
u Sampling error density associated with the historically estimated parameters
u From Bannor et al (2015), it is know that asymptotically
u Gaussian density suitable specification for sampling error density
![Page 24: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/24.jpg)
Energy Model Risk Analysis
u Steps follow closely those of Bannor et al (2015) u We apply the delta method described by the authors in
conjunction with the sampling error density
u Construct a joint market and historical parameter risk induced value density
![Page 25: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/25.jpg)
Energy Model Risk Analysis
![Page 26: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/26.jpg)
Energy Model Risk Analysis
![Page 27: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/27.jpg)
Energy Model Risk Analysis
u Last result gives the storage value variance induced by uncertainty over the estimate of the forward curve covariance matrix
u The relationship can be understood as u First, weighting the matrix by the sensitivity of the model parameters to the
sensitivity of the forward curve covariance matrix
u Second, weighting the result by the sensitivity of the storage value to the model parameters
![Page 28: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/28.jpg)
Model Specifications u Mean-Reverting Variance Gamma (MRVG) … Kiely et al (2015a)
u Mean-Reverting Jump Diffusion (MRJD) u Second model … variant of first … specified by Deng (2000) and used
by Kjaer (2008)
u Choice of stochastic driver is different u Jump-diffusion process with compound Poisson jump process driven by
a double exponential distribution
![Page 29: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/29.jpg)
Model Specifications u MRVG-3x … Kiely et al (2015b)
u First factor accounts for majority of forward curve variability u Composition of MRVG and MR Diffusion
u Parameter b ... proportion of total variance attributed to first factor
u Second factor approximates the typical shape of the sensitivity of the forward curve to the second PC of forward curve returns covariance matrix
![Page 30: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/30.jpg)
Storage Contract and Data
u Simple 20in / 20out storage deal u Deal commences immediately on options quote date
u Lasts for 1 year
u All values in pence / therm
u Options data u 6-month and 1-year monthly options on NBP
u Strike prices ranging from 0.95-1.05 moneyness
u 14 option prices in total
u Quote date 19th December 2012
u Sourced from Bloomberg
![Page 31: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/31.jpg)
Storage Contract and Data
u Data used to estimate historical covariance matrix u 3 years ending 19th December 2012
u Contains relative maturity returns with a day-ahead quote and month-ahead quotes spanning 11-months
u First two models calibrated using FFT-based swaption pricing method … Kiely et al (2015a)
u Third model calibrated using moment matching method with FFT-based option pricing … Kiely et al (2015b)
![Page 32: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/32.jpg)
Mark-Based Calibration Risk
u Consider 2808 parameter combinations
u Retain 807 based on 3% limit around minimum RMSE
![Page 33: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/33.jpg)
![Page 34: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/34.jpg)
Model-Based Calibration Risk
u Consider 2548 parameter combinations
u Retain 795 based on 3% limit around minimum RMSE
![Page 35: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/35.jpg)
![Page 36: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/36.jpg)
Model-Based Calibration Risk
u Conclusion u MRVG and MRJD models carry comparable levels of
calibration risk
u MRVG returns higher expected value than MRJD
u But MRVG also has higher variability than MRJD
![Page 37: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/37.jpg)
Joint Calibration and Parameter Estimation Risk
u Consider 2548 parameter combinations
u Retain 838 based on 3% limit around minimum RMSE
![Page 38: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/38.jpg)
![Page 39: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/39.jpg)
Joint Calibration and Parameter Estimation Risk
u Comparing to MRVG and MRJD u Value is much higher
u Coefficient of variation is almost double
u Confidence in calibrated value is lower
u Clustering of storage value at distinct levels u Relate to different combinations of α and σ
u These combinations appear to fully determine storage value
u The impact of ν is minimal
![Page 40: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/40.jpg)
Joint Calibration and Parameter Estimation Risk
u We now proceed to deriving the parameter risk density associated with the historically estimated parameters
u We can derive a sampling error covariance matrix for each of the volatility parameters
u Which is turn can be used to derive a parameter risk covariance matrix
u We must first estimate the Jacobian of our storage value with respect to our model parameters
![Page 41: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/41.jpg)
![Page 42: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/42.jpg)
Joint Calibration and Parameter Estimation Risk
u Variability has increased dramatically from inclusion of risk associated with the historically estimated parameters
![Page 43: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/43.jpg)
Deriving Risk-Adjusted Price Levels
![Page 44: Calibration and Parameter Risk Analysis for Gas Storage Models · Gas storage capacity presents a prime candidate for model risk analysis given: ! Ever increasing importance of gas](https://reader033.fdocuments.us/reader033/viewer/2022060520/604f0703241bb43d72677dbe/html5/thumbnails/44.jpg)
Deriving Risk-Adjusted Price Levels
u MRVG u Bid-offer price levels … 11.1507-11.2697
u Spread of 1.06% of mean value
u MRJD u Bid-offer prices levels … 11.1473-11.2611
u Spread of 1.02% of mean value
u MRVG-3x u Bid-offer price levels … 16.1866-17.4748
u Spread of 7.66% of mean value