Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity...
Transcript of Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity...
![Page 1: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/1.jpg)
Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario AnalysisT h e A n s w e r i s Ye s !
Proprietary & Confidential – Not for duplication or redistribution
Palisade 2019 Annual ConferenceNovember 13-14San Antonio, Texas
![Page 2: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/2.jpg)
• Introduction
• The Issue
• Definitions
• Examples
• Observations & Recommendations
• Case Study
Overview
Proprietary & Confidential – Not for duplication or redistribution 2
![Page 3: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/3.jpg)
Introduction
Proprietary & Confidential – Not for duplication or redistribution
![Page 4: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/4.jpg)
P e o p l e ▪ P r o c e s s ▪ R e s u l t s
4
Experience on Demand
• Formed in 2008, based in Chesterfield, Missouri
• Currently 17 persons, including 7 Senior Partners who own the firm
• Successfully served over 200 clients
• Broad experience across multiple industries
• Deep technical expertise within multiple industries
Proprietary & Confidential – Not for duplication or redistribution
![Page 5: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/5.jpg)
Mission• Experience on Demand provides
comprehensive services to help our clients tackle today’s problems and achieve tomorrow’s opportunities.
• We partner with our clients, “roll up our sleeves,” and focus on results.
Distinctives• Our Partners are “hands on” experts, highly
skilled in performing technical work
• We pride ourselves in exceedingly responsive customer service
• We are highly flexible in designing work scope and pricing structures to meet our clients’ exact needs
• We continuously strive to maximize net value delivered
P e o p l e ▪ P r o c e s s ▪ R e s u l t s
5
Experience on Demand
Proprietary & Confidential – Not for duplication or redistribution
![Page 6: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/6.jpg)
• S e n i o r P a r t n e r , E x p e r i e n c e o n D e m a n dE n e r g y & U t i l i t i e s P r a c t i c e L e a d e r
• 3 3 Y e a r s c o n s u l t i n g a n d i n d u s t r y e x p e r i e n c e
• Completed over 100 projects across all forms of electric utilities in U.S. and Canada
• D e l o i t t e & To u c h e , L L P
• Director, Governance, Risk, and Regulatory Strategy
• R . W. B e c k , I n c .
• National Practice Leader - Energy Risk Management
• National Practice Leader - Energy Markets Consulting
• A m e r e n C o r p o r a t i o n
• Director, Transmission Capital & Risk Management
• Supervising Engineer, Corporate Planning
• Multiple other roles
• B S N u c l e a r E n g i n e e r i n g
• M B A w i t h h o n o r s
• E x e c u t i v e E d u c a t i o n – T h e W h a r t o n S c h o o l
6
Professional Background
Proprietary & Confidential – Not for duplication or redistribution
![Page 7: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/7.jpg)
7
Career Path
Proprietary & Confidential – Not for duplication or redistribution
BSNE
Union Electric
Nuclear Fuel
Engineer
MBA
Ameren
Energy Marketing & Trading
R. W. Beck
Risk and Energy
Markets Consulting Practices
DeloitteGlobal Energy
Markets / GRRS
Ameren
Transmission Capital & Risk Management
Experience on Demand
Management Consulting
Investor-Owned Utilities
All forms of Electric Utilities,
Banks
EnergyUtilities
GovernmentInfrastructureManufacturing
![Page 8: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/8.jpg)
The Issue
Proprietary & Confidential – Not for duplication or redistribution
![Page 9: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/9.jpg)
9
The Issue
• Many organizations are resistant to Monte Carlo analysis
Proprietary & Confidential – Not for duplication or redistribution
The “cloud” of results is hard to understand and
explain
We prefer to study individual “what if” cases
We want to directly control
the results
We prefer to study discrete
scenarios
![Page 10: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/10.jpg)
10
The Issue
• This is unfortunate because Monte Carlo not only provides more dependable expected values for most business decisions, but also provides richer and more useful sensitivity and scenario analysis capabilities
Proprietary & Confidential – Not for duplication or redistribution
More Information+
Better Information
Monte Carlo Simulation
Yields
![Page 11: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/11.jpg)
Definitions
Proprietary & Confidential – Not for duplication or redistribution
![Page 12: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/12.jpg)
The process of running a model (such as a spreadsheet) many times in a row. Each run
(iteration) uses a different set of inputs. The values used for each input conform to an assumed
probability distribution and an assumed correlation with the other inputs. Results from each
iteration are recorded.
The result of the simulation is obtained by taking the average (mean) of the iteration results.
This becomes the expected result of the model. In most cases, the average of the simulation is
more dependable in reflecting “reality” than a single run of the model using an expected or
most-likely value for each input*.
12
Definitions• Monte Carlo Simulation
Proprietary & Confidential – Not for duplication or redistribution
* This relates to what is known as Jensen’s Inequality
Mathematically, E[f(x,y)] ≠ f[E(x),E(y)]
This holds true for most (but not all) business problems.
![Page 13: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/13.jpg)
13
Definitions• Sensitivity Analysis
Proprietary & Confidential – Not for duplication or redistribution
Understanding the key drivers of a model by performing “what ifs” on
each input variable individually. Goal is to identify the inputs that have
the largest impact on results.
![Page 14: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/14.jpg)
14
Definitions• Scenario Analysis
Proprietary & Confidential – Not for duplication or redistribution
Understanding the behavior of a model for various combinations of
inputs.
Or conversely, studying selected ranges of results to understanding the
combinations of inputs leading to the selected results.
![Page 15: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/15.jpg)
Example & Results
Proprietary & Confidential – Not for duplication or redistribution
![Page 16: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/16.jpg)
16
Example and Results
• Potential power plant investment
• If power price is above marginal operating cost, then plant runs and produces margin• Marginal cost = Fuel price x
energy conversion factor + variable O&M
• However, fixed costs are always incurred
Proprietary & Confidential – Not for duplication or redistribution
Inputs
Generator Size 500 MW
Heat Rate 7.0 MMBtu/MWhr
Variable O&M 2.00 $/MWh
Availability 90%
Fixed Monthly Cost $2,500,000 $/Month
Ave Fuel Price $3.00 $/MMBtu
Fuel Price Volatility 20%
Ave Power Price 30.00 $/MWhr
Power Price Volatility 20%
Gas Forward Purchase Qty - MMBtu/Day
Power Forward Sale Qty - MWhr/Hr
Days in Period 31
![Page 17: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/17.jpg)
17
Example & Results
Proprietary & Confidential – Not for duplication or redistribution
Model in Action
![Page 18: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/18.jpg)
18
Example & Results
Proprietary & Confidential – Not for duplication or redistribution
• Monte Carlo using @Risk• Standard approach
Initial Expected Values
![Page 19: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/19.jpg)
19
Example & Results
Proprietary & Confidential – Not for duplication or redistribution
• Monte Carlo using @Risk• Standard approach
Richness of Information
![Page 20: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/20.jpg)
20
Example & Results
Proprietary & Confidential – Not for duplication or redistribution
• Monte Carlo using @Risk• Standard approach
Sensitivity Analysis
![Page 21: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/21.jpg)
21
Example & Results
Proprietary & Confidential – Not for duplication or redistribution
• Monte Carlo using @Risk• Standard approach
Scenario Analysis
![Page 22: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/22.jpg)
22
Summary
Proprietary & Confidential – Not for duplication or redistribution
• Monte Carlo using @Risk• Dependable expected value
immediately available
• Richer information
• Meaningful sensitivity results
• Meaningful scenario results
• 30 minutes + fun
• Standard approach• Undependable value results
• Limited information
• Unclear sensitivity results
• Unclear scenario results
• 6 hours + Tylenol
![Page 23: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/23.jpg)
Observations & Recommendations
Proprietary & Confidential – Not for duplication or redistribution
![Page 24: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/24.jpg)
24
Observations
• Monte Carlo analysis using Palisade @Risk
• Provides more dependable results for decision-making
• Is significantly less time-intensive for performing proper
sensitivity and scenario analysis
• Features for exploring results are very powerful, yet easy to use
once you learn them
• Even for those ingrained in standard sensitivity and scenario
analysis, @Risk’s benefits should be compelling
Proprietary & Confidential – Not for duplication or redistribution
![Page 25: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/25.jpg)
25
Recommendations
• Take the time to explore @Risk’s features, especially relating to sensitivity analysis and scenario analysis
• When dealing with skeptics…
• Use @Risk’s features to “follow the bread crumbs” to demonstrate to skeptics that @Risk is actually automating the same sensitivity and scenario analyses that are traditionally done manually
Proprietary & Confidential – Not for duplication or redistribution
![Page 26: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/26.jpg)
Case Study
Proprietary & Confidential – Not for duplication or redistribution
![Page 27: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/27.jpg)
27
Case Study
• Texas municipal utility
• We assisted them to develop a highly-flexible Energy Portfolio Analysis model (X-EPM) based on @Risk
• In fourth year of active use
• Fuel & purchased energy (F&PE) cost forecasting
• F&PE rate-setting
• Mid-term hedging analysis
• Long-term contract analysis
• Model and analytic processes have been catalysts leading to more and better questions on actual versus projected results
• Discovered significant lost value relating to operating practices which is now being captured
Proprietary & Confidential – Not for duplication or redistribution
![Page 28: Choosing Among Monte Carlo, Sensitivity Analysis, and Scenario … · 2019. 11. 27. · sensitivity and scenario analysis •Features for exploring results are very powerful, yet](https://reader035.fdocuments.us/reader035/viewer/2022062610/610996b43e18b013e2774af6/html5/thumbnails/28.jpg)
Thank You.P. Glen Justis, Senior Partner
314-619-3430
www.experience-on-demand/enr-util
Proprietary & Confidential – Not for duplication or redistribution