1 Extreme Events Cost-Effectiveness Analysis Scott Matthews Courses: 12-706 / 19-702.
Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.
-
date post
20-Dec-2015 -
Category
Documents
-
view
215 -
download
1
Transcript of Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.
![Page 1: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/1.jpg)
Sensitivity Analysis and Uncertainty
H. Scott Matthews12-706/73-359/19-702Lecture 6
![Page 2: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/2.jpg)
Admin
HW 2 Due todayStill no word on books
We think they’re due today? Unfortunately cant keep pushing back lectures
I will send e-mail with private link to PDF for Wednesday reading (can’t post)
![Page 3: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/3.jpg)
Problem of Unknown Numbers
If we need a piece of data, we can: Look it up in a reference source Collect number through survey/investigation Guess it ourselves Get experts to help you guess it
Often only ‘ballpark’, ‘back of the envelope’ or ‘order of magnitude needed Situations when actual number is unavailable or where rough estimates are good enough
E.g. 100s, 1000s, … (102, 103, etc.)
Source: Mosteller handout
![Page 4: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/4.jpg)
Definition: “Base Case”
Generally uses single values and our ‘best guesses’
Sensitivity Analysis acknowledges uncertainty exists
Incorporate variables instead of constant assumptions
If our ‘Net Benefits’ remain positive over a wide range of reasonable assumptions, then robust results
![Page 5: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/5.jpg)
How many variables?
Choosing ‘variables’ instead of ‘constants’ for all parameters is likely to make model unsolvable
Partial sens. Analysis - change only 1 Equivalent of y/x Do for the most ‘critical’ assumptions
Can use this to find ‘break-evens’
![Page 6: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/6.jpg)
Best and Worst-Case Analysis
Analogous to “upper and lower bounds” used in estimation problems
Does any combination of inputs reverse the sign of our answer? If so, are those inputs reasonable? E.g. using very conservative ests. Might want NB > 0, but know when NB < 0 Similar to ‘breakeven analysis’
![Page 7: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/7.jpg)
Question 2.4 from Boardman
3 projects being considered R, F, W Recreational, forest preserve, wilderness Which should be selected?
Alternative Benefits($)
Costs($)
B/CRatio
NetBenefits ($)
R 10 8 1.25 2R w/ Road 18 12 1.5 6F 13 10 1.3 3F w/ Road 18 14 1.29 4W 5 1 5 4W w/ Road 4 5 0.8 -1Road only 2 4 0.5 -2
![Page 8: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/8.jpg)
Question 2.4
Base Case Net Benefits ($)
-4 -2 0 2 4 6 8
R
R w/ Road
F
F w/ Road
W
W w/ Road
Road only
Project“R with Road”has highest NB
![Page 9: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/9.jpg)
Question 2.4 w/ uncertainty What if we note that Benefits/Costs of each project are uncertain by plus or minus 10%? e.g. instead of Project R having benefits of $10 million, could be as low as $9 million or as high as $11 million
Repeat for all project combinations
Now which project is ‘best’?Alternative Base Case
BenefitsB +10% B -10% Base Case
CostsC -10% C +10%
R 10 11 9 8 7.2 8.8R w/ Road 18 19.8 16.2 12 10.8 13.2F 13 14.3 11.7 10 9 11F w/ Road 18 19.8 16.2 14 12.6 15.4W 5 5.5 4.5 1 0.9 1.1W w/ Road 4 4.4 3.6 5 4.5 5.5Road only 2 2.2 1.8 4 3.6 4.4
![Page 10: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/10.jpg)
Question 2.4 w/ uncertainty
Best Case:R w/Road(same)
Worst Case:Road Only
General Case:Could beseveral-4 -2 0 2 4 6 8 10
R
R w/ Road
F
F w/ Road
W
W w/ Road
Road only
Worst Case Net Benefits ($) Best Case Net Benefits ($)
But difficult to determine that from this chart - can we do better?
![Page 11: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/11.jpg)
Using error/uncertainty bars
Show ‘original’ point as well as range of uncertainty associated with point Range could be fixed number, percentage, standard deviation, other
Excel tutorial available at: http://phoenix.phys.clemson.edu/tutorials/excel/advgraph.html
See today’s spreadsheet on home page Graphs original points, and min/max deviations from that as error bars…
![Page 12: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/12.jpg)
Error bar result
Base Case Net Benefits ($)
R
R w/ Road
FF w/ Road W
W w/ RoadRoad only
-4
-2
0
2
4
6
8
10
Easier to see ‘best case/base case/worst case’ results - imagine moving a straight edge vertically up and down the axis to see result.
![Page 13: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/13.jpg)
Uncertainty
Investment planning and benefit/cost analysis is fraught with uncertainties forecasts of future are highly uncertain applications often made to preliminary designs
data is often unavailableStatistics has confidence intervals – economists need them, too.
![Page 14: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/14.jpg)
Sensitivity Analysis (SA)
Most of our discussions and examples have been simple (e.g. y=mx+b)
Life is not as simple as this E.g. y=x
We need to be able to create models and methods that can incorporate our uncertainties
“SA” is just a fancy phrase for saying “tell me what happens to the decision if the inputs change”
![Page 15: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/15.jpg)
An Easy Visual Clue - Slider Bars
Use built-in excel functions to visually see effects of incremental changes in variables “Scroll bars” (form toolbar) Tutorial at: http://plato.acadiau.ca/aittmodules/excel/viewlets/scrollbar/scrollbar.viewlet/scrollbar_viewlet_swf.html
Let’s look at our old TV estimation problem
![Page 16: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/16.jpg)
EXCEL’s TABLE function
One- and two-input data tablesSort of a built-in tool for sensitivity analysis (without fancy graphs/etc). See PDF posted for examples and instructions
![Page 17: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/17.jpg)
Case: Photo-sensors for lighting
Save electricity by installing sensors in areas where natural light exists Sensors ‘see’ light, only turn light fixtures on when needed
MAIN Posner 2nd floor hallway uses 106 15-watt fluorescent bulbs for 53 fixtures
How could we make a model to determine whether this makes sense? Assume only one year time frame
![Page 18: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/18.jpg)
Photo-sensors for lighting Assume we only care about ‘one year project’ Costs = Labor cost, installation cost, electricity costs, etc. Assume each bulb costs $6
Benefits = ? How should we set up model? Assume equal, set up as ‘show minimum cost’ option
Case 1 ‘Status quo’: assume lights used as is On all the time, bulbs last 10,000 hours ~ burn out once per year)
Case 2 ‘PS’: pay to install sensors now, bulbs off between 1/3 and 1/2 of time
![Page 19: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/19.jpg)
Lighting Case Study - Status Quo
Costs(sq) - lights on all the time Labor cost: cost of replacing used bulbs
“How many CMU facilities employees does it take to change a light bulb?” - and how long does it take?
Assume labor cost = $35/hr, 15 mins/bulb26.5 hours to change all bulbs each year, for a total labor cost of $927.50! Also, bulb cost $636/yr
Electricity: 106*15W ~ 14,000 kWh/yr (on 24-7)Cost varies from 2.5 - 7.5 cents/kWh ~ $350-$1045Cost Replacing bulbs is same ‘order of magnitude’ as the electricity! (Total range [$1,911 - $2,608])
![Page 20: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/20.jpg)
Lighting Case Study - PS sensorsCosts(ps) - probably ‘off’ 1/3 - 1/2 of time
Labor cost: cost of installing sensors = ‘unknown’
Labor cost: cost of installing new bulbsCould assume 1/2 - 2/3 of bulbs changed per year instead of ‘all of them’ [Total $464 - $700]
Bulbs cost [$318-$424] Electricity: 106*15W ~ 7,000 kWh/yr @ 1/2
9,333 kWh if off 1/3 of the timeCost varies [ 2.5 - 7.5 cents/kWh] ~ $175-$700
Total cost (w/o sensors) ~ [$955 - $1,740] How much should we be WTP for sensors if time horizon is only one year?
![Page 21: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/21.jpg)
PS sensors analysisWTP [$170 - $1,700] per year (NB>0)
We basically ‘solved for’ benefit But our main sensitive value was elec. Cost, so range is probably [$919 - $1,129] per yr.
No overlap in ranges - PS always better Should consider effects over several years
Could do a better bulb replacement model Use more ranges - Bulb cost, labor, time
Check sensitivity of model answer to changes Find partial sensitivity results for each Look at spreadsheet model This is fairly complicated - easier way?
![Page 22: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/22.jpg)
Sens. Analysis for Photo Sensors
Several built-in options from the plugins (@RISK-TopRank [Win only] or treeplan [Win/Mac]) to check sensitivity. One-way (one variable at a time) Two-way (two at a time) Tornado (all at a time)
![Page 23: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/23.jpg)
One-Way Sensitivity Analysis
Use @RISK or treeplan plugin. Makes graph that varies a single variable from a low- to high-end range and shows output (eg NPV) as a function of variable
The resulting graph shows how “sensitive” your answer is to changes in the one variable
![Page 24: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/24.jpg)
Tornado Diagrams
Shows results of many one-way plots on single chart
Length of bars tells you how sensitive output is to each variable Bigger bar = more sensitive Software typically “sorts” most to least
Looks like a tornadoSpider diagrams are similar
![Page 25: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/25.jpg)
Two-way SA
Treeplan plug-in cant do thisShows a 2-dimensional plot of what happens when we change 2 variables at once Graph generated is a “frontier”
![Page 26: Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d4e5503460f94a2dd19/html5/thumbnails/26.jpg)
Plug-in notes
See Clemen pp.193- for TopRank tutorial
See course web page for instructions o using treeplan plugins (eg SensIt) Read the PDFs for install instructions!
Should install ok on most cluster computers