An empirical investigation - proceedings · an empirical investigation on the value of combined...
Transcript of An empirical investigation - proceedings · an empirical investigation on the value of combined...
AN EMPIRICAL INVESTIGATION ON THE VALUE OF COMBINED JUDGMENTAL AND
STATISTICAL FORECASTING
PROF. DR. PHILIPPE BAECKEPROF. DR. KARLIEN VANDERHEYDEN
DRS. SHARI DE BAETS
CONTACT: [email protected]
ISF 2014, The Netherlands
© Vlerick Business School
AN EMPIRICAL INVESTIGATION
ON THE VALUE OF COMBINED JUDGMENTAL AND STATISTICAL
FORECASTING
Shari De Baets – ISF 20142
© Vlerick Business School
AN EMPIRICAL INVESTIGATION
� “..the practice of forecasting does not however appear to have improved.” (p,33; Armstrong, Green &
Graefe, 2013)
� Ascher, 1978
� Allen, 1994
� McCarthy, Davis, Golicic & Mentzer, 2006
Shari De Baets – ISF 20143
© Vlerick Business School
AN EMPIRICAL INVESTIGATION
An important task for researchers in our field:
Improving forecastingaccuracy in practice
(Sanders & Manrodt, 2003)
Shari De Baets – ISF 20144
© Vlerick Business School
AN EMPIRICAL INVESTIGATION
�Call for more studies with real company data (Sanders, 2009)
�Going beyond artificial experiments to deriverules for practice
�Expert forecasters
�Context of forecasting task
�A new research model: JUD 3 session(room “Plate”, Wed., 10 AM)
Shari De Baets – ISF 20145
© Vlerick Business School
AN EMPIRICAL INVESTIGATION
ON THE VALUE OF COMBINED JUDGMENTAL AND STATISTICAL
FORECASTING
Shari De Baets – ISF 20146
© Vlerick Business School
THE VALUE OF COMBINED FORECASTING
25%
25%
17%
33%
Forecasting method
judgment alone
statistical methods alone
average of statistical and
judgmental forecast
statistical forecast adjusted
judgmentally
Fildes & Goodwin, 2007
Shari De Baets – ISF 20147
© Vlerick Business School
THE VALUE OF COMBINED FORECASTING
� Potential of judgment in forecasting oftenundermined by unneccessary adjustments(Lawrence et al., 2006)
� Patterns in noise (Harvey, 1995)
� Illusion of control effect (Kotteman, Davis, & Remus,
1994)
� Persists despite warning (Lim & O’Connor, 1995)
Shari De Baets – ISF 20148
© Vlerick Business School
RESEARCH QUESTION
How can we counter damaging adjustments andreap the potential benefits from judgment, andthus, heighten forecasting accuracy?
Shari De Baets – ISF 20149
© Vlerick Business School
HYPOTHESES
� Data augmentation:
� Classic model + judgment:
Var1, Var2, .., Varn -> outcome -> judgmentaladjustment
� Judgment incorporated in model
Var1, Var2, VarJudgment , .., Varn -> outcome
Shari De Baets – ISF 201410
© Vlerick Business School
HYPOTHESES
�Data augmentation:
� H1:
� “integrated judgment”: judgment as part of the model
will outperform
� “restrictive judgment”: judgment as restriction on the model (judgmental adjustment)
Shari De Baets – ISF 201411
© Vlerick Business School
HYPOTHESES (SIMILAR TO FILDES ET AL., 2009)
� H1 applied to
�Direction of the adjustment
�Downward adjustments are beneficial, upwardadjustment are damaging
�Size of the adjustment
�Curvilinear (inverted U-shape) effect: both small and very large adjustments are damaging
�Volatility of the data series
� Judgment is beneficial in high volatility series (volatility due to special events)
Shari De Baets – ISF 201412
© Vlerick Business School
THE COMPANY
Shari De Baets – ISF 201413
Outlet store
Outlet store
Outlet store
International publishing company
Weekly and monthlymagazines
© Vlerick Business School
PROCEDURE
� Demand forecasting
� Predictive model: forecast of expected demandper store
� Optimisation model: profit optimisation – finalnumber for supply per store
Input
Shari De Baets – ISF 201414
© Vlerick Business School
THE COMPANY
� Profit optimization model
Overstock
Stockout
Shari De Baets – ISF 201415
© Vlerick Business School
PROCEDURE
�Predictive model: forecast of expected demandper store
�Optimisation model: profit optimisation – finalnumber for supply per store
�Judgmental adjustment: according to insight
Shari De Baets – ISF 201416
© Vlerick Business School
PROCEDURE
Predictive Optimisation Judgmentalmodel model adjustment
Restrictive judgment
Shari De Baets – ISF 201417
© Vlerick Business School
PROCEDURE – DATA AUGMENTATION
Predictive Optimisation Judgmentalmodel model adjustment
Integrative judgment
Shari De Baets – ISF 201418
© Vlerick Business School
PROCEDURE – DATA AUGMENTATION
Predictive model Optimisation model(incl. judgmentparameter)
Integrative judgment
Shari De Baets – ISF 201419
© Vlerick Business School
GENERAL COMPARISON N = 1223
19%
20%
21%
22%
23%
24%
25%
26%
27%
Basic model Restrictive judgment Integrative judgment
MAPE
Shari De Baets – ISF 201420
© Vlerick Business School
GENERAL COMPARISON
-1,5%
-1,0%
-0,5%
0,0%
0,5%
1,0%
1,5%
2,0%
2,5%
3,0%
3,5%
Restrictive Integrative
FCIMP
Shari De Baets – ISF 201421
© Vlerick Business School
DIRECTION OF ADJUSTMENTS
Direction of adj Restrictive Integrative
Downward 587 464
No adjustment 28 375
Upward 608 384
Countering of ‘tinkering’ withforecasts
Shari De Baets – ISF 201422
© Vlerick Business School
DIRECTION OF ADJUSTMENT
-25%
-20%
-15%
-10%
-5%
0%
5%
10%
15%
20%
25%
Restrictive Integrative
Downward
Upward
Shari De Baets – ISF 201423
© Vlerick Business School
SIZE OF ADJUSTMENT
�Restrictive judgment: 98% adjustments
�Four quantiles of 1192 adjustments (298 per Q)
-10%
-8%
-6%
-4%
-2%
0%
2%
4%
6%
8%
1 2 3 4
Shari De Baets – ISF 201424
© Vlerick Business School
SIZE OF ADJUSTMENT
�Integrative judgment: 70% adjustments
�Four quantiles of 848 adjustments (212 per Q)
-15%
-10%
-5%
0%
5%
10%
15%
20%
1 2 3 4
Restrictive
Integrative
Shari De Baets – ISF 201425
© Vlerick Business School
VOLATILITY (SD)
Shari De Baets – ISF 2014
-0,35
-0,3
-0,25
-0,2
-0,15
-0,1
-0,05
0
0,05
0,1
0,15
1 2 3 4
Restrictive
Integrative
26
© Vlerick Business School
VOLATILITY (CATEGORY)
�Low volatility: n = 850
�High volatility: n 373
or
Shari De Baets – ISF 201427
DVD, CD
© Vlerick Business School
VOLATILITY (CATEGORY)
-15%
-10%
-5%
0%
5%
10%
15%
20%
25%
30%
Restrictive Integrative
Low volatility
High volatility
Shari De Baets – ISF 201428
© Vlerick Business School
CONCLUSION
�General: integrative > restrictive
�Counters harmful adjustments
�Upward
�Small
�Too big
�Low volatility
�But can also limit benefits of restrictive judgment
�Downward
�Big
�High volatility
Shari De Baets – ISF 201429
© Vlerick Business School
CONCLUSION
�Profitability?
74%
75%
76%
77%
78%
79%
80%
81%
82%
83%
84%
Basic model Restrictive Integrative
Profit (% of max profit)
Profit
Shari De Baets – ISF 201430