AN EMPIRICAL INVESTIGATION ON THE VALUE OF COMBINED JUDGMENTAL AND

STATISTICAL FORECASTING

PROF. DR. PHILIPPE BAECKEPROF. DR. KARLIEN VANDERHEYDEN

DRS. SHARI DE BAETS

CONTACT: SHARI.DEBAETS@VLERICK.COM

ISF 2014, The Netherlands

© Vlerick Business School

AN EMPIRICAL INVESTIGATION

ON THE VALUE OF COMBINED JUDGMENTAL AND STATISTICAL

FORECASTING

Shari De Baets – ISF 20142

© Vlerick Business School

AN EMPIRICAL INVESTIGATION

� “..the practice of forecasting does not however appear to have improved.” (p,33; Armstrong, Green &

Graefe, 2013)

� Ascher, 1978

� Allen, 1994

� McCarthy, Davis, Golicic & Mentzer, 2006

Shari De Baets – ISF 20143

© Vlerick Business School

AN EMPIRICAL INVESTIGATION

An important task for researchers in our field:

Improving forecastingaccuracy in practice

(Sanders & Manrodt, 2003)

Shari De Baets – ISF 20144

© Vlerick Business School

AN EMPIRICAL INVESTIGATION

�Call for more studies with real company data (Sanders, 2009)

�Going beyond artificial experiments to deriverules for practice

�Expert forecasters

�Context of forecasting task

�A new research model: JUD 3 session(room “Plate”, Wed., 10 AM)

Shari De Baets – ISF 20145

© Vlerick Business School

AN EMPIRICAL INVESTIGATION

ON THE VALUE OF COMBINED JUDGMENTAL AND STATISTICAL

FORECASTING

Shari De Baets – ISF 20146

© Vlerick Business School

THE VALUE OF COMBINED FORECASTING

25%

25%

17%

33%

Forecasting method

judgment alone

statistical methods alone

average of statistical and

judgmental forecast

statistical forecast adjusted

judgmentally

Fildes & Goodwin, 2007

Shari De Baets – ISF 20147

© Vlerick Business School

THE VALUE OF COMBINED FORECASTING

� Potential of judgment in forecasting oftenundermined by unneccessary adjustments(Lawrence et al., 2006)

� Patterns in noise (Harvey, 1995)

� Illusion of control effect (Kotteman, Davis, & Remus,

1994)

� Persists despite warning (Lim & O’Connor, 1995)

Shari De Baets – ISF 20148

© Vlerick Business School

RESEARCH QUESTION

How can we counter damaging adjustments andreap the potential benefits from judgment, andthus, heighten forecasting accuracy?

Shari De Baets – ISF 20149

© Vlerick Business School

HYPOTHESES

� Data augmentation:

� Classic model + judgment:

Var1, Var2, .., Varn -> outcome -> judgmentaladjustment

� Judgment incorporated in model

Var1, Var2, VarJudgment , .., Varn -> outcome

Shari De Baets – ISF 201410

© Vlerick Business School

HYPOTHESES

�Data augmentation:

� H1:

� “integrated judgment”: judgment as part of the model

will outperform

� “restrictive judgment”: judgment as restriction on the model (judgmental adjustment)

Shari De Baets – ISF 201411

© Vlerick Business School

HYPOTHESES (SIMILAR TO FILDES ET AL., 2009)

� H1 applied to

�Direction of the adjustment

�Downward adjustments are beneficial, upwardadjustment are damaging

�Size of the adjustment

�Curvilinear (inverted U-shape) effect: both small and very large adjustments are damaging

�Volatility of the data series

� Judgment is beneficial in high volatility series (volatility due to special events)

Shari De Baets – ISF 201412

© Vlerick Business School

THE COMPANY

Shari De Baets – ISF 201413

Outlet store

Outlet store

Outlet store

International publishing company

Weekly and monthlymagazines

© Vlerick Business School

PROCEDURE

� Demand forecasting

� Predictive model: forecast of expected demandper store

� Optimisation model: profit optimisation – finalnumber for supply per store

Input

Shari De Baets – ISF 201414

© Vlerick Business School

THE COMPANY

� Profit optimization model

Overstock

Stockout

Shari De Baets – ISF 201415

© Vlerick Business School

PROCEDURE

�Predictive model: forecast of expected demandper store

�Optimisation model: profit optimisation – finalnumber for supply per store

�Judgmental adjustment: according to insight

Shari De Baets – ISF 201416

© Vlerick Business School

PROCEDURE

Predictive Optimisation Judgmentalmodel model adjustment

Restrictive judgment

Shari De Baets – ISF 201417

© Vlerick Business School

PROCEDURE – DATA AUGMENTATION

Predictive Optimisation Judgmentalmodel model adjustment

Integrative judgment

Shari De Baets – ISF 201418

© Vlerick Business School

PROCEDURE – DATA AUGMENTATION

Predictive model Optimisation model(incl. judgmentparameter)

Integrative judgment

Shari De Baets – ISF 201419

© Vlerick Business School

GENERAL COMPARISON N = 1223

19%

20%

21%

22%

23%

24%

25%

26%

27%

Basic model Restrictive judgment Integrative judgment

MAPE

Shari De Baets – ISF 201420

© Vlerick Business School

GENERAL COMPARISON

-1,5%

-1,0%

-0,5%

0,0%

0,5%

1,0%

1,5%

2,0%

2,5%

3,0%

3,5%

Restrictive Integrative

FCIMP

Shari De Baets – ISF 201421

© Vlerick Business School

DIRECTION OF ADJUSTMENTS

Direction of adj Restrictive Integrative

Downward 587 464

No adjustment 28 375

Upward 608 384

Countering of ‘tinkering’ withforecasts

Shari De Baets – ISF 201422

© Vlerick Business School

DIRECTION OF ADJUSTMENT

-25%

-20%

-15%

-10%

-5%

0%

5%

10%

15%

20%

25%

Restrictive Integrative

Downward

Upward

Shari De Baets – ISF 201423

© Vlerick Business School

SIZE OF ADJUSTMENT

�Restrictive judgment: 98% adjustments

�Four quantiles of 1192 adjustments (298 per Q)

-10%

-8%

-6%

-4%

-2%

0%

2%

4%

6%

8%

1 2 3 4

Shari De Baets – ISF 201424

© Vlerick Business School

SIZE OF ADJUSTMENT

�Integrative judgment: 70% adjustments

�Four quantiles of 848 adjustments (212 per Q)

-15%

-10%

-5%

0%

5%

10%

15%

20%

1 2 3 4

Restrictive

Integrative

Shari De Baets – ISF 201425

© Vlerick Business School

VOLATILITY (SD)

Shari De Baets – ISF 2014

-0,35

-0,3

-0,25

-0,2

-0,15

-0,1

-0,05

0

0,05

0,1

0,15

1 2 3 4

Restrictive

Integrative

26

© Vlerick Business School

VOLATILITY (CATEGORY)

�Low volatility: n = 850

�High volatility: n 373

or

Shari De Baets – ISF 201427

DVD, CD

© Vlerick Business School

VOLATILITY (CATEGORY)

-15%

-10%

-5%

0%

5%

10%

15%

20%

25%

30%

Restrictive Integrative

Low volatility

High volatility

Shari De Baets – ISF 201428

© Vlerick Business School

CONCLUSION

�General: integrative > restrictive

�Counters harmful adjustments

�Upward

�Small

�Too big

�Low volatility

�But can also limit benefits of restrictive judgment

�Downward

�Big

�High volatility

Shari De Baets – ISF 201429

© Vlerick Business School

CONCLUSION

�Profitability?

74%

75%

76%

77%

78%

79%

80%

81%

82%

83%

84%

Basic model Restrictive Integrative

Profit (% of max profit)

Profit

Shari De Baets – ISF 201430

THANK YOU

CONTACT: SHARI.DEBAETS@VLERICK.COM

ISF 2014, The Netherlands