International Journal of Forecasting 2 (1986) 401-402 North-Holland

401

EDITORIAL

Simple is Best?

“Forecasting is the art of saying what will happen, and then explaining why it didn’t”.

This hoary old saying reveals a sceptical attitude to forecasting which is probably rather widespread. Indeed the unreliability of the product is paradoxically one reason for the increasing effort being devoted to it. Although we know extrapolation is inherently dangerous, there are many occasions where forecasts have to be made whether we like it or not. The question is how?

The rich variety of different forecasting methods can be rather bewildering for the less experi- enced potential user, and perhaps even for the more knowledgeable. It is now generally accepted that there is no single “best” method, but rather that the choice depends on a variety of considerations such as the objectives, the skill of the analyst, the length of the forecasting horizon, and so on. Despite the increasing wealth of scientific evidence, we should perhaps also admit that our choice of method depends very much on our personal prejudices, which in turn depend largely on the relative importance we attach subjectively to such factors as (a) accuracy, (b) ease of use and interpretation, and (c) cost. It is also strongly influenced by the sort of forecasting problem we regularly meet. In my own case, this has typically been time-series forecasting problems with a fairly short forecasting horizon.

As a result of my experience and general prejudices, I generally prefer a simple approach, wherever possible, not only because other people will be able to understand what I am doing (and that is very important), but also so that Z can understand what I am doing! In any case, I have yet to see convincing evidence that complicated methods generally give much improvement in accuracy (though there will of course be exceptions). My ‘pet’ time-series method is the type of exponential smoothing which is often known as the Holt-Winters method. It allows trend and seasonal terms which are updated by exponential smoothing.

My preference for the simple approach applies not only to forecasting but also to statistical methods in general, as described in my paper [Chatfield (1985)] on the initial examination of data. I suggest that the challenge for statisticians is not just to dev,elop new, and ever more complex, methods, but rather to find ways of making better use of existing methods and to find ways of communicating them to practitioners. I emphasise the importance of looking at data, before choosing a method of analysis. In time-series analysis, this means looking at time plots of the given series. Indeed this is often the most important stage of a time-series analysis. It is usually possible to see if the series has trend and seasonality, whether the irregular variation is large or small, and whether there are outliers or changes in structure which will make routine forecasting methods inappropriate. There is another old saying, sometimes called Newbold’s theorem, which says that if you think you can get good forecasts by eye, then time-series methods will work well: if you can’t, then they won’t!!

The notion that “simple is best” has of course been said before and is backed by a wealth of empirical evidence [e.g., Armstrong (1985), Makridakis (1986, Exhibit l)]. The notion will probably appeal to many readers, but not to many econometricians who are usually more concerned with

0169-2070/86/$3.50 0 1986, Elsevier Science Publishers B.V. (North-Holland)

402 Editorial

explaining the system, rather than in forecasting. Indeed, econometricians tend to belittle univariate projection methods even when they fare better than more complicated multivariate methods. While there is a place for many different approaches, my own prejudices have been reinforced by two recent experiences. An acquaintance who runs a forecasting group in a large company produces quarterly forecasts for management using a multivariate package. Unfortunately, the computer broke down the day before the last quarterly meeting with management, and so my acquaintance cheerfully guesstimated all the forecasts by hand without telling anyone. No-one noticed!

My second example relates to the paper by Harvey and Durbin (summarized later in this issue in ‘Research on Forecasting’) which applies structural modelling to monthly accident data. Two explanatory variables are used to modify the projection forecasts, and in some cases, their effect is found to be ‘significant’. However the improvement in fit (e.g., R2 increases from 0.76 to 0.78) seems of little practical importance and in my view the considerable additional complexity cannot be justified.

Another striking feature of the Harvey-Durbin paper, and its ensuing discussion, is that very similar forecasts are obtained using structural modelling, ARIMA modelling and the Holt-Winters method. This is primarily because the series exhibit fairly regular properties before the advent of seat-belt legislation and so ‘Newbold’s theorem’ applies. While methods sometimes give very different forecasts, it is also important to realise that they often give very similar forecasts. The choice of method may be more crucial when (a) irregular variation is large, (b) there are possible outliers or changes in structure, (c) explanatory variables are thought to be important. Of course it is these sort of situations where forecasts are likely to be more unreliable anyway. In the Harvey-Durbin paper, the main reason for producing ‘forecasts’ (of data which were already available) was to see if there had been a reduction in casualties as a result of the seat-belt legislation. My contribution to the discussion suggested that the reduction for some series was ‘obvious’ just by looking at plots of the series. If this is the case, then the forecasts merely serve to provide a ‘scientific’ justification for the obvious, which is obtained by a very simple method indeed.

The remaining question is to assess why simple is often best. There are a number of contributory reasons, but I would guess that the main explanation arises from the ‘well-known’ fact that fitting the ‘best’ model to historical data does not necessarily produce the ‘best’ post-sample forecasts. One can always improve the fit by making the model more and more complicated, but all models are approximations to truth and may change over time anyway. It appears that simple models are in some way more robust for extrapolation. This is not the place to develop these ideas and so I conclude by asserting that ‘simple is (usually) best’, at least for the sort of problems I have been associated with, and to suggest that the reader tries a complicated method only when dissatisfied with simple procedures.

References

Armstrong, J.S., 1985, Forecasting by extrapolation: Conclusions from 25 years of research, Interfaces 14, 52-61. Chatfield, C., 1985, The initial examination of data (with discussion), Journal of the Royal Statistical Society, Series A, 148,

214-253. Harvey, A.C. and J. Durbin, 1986, The effects of seat-belt legislation on British road casualties: A case study in structural

modelling (with discussion), Journal of the Royal Statistical Society, Series A, 149. Makridakis, S., 1986, The art and science of forecasting: An assessment and future directions, International Journal of

Forecasting 2, 15-39.

Chris Chatfield