On Macroeconomic Forecasting

Macroeconomic forecasts produced with macroeconomic models tend to be little better than intelligent guesswork. That is not an opinion – it is a fact. It is a fact because for decades many reputable and long standing model based forecasters have looked at their past errors, and that is what they find. It is also a fact because we can use models to generate standard errors for forecasts, as well as the most likely outcome that gets all the attention. Doing so indicates errors of a similar magnitude as those observed from past forecasts. In other words, model based forecasts are predictably bad.

The sad news is that this situation has not changed since I was involved in forecasting around 30 years ago. During the years before the Great Recession (the Great Moderation) forecasts might have appeared to get better, but that was because most economies became less volatile. As is well known, the Great Recession was completely missed. Forecasting has not improved, because our ability to explain variables like consumption or investment has not improved.

Does that mean that macroeconomics is not making any progress? I do not want to get sidetracked on this issue, but it could just be that as macroeconomists understand the economy as it was a little better, the nature of the economy also changes because of factors like financial innovation or technical progress. Does this mean macroeconomics is useless? No, in much the same way as medicine cannot predict year by year how your health changes but is quite good at responding to these changes.

What it does mean is that it is very difficult to use forecast performance as a means of judging between alternative models or organisations. Who is better in any one year is largely luck. You need to look at performance over at least a decade to be able to distinguish between luck and a better model or better judgement. Unfortunately models, and the people who use them, rarely remain unchanged for this length of time. The model and the modelling team that the Bank of England uses to forecast is different from its model and team ten years ago. (For much more on this see Ben Broadbent here.) 

I think it is safe to say that this inability to accurately forecast is unlikely to change anytime soon. Which raises an obvious question: why do people still use often elaborate models to forecast? Here it is useful to distinguish between policy making bodies like central banks and the rest.

It makes sense for both monetary and fiscal authorities to forecast. So why use the combination of a macroeconomic model and judgement to do so, rather than intelligent guesswork? (Intelligent guesswork here means some atheoretical time series forecasting technique.) The first point is that it is not obviously harmful to do so. (From my unsystematic reading the only consistent results from forecasting comparisons using alternative techniques is that all forecasts are pretty poor.) The second point is that forecasting using macroeconomic models allows forecasters to combine a large amount of information into a reasonably consistent story that links what has happened to what might happen, and policymakers find these stories helpful. (See, for example, this post by Corola Binder, or some of the work of Deirdre McCloskey.) There are interesting questions about what type of macromodel is best suited for forecasting, and whether the model used for forecasting should also be used for policy analysis, but I’ll leave those for another day.

Many other organisations, not directly involved in policy making, produce macro forecasts. Why do they bother? Why not just use the policy makers’ forecast? A large part of the answer must be that the media shows great interest in these forecasts. Why is this? I’m tempted to say it’s for the same reason as many people read daily horoscopes. However I think it’s worth adding that there is a small element of a conspiracy to deceive going on here too.

To set the scene, consider a similar little conspiracy. There is now a convention that when there are interesting aggregate moves on the stock or currency markets, the media will present some ‘expert’ - typically an economist working for some financial company - who will tell us why the market has so moved. The truth is that no one knows why the market goes up or down, because no one asks each trader why they are making a trade. So all the expert can give us is an unverifiable intelligent guess, but they never tell you that. This small deception suits the media and the experts.

In a similar way, the media likes to pretend that forecasts are much more accurate than they actually are, because that makes small changes newsworthy. In reality forecasts tend to follow each other and change slowly, for reasons Tim Harford notes, so presenting some new forecast every week makes little sense. (Occasionally this conservatism among forecasters can be usefully predicted, as I once noted in an anecdote.) But this conspiracy has the added bonus for the media that it can express horrified shock and surprise when forecasts go wrong.

The rather boring truth is that it is entirely predictable that forecasters will miss major recessions, just as it is equally predictable that each time this happens we get hundreds of articles written asking what has gone wrong with macro forecasting. The answer is always the same - nothing. Macroeconomic model based forecasts are always bad, but probably no worse than intelligent guesses.