Macroeconomic forecasts produced with macroeconomic models tend
to be little better than intelligent guesswork. That is not an opinion – it is
a fact. It is a fact because for decades many reputable and long standing model
based forecasters have looked at their past errors, and that is what they find.
It is also a fact because we can use models to generate standard errors for
forecasts, as well as the most likely outcome that gets all the attention.
Doing so indicates errors of a similar magnitude as those observed from past
forecasts. In other words, model based forecasts are predictably bad.
The sad news is that this situation has not changed since I was
involved in forecasting around 30 years ago. During the years before the Great
Recession (the Great Moderation) forecasts might have appeared to get better,
but that was because most economies became less volatile. As is well known, the Great Recession was completely
missed. Forecasting has not improved, because our ability to explain variables
like consumption or investment has not improved.
Does that mean that macroeconomics is not making any progress?
I do not want to get sidetracked on this issue, but it could just be that as
macroeconomists understand the economy as
it was a little better, the nature of the economy also changes because of
factors like financial innovation or technical progress. Does this mean
macroeconomics is useless? No, in much the same way as medicine cannot predict
year by year how your health changes but is quite good at responding to these
changes.
What it does mean is that it is very difficult to use forecast
performance as a means of judging between alternative models or organisations.
Who is better in any one year is largely luck. You need to look at performance
over at least a decade to be able to distinguish between luck and a better
model or better judgement. Unfortunately models, and the people who use them,
rarely remain unchanged for this length of time. The model and the modelling
team that the Bank of England uses to forecast is different from its model and
team ten years ago. (For much more on this see Ben Broadbent here.)
I think it is safe to say that this inability to accurately
forecast is unlikely to change anytime soon. Which raises an obvious question:
why do people still use often elaborate models to forecast? Here it is useful
to distinguish between policy making bodies like central banks and the rest.
It makes sense for both monetary and fiscal authorities to
forecast. So why use the combination of a macroeconomic model and judgement to
do so, rather than intelligent guesswork? (Intelligent guesswork here means
some atheoretical time series forecasting technique.) The first point is that
it is not obviously harmful to do so. (From my unsystematic reading the only
consistent results from forecasting comparisons using alternative techniques is
that all forecasts are pretty poor.) The second point is that forecasting using
macroeconomic models allows forecasters to combine a large amount of
information into a reasonably consistent story that links what has happened to
what might happen, and policymakers find these stories helpful. (See, for
example, this post by Corola Binder, or some of the work of Deirdre
McCloskey.) There are interesting questions
about what type of macromodel is best suited for forecasting, and whether the
model used for forecasting should also be used for policy analysis, but I’ll
leave those for another day.
Many other organisations, not directly involved in policy
making, produce macro forecasts. Why do they bother? Why not just use the
policy makers’ forecast? A large part of the answer must be that the media
shows great interest in these forecasts. Why is this? I’m tempted to say it’s
for the same reason as many people read daily horoscopes. However I think it’s
worth adding that there is a small element of a conspiracy to deceive going on
here too.
To set the scene, consider a similar little conspiracy. There
is now a convention that when there are interesting aggregate moves on the
stock or currency markets, the media will present some ‘expert’ - typically an
economist working for some financial company - who will tell us why the market
has so moved. The truth is that no one knows why the market goes up or down,
because no one asks each trader why they are making a trade. So all the expert
can give us is an unverifiable intelligent guess, but they never tell you that.
This small deception suits the media and the experts.
In a similar way, the media likes to pretend that forecasts are
much more accurate than they actually are, because that makes small changes
newsworthy. In reality forecasts tend to follow each other and change slowly,
for reasons Tim Harford notes, so presenting some new forecast every
week makes little sense. (Occasionally this conservatism among forecasters can
be usefully predicted, as I once noted in an anecdote.) But this conspiracy has
the added bonus for the media that it can express horrified shock and surprise
when forecasts go wrong.
The rather boring truth is that it is entirely predictable that forecasters will miss major
recessions, just as it is equally predictable that each time this happens we
get hundreds of articles written asking what has gone wrong with macro
forecasting. The answer is always the same - nothing. Macroeconomic model based
forecasts are always bad, but probably no worse than intelligent guesses.
