Monday, 11 August 2014

On Macroeconomic Forecasting

Macroeconomic forecasts produced with macroeconomic models tend to be little better than intelligent guesswork. That is not an opinion – it is a fact. It is a fact because for decades many reputable and long standing model based forecasters have looked at their past errors, and that is what they find. It is also a fact because we can use models to generate standard errors for forecasts, as well as the most likely outcome that gets all the attention. Doing so indicates errors of a similar magnitude as those observed from past forecasts. In other words, model based forecasts are predictably bad.

The sad news is that this situation has not changed since I was involved in forecasting around 30 years ago. During the years before the Great Recession (the Great Moderation) forecasts might have appeared to get better, but that was because most economies became less volatile. As is well known, the Great Recession was completely missed. Forecasting has not improved, because our ability to explain variables like consumption or investment has not improved.

Does that mean that macroeconomics is not making any progress? I do not want to get sidetracked on this issue, but it could just be that as macroeconomists understand the economy as it was a little better, the nature of the economy also changes because of factors like financial innovation or technical progress. Does this mean macroeconomics is useless? No, in much the same way as medicine cannot predict year by year how your health changes but is quite good at responding to these changes.

What it does mean is that it is very difficult to use forecast performance as a means of judging between alternative models or organisations. Who is better in any one year is largely luck. You need to look at performance over at least a decade to be able to distinguish between luck and a better model or better judgement. Unfortunately models, and the people who use them, rarely remain unchanged for this length of time. The model and the modelling team that the Bank of England uses to forecast is different from its model and team ten years ago. (For much more on this see Ben Broadbent here.) 

I think it is safe to say that this inability to accurately forecast is unlikely to change anytime soon. Which raises an obvious question: why do people still use often elaborate models to forecast? Here it is useful to distinguish between policy making bodies like central banks and the rest.

It makes sense for both monetary and fiscal authorities to forecast. So why use the combination of a macroeconomic model and judgement to do so, rather than intelligent guesswork? (Intelligent guesswork here means some atheoretical time series forecasting technique.) The first point is that it is not obviously harmful to do so. (From my unsystematic reading the only consistent results from forecasting comparisons using alternative techniques is that all forecasts are pretty poor.) The second point is that forecasting using macroeconomic models allows forecasters to combine a large amount of information into a reasonably consistent story that links what has happened to what might happen, and policymakers find these stories helpful. (See, for example, this post by Corola Binder, or some of the work of Deirdre McCloskey.) There are interesting questions about what type of macromodel is best suited for forecasting, and whether the model used for forecasting should also be used for policy analysis, but I’ll leave those for another day.

Many other organisations, not directly involved in policy making, produce macro forecasts. Why do they bother? Why not just use the policy makers’ forecast? A large part of the answer must be that the media shows great interest in these forecasts. Why is this? I’m tempted to say it’s for the same reason as many people read daily horoscopes. However I think it’s worth adding that there is a small element of a conspiracy to deceive going on here too.

To set the scene, consider a similar little conspiracy. There is now a convention that when there are interesting aggregate moves on the stock or currency markets, the media will present some ‘expert’ - typically an economist working for some financial company - who will tell us why the market has so moved. The truth is that no one knows why the market goes up or down, because no one asks each trader why they are making a trade. So all the expert can give us is an unverifiable intelligent guess, but they never tell you that. This small deception suits the media and the experts.

In a similar way, the media likes to pretend that forecasts are much more accurate than they actually are, because that makes small changes newsworthy. In reality forecasts tend to follow each other and change slowly, for reasons Tim Harford notes, so presenting some new forecast every week makes little sense. (Occasionally this conservatism among forecasters can be usefully predicted, as I once noted in an anecdote.) But this conspiracy has the added bonus for the media that it can express horrified shock and surprise when forecasts go wrong.

The rather boring truth is that it is entirely predictable that forecasters will miss major recessions, just as it is equally predictable that each time this happens we get hundreds of articles written asking what has gone wrong with macro forecasting. The answer is always the same - nothing. Macroeconomic model based forecasts are always bad, but probably no worse than intelligent guesses.


42 comments:

  1. Blogs with racing you make your blog a candidate to win the promotional prizes. With the support of our sponsors from blogs are organizing events within the first round of our competition. Add your blog and get involved. You refer to your blog. You can find detail information on our website. Thank you for your interested-in.


    Sincerely,
    Web: http://www.bloggercontest.info
    Mail: contact@bloggercontest.info
    Cell: +441269200450


    ReplyDelete
    Replies
    1. Where's the LIKE button? This comment is almost as entertaining as the original post!

      Delete
    2. Button works about as well as the typical forecast.

      Delete
  2. I feel like I am missing your point, but isn't one obvious problem that the macroeconomic models in use conceive of things like the great recession as originating from "shocks" which cannot be forecast by definition.

    It strikes me that even if economists started using models with endogenous crises, they would probably be more useful as early warning systems rather than generators of forecasts to be evaluated against time series data

    ReplyDelete
    Replies
    1. More to the point, George Soros already solve this riddle with his theory on reflexivity.

      Much like when you bop someone in the knee the leg delivers a foot to your face with out any violent intentions, the economy reacts to forecasts. So the models aren't inaccurate per se. The models correctly predict the economic and market outcomes but sans the forecast. Once the forecast is released for public consumption, the economy and the market are incrementally different from the one the forecaster based his predictions on. OK?

      So the only forecasts that are correct must be kept hidden form the market and the public. Does that make sense to anybody?

      Delete
    2. As an investor, I think that the "shock" theory is vastly overrated. I usually think of trends as being driven by and consuming some "fuel". When the "fuel" runs out, whether it be subprime borrowers, internet investors, two income families, retirement savers or what have you, the trend changes dramatically. You can think of it as a shock, but it is not an unpredictable shock. You can't set the date, but you can brace for the impact.

      Delete
  3. I am under the impression (with some modeling to back it up) that there is a basic trend to e.g. RGDP with somewhat normally distributed noise ... all of which derives from pretty basic supply and demand arguments. On top of that are sporadic large shocks (recessions) -- these may be some of the 2-sigma random noise that gets turned into 4-sigma events because of human behavior (feedback), or maybe some other mechanism (I don't know the answer to this). The key to getting the right distribution is getting that trend right.

    This results in a pretty decent forecasting capability:

    http://informationtransfereconomics.blogspot.com/2014/08/prediction-update-not-bad-for-five.html

    I'm going to keep updating and see if/when it fails.

    With sufficient data, one should be able to determine e.g. a mean time between large shocks, however I just don't think enough data is available yet. There might be other indicators like deviation from the trend that may be fruitful, but are as yet unknown.

    ReplyDelete
    Replies
    1. Jason, I'm glad to see you made a comment... I was going to provide a link myself (t your posts). ... shoot, maybe I will anyway: here's a link to another comment on another blog with several of Jason's forecasts, including about the US, Canada, Japan and China:

      http://pragcap.com/forums/topic/against-human-centric-macroeconomics/page/2/#post-71206

      Delete
  4. I was thinking earlier this morning how statistics is such a bad and dangerous practice. It really can get things so wrong when an economic constraint is unknown.
    I use a theoretical model with relationships/laws of economics. Labor share caps a limit on the utilization of labor and capital. It is a real time calculation. The model also estimates the output gap in real time with changes in labor share and capacity utilization. The US economy followed my real-time model in the past and so far it is once again.
    http://angrybearblog.com/2014/08/new-revisions-of-labor-share.html

    Link even shows US productivity continues to be constrained against the effective demand limit... something which should attract attention from the UK in order to explain the "productivity puzzle" there.

    ReplyDelete
  5. As far as I understand, one of the main reasons macroeconomic forecasting under-performs is that it does not take into account disequilibrium and the dynamics of money, profitability and debt.
    For example, Wynne Godley and Steve Keen actually predicted the Great Recession (the last one), by using models. Other economists, like Shaikh, or even non-economists (like David Harvey) had warned as long before, as well.

    The sad truth though is that this kind of economists are marginalized by the majority of academics and media. I would be grateful for any comments on their model based predictions.

    ReplyDelete
  6. I don't think things are nearly this bad. Macro can make reasonable conditional forecasts - and that is valuable.
    One feature of a conditional forecast is it allows you to test various outcomes - such as what happens if stimulus; if zero rates, if China slows or any such question.
    I think this is the primary motive both for private sector consumers of econ forecasts to make their own forecasts and to use forecasts. People want to ask IF? and get some answer THEN...

    I don't know if it has been tested whether Macro models have improved their ability to make conditional forecasts.

    But it is worth a lot to say IF financial crisis THEN, what?? well its naturally impossible to forecast if a few guys in a room will decide to let Lehman go bankrupt or not, but its not so hard to forecast what will happen depending on which choice they make.

    Why isn't that enormously valuable?

    I don't find the complaints about Macro's failings in the forecast department to be all that notable.

    Think about a world where Macro makes reliable forecasts - naturally such a world is impossible.

    ReplyDelete
    Replies
    1. Of course - that is the point of my remark about medicine. That is what academics do with macro models. But most forecasters do not do this - they make unconditional forecasts.

      Delete
    2. Your last statement is unfortunately true if you include all the financial institutions economists. But it is not true if you focus on the core group of specialists in macro analysis at policy institutions or private sector consultancies such as Moody's or Oxford Economics. The IMF produces a yearly spillovers report with different scenarios. Central banks provide lots of impulse response and scenario analysis in various papers and documents. Private sector consultancies like OE put a lot of effort into creating and marketing alternative macro scenarios, as well as their macro model and analysis software that allows you to create your own scenarios. These things are less visible because they often require significant annual subscription fees (which are not that expensive for major financial institutions and large businesses at least). And there is a demand in the press for things like "what would a Chinese hard landing look like? What would be the costs of a Eurozone break up?". Types of questions for which we know the answer is likely to be wrong, but still could be better than guesswork. Finally, there are occasional attempts to evaluate macro models' conditional forecasts. For example,
      http://www.sas.upenn.edu/~egme/pp/CompletePaperRevforDistribution.pdf
      .

      Delete
    3. Ok, yes I agree the medicine example is an excellent one though I think you could sharpen it up and stand it alone as part of the response to the questions you ask. Though I understand your point is to ask since models are good at conditional forecasts, why make unconditional ones where they aren't suitable. I don't think the distinction is always appreciated, even by professional consumers.

      But I love the medicine analogy as a way to ask the question, what is a healthy economy? Here is a simple question that follows naturally: what level of government spending as a percent of GDP represents a healthy economy?

      What economists have direct clear and thorough analysis of that question? I think I could get a response for consumption, investment I don't think I have ever seen one for government beyond Laffer's laugher stating that it can't be 0 or 100.

      Let's ask the question this way - what level of government spending will support the highest level of investment and consumption.

      I know there is lots of discussion about what government does and what the role of government is, and Arthur Lewis naturally explored the relationships between components of economic activity in his growth analysis. I think extending his work there must be something??

      The whole premise of the question is exactly backwards. No one goes to an architect and asks them to predict what their new home will look like, they go and ask them to design one. What will our new economy look like? well it'll look like what we choose to build. lets design one we want to live with.

      Delete
  7. Magnus Carlsen11 August 2014 14:12

    There may be an inherent reason why macro forecasts can never be any better than some simple curve fit.

    The Efficient Market Hypothesis.

    I realise those three words will send half your readership into apoplexy, but basically the idea is just that all current information gets optimally used in current spending decisions. So output, which in the short run is governed by demand, already incorporates everything which is known.

    If I thought there was going to be a crash tomorrow, then I would start cutting spending today. The crash gets brought forward to the present. The point at which spending changes is the point at which new information arrives.

    One way of looking at this is to say that the incapacity to forecast is, paradoxically, a vindication of economic theory.

    PS I very much like the characterisation of the media plus City 'economist' as a conspiracy. Spot on.

    ReplyDelete
  8. If economic theory cannot predict, of what practical use is it? Might as well study Latin or Greek.

    ReplyDelete
    Replies
    1. Magnus Carlsen11 August 2014 14:35

      Because it can tell us which policies to use in particular circumstances.

      Delete
    2. And yet economists can not only agree on which policies to use, they cannot even agree on the specific nature of the particular circumstances which we are experiences.

      Delete
    3. Magnus Carlsen11 August 2014 22:24

      Not true.

      They agreed that quantitative easing was appropriate when interest rates went to zero in the last election. You never know the counterfactual for sure, but the lesson of the 1930s, as well as now mainstream theory, told us this was the right thing to do. There were many non-economist shrieking 'inflation' at this point, but fortunately the mainstream prevailed here.

      They agree that some sort of Taylor rule is appropriate during 'normal times' - I.e. when we're not at zero interest rates.

      They also agree that expansionary fiscal policy helps to stabilise the economy when output is low relative to potential, and when interest rate policy is unavailable. Though it does indeed suit a rightwing media and establishment to portray disagreement on this. Sadly some people get sucked in by this narrative.

      Delete
    4. There is nothing wrong with disagreement on these complex issues. If there was not, that would be a cause for concern.

      It is a myth when people say give me 12 economists and I will give you twelve different answers.

      The truth is there is not much diversity in economics today, especially in macro-economics. And that is certainly not because they have found the answers to today's problems. Far from it. It has more to do with promotion in economics departments being dependent on publications in key journals (AER, JME) which demand work produced a certain way, however questionable and unenlightening it has over and over again been shown to be.

      If there were 12 different answers, things would be a lot healthier!

      Delete
  9. Magnus Carlsen11 August 2014 14:29

    There may be an inherent reason why macro forecasts can never be any better than some simple curve fit.

    The Efficient Market Hypothesis.

    I realise those three words will send half your readership into apoplexy, but basically the idea is just that all current information gets optimally used in current spending decisions. So output, which in the short run is governed by demand, already incorporates everything which is known.

    If I thought there was going to be a crash tomorrow, then I would start cutting spending today. The crash gets brought forward to the present. The point at which spending changes is the point at which new information arrives.

    One way of looking at this is to say that the incapacity to forecast is, paradoxically, a vindication of economic theory.

    PS I very much like the characterisation of the media plus City 'economist' as a conspiracy. Spot on.

    ReplyDelete
  10. "One way of looking at this is to say that the incapacity to forecast is, paradoxically, a vindication of economic theory."

    Basically we know very little about the future. We do not need economic theory to tell us that. I also think there are much better explanations around about how agents act in uncertainty than the REH. I am not sure that all information, current or otherwise gets optimally used by anyone. As someone pointed out here, we have, for example, short memories. And can we really generalise about the behaviour of all participants? I also do not think this is the way to understand how demand is driven. Better to try and find out actually what does happen than make up a lot of nonsense to get a mathematical model.

    I think you find that psychologists or perhaps people like Minsky have more substantial things to say about how people conduct their lives and markets operate in uncertainty.

    ReplyDelete
  11. It's slightly OT, but the 2007 crash was predictable in a general way. It doesn't take DSGE to predict that a radically leverage structure will eventually be hit by a shock that will cause it to collapse.

    This doesn't say when, but from a policy point of view that's irrelevant. We want stability; and, considering the damage caused by the crash, should be willing to sacrifice some amount of growth to get it.

    Larry Summers worries about regulation impairing the efficiency of the financial sector. I worry about the efficiency of the financial sector blowing up the economy.

    ReplyDelete
  12. Surely the answer is that the models that macroeconomists use are simply not suitable for forecasting turning points. All they can do is to project existing patterns ahead as they damp out, but the influence of these gets overwhelmed by the development of what were tiny disturbances at the beginning of the forecast - like the butterfly's wings problem in weather forecasting.

    Perhaps what macroeconomists need to do is a lot more survey research on the changes that occur around turning points - eg asking business managers why they cut investment - to determine how the break occurs, and try to produce some kind of strain gauge that estimates the probability of a break.

    ReplyDelete
  13. In my experience forecasters do the same thing as academic economists: just agree with everyone else and do things the same way as everyone else. It is the very brave who resist this.

    ReplyDelete
  14. Here is a thought provoking extract from Prof Vaclav Smil on the pointlessness of complicated techniques of mathematical modelling in a chapter titled "Against Forecasting" in his book Energy at the Crossoads. And this is a guy with nearly 40 years' experience under his belt: he gives some interesting examples of his own forecasts going wrong in the same book.
    SMIL, Vaclav (2003). Energy at the Crossroads. MIT Press, 2005, pp 170-172
    "If seven billion people (the lowest total used by the IPCC scenarios for the year 2100) were to average 60 GJ of primary energy a year per capita (today's global mean), the total need in the year 2100 would be just 10 GTOE. But if that energy would be used with average efficiency twice as high as it is today (a rather conservative assumption, we have done better during the past 100 years), then every person on this planet would have access to as much useful energy as does an average Italian today (who eats healthier and lives longer than just about anybody else). On the other hand, if today's global mean per capita energy consumption will rise as much as it did during the twentieth century (3.6 times) then a world of 10 billion people (the highest population total used by the IPCC) would need 57 GTOE in the year 2100.
    In a single short paragraph I have outlined two highly plausible extreme
    scenarios bracketing the global energy use in the year 2100 (10-57 GTOE) in an almost identical way (12-64 GTOE) as did 40 elaborate IPCC scenarios stringing together multitudes of much more dubious assumptions. How can we even start guessing, as SRES (2001) did, what will be the per capita income or final energy intensities of economies in transition in the year 2100? What fundamentally new insights have been gained through those complicated exercises?
    ... But there is nothing new about modellers' predilections for built-in complexity. They think they can do better by making their creations progressively more complex by including more drivers and more feedbacks. They do not seem to realize that the greater complexity required to make such interactive models more realistic also necessitates the introduction of more questionable estimates (which can soon turn into pure guesses) and often also of longer chains of concatenated assumptions -- and these necessities defeat the very quest for greater realism. As the variables become more numerous and time horizons more distant, quantifications become inevitably more arbitrary."

    Remarks:
    1. GTOE is Giga Tonnes of Oil Equivalent, where one tonne of oil equivalent is the energy released by burning 1 tonne of oil -- approx 42 Giga Joules (GJ).
    2. I should point out that Smil made an arithmetic error in calculating the figure of 57 GTOE as the uppper limit: the correct figure is 51 GTOE ((3.6*60*10)/42). This correction does not alter the main conclusion.

    To me the take away message is that there are (sharply?) diminishing returns to complexity and long time horizons in model-based forecasting. In other words, it is a delusion to imagine that more complex models will necessarily deliver more accurate forecasts. This I think is undeniable, but something that is all to often forgotten.

    But that is an argument for keeping models simple and tractable, not for doing away with models. After all, even the ancient Romans who forecast by killing geese and examining their entrails were engaged in a form of modelling. Somewhat messy perhaps, but de gustibus non est disputandum.

    ReplyDelete
  15. This post is, of course, not really true.

    So, to take another recent post by S W-L where he similarly said that predictions about the future were impossible to make, he also claimed

    "academic macroeconomists have also got some important things right recently: inflation did not take off following Quantitative Easing, interest rates have stayed low despite bigger deficits, and our models said that Eurozone austerity could cause a second recession."

    http://mainlymacro.blogspot.co.uk/2014/07/what-are-academics-good-for.html

    These are examples of macroeconomic forecasting, that you want the academy to take credit for. You can't have it both ways. Either you are saying something about how the future will turn out, or you are not.

    This is an attempt here to have your cake and eat it. So, the critics of Osborne want the credit for predicting that 'austerity' would lead to less growth (which is trivial, there not being any real world believers in expansionary fiscal contraction, at least in the UK) but no demerit for failing to predict that strong growth would return when the exogenous factors impacting growth (commodity prices mainly, but also the euro) receded.

    Now, THE major reason that economists cannot predict inflection points is that they cannot predict the timing of shocks.

    So, how could an economist predict the oil price shock of the 70s? The pound being forced out of EMU (with consequent monetary policy easing)? Lehman Bros collapse? Nobody would dream of criticising someone for having economic models that could not spot these things.

    The UK 2013 upturn was different. Nothing of this kind happened.

    So, without an unpredictable shock to explain away the failure of foresight, you'll have to forgive those of us who specialise in other fields for ranking economics, for all its claims to mathematical precision, as the least impressive of the social sciences. It all looks a bit half arsed.

    ReplyDelete
    Replies
    1. A predictably muddled, distorted view of the main point raised by SWL. I’d stick to the other field you specialise in (Spinning? Sophistry?) as you appear to be less than 'a bit half-arsed' when it comes to economics.

      For the paragraphs regarding “This is an attempt here to have your cake and eat it”:

      No it isn’t – whilst SWL rightly says that forecasting is very difficult and economists often get it wrong, this doesn’t mean that they never get it right and can’t/shouldn’t claim credit for so doing so…such as correctly forecasting and warning of the impact of Osborne’s austerity policies which cost the economy well over one hundred billion pounds in lost output and unemployment as a result of the prolonged recession…hardly a trivial matter.

      Whilst the strength of the recovery was indeed underestimated by economists, this is simply a prime example of what SWL alludes to illustrating the main point of his post! They often do get it wrong as forecasting is difficult, and have taken in the neck for doing so.

      You note the upturn in 2013 and “exogenous factors impacting growth (commodity prices mainly, but also the euro) receded.” Fine.

      Why do you choose to ignore one of the main catalysts for this upturn?

      As well as the factors you mention, the government’s own 2014 Budget report (formed by Osborne, the OBR and a cross-party committee of MPs) officially record that:
      OBR: “there is slightly less of a drag from fiscal consolidation over this period than in the earlier period as well, so that would have an impact. On the business investment side, in addition to demand issues, you also have a greater need for replacement of investment, and the longer investment is weak the more powerful that becomes as a factor”.
      “…some reduction in the pace of fiscal consolidation have all played a role in the recovery.
      And the figures: “Measured by the change in real-terms public sector net borrowing, there was a fiscal consolidation of £39.2bn between 2011–12 and 2012–13, but a forecast fiscal loosening of £13.3bn between 2012–13 and 2013–14”

      ie. Osborne greatly eased up on austerity which allowed growth to pick up…hence the abandonment of Plan A, despite the ongoing falsehood of claiming to have stuck to their long-term plan and it is now paying off. They didn’t at all as the original plan never worked!

      Likewise from the budget report: “Paul Mortimer-Lee, head of market economics at BNP Paribas, said: What we saw was a removal of headwinds […] the crisis in Europe abated and that allowed optimism to come forward. There were measures to stimulate the housing market. We have to say that the pace of structural fiscal tightening was rather less than it had been in previous years.”
      And…
      “Michael Saunders, Head of West European Economics at Citi said that “the shift from very heavy fiscal restraint to almost no fiscal restraint in 2012, 2013 and 2014” had “lifted a headwind”.”

      http://www.publications.parliament.uk/pa/cm201314/cmselect/cmtreasy/1189/1189.pdf p9-10

      Again, can you read this blog without distorting it to fit your own misguided prejudices? And again, please provide the web-link for your claim that the FT economic forecasts show how badly academic forecasters were compared to city forecasters in the years prior to 2013. The one you have cited above where you falsely accused SWL of cherry-picking data - by actually making up evidence yourself!

      http://mainlymacro.blogspot.co.uk/2014/07/what-are-academics-good-for.html

      Delete
    2. 2011
      http://www.ft.com/cms/s/0/ff88a3b8-175e-11e0-badd-00144feabdc0.html#axzz3ALMSf45S

      2012
      http://www.ft.com/cms/s/0/0b766312-353c-11e1-84b9-00144feabdc0.html#axzz3ALMSf45S

      2013
      http://www.ft.com/cms/s/0/338a7dac-5412-11e2-bb50-00144feab49a.html#axzz3ALMSf45S

      If you want to go further back, you really will have to just google it yourself.

      I am slightly puzzled by your post.

      Are you accepting, as I would claim, that we should mark economists to market (ie test their claims according to how accurately they map on to the future)?

      http://delong.typepad.com/sdj/2012/11/mark-thoma-watches-steven-williamson-once-again-fail-to-mark-his-beliefs-to-market.html

      or, are yo with S W-L in thinking that

      "Macroeconomic forecasts produced with macroeconomic models tend to be little better than intelligent guesswork."

      (NB that is not as you seem to think a claim that it is a difficult exercise, like say brain surgery. I would of course accept that. Rather it is a claim that is just guessing.)

      So that that isn't legitimate?

      Delete
    3. Why have you provided FT web-links which show the full individual survey responses of economists with no analysis of their accuracy, rather than something to support what you (falsely) claimed to indicate the accuracy of city economists vs academics?
      Surely if you are claiming how badly academic forecasters were compared to city forecasters in the years prior to 2013 you ought to provide something which not only documents their responses but also what actually happened subsequently in order to gauge accuracy? - The sort of example that SWL provided in his original post...or the one I provided before from the Treasury where you can clearly see both sets of predictions:
      https://www.gov.uk/government/collections/data-forecasts
      ...and note that in aggregate they differ only slightly, thus completely refuting your claim that "If you look at the same FT survey and the previous year and the year before that, the track record of the academics in forecasting... is very poor, worse than those employed by the evil banksters."
      Perhaps you would take a moment to look at the treasury report above and say whether it supports or refutes your claim? And apologise to SWL (who demonstrates incredible tolerance by allowing your postings on his webpage) for accusing him of cherry-picking data?

      Very strange logic: "that is not as you seem to think a claim that it is a difficult exercise, like say brain surgery. I would of course accept that. Rather it is a claim that is just guessing"
      He's saying forecasting is "little better than intelligent guesswork" BECAUSE it is so difficult - given the myriad of permeatations and dynamic nature of human society, behaviour, shocks and events etc which all impact on the economy from time to time and therefore (potentially) render forecasts made even a week or month ago inaccurate!
      Marking economists to market and the observation made by SWL are not incompatible or mutually exclusive concepts - you can do the former and accept that the latter claim made by SWL is true. His suggestion that forecasts are little better than intelligent guesswork are based on the outcomes/results he has observed - ie. from marking to market - which reveals the poor accuracy of forecasts. Hence his conclusion.

      Like the government, you've kept a wise silence on the 2012-13 easing of austerity as a key reason for the stronger than expected return to growth.

      Delete
    4. (i) I am not going to produce a spreadsheet for you, sorry.

      I am bemused as to why you think the Treasury survey is relevant to the issue of the worth of academic economists. They don't include forecasts by academics (unless you are including quasi-academic institutions like the NIESR and Oxford Economics?)

      (ii) If forecasting is difficult to do, then we should overtime be able to compare the predictions of economists about the future and judge who performs better.
      If it is 'guesswork' as S W-L states, like calling heads on a coin toss, then we can't do that.

      (iii) I have never, and would never, dispute that a looser fiscal policy will, all other things being equal, lead to higher growth. Just as tightening fiscal policy will lead to lower growth. There are no real world believers in expansionary fiscal policy, and Krugman, Portes, and S W-L are attacking strawmen.

      But, all things are never equal. In the UK post 2010 we also had
      -a sharp rise in commodity prices, especially oil
      -an ez crisis
      -de facto tightening of monetary policy as banks reinforced their balance sheets and adjusted to new regulation

      Now, if you are hostile to Osborne you’ll downplay these factors, and focus on austerity. If you are pro-Osborne the reverse. What I would like some analysis of is the relative import of these factors, not trivia stating that fiscal tightening will lower growth.

      (iv) What I have criticised Portes and W-L for is for failing to point out Osborne’s change of course (to the extent that he did) until long after its impact had become apparent in June 2013

      Here is S W-L in October 2012
      http://mainlymacro.blogspot.co.uk/2012/10/the-uk-and-austerity-some-facts.html

      if you look at Portes’ response in th FT in 2013 he was similarly saying fiscal policy was much too tight (which we now know it wasn’t).

      Here is S W-L responding to criticism of this kind

      http://mainlymacro.blogspot.com/2014/04/what-fool-i-have-been.html

      Now there is much I agree with S W-L about (I have no choice he is the professional economist, and simply has better arguments on these points than others). So, do I think that the UK growth we have seen vindicates Osborne’s initial austerity? Not at all, that would be daft. Do we need to use fiscal policy at the zlb? Absolutely we do.

      I do however also think that Osborne did what his critics wanted him to do: eased back of fiscal austerity. I also think that S W-L over eggs the criticism, by downplaying or ignoring the other factors in play and the political constraints applicable to anyone in Osborne’s position.

      I also think that the claims that macroeconomic forecasting is just “guesswork” and so no more capable of being assessed than the ability to judge a coin toss is implausible. Indeed, in relation to what happened in 2013 it wasn’t even forecasting that was required: just one of Osborne’s critics to have pointed out at the relevant time that ‘austerity’ had been abandoned. They didn’t.

      Delete
    5. i) no need for a spread sheet - just can't fathom how you arrived at the claim that "the track record of the academics in forecasting... is very poor, worse than those employed by the evil banksters." by referring to the FT surveys.
      Go on, you can admit you made that bit up just to have a go at SWL?
      Can't fathom why you are bemused about the treasury report as the 'non-city' institutions are packed with - as the name suggests - academic economists with no city affiliations. Not all, but many.
      ii) SWL does not reduce forecasting to the extent that you have in saying it is 'guesswork'...like calling heads on a coin toss. That would be random chance...in which case you'd be right. I fear you misrepresent what he said, which was different... "little better than intelligent guesswork" so not nearly as bad as the toss of a coin.
      iii) They are not attacking straw men - they were attacking the government for claiming that the economy would continue to expand despite austerity - they appeared to be believers in expansionary fiscal policy publicly. (Whether privately, who knows)
      As for the rest...some common ground! I am curious to know when it was evident of a switch to plan B- if it was only evident a long time after, then no-one can be criticised for continuing to predict lower than expected growth as they would think Plan A was still in operation. If evident from when the switch was made, then why not point it out? I struggle to believe guys like Portes and SWL would not make a point of it if they were aware of it sooner.

      Delete
    6. Non-City does not equal academic.

      There was a typo, I meant 'Expansionary Fiscal Contraction'.

      Delete
    7. "Non-City does not equal academic". no, but many are, and its the closest I could find to drawing a comparison between city and academics - which is a lot closer than making up an erroneous claim with imaginary data in order to then accuse someone of cherry-picking data when they did no such thing.
      I don't think critics of Osborne generally, and SWL in particular have been over-critical at all when you fully consider all of the costs and disturbingly the ultimate aims of his policies, not to mention the deception. Aside from the well detailed enormous economic costs (3 years stagnant economy and lost output/jobs etc), there are the social costs that result from cuts which are disproportionately impacting on the poorest sections of society: particularly cuts impacting on the elderly - their care/well being and also the disabled. Those that can least afford (and least deserve given the roots/perpetrators of the economic crisis) the cuts imposed with the slow dismantling of the welfare state. If there are to be cuts, should he not spread them around a little more evenly and think about getting those who caused the crisis to stump up a little more?

      Delete
  16. Magnus Carlsen12 August 2014 12:48

    You need to learn how to tell the difference between unconditional forecasting, and asking what will happen if policy X is implemented. This is not having your cake and eating it.

    ReplyDelete
  17. I thought the standard practice was to link to articles that support your point, not ridicule it. But the Carola Binder link absolutely destroys your point about the usefulness of story telling.

    ReplyDelete
    Replies
    1. Because the Binder post demonstrates that macroeconomic stories are totally underdetermined by macroeconomic theory. And that, of course, is why macroeconomic theory has been wielded to justify radically upward redistributionist policies in the name of Reagan and Thatcher and Friedman.

      Delete
  18. Spot on Simon. The general purpose of all forecasts is for storytelling conditioned on certain assumptions. Unexpected (stochastic) events cause errors, as do errors in assumptions and the models used. This does not mean that forecasting is pointless, but it is imperative that forecasters try to learn lessons from their errors.

    One recent example is the OECD: http://www.oecd.org/eco/outlook/OECD-Forecast-post-mortem-policy-note.pdf

    ReplyDelete
  19. Consider:

    1. Economic theory provides the language and the structure used to tell economic stories.
    2. Economic stories can, and frequently are, normative. They claim to describe the world and how the world should be in order to be "more efficient" and "make a bigger pie."
    3. Normative economic stories are backed by credentialed economists.
    4. Normative economic stories told by credentialed economists are successful in shaping policy.
    5. Economic theory provides little guidance and no constraint on economic stories. It provides a vocabulary of actors, causes, and effects that can be arranged in a variety of internally consistent ways.
    6. Everyone who obtains credentials is deeply invested in the status quo of society and economics.
    7. 99% of the credentialed are the sons and daughters of the rich and powerful.
    8. In politics, the disenfranchised are, by definition, at a disadvantage in the competition to shape politics.
    Conclusion
    The destruction of Reagan, Thatcher, the boys from Chicago in South America, austerity, and the rest, are the inevitable consequence of academic economists paradigm of abstract theory and utter disdain for the scientific method which requires data first, theory second.

    ReplyDelete
  20. Does forecast accuracy have any effect on the career success of a macroeconomist?

    ReplyDelete