Why have pundits got politics so wrong since 2015?

I have just read a paper called “Political science, punditry, and the Corbyn problem’ by Peter Allen, a Reader in comparative politics at Bath. It reflects on how most pundits, including some political scientists, got Corbyn’s initial success and then survival completely wrong. I will not attempt to summarise the paper here. It is well worth reading. I am going to take it as read that many pundits did get Corbyn completely wrong in 2015 and 2017. This has nothing to do with whether the left ascendency is a good or bad thing, but just the failure of pundits to see why it was happening.

Allen notes a kind of epistemic snobbery “‘whereby people who do not meet the above criteria of political inclusion are not seen as worthy participants or contributors in political discussions, or whereby their political opinions are devalued in some way”. It was a kind of “othering” that I felt personally when I joined Labour’s Economic Advisory Council. I was told, by people who I respect, that my academic standing would be harmed if I joined the group. It was if I had decided to give economic advice to the BNP rather than the Labour party.

Part of this represented a longstanding dislike by the centre and centre-left of the left in the UK that stems from the political battles within Labour in the 1980s. Andy Beckett tells some of the story here. There was a lot wrong with the Labour left at that time, and Labour leaders from Kinnock to Blair found they could gain a certain credibility by attacking both the left and the unions. Indeed some of those who attack the left today were part of the left back then, and now see the error of their ways. The Labour left came to be seen as generically toxic.

As Allen notes, another element in this failure to understand Corbyn was a belief in triangulation. In the world that takes triangulation as the theory rather than just a useful model with limitations, moving sharply to the left when a party of the right wins an election makes no sense. But why were the same pundits not already noting that the theory of triangulation had broken down, because the Conservative party from 2010 to 2015 had moved sharply to the right and yet had won a general election? This is what the rest of this post is about.

Allen does not mention austerity specifically, but I think misunderstanding austerity plays a large role in failing to see how far right the Conservatives were moving, and therefore Corbyn’s rise in 2015 and Labour’s gains during the 2017 campaign. If you look at what the Coalition did collectively there can be no doubt about what was going on. The hostile environment, privatisation of the NHS, demonisation of those on welfare and so on. Yet perhaps all of these things could be explained away individually if that is what you want to do: continuing Blairs policy on the NHS, responding to popular opinion on immigration and welfare. The dominant narrative, at least to begin with, was of Cameron the moderniser.

The clearest indicator of a rightward shift was austerity. It should have been clear by 2012 if not earlier that the recovery was stalling. Thatchers experiment with austerity had been brief and was quickly reversed, but Osborne was not for turning. We had for the first time since WWII a government attempting sustained austerity during a recovery phase of a recession. Perhaps too many placed their faith in City folk that told stories of imminent bond strikes, so they believed deficit reduction had to be done. But when interest rates on government debt started falling curious academic minds at least should have begun to smell a rat. Did pundits not notice that the majority of economists were against austerity? This is a genuine question rather than a rebuke, because you had to do a little research to find out they were.

Once you miss the rightward move of the Coalition government, and note that it would have been worse still but for the Liberal Democrats, then you also fail to see that Labour from 2010 to 2015 had been following a triangulation strategy and failed. Did pundits put everything down to Miliband’s unpopularity? Once you understood that Labour had moved to the right and lost, then Corbyn’s victory should have come as no surprise, as I argued here before the result.

Understanding the deep damage that the austerity policy did to the country means that it is hardly surprising that under a left leader opposed to austerity the Labour party should attract half a million members. Too many pundits talked about this in terms that applied to the Labour party in the 1970s and early 80s, but this was a danger for rather than a description of the mass movement that Labour were becoming.

There was one feature of received wisdom that seemed to be holding true, however, and that was that Labour led from the left would be defeated decisively in any general election. Poll after poll suggested this was true. I was told too many times that the left were only interested in controlling the party (how surprising) and not interested in winning elections. It was nonsense of course.

As soon as Labour's position in the polls started rising in the middle of the campaign I suggested that Corbyn’s unpopularity before the campaign told us more about the media than anything else, but I’m not sure this is accepted by most pundits. Many will blame the Tories bad campaign, but what that showed us was that May and her team were pretty bad at doing politics, which was something that should have been clear given the evidence if the media had been doing its job properly. But underestimating the role of austerity is important here too.

Austerity was, after a time if not initially, designed to shrink the UK state. And it succeeded. Attitude surveys tell us that is very unpopular, with less than 10% of the population wanting lower taxes and spending. So a party proposing the opposite, with a tax financed fiscal expansion that was at the heart of the Labour campaign, was bound to be popular on that account. Again the Labour surge was a consequence of a media that preferred talking about Labour divisions and personalities rather than policies, so Labour's policy stance came to voters as a surprise.

Thus in my view the failure to see austerity for what it really was is crucial in understanding why pundits got Corbyn so wrong. However I would be fascinated to know how some of those same pundits themselves account for this failure, and whether they see my account having some validity or not.