The BBC and Statistics

If you think this is not the most exciting blog post title, then you will get part of the problem identified in a report on that very subject. The report does not pull its punches. For example:

"We were surprised that an organisation the size of the BBC, with a high (and increasing) volume of statistics in its outputs, does not itself have an explicit expert statistical capability. If guidelines were issued advising journalists to seek statistical advice, virtually noone we talked to could complete the sentence “For advice, contact ….””

There was until recently a Head of Statistics, but that job has been abolished.

However, having statistical expertise within house is only a partial answer to these kind of problems. As the report makes clear, journalists need to have some knowledge of their own, so that they can challenge politicians in real time. If an education minister says spending has gone up in real terms, a good journalist should immediately ask whether it has also increased per pupil. Lot of this is about the appropriate normalisation of statistics, which was the central message of my own submission to this inquiry.

That submission, and the report itself, is about the detail of how to improve the BBC’s use of statistics. Three more general points also occurred to me.

The first is about economics. Much that is written in this report about statistics could also be said about economics. The current view about economics reporting in much of the media is that this is a specialist (albeit generally topical) area that needs one or more specialist reporters, to be filed under the heading of ‘business reporting’. In reality a great deal, perhaps the majority, of political discussion depends crucially on economics. Mediamacro is not the product of the occasional confused economics editor, but the lack of economics understanding among political reporters.

The second is about balance. Many have rightly complained that an obsession with balance during the Brexit campaign allowed Leave lies to go unchallenged. But moving beyond balance requires the ability to access and judge expertise: to tell the lies from the truth. If that expertise is not there, balance is the cheap and only feasible option.

Which brings me to my third, and perhaps most contentious, point. This report makes many sensible suggestions. They generally involve more resources: training, in house expertise, better research. Yet journalism is under pressure like never before, largely as a result of the medium you are using to read this post. For most that will inevitably mean less time to check statistics, consult and research. The BBC is often said to be in a privileged position in this respect, but I would turn that point on its head. We may need somebody with deep pockets, rather than the competitive market place, to sustain quality journalism.

Good journalism may be what economists call a public good. The obvious body to produce public goods is the public sector. But good journalism is also about holding the public sector and the politicians that run it to account. [1] Implementing this report requires more resources, at the same time as the government is squeezing those resources because elements within it dislike the BBC. This illustrates a contradiction that goes well beyond the BBC’s use of statistics.

[1] Good journalism also often requires holding the private sector, and in particular large corporations, to account. If that journalism is financed through advertising, we have a similar problem.

Using economic statistics in an impartial and informed way

The BBC Trust is conducting a review on this issue, and has put out a public call for evidence. I thought I might reproduce my own evidence here because it may be of wider interest. I've decided to do this prior to actually submitting it, so that readers can add in comments any examples that go with the theme of what I am saying that would bolster my case. Note that the review is specifically about the use of statistics rather than economics.

Draft submission

I would just like to make a few specific points about the use of economic and financial data. This data should be presented in a way that informs the public, rather than in a way that is meaningless to everyone except experts, or worse still in a way that fosters some political position.

Unfortunately this standard is not upheld at present. To take just one example, I have seen a well known BBC financial journalist quote, without qualification, numbers for how much UK government debt is increasing every day, in a manner that is clearly designed to suggest that this is a very serious problem. But numbers like this are meaningless on their own. For example, we could do exactly the same for nominal GDP to give a positive impression about the health of the economy. To the layperson any number would seem large, but it would be a meaningless measure of economic health, particular if all the growth was coming from inflation. Needless to say the view that rising debt in nominal terms represent an urgent problem is a political position, and not one shared by many macroeconomists.  

Mistakes like this could be avoided to a large extent by applying some normalisation. To return to the debt example, numbers for debt and deficits should routinely be given as shares of GDP. Economic journalists should also point out that the debt to GDP ratio will not rise if the deficit is positive, but (based on current numbers) only if the deficit as a share of GDP is above something over 3%. An alternative normalisation that would make such figures more meaningful would be to divide them by total household income. Figures for aggregate personal debt should always be normalised with respect to household income, because only then can we really see if rising debt is something we should be concerned about, or just the result of growing incomes

When presenting movements over time, journalists are familiar with the danger of presenting increases in nominal terms, where any growth may just reflect inflation. However in a period like the present with large scale net inward migration, it becomes increasingly important to normalise by the number of people in the economy. The most frequent case where this point is ignored is for GDP growth. Per capita growth is much more relevant for the public, because it is closer to how fast average incomes are growing. Another example is for data on employment: with large inward migration employment will routinely be at record levels. For most purposes some measure of participation is more appropriate.  

The best way on television to put data into historical context is to show a chart. This is occasionally done, but it should happen far more often. I remember watching in despair the coverage of the 2013/4 winter floods, where the Prime Minister repeatedly said that the coalition had maintained previous levels of spending, by including spending committed by the previous administration in the numbers he used. One glance at a chart would show that in reality spending had been cut back sharply in 2011 and 2012, yet I never saw the relevant data shown by the BBC, and I could not find it on the BBC’s website. To their credit, recent Newsnight coverage of the recent floods did show this data.
 

Charts together with normalisation would also put the ‘protection’ given to some departments in recent Budgets and Autumn Statements into context. Health spending is ‘protected’, but if a chart of health spending as a share of GDP is shown, this makes it clear that the planned reduction in spending as a share of GDP coupled with the reductions that have already taken place are unprecedented. Viewers who otherwise may find it difficult to understand why the NHS seems to be in current crisis even though it has been ‘protected’ will immediately understand on seeing this chart. Education spending may be ‘protected’ in real terms, but as the number of pupils is likely to grow real spending per pupil will fall.

Although charts cannot be shown on radio, journalists should have them at their disposal so they can describe their main features, or as ammunition when interviewing politicians. If a chart cannot be shown, historical averages can put figures into context. Consider recent economic growth, using the more relevant and comparable measure of GDP per head. Has growth since 2013 been a real recovery? Showing a chart of growth going back 50 years or so immediately puts such claims into perspective, but failing this merely noting that recent growth is at best close to historic averages suggest that this is no recovery in the normal macroeconomic meaning of that term. (We normally think of a recovery as GDP per head returning to some trend line, whereas recent growth has simply maintained our distance below it.)

This was one example where a failure to put data into context meant that BBC coverage of the election failed to be impartial. Here are two more examples. (A more complete list of what I have called ‘mediamacro myths’ can be found at http://mainlymacro.blogspot.co.uk/2015/04/mediamacro-myths-summing-up.html) The first is the claim, repeated endlessly by Conservatives, that they had to clear up the mess left by Labour profligacy. The claim was absolutely central to their attempt to blame the previous Labour government for austerity. This claim was echoed by the right wing press, and largely believed by much of the electorate.

While it was true that in the last few years of his Chancellorship Gordon Brown did slightly bend the rules to achieve his fiscal targets, there is no way under any conceivable definition that this could be called profligate behaviour. In no conceivable way did it put the economy at risk, or require substantial austerity to correct it. The Conservative claim was therefore false, but I do not recall it ever being challenged on the BBC. Once again simply showing a chart for the debt or deficit relative to GDP over time would have gone a long way to establishing this point.

The second relates to the necessity of reducing the deficit quickly. It was austerity over the first two years of the coalition government that help stall a nascent recovery: OBR analysis suggests GDP growth was reduced by about 1% as a result of fiscal consolidation in both those first two years (figures which I never recall seeing quoted on the BBC, still less used to challenge George Osborne) . The justification given by the government for this was that austerity was needed to restore the ‘trust of the markets’. The BBC appears to have relied on economists working in the financial sector to support this claim. Unfortunately this source is biased, both politically, and in its own self interest: playing up the unpredictability of the markets fosters their self-appointed position as ‘high priests’.

At the very least journalists should have consulted those in the academic community more about this claim. (There is, for example, a widely held view among academics that since the recession there has been an acute shortage of safe assets like government debt.) However some simple use of data would have thrown doubt on the idea that the UK was about to suffer a debt funding crisis. Interest rates on government debt were lower in 2009 and 2010 compared to 2007 or 2008, both in absolute terms and relative to US rates. If you look at rates just before and after the election, they show no impact from the election result: if rates were being driven by prospective deficit levels then you would expect them to fall immediately after the election. By failing to use these simple statistics to challenge the government’s narrative the BBC failed to be impartial.