Wednesday, 1 August 2018

How BBC balance and bad think tanks discourage evidence based policy


The Knowledge Transmission Mechanism (KTM) is how knowledge produced by academics and other researchers is translated into public policy. Evidence based policy is the result of this mechanism working. The media is, in theory, an important conduit for the KTM: media publicises research, policy maker sees/hears/reads media and gets their civil servants to investigate research. Or media communicates policy consensus on issue, and politician is questioned by the media on why they are not following this consensus.

The rigid application of political balance in the broadcast media is in danger of negating the KTM, and therefore evidence based policy. The moment an issue (call it issue X) is deemed ‘political’ by the media, balance dictates that any view expressed on issue X is an opinion rather than knowledge. As a result, when the media want to talk to non-politicians (‘experts’) about issue X, the imperative of balance remains.

Now suppose that in the knowledge world there is in fact a consensus on issue X. That would be a problem for balance broadcasting, because it would be difficult to get an expert to argue against the consensus. The BBC overcame this problem valiantly during Brexit, using Patrick Minford (who is not known as a trade economist) time and again to balance the IMF, the OECD, more than 90% of academic opinion etc. But another way of solving this problem is to use certain think tanks.

There are two types of think tank. The good kind can be a vital part of the KTM. There is often a genuine need for think tanks to help translate academic research into policy. Sometimes these think tanks will be very like universities (like the IFS for example). Other times they will be think tanks that have a broad left or right orientation. These think tanks are an important part of the KTM, because they can establish what the academic consensus is, translate academic ideas into practical policy, and match policy problems to evidence based solutions. The IPPR is an obvious example of this type of think tank. They are part of evidence based policy making.

The bad kind are rather different. These produce ‘research’ that conforms to a particular line or ideology, rather than conforming to evidence or existing academic knowledge. Sometimes these think tanks can even become policy entrepreneurs, selling policies to politicians. This is often called policy based evidence making. It would be nice to be able to distinguish between good and bad think tanks in an easy way. The good type seeks to foster the KTM, and ensure policy is evidence based, and the bad type seek to negate the KTM by producing evidence or policies that fit preconceived ideas or the policymaker’s ideology.

I would argue that transparency about funding sources provides a strong indicator of which type a think tank is. Why is this? One obvious reason why you would not want, say, company Y listed as a funder is if you subsequently produced a report that was directly in the interests of Y. A clear example is the IEA’s arguments against plain packaging of cigarettes, and its funding from tobacco companies. To be clear I am not suggesting that the IEA (who have been in the news recently) is insincere about the arguments it makes, but if funding was transparent it would be very easy to suggest their arguments on plain packaging were contrived by just pointing out its funding sources. So that is an obvious reason to keep funding secret. In contrast, an IFS type think tank has no reason to hide its funders, because it is not in the business of producing policies that meet the specific wishes of these funders. [1]

It has been suggested to me that IEA type think tanks need to keep their funding secret because some personal donations would get individuals into trouble with their employers if they became public. It would be interesting to know from some IFS type think tanks how much funding they lose for this reason. Perhaps the answer is not much, and that in any case the principle of transparency is more important than a few extra donations.

Another good indicator of a bad think tank is their relationship to academia. I have told the story about how the IEA under Philip Booth tried to cultivate the idea that the 364 academics who famously objected to the Thatcher government's 1981 budget were embarrassingly wrong, when in fact they were proved right. The IEA’s media and political connections are sufficiently strong that even good BBC economics journalists were taken in by their line. To the extent that it emboldened Osborne to ignore the majority of academic economists over austerity it was a dangerous myth to cultivate.

In the case of global warming the BBC has been forced (I don’t think that is an inappropriate word, as it often breaches the guidance) to treat man made climate change as a fact rather than an opinion that always has to be balanced. That is not going to happen for some time over any economic issue, however strong the academic consensus (like Brexit). This is partly because the pressure from academia is much less, and partly there is still a prejudice against social science (as if evidence based policy making cannot occur for economic or social policy!). But the BBC does need to explain their attitude to the use of think tanks. Why do they use think tanks that do not declare their funding sources, and when they do why is this information not passed on to its audience?

[1] To understand why arguments like the IFS gets money from the government or the EU and therefore it is biased do not work, see here.


10 comments:

  1. I used to edit a small trade publication. It was distributed f.o.c., so our revenue depended entirely on advertising; we'd have a few clients who you could rely on to take space in every issue and a lot more who would come in three or four times a year. So you'd rotate through them; if Fred had booked last month you'd leave him alone for a while, if George hadn't been in touch for six months you'd give him a call.

    But the magazine didn't consist entirely of advertising - and my job was to fill the remaining space with news items and features. From the outset I was under huge pressure (direct and indirect, i.e. budgetary) to organise issues around advertising, to tailor features to make major advertisers look good, to take features supplied by companies' PR departments, to disguise advertorial as genuine features, and so on. Some requests I resisted, others - perhaps more than I'd like to think - I went along with; it's how that world works.

    I tend to see the IEA in a similar light; not as a research institute which happens to have commercial backers, but as a commercial publishing operation which happens to employ researchers. I imagine that they have a roster of funders (just as we had a roster of advertisers) and rotate through them, producing research to suit (just as we produced features to suit our advertisers). Transparency should be the least we can expect, but in this case I suspect it would kill the goose that lays the golden eggs.

    ReplyDelete
  2. For those interested in knowledge and its transmission, this raises a large red warning flag... "To be clear I am not suggesting that the IEA (who have been in the news recently) is insincere about the arguments it makes, but if funding was transparent it would be very easy to suggest their arguments on plain packaging were contrived by just pointing out its funding sources." To my mind, rather than 'very easy' this should read 'all too easy' because instead of actually examining the evidence base used and the quality of analysis, those for whom the conclusion is unpalatable need 'just' point out the funding source to dismiss the conclusion regardless of any evidence proffered or the analysis of it. So, ironically, that *is* an obvious reason to keep funding secret. I'm no apologist for the IEA and indeed I wrote a highly-critical blog on their response to the Uber Employment Tribunal decision https://medium.com/@StrongerInNos/is-everything-alright-hun-dbb907cb6e60 Of course it's reasonable to ask 'cui bono' and it might even be reasonable to think it necessary, but it's certainly not sufficient to come to any judgement on the work itself.

    ReplyDelete
  3. I think you view is too purist.?I'm sure you are right in some respects but you implicitly assume that "good" think tanks are not politically biased whereas this is almost certainly no true. A segregation of think tanks as you suggest may improve things but I doubt if it is a total solution to the issue of bias.

    A further point is that you seem to implicitly assume that the target audience is unable to sift through these issues themselves. It seems odd to assume that policy makers are not able to sort the wheat from the chaff here as you have done in your piece.

    The other problem with these sorts of rule is: where does it end? before you know it there is a host of rules to fulfil and this might be tantamount to a form of censorship. Are we not better off with very few rules with a maximum of freedom and apply caveat emptor?

    ReplyDelete
  4. I think the media are better informed than you think. This is an arrogant post. Before the financial crisis the media did look up to economists. So did New Labour. In fact until 2008 economists got pretty much what they wanted. We have seen the result. Now there is a growing realisation of what even some economists(particularly those who are genuinely multidisciplinary in their approaches) openly admit: economists don't know much. Until then journalists are wise to be cautious of what economists say and field widely for opinion, especially when economists say things like "there is a consensus in the economist profession..."

    ReplyDelete
  5. "The BBC overcame this problem valiantly during Brexit, using Patrick Minford (who is not known as a trade economist) time and again to balance the IMF, the OECD, more than 90% of academic opinion etc."

    Both sides made errors on forecasting the short term consequences of Brexit. But Minford's forecasts were closer to what actually happened. If you believe in evidence based social science, as you state, then you should accept this fact. I don't see how the BBC can be criticised on this issue.

    The Treasury short term forecasts for Brexit have now been exposed as a poor joke. Their excuses for this are laughable. They had their 'Michael Fish' moment but unlike the Met Office they haven't gone back to basics and started again.

    ReplyDelete
    Replies
    1. This is a Reuters report on the post-Brexit forecast from the Treasury:

      The top official at Britain’s [Treasury, Tom Scholar,] said his department’s forecasts of a big hit to the economy from Brexit, made shortly before the June 2016 EU membership referendum, were no longer applicable.

      Brexit supporters have long criticised the projections as part of a “Project Fear” they say was led by former prime minister David Cameron and his then-Chancellor George Osborne.

      The forecasts said that within 15 years Britain’s economy could be between 3.4 and 9.5 percent smaller if it left the EU than if it had stayed in.

      Tom Scholar told lawmakers on Wednesday the forecasts were based on an assumption that Britain would immediately start the process of leaving the European Union, and they did not include any stimulus measures for the economy.

      In fact, Britain took nine months to launch the Brexit process and the Bank of England pumped in stimulus shortly after the vote. Last November, the government announced measures which were also aimed at helping the economy.

      “The third (factor) I would say is that the global economy has recovered much more strongly ...than we expected at the time,” Scholar said.

      Britain’s economy has withstood the Brexit vote shock better than most private economists expected, but it has lagged behind growth rates achieved in other advanced economies.

      Speaking to the Treasury Committee in the [House of Commons], Scholar also said the forecasts were based on three potential scenarios for Britain’s future relationship outside the EU, but not the one that British Prime Minister Theresa May wants: a tailor-made trade deal with the bloc.

      “I don’t think the pre-referendum analysis is useful in the current debate about our attempts to secure a deep and special partnership because that’s a different thing to any of the three scenarios that were illustrated in that paper,” he said.

      Delete
    2. I was aware of Tom Scholar's remarks, which is why I wrote that the Treasury's excuses for their poor predictions are laughable.

      Delete
    3. So the Treasury admits that it made the wrong assumptions when forecasting the short term effects of Brexit. Clearly then the BBC when reporting on long term forecasts for Brexit should act under the assumption that the forecasts may also be based upon spurious assumptions and that the actual outcome may be vastly different from those forecasts. It would be failing in its public duty to do otherwise.

      Delete
  6. Only today the Guardian published an article suggesting that even in respect of climate change the BBC still thinks it necessary to "balance" any discussion of the subject with a climate change denier like Nigel Lawson. So I don't think its attitude to so-called balance is limited to economics. Perhap because of its fear of politicians the BBC has relegated its duty to inform to niche programmes like "More or Less" and "Inside Science

    ReplyDelete
  7. As an Aussie, its very interesting that the IEA has taken tobacco compnany money to argue against plain packaging. Because its Antipodean counterpart did exactly the same (remembering that Australia was the first country to do plain packaging legislation). I think you're being very generous in saying they were sincere in their arguments.

    Your point about funding transparency is very clear in Australia. There are two signifcant thinktanks on the right here - the Institute for Public Affairs and the Centre for Independent Studies. The IPA is notorious for talking its funder's book, the CIS has a reputation for being ideological but precisely on that account having integrity (think pre-Koch Cato in the US). The CIS gets some traction with academics and policy wonks but it is the IPA that is really listened to by tory politicians.

    Of course, that is precisely because they know there are very rich and powerful people behind the IPA, not true believers in markets.

    ReplyDelete

Unfortunately because of spam with embedded links (which then flag up warnings about the whole site on some browsers), I have to personally moderate all comments. As a result, your comment may not appear for some time. In addition, I cannot publish comments with links to websites because it takes too much time to check whether these sites are legitimate.