Winner of the New Statesman SPERI Prize in Political Economy 2016


Showing posts with label REF. Show all posts
Showing posts with label REF. Show all posts

Monday, 7 March 2016

The 'strong case' critically examined

Perhaps it was too unconventional setting out an argument (against independent central banks, ICBs) that I did not agree with, even though I made it abundantly clear that was what I was doing. It was too much for one blogger, who reacted by deciding that I did agree with the argument, and sent a series of tweets that are best forgotten. But my reason for doing it was also clear enough from the final paragraph. The problem it addresses is real enough, and the problem appears to be linked to the creation of ICBs.


The deficit obsession that governments have shown since 2010 has helped produce a recovery that has been far too slow, even in the US. It would be nice if we could treat that obsession as some kind of aberration, never to be repeated, but unfortunately that looks way too optimistic. The Zero Lower Bound (ZLB) raises an acute problem for what I call the consensus assignment (leaving macroeconomic stabilisation to an independent, inflation targeting central bank), but add in austerity and you get major macroeconomic costs. ICBs appear to rule out the one policy (money financed fiscal expansion) that could combat both the ZLB and deficit obsession. I wanted to put that point as strongly as I could. Miles Kimball does something similar here, although without the fiscal policy perspective


Of course many macroeconomists do see the problem, but the solutions they propose are often just workarounds. Things like Quantitative Easing, or NGDP targets, or a higher inflation target. [1] None completely remove the basic difficulty created by the ZLB. (One proposal that does is negative interest rates coupled with eliminating paper money, which I will come back to.) As a result, these workarounds mean that in response to a sharp enough recession, we would still regret no longer having the possibility of undertaking a money financed fiscal stimulus.


I also think there is a grain of truth in the argument that ICBs created an environment where deficit obsession became easier. Take the UK for example. In the 2000s the organisation that came to dominate budget analysis was the IFS. They are excellent on both the microeconomics of particular budget measures and their costing. Before the Great Recession that meant that all the IFS needed was a macro forecast (of which there are many) and they had all they required to provide excellent budget analysis. The IFS did not have strong macro policy expertise, and sometimes this shows, but as long as the consensus assignment worked that did not matter.


One of the those working at the IFS during this Labour government period was Rupert Harrison. In 2006 he became chief of staff to George Osborne. He helped introduce, perhaps reflecting his IFS experience, two important and positive policy innovations: setting up the OBR (with some minor assistance from a certain UK academic), and a form of fiscal rule (a five year deficit target) which allowed debt to be a shock absorber. But he also appeared to bring the received wisdom on the consensus assignment untroubled by the ZLB, which meant that Osborne could give a speech in 2009 outlining the macroeconomic basis of his strategy in which the ZLB was not mentioned.


This is an example of a more general point, which Robert in comments reminded me I could have made to strengthen the strong argument still further. With ICBs, macroeconomic expertise can move from finance ministries to central banks, leaving finance ministries unprepared for what they may need to do in a major recession.


But this grain of truth runs up against a real difficulty, which is the major flaw in the ‘strong argument’ I set out in the earlier post. To see the flaw ask the following question: in the absence of ICBs, would our deficit obsessed governments actually have undertaken a money financed fiscal stimulus? To answer that you have to ask why they are deficit obsessed. If it is out of ignorance (my Swabian syndrome), then another piece of macro nonsense that ranks alongside deficit obsession is the evil of printing money in any circumstances. I suspect a patient suffering Swabian syndrome would also be subject to this fallacy. If the reason is strategic (the desire for a smaller state) the answer is obviously no. We would simply be told it could not be done because it would open the inflation floodgates.


Following my grain of truth idea you might counter that without ICBs the knowledge within or outside government that these excuses were without foundation would be greater, and so governments could not get away with them so easily. But you would still have plenty of economists from the financial sector telling you that not only did you need to reduce debt rapidly to appease the markets, but also that any government printing money would scare the markets even more. Indeed, would governments alone have had the courage to undertake the scale of QE that we have seen ICBs undertake?


As for the argument that macroeconomic expertise gets concentrated in central banks, surely the answer here is to allow that expertise into the public domain by making central banks more open, and to directly combat the forces that make some central bank leaders routinely argue for austerity when they can no longer effectively combat deflation.
The basic flaw with my strong argument against ICBs is that the ultimate problem (in terms of not ending recessions quickly) lies with governments. There would be no problem if governments could only wait until the recession was over (and interest rates were safely above the ZLB) before tackling their deficit, but the recession was not over in 2010. Given this failure by governments, it seems odd to then suggest that the solution to this problem is to give governments back some of the power they have lost. Or to put the same point another way, imagine the Republican Congress in charge of US monetary policy.


But if abolishing ICBs is not the answer to the very real problem I set out, does that mean we have to be satisfied with the workarounds? One possibility that a few economists like Miles Kimball have argued for is to effectively abolish paper money as we know it, so central banks can set negative interest rates. Another possibility is that the government (in its saner moments) gives ICBs the power to undertake helicopter money. Both are complete solutions to the ZLB problem rather than workarounds. Both can be accused of endangering the value of money. But note also that both proposals gain strength from the existence of ICBs: governments are highly unlikely to ever have the courage to set negative rates, and ICBs stop the flight times of helicopters being linked to elections.
      
These are big (important and complex) issues. There should be no taboos that mean certain issues cannot be raised in polite company. I still think blog posts are the best medium we have to discuss these issues, hopefully free from distractions like partisan politics.  
   

[1] Please do not misunderstand what I mean by workarounds. The workaround may be still be useful in its own right (I have argued that monetary policy should be guided by the level of NGDP), but it does not completely remove the problem of the ZLB.
 

Tuesday, 23 December 2014

How to reference the REF

Or how to go beyond a league table position in evaluating a UK university department

Every five years or so all UK university departments get their research assessed in one gigantic peer review exercise, which is now called the REF. Each discipline is assessed separately, and scores can be used to compile a league table. The exercise has direct financial implications: the better the research, the more money universities get from the government. But if you know what academics are like, you will not be surprised to learn that those in the UK obsess about this exercise and its results to a far greater extent than the money involved would justify. The results of the latest exercise have just been published, and turned into league tables by Times Higher Education (THE) here

You could say that the REF now provides the same sort of incentive system for UK universities as profit does for a firm. In some cases academics whose research is below their departmental average are put under pressure to leave by one means or another, and most academics feel acutely the pressure to improve on how their own output will be assessed by this exercise. In contrast, poor performance in teaching or administration is not nearly such a serious issue.

Many academics complain bitterly about the indignity of all this. An alternative system would be one where getting tenure was the last performance hurdle an academic had to pass, and from then on they were free to do what they liked. Research money could all be allocated on a project by project basis. I personally doubt that would be a better system from society’s point of view, and I do find it annoying how academics can complain so much about pressures that are taken for granted elsewhere.

It would be a mistake, however, to think that the position in some REF league table told you all you needed to know to evaluate the quality of research in a department. The REF releases a wealth of data, and going beyond the headline number (usually the GPA score) can be informative. In the latest exercise departments were evaluated under three headings: outputs, environment and impact. Details about what is involved for each category can be found here.

Outputs, which has the highest weight in the total (65%), looks at the quality of the four best recent publications of each submitted member of staff. The key word to note here is ‘submitted’. A department/university can choose not to submit all its staff to the REF, and by not submitting staff that a department/university considers are well below average it can raise its GPA score (if it gets its assessments right). So to the extent that staff are not submitted, the GPA will overestimate the average quality of the research done in that department. As I said, league tables normally just look at the GPA score [1].

To some it may seem strange that this is allowed, but there are arguments to justify it. Departments do pay a significant financial penalty for leaving staff out - they only get money for submitted staff. To get a guide to the total amount of quality adjusted research done in a department, simply multiply the GPA score by the number of people submitted (called ‘power’ by THE).

The decision about whether not to submit a member of staff is an agonising one [2] that involves many difficult trade-offs. To the individual not being submitted it is a nasty slap in the face. For the department, the perceived benefits in getting a higher position in GPA based league tables may outweigh the financial cost of not submitting staff members. Decisions on this front do vary from university to university, and from department to department: in economics, compare the third and fourth columns of the table below.

Although it only counts for 15% of the total GPA score, the ‘environment’ heading may be of particular interest to potential PhD students. It is based on a number of different criteria, including the number of PhDs, the support provided for research, and research income from outside grants. Only three economics departments had all elements of environment judged to be of the highest (4*) quality this time: UCL, LSE and Oxford.

Impact is a new category, accounting for 20% of the total. It is based on case studies where research has engaged with public, private and third sector organisations, or directly with the public. For example, one of Oxford’s case studies for economics was my own work on fiscal councils. A quick look at the results suggests that this new element has had a significant influence on the overall results. In economics, for example, the only department where all the submitted case studies were judged to be of the highest quality was Bristol. So while Bristol only came 12th= on published outputs, a strong impact and environment score lifted them to 6th in the overall ranking.

As with any evaluation system, there are difficult judgements to make on the details, and these can lead to possibilities to ‘play the system’. Chris Bertram focuses on one particular issue at Crooked Timber. Each iteration of the assessment exercise attempts to change the details of the rules to avoid this, only to allow some new possibility to exploit the system. Partly as a result, after each exercise many academics feel that there must be a better and less time consuming way to judge the quality of research produced by individual academics or departments, but perhaps the fact that we keep returning to the same basic procedure suggests otherwise.

REF 2014 results: economics and econometrics
University
GPA
Score
No. of staff
submitted
Eligible
staff
Power
% 4*
Outputs
% 4*
Environ.
% 4*
Impact
UCL
3.78
37
45
139
70
100
92
LSE
3.55
51
56
182
56
100
87
Oxford
3.44
84
97
289
43
100
64
Cambridge
3.42
27
38
92
55
13
50
Warwick
3.41
42
52
142
43
38
60
Bristol
3.32
19
25
62
22
63
100
Essex
3.25
33
40
108
29
63
20
Edinburgh
3.14
18
28
55
31
50
13
Royal Holloway
3.11
14
23
45
35
0
60
Nottingham
3.05
35
46
107
20
13
18
UEA
3.04
14
22
43
20
0
20
Surrey
3.01
21
25
62
27
13
0
Queen Mary
2.98
24
31
73
20
13
13
York
2.93
28
46
82
14
0
40
St. Andrews
2.92
21
31
60
24
0
0
Manchester
2.89
33
45
96
11
13
40
Glasgow
2.86
24
30
68
18
0
0
Sussex
2.84
17
24
49
15
0
37
Exeter
2.78
25
31
68
13
25
13
Birmingham
2.78
24
27
67
8
0
27
Southampton
2.70
22
28
59
22
0
10
Birkbeck
2.60
25
32
65
10
0
0
Leicester
2.59
22
29
58
19
0
0
Sheffield
2.58
15
26
38
8
0
40
Aberdeen
2.48
19
26
48
5
0
0
City
2.44
14
26
33
17
0
20
Kent
2.32
22
26
51
3
0
13
Brunel
2.20
26
29
58
2
0
0
Note that many economics departments are assessed under Business and Management, and are not included here. Sources: columns 2,3 and 5: Times Higher Education, column 4: HESA, columns 6-8, REF.



[1] THE publishes an alternative university wide ranking (aggregated across departments) that multiplies the GPA by the proportion of staff submitted, but that implicitly gives the research of non-submitted staff a score of zero, which is likely to be too extreme. It is better to simply note either the power score, or the proportion of staff submitted. Approximate information on the number of staff eligible for submission by department can be found here.

[2] This is based on my own experience at my previous university, where I was research director for the school. Thankfully I have played no role in these decisions at Oxford!