Delegating to the wrong people: the strange case of interviewing undergraduate candidates at Oxford

This post starts off talking about something rather parochial, but it then moves to a wider point about decision making in a particular type of large organisation.

This is a strange time of year at Oxford University. Very few UK universities interview prospective undergraduates to help select who should be admitted, but both Oxford and Cambridge do. As a result, all but the most senior academics will generally spend at least two solid days interviewing something like 3 times as many undergraduate candidates as there are places. The personal and social cost of this is pretty high - quite a few research papers will not get written as a result, and maybe the odd important discovery will be delayed by years. Yet Oxford and Cambridge are also relatively unusual among UK universities in still allowing academics considerable control over the way the institution is run. Is this a paradox?

Now all this might make sense from a social point of view if interviewing meant we clearly got much better students at Oxford than we would otherwise. However all the evidence I have ever seen on interviews is that they are pretty unreliable as a selection procedure. In this particular case, there are well known biases: those from certain schools or families will be practiced at giving a good impression in such situations, whereas others will not. We (and I think I’m justified in using that collective pronoun) do our very best to get round those biases, but it is hard. We can fail to compensate, or over compensate. We all think we can judge someone by talking to them for 30 minutes, but in reality our ability to assess academic potential that way is pretty small. Indeed, even if information gleaned from interviews contains some useful information, I strongly suspect those making decisions give this information far more weight than it deserves. My suspicions receive some local support from Bhattacharya et al (2012)[1].

So, if there is no clear evidence that students are much better selected as a result of interviews, why do we continue with them? One familiar idea is that this is a form of concealed class bias, but I suspect this is not the general case. The evidence from Bhattacharya et al suggest that for PPE at Oxford, the marginal student from an independent (private) school faces significantly higher admission thresholds than those from state schools. A slightly less politically charged version of this argument is that we want to select students who are easy and interesting to teach, and as the interviews are like mini tutorials they achieve this goal. This would also explain why interviewing is largely confined to the two universities that still have a traditional tutorial system (teaching in small groups of two to four).

Even if there is some truth in this, there still seems to be a puzzle. From the self-interested point of view of the research oriented and highly competitive academic, the effort involved in interviewing seems out of proportion to any gain in getting more congenial students. I spend the equivalent of 12 full days a year teaching in tutorials, so spending 2+ days each year in a process that might make those 12 days slightly easier is just not worth it.

One of the other unusual aspects of Oxford and Cambridge is that they remain - in theory at least - self-governing. The individual colleges that make the admission decisions certainly are run by the academics.[2] So why are Oxford academics so attached to interviewing, when it just does not seem to be in their interests?

But maybe this is not the right question. Perhaps the majority of Oxbridge academics would happily give up interviewing, but for some reason this preference is not getting realised. I could at this point talk about the peculiarities of decision making at Oxford (and peculiar is the right word), but I think there may be a more general issue here.

Decision making in large organisations like universities is often run through committees, or through delegation to ‘volunteers’. Most academics, even though they may enjoy teaching students, are primarily interested in research (either intrinsically, or because it determines future career prospects). As a result, they will tend to avoid taking positions where they have to think about non-research related issues (like being on department or university teaching committees), and try and avoid administrative duties more generally. However a minority will have different priorities, and so they will take those positions. They may as a result take decisions which divert academics away from research, given their own interests and responsibilities. It may be these people who are taking decisions about interviewing, and not your more typical research focused academic.

Some of my former (non-Oxford) colleagues wrote a paper about this general issue some time ago [3], and although they did not apply it to universities, I suspect it may have been their inspiration. In many organisations this problem is dealt with by a strong central authority, but universities often do not have that, and Oxford and Cambridge certainly do not. So even if the majority of academics did want to stop interviewing, they cannot exercise this preferences, because they have delegated decision making to others who have different interests and priorities. Who knows if this is right, but maybe I have just thought up a good interview question!
    
[1] Bhattacharya, Kanaya and Stevens (2012), Are University Admissions Academically Fair? Oxford University Discussion Paper No. 608. Note (Table 2) that interview scores are very significant in deciding who is admitted, but have a weaker correlation with first year exam results compared to other pre-admission evidence.

[2] For UK universities, research performance is the key incentive, because of the UK’s research assessment exercise. So I suspect a strong, top down central authority would quickly decide that interviewing was a non productive distraction. 

[3] Bulkley, G., Myles, G.D. and Pearson, B.R. (2001), On the Membership of Decision-Making Committees, Public Choice, 106, 1-22.