This post starts off talking about something rather parochial, but it then moves to a wider point about decision making in a particular type of large organisation.
This is a strange time of year at Oxford University. Very few UK universities interview prospective undergraduates to help select who should be admitted, but both Oxford and Cambridge do. As a result, all but the most senior academics will generally spend at least two solid days interviewing something like 3 times as many undergraduate candidates as there are places. The personal and social cost of this is pretty high - quite a few research papers will not get written as a result, and maybe the odd important discovery will be delayed by years. Yet Oxford and Cambridge are also relatively unusual among UK universities in still allowing academics considerable control over the way the institution is run. Is this a paradox?
Now all this might make sense from a social point of view if interviewing meant we clearly got much better students at Oxford than we would otherwise. However all the evidence I have ever seen on interviews is that they are pretty unreliable as a selection procedure. In this particular case, there are well known biases: those from certain schools or families will be practiced at giving a good impression in such situations, whereas others will not. We (and I think I’m justified in using that collective pronoun) do our very best to get round those biases, but it is hard. We can fail to compensate, or over compensate. We all think we can judge someone by talking to them for 30 minutes, but in reality our ability to assess academic potential that way is pretty small. Indeed, even if information gleaned from interviews contains some useful information, I strongly suspect those making decisions give this information far more weight than it deserves. My suspicions receive some local support from Bhattacharya et al (2012)[1].
So, if there is no clear evidence that students are much better selected as a result of interviews, why do we continue with them? One familiar idea is that this is a form of concealed class bias, but I suspect this is not the general case. The evidence from Bhattacharya et al suggest that for PPE at Oxford, the marginal student from an independent (private) school faces significantly higher admission thresholds than those from state schools. A slightly less politically charged version of this argument is that we want to select students who are easy and interesting to teach, and as the interviews are like mini tutorials they achieve this goal. This would also explain why interviewing is largely confined to the two universities that still have a traditional tutorial system (teaching in small groups of two to four).
Even if there is some truth in this, there still seems to be a puzzle. From the self-interested point of view of the research oriented and highly competitive academic, the effort involved in interviewing seems out of proportion to any gain in getting more congenial students. I spend the equivalent of 12 full days a year teaching in tutorials, so spending 2+ days each year in a process that might make those 12 days slightly easier is just not worth it.
One of the other unusual aspects of Oxford and Cambridge is that they remain - in theory at least - self-governing. The individual colleges that make the admission decisions certainly are run by the academics.[2] So why are Oxford academics so attached to interviewing, when it just does not seem to be in their interests?
But maybe this is not the right question. Perhaps the majority of Oxbridge academics would happily give up interviewing, but for some reason this preference is not getting realised. I could at this point talk about the peculiarities of decision making at Oxford (and peculiar is the right word), but I think there may be a more general issue here.
Decision making in large organisations like universities is often run through committees, or through delegation to ‘volunteers’. Most academics, even though they may enjoy teaching students, are primarily interested in research (either intrinsically, or because it determines future career prospects). As a result, they will tend to avoid taking positions where they have to think about non-research related issues (like being on department or university teaching committees), and try and avoid administrative duties more generally. However a minority will have different priorities, and so they will take those positions. They may as a result take decisions which divert academics away from research, given their own interests and responsibilities. It may be these people who are taking decisions about interviewing, and not your more typical research focused academic.
Some of my former (non-Oxford) colleagues wrote a paper about this general issue some time ago [3], and although they did not apply it to universities, I suspect it may have been their inspiration. In many organisations this problem is dealt with by a strong central authority, but universities often do not have that, and Oxford and Cambridge certainly do not. So even if the majority of academics did want to stop interviewing, they cannot exercise this preferences, because they have delegated decision making to others who have different interests and priorities. Who knows if this is right, but maybe I have just thought up a good interview question!
[1] Bhattacharya, Kanaya and Stevens (2012), Are University Admissions Academically Fair? Oxford University Discussion Paper No. 608. Note (Table 2) that interview scores are very significant in deciding who is admitted, but have a weaker correlation with first year exam results compared to other pre-admission evidence.
[2] For UK universities, research performance is the key incentive, because of the UK’s research assessment exercise. So I suspect a strong, top down central authority would quickly decide that interviewing was a non productive distraction.
[3] Bulkley, G., Myles, G.D. and Pearson, B.R. (2001), On the Membership of Decision-Making Committees, Public Choice, 106, 1-22.
Writing as someone who was responsible for the Oxbridge admissions programme at a mixed independent school from 1974 until 2008, one thing I think I can say with confidence is that admissions decisions at both universities have become far more consistent over that 34-year span, in that there have been far fewer shocks (i.e. outstanding students rejected or no-hopers admitted). Since the emphasis on aptitude tests (at Oxford), GCSE results, AS marks (at Cambridge), and other relatively objective measures, and the (attempted) standardisation of interview questions and techniques, decision-making has become much less erratic. I support the view that the interview process clings on because the tutorial system still exists, and maybe one intention is to weed out pupils who (through shyness or lack of interest and commitment) cannot verbally contribute much to an intensive one-hour session. Also, it may be a way of preselecting out pupils who would otherwise apply just for the hell of it - easily the most daunting aspect for candidates is the interview. Only those who are determined to get in expose themselves to it. If you abolish the interview process, you (or delegated assistants) may actually end up committing far more time to the process!
ReplyDeleteIf you omit the interview then there is more scope for the personal statement and reference to exaggerate abilities and interest without fear of being challenged. That would leave the only objective source of information the exam and aptitude test grades but there are some random elements and difficulties in comparison in grades as well.
DeleteSo the interview could perform a function as an audit of other data, thereby making it more reliable, and possibly provide a small amount of additional information. Maybe this is worth the cost, maybe not. I agree that the pure interview score should not be highly weighted versus exam results. I would guess that this is now the view of most Oxbridge college admissions officers, but in practice when faced with a large number of borderline candidates with similar exam results, the interview probably does assume some importance as a tiebreaker.
I find it a little strange that Cambridge publishes a lot of research on the predictive power of AS level UMS scores but very little on interviews despite having an apparently standardised interview scoring system. This suggests to me they might have looked at interview reliability but haven't yet found anything they want to publicise.
At risk of focusing too much attention on the "parochial" element of your post, I thought I would pass along a recent post by Arnold Kling.
ReplyDeletehttp://www.arnoldkling.com/blog/college-admissions-merit-and-ethnicity/
One thing you don't mention that seems likely to have some bearing is the importance of the interviews in maintaining the Oxbridge image among kids (and teachers/parents). The institutions have a vested interest in maintaining their (partly justified, but - for some subjects at least - out of proportion) aura of being a class apart from other unis. The extra hoops you have to jump through in applying is a big part of that.
ReplyDeleteI always thought that was a major factor. It's part signalling ("we're so important that people have to come see us") and part indoctrination (face-to-face conversations: very important for mind control).
DeleteCertainly the overall effect on me was that other universities looked second-rate by comparison. It built a personal emotional connection that e.g. Bath Uni couldn't emulate. Even though I actually preferred the course at Bath, since most of my interaction with them was via forms, they came across as very impersonal and bureaucratic.
If Cambridge wants to keep its image among prospective students, I strongly recommend that interviewing continue.
-- Cambridge grad
This debate may be in danger of overlooking the perspective of the young candidates who take part in the Oxford interview process. As a former PPE student at St Catz, I am amazed at how protracted and opaque the interview process has become. My son spent Tue - Fri last week in Oxford; four interviews spread over four days at three colleges. He experienced long periods of excrutiating boredom interspersed with frantic activity. Like RAF fighter pilots during the Battle of Britain, candidates pretended to relax while waiting for a sudden summons to dash across the city for 30 minutes of intense questioning at another college which could shape the course of their life. Nobody bothered to explain what was going on. It became as much a test of fortitude and endurance as academic potential. There may well be method in this madness, but I gather many candidates found it confusing, exhausting and dispiriting. If anything deters state school students from applying to Oxford the gruelling interview merry-go-round rising towards the top of the list.
ReplyDeleteI very much agree that we should think about the experience of those being interviewed. Those that are successful will be fine, but the majority will not be successful. The problem with the interview process is that rejection can seem much more personal, particularly given the intense nature of the experience that you describe. Over the years I have met a number of highly successful people in academia or government who still speak with bitterness about being rejected by Oxbridge, and I'm sure the interview has a lot to do with that.
DeleteIt is not only Oxbridge that undertakes interviews to select candidates. Durham University always has, or at least did thirty years ago when I was applying to universities. Now other universities are following suit, particularly in departments which attract large numbers of students with A grades at GCSE, AS and A-level. So this certainly applies to many of the top science, engineering and computer science departments in the country. This trend also helps to explain why the interview process persists. It is there because universities increasingly need a tool to distinguish between the ever-growing number of students with perfect grades.
ReplyDeleteIn the past both Oxford and Cambridge had entrance exams to perform that function (as well as interviews). The entrance exams were scrapped in the late 1980s in part because they were seen as too inaccessible to state school students and also there was a growing belief at the time that they gave public school applicants an advantage by virtue of the greater amount of preparation time and resources at their disposal. I'm not sure I ever bought that argument though, particularly as the entrance exam route always seemed more objective to me. So I suppose it is somewhat ironic that it is now the interview process that is under scrutiny and criticism for the same reasons.
When I applied to Cambridge from my state comprehensive I had to sit the entrance exam without any A-level further maths (my school didn't teach it). I only had 8 weeks to prepare post-A-level (including covering the entire further maths syllabus). I still got the top grade, won an exhibition prize and was given a place without being interviewed. I'm still not convinced that the interview-only route (which was favoured at the time for pre-A-level applicants) would have resulted in the same outcome.
For me objectivity is the key. An exam is objective provided both the person sitting the exam and the person marking it both agree what the right answer is to each question. Unfortunately that cannot always be said of aptitude, IQ or personality tests. It is rarely true with interviews. So the sooner Oxbridge gets back to the entrance exam the better.
A more interesting question is why do so many employers persist with interviews for technical jobs where knowledge and intelligence surely are the primary concern. I can understand using interviews to fill a job in sales, marketing or PR. For those posts image, personality and appearance matter. But why should universities use them to fill research posts in science, or tenured academic posts? If you want the best academics at your university then the CV doesn't lie. So here is a question for you Simon. How fair was the selection process for you in your current post, or that for all your colleagues?