Hard evidence: Is Oxford biased against state students?

It's autumn, and a new batch of students are starting university. Some are walking through the ancient gates of an Oxbridge college. Others are joining a redbrick university like Manchester or Bristol. A few may even be arriving in Warwick as I did (only to realise the University of Warwick is actually in Coventry).

At this time of the year there is an oft-quoted debate, and Oxford and Cambridge Universities tend to be at the heart of it. Do top universities favour privately educated students or state school applicants?

Nobody disputes that private school pupils are more likely to apply to Oxford or Cambridge than state school pupils. Many private schools are also prepped to face the Oxbridge admission process. This is a problem, but recently the Guardian suggested an even more worrying trend.

Looking at data on Oxford University admissions from 2010 to 2012, they compared the fates of independent and state school applicants who went on to achieve the highest possible school marks: three or more A* grades at A Level. Their results were as follows:

• Independent school applicants with at least three A*s: 2,175 applied; 1,098 accepted.
• State school applicants with at least three A*s: 3,196 applied; 1,474 accepted.

Over 50% of independent school applicants got in, but only 46% of state school hopefuls did. The difference may not seem large at first glance, but it is actually substantial. If we put both groups of applicants' names in a hat and randomly picked out candidates until the 2,572 places were filled, there is a less than 0.1% chance we'd pick so many independent school students.

If we didn't like Oxford or private schools, we might just finish our analysis there. After all, there seems to be convincing evidence of a bias, and numbers can't be wrong. Or can they?

The disappearing bias

During the autumn of 1973, the University of Berkeley gave postgraduate places to 44% of male applicants but only 35% of female ones. It was a huge discrepancy, and the university was soon taken to court for discriminating against women.

Berkeley gathered a committee to examine the data. The team included Peter Bickel, a statistician at the university. Along with two colleagues, Bickel started by tallying up admissions for each department separately. Perhaps only a few faculties were to blame for the gap?

Once the team had excluded departments with fairly even acceptance rates, or ones that no women applied to, the six biggest departments (labelled A to F) remained:

The results were startling. Comparing male/female acceptance rates for each individual department, the committee couldn't find any substantial bias in favour of men. If anything, there was a slight bias towards women. So what was going on?

It was clear from the data that some departments (such as A) were easier to get into than others (such as F). When the committee looked at which of these courses men and women tended to apply to, there was a big difference in preferences:

Most men had applied for the less competitive subjects, whereas a lot of the female applicants had to fight it out for places on the popular courses. This explained why more men had got into Berkeley: they'd applied for courses that it was easier to get a place on. Although no individual subject favoured men over women, when all the subjects were bundled together it therefore looked like there was a bias towards male applicants.

The contradiction is known as "Simpson's paradox", after statistician Edward Simpson who first outlined the problem in 1951. It can happen whenever we combine sets of data – like departmental admission rates – into a single statistic, when each piece of information should really be analysed individually.

Paradoxes and private schools

Could the recently alleged private school advantage in Oxford admissions be a case of Simpson's paradox? To investigate, The Conversation obtained department-level admissions data from Oxford University through a Freedom of Information request. The criteria was the same as in The Guardian article: the 2010-12 success rates of private and state school applicants who had achieved three or more A* grades at A Level.

First, let's look at the overall acceptance rates:

• Independent school applicants with at least three A*s: 2579 applied; 1208 accepted.
• State school applicants with at least three A*s: 3247 applied; 1460 accepted.

These figures don't quite match the ones given in The Guardian article. (The Guardian responded that it may be because of different dataset requests). But assuming the figures Oxford gave The Conversation are correct, 45.0% of state school applicants who went on to get 3 A*s were given places, and 46.8% of private school pupils. Unlike the large gap reported by the Guardian, this difference – a mere 1.8% – is not particularly unusual. It is plausible that such a result could have occurred just by chance.

Alternatively, it could have been down to the courses that students opted for. A closer look at the data shows a disproportionate number of private school pupils applied for less popular courses like Classics, and relatively more state school pupils went for competitive subjects like mathematics and medicine.

Across all subjects and both school types, 45.8% of applicants who got at least three A*s were accepted between 2010-12. If we look at the least competitive courses (i.e. subjects that took more applicants than the average), we find that 46% of the independent school applicants went for these subjects whereas 42% of state school pupils did:

The pattern is reversed when we look at the most competitive group of courses (which took fewer applicants than average): more state school pupils apply for such subjects. This suggests Simpson's Paradox is at play, and would explain why slightly more independent school applicants gained places overall.

Most of us agree that more needs to be done to make top-level education accessible to pupils from all backgrounds. However, there are wildly differing opinions about who is to blame and what needs to be done to fix the problem – opinions that often depend on people's political and social views.

While it is important to debate how to improve the situation, and to support these arguments with relevant data, we must also watch out for quirks like Simpson's paradox. If we don't, there is a risk we will spend time and effort counteracting biases that don't actually exist.

This story is published courtesy of The Conversation (under Creative Commons-Attribution/No derivatives).