EU researchers saw Brexit coming

Following the United Kingdom's decision to leave the European Union on 23 June 2016, the EU-funded SENSEI project has revealed that it accurately predicted the result of the referendum by analysing 6 million social media conversations in the weeks preceding the vote.

The SENSEI project aims to make sense of the millions of blog posts and social media conversations that take place every day. To do this, the overall goals of the project have been twofold: Firstly, project partners have developed sophisticated summarisation and analytics technology to help users make sense of human interaction from diverse media channels. Secondly, they have designed and evaluated the SENSEI summarisation technology in real-world environments, with the aim to improve task performance and the productivity and efficiency of end-users, including, for example, data and media analysts, journalists and editors.

However, the project does not just rely on technology – human intervention is also a crucial element in how the project has been analysing social media conversations, with an emphasis by the project on using a combination of 'man and machine' to predict the outcome of major votes and elections. This has included the inconclusive December 2015 Spanish general election and the Brexit referendum.

Predicting Brexit

'Over the course of this campaign, the SENSEI project's unique combination of humans and machine reading algorithms has listened to more than 6 million social media conversations relating to the Brexit vote to identify and predict voting sentiment,' commented Prof. Giuseppe Riccardi, SENSEI project coordinator. 'We are delighted that, following our equally accurate prediction of the Spanish general election, we have once again predicted the outcome of a major political event with very high accuracy. It appears that the momentum on UK social media started to change on 21 June [two days before the referendum vote] and we watched it move.'

Whilst traditional pollsters and bookmakers were predicting a very narrow win for the Remain campaign right up until the evening of 23 June, the online conversations monitored by the SENSEI project team began to tell a different story. Earlier during the day of 23 June as polling stations opened across the country, social media chatter in the UK was 49.63 % in favour of Leave and 50.37 % in favour of Remain and it was too close to call.

But by late afternoon, social media chatter had changed as more and more undecided voters took their decision and there was a dramatic swing to Leave. The SENSEI system predicted a final vote breakdown of 48 % Remain and 52 % Leave, which was precisely the final result of the referendum that was announced in the early morning hours of 24 June. Social media, in essence, had gotten it right.

Dr. Hugo Zaragoza, a member of the SENSEI project team and an expert in opinion analytics commented: 'Once again, our data told a very different story to the pollsters and we were proved to be more accurate. This is a great result for the project. The ability to listen to millions of pieces of conversations and then analysing them for sentiment, using a combination of humans and machines, has proved once again to be more successful than traditional polling methods... at times, the social media chatter was intense and passionate. Since we started this project, we have listened to and analysed 300 000 social media conversations on Brexit every day and we're delighted to have called the result accurately.'

Commercial opportunities

By highlighting how using a powerful combination of technology-driven analytics, combined with a unique qualitative approach using human intervention can fundamentally change the way social media chatter is understood and how it can be applied commercially, the SENSEI project is providing a highly valuable tool to help commentators to understand what is being said.

'This is a really powerful tool for politics and business after the disastrous set of predictions from the main players,' said Dr Zaragoza. 'We have demonstrated how accurate our technique is.'