Sochi: Our tweeted emotions decrypted in real time

EPFL researchers will track emotions of the viewing public during the Olympic Games in Sochi. Via social media, they will show in real time what people are feeling during the competitions.

Online discussions offer an extraordinary windfall of information. They are free and open to scientists studying ever more precise data. Extracting keywords from tweets, it is possible to reconstruct a particular event. On this basis, the team of Human Computer Interaction Group has decided to use the upcoming Winter Olympics to test emotion recognition software. What are the strongest emotions, and how do they evolve during the competition? Which athletes do the Internet users prefer?

To achieve this performance in real time, researchers have developed an application, called EmotionWatch, that displays the various emotions in the form of a colored pie chart. This virtual watch displays the strongest trends on the computer screen, and the disk then changes color depending on the intensity of emotion. Each of the 20 feelings considered has its own color – yellow for joy, red for anger, and blue for worry.

A student project launches the concept

It all began with a semester project. The laboratory sought to use and develop the considerable windfall of conversations exchanged on the web at the Olympics in 2012. Student Renato Kempter was interested in human interactions on the Internet. During three months the young researcher and two PhD students set out to decrypt and codify some 50 million tweets posted at the London Olympics. "What was interesting is that we could perfectly follow the event just by reading the tweets. The reaction times were incredibly fast. People described their pride, their sadness .... There was even a scandal with badminton, and it was plain to see in the tweets."

EmotionWatch

To understand and classify the opinions of Internet users, Renato Kempter had to imagine how to interpret and represent them. "I relied on a wheel of emotions designed by psychology students at the University of Geneva." Based on this work, the young researcher then enhanced this tool by adding an extra visual element: color. He developed a disk surrounded by 20 colored dots, each representing a particular emotion. Through keywords detected in the conversations, the software dispatches emotions and displays the color of the most intense feeling. "It became increasingly complex because I had to also consider the factor of time and the frequency of tweets."

The language of Twitter and Chinese emoticons

Alongside the development of the software, the researchers extracted the most significant words found in the sports vocabulary. They created a lexicon of 20 emotions and associated positive and negative values. Pearl Pu Faltings, who heads the project, also focused on exchanges conducted on Sina.com – China's Twitter. Computer scientists have thus added an additional challenge to deciphering opinions: "The Chinese love to punctuate their messages with emoticons. They use more than 70. We had to choose 20 which we associated with an emotion related to words already included in our software, such as pride, joy, sadness or disappointment."

The challenge: capture live tweets

Undoubtedly the major challenge will be the capture and storage of all these messages with the underlying fear of a crash, because with such an operation, there is no second chance. In the meantime, the accuracy of the software has been improved. It must be able to sort the tweets by keywords to recognize all sports. "The application must be more than just attractive or fun. For us scientists, it is important that our system is operational in other competitions and reliable in the recognition of Twitter vocabulary," says Pearl Pu Faltings.

PhD student Valentina Sintsova focuses on refining the data: "The program was designed to recognize English and Chinese emoticons, and now it must also accept the contractions and the specific vocabulary used on Twitter such as 'love u' or '4 you,' etc., and even be able to separate emotions vis-à-vis two players in a game of tennis, for example."

After the live test at the Olympic Games in Sochi, the application could be used for commercial purposes or marketed to see consumers' reactions to a new product, or those of voters during the presentation of candidates in a political election.