Solving the crisis of choice online

In an age of information overload, a team of European researchers are developing technology to solve the “crisis of choice” people face when surfing the web, shopping for products online or watching TV.

Content is everywhere these days, forcing consumers to make decisions at every turn: Which TV channel to watch? Which website to go to for news? Which video to download? In what format? What book to buy from where? Consumers feel overwhelmed and standard search technology can only do so much to help. Dynamic personalised recommendations, generated automatically by smart software that gets to know your likes and dislikes over time, promise to solve the problem.

“There is so much information available today that people often don’t know where to find content that interests them. That is why a lot of companies are paying a lot of attention to machine learning technology to provide personalised searches and recommendations,” explains Alexander Voss, a researcher at Microsoft’s European Innovation Centre in Aachen, Germany.

Voss is coordinating a team of researchers from seven companies and universities who are developing advanced methods, models and algorithms to bring personalised recommendations to everything from web content and interactive TV channel guides to e-commerce. Their algorithms, developed in the EU-funded MyMedia project, are being used experimentally by project partners Microsoft, BT, the BBC and Spanish software firm Microgénesis, and are likely to be used commercially in the near future.

“We use several types and sources of information to personalise the system’s recommendations,” Voss says. “It can be both explicit and implicit,” he adds.

Explicit information is sourced from profiles created by the user. They can state their likes and dislikes, along with information about themselves, such as age, gender and place of residence, that will help the system better understand what they are interested in. It can also be drawn from their explicit interactions, such as giving a star rating to a video or news article. “This is what we call the stable information,” Voss explains.

The ‘unstable’ information is gained implicitly and is more likely to change depending on factors such as the users’ mood, context and variations in taste. It is based largely on the history of users’ choices, such as, in the case of multimedia content, what types of music they tend to listen to or what videos they watch.

Most personalised recommendation technology looks at implicit information as a black and white issue: if a user watched one video they must want to see more like it; if they didn’t watch one it must not have interested them. In a key innovation, the MyMedia researchers are adding a third and very important variable when it comes to finding what people really want.

Spoilt for choice, maybe

“We’re not just looking at choices in terms of yes or no, but also maybe,” Voss explains. “Just because someone didn’t watch a certain video doesn’t mean they wouldn’t want to, they simply might not have had the time or felt like it at that moment. Instead of excluding it, we add it to a relative ranking of recommended content that changes over time as the system builds up a better idea of the user’s interests.”

In addition, the system allows content providers to change different parameters and see how the changes affect users’ responses to recommendations. “There is no other recommender system that can do that,” Voss says.

Researchers at the University of Hildesheim, another project partner, have won several awards for the innovative approach, including the PKDD Discovery Challenge 2009 at the European Conference on Machine Learning in Bled, Slovenia. A team from the Technical University of Eindhoven, meanwhile, received the Best Short Paper and Poster Award at the 3rd ACM Conference on Recommender Systems in New York last year.

Dutch research institute Novay is also contributing to the system’s development.

The technology has proven so effective in trials that Microgénesis has formally announced plans to develop it commercially as part of its OFCommerce platform for e-commerce services, where it will help customers of online retailers find media content and other products that interest them.

"From the perspective of an SME like Microgénesis, MyMedia happens to be an excellent opportunity to enhance our e-commerce application features,” notes Laura de la Rua, R+D operation manager at Microgénesis. “In the increasingly competitive world of digital content distribution, a powerful recommender system... is a true advantage over our competitors."

In the United Kingdom, BT is rolling out an experimental application of the MyMedia machine learning system to help users of its BT Vision IPTV services.

“They have around 500,000 subscribers to their TV services with a set-top-box who can watch TV or video on demand. They want to help them find content,” Voss says.

Similarly, the BBC is exploring the application of MyMedia technology in the context of an online catch-up service that is regularly used by several million people to watch programmes on the internet that they may have missed on the TV or radio.

Microsoft, meanwhile, is using a prototype of the system in a limited trial on its German MSN site.

“We’re using it to recommend content through ClipClub, an application that marries online video sharing with social networking aimed particularly at young people. The goal is to help people not only find content but also make new friends online,” Voss explains.

The MyMedia project coordinator expects the trial applications to form the basis for commercial applications in the near future, and not just by the project partners.

The MyMedia software was released last year under an open source licence, providing all the components necessary for anyone to build a state-of-the-art recommender system. Voss sees it being taken up not only by companies involved in the distribution of multimedia content, but also by the financial services industry to help recommend investment products or even in the bio-science sector to help researchers create complex models of proteins and compounds.