Linking television and the Internet

In the "LinkedTV" project, researchers are seamlessly connecting TV offerings with the Internet. Audiences will benefit from an informative and personalized viewing experience.

The panel discussion is getting heated – but what exactly is in the new proposed law that the experts on TV are arguing about so vigorously? Up until now, spontaneous questions such as these that arise during a program had to be clarified through a viewer's own research on the Internet.

If it's up to researchers at the Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS in Sankt Augustin, Germany, viewers will no longer have to look up such additional information in the future. In the project "LinkedTV", the institute is working with eleven partners from seven countries on a new television concept. "We want to seamlessly combine television and the Internet so that viewers can directly access background information about the current program during the show, without having to spend a lot of time and effort themselves in searching for it," explains project manager Heike Horstmann. Some channels are already taking steps toward such a linking of TV and the Web – but this requires a relatively high amount of effort by personnel: The editorial staff has to scour the Internet or the broadcaster archive for suitable material and provide the links to the broadcast manually.

Software accelerates the editorial team's research

The project partners have developed intelligent software which will greatly simplify this laborious work in the future. The solution scans the contents of a show prior to its broadcast via speech analysis and image processing for topic-related content from the Web. Since the software provides a great number of unfiltered hits and semantic links, the researchers have developed methods to narrow down the results according to cer-
tain criteria. "For example, content will be displayed only if it complies with laws for the protection of minors and for which the copyright is no problem," Horstmann explains. The editorial team adds the final touches, checking suggested content for relevance and eliminating any duplications from the hit list.

Parallel to the broadcast, the viewer then finds information sorted according to topic for each chapter of a TV program. If desired, he or she can specify filter criteria that determine which content is displayed – defining topics of particular interest or hiding results for specific content. In addition, the software is also capable of learning and adjusts the offer of information to the user's behavior. "If the viewer repeatedly accesses the weather report or information on a particular topic, for instance, this content will be displayed in a prioritized fashion," says Horstmann. The viewer can view both broadcast and additional content directly on the screen of an Internet-capable television or on another end device with Web access, such as a tablet or laptop. The best approach would be to use two screens which are synchronized with each other in parallel. Generally, the show then runs on the television, while further content is displayed on the "second screen." "The user controls both devices via the second screen. He can then, for example, interrupt the program on the television to inform himself of a particular detail via the tablet," explains Horstmann.

To give a practical idea of how linked television might look, IAIS researchers and their project partners have enriched two real formats – a newscast by Rundfunk Berlin-Brandenburg and a documentary series of the Dutch TV station AVRO – with additional virtual content. The results will be presented this year at several professional events – including IBC 2014 in Amsterdam.