Startup launches new app that could shake up conventional video viewing

A start-up from Nanyang Technological University, Singapore (NTU Singapore) has launched an innovative mobile app that allows viewers to interact with television or video screens. This could spell new opportunities for marketers as Singapore moves towards more digital applications in a SMART Nation.

As a pilot test of the technology, NTU launched this new app named HEY! Shake during this year's Freshmen Welcome. Students interacted with videos playing on campus TV screens to get access to exclusive content and win prizes.

Available free on iOS App Store and Google Play for Android phones, HEY! Shake demonstrates the potential of converting passive viewers into potential customers for broadcasters, video content providers, movie theatres and digital advertising screens.

For example, a viewer watching a television drama at home could buy the same outfit or accessories worn by an actor with a single shake of the mobile phone.

Developed by NTU spin-off company Cloud Wings, the technology powering the app was invented by Assoc Prof Wen Yonggang, Director of the Innovation Lab at the School of Computer Science and Engineering, NTU Singapore.

The app which was rolled out at the beginning of August was ranked as one of the hottest apps on both iOS App Store and Google Play in that week. It was successfully tested out during NTU's recent Freshmen Welcome Ceremony with more than 3,500 downloads and "shakes".

What users have to do is to open the app, shake their phone at a video screen, and they will be brought to various online content sites, such as Facebook, Instagram, or the NTU HEY! Magazine to win prizes.

NTU Associate Provost for Student Life, Professor Kwok Kian Woon, said the new app has enhanced the freshman experience of a vibrant campus life at NTU, a living test bed for smart technologies.

"As a technological university, we walk the talk, by embracing smart technologies which can enhance the everyday lives of our students, faculty and staff. The new HEY! Shake opens up a new avenue of internal communication via our campus screens, which can inform our students of interesting activities and news," Prof Kwok said.

New era of television engagement

Assoc Prof Wen said that the technology could usher in a new era for television and movie theatres, where information is not just a one-way street from the screen to the consumer.

"Not only can our technology make video social again but also interactive by allowing product placements and relevant information to be directly accessible to the viewer with just a simple shake of the phone," Prof Wen added.

Prof Wen says the app is another example of how basic research at the university can be translated and developed into a product. The underlying technology has already received international recognition, winning the 2015 IEEE Multimedia Magazine Best Paper Award, and the Golden Medal at the ASEAN ICT Awards 2013.

HEY! Shake uses a patent-pending technology called AirSense, which detects sound patterns encoded in the audio track to recognise which video the user is watching.

The app then connects to the cloud servers to pull related content such as additional videos, websites or lucky draws.

Mr Xia Ye, Head of Product & Business Development, Cloud Wings, said their innovation could play a part in Singapore's transformation into a smart nation, making businesses and people even more connected.

"We hope that this 'Shake Fever' will continue to spread across the NTU campus and the country, as the technology can be scaled up to all displays screens, such as those at MRT stations, bus stops, movie theatres and even on public transport," said Mr Xia, an NTU alumnus. "As long as there is a display screen and speakers, our technology can work seamlessly."

After the successful launch of HEY! Shake, NTU will continue to use the app across its campus to engage its students, while Cloud Wings is looking to expand into three business sectors.

The first is embedded advertisements and product placement which marks a shift from conventional video advertising. Users can "shake" their phones when they see a product that interests them to access the e-commerce site to purchase the product advertised.

The second is to convert home-shopping TV channels into e-commerce platforms while the third is the development of virtual coupons, where viewers can shake their phone to receive e-coupons or discount vouchers to be redeemed at physical stores.

Cloud Wings is incubated at NTU's innovation and enterprise arm, NTUitive. The founding team has received grants from various government agencies and the Singapore MIT Alliance for Research and Technology (SMART).