Skype eye contact finally possible (w/ Video)

Aug 27, 2013 by Angelika Jacobs
Innovative software rotates the face of the person on screen during video conferences in order to make eye contact. Credit: Computer Graphics Laboratory / ETH Zurich

(Phys.org) —Those separated from family and friends by long distances often use video conferencing services such as Skype in order to see each other when talking. But who hasn't experienced the frustration of your counterpart not making direct eye contact during the conversation? A software prototype from the Computer Graphics Laboratory ETH Zurich may be able to help.

"We want to make calls as similar as possible to a real meeting," explains Claudia Kuster, a doctoral student at the Computer Graphics Laboratory ETH Zurich. Lack of is said to be a considerable obstacle to the feel of a 'real' conversation. This problem arises because the speaker looks mainly at their counterpart's picture instead of at the camera. Kuster and her colleagues are now offering a solution to the problem for everyday use: that recognises the face in the and rotates it so that the person appears to be looking at the camera.

Until now, only larger companies have been able to afford the luxury of creating artificial eye contact during video conferences; this has required complex mirror systems or several cameras and special software. No satisfactory solution to the problem has existed for private use.

Depth map and facial recognition

This is now changing thanks to the new software that Kuster has developed under the guidance of Markus Gross, Professor of Computer Science at ETH Zurich. Thanks to Kinect, a new generation of cameras that collect colour and depth information simultaneously, the system is available for home use. The software developed by Kuster uses a depth map calculated from the image information and a programme that recognises faces in real-time video.

In contrast to previous solutions, Kuster and her colleagues do not turn the entire video image including the background, thus avoiding the problem of missing information in the original image resulting in gaps appearing in the rotated picture. Instead, her algorithm turns only the face and inserts it seamlessly into the original image. The software looks for a contour around the face in which the border pixel in the original and the corresponding pixel in the rotated image have as many similar colour values as possible.

This video is not supported by your browser at this time.

Robust under adverse conditions

"The software can be user adjusted in just a few simple steps and is very robust," says Kuster. If the programme temporarily loses sight of the face – for example, if the person turns their head or disappears behind an object such as a cup – the software leaves the original image in place. The software can cope effortlessly with changing light conditions and even two faces at the same time, as the researchers demonstrate in the above video. However, in the current version of the software, glasses prevent , so wearers must take them off for video conferences.

Although the new generation of cameras with depth sensors are affordable, they are still more expensive than a standard webcam. In addition, current laptops, tablets and smartphones are not yet equipped with this technology. Kuster and her colleagues are planning to further develop the software for mobile devices with standard webcams, and simplify it for the user as much as possible. The researchers also want to develop a Skype plug-in that users can easily install to maintain eye contact in the future.

Explore further: Lifelike wobble thanks to new software

More information: Kuster C, et al. Gaze Correction for Home Video Conferencing. Proceedings of ACM SIGGRAPH Asia (Singapore, November 28 - December 1, 2012), ACM Transactions on Graphics, 31 (6), 174:1-174:6. graphics.ethz.ch/publications/papers/paperKus12.php

Related Stories

Your next phone camera might be able to sense depth

Apr 12, 2013

The camera in your smartphone may soon have a new trick: depth perception. Toshiba, Samsung and Silicon Valley startup Pelican Imaging are developing image sensors and software that would allow cameras to detect the distance ...

Lifelike wobble thanks to new software

Jul 25, 2013

The cartoon characters so loved by young and old audiences alike are becoming increasingly lifelike. In order for them to look realistic, animators invest a lot of time in making a fat belly wobble as naturally ...

Crowd sourcing project to allow 3D scan-to-print web app

Aug 05, 2013

Technology to allow for printing three dimensional objects is evolving rapidly, making it difficult for some to keep up. It's also still relatively expensive. Currently, people who wish to print such an object ...

Recommended for you

User comments : 3

Adjust slider to filter visible comments by rank

Display comments: newest first

verkle
1 / 5 (4) Aug 27, 2013
Really cool. The examples photos given above seem backward though. In my experience the camera is usually above the screen, so people are looking down lower than my eye level. In these examples the people are looking up higher for some reason.

ahaveland
1 / 5 (3) Aug 27, 2013
If it's a problem, why not just temporarily tape a really small webcam on the screen nearer the face?
One day, perhaps screens could have a stereo camera built in and thus be truly bidirectional.
ahmedgnz
5 / 5 (1) Aug 27, 2013
Great, so now we will be having fake eye contact rather than no eye contact. I think I would still prefer the honesty of the latter.

More news stories

Ex-Apple chief plans mobile phone for India

Former Apple chief executive John Sculley, whose marketing skills helped bring the personal computer to desktops worldwide, says he plans to launch a mobile phone in India to exploit its still largely untapped ...

Airbnb rental site raises $450 mn

Online lodging listings website Airbnb inked a $450 million funding deal with investors led by TPG, a source close to the matter said Friday.

Health care site flagged in Heartbleed review

People with accounts on the enrollment website for President Barack Obama's signature health care law are being told to change their passwords following an administration-wide review of the government's vulnerability to the ...

A homemade solar lamp for developing countries

(Phys.org) —The solar lamp developed by the start-up LEDsafari is a more effective, safer, and less expensive form of illumination than the traditional oil lamp currently used by more than one billion people ...

NASA's space station Robonaut finally getting legs

Robonaut, the first out-of-this-world humanoid, is finally getting its space legs. For three years, Robonaut has had to manage from the waist up. This new pair of legs means the experimental robot—now stuck ...

Filipino tests negative for Middle East virus

A Filipino nurse who tested positive for the Middle East virus has been found free of infection in a subsequent examination after he returned home, Philippine health officials said Saturday.

Egypt archaeologists find ancient writer's tomb

Egypt's minister of antiquities says a team of Spanish archaeologists has discovered two tombs in the southern part of the country, one of them belonging to a writer and containing a trove of artifacts including reed pens ...