Skype eye contact finally possible (w/ Video)

Aug 27, 2013 by Angelika Jacobs
Innovative software rotates the face of the person on screen during video conferences in order to make eye contact. Credit: Computer Graphics Laboratory / ETH Zurich

(Phys.org) —Those separated from family and friends by long distances often use video conferencing services such as Skype in order to see each other when talking. But who hasn't experienced the frustration of your counterpart not making direct eye contact during the conversation? A software prototype from the Computer Graphics Laboratory ETH Zurich may be able to help.

"We want to make calls as similar as possible to a real meeting," explains Claudia Kuster, a doctoral student at the Computer Graphics Laboratory ETH Zurich. Lack of is said to be a considerable obstacle to the feel of a 'real' conversation. This problem arises because the speaker looks mainly at their counterpart's picture instead of at the camera. Kuster and her colleagues are now offering a solution to the problem for everyday use: that recognises the face in the and rotates it so that the person appears to be looking at the camera.

Until now, only larger companies have been able to afford the luxury of creating artificial eye contact during video conferences; this has required complex mirror systems or several cameras and special software. No satisfactory solution to the problem has existed for private use.

Depth map and facial recognition

This is now changing thanks to the new software that Kuster has developed under the guidance of Markus Gross, Professor of Computer Science at ETH Zurich. Thanks to Kinect, a new generation of cameras that collect colour and depth information simultaneously, the system is available for home use. The software developed by Kuster uses a depth map calculated from the image information and a programme that recognises faces in real-time video.

In contrast to previous solutions, Kuster and her colleagues do not turn the entire video image including the background, thus avoiding the problem of missing information in the original image resulting in gaps appearing in the rotated picture. Instead, her algorithm turns only the face and inserts it seamlessly into the original image. The software looks for a contour around the face in which the border pixel in the original and the corresponding pixel in the rotated image have as many similar colour values as possible.

This video is not supported by your browser at this time.

Robust under adverse conditions

"The software can be user adjusted in just a few simple steps and is very robust," says Kuster. If the programme temporarily loses sight of the face – for example, if the person turns their head or disappears behind an object such as a cup – the software leaves the original image in place. The software can cope effortlessly with changing light conditions and even two faces at the same time, as the researchers demonstrate in the above video. However, in the current version of the software, glasses prevent , so wearers must take them off for video conferences.

Although the new generation of cameras with depth sensors are affordable, they are still more expensive than a standard webcam. In addition, current laptops, tablets and smartphones are not yet equipped with this technology. Kuster and her colleagues are planning to further develop the software for mobile devices with standard webcams, and simplify it for the user as much as possible. The researchers also want to develop a Skype plug-in that users can easily install to maintain eye contact in the future.

Explore further: Crowd sourcing project to allow 3D scan-to-print web app

More information: Kuster C, et al. Gaze Correction for Home Video Conferencing. Proceedings of ACM SIGGRAPH Asia (Singapore, November 28 - December 1, 2012), ACM Transactions on Graphics, 31 (6), 174:1-174:6. graphics.ethz.ch/publications/… apers/paperKus12.php

Related Stories

Your next phone camera might be able to sense depth

Apr 12, 2013

The camera in your smartphone may soon have a new trick: depth perception. Toshiba, Samsung and Silicon Valley startup Pelican Imaging are developing image sensors and software that would allow cameras to detect the distance ...

Lifelike wobble thanks to new software

Jul 25, 2013

The cartoon characters so loved by young and old audiences alike are becoming increasingly lifelike. In order for them to look realistic, animators invest a lot of time in making a fat belly wobble as naturally ...

Crowd sourcing project to allow 3D scan-to-print web app

Aug 05, 2013

Technology to allow for printing three dimensional objects is evolving rapidly, making it difficult for some to keep up. It's also still relatively expensive. Currently, people who wish to print such an object ...

Recommended for you

Designing exascale computers

Jul 23, 2014

"Imagine a heart surgeon operating to repair a blocked coronary artery. Someday soon, the surgeon might run a detailed computer simulation of blood flowing through the patient's arteries, showing how millions ...

User comments : 3

Adjust slider to filter visible comments by rank

Display comments: newest first

verkle
1 / 5 (4) Aug 27, 2013
Really cool. The examples photos given above seem backward though. In my experience the camera is usually above the screen, so people are looking down lower than my eye level. In these examples the people are looking up higher for some reason.

ahaveland
1 / 5 (3) Aug 27, 2013
If it's a problem, why not just temporarily tape a really small webcam on the screen nearer the face?
One day, perhaps screens could have a stereo camera built in and thus be truly bidirectional.
ahmedgnz
5 / 5 (1) Aug 27, 2013
Great, so now we will be having fake eye contact rather than no eye contact. I think I would still prefer the honesty of the latter.