Software developer shows face-swapping in realtime (w/ video)

September 23, 2011 by Nancy Owano, report
Software developer shows face-swapping in realtime (w/ video)

( -- Software developer Arturo Castro and media artist Kyle MacDonald have put out a video demo of their software that replaces their faces with other faces in realtime. Their face-swapping, face-morphing technology shows how their software can transform new or celebrity faces with your own. The two of them pout, pucker up, grimace, and perform other facial antics to breathe even more life into their changing faces of the likes of Paris Hilton, Marilyn Monroe, Chairman Mao, Lenin and Andy Warhol.

Castro calls the technology Realtime Face Substitution, and its underpinning is the openFrameworks, which Castro helps to drive along with Zachary Lieberman and Theodore Watson.

OpenFrameworks is a coding toolkit that has been simplified for designers and artists, with C++ as the programming language.The popular openFrameworks toolkit was born out of the conviction that better tools make better projects, and that artists are generally turned on, not turned off, by using C++ for artistic expression.

The code is written to be cross platform (PC, Mac, Linux, , ) and cross compiler. The API is designed to be minimal and easy to grasp.

Castro also makes use of Jason Saragih’s FaceTracker library, a C/C++ API for real time face alignment and tracking.

FaceTracker is open source for non-commercial use, and is available from Saragih. Castro pays tribute to FaceTracker for having contributed "much of the magic that happens in the video. The library returns a mesh that matches contours of the facial features, such as eyes, nose and mouth.

"That way the mesh obtained from a photo is matched to my own face in the video. Applying some color interpolation algorithms from Kevin Atkinson’s image clone code gives it the blending effect that can be seen in the final footage," he said.

The other technology component used is Kyle McDonald’s ofxFacetracker. "I’m also using Kyle McDonald’s ofxFaceTracker addon for openFrameworks which wraps Jason’s library for easier use," he said.

Castro said before producing the video he had thought about identity theft and what would happen if a face could be grabbed and put on someone else.

His next step is to use the technology for his art-making projects. He is planning an installation soon. His inspirations have been the art center in Valencia, Spain, lalalab, and an electronic musician, Aphex Twin, both with videos that use face-swapping technology.

Explore further: I like your face -- but why? A new computer model pinpoints the reasons

Related Stories

The nose knows: 2 fixation points needed for face recognition

October 20, 2008

Many of us are bad at remembering names but we are very quick to point out that at least we never forget a face. Never mind recognizing a familiar face- how is it that we recognize faces at all? Facial recognition is so automatic ...

Facial recognition at first glance

July 1, 2010

We meet a multitude of people on a daily basis: the nice waitress in the coffee shop around the corner, the bus driver or the colleagues at the office. Without the ability to recognize faces at first glance we would not be ...

Neuroscience: Blue in the face

March 25, 2011

The way that humans perceive each other is strongly affected by the configuration, contour, and complexion of faces. Researchers from Toyohashi Tech report the importance of facial color on neural responses underlying perception.

Can't place that face? The trouble may be in your neurons

July 28, 2010

A specific area in our brains is responsible for processing information about human and animal faces, both how we recognize them and how we interpret facial expressions. Now, Tel Aviv University research is exploring what ...

Recommended for you

Self-assembling 3-D battery would charge in seconds

May 17, 2018

The world is a big place, but it's gotten smaller with the advent of technologies that put people from across the globe in the palm of one's hand. And as the world has shrunk, it has also demanded that things happen ever ...

HSBC, ING banks announce blockchain first

May 14, 2018

Banking giants HSBC and ING on Monday said they had carried out a landmark blockchain transaction aimed at speeding up payment processes and making them more secure.


Adjust slider to filter visible comments by rank

Display comments: newest first

3.8 / 5 (5) Sep 23, 2011
Impressive. Won't be long before the accuracy exceeds our ability to detect with the naked eye.
5 / 5 (2) Sep 23, 2011
Looks like some plastic surgery gone horribly wrong.
not rated yet Sep 23, 2011
The two faced SOBs (sons of bitmaps).
not rated yet Sep 23, 2011
Looks like some plastic surgery gone horribly wrong.

That happens not just on webcams.. Welcome to Hollywood!
not rated yet Oct 07, 2011
Face talk scams will abound. The newest versions of the Nigerian pig in a poke scam will include this tech.

You receieved a call from a famous person trapped in africa during a red cross promotional event.

"Hi this is brad pitt, I was in africa _________
A) helping people with Food,
B)helping people with Water,
C) Collecting different kinds of Children.
And got trapped please send me money and ill let you be in a movie!"

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.