Software developer shows face-swapping in realtime (w/ video)

Sep 23, 2011 by Nancy Owano report
Software developer shows face-swapping in realtime (w/ video)

(PhysOrg.com) -- Software developer Arturo Castro and media artist Kyle MacDonald have put out a video demo of their software that replaces their faces with other faces in realtime. Their face-swapping, face-morphing technology shows how their software can transform new or celebrity faces with your own. The two of them pout, pucker up, grimace, and perform other facial antics to breathe even more life into their changing faces of the likes of Paris Hilton, Marilyn Monroe, Chairman Mao, Lenin and Andy Warhol.

Castro calls the technology Realtime Face Substitution, and its underpinning is the openFrameworks, which Castro helps to drive along with Zachary Lieberman and Theodore Watson.

OpenFrameworks is a coding toolkit that has been simplified for designers and artists, with C++ as the programming language.The popular openFrameworks toolkit was born out of the conviction that better tools make better projects, and that artists are generally turned on, not turned off, by using C++ for artistic expression.

The code is written to be cross platform (PC, Mac, Linux, , ) and cross compiler. The API is designed to be minimal and easy to grasp.

Castro also makes use of Jason Saragih’s FaceTracker library, a C/C++ API for real time face alignment and tracking.

FaceTracker is open source for non-commercial use, and is available from Saragih. Castro pays tribute to FaceTracker for having contributed "much of the magic that happens in the video. The library returns a mesh that matches contours of the facial features, such as eyes, nose and mouth.

"That way the mesh obtained from a photo is matched to my own face in the video. Applying some color interpolation algorithms from Kevin Atkinson’s image clone code gives it the blending effect that can be seen in the final footage," he said.

The other technology component used is Kyle McDonald’s ofxFacetracker. "I’m also using Kyle McDonald’s ofxFaceTracker addon for openFrameworks which wraps Jason’s library for easier use," he said.

Castro said before producing the video he had thought about identity theft and what would happen if a face could be grabbed and put on someone else.

His next step is to use the technology for his art-making projects. He is planning an installation soon. His inspirations have been the art center in Valencia, Spain, lalalab, and an electronic musician, Aphex Twin, both with videos that use face-swapping technology.


Explore further: Many docs believe mobile health apps can improve patient care

Related Stories

The nose knows: 2 fixation points needed for face recognition

Oct 20, 2008

Many of us are bad at remembering names but we are very quick to point out that at least we never forget a face. Never mind recognizing a familiar face- how is it that we recognize faces at all? Facial recognition is so automatic ...

Facial recognition at first glance

Jul 01, 2010

We meet a multitude of people on a daily basis: the nice waitress in the coffee shop around the corner, the bus driver or the colleagues at the office. Without the ability to recognize faces at first glance ...

Neuroscience: Blue in the face

Mar 25, 2011

The way that humans perceive each other is strongly affected by the configuration, contour, and complexion of faces. Researchers from Toyohashi Tech report the importance of facial color on neural responses ...

Can't place that face? The trouble may be in your neurons

Jul 28, 2010

A specific area in our brains is responsible for processing information about human and animal faces, both how we recognize them and how we interpret facial expressions. Now, Tel Aviv University research is exploring what ...

Recommended for you

Facebook sues law firms, claims fraud

9 hours ago

Facebook is suing several law firms that represented a man who claimed he owned half of the social network and was entitled to billions of dollars from the company and its CEO Mark Zuckerberg.

IBM 3Q disappoints as it sheds 'empty calories'

9 hours ago

IBM disappointed investors Monday, reporting weak revenue growth again and a big charge to shed its costly chipmaking division as the tech giant tries to steer its business toward cloud computing and social-mobile ...

User comments : 5

Adjust slider to filter visible comments by rank

Display comments: newest first

Code_Warrior
3.8 / 5 (5) Sep 23, 2011
Impressive. Won't be long before the accuracy exceeds our ability to detect with the naked eye.
Nikola
5 / 5 (2) Sep 23, 2011
Looks like some plastic surgery gone horribly wrong.
CarolinaScotsman
not rated yet Sep 23, 2011
The two faced SOBs (sons of bitmaps).
Star_Gazer
not rated yet Sep 23, 2011
Looks like some plastic surgery gone horribly wrong.

That happens not just on webcams.. Welcome to Hollywood!
Zomax
not rated yet Oct 07, 2011
Face talk scams will abound. The newest versions of the Nigerian pig in a poke scam will include this tech.

You receieved a call from a famous person trapped in africa during a red cross promotional event.

"Hi this is brad pitt, I was in africa _________
A) helping people with Food,
B)helping people with Water,
C) Collecting different kinds of Children.
And got trapped please send me money and ill let you be in a movie!"