Software developer shows face-swapping in realtime (w/ video)

Sep 23, 2011 by Nancy Owano report
Software developer shows face-swapping in realtime (w/ video)

(PhysOrg.com) -- Software developer Arturo Castro and media artist Kyle MacDonald have put out a video demo of their software that replaces their faces with other faces in realtime. Their face-swapping, face-morphing technology shows how their software can transform new or celebrity faces with your own. The two of them pout, pucker up, grimace, and perform other facial antics to breathe even more life into their changing faces of the likes of Paris Hilton, Marilyn Monroe, Chairman Mao, Lenin and Andy Warhol.

Castro calls the technology Realtime Face Substitution, and its underpinning is the openFrameworks, which Castro helps to drive along with Zachary Lieberman and Theodore Watson.

OpenFrameworks is a coding toolkit that has been simplified for designers and artists, with C++ as the programming language.The popular openFrameworks toolkit was born out of the conviction that better tools make better projects, and that artists are generally turned on, not turned off, by using C++ for artistic expression.

The code is written to be cross platform (PC, Mac, Linux, , ) and cross compiler. The API is designed to be minimal and easy to grasp.

Castro also makes use of Jason Saragih’s FaceTracker library, a C/C++ API for real time face alignment and tracking.

FaceTracker is open source for non-commercial use, and is available from Saragih. Castro pays tribute to FaceTracker for having contributed "much of the magic that happens in the video. The library returns a mesh that matches contours of the facial features, such as eyes, nose and mouth.

"That way the mesh obtained from a photo is matched to my own face in the video. Applying some color interpolation algorithms from Kevin Atkinson’s image clone code gives it the blending effect that can be seen in the final footage," he said.

The other technology component used is Kyle McDonald’s ofxFacetracker. "I’m also using Kyle McDonald’s ofxFaceTracker addon for openFrameworks which wraps Jason’s library for easier use," he said.

Castro said before producing the video he had thought about identity theft and what would happen if a face could be grabbed and put on someone else.

His next step is to use the technology for his art-making projects. He is planning an installation soon. His inspirations have been the art center in Valencia, Spain, lalalab, and an electronic musician, Aphex Twin, both with videos that use face-swapping technology.


Explore further: Fujitsu develops technology to quickly detect latent malware activity in internal networks

Related Stories

The nose knows: 2 fixation points needed for face recognition

Oct 20, 2008

Many of us are bad at remembering names but we are very quick to point out that at least we never forget a face. Never mind recognizing a familiar face- how is it that we recognize faces at all? Facial recognition is so automatic ...

Facial recognition at first glance

Jul 01, 2010

We meet a multitude of people on a daily basis: the nice waitress in the coffee shop around the corner, the bus driver or the colleagues at the office. Without the ability to recognize faces at first glance ...

Neuroscience: Blue in the face

Mar 25, 2011

The way that humans perceive each other is strongly affected by the configuration, contour, and complexion of faces. Researchers from Toyohashi Tech report the importance of facial color on neural responses ...

Can't place that face? The trouble may be in your neurons

Jul 28, 2010

A specific area in our brains is responsible for processing information about human and animal faces, both how we recognize them and how we interpret facial expressions. Now, Tel Aviv University research is exploring what ...

Recommended for you

Microsoft CEO is driving data-culture mindset

20 hours ago

(Phys.org) —Microsoft's future strategy: is all about leveraging data, from different sources, coming together using one cohesive Microsoft architecture. Microsoft CEO Satya Nadella on Tuesday, both in ...

Enabling dynamic prioritization of data in the cloud

Apr 14, 2014

IBM inventors have patented a cloud computing invention that can improve quality of service for clients by enabling data to be dynamically modified, prioritized and shared across a cloud environment.

User comments : 5

Adjust slider to filter visible comments by rank

Display comments: newest first

Code_Warrior
3.8 / 5 (5) Sep 23, 2011
Impressive. Won't be long before the accuracy exceeds our ability to detect with the naked eye.
Nikola
5 / 5 (2) Sep 23, 2011
Looks like some plastic surgery gone horribly wrong.
CarolinaScotsman
not rated yet Sep 23, 2011
The two faced SOBs (sons of bitmaps).
Star_Gazer
not rated yet Sep 23, 2011
Looks like some plastic surgery gone horribly wrong.

That happens not just on webcams.. Welcome to Hollywood!
Zomax
not rated yet Oct 07, 2011
Face talk scams will abound. The newest versions of the Nigerian pig in a poke scam will include this tech.

You receieved a call from a famous person trapped in africa during a red cross promotional event.

"Hi this is brad pitt, I was in africa _________
A) helping people with Food,
B)helping people with Water,
C) Collecting different kinds of Children.
And got trapped please send me money and ill let you be in a movie!"

More news stories

Tiny power plants hold promise for nuclear energy

Small underground nuclear power plants that could be cheaper to build than their behemoth counterparts may herald the future for an energy industry under intense scrutiny since the Fukushima disaster, the ...

Hand out money with my mobile? I think I'm ready

A service is soon to launch in the UK that will enable us to transfer money to other people using just their name and mobile number. Paym is being hailed as a revolution in banking because you can pay peopl ...