Software developer shows face-swapping in realtime (w/ video)

Castro calls the technology Realtime Face Substitution, and its underpinning is the open source platform openFrameworks, which Castro helps to drive along with Zachary Lieberman and Theodore Watson.

OpenFrameworks is a coding toolkit that has been simplified for designers and artists, with C++ as the programming language.The popular openFrameworks toolkit was born out of the conviction that better tools make better projects, and that artists are generally turned on, not turned off, by using C++ for artistic expression.

The code is written to be cross platform (PC, Mac, Linux, iPhone, Android) and cross compiler. The API is designed to be minimal and easy to grasp.

Castro also makes use of Jason Saragih’s FaceTracker library, a C/C++ API for real time face alignment and tracking.

FaceTracker is open source for non-commercial use, and is available from Saragih. Castro pays tribute to FaceTracker for having contributed "much of the magic that happens in the video. The library returns a mesh that matches contours of the facial features, such as eyes, nose and mouth.

"That way the mesh obtained from a photo is matched to my own face in the video. Applying some color interpolation algorithms from Kevin Atkinson’s image clone code gives it the blending effect that can be seen in the final footage," he said.

The other technology component used is Kyle McDonald’s ofxFacetracker. "I’m also using Kyle McDonald’s ofxFaceTracker addon for openFrameworks which wraps Jason’s library for easier use," he said.

Castro said before producing the video he had thought about identity theft and what would happen if a face could be grabbed and put on someone else.

His next step is to use the technology for his art-making projects. He is planning an installation soon. His inspirations have been the art center in Valencia, Spain, lalalab, and an electronic musician, Aphex Twin, both with videos that use face-swapping technology.

© 2011 PhysOrg.com