Facial motion capture helps bring VR documentary to life

CAMERA, the University of Bath's motion capture research centre, has collaborated with the Interactive arm of Aardman on a BBC virtual reality documentary about the story of twin sisters whose lives were torn apart by a terrible accident.

The VR documentary, called "Is Anna OK?" allows the viewer to step into the shoes of Anna and her sister Lauren to discover how their lives are changed after Anna suffers a head injury from being hit by a car.

Viewers can experience the documentary from multiple points of view, exploring Anna's fragmented memories and piecing together the story of what happened and follow Anna's journey of rehabilitation.

Researchers at CAMERA worked closely with the team at Aardman using motion capture technology to make the movement of the characters more realistic. In particular, they used the latest research methods in facial animation to analyse the 3-D movement of the actor's face and transfer it onto the face of the animated character.

Professor Darren Cosker, Director of CAMERA, led the research. He explained: "One actor played the parts of both identical twins for the documentary.

"We performed a 3-D scan of the actor's face and built an animated model from this. We then used a mixture of different research methods to analyse the fine movements of the actor's face, such as her smile and eyebrow movements.

"Each movement of the actor's face was assigned values which were then transferred to the model of the character's face. This method is much easier for the animators to edit and tweak movements rather than building a new animated face from scratch.

"This work is a result of several years of research, so it's great to be able to put our research into use on a commercial project for the first time."

Lorna Probert, Head of Interactive Production, Aardman added: "When telling the narrative of real individuals that have been affected in such a dramatic way it was really important to us that the characters felt authentic. Working with CAMERA to motion capture our actress to this level of facial detail helped us convey the emotions of the performance so the audience could feel properly immersed in their story."

The documentary was nominated for several awards last year, including the 2018 Raindance Film Festival and the 2018 Sheffield Doc Fest. It is available to download for free on Oculus Rift.

The team recently worked with Aardman on the narrative adventure game 11-11: Memories Retold, which used motion capture technology to animate the characters.