Reality substitution on track to replace traditional virtual reality

Until recently, virtual reality's widespread use, both commercially and in scientific research, has been hampered by the need to develop custom virtual worlds using labor-intensive 3D animation. Researchers and engineers from EPFL's Laboratory of Cognitive Neuroscience and the W Science Initiative are unveiling a Reality Substitution prototype on March 30th at The Brain Forum that could change all of that. The project, known as RealiSM, has developed an easy-to-use virtual world creator that captures real-world situations to be played back in head-mounted displays (HMD). The system will soon be employed in the lab to study memory and peri-personal space (the space defined by what is within one's reach) and will have numerous clinical uses for treating phobias and PTSD therapy.

"By combining existing technology with our video editing software and insights from the cognitive neurosciences, we are not far from making immersive experiences in the lab, the clinic and the boardroom a part of our daily lives," says W Science co-chairman Dr Jamil El-Imad "Our engineers have created a system that is almost as simple as making a home video: the user can set up the video camera unit, record, and then play the entire scene in an HMD like the Oculus Rift."

Next generation virtual reality

The prototype is essentially a video recording, editing and playback platform for the next generation of virtual reality production and experience. The video recorder, comprised of 18 high-resolution cameras pointed in all directions, captures a spherical point of view in three dimensions, and is also coupled with four pairs of electronic ears for sound recording in 360 degrees. Editing software developed by the EPFL team stitches the images together and allows for basic editing. Finally, the spherical video is played back in a custom HDM, which is itself equipped with a stereoscopic camera on the front of the display to incorporate the users body into the virtual world.

According to previous work by the researchers at EPFL, this added sense of "bodily presence" is essential in creating a truly immersive experience. "By including the users body and spatialized sound in the virtual world, the experience goes from being a floating pair of eyes in a foreign environment to an authentically lived experience with real cognitive implications," explains EPFL Professor and project co-chair Olaf Blanke.

For research, clinical applications and real-time teleconferencing

"There is a positive feedback loop between virtual reality and cognitive neuroscience," says EPFL researcher and project leader Bruno Herbelin. "On one hand, with a VR setup we have an environment that can be completely controlled and endlessly repeated—which are ideal experimental conditions. On the other hand, insights from the cognitive sciences are leading to more immersive, extremely realistic experiences for increasingly effective clinical therapies, behavioral experiments and even better gameplay for entertainment."

VR technology is currently being used across the globe for phobia treatment and PTSD therapy, often with outstanding results. The RealiSM project aims to make this technology more accessible to the average clinician, who may not have a team of engineers working for her, and could also bring the technology to developing countries. Other potential uses include immersive, real-time video recording to bring business calls to the next level or even help those working or serving their country abroad feel closer to home.