Crowdsourcing for creating 3D videos

Mar 14, 2014

Advanced graphics processors, new algorithms and advanced mathematics will soon make a new 3D video technology gathering feed from multiple sources possible.

As smart phones are becoming ubiquitous, they increasingly serve as a link to social networks. Networks sharing of large concerts or sporting events with 3D imagery will soon be made possible thanks to a developed under SceneNet, an EU funded project, due to be completed in 2016. Chen Sagiv is an applied mathematician, co-founder and CEO of image and signal processing company SagivTech, based in Ra'anana, Israel, who is also the project coordinator. She talks to about the promises of this emerging technology, based on heavy-duty image processing that require mathematical wizardry.

What novel technology have you been involved in developing?

The main idea of the EU project is to use the power of the crowd to generate 3D scenes. For example, at rock concerts you see many people taking videos with their smartphones. At the end of the day each person is left with their own video, which was taken from a specific angle, and perhaps the quality is also poor. We are going to take the input of many participants, taken from different viewing angles. We will upload these videos to a server where they are synchronised, and thus obtain a 3-D reconstruction of the scene. This will allow people to view the scene from any angle. They will then be able to share the video in social networks, maybe by creating ad hoc communities.

What has been the main challenge towards creating 3-D videos?

For videos of static objects we use tools that already exist. One of our main challenges is to produce a viewer for dynamic scenes. Let's say we have the basic version of the entire pipeline, the sequence of operations leading to the 3D video. We have clients running on mobile devices for capturing images and then uploading them to the server. On the server we have the software to synchronise the different videos and to produce the 3D scene. We need a lot of computing power to do this. We have a preliminary proof of concept, but at this point, it is an internal demo, which will be refined for a presentation at the end of the project.

What are the remaining challenges?

One of our main challenges is to actually produce a viewer for the dynamic scenes. Actually, it would be nice to present it as a hologram, maybe this is an interesting idea. It would however be very complex, and require very sophisticated optics. We would not create such a display system ourselves. We plan to use technology that already exists.

What kind of expertise is required to develop such technology?

We are experts in GPU computing, and this is our leverage in this project. GPUs are graphic processing units, processors that are very fast for graphics processing. We also use them for mathematical computation. Our main aim is to develop the by looking at the mathematical aspects, and by developing new algorithms; we are developing new software for synchronising dynamic scenes with a lot of movement.

What are the potential applications of such technology?

Our project is part of the future emerging technologies EU funding scheme. It could be applied in many areas, such as surveillance or tourism. At this point we are looking at the music community. On the advisory board of the EU project, we have a musician and someone from a company which organises events. As a test case we are using the concept of a musical event due to be organised around January 2016.

Explore further: Australian police get hand-held 3D crime scene laser scanner

More information:

add to favorites email to friend print save as pdf

Related Stories

Hitachi demos 3D real-world object projector

Oct 05, 2011

( -- In a feat of technical wizardry combined with several doses of panache, Hitachi has demoed a 3D projector that can project images onto real-world objects in stunning fashion. For the demo, ...

Novel equations improve image processing

Jul 06, 2012

A specific class of mathematical equations is helping to solve major challenges in the field, facilitating advanced modelling in a number of applications from climate change to desertification.

Recommended for you

Computerized emotion detector

Sep 16, 2014

Face recognition software measures various parameters in a mug shot, such as the distance between the person's eyes, the height from lip to top of their nose and various other metrics and then compares it with photos of people ...

Cutting the cloud computing carbon cost

Sep 12, 2014

Cloud computing involves displacing data storage and processing from the user's computer on to remote servers. It can provide users with more storage space and computing power that they can then access from anywhere in the ...

Teaching computers the nuances of human conversation

Sep 12, 2014

Computer scientists have successfully developed programs to recognize spoken language, as in automated phone systems that respond to voice prompts and voice-activated assistants like Apple's Siri.

Mapping the connections between diverse sets of data

Sep 12, 2014

What is a map? Most often, it's a visual tool used to demonstrate the relationship between multiple places in geographic space. They're useful because you can look at one and very quickly pick up on the general ...

User comments : 0