Crowdsourcing for creating 3D videos

Mar 14, 2014

Advanced graphics processors, new algorithms and advanced mathematics will soon make a new 3D video technology gathering feed from multiple sources possible.

As smart phones are becoming ubiquitous, they increasingly serve as a link to social networks. Networks sharing of large concerts or sporting events with 3D imagery will soon be made possible thanks to a developed under SceneNet, an EU funded project, due to be completed in 2016. Chen Sagiv is an applied mathematician, co-founder and CEO of image and signal processing company SagivTech, based in Ra'anana, Israel, who is also the project coordinator. She talks to youris.com about the promises of this emerging technology, based on heavy-duty image processing that require mathematical wizardry.

What novel technology have you been involved in developing?

The main idea of the EU project is to use the power of the crowd to generate 3D scenes. For example, at rock concerts you see many people taking videos with their smartphones. At the end of the day each person is left with their own video, which was taken from a specific angle, and perhaps the quality is also poor. We are going to take the input of many participants, taken from different viewing angles. We will upload these videos to a server where they are synchronised, and thus obtain a 3-D reconstruction of the scene. This will allow people to view the scene from any angle. They will then be able to share the video in social networks, maybe by creating ad hoc communities.

What has been the main challenge towards creating 3-D videos?

For videos of static objects we use tools that already exist. One of our main challenges is to produce a viewer for dynamic scenes. Let's say we have the basic version of the entire pipeline, the sequence of operations leading to the 3D video. We have clients running on mobile devices for capturing images and then uploading them to the server. On the server we have the software to synchronise the different videos and to produce the 3D scene. We need a lot of computing power to do this. We have a preliminary proof of concept, but at this point, it is an internal demo, which will be refined for a presentation at the end of the project.

What are the remaining challenges?

One of our main challenges is to actually produce a viewer for the dynamic scenes. Actually, it would be nice to present it as a hologram, maybe this is an interesting idea. It would however be very complex, and require very sophisticated optics. We would not create such a display system ourselves. We plan to use technology that already exists.

What kind of expertise is required to develop such technology?

We are experts in GPU computing, and this is our leverage in this project. GPUs are graphic processing units, processors that are very fast for graphics processing. We also use them for mathematical computation. Our main aim is to develop the by looking at the mathematical aspects, and by developing new algorithms; we are developing new software for synchronising dynamic scenes with a lot of movement.

What are the potential applications of such technology?

Our project is part of the future emerging technologies EU funding scheme. It could be applied in many areas, such as surveillance or tourism. At this point we are looking at the music community. On the advisory board of the EU project, we have a musician and someone from a company which organises events. As a test case we are using the concept of a musical event due to be organised around January 2016.

Explore further: Dartmouth contests showcase computer-generated creativity

More information: scenenet.uni-bremen.de/
www.sagivtech.com/

Related Stories

Hitachi demos 3D real-world object projector

Oct 05, 2011

(PhysOrg.com) -- In a feat of technical wizardry combined with several doses of panache, Hitachi has demoed a 3D projector that can project images onto real-world objects in stunning fashion. For the demo, ...

Novel equations improve image processing

Jul 06, 2012

A specific class of mathematical equations is helping to solve major challenges in the field, facilitating advanced modelling in a number of applications from climate change to desertification.

Recommended for you

EU open source software project receives green light

Jul 01, 2015

An open source software project involving the University of Southampton to extend the capacity of computational mathematics and interactive computing environments has received over seven million euros in EU funding.

Can computers be creative?

Jul 01, 2015

The EU-funded 'What-if Machine' (WHIM) project not only generates fictional storylines but also judges their potential usefulness and appeal. It represents a major advance in the field of computational creativity.

Algorithm detects nudity in images, offers demo page

Jul 01, 2015

An algorithm has been designed to tell if somebody in a color photo is naked. Isitnude.com launched earlier this month; its demo page invites you to try it out to test its power in nudity detection. You ...

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.