From 3-D to 6-D: Researchers developing super-realistic image system

Aug 08, 2008
Traditional flat screen displays (bottom left) present 2D images. 3D and 4D displays have been proposed making use of lenslet arrays to shape a fixed outgoing light field for horizontal or bidirectional parallax (top left). Image / Martin Fuchs; Ramesh Raskar; Hans-Peter Seidel; and Hendrik P. A. Lensch

( -- By producing "6-D" images, an MIT professor and colleagues are creating unusually realistic pictures that not only have a full three-dimensional appearance, but also respond to their environment, producing natural shadows and highlights depending on the direction and intensity of the illumination around them.

The process can also be used to create images that change over time as the illumination changes, resulting in animated pictures that move just from changes in the sun's position, with no electronics or active control.

To create "the ultimate synthetic display," says Ramesh Raskar, an associate professor at the MIT Media Lab, "the display should respond not just to a change in viewpoint, but to changes in the surrounding light."

Raskar and his colleagues will describe the system, which is based entirely on an arrangement of lenses and screens, on Aug. 11 at the annual SIGGRAPH (Special Interest Group on Graphics and Interactive Techniques) conference of the Association for Computing Machinery held in Los Angeles.

Three-dimensional images, created using a variety of systems that make separate images for each eye, have been around for many decades. The new MIT process could bring an unprecedented degree of realism to such images.

The basic concept is similar to those inexpensive 3-D displays sometimes used on postcards and novelty items, that use an overlay of plastic that contains a series of parallel linear lenses that create a visible set of vertical lines over the image. (It is a different approach from that used to create holograms, which require laser light to create.) In addition to three-dimensional images, these are sometimes used to present a series of images that change as you view them from different angles from side to side. This can simulate simple motion, such as a car moving along a road.

By using an array of tiny square lenses instead of the linear ones, such displays can also be made to change as you change the viewing angle up or down - making a "4-D" image. This reveals different views with horizontal as well as vertical movement of the viewer. The new "lighting aware" system adds additional layers of lenses and screens to add two more dimensions of change. The image that is seen is then not only based on the position of the viewer, but also on the direction of the illumination.

In an initial test of the principle, Raskar and his team created an image of a glass wine bottle, whose caustic (a term for the collection of light rays coming from a curved surface), shadows and highlights change with the illumination.

"Even if you have the best hologram out there," explains Raskar, when the angle of the light changes "if I have a hologram of a flower, and a real flower next to it, the hologram doesn't look real. All the shadows and all the reflections on that flower are not mimicked on that hologram."

The new system, still in a relatively low-resolution laboratory proof-of-concept, could have applications including pictures used for training purposes, he said. In training someone how to carry out industrial inspections, an image of the device to be inspected would respond just like a real object when the inspector shines lights on it from different angles, for example.

Because the system is being built by hand from custom-made parts, Raskar says, the present version costs about $30 per pixel to make. Since it takes thousands of pixels to create a recognizable image, practical devices at an affordable price will require significant further development. "It will be at least 10 years before we have any realistic practical-sized displays," he estimates.

The main applications ultimately would be for advertising and for entertainment, Raskar says. A similar system could even be adapted to produce motion pictures and moving computer displays as well, he says.

The research was done in collaboration with Martin Fuchs, Hans-Peter Seidel, and Hendrik P.A. Lensch, all of MPI Informatik, The work was partly funded by Mitsubishi Electric Research Laboratories.

Provided by MIT

Explore further: User-friendly electronic 'EyeCane' enhances navigational abilities for the blind

add to favorites email to friend print save as pdf

Related Stories

Money grows on trees with great walnuts of China

16 hours ago

Grinning with pride, a Chinese farmer held out two precious walnuts—globes so precisely symmetrical that consumers in search of hand massages value them more highly than gold.

Do we want an augmented reality or a transformed reality?

10 hours ago

It seems we are headed towards a world where augmented reality (AR) systems will be as common as smartphones are today – it's already about to revolutionise medicine, entertainment, the lives of disabled peop ...

NASA sees a weaker Typhoon Vongfong near Amami Oshima

9 hours ago

The once-powerful Category 5 Typhoon Vongfong has fortunately weakened to a barely Category 1 typhoon as it approaches the big islands of Japan. NASA-NOAA's Suomi NPP satellite and NASA's Aqua satellite passed ...

Recommended for you

When emotions control objects

16 hours ago

Dimming a light, immersive playing on a computer, and tracking yoga exercises in real time – sensors developed by SmartCardia use various vital signs to transmit data to a host of everyday objects.

Tailored 'activity coaching' by smartphone

Oct 17, 2014

Today's smartphone user can obtain a lot of data about his or her health, thanks to built-in or separate sensors. Researcher Harm op den Akker of the University of Twente (CTIT Institute) now takes this health ...

WASP has printer, will travel, to make houses

Oct 16, 2014

At Maker Faire Rome, an Italian 3D printer company is demonstrating a tall, portable machine that will bring 3D-printed dwellings to impoverished countries. WASP has been exploring low-cost solutions to ...

User comments : 4

Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Aug 08, 2008
why not take it to the final level.. instead of linear, they make it a square. Instead of a square, make it a circle. That way, any direction you look you always get the best possible image. I've thought this since the first time I saw a hologram. It was never invented until now? How odd.
not rated yet Aug 08, 2008
^interesting preposition. but, what about the spaces in between the circular lenses? wont those "empty spaces" distort the final image, at least to some degree?
not rated yet Aug 08, 2008
This idea has been popping up with tedious regularity as "the next big thing in video" for the past 40 years. The other Big Thing is the "optical transistor" that "might one day" allow computers to operate 1000x faster than today's. Yawn. Scientists with no knowledge of tech history.
not rated yet Aug 08, 2008
There were fantastic computer games 15 years ago, created by just a handful of people. Now, it takes dozens of people, and high powered computers to play them on. What's the difference? Game quality? Well, no, graphics quality, to be precise.

Consumers are just nuts for graphics quality, because the "difference" can be noticed by a 6 year old in 10 seconds. What? You're willing to pay $50 for a new First Person Shooter, when you can buy any of dozens of the top rated games of all time (repackaged economically) for $5?

It's the mindless fascination with the obvious that drives this market. And in this case, they're talking about something that will make a huge, huge difference in First Person Shooters. So, definitely plan on seeing it in the store. Join the 6 year old video game lemmings.