UIUC team will show can't-tell photo inserts at Siggraph (w/ video)

Dec 08, 2011 by Nancy Owano report
Image credit: Kevin Karsch

(PhysOrg.com) -- Visitors to this month's Siggraph Asia conference on computer graphics from December 12 to 15 will witness a presentation from a team at the University of Illinois in Urbana Champaign on how to tweak photos by adding in something that was not there before. They will present their study, Rendering Synthetic Objects into Legacy Photographs, which details their approach.

So what? What could possibly be new about this? Their method has more going for it than older techniques used by the Kremlin or budding Photoshop enthusiasts. The team, Kevin Karsch, Varsha Hedau, David Forsyth, Derek Hoiem, can simulate so that the object looks realistic.

Humans can quickly detect photo fraud, maintains Karsch. They can do so in spotting lighting inconsistencies in a doctored photograph.

In contrast, the university team’s method, he says, is successfully confusable even for people who pride themselves in spotting differences.

If you don’t know the perspective, if you don’t know the geometry of an object, then you are just manipulating pixels, he commented, with unconvincing results.

In their computer program, a user is asked to select light sources in the picture. An algorithm recreates the 3-D geometry and lighting of the scene and the artificial object is inserted into its new environment. The program adds shadows and highlights to the object before converting it back to 2-D.

The weakness in existing photo editing programs, they say, is that they simply insert a 2-D object. Karsch, a computer science doctoral student whose advisor is David Forsyth, explains that image editing software that only allows 2-D manipulations does not account for high-level spatial information that is present in a given scene, yet 3-D modeling tools may be complex and tedious for novice users.

The team set out to extract the 3-D scene information from single images, to allow for seamless object insertion, removal, and relocation.

The process involves three phases: luminaire inference, perspective estimation (depth, occlusion, camera parameters), and texture replacement. The team, in their paper, says their method can realistically insert synthetic objects into existing without requiring access to the scene or any additional scene measurements.

“With a single image and a small amount of annotation, our method creates a physical model of the scene that is suitable for realistically rendering synthetic objects with diffuse, specular, and even glowing materials while accounting for lighting interactions between the objects and the scene.”

Potentially useful applications include interior design, where decorators might take a photo of a room and experiment with different furniture and object insertions. Other possibilities include entertainment and gaming.

Explore further: Artificial intelligence that imitates children's learning

More information: kevinkarsch.com/publications/sa11.html

Related Stories

Faster computer graphics

Jun 13, 2011

Photographs of moving objects are almost always a little blurry — or a lot blurry, if the objects are moving rapidly enough. To make their work look as much like conventional film as possible, game and ...

Extreme makeover: computer science edition

Nov 12, 2008

(PhysOrg.com) -- Suppose you have a cherished home video, taken at your birthday party. You're fond of the video, but your viewing experience is marred by one small, troubling detail. There in the video, framed ...

Laser-based camera can see around corners

Nov 17, 2010

(PhysOrg.com) -- Researchers from MIT have developed a camera that can capture images of a scene that is not in its direct line of sight. The camera is equipped with a femtosecond laser, which fires extremely ...

Oscar Worthy Science And Engineering

Feb 18, 2010

When audiences watch a movie, they know that what they are seeing is an illusion -- and making the images appear as real as possible can be a major undertaking for any filmmaking team.

Recommended for you

Artificial intelligence that imitates children's learning

13 hours ago

The computer programmes used in the field of artificial intelligence (AI) are highly specialised. They can for example fly airplanes, play chess or assemble cars in controlled industrial environments. However, a research ...

Oculus unveils new prototype VR headset

Sep 20, 2014

Oculus has unveiled a new prototype of its virtual reality headset. However, the VR company still isn't ready to release a consumer edition.

Who drives Alibaba's Taobao traffic—buyers or sellers?

Sep 18, 2014

As Chinese e-commerce firm Alibaba prepares for what could be the biggest IPO in history, University of Michigan professor Puneet Manchanda dug into its Taobao website data to help solve a lingering chicken-and-egg question.

Computerized emotion detector

Sep 16, 2014

Face recognition software measures various parameters in a mug shot, such as the distance between the person's eyes, the height from lip to top of their nose and various other metrics and then compares it with photos of people ...

User comments : 10

Adjust slider to filter visible comments by rank

Display comments: newest first

that_guy
1 / 5 (3) Dec 08, 2011
umm yeah. You can still tell it has been inserted. It is very effective and clever - but it is not better than a great photoshop artist...it's on par with a seasoned professional.

This is an awesome tool to use in conjunction with 3d modelling or photoshop, as it takes out 90 percent of the work, but to say it's out and out better is purely mental masterbation by the students who created it.
0FET
2.1 / 5 (7) Dec 08, 2011
"THIS LOOKS SHOPPED / I CAN TELL FROM SOME OF THE PIXELS AND FROM SEEING QUITE A FEW SHOPS IN MY TIME.
Skepticus
4.3 / 5 (6) Dec 08, 2011
Excellent. Another improvement in our ability to doctor ANY evidence that is not favorable to us, or to cook up convincingly evidences to nail our enemies to the nearest barn door!
expert
4 / 5 (4) Dec 08, 2011
Excellent. Another improvement in our ability to doctor ANY evidence that is not favorable to us, or to cook up convincingly evidences to nail our enemies to the nearest barn door!


Yeah, just in time for invasion of Iran :)
Deesky
5 / 5 (1) Dec 08, 2011
It's a nice tool for non-professionals. Professionals do take similar things into consideration when compositing images to make the scene look realistic.

However, the big limitation of this technique is that it can only deal with synthetic (generated) objects, rather than other picture sources of say people, animals, etc.
Baseline
5 / 5 (4) Dec 08, 2011
"THIS LOOKS SHOPPED / I CAN TELL FROM SOME OF THE PIXELS AND FROM SEEING QUITE A FEW SHOPS IN MY TIME.


It's unfortunate that when you were seeing all those "shops" in your time you didn't happen to notice the Caps lock key...
Tausch
1 / 5 (2) Dec 09, 2011
A 'Turing' test for visuals.
No amount of observation will distinguished between reality and synthesized - the brain is limited to the resolution of inputs the senses are able to resolve.
Eventually all senses will have indistinguishable 'perfection'.

Such a digital representation will serve any effort to alleviate any deficiency any sense exhibits. (Misuse excluded)
tadchem
1 / 5 (2) Dec 09, 2011
There are still bugs. In all three examples provided, the shadows cast on the objects appear roughly correct, but the shadows cast BY the inserted objects are obviously flawed. The billiard balls, for example, cast their shadows in disparate directions as if each has an independent light source. The statues in the other two examples do not seem to properly block the direct light that illuminates them. The 'sunbeams' on the floor show no signs of interception by the statues.
When I showed them to a photointerpreter friend, she laughed. I have seen better photo insertions by amateurs on Worth1000.
Neophile
not rated yet Dec 09, 2011
"The billiard balls, for example, cast their shadows in disparate directions as if each has an independent light source." A point source near the objects will do just that (like the overhead lamp for instance).
Zander
not rated yet Dec 10, 2011
"THIS LOOKS SHOPPED / I CAN TELL FROM SOME OF THE PIXELS AND FROM SEEING QUITE A FEW SHOPS IN MY TIME.


It's unfortunate that when you were seeing all those "shops" in your time you didn't happen to notice the Caps lock key...


He was in fact quoting a meme.

http://knowyourme...-shopped