Mathematics and fine art: Digitizing paintings through image processing

Sep 25, 2012

The current trend to digitize everything is not lost on fine art. Documenting, distributing, conserving, storing and restoring paintings require that digital copies be made. The Google Art Project, which brings art from galleries around the world to online audiences, was launched in early 2011 for precisely these reasons. Google's project has been a complex undertaking, however, carried out under carefully controlled settings using state-of-the-art equipment and requiring rigorous postproduction work.

In a paper published this month in the SIAM Journal on Imaging Sciences, authors Gloria Haro, Antoni Buades and Jean-Michel Morel propose a far simpler technique that can achieve reliable reproductions of paintings using fusion of photographs taken from different angles through statistical methods. The simple photographic procedure eliminates the need for sophisticated and acquisition requirements. The postproduction process, while intensive, is fully automated.

The fusion of multiple images of a painting from well-chosen angles can eliminate glare, highlights and motion blur. Robust statistical methods reduce noise and compensate for optical distortion, thus addressing the problem of uncontrolled illumination and destructive reflection that tends to be seen in many digitized paintings.

One of the main advantages of the method described is that image fusion obviates the need for a high-performance camera. "This article demonstrates the possibility of acquiring a good quality image of a painting from amateur snapshots taken in bursts from different angles, in normal museum illumination," says senior author Jean-Michel Morel. "The photographing procedure is simple and can be done with a commercial hand-held camera by an amateur photographer." Thus, paintings can be digitized even under poor light conditions, and this includes museum pieces that may be protected by glass screens that reflect light from other objects in the room.

The only requirement is for the photographer to take as many shots from as many angles as possible, making sure that no particular detail is affected by glare in a majority of the photographs. "This acquisition is then followed by an intensive (but fully automatic) post-production chain, whose mathematical and algorithmic definition is precisely the object of the article," author Morel explains.

First, each series of images taken from the same position at the same instant—termed "bursts"—is fused to get single views with little noise and motion blur. A weighted average is used to obtain the sharpest possible image. Fusion of multiple images taken from the same position allows removal of noise and compression artifacts, while permitting emphasis of details. This is called burst denoising.

Highlights—regions of an image where extreme brightness causes information to be lost—can also be removed using photographs taken from various viewpoints. Finally, Poisson editing, a technique which allows seamless blending of images, and robust gradient statistics are used to fuse all final views from each burst. This brings images taken from different angles containing many reflections, highlights and shadows, into one final frontal-view image.

Fusion of images also does away with the need for controlled illumination, which is absolutely necessary in the case of single photograph techniques as used in the Project. The authors point out that with a single photograph it is much harder to eliminate glare and reflectance noise, which they find evident in white saturated regions caused by reflections in some photographs in Google's database. The chain described in this paper takes the best advantage of a group of photographs and can work with as few as three. "The image fusion algorithm described in the paper can be applied to a video of the painting as well, though it would be very slow. An extension of the technique to video therefore requires a technological effort," Morel explains.

Likening photography of a painting to viewing it at a museum (several walkarounds are needed to find an optimal viewing position), the authors assert that a quality image of a painting with a complete view cannot be obtained by a single photograph, or even multiple photographs taken from the same position, regardless of the capabilities of a camera.

Once an image clear of imperfections caused by setup and illumination is obtained, color mapping, contrast, and other subjective features are left to photographers and artists. As the authors explain, "There is no absolute definition of a "good" photograph of a painting. This is an artistic question that goes beyond the scope of image processing as a technique."

The methods described in the paper could potentially be used in image restoration. In addition, these methods can be applied to 3-D objects, though the process is slow and success cannot be guaranteed. Future research will be directed toward better digitization of three-dimensional art using these techniques. "The digital recreation of 3D pieces of art is the subject of very active research to resolve its obviously more complex issues," Morel says.

This paper describes just one of many uses for the highly versatile field of image processing. View a video overview of image processing from SIAM's Annual Meeting:

This video is not supported by your browser at this time.


Explore further: Professor quantifies how 'one thing leads to another'

More information: epubs.siam.org/doi/abs/10.1137/120873923

add to favorites email to friend print save as pdf

Related Stories

Google Art Project premieres upgrades in Chicago

Apr 03, 2012

(AP) -- High technology merged with high culture Tuesday at The Art Institute of Chicago when Google Inc. announced an upgrade to its Google Art Project initiative, adding thousands of works in dozens more countries.

Imaging method for eye disease used to eye art forgeries

Feb 03, 2010

Scientists in Poland are describing how a medical imaging technique has taken on a second life in revealing forgery of an artist's signature and changes in inscriptions on paintings that are hundreds of years ...

Recommended for you

Fossil arthropod went on the hunt for its prey

9 hours ago

A new species of carnivorous crustacean has been identified, which roamed the seas 435 million years ago, grasping its prey with spiny limbs before devouring it. The fossil is described and details of its lifestyle are published ...

User comments : 0