Mathematics and fine art: Digitizing paintings through image processing

September 25, 2012, Society for Industrial and Applied Mathematics

The current trend to digitize everything is not lost on fine art. Documenting, distributing, conserving, storing and restoring paintings require that digital copies be made. The Google Art Project, which brings art from galleries around the world to online audiences, was launched in early 2011 for precisely these reasons. Google's project has been a complex undertaking, however, carried out under carefully controlled settings using state-of-the-art equipment and requiring rigorous postproduction work.

In a paper published this month in the SIAM Journal on Imaging Sciences, authors Gloria Haro, Antoni Buades and Jean-Michel Morel propose a far simpler technique that can achieve reliable reproductions of paintings using fusion of photographs taken from different angles through statistical methods. The simple photographic procedure eliminates the need for sophisticated and acquisition requirements. The postproduction process, while intensive, is fully automated.

The fusion of multiple images of a painting from well-chosen angles can eliminate glare, highlights and motion blur. Robust statistical methods reduce noise and compensate for optical distortion, thus addressing the problem of uncontrolled illumination and destructive reflection that tends to be seen in many digitized paintings.

One of the main advantages of the method described is that image fusion obviates the need for a high-performance camera. "This article demonstrates the possibility of acquiring a good quality image of a painting from amateur snapshots taken in bursts from different angles, in normal museum illumination," says senior author Jean-Michel Morel. "The photographing procedure is simple and can be done with a commercial hand-held camera by an amateur photographer." Thus, paintings can be digitized even under poor light conditions, and this includes museum pieces that may be protected by glass screens that reflect light from other objects in the room.

The only requirement is for the photographer to take as many shots from as many angles as possible, making sure that no particular detail is affected by glare in a majority of the photographs. "This acquisition is then followed by an intensive (but fully automatic) post-production chain, whose mathematical and algorithmic definition is precisely the object of the article," author Morel explains.

First, each series of images taken from the same position at the same instant—termed "bursts"—is fused to get single views with little noise and motion blur. A weighted average is used to obtain the sharpest possible image. Fusion of multiple images taken from the same position allows removal of noise and compression artifacts, while permitting emphasis of details. This is called burst denoising.

Highlights—regions of an image where extreme brightness causes information to be lost—can also be removed using photographs taken from various viewpoints. Finally, Poisson editing, a technique which allows seamless blending of images, and robust gradient statistics are used to fuse all final views from each burst. This brings images taken from different angles containing many reflections, highlights and shadows, into one final frontal-view image.

Fusion of images also does away with the need for controlled illumination, which is absolutely necessary in the case of single photograph techniques as used in the Project. The authors point out that with a single photograph it is much harder to eliminate glare and reflectance noise, which they find evident in white saturated regions caused by reflections in some photographs in Google's database. The chain described in this paper takes the best advantage of a group of photographs and can work with as few as three. "The image fusion algorithm described in the paper can be applied to a video of the painting as well, though it would be very slow. An extension of the technique to video therefore requires a technological effort," Morel explains.

Likening photography of a painting to viewing it at a museum (several walkarounds are needed to find an optimal viewing position), the authors assert that a quality image of a painting with a complete view cannot be obtained by a single photograph, or even multiple photographs taken from the same position, regardless of the capabilities of a camera.

Once an image clear of imperfections caused by setup and illumination is obtained, color mapping, contrast, and other subjective features are left to photographers and artists. As the authors explain, "There is no absolute definition of a "good" photograph of a painting. This is an artistic question that goes beyond the scope of image processing as a technique."

The methods described in the paper could potentially be used in image restoration. In addition, these methods can be applied to 3-D objects, though the process is slow and success cannot be guaranteed. Future research will be directed toward better digitization of three-dimensional art using these techniques. "The digital recreation of 3D pieces of art is the subject of very active research to resolve its obviously more complex issues," Morel says.

This paper describes just one of many uses for the highly versatile field of image processing. View a video overview of image processing from SIAM's Annual Meeting:

Explore further: Team creates computerized method for matching images in photos, paintings, sketches

More information:

Related Stories

Google Art Project premieres upgrades in Chicago

April 3, 2012

(AP) -- High technology merged with high culture Tuesday at The Art Institute of Chicago when Google Inc. announced an upgrade to its Google Art Project initiative, adding thousands of works in dozens more countries.

Imaging method for eye disease used to eye art forgeries

February 3, 2010

Scientists in Poland are describing how a medical imaging technique has taken on a second life in revealing forgery of an artist's signature and changes in inscriptions on paintings that are hundreds of years old. A report ...

Recommended for you

Unprecedented study of Picasso's bronzes uncovers new details

February 17, 2018

Musee national Picasso-Paris and the Northwestern University/Art Institute of Chicago Center for Scientific Studies in the Arts (NU-ACCESS) have completed the first major material survey and study of the Musee national Picasso-Paris' ...

Using Twitter to discover how language changes

February 16, 2018

Scientists at Royal Holloway, University of London, have studied more than 200 million Twitter messages to try and unravel the mystery of how language evolves and spreads.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.