Mathematics and fine art: Digitizing paintings through image processing

September 25, 2012

The current trend to digitize everything is not lost on fine art. Documenting, distributing, conserving, storing and restoring paintings require that digital copies be made. The Google Art Project, which brings art from galleries around the world to online audiences, was launched in early 2011 for precisely these reasons. Google's project has been a complex undertaking, however, carried out under carefully controlled settings using state-of-the-art equipment and requiring rigorous postproduction work.

In a paper published this month in the SIAM Journal on Imaging Sciences, authors Gloria Haro, Antoni Buades and Jean-Michel Morel propose a far simpler technique that can achieve reliable reproductions of paintings using fusion of photographs taken from different angles through statistical methods. The simple photographic procedure eliminates the need for sophisticated and acquisition requirements. The postproduction process, while intensive, is fully automated.

The fusion of multiple images of a painting from well-chosen angles can eliminate glare, highlights and motion blur. Robust statistical methods reduce noise and compensate for optical distortion, thus addressing the problem of uncontrolled illumination and destructive reflection that tends to be seen in many digitized paintings.

One of the main advantages of the method described is that image fusion obviates the need for a high-performance camera. "This article demonstrates the possibility of acquiring a good quality image of a painting from amateur snapshots taken in bursts from different angles, in normal museum illumination," says senior author Jean-Michel Morel. "The photographing procedure is simple and can be done with a commercial hand-held camera by an amateur photographer." Thus, paintings can be digitized even under poor light conditions, and this includes museum pieces that may be protected by glass screens that reflect light from other objects in the room.

The only requirement is for the photographer to take as many shots from as many angles as possible, making sure that no particular detail is affected by glare in a majority of the photographs. "This acquisition is then followed by an intensive (but fully automatic) post-production chain, whose mathematical and algorithmic definition is precisely the object of the article," author Morel explains.

First, each series of images taken from the same position at the same instant—termed "bursts"—is fused to get single views with little noise and motion blur. A weighted average is used to obtain the sharpest possible image. Fusion of multiple images taken from the same position allows removal of noise and compression artifacts, while permitting emphasis of details. This is called burst denoising.

Highlights—regions of an image where extreme brightness causes information to be lost—can also be removed using photographs taken from various viewpoints. Finally, Poisson editing, a technique which allows seamless blending of images, and robust gradient statistics are used to fuse all final views from each burst. This brings images taken from different angles containing many reflections, highlights and shadows, into one final frontal-view image.

Fusion of images also does away with the need for controlled illumination, which is absolutely necessary in the case of single photograph techniques as used in the Project. The authors point out that with a single photograph it is much harder to eliminate glare and reflectance noise, which they find evident in white saturated regions caused by reflections in some photographs in Google's database. The chain described in this paper takes the best advantage of a group of photographs and can work with as few as three. "The image fusion algorithm described in the paper can be applied to a video of the painting as well, though it would be very slow. An extension of the technique to video therefore requires a technological effort," Morel explains.

Likening photography of a painting to viewing it at a museum (several walkarounds are needed to find an optimal viewing position), the authors assert that a quality image of a painting with a complete view cannot be obtained by a single photograph, or even multiple photographs taken from the same position, regardless of the capabilities of a camera.

Once an image clear of imperfections caused by setup and illumination is obtained, color mapping, contrast, and other subjective features are left to photographers and artists. As the authors explain, "There is no absolute definition of a "good" photograph of a painting. This is an artistic question that goes beyond the scope of image processing as a technique."

The methods described in the paper could potentially be used in image restoration. In addition, these methods can be applied to 3-D objects, though the process is slow and success cannot be guaranteed. Future research will be directed toward better digitization of three-dimensional art using these techniques. "The digital recreation of 3D pieces of art is the subject of very active research to resolve its obviously more complex issues," Morel says.

This paper describes just one of many uses for the highly versatile field of image processing. View a video overview of image processing from SIAM's Annual Meeting:

The video will load shortly

Explore further: Team creates computerized method for matching images in photos, paintings, sketches

More information:

Related Stories

Google Art Project premieres upgrades in Chicago

April 3, 2012

(AP) -- High technology merged with high culture Tuesday at The Art Institute of Chicago when Google Inc. announced an upgrade to its Google Art Project initiative, adding thousands of works in dozens more countries.

Imaging method for eye disease used to eye art forgeries

February 3, 2010

Scientists in Poland are describing how a medical imaging technique has taken on a second life in revealing forgery of an artist's signature and changes in inscriptions on paintings that are hundreds of years old. A report ...

Recommended for you

Science: Public interest high, literacy stable

October 28, 2016

While public interest in science continues to grow, the level of U.S. scientific literacy remains largely unchanged, according to a survey by the University of Michigan Institute for Social Research.

Experts uncover hidden layers of Jesus' tomb site

October 27, 2016

In the innermost chamber of the site said to be the tomb of Jesus, a restoration team has peeled away a marble layer for the first time in centuries in an effort to reach what it believes is the original rock surface where ...

Important ancient papyrus seized from looters in Israel

October 27, 2016

(—Eitan Klein, a representative of the Israel Antiquities Authority, has announced that an important papyrus document dated to 2,700 years ago has been seized from a group of Palestinian looters who reportedly ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.