Software can automatically critique composition of digital photographs

August 14, 2015, Pennsylvania State University
Photo of image to be critiqued by software. Credit: Patrick Mansell, Penn State

Everyone may be a critic, but now Penn State researchers are paving a way for machines to get in on the act. However, the researchers add that their photo-analysis algorithm is designed to offer constructive feedback, not to replace photographers.

The researchers have developed an algorithm that analyzes the arrangement of visual elements—the —of digital photographs. It also offers feedback about the perceived composition of the photograph and provides examples of similarly composed pictures of high aesthetic value, said James Wang, professor of information sciences and technology. Wang and colleagues recently received a patent for the system.

"If you think about aesthetics, everything is about composition," said Wang. "You can look into colors, or textures, or shapes, but, if you boil it down, you eventually have to consider all of these elements as part of composition."

Training a machine to become an art critic is not easy, according to the researchers. A machine must be trained with examples of highly regarded photographs in order to recognize good compositional elements, said Jia Li, professor of statistics, who worked with Wang.

Li added that the software the team developed relies on psychological theories of human vision as well as on the opinions of people to help classify images. For example, the researchers considered work done in Gestalt psychology, which focuses on how people perceive shape and form, in the development of the software.

A group of graduate students helped develop the algorithm by manually labeling hundreds of images from a collection of pictures on the website The composition of each picture is categorized into horizontal, vertical, centered, diagonal and textured types. These labels, then, serve as the training dataset that teaches the algorithm how to classify photographs.

"Aesthetics is, of course, subjective," said Li. "What one person might find pleasing may not be pleasing to another, but there are elements that many good photos have in common."

The software performs a pixel-by-pixel analysis to extract features from a photograph and then uses statistical analysis to classify and compare it to highly aesthetic pictures.

Often, photos may have a blend of compositional elements. The software can detect and indicate cases when there may be several compositional options for the photographer.

"For example, a group of people could look at a photograph of, let's say, a canyon, and some would see it as a vertical composition, some people could see it as a diagonal composition, and some might even say it's a horizontal picture," Wang said. "So, sometimes, composition is not well defined."

Wang said he doubts the application will replace photographers any time soon. He hopes, instead, that it creates better photographers.

"Our goal from the beginning was to help ," said Wang. "If you are an amateur photographer then, potentially, a computer can analyze your photograph's composition and help you improve it."

The software can be installed on a server that can be accessed by a mobile phone. The researchers expect that, as smart phones become more powerful, the application will be able to rest on the phone. Eventually, as gain more memory and power, the software would not need to be stored on a server, but could be added to the phone itself.

"It's not even necessary to store very high resolution images for the system to work, so putting the on a is possible," said Li. "The system was developed so that it doesn't consume a lot of memory."

Explore further: Online system rates images by aesthetic quality

Related Stories

Online system rates images by aesthetic quality

May 5, 2009

( -- An online photo-rating system developed at Penn State is the first publicly available tool for automatically determining the aesthetic value of an image, according to a Penn State researcher involved with ...

Flickr photo data used to predict people's locations

August 12, 2015

A team of researchers with University College in England has found a way to use photo information attached to images uploaded to the sharing site Flickr to create an application that can predict where people will be at a ...

Tactile photographs: 'Seeing' by touching

June 12, 2013

( —Arizona State University computer scientist Baoxin Li is demonstrating the artistic side of his technical field by creating computer-generated tactile photographs for people living with visual impairment.

Computer arranges pictures based on their artistic aspects

February 18, 2014

Until now, it has been a time-consuming process for a program to arrange pictures in a consistent order. It is even more complex to order them on the basis of visual characteristics. Computer scientists from Saarbrücken ...

Recommended for you

Scientists solve mystery shrouding oldest animal fossils

March 25, 2019

Scientists from The Australian National University (ANU) have discovered that 558 million-year-old Dickinsonia fossils do not reveal all of the features of the earliest known animals, which potentially had mouths and guts.

Earth's deep mantle flows dynamically

March 25, 2019

As ancient ocean floors plunge over 1,000 km into the Earth's deep interior, they cause hot rock in the lower mantle to flow much more dynamically than previously thought, finds a new UCL-led study.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.