Analyzing pixel correlations in photographs improves image analysis

May 21, 2014, Agency for Science, Technology and Research (A*STAR), Singapore
Analyzing pixel correlations in photographs improves image analysis
Saliency map (right) of an image (left), which illustrates how a computer model is able to identify salient information such as the high visibility vest from an image based on statistical analysis. Credit: A*STAR Institute for Infocomm Research

A visual saliency technique that can detect and extract relevant information from both still and moving images has many applications for computer image processing. Such a technique can be used to detect motion, distinguish different objects and improve the quality of specific parts of an image through selective compression.

Shijian Lu and colleagues from the A*STAR Institute for Infocomm Research in Singapore have developed a robust and efficient method for capturing such salient information from images and movies. They found that the key lies in the distribution of brightness and color between pairs of pixels.

Digital images are encoded as pixels, or points in an image. To detect an object (for example, a person standing in the foreground), brightness variations between neighboring pixels could be compared. However, considering just individual pixels can be deceiving, as the context is important when seeking to distinguish between important details and unimportant background information.

The technique developed by the A*STAR researchers hence involves counting the pixels in an image based on their color and then plotting the distribution. This makes not only the distribution of colors apparent, but also the frequency at which pairs or neighboring appear. A low frequency of pixel pairs with a certain difference in color indicates a region of high interest, as it denotes clear boundaries between objects. In this way, the salient features can be easily identified; for example, not only large areas of contrast in a photograph, such as a yellow school bus in front of a neutral background, but also contrasts in smaller areas, such as a person wearing a safety vest (see image).

"Our model has great potential for predicting the point in an image that will attract the human eye," comments Lu. "Apart from generic object detection, it can be applied to tasks such as guiding robots or to the smart design of web pages and advertisements."

The next step for the researchers will be to apply this scheme to detecting motion in videos, which follows similar rules as identifying relevant information in still photographs. Moreover, Lu says that their algorithm enables more complex approaches to image analysis.

"An example is the development of computational modeling of the top-down approach of humans looking at a scene," says Lu. "Combining our bottom-up modeling algorithm with a top-down visual search could solve many challenging computer vision problems—such as anomaly detection or target search—in a more robust, efficient and cognitive manner."

Explore further: Mathematically correcting over- and underexposure in photographs

More information: Lu, S., Tan, C. & Lim, J.-H. "Robust and efficient saliency modeling from image co-occurrence histograms." IEEE Transactions on Pattern Analysis and Machine Intelligence 36, 195–201 (2014).

Related Stories

Pixels guide the way for the visually impaired

February 28, 2013

(—Images have been transformed into pixels and projected onto a headset to help the visually impaired in everyday tasks such as navigation, route-planning and object finding.

Filtering web images effectively

June 25, 2012

You’re looking for a photo of a flower. Not just any photo—it needs to be horizontal in shape. And not just any flower—it needs to be a purple flower.

New method to magnify digital images is 700 times faster

June 21, 2013

Aránzazu Jurío-Munárriz, a graduate in computer engineering from the NUP/UPNA-Public University of Navarre, has in her PhD thesis presented new methods for improving two of the most widespread means used in digital image ...

Recommended for you

Cryptocurrency rivals snap at Bitcoin's heels

January 14, 2018

Bitcoin may be the most famous cryptocurrency but, despite a dizzying rise, it's not the most lucrative one and far from alone in a universe that counts 1,400 rivals, and counting.

Top takeaways from Consumers Electronics Show

January 13, 2018

The 2018 Consumer Electronics Show, which concluded Friday in Las Vegas, drew some 4,000 exhibitors from dozens of countries and more than 170,000 attendees, showcased some of the latest from the technology world.

Finnish firm detects new Intel security flaw

January 12, 2018

A new security flaw has been found in Intel hardware which could enable hackers to access corporate laptops remotely, Finnish cybersecurity specialist F-Secure said on Friday.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.