JPEG for the mind: How the brain compresses visual information

February 10, 2011
Ed Connor is the director of the Zanvyl Krieger Mind/Brain Institute at the Johns Hopkins University. Credit: Will Kirk/JHU

Most of us are familiar with the idea of image compression in computers. File extensions like ".jpg" or ".png" signify that millions of pixel values have been compressed into a more efficient format, reducing file size by a factor of 10 or more with little or no apparent change in image quality. The full set of original pixel values would occupy too much space in computer memory and take too long to transmit across networks.

The brain is faced with a similar problem. The images captured by light-sensitive in the retina are on the order of a megapixel. The brain does not have the transmission or to deal with a lifetime of megapixel images. Instead, the brain must select out only the most vital information for understanding the visual world.

In today's online issue of , a Johns Hopkins team led by neuroscientists Ed Connor and Kechen Zhang describes what appears to be the next step in understanding how the brain compresses down to the essentials.

They found that cells in area "V4," a midlevel stage in the primate brain's object vision pathway, are highly selective for image regions containing acute curvature. Experiments by doctoral student Eric Carlson showed that V4 cells are very responsive to sharply curved or angled edges, and much less responsive to flat edges or shallow curves.

To understand how selectivity for acute curvature might help with compression of visual information, co-author Russell Rasquinha (now at University of Toronto) created a computer model of hundreds of V4-like cells, training them on thousands of natural object images. After training, each image evoked responses from a large proportion of the virtual V4 cells -- the opposite of a compressed format. And, somewhat surprisingly, these virtual V4 cells responded mostly to flat edges and shallow curvatures, just the opposite of what was observed for real V4 cells.

The results were quite different when the model was trained to limit the number of virtual V4 cells responding to each image. As this limit on responsive cells was tightened, the selectivity of the cells shifted from shallow to acute curvature. The tightest limit produced an eight-fold decrease in the number of cells responding to each image, comparable to the file size reduction achieved by compressing photographs into the .jpeg format. At this level, the computer model produced the same strong bias toward high curvature observed in the real V4 cells.

Why would focusing on acute curvature regions produce such savings? Because, as the group's analyses showed, high-curvature regions are relatively rare in natural objects, compared to flat and shallow curvature. Responding to rare features rather than common features is automatically economical.

Despite the fact that they are relatively rare, high-curvature regions are very useful for distinguishing and recognizing objects, said Connor, a professor in the Solomon H. Snyder Department of Neuroscience in the School of Medicine, and director of the Zanvyl Krieger Mind/Brain Institute.

"Psychological experiments have shown that subjects can still recognize line drawings of objects when flat edges are erased. But erasing angles and other regions of high curvature makes recognition difficult," he explained

Brain mechanisms such as the V4 coding scheme described by Connor and colleagues help explain why we are all visual geniuses.

"Computers can beat us at math and chess," said Connor, "but they can't match our ability to distinguish, recognize, understand, remember, and manipulate the objects that make up our world." This core human ability depends in part on condensing visual information to a tractable level. For now, at least, the .brain format seems to be the best compression algorithm around.

Explore further: Out of sight, out of mind? Not really

Related Stories

Out of sight, out of mind? Not really

August 23, 2005

By playing a trick on the brain, neuroscientists at MIT's McGovern Institute for Brain Research have discovered one way that humans naturally recognize objects.

Study: How the Brain Makes a Whole out of Parts

January 18, 2006

When a human looks at a number, letter or other shape, neurons in various areas of the brain's visual center respond to different components of that shape, almost instantaneously fitting them together like a puzzle to create ...

Shape encoding may start in the retina

September 12, 2007

New evidence from the University of Southern California suggests that there may be dedicated cells in the retina that help compile small bits of information in order to recognize objects. The research was conducted by Ernest ...

Researcher discovers brain cells have 'memory'

April 2, 2009

As we look at the world around us, images flicker into our brains like so many disparate pixels on a computer screen that change every time our eyes move, which is several times a second. Yet we don't perceive the world as ...

Finding our color center

November 1, 2010

The colorful Australian film Strictly Ballroom has been used in a breakthrough scientific experiment to locate the colour processing center in the human brain.

Important brain area organized by color and orientation

November 16, 2010

A brain area known to play a critical role in vision is divided into compartments that respond separately to different colors and orientations, Vanderbilt University researchers have discovered. The findings have important ...

Recommended for you

How the finch changes its tune

August 3, 2015

Like top musicians, songbirds train from a young age to weed out errors and trim variability from their songs, ultimately becoming consistent and reliable performers. But as with human musicians, even the best are not machines. ...

Cow embryos reveal new type of chromosome chimera

May 27, 2016

I've often wondered what happens between the time an egg is fertilized and the time the ball of cells that it becomes nestles into the uterine lining. It's a period that we know very little about, a black box of developmental ...

Shaving time to test antidotes for nerve agents

February 29, 2016

Imagine you wanted to know how much energy it took to bike up a mountain, but couldn't finish the ride to the peak yourself. So, to get the total energy required, you and a team of friends strap energy meters to your bikes ...


Adjust slider to filter visible comments by rank

Display comments: newest first

5 / 5 (3) Feb 10, 2011
"Computers can beat us at math and chess," said Connor, "but they can't match our ability to distinguish, recognize, understand, remember, and manipulate the objects that make up our world."

The computers are only as smart as the humans that program them. Just give it time.
1 / 5 (1) Feb 10, 2011
Since humans program them now, and they "can't match our ability to distinguish, recognize, understand, remember, and manipulate the objects that make up our world." there must be something other than how "smart" one is to compressing visual information in the brain.
2 / 5 (8) Feb 10, 2011
When computers acquire human consciousness, intuition, instinct, and aesthetic sensibility, they will win all
the marbles.
1.5 / 5 (8) Feb 10, 2011
Why does the researcher in the photo look like you just called him a big pussy?
1 / 5 (1) Feb 11, 2011
When computers acquire human consciousness, intuition, instinct, and aesthetic sensibility, they will win all the marbles.

When that happens I want my consciousness to be transferred into the synthetic body they make for it. Then it can augment my already awesome intellect.

Did I mention my other amazing trait, humility?
not rated yet Feb 14, 2011
Unlike perceptually encoded images, Portable Network Graphics (.PNG files) are, in fact, lossless.

No visual information is lost in the compression.

It isn't merely "little or no apparent change in image quality", it offers a file-size shrinking method with no change in image quality.
not rated yet Feb 23, 2011
@Telekinetic ... because his computer DID just beat him a chess! :-)
not rated yet Feb 23, 2011

one thing that may be more of a challenge for 'computer consciousnesses' is empathy. It seems to be excessively challenging for all too many humans as it is. Perhaps that will hinge mostly on what/who "they" the non-biologicals, regard as their equals. Food for thought there.

BTW, I just finished reading "Skinned" by Robin Wassermann. My daughter borrowed it from our local library's 'Young Adult' section, and it's a very sophisticated story for persons of any age I think. I covers the process you hope the machines will do for you. In a nutshell - in my opinion - the key concept is reincarnation; no matter what the subjective beliefs and feelings or the new entity, he will be your [momentarily identical] twin brother.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.