Seeing two figures in coordinated action helps brain pick out movements of one

September 7, 2006
Seeing two figures in coordinated action helps brain pick out movements of one
Researchers used colored dots to represent the movement of dancers (above) and fighters in a pair of videos illustrating visual discrimination.

A new study by vision scientists at the University of California, Berkeley, finds that the human visual system is better able to discriminate the movements of a single person when his or her actions are coordinated in a meaningful way with a second individual.

When shown a pair of figures in motion, for instance, the brain is better able to pick out an individual if it perceives that one person is throwing a punch while another is making a defensive block, the researchers said.

This is especially important when the view is somehow obscured or impaired, according to the study, published in the September issue of the journal Nature Neuroscience.

"Our study reveals a greater degree of complexity in human visual processing than previously understood," said Dennis Levi, UC Berkeley dean of optometry and principal investigator of the study. "When we watch two people interacting, knowing what one is doing helps us understand the actions of the other. We think of it as 'It takes two to Tango.'"

Other researchers on the team are Peter Neri, a UC Berkeley post-doctoral fellow in Levi's laboratory, and Jennifer Luu, an undergraduate who joined the group as part of the UC Berkeley Undergraduate Research Apprentice Program. Luu is now a UC Berkeley Ph.D. student in bioengineering.

This research provides insight into how accurately we can interpret what we see from grainy security cameras, particularly when identifying whether a crime is taking place. There is even research taking place on the development of artificial intelligence computer programs that can automatically detect which actions are suspicious.

At its most fundamental level, the human brain's ability to interpret and react to the action it sees within a fraction of a second is a matter of life and death, such as identifying that a tiger lunging forward with teeth bared is a threat and then deciding how best to get away.

But this skill - when not impaired by such disorders as autism - is also essential to our success as a social species, allowing us to determine whether someone is happy, sad or nervous based upon visual cues both subtle and obvious.

"Our study is the first to provide quantitative evidence that this ability to interpret action enhances the processing of visual discrimination," said Neri, the lead author.

To build their dataset of human actions, the researchers outfitted members of the UC Martial Arts Program and the UC Ballroom Dance Team with sports suits that each contained 13 lights marking key body parts, such as the head, hands, knees and feet. The "actors" were then asked to fight or dance in a dark room while the researchers filmed the movements.

Using motion capture technology, the lights in the video were converted into dots that the researchers could manipulate. The original samples of each action - fighting or dancing - lasted 22 seconds each. In the synchronized video files, figures' actions corresponded naturally with each other.

The researchers also created desynchronized files in which the movements of one figure filmed in the first 11 seconds were paired with those for the other figure in the second 11 seconds. In these files, the movements of the two figures were no longer coordinated with each other.

Four UC Berkeley undergraduate students with normal vision who had been recruited for the study were then shown 1.5- to 3-second sequences that were randomly selected from either the synchronized or desynchronized files. In half of the sequences, the researchers had further scrambled the dots of one figure and added many other dots to the sequence to mask the action. The researchers were able to vary the amount of dots to measure participants' tolerance for "noise" or "fog."

The participants had to determine whether the dots they were viewing represented one or two figures.

"We found that people can tolerate more masking dots and answer correctly when the figures' actions are interacting in a meaningful way," said Neri, who also participated in the study. "If the sequence is desynchronized, we can tolerate 10 dots. When viewing synchronized action, we can tolerate a minimum of 20 and up to 100 dots. Having the action coordinated improves the performance by at least a factor of two."

The researchers noted that the relationship between two figures held true whether the movements were coordinated but antagonistic, as in the fighting sequences, or coordinated and cooperative, as in the dancing movements.

"This study shows that the cortex encodes complex information about actions, and that there can be a clear evolutionary advantage - improved visual discrimination - to doing so," said Neri.

The researchers pointed out that the human brain is constantly adapting to limited amounts of visual information, enabling us to imagine a full picture of a tiger, for instance, even if trees and bushes obscure parts of its body.

The brain also allows us to understand when a group of dots or lines represents a human being. When presented with a group of dots in motion, the human visual system can distinguish whether the dots comprise a male or a female, and even judge the figure's emotional state, the researchers said.

"We used to talk about how the eye is a camera that passes information intact to the brain, but that's actually not how we see," said Levi. "Things are always changing before our eyes, and our brain is constantly making best guesses about what it's seeing."

When the view is somehow obscured, that task naturally becomes more difficult. This guessing game played by the visual system is also why the brain is susceptible to optical illusions, Levi added. This best-guess approach to interpreting what we see is also an important factor when considering eyewitness accounts of events, the researchers said.

Source: UC Berkeley, By Sarah Yang

Explore further: Ray-tracing software lets researchers visualize science with greater fidelity

Related Stories

What neuroscience can learn from computer science

August 10, 2015

What do computers and brains have in common? Computers are made to solve the same problems that brains solve. Computers, however, rely on a drastically different hardware, which makes them good at different kinds of problem ...

'Bathtub rings' suggest Titan's dynamic seas

July 28, 2015

Saturn's moon, Titan, is the only object in the Solar System other than Earth known to have liquid on its surface. While most of the lakes are found around the poles, the dry regions near the equator contain signs of evaporated ...

Exposing breast cancer using nanoscale polymers

May 13, 2015

Photoacoustic imaging is a ground-breaking technique for spotting tumors inside living cells with the help of light-absorbing compounds known as contrast agents. A*STAR researchers have now discovered a way to improve the ...

Recommended for you

New Horizons team selects potential Kuiper Belt flyby target

August 29, 2015

NASA has selected the potential next destination for the New Horizons mission to visit after its historic July 14 flyby of the Pluto system. The destination is a small Kuiper Belt object (KBO) known as 2014 MU69 that orbits ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.