Genius of Einstein, Fourier key to new humanlike computer vision

Jun 20, 2011 by Emil Venere
This graphic illustrates a new computer-vision technology that builds on the basic physics and mathematical equations related to how heat diffuses over surfaces. The technique mimics how humans perceive three-dimensional shapes by instantly recognizing objects no matter how they are twisted or bent, an advance that could help machines see more like people. Here, a "heat mean signature" of a human hand model is used to perceive the six segments of the overall shape and define the fingertips. Credit: Purdue University image/Karthik Ramani and Yi Fang

(PhysOrg.com) -- Two new techniques for computer-vision technology mimic how humans perceive three-dimensional shapes by instantly recognizing objects no matter how they are twisted or bent, an advance that could help machines see more like people.

The techniques, called mapping and heat distribution, apply mathematical methods to enable machines to perceive three-dimensional objects, said Karthik Ramani, Purdue University's Donald W. Feddersen Professor of Mechanical Engineering.

"Humans can easily perceive 3-D shapes, but it's not so easy for a computer," he said. "We can easily separate an object like a hand into its segments - the palm and five fingers - a difficult operation for computers."

Both of the techniques build on the basic physics and related to how heat diffuses over surfaces.

" made contributions to diffusion, and 18th century physicist Jean Baptiste Joseph Fourier developed Fourier's law, used to derive the heat equation," Ramani said. "We are standing on the shoulders of giants in creating the algorithms for these new approaches using the heat equation."

As heat diffuses over a surface it follows and captures the precise contours of a shape. The system takes advantage of this "intelligence of heat," simulating heat flowing from one point to another and in the process characterizing the shape of an object, he said.

Findings will be detailed in two papers being presented during the IEEE Computer Vision and Pattern Recognition conference on June 21-23 in Colorado Springs. The paper was written by Ramani, Purdue doctoral students Yi Fang and Mengtian Sun, and Minhyong Kim, a professor of pure mathematics at the University College London.

A major limitation of existing methods is that they require "prior information" about a shape in order for it to be analyzed.

"For example, in order to do segmentation you have to tell the computer ahead of time how many segments the object has," Ramani said. "You have to tell it that you are expecting, say, 10 segments or 12 segments."

The new methods mimic the human ability to properly perceive objects because they don't require a preconceived idea of how many segments exist.

"We are trying to come as close as possible to human segmentation," Ramani said. "A hot area right now is unsupervised machine learning. This means a machine, such as a robot, can perceive and learn without having any previous training. We are able to estimate the segmentation instead of giving a predefined number of segments."

The work is funded partially by the National Science Foundation. A patent on the technology is pending.

Researchers developing a new machine-vision technique tested their method on certain complex shapes, including the human form or a centaur, a mythical half-human, half-horse creature. The heat mapping allows a computer to recognize the objects no matter how the figures are bent or twisted and is able to ignore "noise" introduced by imperfect laser scanning or other erroneous data. Credit: Purdue University image/Karthik Ramani and Yi Fang

The methods have many potential applications, including a 3-D search engine to find mechanical parts such as automotive components in a database; robot vision and navigation; 3-D medical imaging; military drones; multimedia gaming; creating and manipulating animated characters in film production; helping 3-D cameras to understand human gestures for interactive games; contributing to progress of areas in science and engineering related to pattern recognition; machine learning; and computer vision.

The heat-mapping method works by first breaking an object into a mesh of triangles, the simplest shape that can characterize surfaces, and then calculating the flow of heat over the meshed object. The method does not involve actually tracking heat; it simulates the flow of heat using well-established mathematical principles, Ramani said.

Heat mapping allows a computer to recognize an object, such as a hand or a nose, no matter how the fingers are bent or the nose is deformed and is able to ignore "noise" introduced by imperfect laser scanning or other erroneous data.

"No matter how you move the fingers or deform the palm, a person can still see that it's a hand," Ramani said. "But for a computer to say it's still a hand is going to be hard. You need a framework - a consistent, robust algorithm that will work no matter if you perturb the nose and put noise in it or if it's your nose or mine."

The method accurately simulates how heat flows on the object while revealing its structure and distinguishing unique points needed for segmentation by computing the "heat mean signature." Knowing the heat mean signature allows a computer to determine the center of each segment, assign a "weight" to specific segments and then define the overall shape of the object.

"Being able to assign a weight to segments is critical because certain points are more important than others in terms of understanding a shape," Ramani said. "The tip of the nose is more important than other points on the nose, for example, to properly perceive the shape of the nose or face, and the tips of the fingers are more important than many other points for perceiving a hand."

In temperature distribution, heat flow is used to determine a signature, or histogram, of the entire object.

"A histogram is a two-dimensional mapping of a three-dimensional shape," Ramani said. "So, no matter how a dog bends or twists, it gives you the same signature."

The temperature distribution technique also uses a triangle mesh to perceive 3-D shapes. Both techniques, which could be combined in the same system, require modest computer power and recognize shapes quickly, he said.

"It's very efficient and very compact because you're just using a two-dimensional histogram," Ramani said. "Heat propagation in a mesh happens very fast because the mathematics of matrix computations can be done very quickly and well."

The researchers tested their method on certain complex shapes, including hands, the human form or a centaur, a mythical half-human, half-horse creature.

Explore further: Computerized emotion detector

More information: Heat-Mapping: A Robust Approach Toward Perceptually Consistent Mesh Segmentation

ABSTRACT
3-D mesh segmentation is a fundamental low-level task with applications in areas as diverse as computer vision, computer-aided design, bio-informatics, and 3-D medical imaging. A perceptually consistent mesh segmentation (PCMS), as defined in this paper is one that satisfies 1) invariance to isometric transformation of the underlying surface, 2) robust to the perturbations of the surface, 3) robustness to numerical noise on the surface, and 4) close conformation to human perception. We exploit the intelligence of the heat as a global structure-aware message on a meshed surface and develop a robust PCMS scheme, called Heat-Mapping based on the heat kernel. There are three main steps in Heat-Mapping. First, the number of the segments is estimated based on the analysis of the behavior of the Laplacian spectrum. Second, the heat center, which is defined as the most representative vertex on each segment, is discovered by a proposed heat center hunting algorithm. Third, a heat center driven segmentation scheme reveals showing PCMS with high consistency towards human perception. Extensive experimental results on various types of models verify the performance of Heat-Mapping with respect to the consistent segmentation of articulated bodies, the topological changes, and various levels of numerical noise.

Related Stories

Software promises more efficient design process

Mar 02, 2006

Mechanical engineers at Purdue University have developed software that promises to increase the efficiency of creating parts for everything from cars to computer hardware by making it possible to quickly evaluate and optimize ...

See no shape, touch no shape, hear a shape?

Oct 18, 2010

(PhysOrg.com) -- Scientists at The Montreal Neurological Institute and Hospital – The Neuro, McGill University have discovered that our brains have the ability to determine the shape of an object simply by processing ...

Human eye inspires advance in computer vision (w/Video)

Jun 18, 2009

Inspired by the behavior of the human eye, Boston College computer scientists have developed a technique that lets computers see objects as fleeting as a butterfly or tropical fish with nearly double the accuracy and 10 times ...

Recommended for you

Computerized emotion detector

21 hours ago

Face recognition software measures various parameters in a mug shot, such as the distance between the person's eyes, the height from lip to top of their nose and various other metrics and then compares it with photos of people ...

Cutting the cloud computing carbon cost

Sep 12, 2014

Cloud computing involves displacing data storage and processing from the user's computer on to remote servers. It can provide users with more storage space and computing power that they can then access from anywhere in the ...

Teaching computers the nuances of human conversation

Sep 12, 2014

Computer scientists have successfully developed programs to recognize spoken language, as in automated phone systems that respond to voice prompts and voice-activated assistants like Apple's Siri.

Mapping the connections between diverse sets of data

Sep 12, 2014

What is a map? Most often, it's a visual tool used to demonstrate the relationship between multiple places in geographic space. They're useful because you can look at one and very quickly pick up on the general ...

User comments : 9

Adjust slider to filter visible comments by rank

Display comments: newest first

hush1
not rated yet Jun 20, 2011
Kudos
Why just heat?
The human brain does the same for sound.
Sound mapping allows a human to recognize sound, (language).
The meaning assigned, (associated) occurs at a later point in embryonic development and of course, after birth.
Alburton
5 / 5 (1) Jun 20, 2011
Beautiful idea!
Now it is easy to exclaim it..but I still will =)
But of course!Using heat transport equations as a description of the geometry in which it happens!! EUREEEEEEEEEEEEEEEEEEEEEKAAA (not just greek but texan,right? =)

hush1
not rated yet Jun 20, 2011
In any language, a revelation.
lol, Even in texan. lol
kevinrtrs
1 / 5 (4) Jun 21, 2011
OK, this is indeed a new and exciting development. I just wonder why they credited only Einstein and Fourier and left poor Laplace out of it, whilst relying on the transformations from time to frequency domain and back again?

That aside, notice how man is tending to create robots in his own image. Everything the robot can or should be able to do is based on mimicking what mankind is capable of. This reminds one of us being made in the image of our own creator.
antialias_physorg
1 / 5 (1) Jun 21, 2011
This is nice but it has one serious drawback (one that has been plagueing me and my colleagues in segmentation of medically relevant structures for decades now): Pathologiies tend to be wildly aberrant in how they alter the shape of the structure (think broken bones, spreading tumors, missing parts due to trauma, etc. )

The idea, however, is very interesting. I can see where that would render great results in areas which are not prone to such pathological aberrations of shape.
hush1
not rated yet Jun 21, 2011
antialias physorg:

The method is independent from the susceptibility (proneness) of aberrations in shape.

The example I have in mind: A rubber jigsaw puzzle.
You can assess the 'amount' of 'pathology' when the contortions/distortions exceed the physical limitations of the rubber, in the case where the puzzle still has all its' pieces.

In the case where the puzzle is missing pieces and is subject to contortions/distortions as well, the pathological state(s) is/are still discernible from the states that still lie within the physical limits of a puzzle.

A comparison and the difference of possible states with the states of the sample probe (using heat diffusion) is the pathological aberration, apart from 'normal' aberrations.

The numerical segmentation plays no role. Except in defining how fine you want to measure shape. You know all the variations. Fourier's law shows exactly which aberrations are pathological and which aberrations are not pathological.

Where's the drawback?
antialias_physorg
1 / 5 (1) Jun 21, 2011
The drawback is that the described method only works on continuous surfaces. thus it would not be able to segment (properly):
- broken bones
- lesions in tissues
- metastatic tumors
...all these are hot areas of research.

The real drawback of the methid is, however, that you need to be able to generate the surface mesh. In effect this means that you must have already segmented the structure before segmenting it. (e.g. in my work I segment a certain structure within bones which is not easily differntiated from surrounding trabecular regions and 'peters out' into these areas. There is no sharp boundary to which one could easuly limit the mesh.)

This is common to many medical problems (especially when segmenting tumors)
TheGhostofOtto1923
1 / 5 (2) Jun 22, 2011
This reminds one of us being made in the image of our own creator.
Sorry kev you've got that one backwards too. And we are creating robots to be BETTER than we are, just like we did with god, so you're doubly in error.
hush1
not rated yet Jun 23, 2011
"The drawback is that the described method only works on continuous surfaces. thus it would not be able to segment (properly):" - ap

What are a 'discontinuous' surfaces for you?

"The real drawback of the methid is, however, that you need to be able to generate the surface mesh. In effect this means that you must have already segmented the structure before segmenting it." - ap

That's just the point. The segmentation is arbitrary. Your choice.

"There is no sharp boundary to which one could easuly limit the mesh.) " - ap

You limit the mesh. It's arbitrary.

"This is common to many medical problems (especially when segmenting tumors)" -ap

Diagnostic? The method is 3D representation.