Human hearing beats the Fourier uncertainty principle

Feb 04, 2013 by Lisa Zyga feature
Each dot represents a subject’s performance on Task 5 (simultaneously measuring the duration and frequency of a sound), with temporal acuity on the x-axis and frequency acuity on the y-axis. All dots within the black rectangle beat the Fourier uncertainty principle. Credit: Oppenheim and Magnasco ©2013 American Physical Society

(Phys.org)—For the first time, physicists have found that humans can discriminate a sound's frequency (related to a note's pitch) and timing (whether a note comes before or after another note) more than 10 times better than the limit imposed by the Fourier uncertainty principle. Not surprisingly, some of the subjects with the best listening precision were musicians, but even non-musicians could exceed the uncertainty limit. The results rule out the majority of auditory processing brain algorithms that have been proposed, since only a few models can match this impressive human performance.

The researchers, Jacob Oppenheim and Marcelo Magnasco at Rockefeller University in New York, have published their study on the first direct test of the Fourier uncertainty principle in human hearing in a recent issue of .

The Fourier uncertainty principle states that a time-frequency exists for sound signals, so that the shorter the duration of a sound, the larger the spread of different types of frequencies is required to represent the sound. Conversely, sounds with tight clusters of frequencies must have longer durations. The uncertainty principle limits the precision of the simultaneous measurement of the duration and frequency of a sound.

To investigate human hearing in this context, the researchers turned to psychophysics, an area of study that uses various techniques to reveal how affect human . Using physics, these techniques can establish tight bounds on the performance of the senses.

An ear for precision

To test how precisely humans can simultaneously measure the and frequency of a sound, the researchers asked 12 subjects to perform a series of listening tasks leading up to a final task. In the final task, the subjects were asked to discriminate simultaneously whether a test note was higher or lower in frequency than a leading note that was played before it, and whether the test note appeared before or after a third note, which was discernible due to its much higher frequency.

When a subject correctly discriminated the frequency and timing of a note twice in a row, the difficulty level would increase so that both the difference in frequency between the notes and the time between the notes decreased. When a subject responded incorrectly, the variance would increase to make the task easier.

(a) In task 5, subjects are asked to discriminate simultaneously whether the test note (red) is higher or lower in frequency than the leading note (green), and whether the test note appears before or after the high note (blue). (b) Tasks 1 through 4 lead up to task 5: task 1 is frequency only, task 2 is timing only, task 3 is frequency only but with the high note (blue) as a distractor, and task 4 is timing only, but with the leading (green) note as a distractor. Credit: Oppenheim and Magnasco ©2013 American Physical Society

The researchers tested the subjects with two different types of sounds: Gaussian, characterized by a rise and fall that follows a bell curve shape; and note-like, characterized by a rapid rise and a slow exponential decay. According to the uncertainty principle, note-like sounds are more difficult to measure with high precision than Gaussian sounds.

But as it turns out, the subjects could discriminate both types of sounds with equally impressive performance. While some subjects excelled at discriminating frequency, most did much better at discriminating timing. The top score, achieved by a professional musician, violated the uncertainty principle by a factor of about 13, due to equally high precision in frequency acuity and timing acuity. The score with the top timing acuity (3 milliseconds) was achieved by an electronic musician who works in precision sound editing.

This video is not supported by your browser at this time.
A Gaussian note, characterized by a rise and fall that follows a bell curve shape

This video is not supported by your browser at this time.
A note-like note, characterized by a rapid rise and a slow exponential decay

This video is not supported by your browser at this time.
A reverse note-like note, characterized by a slow exponential rise and a rapid decay

This video is not supported by your browser at this time.
An example of the simultaneous time-frequency discrimination task for Gaussian notes

This video is not supported by your browser at this time.
An example of the simultaneous time-frequency discrimination task for note-like notes

This video is not supported by your browser at this time.
An example of the simultaneous time-frequency discrimination task for reversed note-like notes. All samples: Credit: Oppenheim and Magnasco

The researchers think that this superior human listening ability is partly due to the spiral structure and nonlinearities in the cochlea. Previously, scientists have proven that linear systems cannot exceed the time-frequency uncertainty limit. Although most nonlinear systems do not perform any better, any system that exceeds the uncertainty limit must be nonlinear. For this reason, the nonlinearities in the cochlea are likely integral to the precision of human auditory processing. Since researchers have known for a long time about the cochlea's nonlinearities, the current results are not quite as surprising as they would otherwise be.

"It is and it is not [surprising]," Magnasco told Phys.org. "We were surprised, yet we expected this to happen. The thing is, mathematically the possibility existed all along. There's a theorem that asserts uncertainty is only obeyed by linear operators (like the linear operators of quantum mechanics). Now there's five decades of careful documentation of just how nastily nonlinear the cochlea is, but it is not evident how any of the cochlea's nonlinearities contributes to enhancing time-frequency acuity. We now know our results imply that some of those nonlinearities have the purpose of sharpening acuity beyond the naïve linear limits.

"We were still extremely surprised by how well our subjects did, and particularly surprised by the fact that the biggest gains appear to have been, by and large, in timing. You see, physicists tend to think hearing is spectrum. But spectrum is time-independent, and hearing is about rapid transients. We were just told, by the data, that our brains care a great deal about timing."

This video is not supported by your browser at this time.
A famous excerpt from Casablanca with the characters Ilsa and Sam.

This video is not supported by your browser at this time.
In a “surrogated” version, an identical copy of the spectrum is retained, but all phase information is destroyed, destroying all the timing information and rendering the time series statistically stationary.

This video is not supported by your browser at this time.
In a “whitened” version, the spectrum is destroyed, but all phases are preserved, creating a time series which is spectrally perfectly white and statistically uncorrelated. The whitened version sounds like a bad-quality and noisier copy of the original, but everything is clearly recognizable. All samples: Credit: Oppenheim and Magnasco

New sound models

The results have implications for how we understand the way that the brain processes sound, a question that has interested scientists for a long time. In the early 1970s, scientists found hints that human hearing could violate the , but the scientific understanding and technical capabilities were not advanced enough to enable a thorough investigation. As a result, most of today's sound analysis models are based on old theories that may now be revisited in order to capture the precision of human hearing.

"In seminars, I like demonstrating how much information is conveyed in sound by playing the sound from the scene in Casablanca where Ilsa pleads, "Play it once, Sam," Sam feigns ignorance, Ilsa insists," Magnasco said. "You can recognize the text being spoken, but you can also recognize the volume of the utterance, the emotional stance of both speakers, the identity of the speakers including the speaker's accent (Ingrid's faint Swedish, though her character is Norwegian, which I am told Norwegians can distinguish; Sam's AAVE [African American Vernacular English]), the distance to the speaker (Ilsa whispers but she's closer, Sam loudly feigns ignorance but he's in the back), the position of the speaker (in your house you know when someone's calling you from another room, in which room they are!), the orientation of the speaker (looking at you or away from you), an impression of the room (large, small, carpeted).

"The issue is that many fields, both basic and commercial, in sound analysis try to reconstruct only one of these, and for that they may use crude models of early hearing that transmit enough information for their purposes. But the problem is that when your analysis is a pipeline, whatever information is lost on a given stage can never be recovered later. So if you try to do very fancy analysis of, let's say, vocal inflections of a lyric soprano, you just cannot do it with cruder models."

By ruling out many of the simpler models of auditory processing, the new results may help guide researchers to identify the true mechanism that underlies human auditory hyperacuity. Understanding this mechanism could have wide-ranging applications in areas such as speech recognition; sound analysis and processing; and radar, sonar, and radio astronomy.

"You could use fancier methods in radar or sonar to try to analyze details beyond uncertainty, since you control the pinging waveform; in fact, bats do," Magnasco said.

Building on the current results, the researchers are now investigating how human hearing is more finely tuned toward natural sounds, and also studying the temporal factor in hearing.

"Such increases in performance cannot occur in general without some assumptions," Magnasco said. "For instance, if you're testing accuracy vs. resolution, you need to assume all signals are well separated. We have indications that the hearing system is highly attuned to the sounds you actually hear in nature, as opposed to abstract time-series; this comes under the rubric of 'ecological theories of perception' in which you try to understand the space of natural objects being analyzed in an ecologically relevant setting, and has been hugely successful in vision. Many sounds in nature are produced by an abrupt transfer of energy followed by slow, damped decay, and hence have broken time-reversal symmetry. We just tested that subjects do much better in discriminating timing and frequency in the forward version than in the time-reversed version (manuscript submitted). Therefore the nervous system uses specific information on the physics of sound production to extract information from the sensory stream.

"We are also studying with these same methods the notion of simultaneity of sounds. If we're listening to a flute-piano piece, we will have a distinct perception if the flute 'arrives late' into a phrase and lags the piano, even though flute and piano produce extended sounds, much longer than the accuracy with which we perceive their alignment. In general, for many sounds we have a clear idea of one single 'time' associated to the sound, many times, in our minds, having to do with what action we would take to generate the sound ourselves (strike, blow, etc)."

Explore further: Experiment with speeding ions verifies relativistic time dilation to new level of precision

More information: Jacob N. Oppenheim and Marcelo O. Magnasco. "Human Time-Frequency Acuity Beats the Fourier Uncertainty Principle." PRL 110, 044301 (2013). DOI: 10.1103/PhysRevLett.110.044301

Journal reference: Physical Review Letters search and more info website

4.6 /5 (38 votes)

Related Stories

Sound, vision & hearing loss

Jun 24, 2011

(Medical Xpress) -- The mechanisms used by the brain to distinguish contrasting sounds may be similar to those used to visually pick out a face in the crowd.

Recommended for you

How Paramecium protozoa claw their way to the top

Sep 19, 2014

The ability to swim upwards – towards the sun and food supplies – is vital for many aquatic microorganisms. Exactly how they are able to differentiate between above and below in often murky waters is ...

User comments : 22

Adjust slider to filter visible comments by rank

Display comments: newest first

Anda
3.8 / 5 (8) Feb 04, 2013
I didn't know the Fourier uncertainty principle but I'm not surprised.
Music is one of the most amazing achievements of humans.
verkle
1.1 / 5 (29) Feb 04, 2013
Music is one of those amazing gifts from God himself.
Incredible how man has taken and used it.
I can't hardly go through a day without music.

kochevnik
2.1 / 5 (11) Feb 04, 2013
This inequality is also the Heisenberg uncertainty principle. Perhaps it's treatment as an absolute barrier to knowledge should be reexamined
wiyosaya
1.5 / 5 (8) Feb 04, 2013
This inequality is also the Heisenberg uncertainty principle. Perhaps it's treatment as an absolute barrier to knowledge should be reexamined

I have to agree.

As I see it, there are too many things taken as "gospel truth" because they are widely accepted as verbatim fact in mainstream science. I think the best of scientists keep a modicum of skepticism in such matters.
Mike_Massen
2.8 / 5 (13) Feb 04, 2013
verkle mumbled or waffled incoherently lying about a god creating music
Music is one of those amazing gifts from God himself.
Really, how so, which music did he compose and how was it communicated and btw: Which particular deity are you waffling about, the one by moses or the different one by jesus (who never appeared) or the different one by mohammed (who never appeared) or the different buddha nature/principle of all things or some other one, perhaps a pantheist one - that also cant communicate except through the brutality and selection of nature !

Seriously verkle, you say its from "god", how did this happen, cant see any musical notes in the old testament or new for that matter ?

Pray tell, get your deity to say hullo please, now don't let him/her/it be shy - ohhhkaaaayyy ?
Infinum
2.1 / 5 (7) Feb 04, 2013
Since most aspects of our surroundings is non-linear it is no wonder people evolved organs to perceive and interpret those non-linearities. We would not survive otherwise, so it is somewhat naive to assume that human perception can be described by linear models.

Great experiment.
AlexDSP
5 / 5 (3) Feb 04, 2013
Here is the paper with free access: arxiv.org/pdf/1208.4611.pdf. The Fourier uncertainty principle conveniently ignores phase information - that carries timing information - so it's little wonder it can be violated.
sigfpe
3 / 5 (2) Feb 04, 2013
The Fourier uncertainty principle doesn't set a limit on how accurately, say, the centre of a gaussian wavepacket, and the central frequency of a wavepacket, can be simultaneously be determined. There's no theorem in mathematics that says this is so and this experiment clearly demonstrates there is no such limit. So the title is a bit misleading. This is a different situation to Heisenberg's uncertainty principle in quantum mechanics where we don't get to directly take measurements from a wavefunction.
NeptuneAD
3 / 5 (2) Feb 04, 2013
I am blessed to have a good ear for music as is my daughter, my son isn't, so it makes me wonder what it is that is different that makes it so, instantly recognizing when a note is not right and when the timing is off seems natural to me but a miracle to others.

Btw, please don't take any of my words to mean anything religious, it will just make you look silly and that you have no confidence in your beliefs.
tigger
5 / 5 (1) Feb 04, 2013
My co-worker is tone deaf... it drives my near-perfect-pitch ear insane as he 'sings' along with the radio!
Ojorf
2 / 5 (4) Feb 04, 2013
The differences are mostly in your mind and not your ears. How much brain you have allocated to processing the data and how efficient it is at doing so.
Wistrolf
4.5 / 5 (2) Feb 04, 2013
I see no connection to the heissenberg principle here and theoretical limitations for the perceptions from that point of view. I'm no expert in Fourier but I think its limitations come from the fact that it is a dead function. No learning capabilities. The amazing ability of our voice perception I think is due to a functionality that could be described as bayesian or iterative. This paper reviel, once again, the adaptive beautiness of Evolution and its constructions
kochevnik
1 / 5 (1) Feb 04, 2013
Wave equations are often borken out in Fourier series. As to the remark that no phase information is included, I respectfully dissent. Phase and volume holograms can be encoded with Fourier series even using a normalized amplitude. In the case of holograms the quantum light waves are completely described by the spectra. Open any quantum mechanics book and you will see Fourier series abound.
ralph638s
1 / 5 (2) Feb 04, 2013
Fourier analysis produces both frequency and phase information but, in many audio processing algorithms, the phase is usually discarded since it is deemed unnecessary. Clearly, that is not the case...
tkjtkj
5 / 5 (2) Feb 05, 2013
Mike_Masser .. yes thank you for the contribution : " Pray tell, get your deity to say hullo please, now don't let him/her/it be shy - ohhhkaaaayyy ? "


It doesnt take long for the zealots to pop-up here .. unabashedly endorsing their concepts of 'a perfect being' making perfect music .. Music of the Spheres .. yes of course, .. spheres being perfect, not ugly like those evil elipses .. and tough, too, having dominated celestial science into the submission of ignorance that constrained science for so many centuries ..
Why can not our moderators here put on their academic robes and just declare that aNY comment here referring to the magic of religion has no place in a meeting-place dedicated to science ...
steelerobert
1 / 5 (4) Feb 06, 2013
@tkjtkj I have an electrical engineering degree and studied Fourier analysis on signals in college. ALL of the creators of modern science believed in a supreme intelligent creator: Bacon, Newton, Boyle, Liebnitz, Pascal, Kepler, Copernicus, Galileo, Leeuwenhoek. The list goes on and on. There are many scientists today who believe such new findings show a lack of randomness in the design of the world around us. That is the philosophy that gave birth to modern science, that there was a design, that there were natural laws, a lawgiver and it was possible to understand what He had made. The modern movement to deny this truth is politics not science.
Mike_Massen
2 / 5 (4) Feb 06, 2013
steelerobert mumbled something akin to idolatry & hypnosis with this weirdness
..lawgiver and it was possible to understand what He had made. The modern movement to deny this truth is politics not science.
Please explain in direct terms:-

1. Is this the deity of Moses & is he/she/it any sort of good communicator at all ?
2. Is the deity you claim to be a 'lawgiver' have good & just ideas about punishment fitting a crime, eg A young girl being misled by a serpent ?

These seem to be the core issues why most intelligent people throw the deity of Moses in the dustbin... As to Jesus, there are several other issues that show this is also a mere story.

And I hope you are not a devotee of Islam, as there are just as obvious issues re care of humanity and logic that arise...

Your best answer, oh and please take your time - I can wait a few hours but, I am hoping they are obvious to you from your best method of getting the so called laws from some 'lawgiver' ?

Thankyou :-)
kochevnik
4.2 / 5 (5) Feb 06, 2013
ALL of the creators of modern science believed in a supreme intelligent creator: Bacon, Newton, Boyle, Liebnitz, Pascal, Kepler, Copernicus, Galileo, Leeuwenhoek.
How many were not under threat of death by the church for heresy? Obviously you slept through history class
Mike_Massen
3.3 / 5 (7) Feb 06, 2013
kochevnik absolutely correct
How many were not under threat of death by the church for heresy?
Especially Copernicus who rattled the church's view re earth at centre & Galileo who was under house arrest, rest of his life !

http://en.wikiped..._Galilei

Dont forget church bureaucracy power structure maintained persecution due to great insecurity, they have original documents re the old/new testament never released showing OT is clearly allegory & NT a hodgepodge of stories written long after Jesus died/left/vanished/launched.

Many researchers studied & experimented despite the urging of the church to "..not look too closely into works of the Lord.."

Imagine where we would be today if microbiology & research into the causes of suffering were done with steadfastness & without fear from any church.

Do you think we would have a cure for cancer, alzheimers etc & advanced treatment of injuries after 3000 plus years of investigations instead of a mere 300 or so !?!
Tausch
1 / 5 (3) Feb 07, 2013
Bravo!
I will get you all started:
http://brainconne...ory-phys

Is that not a most wonderful illustration?!
And from the MIT press the primers are: Musimathics Vol. 1&2.

I'm researching and writing on the origin of human language.
A book is planned.
Kudos AlexDSP!
For those of you not entering related fields of science a cursory review of tonotopy is all that is needed.
Once a philosophical question, the answer to the harmless statement:
Can you hear the shape of sound?
will take you on a scientific endeavor beyond anything you have ever imagined.
The journey and narration starts with classical physics, the wave, enters classical information theory, transitions to quantum information theory and physics and ends at molecular chemistry and biology within the brain. No field of science is neglected.

The research is all a repeat. Been there long ago.
Still. Nothing is so wonderful it can not be repeated.
(Bite me, double negative watchers!!)
TyroneTork
not rated yet Feb 10, 2013
This paper reviel, once again, the adaptive beautiness of Evolution and its constructions


Sir, I must agree, however the use of Majuscule in the word "Evolution" is somewhat troublesome... Are you subtly implying the existence of a Intelligent Capitilizer?
musicphile
not rated yet Feb 17, 2013
The Fourier uncertainty principle doesn't set a limit on how accurately, say, the centre of a gaussian wavepacket, and the central frequency of a wavepacket, can be simultaneously be determined. There's no theorem in mathematics that says this is so and this experiment clearly demonstrates there is no such limit. So the title is a bit misleading [...]


I agree 100% with sigfpe. Moreover, is that experiment really showing something exceptional from the human ear? Can't a computer do even better? Take a computer with 2 processors: proc 1 is equipped with a sonagram with high frequency resolution, proc 2 with a sonagram with high time resolution. The same experiment conducted once with this computer will provide an even higher precision for measuring the centre of the wavepackets: proc 1 will discriminate the green and red packets (though it won't discriminate red from blue) while proc 2 will discriminate the red and blue packets (though it won't discriminate green from red).