Visualizing the emotional power of music

Musical styles and genres differ around the world, but the emotional power of music is universally felt. To understand this evocative force, researchers in many fields, including information science, neural perception, and signal processing, investigate music's underlying structure, examining features such as the tone, timbre, and auditory and rhythmic features of a piece.

Now a team of Japanese scientists from the University of Tokyo has developed a new approach to analyzing musical structure. The new method overcomes many of the limits of previous tools. The researchers publish their results in the journal Chaos.

"The new and important point of our method is that it analyzes the local and global features in one framework without corrupting the underlying regularities of the original time series," said Miwa Fukino, a doctoral student at the University of Tokyo and first author on the paper. "For example, when one uses Fast Fourier Transform to get the frequency spectrum and then uses it to calculate the feature vectors of tonality, the timing information is lost in the resulting data. So, the result cannot be used for further analysis related to the timing factors," she said.

Fukino and her colleagues' new approach is based on a graphing technique called a recurrence plot, a two-dimensional analytical tool that visualizes recurrences of data. Recurrence is a basic property of complex systems, first described mathematically in the late 19th century by French mathematician and scientist Henri Poincaré, in which systems return close to their initial state after a finite amount of time. The new twist the Japanese researchers added is to make a recurrence plot of recurrence plots. This provides layers of time series data to visualize simultaneously both local and global attributes of music. The researchers applied their new approach to selected classical piano pieces by Bach, Beethoven and Mozart.

Structural knowledge of music is useful for many professional applications, from composition of performance pieces to video game design to music therapists seeking a healing response. The researchers are continuing their work to reveal a scientific understanding of its evocative power.