Researchers develop virtual composer assistant

June 11, 2018, National Research Nuclear University

Experts from the Institute of Cyber Intelligence Systems at the National Research Nuclear University MEPhI (Russia) are currently developing a virtual composer assistant that will be able to analyze the emotional state of music composers and follow their logic. Despite the high level of development of music theory, the process of creating music is still hard to formally algorithmize, since it is inseparably linked with the emotional experiences of the composer. This aspect of the creative process is of particular interest to experts in machine intelligence.

Scientists can already create synergy between humans and software in a way that allows the computer running the software to gather data on the and the intentions of the creator, and thereby become a continuation of the human mind. The virtual assistant is software that can create human-level music of high aesthetic quality on its own. It is a creative smart assistant that can add its own notes, chords and combinations to a melody written by the composer.

In order to achieve this goal, researchers had to identify musical sounds, chords and their combinations, create semantic maps based on this analysis, and connect them to models of humans' emotional perception of music. The new software is fundamentally different from other approaches, since the virtual assistant can maintain emotional contact with the composer, said Professor Alexei Samsonovich from the MEPhI Institute of Cyber Intelligence Systems.

"Most such programs represent neural networks taught to write music or make art," he said. "But our idea was to identify the emotional state of the composer and follow its logic."

This method is based on using semantic maps instead of neural networks, Samsonovich said. In semantic maps, the emotional state of one's mind is represented by a point in the affective space. In this space, every possible action has its own coordinates, and there is a certain rule that allows researchers to connect various coordinates with the others by following the logic of emotional perception. The laws of musical harmony and rhythm impose certain limitations on this process.

"These limitations, however, cannot define everything; they leave some freedom for creativity," Samsonovich explained. "This is the freedom that our software uses, while being guided by the semantic map and the rules of choosing the coordinates on it, and while taking into account the evolution of the emotional state. I would like to point out that we are not talking about machine learning, as the model is based on the principles of psychology (basic psychological aspects of behavior). In this sense, unlike , models like this can explain the essence of the phenomenon of creativity."

The mutual understanding between the composer and the virtual assistant can be achieved through organizing and storing various types of emotional reactions to musical elements—chords, their sequence and intervals—in the software environment.

In order to ensure storage and use of the information about the most successful and fitting combinations of sounds, the virtual assistant uses a semantic map, which allows the virtual composer not only to build on a melody set by the user according to his taste, but also monitor his mood, suggesting different versions and creating the proper accompaniment.

In the course of the method's development, researchers used a separate application in order to test how participants evaluate various combinations of chords according to three measures, each judged on a scale from one to 10: pleasant-unpleasant, cheerful-sad and exciting-boring. This evaluation system allowed researchers to create three-dimensional semantic maps.

"We assumed that one's choice of behavior based on social emotions, feelings and relations can be described by relatively simple dynamic equations," Samsonovich said. "The simple model of behavior based on the semantic map and two equations is indistinguishable from the human's behavior in the simplest of paradigms of social interaction in a virtual environment."

Samsonovich also added that the most promising approach in this field of research is based on cognitive architectures that are created using psychology and neuroscience data—the so-called biologically inspired cognitive architectures (BICA).

The researchers believe that the new development can be scaled up and used not only in music, but also in many other areas of digital creativity, as well as in human-machine systems. Creating and implementing such innovations is an important step in the study of the socio-emotional aspect of intelligence, which is responsible for the recognition and expression of emotions, intentions, motivations and desires. The key results of this research were published in the Procedia Computer Science journal.

Explore further: Researcher proposes social emotions test for artificial intelligence

Related Stories

EEG headset for emotion detection

January 9, 2018

At the International Consumer Electronics Show (CES) taking place in Las Vegas, Nev. Jan. 9-12, 2018, imec and Holst Centre will demonstrate a prototype of an electroencephalogram (EEG) headset that can measure emotions and ...

Recommended for you

Archaeologists discover Incan tomb in Peru

February 16, 2019

Peruvian archaeologists discovered an Incan tomb in the north of the country where an elite member of the pre-Columbian empire was buried, one of the investigators announced Friday.

Where is the universe hiding its missing mass?

February 15, 2019

Astronomers have spent decades looking for something that sounds like it would be hard to miss: about a third of the "normal" matter in the Universe. New results from NASA's Chandra X-ray Observatory may have helped them ...

What rising seas mean for local economies

February 15, 2019

Impacts from climate change are not always easy to see. But for many local businesses in coastal communities across the United States, the evidence is right outside their doors—or in their parking lots.

The friendly extortioner takes it all

February 15, 2019

Cooperating with other people makes many things easier. However, competition is also a characteristic aspect of our society. In their struggle for contracts and positions, people have to be more successful than their competitors ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.