Creativity research in 12 languages: Research team expands automatic semantic evaluation methods

"What could you use a brick for if not to build a house?" In human creativity research, study participants are often asked to to come up with unusual uses for various objects. This phase is then followed by a subjective and very time-consuming coding process. For this reason, researchers have long been striving to find faster and more objective ways to assess study participants' creativity.

A new development that has emerged in recent years is automated and computer-based scoring, where an algorithm calculates the semantic distance between participants' responses on creativity tasks.

So far, however, this method has mainly been used with English-language data. Led by John D. Patterson and Roger Beaty from Pennsylvania State University, U.S., a large international research team, including researchers from the Max Planck Institute for Empirical Aesthetics (MPIEA) in Frankfurt am Main, Germany, has now tested the method for use in other languages.

The study will be published in the journal Psychology of Aesthetics, Creativity and the Arts.

In a multi-laboratory study involving more than 6,500 participants, a total of 28 researchers collected creativity data in 12 different languages: Arabic, Chinese, Dutch, English, French, German, Hebrew, Italian, Persian (Farsi), Polish, Russian, and Spanish. They used two models based on the AI method of "deep learning" to calculate the semantic distance in the different languages. The researchers then compared the automatic metrics with human ratings.

"The best performing model for each language was consistent with human ratings. This confirmed the validity of the method for all 12 languages," reports Julia F. Christensen of the MPIEA.

The authors will provide open access to the multilingual dataset for future algorithm development, along with the Python code to compute the semantic distance in all languages mentioned.