Using hand gestures while speaking found to influence how words are heard by others

talking hand
Credit: Unsplash/CC0 Public Domain

A team of researchers from the Max Planck Institute for Psycholinguistics, Radboud University and TiCC Tilburg University, all in The Netherlands, has found that hand gestures used by people when speaking can influence how their words are being heard and interpreted by others. In their paper published in the journal Proceedings of the Royal Society B, the group describes experiments with volunteers who watched videos of people speaking with and without hand gestures.

Using while speaking is common to politicians and people in television commercials—but do such gestures have an impact on their audience? The researchers with this new effort sought to find the answer to that question by showing volunteers videos of people speaking under different conditions and then asking the volunteers questions about what they heard.

The conditions involved the speaker stressing different parts of words in a sentence, such as "ob" in the word "object" or "ject." Other conditions involved the speaker making different types of hand gestures, such as chopping, or sweeping motions—sometimes coinciding with the part of the word being stressed and at other times at random. The researchers recorded the volunteers as they watched and listened to the recordings, and participants were questioned afterward.

The researchers found that listeners gave more weight to syllables being spoken in conjunction with hand gestures—in 20% of the cases, they were more likely to hear and interpret the word being spoken. When mismatches between words being spoken and hand gestures occurred, however, listeners were 40% more likely to hear the wrong sound.

What we see shapes what we hearPeople often move their hands up and down to 'highlight' what they are saying. Are such 'beat gestures' important for communication? Hans Rutger Bosker from the Max Planck Institute for Psycholinguistics and David Peeters from Tilburg University created words with an ambiguous stress pattern and asked listeners what they heard (DIScount or disCOUNT?). The beat gestures people saw influenced what they heard, showing that listeners quickly integrate verbal and visual information during speech recognition. Credit: Hans Rutger Bosker/David Peeters - Max Planck Institute for Psycholinguistics

The researchers suggest their findings show that hand gestures are an important part of face-to-face communications because they clearly have an impact on what the listener is hearing. They also suggest that responding to gestures while someone is speaking might be learned as a person is growing up. But they also note that it is equally likely that there is an evolutionary reason for it. They note that the experiments were conducted with Dutch speakers, but suggest it is likely they would find the same results with other languages.


Explore further

Mix-StAGE: A model that can generate gestures to accompany a virtual agent's speech

More information: Hans Rutger Bosker et al. Beat gestures influence which speech sounds you hear, Proceedings of the Royal Society B: Biological Sciences (2021). DOI: 10.1098/rspb.2020.2419

© 2021 Science X Network

Citation: Using hand gestures while speaking found to influence how words are heard by others (2021, January 28) retrieved 28 February 2021 from https://phys.org/news/2021-01-gestures-words-heard.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.
519 shares

Feedback to editors

User comments