Sound waves bypass visual limitations to recognize human activity

Sound waves bypass visual limitations to recognize human activity
Video cameras continue to gain widespread use, but there are privacy and environmental limitations in how well they work. Acoustical waves are an alternative medium that may bypass those limitations. Unlike electromagnetic waves, acoustical waves can be used to find objects and also identify them. As described in a new paper in Applied Physics Letters, the researchers used a 2D acoustic array and convolutional neural networks to detect and analyze the sounds of human activity.

By using a two-dimensional acoustic array with 256 receivers and four ultrasonic transmitters, the researchers were able to gather data related to four different human activities: sitting, standing, walking and falling. Credit: Xinhua Guo

Video cameras continue to gain widespread use to monitor human activities for surveillance, health care, home use and more, but there are privacy and environmental limitations in how well they work. Acoustical waves, such as sounds and other forms of vibrations, are an alternative medium that may bypass those limitations.

Unlike , such as those used in radar, acoustical waves can be used not only to find objects but also to identify them. As described in a new paper in the May 28 issue of Applied Physics Letters, the researchers used a two-dimensional acoustic array and to detect and analyze the sounds of human activity and identify those activities.

"If the is high enough, a large number of applications could be implemented," said Xinhua Guo, associate professor at Wuhan University of Technology. "For example, a medical alarm system could be activated if a person falls at home and it is detected. Thus, immediate help could be provided and with little privacy leaked at the same time."

By using a two-dimensional acoustic array with 256 receivers and four ultrasonic transmitters, the researchers were able to gather data related to four different human activities—sitting, standing, walking and falling. They used a 40-kilohertz signal to bypass any potential contamination from ordinary room noise and distinguish it from the identifying sounds.

Their tests achieved an overall of 97.5% for time-domain data and 100% for frequency-domain data. The scientists also tested arrays with fewer receivers (eight and four) and found them to produce results with lower accuracy of the human activity.

Guo said acoustic systems are a better detection device than vision-based systems because of the lack of widespread acceptance of cameras due to privacy issues. In addition, low lighting or smoke can also hamper vision recognition, but sound waves are not affected by those special environmental situations.

"In future, we will go on studying complex activity and situation of random positioning," Guo said. "As we know, human activities are complicated, taking falling as an example, and can present in various postures. We are hoping to collect more datasets of falling activity to reach higher accuracy."

Guo said they will be experimenting with various numbers of sensors and their effectiveness in detecting and determining human activities. He said there is an optimal number for the that would make this viable for commercial and personal use in homes and buildings.


Explore further

Show your hands: Smartwatches sense hand activity

More information: "A single feature for human activity recognition using two-dimensional acoustic array," Applied Physics Letters (2019). DOI: 10.1063/1.5096572
Journal information: Applied Physics Letters

Citation: Sound waves bypass visual limitations to recognize human activity (2019, May 28) retrieved 19 June 2019 from https://phys.org/news/2019-05-bypass-visual-limitations-human.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.
184 shares

Feedback to editors

User comments

May 28, 2019
These guys sound like they are looking for a better "mousetrap" to sell...

May 28, 2019
The need for detection of falling is very real. Many elderly people fall in their homes. It can happen almost soundlessly, as when a person slides from sitting in a chair to sprawling on the floor. Once down, elderly persons often cannot raise themselves even if they are uninjured. Unable to reach an accessible communication device, they may be stuck on the floor for hours or days until someone visits the home.

May 28, 2019
The need for detection of falling is very real. Many elderly people fall in their homes. It can happen almost soundlessly, as when a person slides from sitting in a chair to sprawling on the floor. Once down, elderly persons often cannot raise themselves even if they are uninjured. Unable to reach an accessible communication device, they may be stuck on the floor for hours or days until someone visits the home.
Granted, but I fail to see how a deep-learning v i s u a l analysis network wouldn't do a far better job at the same task, and be able to do so with the input video stream destroyed in real time, preserving privacy, which was one of the points raised by the article.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more