More objective than human hearing

In industrial production, the testing of machines and products by means of acoustic signals still takes a niche role. At the Hannover Messe 2017, Fraunhofer is exhibiting a cognitive system that detects erroneous sounds more objectively than the human ear (Hall 2, Booth C16/C22). The technology has successfully passed the initial practical tests and there detected up to 99 percent of the errors.

In industrial production, it is crucial that the machines work and that the product does not have any defects. The production process is therefore continuously monitored. By humans, but also by more and more sensors, cameras, software and hardware. In most cases, machine-based automated testing is based on visual or physical criteria. Only people also use their ears naturally: if something sounds unusual, a person switches the machine off for safety. The problem is this: Everyone perceives noises somewhat differently. Whether something goes wrong is therefore rather a subjective feeling and presents an increased susceptibility to error.

Training with millions of data records

The Fraunhofer Institute for Digital Media Technology IDMT develops cognitive systems that accurately identify faults based on acoustic signals. The technological approach combines intelligent acoustic measurement technology and signal analysis, machine learning as well as data-safe, flexible data storage. "We integrate the intelligence of listening into the industrial condition control of machines and automated test systems for products," explains Steffen Holly of IDMT's "Industrial Media Applications" business unit. Once they have been trained, cognitive systems can hear more objectively than human hearing: instead of two ears, they have, so to speak, many thousands of them at their disposal, in the form of millions of neutral data records. Initial pilot projects with industry are already under way. The researchers have been able to detect up to 99 percent of the defects purely acoustically.

Assigning sounds distinctly

The scientists identify possible sources of noises and analyze their causes, create a noise model of the environment, and focus their microphones there. "It is ideal to simulate the human ear: it receives sounds through the air," says Holly. From the total signal, the system calculates out background sounds, such as voices or from a forklift driving by. This is then repeatedly compared with previously determined, laboratory-pure reference noise. With the help of artificial neural networks, the scientists are gradually developing algorithms that are able to detect noises which occur from errors. "The cleaner the acoustic signal is, the better the cognitive system recognizes deviations," Holly explains. The technology is so sensitive that it also displays nuances in error intensity and manages complex tasks. An example from the field of automotive production: In modern car seats, a large number of individual motors are installed, with the aid of which the driver can adjust his seat individually. The design of the motors is not the same, their noises are different and they are installed in different places. "In a pilot project with an automotive supplier, our acoustic monitoring system was able to detect all of the error sources perfectly", Holly reports.

Flexible, secure data storage in the cloud

The Fraunhofer researchers are able to ensure the data security of the collected acoustic signals through user authorizations as well as rights and identity management. An example is the decoupling of real and virtual identities in order to not violate user rights when evaluating the data by different persons. Machines and test systems are usually installed in the production line. The researchers store their acoustic data records in a secure cloud. "We can react very flexibly to changes in the production process and adjust our cognitive system accordingly," Holly mentions as an additional advantage.