Expert help from a distance

July 11, 2011
The system uses the 2-D barcodes attached to various parts of the malfunctioning machine to determine the camera's position and direction of view and then sends this data to the experts. Credit: © Fraunhofer FKIE

When electronic devices refuse to work, you rarely find the solution in the manual. Technicians often face similar problems with industrial machines, and companies end up flying in experts from the manufacturer to get things running again. A new system aims to help manufacturers lend a hand regardless of how far away they are located.

Machines stretch from one end of the production hall to another, each of them an important part of the . When one of these complex pieces of machinery stops working, the on-site technicians grab their tools and the manual and try to fix it – but sometimes the only solution is to call the manufacturer for tips on how to get it working again. The problem is that giving advice over the telephone is never easy: Do they mean the screw on the right or the screw on the left? Well, that depends on which side of the machine you are standing on! Even putting things down in writing can only get you so far. Often the only choice left is to fly experts in, sometimes from the other side of the world. But until they arrive, the machine stays stubbornly offline, perhaps bringing the entire production process to a halt.

Soon experts will have an easier and quicker method of supporting the on-site technicians without having to set foot outside their office: the augmented reality system developed by researchers from the Fraunhofer Institute for Communication, Information Processing and Ergonomics FKIE in Wachtberg. This allows technicians to record the malfunctioning machine with a camera fixed to the back of their laptop monitor. The computer is mounted on a swivel arm so that the technicians can view the screen while carrying out repairs. An image processing program calculates the camera's position and direction of view and sends this information to the manufacturer over standard telecommunications networks. This enables the experts to view the machine on their monitor from the same perspective as the technicians. They can even use the software to write instructions on specific parts of the machine such as 'Remove this screw'. These instructions then pop up on the technicians' screen on exactly the same part of the machine. And if a technician walks around the machine with the laptop, the image moves accordingly – and the written information stays where it was intended to be, for example hovering over a specific screw. Once the technicians have carried out the experts' instructions, the pop-up information can be deleted by simply clicking on it. The system is based on a chat protocol, which means everyone involved can communicate either through the chat function or by telephone.

The researchers managed to minimize the quantity of data transmitted to allow the system to function over a cellphone network. That means there is no need for a broadband connection, so can call on experts even from remote locations such as wind turbines in the middle of a field or in newly industrialized and developing countries. "We only transmit location data, not pictures," says Dr. Thomas Alexander, who heads up the research team at the FKIE. "At the moment we do that by attaching 2D barcodes to various parts of the machine. When the repair technician takes a picture of the malfunctioning machine, the software on the laptop reads those markings and links them to information in the database – for example the machine's identification code and the position and location of the barcodes that appear in the picture. And this is the only information that actually gets transmitted." Once the data reaches the manufacturer, another software program links it to the machine's CAD data to enable the experts to view the machine from the same angle as the on-site technician. The researchers have already developed a prototype of the system, and the next step is to carry out a study in which users will put the system through its paces. The results should help the scientists to optimize the system and tailor it more closely to user requirements.

Explore further: Virtual engineer to predict machine failure

Related Stories

Virtual engineer to predict machine failure

October 26, 2010

Scientists at the University of Portsmouth have created a ‘virtual engineer’ which uses artificial intelligence techniques to predict when machines need repairing.

Software tackles production line machine 'cyclic jitters'

April 2, 2008

Electronic commands passed from machine to machine over data networks increasingly drive today’s precisely timed and sequenced manufacturing production lines. However timing irregularities in the signals from even one machine—a ...

Handy remote monitoring system works anywhere

September 6, 2004

The VTT Technical Research Centre of Finland has teamed up with ten other European research bodies and companies from six European countries to create a unique remote monitoring system for machines. The product of a three-year ...

Training to promote health

May 10, 2011

Many people use fitness studios to get back into shape after an injury. Fraunhofer researchers are now developing a system that will provide doctors with data on the exercises people do on the various machines. This should ...

Startups offer new ways to fix your computer

February 11, 2009

Getting help for a problem PC can be a confusing and frustrating process. Tech support can be costly and impersonal. And often it doesn't fix the problem, or it requires consumers to call multiple times before getting help.

Recommended for you

'Droneboarding' takes off in Latvia

January 22, 2017

Skirted on all sides by snow-clad pine forests, Latvia's remote Lake Ninieris would be the perfect picture of winter tranquility—were it not for the huge drone buzzing like a swarm of angry bees as it zooms above the solid ...

Singapore 2G switchoff highlights digital divide

January 22, 2017

When Singapore pulls the plug on its 2G mobile phone network this year, thousands of people could be stuck without a signal—digital have-nots left behind by the relentless march of technology.

Making AI systems that see the world as humans do

January 19, 2017

A Northwestern University team developed a new computational model that performs at human levels on a standard intelligence test. This work is an important step toward making artificial intelligence systems that see and understand ...

Firms push hydrogen as top green energy source

January 18, 2017

Over a dozen leading European and Asian firms have teamed up to promote the use of hydrogen as a clean fuel and cut the production of harmful gasses that lead to global warming.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.