Teaming up with robots

March 27, 2013
With the aid of camera systems, the “Link4Save” software scans the production hall and can immediately stop robots in critical situations. Credit: Fraunhofer IDMT

Critical situations are occurring with greater frequency at industrial workplaces – situations that could lead to serious job-related accidents. With the "4Save" toolbox from Fraunhofer, these dangers do not arise at all. Researchers exhibit the latest version of the safety system, with "Eye4Save" visual surveillance, at the Hannover Messe from April 8 to 12, 2013.

Robots are taking over a greater number of tasks in industrial production: They weld and screw together components with millimeter precision, transport materials weighing several tons and package, palette and stack the finished products. Even their numbers are steadily rising. Germany – together with Japan and South Korea – has meanwhile become one of the countries with the world's highest density of robots. The space in factory halls is getting increasingly tighter. Electronic machines, and the technical staff in charge of them, are regularly getting in the way on the assembly lines and robotic pathways. That leads to an increased risk of accidents to the employees. Work areas that are tough to see into, or blind spots, hide the greatest risk potential. Thus security systems that also ensure increased employee safety are in demand as the robot population and the pace of work continue to rise.

Software prevents workplace accidents and unnecessary production stops

Researchers at the Fraunhofer Institute for IDMT in Ilmenau present the latest version of their "4Save" technologies at the Hannover Messe Digital Factory (Hall 7, Booth B10). The institute jointly developed this technology with Thüringen-based companies through the "BildRobo" collaborative project of the European Regional Development Fund (ERDF). It consists of the "Sim4Save" , the "Link4Save" software platform and the new "Eye4Save" surveillance system, made by the Zentrum für Bild- und Signalverarbeitung e.V. Ilmenau (ZBS). "The interaction between the individual components not only elevates safety, but also the efficiency of robot-supported workstations at the same time. This makes it possible to reduce work-related accidents and unnecessary production stops to a minimum," says Peter Pharow, Head of "Data Representation and Interfaces" at IDMT and co-developer of the "4Save" technology components, summarizing the advantages.

The core of the security system is the "Link4Save" . It analyzes signals from linked visual, acoustic and tactile surveillance systems in real time, and stops the equipment in before a collision could occur. Because it is often the minor inadvertent actions, like stumbling, that could lead to serious accidents with robots. The system precisely monitors how the individual equipment is moving, and steps in if the safe distance between human and robot is undercut. Depending on the power and speed of the operating equipment, the responses range from acoustic warning signals or slowing down the job process, to stopping the robot immediately.

Among other things, it incorporates data from the new "Eye4Save" visual . "Eye4Save" uses the images from synchronized stereo cameras to record objects in rooms in three dimensions. In the current configuration, four pair of stereo cameras are used, which are mounted in the upper corners of the room, and monitor the room from an oblique viewpoint. In an initial calibration process, the stereo camera pairs are calibrated both in terms of geometry and with respect to color. A miniature spectrometer from ZBS, which will also be presented at the trade show, acts as a tool here.

Camera system also recognizes occluded objects

Analogous to human depth perception, the depth information is taken from each pair of the stereo cameras for their respective viewpoint. The perception of objects in a space relates to the analysis of this depth information in combination with the recorded color values. As a reference, each stereo camera pair uses a distance-based, color-adaptable background model that it learns in advance. The foreground objects, detected from the various viewpoints, are put together and mapped out in the three-dimensional form of their visual hulls. "The more of the cameras see an object, the better the hulls represent the appearance of the object's actual shape. If there is any doubt – for instance, if a camera cannot see an object completely, because it is covered up by another object – the hull remains the same size. That means that occlusions are always treated in favor of the safety," explains Daniel Kapusi of ZBS, which is cooperating with IDMT.

The optimal number of cameras and sensors as well as their ideal positioning and orientation are determined interactively with the aid of the "Sim4Save" simulation tool. This tool simulates the production hall in three dimensions, and ensures the optimal coverage of the entire work area with safety technology. "Occluded objects or excessive distances often diminish the quality of camera recordings. These situations are taken into consideration with the simulation. 'Sim4Save' makes blind spots and obscured work areas a thing of the past," explains Pharow.

The "4Save" technologies could be used both in synergy as well as individually. While architects and planners of new factory buildings can use "Sim4Save" as a separate service, the "Link4Save" platform also works without the other elements of the "4Save" Toolbox, but not without upstream sensors. "Eye4Save" can be coupled with "Link4Save," but also operated on a platform-independent basis.

Explore further: 'Good-bye, blind spot' -- man and machine always in view

Related Stories

'Good-bye, blind spot' -- man and machine always in view

March 22, 2011

Particular care must be taken in a production hall where robots and men work together, where even minor carelessness could result in serious accidents or stop production. At the Hannover Messe trade fair that is taking place ...

Robotics: Safety without protective barriers

July 8, 2011

The modern working world is no longer conceivable without robots. They assist humans in manufacturing, laboratories or medicine. In the future, a new projection and camera-based system will prevent collisions between robots ...

Vigilant camera eye

September 20, 2010

An innovative camera system could in future enhance security in public areas and buildings. Smart Eyes works just like the human eye. The system analyzes the recorded data in real time and then immediately flags up salient ...

Care-O-bot 3: Always at your service

July 1, 2008

Who doesn’t long for household help at times? Service robots will soon be able to relieve us of heavy, dirty, monotonous or irksome tasks. Research scientists have now presented a new generation of household robots, the ...

3D television without glasses

September 1, 2011

When the boundaries merge between the action and the viewer, television becomes a special experience. Fraunhofer research scientists are optimizing the technologies that make it possible to watch TV in 3D without technical ...

Recommended for you

Firms push hydrogen as top green energy source

January 18, 2017

Over a dozen leading European and Asian firms have teamed up to promote the use of hydrogen as a clean fuel and cut the production of harmful gasses that lead to global warming.

WhatsApp vulnerable to snooping: report

January 13, 2017

The Facebook-owned mobile messaging service WhatsApp is vulnerable to interception, the Guardian newspaper reported on Friday, sparking concern over an app advertised as putting an emphasis on privacy.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.