Teaming up with robots

Critical situations are occurring with greater frequency at industrial workplaces – situations that could lead to serious job-related accidents. With the "4Save" toolbox from Fraunhofer, these dangers do not arise at all. Researchers exhibit the latest version of the safety system, with "Eye4Save" visual surveillance, at the Hannover Messe from April 8 to 12, 2013.

Robots are taking over a greater number of tasks in industrial production: They weld and screw together components with millimeter precision, transport materials weighing several tons and package, palette and stack the finished products. Even their numbers are steadily rising. Germany – together with Japan and South Korea – has meanwhile become one of the countries with the world's highest density of robots. The space in factory halls is getting increasingly tighter. Electronic machines, and the technical staff in charge of them, are regularly getting in the way on the assembly lines and robotic pathways. That leads to an increased risk of accidents to the employees. Work areas that are tough to see into, or blind spots, hide the greatest risk potential. Thus security systems that also ensure increased employee safety are in demand as the robot population and the pace of work continue to rise.

Software prevents workplace accidents and unnecessary production stops

Researchers at the Fraunhofer Institute for Digital Media Technology IDMT in Ilmenau present the latest version of their "4Save" technologies at the Hannover Messe Digital Factory (Hall 7, Booth B10). The institute jointly developed this technology with Thüringen-based companies through the "BildRobo" collaborative project of the European Regional Development Fund (ERDF). It consists of the "Sim4Save" simulation tool, the "Link4Save" software platform and the new "Eye4Save" surveillance system, made by the Zentrum für Bild- und Signalverarbeitung e.V. Ilmenau (ZBS). "The interaction between the individual components not only elevates safety, but also the efficiency of robot-supported workstations at the same time. This makes it possible to reduce work-related accidents and unnecessary production stops to a minimum," says Peter Pharow, Head of "Data Representation and Interfaces" at IDMT and co-developer of the "4Save" technology components, summarizing the advantages.

The core of the security system is the "Link4Save" software platform. It analyzes signals from linked visual, acoustic and tactile surveillance systems in real time, and stops the equipment in critical situations before a collision could occur. Because it is often the minor inadvertent actions, like stumbling, that could lead to serious accidents with robots. The system precisely monitors how the individual equipment is moving, and steps in if the safe distance between human and robot is undercut. Depending on the power and speed of the operating equipment, the responses range from acoustic warning signals or slowing down the job process, to stopping the robot immediately.

Among other things, it incorporates data from the new "Eye4Save" visual surveillance system. "Eye4Save" uses the images from synchronized stereo cameras to record objects in rooms in three dimensions. In the current configuration, four pair of stereo cameras are used, which are mounted in the upper corners of the room, and monitor the room from an oblique viewpoint. In an initial calibration process, the stereo camera pairs are calibrated both in terms of geometry and with respect to color. A miniature spectrometer from ZBS, which will also be presented at the trade show, acts as a tool here.

Camera system also recognizes occluded objects

Analogous to human depth perception, the depth information is taken from each pair of the stereo cameras for their respective viewpoint. The perception of objects in a space relates to the analysis of this depth information in combination with the recorded color values. As a reference, each stereo camera pair uses a distance-based, color-adaptable background model that it learns in advance. The foreground objects, detected from the various viewpoints, are put together and mapped out in the three-dimensional form of their visual hulls. "The more of the cameras see an object, the better the hulls represent the appearance of the object's actual shape. If there is any doubt – for instance, if a camera cannot see an object completely, because it is covered up by another object – the hull remains the same size. That means that occlusions are always treated in favor of the safety," explains Daniel Kapusi of ZBS, which is cooperating with IDMT.

The optimal number of cameras and sensors as well as their ideal positioning and orientation are determined interactively with the aid of the "Sim4Save" simulation tool. This tool simulates the production hall in three dimensions, and ensures the optimal coverage of the entire work area with safety technology. "Occluded objects or excessive distances often diminish the quality of camera recordings. These situations are taken into consideration with the simulation. 'Sim4Save' makes blind spots and obscured work areas a thing of the past," explains Pharow.

The "4Save" technologies could be used both in synergy as well as individually. While architects and planners of new factory buildings can use "Sim4Save" as a separate service, the "Link4Save" platform also works without the other elements of the "4Save" Toolbox, but not without upstream sensors. "Eye4Save" can be coupled with "Link4Save," but also operated on a platform-independent basis.