Astronaut Luca feeling the force, to advance rover control

ESA astronaut Luca Parmitano has made robotics history, reaching out from the International Space Station in orbit around Earth at 8 km/s, to control an Earth-based rover, equipped with an advanced gripper possessing the equivalent mobility and dexterity of a human hand.

This gripper was able to pick up and collect rock samples from the mock-lunar environment. In the future a comparable system may well be used to explore alien environments, with astronauts controlling surface rovers from the safety and comfort of a surface habitat or an orbiting spacecraft.

The Analog-1 test project, which concluded with this two-hour space-to-ground test on 25 November, had multiple technical goals. High among them was to assess the use of 'force-feedback' controls—like a high-end gaming joystick pushing back on their user, giving them a sense of touch—in space, to evaluate if this technology would enable high-precision robotic control in weightless conditions.

"Imagine the robot as Luca's avatar on Earth, providing him with both vision and touch," says ESA engineer Kjetil Wormnes, heading the Analog-1 test campaign. "It was equipped with two cameras—one in the palm of its hand, the other in a manoeuvrable arm—to let Luca and the remotely-located scientists observe the environment and get a close-up on the rocks."

As journalists watched, the Luca-controlled rover completed its sampling campaign right on schedule, traversing between a trio of sampling sites along challengingly narrow pathways. While selecting rocks Luca received advice from a team of geological experts based at the European Astronaut Centre (EAC) in Germany, simulating a real-life surface exploration survey.

"We benefitted from Luca's previous training through our Pangaea programme, giving astronauts practical experience in geology," adds Jessica Grenouilleau, Meteron project lead at ESA's Exploration Systems Group. "It helped tremendously in having an efficient discussion between the crew and the scientists."

In addition, Luca could feel whenever the robot touched the ground or picked up a rock, using a Sigma 7 force-feedback device with six degrees of freedom. This test was the first time that such a device was used on the ISS to control a robot on the ground.

The two-way control link between the rover and the ISS extended via communication satellites in geostationary orbit to EAC and on to the hangar, resulting in a latency or communication lag of 0.8 seconds, which Luca had to take into account when planning his moves.

The hardware and software used by Luca aboard ISS was developed by ESA's Human Robot Interaction Laboratory, based at ESA's technical heart in the Netherlands, with the support of the DLR German Aerospace Center's Institute for Robotics and Mechatronics—including the integration of control software to customise the system's force-feedback and haptic feedback for the inescapable time delay experienced within the system.

ESA robotic engineer Thomas Krueger, heading the HRI Lab, explains: "For this exploration scenario, involving a relatively short time delay, we've been able to combine the relative advantages of humans and robots: a human for their ability to deal with complex and unstructured environments and decision making, and a dexterous robot able to cope with harsh environments and to precisely execute its operator's commands.

"By enhancing the operator experience with force-feedback and intuitive controls we can make previously impossible robotic control tasks feasible, and open up new methods of exploring space. We are now keen on analysing the data and feedback from Luca to see the details of how he performed, and find out where we can improve and prepare future exploration plans."

The rover itself was also built by the HRI Lab, taking on the role of the project's payload developer. To establish the communication link with the ISS the team was supported by EAC.

Analog-1 has been the latest in a series of progressively more challenging human-robot test campaigns involving the ISS, collectively called Meteron—Multi-purpose End-to-End Robotic Operation Network. The first 1 degree of freedom force-feedback test took place back in 2015 with ESA's Haptics-1 experiment, progressing to DLR's 2 degrees of freedom Kontur-2 the following year—advancing now to a full 6 degrees of freedom movement.

The next step will be an outdoor test campaign in a Moon-like terrestrial location, planned for next year. A rover would examine and collect genuine rocks in an operational scenario resembling the complexity of a full mission on the Moon.