This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:


trusted source


Orbiting astronaut oversees robot team on Earth

Orbiting astronaut oversees robot team on Earth
The Rollin' Justin robot in DLR's planetary surface test environment at the Agency's German Space Operations Center in Operpfaffenhofen near Munich, used for the Surface Avatar test on 24 July 2023, in which NASA astronaut Frank Rubio controlled a trio of robots on the ground to accomplish a set of tasks.  Credit: DLR

Astronaut Frank Rubio aboard the International Space Station collaborated with a small team of robots on Earth to accomplish a complex task—a first test of a new approach to combine human and robotic capabilities for our return to the moon and beyond.

The two-hour telerobotics test took place in a simulated planetary environment in the German Aerospace Center, DLR, at Oberpfaffenhofen near Munich. Working from Europe's Columbus module, NASA astronaut Frank Rubio directed a trio of robotic assets to remove a seismometer from a and place it on the ground to detect moonquakes.

Frank served as overseer for the robot team, working on a "point and click" basis. He took as needed using a force-feedback device. A DLR robotic lander arm was combined with DLR's Rollin' Justin humanoid robot and ESA's four-wheeled, two-armed Interact rover.

The goal is to shed light on what astronauts need when operating robots with varying levels of independence, known as "scalable autonomy." This approach enables astronauts to have more control over the robots.

In the test scenario, the first in the "Surface Avatar" series, a stuck pin had to be removed from the lander before the seismometer could be positioned by Justin. Meanwhile the Interact rover had been damaged by a quake during a surface sampling operation, so that the sample still stuck in its gripper needed to be retrieved by Justin and be placed on the lander.

Orbiting astronaut oversees robot team on Earth
NASA astronaut and Expedition 69 Flight Engineer Frank Rubio poses for a portrait while working inside the International Space Station's Destiny laboratory module. Credit: NASA/ESA

In the event, Frank accomplished all the tasks in time, while also giving the team valuable feedback on features of the user interface and the robots. Those will be implemented and validated in the next session of Surface Avatar in January 2024.

This first test opens the way to a follow-up Surface Avatar test with a larger team of robots, planned to be undertaken by Danish ESA astronaut Andreas Mogensen during his Huginn mission later this year.

The distances involved in spaceflight make it impractical to perform direct remote control of robots on the moon or other planetary surfaces from Earth. Instead, the idea is to have astronauts in orbit control these robots—to explore and take action on the moon, then later other alien environments, without the cost and risk of landing.

"Our METERON project undertaken with DLR, proved the feasibility of this force-feedback control technique from local orbit, expanding to work with astronauts on the ISS and culminating in an outdoor campaign in the moon-like environment of Mount Etna in Sicily," explains Thomas Krueger, head of ESA's Human Robot Interaction Lab, partnering with DLR's Institute of Robotics and Mechatronics.

Orbiting astronaut oversees robot team on Earth
The Rollin' Justin robot places a seismometer on the ground during the Surface Avatar test at DLR in Oberpfaffenhofen on 24 July 2023. NASA astronaut Frank Rubio controlled a trio of robots on the ground, generally seeing through one robot's eyes at a time. On the bottom left is a minimap and on the right the available options are listed. Augmented reality blue overlays are displayed around the objects when recognized by the system. Credit: ESA/DLR

"With Surface Avatar we combining direct teleoperation approach with supervised autonomy to control multiple robotic assets to accomplish : picture a on Earth, where a crane might work alongside a bulldozer or excavator to get a big task done.

"What we want to find out with this testing is how the astronauts perform in practice. What control mode do they choose? Do they have by seeing the world through the eyes of multiple robots? And what judgements they make in their head about how to act?"

The Surface Avatar test scenario assumes the relatively structured environment of a planetary surface base, where the robots already know how to interact with some of the objects around them and the other robots in place. So, an augmented reality display produces a list of options for the controller to select.

  • Orbiting astronaut oversees robot team on Earth
    A sample canister is placed on the lander by the Rollin' Justin robot during the initial Surface Avatar test campaign at DLR Oberpfaffenhofen on 24 July 2023. The robots were overseen by NASA astronaut Frank Rubio aboard the International Space Station on a "scalable autonomy" basis, switching to direct control when needed. Credit: DLR
  • Orbiting astronaut oversees robot team on Earth
    Control setup for the Surface Avatar system, allowing an astronaut aboard the International Space Station to oversee multiple robots on a given task. The laptop displaying views from the robots' cameras is supplemented by a joystick and also a haptic or "force feedback" device for the astronaut's hand, allowing precise control of a given robot's arm. The operational system is set up in the Columbus module in orbit, but this photo was taken on a mockup on the ground. Credit: ESA/DLR

If the desired option is not available, then the astronaut switches to direct control, feeling precisely what the robot arm experiences through a force-feedback control device, allowing a high level of precision.

"We are excited to be a step closer to giving astronauts and experts on Earth a wide range of possibilities to command and manage teams of different robots in space," says Principal Investigator Neal Y. Lii from the DLR Institute of Robotics and Mechatronics. "We will be able to use our robots on the surface as their physical avatars and intelligent co-workers to execute ever more complex tasks."

Thomas concurs, "This human- collaboration paves the way for future missions and permanent outposts on the moon and beyond."

Citation: Orbiting astronaut oversees robot team on Earth (2023, July 26) retrieved 4 March 2024 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

In simulated mission, astronauts obtained direct haptic feedback from robots they control on the ground


Feedback to editors