New augmented reality applications assist astronaut repairs to space station

New augmented reality applications assist astronaut repairs to space station
Japan Aerospace Exploration Agency (JAXA) astronaut Soichi Noguchi prepares to conduct routine maintenance on the T2 Treadmill while receiving procedures through his augmented reality goggles. Credit: NASA

Most often, communications delays between the International Space Station crew and ground are nearly unnoticeable as they are routed from one Tracking and Data Relay Satellite to another as the station orbits about 250 miles above Earth. As NASA prepares to explore the Moon, about 240,000 miles away, and eventually Mars, which averages about 245 million miles away, NASA is developing tools to increase astronaut autonomy to operate spacecraft or systems without assistance from the Mission Control Center at NASA's Johnson Space Center in Houston because communication delays from Earth will last longer.

The T2 Augmented Reality (T2AR) project demonstrates how station crew members can inspect and maintain scientific and critical to maintaining crew health and achieving research goals without assistance from ground teams.

To kick off the T2AR activities in orbit in April, astronaut Soichi Noguchi of the Japan Aerospace Exploration Agency (JAXA) was tasked with maintenance for one of the space station crew's pieces of exercise equipment, the T2 Treadmill. The inspection procedure is typically available as a PDF document to be accessed on a computer or tablet, which can be hard to hold while also operating tools or flashlights or examining equipment in a tight space. This time, no extra handheld instructions or communication with ground teams in Mission Control were necessary since the information was all in plain sight. Using the Microsoft HoloLens augmented reality (AR) goggles and armed with novel procedure tracking software NASA developed, Noguchi had step-by-step guidance and cues to assist in the work without referring to a separate screen.

New augmented reality applications assist astronaut repairs to space station
This image from ground testing shows what guidance and information an astronaut might see when using augmented reality goggles to conduct maintenance on the space station. Credit: NASA

T2AR is the first in-space operational use of the HoloLens in combination with custom-built AR software, which enables an astronaut to perform unassisted maintenance and inspections on a major piece of crew support hardware. This investigation builds on the Sidekick experiment former NASA astronaut Scott Kelly conducted in 2016.

This novel demonstration used 3-D directional cues to direct the astronaut's gaze to the proper work sites and displayed the procedure instructions. The device followed an astronaut's verbal instructions to navigate procedures and displayed AR cues and procedure text over the hardware as appropriate for the procedure step being performed. The system also provided supplemental information, such as instructional videos and system overlays, to assist in performing the procedure.

"AR tools hold the promise of allowing us to pre-package guidance and expertise," says International Space Station associate scientist Bryan Dansberry at Johnson. "The space station is the perfect platform to test out AR systems and refine these tools so they will be ready when future astronauts need them. Closer to home, these tests help to mature software and AR technology now so expertise and support are available in remote locations around the world."

New augmented reality applications assist astronaut repairs to space station
NASA astronaut and Expedition 65 Flight Engineer Megan McArthur wears the specialized Sidekick headset and tests using augmented reality aboard the International Space Station. Credit: NASA

Since that first activity with Noguchi, astronaut Thomas Pesquet of ESA (European Space Agency) and NASA astronaut Megan McArthur have also used the AR application aboard the space station. The tests provided researchers with information about how the technology does and does not work to assist crew inspection maintenance procedures. With the completion of this maintenance activity, nine more test sessions remain in the technology demonstration plan.

While this demonstration was currently limited to the critical T2 Treadmill, the platform is designed to be used across a wide variety of space station needs in the future, leading to increased crew efficiency and activity execution accuracy. In addition to use on the station, this technology could aid astronauts on future journeys to the Moon and Mars, when communication delays are greater, by allowing them to conduct tasks without waiting for further direction from Earth.

NASA's Advanced Exploration Systems division sponsors this technology demonstration aboard the microgravity laboratory to advance future human and robotic exploration missions as part of NASA's Moon and Mars exploration approach.

Provided by NASA

Citation: New augmented reality applications assist astronaut repairs to space station (2021, September 20) retrieved 5 December 2022 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

2 Russian crew do spacewalk at International Space Station


Feedback to editors