As it turns out, even with these virtual tools, some old manual habits are hard to break and apparently spacefaring specialists make mistakes just like the rest of us.
Astronaut Scott Kelly wearing a HoloLens headset on the International Space Station.
Image: NASA
Across industries, augmented and virtual reality are adding a 21st-century twist on standard operations and redefining business as usual. This ranges from mixed reality manufacturing to a more literal take on modern medicine, but AR and VR are also being implemented well beyond our Earthly confines, as NASA and JPL tap these capabilities to reimagine space exploration and mission procedures. Sometimes with these tools, as the saying goes, seeing is believing, as engineers and project scientists take a dive into the digital ether to bring tomorrow's spacecraft to life in 3D. But, as it turns out, some old manual habits are hard to break and apparently spacefaring specialists make mistakes just like the rest of us.
SEE:IT expense reimbursement policy(TechRepublic Premium)
In a recent post, NASA detailed a number of ways the agency taps AR and VR capabilities aboard the space station, ranging from remotely controlling robots to AR maintenance procedures. At the same time, JPL is using mixed reality to create an immersive Martian experience, allowing scientists to take a virtual stroll on our cosmic neighbor.
In one particular project, dubbed Sidekick, astronaut Scott Kelly used a Microsoft HoloLens headset to conduct ISS training and future mission prep. During these tests, a member of mission control on Earth could stream Kelly's field of view via the headset and also draw visuals the astronaut could visualize in 3D on the HoloLens display, explained Benjamin Nuernberger, an AR specialist at JPL.
"So instead of just talking about something spatially, just verbally, now you can give some more spatial 3D instructions drawing into his field of view," he said.
Situationally, Nuernberger discussed the benefits of these VR and AR capabilities alongside NASA's detailed step-by-step iPad instructions the astronauts have at their disposal. While he gave the agency its props, stating NASA does a "great job" of providing detailed procedures, in some instances, such as "unforeseen repairs" or if the instructions are particularly complicated, "it's hard to write up these procedures."
"This kind of technology can help, especially in those situations and unforeseen circumstances, because you could see their perspective, and you can draw on the screen, and those screen drawings can then be displayed in 3D for the astronaut," he said.
A NASA concept image related to the Sidekick project.
Image: NASA
These astronauts may be conducting mission operations on an orbiting space laboratory hundreds of miles overhead, but the conundrum associated with implementing a plan or following a set of instructions isn't necessarily so different from a familiar mundane experience here on Earth.
"Just imagine building IKEA furniture at home. You read the instructions, and you try to build it. Many times we mess up on a certain step," Nuernberger said. "Astronauts, they're extremely smart, and they know how to transfer the instructions from the paper or the iPad into the spatial realm, but still everyone's human in the end."
Scandinavian furniture snafus aside, he said these AR and VR capabilities can help save time, reduce risk and decrease the "cognitive load" on astronauts.
Over time, this project transformed into the ProtoSpace project Nuernberger currently works on, an undertaking that transforms spacecraft designs into 3D interactive visuals using uploaded CAD files.
"Basically, we want to visualize the spacecraft before it's built, while it's being built [and] maybe even after it's built," he said.
To understand the real value-add of these spatial visualizations, it's helpful to take a look at the way scientists and engineers would approach design and assembly before the days of AR and VR headsets. Prior to ProtoSpace, Nuernberger said this planning phase could involve building mockups with costs pushing "tens of thousands of dollars," lots of cabling to support the craft and several day's time putting it all together.
"Now, you can virtually see that almost instantaneously," he said. "You have a CAD model. You upload it with those cables, and then you put on the headset and you're ready to go."
During the design process, Nuernberger said members of the team would "go back and forth over email or show PowerPoints to each other for a long time," adding that he's heard anecdotes where these scientists will put on one of the AR/VR headsets and "almost immediately" come up with the answer to a design or assembly question.
A screenshot of an OpsLab JPL video highlighting these AR and VR uses.
Image: YouTube/OpsLab JPL
SEE:Juggling remote work with kids' education is a mammoth task. Here's how employers can help (free PDF)(TechRepublic)
A lot of these AR and VR-enabled capabilities and new efficiencies comes back to one critical point: NASA builds very specific spacecraft for a very specific environment and purpose. This means the team is in-essence reinventing the roving wheel with each new mission; essentially creating an exploration recipe from scratch with each go-around.
"We're not like a car manufacturer where we're doing the same thing over and over and over again," he said. "We're building one rover, maybe two [...] and if we get something wrong, the risk is very high, especially, in the assembly phase because there's eventually a launch window."
Interestingly, there are also AR and VR workplace safety benefits at play. While NASA may routinely tackle extraterrestrial feats and maneuvers on distant planetary bodies, the agency must still abide by OSHA standards here on terra firma.
In one instance, Nuernberger said the NASA team was planning spacecraft and support equipment configurations when a Microsoft HoloLens revealed a critical flaw in their approach.
"They realized that if [they arranged the craft] in this configuration, someone's going to have to be working under a suspended load, which is a very risky and dangerous thing, which is not allowed under OSHA standards," Nuernberger said.
A screenshot of an OpsLab JPL video highlighting these AR and VR uses.
Image: YouTube/OpsLab JPL
When it comes to the advantages of these tools, Nuernberger said the engineers have "seen that it's not just a toy." As the headset slides on and the visual capabilities take hold, there's often this moment of realization that these AR and VR tools have real functionality, he explained.
"Typically it's when someone puts on the headset for the first time and they look at these massive spacecraft, not just like a little augmented reality experience on your tabletop, but they look at this full-scale spacecraft and they're like, 'Oh my goodness, how big is this?!'"
Self-admittedly, this experience even surprises Nuernberger sometimes, as he slides on the headset and sees the Mars Rover in a living room or conference room and realizes just how large the craft is in a stereoscopic augmented reality as opposed to a pixelated vehicle framed on a computer screen.
Watching the NASA and JPL videos highlighting this technique can be a somewhat surreal experience: A team of JPL employees huddled around a seemingly massive yet-invisible object, each donning virtual reality headsets, manipulating components within a multi-million dollar craft that currently exists as mere digital ether in the middle of it all.
Situationally, sometimes the virtual and the physical worlds blend for the members of the team in this mixed reality mashup and old manual habits can be hard to break.
"We've had people come in with physical tools and put the physical tool on the virtual model, which is kind of cool," Nuernberger said.
Our editors highlight the TechRepublic articles, downloads, and galleries that you cannot miss to stay current on the latest IT news, innovations, and tips. Fridays
Read more here:
Seeing is believing: AR and VR bring the future of space exploration to life in 3D - TechRepublic
- A Q&A with Katya Echazarreta, the first Mexican-born woman to travel to space - Astronomy Magazine - April 24th, 2024 [April 24th, 2024]
- Japans planning its first lunar steps with the Artemis program - The Hill - April 24th, 2024 [April 24th, 2024]
- DC Eyes Beijing's Investments In Space - Payload - Payload - April 24th, 2024 [April 24th, 2024]
- THIS WEEK @NASA: More Partners in Space Exploration, Upgrade to Testing Facilities for Artemis II - SpaceCoastDaily.com - April 24th, 2024 [April 24th, 2024]
- Explore Space and Poetry with NASA and Poet Laureate Ada Limn - sxsw.com - April 24th, 2024 [April 24th, 2024]
- NASA warns of secret military presence in space disguised as civilian programs - TweakTown - April 24th, 2024 [April 24th, 2024]
- NEW: UAH wins first place in college division in NASA's 30th Human Exploration Rover Challenge - Hville Blast - April 24th, 2024 [April 24th, 2024]
- Are we prepared for Chinese preeminence on the moon and Mars? (op-ed) - Space.com - April 24th, 2024 [April 24th, 2024]
- As space exploration increases, effective space regulation is a necessity - Verdict - April 24th, 2024 [April 24th, 2024]
- NASA Announces Winners of Power to Explore Challenge - NASA - April 24th, 2024 [April 24th, 2024]
- NASA tests walking robot on Mount Hood for space exploration with universities - KTVL - April 24th, 2024 [April 24th, 2024]
- Five future astronauts certified from European Space Agency's 2022 graduating class - UPI News - April 24th, 2024 [April 24th, 2024]
- Sweden becomes 38th country to sign NASA's Artemis Accords for moon exploration - Space.com - April 24th, 2024 [April 24th, 2024]
- Space start-up claims to have discovered propellant-free propulsion system - NewsBytes - April 24th, 2024 [April 24th, 2024]
- Soyuz Spacecraft Launches to Space Station With NASA Astronaut - SciTechDaily - March 24th, 2024 [March 24th, 2024]
- This Week In Space podcast: Episode 103 Starship's Orbital Feat - Space.com - March 24th, 2024 [March 24th, 2024]
- From 3D To Outer Space: Register For WCC Youth Summer Camps - mitechnews.com - March 24th, 2024 [March 24th, 2024]
- NASA and SpaceX: Enhancing Space Exploration with the 30th Resupply Mission - Space Daily - March 24th, 2024 [March 24th, 2024]
- The Cosmic Enigma: How Space Exploration is Unraveling the Universe's Birth Secrets - yTech - March 24th, 2024 [March 24th, 2024]
- Following Safe Return, NASA's SpaceX Crew-7 to Recount Space Mission - NASA - March 24th, 2024 [March 24th, 2024]
- The Heat: Space Exploration - CGTN America - March 24th, 2024 [March 24th, 2024]
- SpaceX Time-Lapse Video Captures the Essence of Space Travel - yTech - March 24th, 2024 [March 24th, 2024]
- TOP 4 Reasons to Return to the Moon - The Universe. Space. Tech - March 24th, 2024 [March 24th, 2024]
- SpaceX Dragon Successfully Attaches to ISS Bringing Vital Research Payloads - yTech - March 24th, 2024 [March 24th, 2024]
- End of an Era: Delta IV Heavy Makes Final Journey to the Stars - yTech - March 24th, 2024 [March 24th, 2024]
- Photography exhibition at the Mound delves into space exploration - The National - March 24th, 2024 [March 24th, 2024]
- Lunar Node 1: the trailblazing lunar beacon guiding NASA's future in space exploration - Boing Boing - March 24th, 2024 [March 24th, 2024]
- SpaceX's CRS-30 Mission: Delivering Innovations and Research to the ISS - yTech - March 24th, 2024 [March 24th, 2024]
- Remembering Tom Stafford, the Space Races Peacemaker (1930-2024) - AmericaSpace - March 24th, 2024 [March 24th, 2024]
- 9 Hints That Kingdom Of The Planet Of The Apes Involves Space Travel - Screen Rant - March 24th, 2024 [March 24th, 2024]
- NASA sends science projects and hardware aboard SpaceX's 30th resupply launch to ISS SatNews - SatNews - March 24th, 2024 [March 24th, 2024]
- SpaceX Dragon Spacecraft Docks to Space Station With New Science and Supplies - SciTechDaily - March 24th, 2024 [March 24th, 2024]
- Moon Race 2.0: Why so many nations and private companies are aiming for lunar landings - BBC.com - February 16th, 2024 [February 16th, 2024]
- NASA Selects Texas A&M As First Approved Exploration Park Facility - Texas A&M University Today - February 16th, 2024 [February 16th, 2024]
- The Space Economy in 2024 - OODA Loop - February 16th, 2024 [February 16th, 2024]
- Opinion | It's an exciting time in space exploration. But U.S. leadership is at risk. - The Washington Post - February 16th, 2024 [February 16th, 2024]
- NASA Experiment With Fire in Space Ends When Entire Spacecraft Burns Up - Futurism - February 16th, 2024 [February 16th, 2024]
- NASA Selects Texas A&M as First Approved Exploration Park Facility - NASA - February 16th, 2024 [February 16th, 2024]
- Ground-Based Lasers Could Accelerate Spacecraft to Other Stars - Universe Today - February 16th, 2024 [February 16th, 2024]
- 8 Space Exploration Books to Add to Your TBR - Book Riot - February 16th, 2024 [February 16th, 2024]
- Meet the Elected Members of the African Space Council - Space in Africa - February 16th, 2024 [February 16th, 2024]
- NASA working with Texas A&M to build new spaceflight research facilities - 25 News KXXV and KRHD - February 16th, 2024 [February 16th, 2024]
- Inflatable Habitats: Expanding Space Exploration - Lockheed Martin - February 16th, 2024 [February 16th, 2024]
- China's Rising Ambitions in Space Exploration: Chang'e Missions and International Cooperation - Medriva - February 16th, 2024 [February 16th, 2024]
- Pressured Space Exploration in Today's Era - Mirage News - February 16th, 2024 [February 16th, 2024]
- Exoplanet Hunter Sets Its Sights on Jupiter: A New Twist in Space Exploration - SciTechDaily - February 16th, 2024 [February 16th, 2024]
- Frontier Aerospace TALOS Engines Used For Space Exploration - PR Newswire - February 16th, 2024 [February 16th, 2024]
- Exploring the Potential of Minimoons - Medriva - February 16th, 2024 [February 16th, 2024]
- Understanding the Role of TRISH in Space Health and its Future Innovations - Medriva - February 16th, 2024 [February 16th, 2024]
- The Future of Satellite Refueling and In-Orbit Servicing - Medriva - February 16th, 2024 [February 16th, 2024]
- Japan makes third attempt to launch next-gen rocket - Yahoo Singapore News - February 16th, 2024 [February 16th, 2024]
- Uruguay becomes latest nation to join NASA's Artemis Accords - UPI News - February 16th, 2024 [February 16th, 2024]
- An astronomer's lament: Satellite megaconstellations are ruining space exploration - The Conversation - February 16th, 2024 [February 16th, 2024]
- Asteroid Mining: A Glimpse into the Future - Medriva - February 16th, 2024 [February 16th, 2024]
- Singapore's ESS and Australia's ELA Join Forces to Redefine Space Exploration - BNN Breaking - February 16th, 2024 [February 16th, 2024]
- Intuitive Machines makes history with lunar lander launch - InnovationMap - February 16th, 2024 [February 16th, 2024]
- The Potential of Minimoons in Interplanetary Exploration and Space Travel - Medriva - February 16th, 2024 [February 16th, 2024]
- In space exploration, Switzerland punches above its weight - SWI swissinfo.ch - SWI swissinfo.ch in English - February 16th, 2024 [February 16th, 2024]
- Odysseus Mission: Pioneering the Lunar Economy and Space Exploration - BNN Breaking - February 16th, 2024 [February 16th, 2024]
- Starlab Announces Leadership Team With new CEO and CFO - SpaceWatch.Global - February 16th, 2024 [February 16th, 2024]
- The New Space Race: Private American Companies Compete to Land on the Moon - Medriva - February 16th, 2024 [February 16th, 2024]
- Spectrum Advanced Manufacturing Technologies Plays Vital Role in NASA's Deep Space Optical Communications - PR Newswire - February 16th, 2024 [February 16th, 2024]
- Intuitive Machines Odyssey lunar lander to be launched early Wednesday - The Washington Post - February 16th, 2024 [February 16th, 2024]
- New funding for space projects ensures the UK's role in global missions - Innovation News Network - February 16th, 2024 [February 16th, 2024]
- SpaceX and Intuitive Machines Lunar Mission: Challenges, Expectations, and Future - Medriva - February 16th, 2024 [February 16th, 2024]
- Farewell to a NASA Hero: Apollo Astronaut Thomas K. Mattingly II Departs at 87 - SciTechDaily - November 4th, 2023 [November 4th, 2023]
- NASA Welcomes Netherlands as Newest Artemis Accords Signatory - NASA - November 4th, 2023 [November 4th, 2023]
- China's astronauts tend orbital garden to aid deep space exploration - Space.com - November 4th, 2023 [November 4th, 2023]
- All-UK astronaut mission shows that private enterprise is vital to the ... - The Conversation Indonesia - November 4th, 2023 [November 4th, 2023]
- Former Astronaut Discusses Space Travel and Its Future With ... - Duke Today - November 4th, 2023 [November 4th, 2023]
- Returning astronauts to the moon is NASA's biggest challenge, but ... - Space.com - November 4th, 2023 [November 4th, 2023]
- Australias IPSEC Conference Shows Synergies Between the Space ... - Via Satellite - November 4th, 2023 [November 4th, 2023]
- Mice embryos successfully grown in space, a promising milestone ... - Salon - November 4th, 2023 [November 4th, 2023]
- Dava Newman talks Mars exploration and comunication - CMU The Tartan Online - November 4th, 2023 [November 4th, 2023]
- Science Fact or Fiction: Leveraging Space to Transform Medicine - Kennedy Space Center - November 4th, 2023 [November 4th, 2023]
- Now astronauts will not get lost in space - SAMAA - November 4th, 2023 [November 4th, 2023]
- Sci-fi inspired tractor beams are real, and could solve a major space ... - Livescience.com - November 4th, 2023 [November 4th, 2023]
- Mining Meets Space: Revolutionizing Industry Innovation with Cross ... - AZoMining - November 4th, 2023 [November 4th, 2023]
- Surf's up: Catch an atmospheric wave as ASU research team ... - ASU News Now - November 4th, 2023 [November 4th, 2023]
- Ron Baron says Tesla's valuation can hit $4 trillion, but 'SpaceX has ... - Morningstar - November 4th, 2023 [November 4th, 2023]