The Lynx R1 MR headset.
XR typically refers to the spectrum of spatial technologies that cover the gamut between virtual (VR) reality and augmented reality (AR). The world is largely familiar with the concept of VR, the complete immersion in a digital space. It helps that VR is already available on consumer platforms such as Oculus Quest and PlayStation VR. Less understood is AR, which seeks to enhance the real world by overlaying the digital world on top it. Even within these two categories of XR, there are additional levels of immersionsome smart glasses feature a single monochrome display. Others, such as Hololens or Magic Leap, provide more advanced iterations of AR.
While many are familiar with the concept of VR, the market and the world are still becoming acquainted with AR. While VR is already available on consumer platforms such as Oculus Quest and PlayStation VR, consumer AR that everyone wants is still years away. People want AR glasses to have the graphical capabilities of todays Oculus Quest, inside the fashionable form factor of todays Ray Ban Stories. However, thats simply not possible with todays optical, wireless, battery and processing capabilities. Qualcomm is at the forefront of enabling headworn AR and VR solutions, including the Oculus Quest, Ray Ban Stories, and many other headsets.
I recently spoke with Qualcomms GM of XR, Hugo Swart about the companys vision and foundational work on Mixed Reality. The company has been building chipsets and platforms for XR since the early days and had one of the earliest AR platforms called Vuforia in 2012 and enabled pioneers in headworn AR like ODG. Qualcomm Technologies were also pivotal in enabling early smartphone VR solutions as well as the first Standalone VR headsets including the wildly popular Oculus Quest 2.
Why we need Mixed Reality
In the meantime, technologies like RGB video pass-through AR or Mixed Reality are bridging the gap on the way to AR. You may have heard Microsoft use the term Mixed Reality to describe its XR platform. While this is similar, in the context of this column, Mixed Reality refers to enabling both AR and VR within a single VR device. Microsofts definition refers to different devices sharing the same spatial computing platform and potentially space. In addition to possessing VR capabilities, Qualcomms vision for a Mixed Reality device must be able to use passthrough color cameras to recreate a feed as close to the real world as possible for the purpose of AR.
Mixed Reality capable VR devices must also be lighter and more compact than current VR devicesthey need to feel almost invisible to the user much like a pair of AR glasses would be. Were already seeing this in devices targeted towards Mixed Reality, such as the Lynx R-1 which runs on Qualcomm Snapdragon XR2 chipset. I expect that well see similar devices from Facebook, Apple and other tech giants pursuing AR very soon
Right now, Qualcomm believes there are four paths within AR that are evolving in parallel, on different parts of the XR spectrum:
Eventually, these capabilities will merge to the point where it becomes difficult to tell the difference between each category. The lines will become very blurry, like what were seeing with VR today.
Mixed Reality features
In speaking with Hugo I learned that there are a common series of challenges that must be overcome in order to fully enable this new form of spatial computing. First, Mixed Reality headsets require the lowest latency possible along the entire image processing pipeline, since they must capture whats happening in the real world, process it, render it and then display it inside of the headset, in almost real-time. If things are too out of sync, the user will feel like they are drunk or have motion sickness. There cannot be any weak links in the chain, not even in the display. Every millisecond matters in Mixed Reality, even more than in VR or AR. This is a problem that Qualcomm recognizes and is working to mitigate. According to Hugo Swart, the companys GM of XR, Qualcomm has managed to bring photon-to-photon latency down to under 10 milliseconds from the previously standard 30ms+.
Second, Mixed Reality needs to have a deep understanding of the environment and be able to utilize AI algorithms to recognize objects and map the users surroundings. This is what allows the headset to overlay digital information over the real-world environment for AR functions. Additionally, even if the user is using the headset in VR mode, it needs to be able to help with boundaries and object avoidance. Snapdragon XR2 chipset, which is the standard for todays XR headsets, has these capabilities built-in.
In addition to low-latency and environmental awareness, a Mixed Reality device needs to have enhanced interactivity. In other words, users should be able to interact in AR, VR and MR experiences using different user interfaces and control schemesfor example, hand tracking for base-level interactions and controllers for more intensive applications like gaming and creative work.
A Mixed Reality device must also consider how the user interacts with both virtual and physical objects while inside of the headset and what that might mean for the users immersion. For example, it will need to understand the differences between different types of objects, applying the proper level of opacity to them depending on their degree of relevance. When you look at the previous generation of XR chipsets, the Snapdragon XR2 increased AI performance by 11 times, which is crucial in enabling such features.
Last but certainly not least is the fact that Mixed Reality headsets require a fundamentally better field of view (FoV) than most AR headsets, which typically feature between 30 and 50 degrees. This is because bending light is difficult, and these devices light engines are limited by what can fit inside their necessarily lightweight and compact form factors. Todays technology simply cannot produce anything beyond 50 degrees, for the most part. There are a few exceptions, but these solutions tend to be more appropriate for engineering environments versus consumer, due to their size and weight.
One workaround to this bottleneck is RGB pass-through technology, which delivers the FoV that users expect, on a much more accessible timeline. Snapdragon XR2 chipset was specifically designed to support up to 7 different simultaneous camera streams including ultra-low latency processing to enable a lag-free RGB passthrough experience. Qualcomms powerful Adreno is designed to enable the higher resolutions of Mixed Reality displays to mitigate the loss of visual fidelity that results from an increase in FoV. People want access to AR and MR today, not in three to five years when lightweight AR headsets finally support an acceptable field of view and levels of performance. For AR to be accessible, Mixed Reality headsets must flourish. This will allow developers to build AR experiences on todays available hardware for tomorrows AR headsets.
Mixed Reality: a consumer or enterprise technology?
With how much Mixed Reality borrows from both VR and AR, it would not be surprising for it to inherit many of the same use cases. In fact, consumer and enterprise use cases for AR and VR translate almost perfectly to Mixed Reality. However, there are some use cases that are even better in Mixed Reality than VR or AR by themselves.
One of those is enterprise training. Imagine being able to take advantage of Mixed Realitys wide FoV, overlaying graphics onto, but still very much seeing the real world. This makes Mixed Reality a powerful proposition for job training.
There are additional Mixed Reality applications already in use in the enterprise, such as remote assistance and guided work instructions. However, these stand to become much more powerful and capable than the simple 2D head-mounted solutions we have today. There are also consumer use cases aplenty, since a user can turn virtually any surface in their home into a gigantic display without losing their real-world spatial and situational awareness.
Final thoughts
Many in the industry agree the convergence of AR and VR is inevitable. Mixed Reality is still in its early days, but much of the XR industry appears to be moving in that direction in the near term. Mixed Reality will not only benefit from the technological developments of AR and VR, but also facilitate them at the same time. There will be many users who will want a single, Mixed Reality device to cover the gamut of XR. Qualcomms technological innovations inside of the XR2 are at the core of making that possible today. However, not everyone will want this unified device and may prefer to only have AR or only have VR. Users want choice and hopefully the market will give them that with a diversity of XR devices and approaches. Much like the XR headsets and Android smartphones of today, many will likely be powered by Qualcomm technologies.
Disclosure:My firm, Moor Insights & Strategy, like all research and analyst firms, provides or has provided research, analysis, advising, and/or consulting to many high-tech companies in the industry, including Qualcomm. I do not hold any equity positions with any companies cited in this column
Read more:
Qualcomm And Mixed Reality: The Next Step In The Evolution Of XR? - Forbes
- Days of our Lives' Suzanne Rogers on the Evolution of Maggie: "She Knows Who She Is Now, and She's Not Relying ... - Michael Fairman TV - March 14th, 2024 [March 14th, 2024]
- Kylie Jenner Talks About Her Style Evolution - The Cut - March 14th, 2024 [March 14th, 2024]
- Equator Coffees Unveils New Packaging Design, Reflecting Brand Evolution & Vision For The Future - Sprudge - March 14th, 2024 [March 14th, 2024]
- Rosewood Hotel Group Accelerates Growth And Evolution Across Its Four Distinctive Brands - Hospitality Net - March 14th, 2024 [March 14th, 2024]
- Thomson Reuters Unveils New Brand Evolution - Adweek - March 14th, 2024 [March 14th, 2024]
- Is It Becoming Acceptable to Speak of Design? - Discovery Institute - March 14th, 2024 [March 14th, 2024]
- Did Charles Darwin Convert to Christianity and Discredit Evolution on His Deathbed? - Snopes.com - March 14th, 2024 [March 14th, 2024]
- Milk, it's not just for mammals: An amphibian makes it too - NPR - March 14th, 2024 [March 14th, 2024]
- Discover Puerto Rico Debuts Evolution of Its Successful 'Live Boricua' Brand Campaign Aimed at Engaging Visitors ... - Yahoo Finance - March 14th, 2024 [March 14th, 2024]
- A Journey Through Time: The Evolution of Ras Al Khaimah Art - Business Wire - March 14th, 2024 [March 14th, 2024]
- Empowering Women: The Evolution and Innovation of coto Social Platform - CXOToday.com - March 14th, 2024 [March 14th, 2024]
- The Evolution of Da'Vine Joy Randolph - The Root - March 14th, 2024 [March 14th, 2024]
- Study on mating behaviors offers clues into the evolution of attraction - Phys.org - March 14th, 2024 [March 14th, 2024]
- Dragonball Evolutions live-action Goku says goodbye to Toriyama: Sorry we messed up - AS USA - March 14th, 2024 [March 14th, 2024]
- Investec, evolution of SMEs in the materials handling sector - Leasing Life - March 14th, 2024 [March 14th, 2024]
- Pride & Prejudice and the evolution of the female gaze on screen - Yahoo News UK - March 6th, 2024 [March 6th, 2024]
- Joe Wong's Musical Evolution - Shepherd Express - March 6th, 2024 [March 6th, 2024]
- A global survey of prokaryotic genomes reveals the eco-evolutionary pressures driving horizontal gene transfer - Nature.com - March 6th, 2024 [March 6th, 2024]
- Redefining Intelligence: Chimpanzees Break Through the Cultural Evolution Barrier - Medriva - March 6th, 2024 [March 6th, 2024]
- Mollusk Eyes Reveal How Future Evolution Depends on the Past - Quanta Magazine - March 6th, 2024 [March 6th, 2024]
- Levy Delves Into the Evolution of ADCs in NSCLC - OncLive - March 6th, 2024 [March 6th, 2024]
- The Snake Is The Spearhead of Reptile Evolution, But Why? - ScienceAlert - March 6th, 2024 [March 6th, 2024]
- 'A very special day: Birds linked to Darwins theory of evolution reintroduced to Galapagos Islands - Euronews - March 6th, 2024 [March 6th, 2024]
- Why the Powerhouses of Cells Evolve Differently in Plants - College of Natural Sciences - March 6th, 2024 [March 6th, 2024]
- Driving the DevOps Evolution: ArgoCD, Tekton and Seamless Migrations - DevOps.com - March 6th, 2024 [March 6th, 2024]
- Finding the Balance: The Evolution of Public Health Guidance Amidst Controversy - Medriva - March 6th, 2024 [March 6th, 2024]
- Insider Podcast: Paolini dishes on her Polish roots and hard-court evolution - WTA Tennis - March 6th, 2024 [March 6th, 2024]
- Interview: Sara Gruen and Rick Elice Talk About the Inspiration and Evolution of the New Musical Water for Elephants - TheaterMania.com - March 6th, 2024 [March 6th, 2024]
- The Evolution of the Laravel Welcome Page - Laravel News - March 6th, 2024 [March 6th, 2024]
- A Serpentine 'Explosion' 125 Million Years Ago Primed Snakes for Rapid, Diverse Evolution - Smithsonian Magazine - March 6th, 2024 [March 6th, 2024]
- The Evolution of Modern Technologies in Car Development - FinSMEs - March 6th, 2024 [March 6th, 2024]
- Milwaukee Transformed: From Bronzeville to Veterans Park, Aerial Timelapses Reveal City's Evolution - BNN Breaking - March 6th, 2024 [March 6th, 2024]
- The eyes are a gateway to evolution of daddy longlegs at least. - University of Wisconsin-Madison - March 6th, 2024 [March 6th, 2024]
- Adrian Newey: RB20 is the next step in Red Bull's design evolution - PlanetSport - March 6th, 2024 [March 6th, 2024]
- LiveScore releases its 'Evolution of Fan' report - Gambling Insider - March 6th, 2024 [March 6th, 2024]
- The loyalty program evolution makes its way to the full-service restaurant category - Nation's Restaurant News - March 6th, 2024 [March 6th, 2024]
- Teenage Mutant Ninja Turtles: The Last Ronin II - Re-Evolution #1 spoiler-free review: goes hard on the action, but ... - Gamesradar - March 6th, 2024 [March 6th, 2024]
- Exploring U.S. Financial Evolution: DAR Hosts Talk on Federal Reserve History in Thomasville - BNN Breaking - March 6th, 2024 [March 6th, 2024]
- Why cloud evolution needs a cohesive approach to succeed - CIO - March 6th, 2024 [March 6th, 2024]
- Gilead Sciences CEO on Company's Evolution and Commitment to the Bay Area - BioSpace - March 6th, 2024 [March 6th, 2024]
- Navigating the AI Quandary: Human Supremacy vs Machine Intelligence Evolution - BNN Breaking - March 6th, 2024 [March 6th, 2024]
- Denis Villeneuve breaks down the evolution of sandworms in 'Dune: Part Two' - Mashable - March 6th, 2024 [March 6th, 2024]
- Continued evolution of law improves governing capacity - Chinadaily.com.cn - China Daily - March 6th, 2024 [March 6th, 2024]
- The Evolution of the DEX Space with dYdX's CEO Antonio Juliano - Blockster - March 6th, 2024 [March 6th, 2024]
- Quick Commerce Evolution: 3PL Firms Aim for Same Day Delivery, Chasing Blinkit and Zepto's Lead - BNN Breaking - March 6th, 2024 [March 6th, 2024]
- What If...? Star Jeffrey Wright Addreses the Watcher's Evolution and 'Epic' Season 2 Finale - CBR - Comic Book Resources - December 31st, 2023 [December 31st, 2023]
- Evolution of the Connected Autonomous Vehicle - Ward's Auto - December 31st, 2023 [December 31st, 2023]
- A project to capture the evolution of human culture. - Psychology Today - December 31st, 2023 [December 31st, 2023]
- The Evolution of a Digital Soul. Beyond Code: A Journey of Heart and | by Mark Randall Havens | Dec, 2023 - Medium - December 31st, 2023 [December 31st, 2023]
- 4 Clues That Reid Is Finally Returning In Criminal Minds: Evolution Season 2 - Screen Rant - December 31st, 2023 [December 31st, 2023]
- Evolution of Samoyed and Kitten's Friendship Delights Internet: 'Wholesome' - Newsweek - December 31st, 2023 [December 31st, 2023]
- Crypto Evolution: Pullix (PLX) vs OKB (OKB) & KuCoin (KCS) - Crypto Reporter - December 31st, 2023 [December 31st, 2023]
- Alfa Romeos mediocre F1 season heralded its era of evolution: Prime Tire - The Athletic - December 31st, 2023 [December 31st, 2023]
- Beyond The Uniform: 10 Years of Evolution in SYNC Performance's Custom Program - SkiRacing.com - December 31st, 2023 [December 31st, 2023]
- Why SZA's evolution into a popstar has earned her recognition as artist of the year - Salon - December 31st, 2023 [December 31st, 2023]
- AI in 2023 Rises, Falls and Evolution - Finance Magnates - December 31st, 2023 [December 31st, 2023]
- Indonesia's Indosat pursues evolution from telecom to tech company - Nikkei Asia - December 31st, 2023 [December 31st, 2023]
- EdTech Evolution: 3 Stocks Educating the Next Generation - InvestorPlace - December 31st, 2023 [December 31st, 2023]
- Informa Tech Interview with Huawei about voice evolution and innovations at 5G Core Summit 2023 - Informa Tech ... - Light Reading - December 31st, 2023 [December 31st, 2023]
- Looking ahead: What will the DeFi evolution look like in 2024? - Ledger Insights - Ledger Insights - December 31st, 2023 [December 31st, 2023]
- Why Cat Bohannon wrote 'Eve, How the Female Body Drove 200 Million Years of Human Evolution' | India News ... - IndiaTimes - December 31st, 2023 [December 31st, 2023]
- The smart-design evolution of the laboratory space - pharmaphorum - December 31st, 2023 [December 31st, 2023]
- The WILD Evolution of Teenage Mutant Ninja Turtles TMNT (VIDEO) - FandomWire - December 31st, 2023 [December 31st, 2023]
- The supernatural invades American museums via indigenous artifacts - Why Evolution Is True - December 31st, 2023 [December 31st, 2023]
- Baleen Whales First Evolved Large Body Size in Cold Southern Waters, New Fossil Shows - Sci.News - December 31st, 2023 [December 31st, 2023]
- The Evolution of Identity in Taiwan The Diplomat - The Diplomat - December 31st, 2023 [December 31st, 2023]
- From the Archive: The Evolution Of Hockey Pools - The Hockey News - December 31st, 2023 [December 31st, 2023]
- 'X-Men: Evolution' Is Better Than 'X-Men: The Animated Series' - Collider - December 31st, 2023 [December 31st, 2023]
- Unveiling the Silver Screen: The Evolution of Celebrity Nudity in Cinema - The Hype Magazine - December 31st, 2023 [December 31st, 2023]
- Are Humans Still Evolving? 'Maybe More Rapidly Than Ever,' Says Scientist - Newsweek - December 31st, 2023 [December 31st, 2023]
- The Intersection of Real Estate and Fintech: Evolution, Impact of Policies, and Global Dynamics - CXOToday.com - December 31st, 2023 [December 31st, 2023]
- Kyle Richards' Style Evolution: Her Best Looks - Us Weekly - December 31st, 2023 [December 31st, 2023]
- Criminal Minds: Evolution Season 2's "Deeper Secrets" Teased By Aisha Tyler - Screen Rant - December 31st, 2023 [December 31st, 2023]
- Saturday: Hili dialogue Why Evolution Is True - Why Evolution Is True - December 31st, 2023 [December 31st, 2023]
- NBA 2K24 MyTEAM New Year Resolution Adds 14 Evolution Cards - ClutchPoints - December 31st, 2023 [December 31st, 2023]
- dive into the history of NASA's logo evolution from the space ... - Designboom - November 8th, 2023 [November 8th, 2023]
- Resolving the puzzle of same-sex sexual interactions - Nature.com - November 8th, 2023 [November 8th, 2023]
- The History and Evolution of Black Friday And How It Got Its Name - Yahoo Life - November 8th, 2023 [November 8th, 2023]
- Evolution of Terran R, with Tim Ellis (Relativity Space) - Payload - November 8th, 2023 [November 8th, 2023]
- Brownell Raves About Breakout Junior's Evolution - The Clemson Insider - November 8th, 2023 [November 8th, 2023]