Youre walking down a narrow corridor. Someone is walking toward you, so you step to one side. But in that moment, they step to the same side. You make eye contact, grin awkwardly and then, without a word, negotiate a way around each other.
Our lives are full of these delicate social dances. Whether were having a conversation, playing a game or trying to avoid collisions with passersby, our social interactions are reciprocal. My behavior affects your behavior, which in turn affects my behavior.
But until the past few years, research into social cognitionthe psychology of human interactionhas been decidedly non-interactive. Participants looked at images of faces, read short stories about social scenarios, or watched videos of other people interacting. They didnt actually interact with another person.
Take the Sally-Anne task, which is widely used in studies of autism to test theory of mind, the ability to understand other peoples beliefs, intentions and emotions. The participant watches an interaction between two dolls and is asked to predict the behavior of one of the dolls based on an understanding of what the doll believes.
When children with autism answer incorrectly, the assumption is that they have failed to read the dolls mental state and that similar failures explain their difficulties interacting with other people. However, many adults with autism pass this test, and even others that are more challenging, yet still experience severe social difficulties.
These observations clearly demonstrate that traditional tests of social cognition fail to capture key aspects of social interactions, particularly in adults, that are essential to understanding autism.
We need tests that allow us to precisely measure behavior in complex, reciprocal social interactions. To achieve this goal, we and others are investigating the use of virtual-reality technology as a tool for research and, potentially, therapy.
Using these technologies, we have confirmed that problems with joint attentionthe ability to coordinate with someone else so that you are both paying attention to the same thingpersist into adulthood. Weve also gained important insights about the roots of these problems. We also hope that adults with autism can one day practice their social skills within specially designed virtual environments.
In 2013, a team led byLeonhard Schilbach, now at the Max Planck Institute of Psychiatry in Munich, Germany, published anew manifestofor social cognition research. These researchers argued that social cognition should be investigated using a second-person neuroscience approach, in which behavior and brain responses are measured while people engage in reciprocal interactions.
This emerging field offers exciting possibilities for understanding autism. But it also presents serious challenges: Experimental investigations require precise control of the conditions so they can be repeated and manipulated consistently. Achieving this control in the context of a realistic social interaction is far from straightforward.
In a review article published earlier this year, we addressed these issues. We focused on studies of joint attention, which involves both responding to your partner to attract his attention, and initiating joint attention, by guiding that person to an object or location of interest. Joint attention is important in the development of language and social skills, and a delay in its development is one of the mostreliable early signs of autism.
One approach used in a number of studies has been to measure brain responses while participants are engaged in a joint-attention game with another person, either face to face or through a live video feed. However, this relies on the partner behaving consistently for all participants.
Differences in the brain responses of people with and without autism may reflect differences in the neural mechanisms of joint attention. But they could just as well reflect variation in the behavior of the partner, or in the participants sensitivity to other social cues, such as smiles or eyebrow raises, from the partner.
To address these issues, we and other researchers have replaced the human partner with a virtual partner, or avatar, whose behavior is controlled by a computer.
In our own studies, participants interact with an animated virtual character called Alan whose face appears in the center of a computer screen. We use an eye tracker to see where on the screen the participant is looking, and program Alan to respond to her eye movements. This gives us complete control over the interaction.
The participant works with Alan to catch a burglar who is hiding in one of six houses on the screen. Each trial begins with both Alan and the participant searching the houses. If the participant finds the burglar, she initiates joint attention, guiding Alan to the burglar by making eye contact and then looking at the correct house. If, on the other hand, Alan finds the burglar, he initiates joint attention and the participant responds.
The game requires participants to coordinate their behavior with Alan, make use of eye contact and flexibly assume different roles in the joint-attention process.
The search phase of our task also adds a complexity that is absent from other studies. Because Alan makes many eye movements during the trial, the participant has to decide whether a particular eye movement is intended to guide her to the burglar or is simply part of Alans ongoing search.
We have found that participants respond much faster if we remove the search phase so Alans eye movements always indicate the burglars location. This suggests that what we call intention monitoringworking out whether a cue such as an eye movement is intended to be communicativeis an important part of joint attention.
In research published in April, we used this task with a group of adults with autism. Overall, they made slightly more errors than controls did. They were also slower to respond to the avatars eye-gaze cue, but were just as fast as controls when we replaced Alans eye-gaze cues with an arrow pointing to the burglars location.
This finding suggests that the difficulties of adults with autism are specific to the social interaction involved in the task, and cannot be explained by other factors that might affect performancesuch as the ability to orient attention or control eye movements.
Our findings suggest that subtle joint-attention difficulties continue into adulthood, at least for some people with autism. This contrasts with evidence from other studies suggesting that children and adults with autism have no difficulty responding to eye-gaze cues on a computer screen.
We think this may reflect the intention-monitoring component of our task, which makes it more akin to a real-life interaction.
In our research so far, participants have interacted with a virtual character on a computer screen. The next step is to use fully immersive virtual-reality headsets to recreate more realistic social interactions, in which individuals must evaluate multiple social cues at once, including eye gaze, head orientation, hand gestures, speech and facial expressions.
We, among others, are also considering clinical applications of new immersive virtual-reality technologies. Virtual simulations could perhaps be used for social-skills training in which elements of a social interaction are introduced gradually. Virtual meeting spaces could also allow people with and without autism to interact in a safe and controlled environment that reduces anxiety and sensory overload.
Many of the insights in our research have come from adults with autism. Theyve told us how to make our task easier to understand, and theyve described the strategies theyve used to complete the task. Many have told us that although the virtual interaction is cognitively challenging, it is less intimidating and anxiety-provoking than real-life interactions.
Involving people with autism in research is key to its success. As virtual-reality technology improves and becomes increasingly affordable, the possibilities may be limited only by our collective imagination.
This story wasoriginaly publishedonSpectrum.
Read this article:
Virtual Reality May Reveal New Clues About Autism Social Difficulties - Scientific American
- UNL program using virtual reality to enhance research - KOLN - May 13th, 2024 [May 13th, 2024]
- 3D augmented reality with regular glasses | Stanford News - Stanford University News - May 13th, 2024 [May 13th, 2024]
- LG Display's Research Results on OLEDoS for VR Selected as Distinguished Paper at SID Display Week 2024 - PR Newswire - May 13th, 2024 [May 13th, 2024]
- Full-colour 3D holographic augmented-reality displays with metasurface waveguides - Nature.com - May 13th, 2024 [May 13th, 2024]
- The Future of Onboarding: Inside Serviceplan's VR-Based Employee Integration | LBBOnline - Little Black Book - LBBonline - May 13th, 2024 [May 13th, 2024]
- Virtual reality space experience returns to Silver Street Studios - Houston Chronicle - May 13th, 2024 [May 13th, 2024]
- Rain Technology to Unveil New Anamorphic-XR Optical Engine at Display Week 2024 - Display Daily - May 13th, 2024 [May 13th, 2024]
- Virtual reality used as a tool for brain health at assisted living center - Eyewitness News 3 - May 13th, 2024 [May 13th, 2024]
- Virtual Reality in Gaming Market to Reach US$75 Billion by 2028 - Analytics Insight - May 13th, 2024 [May 13th, 2024]
- Disney+ to release VR Marvel series exclusively for Apple Vision Pro - CyberNews.com - May 13th, 2024 [May 13th, 2024]
- Startup aimed at VR entertainment in cars files for insolvency - PhoneArena - May 13th, 2024 [May 13th, 2024]
- After Decades of Promise, the Virtual Reality Era Has Finally Arrived - Kiplinger's Personal Finance - May 13th, 2024 [May 13th, 2024]
- Hahnville senior signs with virtual reality gaming company - St. Charles Herald Guide - May 13th, 2024 [May 13th, 2024]
- Sandbox VR Forges Franchise Partnership with Apparel Group to Pioneer Virtual Reality in the Middle East - Business Wire - May 13th, 2024 [May 13th, 2024]
- Best VR games in 2024 - TechRadar - May 13th, 2024 [May 13th, 2024]
- Researchers rely on Virtual Reality as a complementary therapy for eating disorders - MIXED Reality News - May 13th, 2024 [May 13th, 2024]
- Beyond therapy: Virtual reality shows promise in fighting depression - Medical Xpress - May 13th, 2024 [May 13th, 2024]
- Apparel Group And Sandbox VR Announce Partnership - Retail & Leisure International - Retail & Leisure International - May 13th, 2024 [May 13th, 2024]
- Batman returns to VR - theSun - May 13th, 2024 [May 13th, 2024]
- Virtual Reality Shows Promise in Treating Depression - Neuroscience News - May 13th, 2024 [May 13th, 2024]
- Augmented And Virtual Reality in Manufacturing Market Size, Share And Growth Analysis For 2024-2033 - EIN News - May 13th, 2024 [May 13th, 2024]
- Sandbox VR Announces U.S. Franchise Initiative and International Deal - Franchise Times - May 13th, 2024 [May 13th, 2024]
- MINI Mixed Reality Experience: Driving into the Future. - BMW Press - May 13th, 2024 [May 13th, 2024]
- Virtual reality project will allow users to witness October 7 first hand - The Jewish Chronicle - May 13th, 2024 [May 13th, 2024]
- Spin Rhythm XD getting PlayStation and VR ports this summer - Niche Gamer - May 13th, 2024 [May 13th, 2024]
- Students at the Hong Kong University of Science and Technology use virtual reality headsets in class - Islander News.com - May 13th, 2024 [May 13th, 2024]
- SFU SIAT researcher uses virtual reality to help teen mental health - Simon Fraser University News - May 13th, 2024 [May 13th, 2024]
- The Navy Brings Training to the Fleet With VR - ATD - May 13th, 2024 [May 13th, 2024]
- Manchester Airport launches gaming area with virtual reality escape room Business Traveller - Business Traveller - May 13th, 2024 [May 13th, 2024]
- Can virtual reality help ease the end of life? - Universit de Montral - May 13th, 2024 [May 13th, 2024]
- Fans still waiting for a virtual reality view of the Buffalo Bills' new ... - Buffalo News - November 28th, 2023 [November 28th, 2023]
- Immerse yourself in new, augmented worlds - University of Miami: News@theU - November 28th, 2023 [November 28th, 2023]
- Who are the leading innovators in vehicle digital twins for the ... - just-auto.com - November 28th, 2023 [November 28th, 2023]
- How Virtual and Augmented Reality Are Redefining Construction - TimesTech - November 28th, 2023 [November 28th, 2023]
- Virtualware Appoints VRARA's John A Cunningham to Advisory Board - XR Today - November 28th, 2023 [November 28th, 2023]
- University Project Aims to Dissolve Borders Between Virtual and ... - XR Today - November 28th, 2023 [November 28th, 2023]
- Are real cadavers better than virtual ones? - Slate - November 28th, 2023 [November 28th, 2023]
- Samsung Glasses could be the name of a new pair of Samsung ... - TechRadar - November 28th, 2023 [November 28th, 2023]
- The Best Augmented Reality Accessories for AR Immersion - XR Today - November 28th, 2023 [November 28th, 2023]
- The best VR apps for 2022 | Digital Trends - February 18th, 2023 [February 18th, 2023]
- What is virtual reality? - TechTarget - February 18th, 2023 [February 18th, 2023]
- Forget the metaverse, Sony's PlayStation VR2 will make you a virtual reality believer again - British GQ - February 18th, 2023 [February 18th, 2023]
- Time to play! New virtual reality gaming coming to Northern Virginia - WUSA9.com - January 31st, 2023 [January 31st, 2023]
- Virtual reality training content launched for footwear and leather sector - The Hindu - January 31st, 2023 [January 31st, 2023]
- Texas A&M researchers are using multisensory virtual reality to help NASA astronauts - KAGSTV.com - January 31st, 2023 [January 31st, 2023]
- Hollywoods Virtual Reality Guru and American Millionaire Expanded His Soccer Portfolio to Europe With a Deal Worth $940 Million - EssentiallySports - January 8th, 2023 [January 8th, 2023]
- TikTok owner ByteDance takes first step into virtual reality with ... - December 26th, 2022 [December 26th, 2022]
- Virtual Reality inspires students to think about long-term careers - Turn to 10 - December 21st, 2022 [December 21st, 2022]
- Virtual Reality Arcade with Best VR Game Experiences - Main Event - December 12th, 2022 [December 12th, 2022]
- Virtual Reality - reddit - November 16th, 2022 [November 16th, 2022]
- The Future Of Virtual Reality (VR) - Forbes - November 16th, 2022 [November 16th, 2022]
- Virtual Reality (VR) For Professional Workflows | NVIDIA - October 28th, 2022 [October 28th, 2022]
- Virtual reality in fiction - Wikipedia - October 28th, 2022 [October 28th, 2022]
- Meta will release a new consumer-grade VR headset next year - TechCrunch - October 28th, 2022 [October 28th, 2022]
- Virtual Reality [VR] Market Size & Share | Forecast Report, 2029 - October 25th, 2022 [October 25th, 2022]
- Virtual Reality in the Healthcare Industry - VISUALISE - October 25th, 2022 [October 25th, 2022]
- Virtual Reality vs. Augmented Reality vs. Mixed Reality - Intel - October 25th, 2022 [October 25th, 2022]
- Applications Of Virtual Reality - Virtual Reality Society - October 25th, 2022 [October 25th, 2022]
- The Global Augmented Reality & Virtual Reality In Manufacturing Market size is expected to reach $27.9 billion by 2028, rising at a market growth... - October 25th, 2022 [October 25th, 2022]
- We Met in Virtual Reality Explores the Potential of the Metaverse - Seven Days - October 19th, 2022 [October 19th, 2022]
- What If the Metaverse Is Better Without Virtual Reality? - WIRED - October 19th, 2022 [October 19th, 2022]
- XRHealth and HTC Introduce The First Distraction Therapy Virtual Reality Platform to Improve Patient Experience During Painful or Anxiety Provoking... - October 19th, 2022 [October 19th, 2022]
- Department of Corrections Introduces Virtual Reality Technology to Augment Programming for Incarcerated Parents and Their Children - Pennsylvania... - October 19th, 2022 [October 19th, 2022]
- Arden Bendler Browning conquers virtual reality, Luve More LC + Thinkers Makers Society, and opportunity - theartblog.org - October 19th, 2022 [October 19th, 2022]
- How Virtual or Augmented Reality Can Help Your Packaging Line - Packaging Digest - October 19th, 2022 [October 19th, 2022]
- A marriage of virtual reality and physical therapy: Immergo Labs emerges from its UCSC roots - Lookout Santa Cruz - October 19th, 2022 [October 19th, 2022]
- Senior living residents explore the world through virtual reality - Rocky Mountain PBS - October 19th, 2022 [October 19th, 2022]
- Orthopedic Residency Program and PrecisionOS Team up For Exclusive 3-Year Collaboration: Virtual Reality Development and Curriculum Integration - PR... - October 19th, 2022 [October 19th, 2022]
- 6 Insightful Construction XR Cases to Read in 2022 - XR Today - October 19th, 2022 [October 19th, 2022]
- Video footage emerges from an MMA event that put virtual reality users into the thick of the fighting action - Yahoo News - October 19th, 2022 [October 19th, 2022]
- Carmack: Theres a bunch that Im grumpy about in virtual reality - Ars Technica - October 19th, 2022 [October 19th, 2022]
- The Peripheral Co-Creator on the Shows Line Between Reality & Virtual Reality - TV Insider - October 19th, 2022 [October 19th, 2022]
- Clarkson's Dr. Dana Barry Gave a Presentation at International Conference in Italy - Clarkson University News - October 19th, 2022 [October 19th, 2022]
- Apple VR glove with finger- and hand-tracking patent hints at companion accessories for the "Reality Pro" headset - Notebookcheck.net - October 19th, 2022 [October 19th, 2022]
- Virtual reality spectacle of 2010 sockeye run returns to Shuswap Salmon Arm Observer - Salmon Arm Observer - October 19th, 2022 [October 19th, 2022]
- The virtual reality push brings together Microsoft, Meta and the art of war - Stuff - October 19th, 2022 [October 19th, 2022]
- What Is Virtual Reality All About? - Oculus VR - September 20th, 2022 [September 20th, 2022]
- What Is Virtual Reality? | Tech & Learning - September 20th, 2022 [September 20th, 2022]
- 10 Advantages And Disadvantages Of Virtual Reality (2022) - September 20th, 2022 [September 20th, 2022]
- Insights on the Tactile Virtual Reality Global Market Research Report to 2027 - Growing Advancements in Virtual Reality Products & Rising... - September 20th, 2022 [September 20th, 2022]