A virtual operating theatre is helping train up surgeons on new procedures.
Virtual reality is often touted as a way of creating fantasy universes, but it could also turn out to be an effective way of teaching skills that are hard to practice in the real world.
Take training up the doctors of tomorrow, for example. US university Case Western has already announced it plans to do away with its anatomy labs, and the cadavers that go with them, and teach medical students with Microsoft's HoloLens 'mixed reality' system instead. Aspiring doctors will be able to wear HoloLens headsets, and view the different layers of a body -- skin, muscle, blood vessels, and so on -- in 3D.
But going one step further, one UK company is trying to recreate the hands-on aspects of surgery in a VR setting, allowing students to get a sense of how the human body feels in using haptic feedback.
Fundamental VR, based in London and Guildford, has added a haptics element to virtual reality to allow medics to train without having to test out their nascent skills on an actual patient.
The system combines the HoloLens headset and the company's software with a stylus connected to a standard-issue mechanical arm.
The stylus appears as a syringe in the VR world the wearer sees, with one button to empty the syringe, and another to refill it.
Moving the stylus in the real world moves the syringe in the simulation, and when the virtual needle meets the virtual skin, flesh, or bone, the varying resistance of the material is transmitted through the stylus to the user, giving them a powerful facsimile of a real-live body.
The idea is that encountering different elements of the body -- like fat or bone -- should feel very different.
The first system, set up to resemble a total knee arthroscopy, was custom-built for the drug company Pacera to teach clinicians how to do a procedure using one of its products, an anaesthetic called Exparel.
Unlike traditional most anaesthetics, where a larger dose is injected in one go and spreads out widely from the injection site, Exparel is injected in several doses and stays largely where it's put. For some surgeons, the change in procedure was difficult to grasp, and so the VR teaching tool was born.
The imagery for the system was created by taking a series of photos of a knee to build up its 3D counterpart. Off-the-shelf haptic hardware is used to stand in for the syringe in the Pacera system, but could equally act as any medical tool that's needed.
Surgeons weigh in
Building a system that could faithfully recreate the experience of surgery required a mixture of human and technological smarts. In order to build the VR setup for knee replacement surgery, the company canvassed the opinions of orthopaedic surgeons on the steps that make up each procedure.
"Surgery is about science, but also about art, and where there's art, there's opinion. Getting to a common standard where people agree what's the right way to do that and on best practice, that took some time. Once we had got that, we were ready to start embracing some of the challenges of texture and tissue types and how those change throughout the procedure," Richard Vincent, cofounder of Fundamental VR, told ZDNet.
Next the surgeons were enlisted to help convert the real-life experience of surgery into a virtual version, with the company's haptics development engine the bridging the real and VR world.
"We built a calibration tool: it's the core of our software that allows us to quickly translate what are quite difficult things to communicate into numbers... we started with people saying 'it's like sticking a needle into an orange' or 'it's like chicken', then you can basically adjust it in real time until they agree it's how they feel and average that out," Vincent said.
Dr Stan Dysart, an orthopaedic surgeon that specialises in joint replacements at Georgia-based Pinnacle Orthopaedics, was among the surgeons that contributed their first-person perspective of knee operations to help the system recreate the authentic feel of surgery, assigning each element of the human body a number that corresponds to a certain texture.
"The haptic device has a scoring system, and I helped them decide what a needle feels like in capsule, what it feels like in muscle, in fat, in periosteum, and what it feels like on bone," he said. Fat, for example, is extremely forgiving, while the capsule has a fibrous, plastic-like texture.
"The capsule has a certain resistance, and when you go through the capsule, resistance releases, so you can score that -- you can score that [level of haptic] feedback, and score a different feedback for every part of the knee. You give it a number, and computers understand numbers -- the higher the number, the greater the resistance the surgeon will feel," Dysart said.
Once the surgeons have agreed on the haptic-feedback rating for each layer of the body, from muscle to bone, the haptics system can translate that back into the level of feedback the VR wearer will feel when they apply the virtual syringe -- a matter of balancing the amount of processing that the scenario needs with the abilities of the GPU underpinning the system to produce a smooth experience for the user.
The total knee arthroscopy-related system is already being used by surgeons in centres across the US, and it's helping surgeons refine their techniques, according to Dysart.
"Surgeons love it. They enjoy the experience, they enjoy practicing without potentially damaging a live patient. That's where it's important. Everything we do in live surgery has a consequence -- how deep do you cut? where do you cut? where do you inject? -- because there are nerves and arteries all about the knee.
"In virtual reality, if you plunge the needle too deeply, nothing is injured. You realise you've done it incorrectly, and you can do it over and over until you have the right technique. That's the beauty of this," he said.
Alongside the total knee arthroscopy, Fundamental VR has three more custom setups in the pipeline including a soft tissue and a spine procedure that it expects will go live at some point this summer.
Teaching tool
Fundamental VR is already talking to educational institutions about how haptics-based systems could be used to teach students to improve their skills or help established doctors learn new procedures before they try them out on the wards. For now, Fundamental VR is concentrating on the US market, though it has had conversations with teaching facilities both in London and abroad.
As well as building more specific one-off systems for clients in future, the company also expects to create a library of common procedures that can be accessed on a subscription basis. Long with 'standard' anatomy, the company could potentially create variants to introduce students to some of the rarer anatomical variations or conditions.
"[Removal of] the appendix is still the most performed operation, so having a better way of teaching that would be useful for lots of people, but on the flip side, there's a lot of opportunity [for doctors] to be around that and observe that," Fundamental VR's Vincent said.
"But if you go into neurology, there may be something you only see three times a year, but it's a life-and-death situation. The number of people that need that training is much less and it might be harder to make the business case, but the human case in much stronger," he continued.
While haptics system might not go over well with all surgeons -- some more senior clinicians found the simulations a bit too close to computer gaming -- the company prefers to liken it to the way pilots use flight simulators.
"We go to lots of conferences where we talk to lots of surgeons about how, say, when you face this bleed at this moment, and you've got five minutes to deal with it, it's never going to make that a less traumatic moment when it happens, but if you go through a simulator that gets you close to it a few times, that has got to be good thing," Vincent explained.
Practicing on VR, not patients
Once mixed and virtual reality become cheaper and more common, haptics and VR could be used to create models of individual patients before they undergo surgery.
"If you could create from those scans something where we could share the kidney, move it around, agree how to get in there, what's the plan, how do we make the surgery the quickest and most effective, that would be good for patient safety," Chris Scattergood, Fundamental VR's co-founder, said.
The future of medical VR, then, will be a mix of teaching students and professionals how to do high-volume, routine operations of the kind that are done hour after hour in hospitals across the world, as well as to understand niche procedures that clinicians at the highest level may only see once or twice in their lives.
Either way, while doctors are practising the skills they need to perform the procedures, they'll be learning virtually, making their mistakes away on a computer system and perfecting their techniques long before they get to use them on their patients.
Visit link:
Surgery in virtual reality: How VR could give trainee doctors the feel of real patients - ZDNet
- UNL program using virtual reality to enhance research - KOLN - May 13th, 2024 [May 13th, 2024]
- 3D augmented reality with regular glasses | Stanford News - Stanford University News - May 13th, 2024 [May 13th, 2024]
- LG Display's Research Results on OLEDoS for VR Selected as Distinguished Paper at SID Display Week 2024 - PR Newswire - May 13th, 2024 [May 13th, 2024]
- Full-colour 3D holographic augmented-reality displays with metasurface waveguides - Nature.com - May 13th, 2024 [May 13th, 2024]
- The Future of Onboarding: Inside Serviceplan's VR-Based Employee Integration | LBBOnline - Little Black Book - LBBonline - May 13th, 2024 [May 13th, 2024]
- Virtual reality space experience returns to Silver Street Studios - Houston Chronicle - May 13th, 2024 [May 13th, 2024]
- Rain Technology to Unveil New Anamorphic-XR Optical Engine at Display Week 2024 - Display Daily - May 13th, 2024 [May 13th, 2024]
- Virtual reality used as a tool for brain health at assisted living center - Eyewitness News 3 - May 13th, 2024 [May 13th, 2024]
- Virtual Reality in Gaming Market to Reach US$75 Billion by 2028 - Analytics Insight - May 13th, 2024 [May 13th, 2024]
- Disney+ to release VR Marvel series exclusively for Apple Vision Pro - CyberNews.com - May 13th, 2024 [May 13th, 2024]
- Startup aimed at VR entertainment in cars files for insolvency - PhoneArena - May 13th, 2024 [May 13th, 2024]
- After Decades of Promise, the Virtual Reality Era Has Finally Arrived - Kiplinger's Personal Finance - May 13th, 2024 [May 13th, 2024]
- Hahnville senior signs with virtual reality gaming company - St. Charles Herald Guide - May 13th, 2024 [May 13th, 2024]
- Sandbox VR Forges Franchise Partnership with Apparel Group to Pioneer Virtual Reality in the Middle East - Business Wire - May 13th, 2024 [May 13th, 2024]
- Best VR games in 2024 - TechRadar - May 13th, 2024 [May 13th, 2024]
- Researchers rely on Virtual Reality as a complementary therapy for eating disorders - MIXED Reality News - May 13th, 2024 [May 13th, 2024]
- Beyond therapy: Virtual reality shows promise in fighting depression - Medical Xpress - May 13th, 2024 [May 13th, 2024]
- Apparel Group And Sandbox VR Announce Partnership - Retail & Leisure International - Retail & Leisure International - May 13th, 2024 [May 13th, 2024]
- Batman returns to VR - theSun - May 13th, 2024 [May 13th, 2024]
- Virtual Reality Shows Promise in Treating Depression - Neuroscience News - May 13th, 2024 [May 13th, 2024]
- Augmented And Virtual Reality in Manufacturing Market Size, Share And Growth Analysis For 2024-2033 - EIN News - May 13th, 2024 [May 13th, 2024]
- Sandbox VR Announces U.S. Franchise Initiative and International Deal - Franchise Times - May 13th, 2024 [May 13th, 2024]
- MINI Mixed Reality Experience: Driving into the Future. - BMW Press - May 13th, 2024 [May 13th, 2024]
- Virtual reality project will allow users to witness October 7 first hand - The Jewish Chronicle - May 13th, 2024 [May 13th, 2024]
- Spin Rhythm XD getting PlayStation and VR ports this summer - Niche Gamer - May 13th, 2024 [May 13th, 2024]
- Students at the Hong Kong University of Science and Technology use virtual reality headsets in class - Islander News.com - May 13th, 2024 [May 13th, 2024]
- SFU SIAT researcher uses virtual reality to help teen mental health - Simon Fraser University News - May 13th, 2024 [May 13th, 2024]
- The Navy Brings Training to the Fleet With VR - ATD - May 13th, 2024 [May 13th, 2024]
- Manchester Airport launches gaming area with virtual reality escape room Business Traveller - Business Traveller - May 13th, 2024 [May 13th, 2024]
- Can virtual reality help ease the end of life? - Universit de Montral - May 13th, 2024 [May 13th, 2024]
- Fans still waiting for a virtual reality view of the Buffalo Bills' new ... - Buffalo News - November 28th, 2023 [November 28th, 2023]
- Immerse yourself in new, augmented worlds - University of Miami: News@theU - November 28th, 2023 [November 28th, 2023]
- Who are the leading innovators in vehicle digital twins for the ... - just-auto.com - November 28th, 2023 [November 28th, 2023]
- How Virtual and Augmented Reality Are Redefining Construction - TimesTech - November 28th, 2023 [November 28th, 2023]
- Virtualware Appoints VRARA's John A Cunningham to Advisory Board - XR Today - November 28th, 2023 [November 28th, 2023]
- University Project Aims to Dissolve Borders Between Virtual and ... - XR Today - November 28th, 2023 [November 28th, 2023]
- Are real cadavers better than virtual ones? - Slate - November 28th, 2023 [November 28th, 2023]
- Samsung Glasses could be the name of a new pair of Samsung ... - TechRadar - November 28th, 2023 [November 28th, 2023]
- The Best Augmented Reality Accessories for AR Immersion - XR Today - November 28th, 2023 [November 28th, 2023]
- The best VR apps for 2022 | Digital Trends - February 18th, 2023 [February 18th, 2023]
- What is virtual reality? - TechTarget - February 18th, 2023 [February 18th, 2023]
- Forget the metaverse, Sony's PlayStation VR2 will make you a virtual reality believer again - British GQ - February 18th, 2023 [February 18th, 2023]
- Time to play! New virtual reality gaming coming to Northern Virginia - WUSA9.com - January 31st, 2023 [January 31st, 2023]
- Virtual reality training content launched for footwear and leather sector - The Hindu - January 31st, 2023 [January 31st, 2023]
- Texas A&M researchers are using multisensory virtual reality to help NASA astronauts - KAGSTV.com - January 31st, 2023 [January 31st, 2023]
- Hollywoods Virtual Reality Guru and American Millionaire Expanded His Soccer Portfolio to Europe With a Deal Worth $940 Million - EssentiallySports - January 8th, 2023 [January 8th, 2023]
- TikTok owner ByteDance takes first step into virtual reality with ... - December 26th, 2022 [December 26th, 2022]
- Virtual Reality inspires students to think about long-term careers - Turn to 10 - December 21st, 2022 [December 21st, 2022]
- Virtual Reality Arcade with Best VR Game Experiences - Main Event - December 12th, 2022 [December 12th, 2022]
- Virtual Reality - reddit - November 16th, 2022 [November 16th, 2022]
- The Future Of Virtual Reality (VR) - Forbes - November 16th, 2022 [November 16th, 2022]
- Virtual Reality (VR) For Professional Workflows | NVIDIA - October 28th, 2022 [October 28th, 2022]
- Virtual reality in fiction - Wikipedia - October 28th, 2022 [October 28th, 2022]
- Meta will release a new consumer-grade VR headset next year - TechCrunch - October 28th, 2022 [October 28th, 2022]
- Virtual Reality [VR] Market Size & Share | Forecast Report, 2029 - October 25th, 2022 [October 25th, 2022]
- Virtual Reality in the Healthcare Industry - VISUALISE - October 25th, 2022 [October 25th, 2022]
- Virtual Reality vs. Augmented Reality vs. Mixed Reality - Intel - October 25th, 2022 [October 25th, 2022]
- Applications Of Virtual Reality - Virtual Reality Society - October 25th, 2022 [October 25th, 2022]
- The Global Augmented Reality & Virtual Reality In Manufacturing Market size is expected to reach $27.9 billion by 2028, rising at a market growth... - October 25th, 2022 [October 25th, 2022]
- We Met in Virtual Reality Explores the Potential of the Metaverse - Seven Days - October 19th, 2022 [October 19th, 2022]
- What If the Metaverse Is Better Without Virtual Reality? - WIRED - October 19th, 2022 [October 19th, 2022]
- XRHealth and HTC Introduce The First Distraction Therapy Virtual Reality Platform to Improve Patient Experience During Painful or Anxiety Provoking... - October 19th, 2022 [October 19th, 2022]
- Department of Corrections Introduces Virtual Reality Technology to Augment Programming for Incarcerated Parents and Their Children - Pennsylvania... - October 19th, 2022 [October 19th, 2022]
- Arden Bendler Browning conquers virtual reality, Luve More LC + Thinkers Makers Society, and opportunity - theartblog.org - October 19th, 2022 [October 19th, 2022]
- How Virtual or Augmented Reality Can Help Your Packaging Line - Packaging Digest - October 19th, 2022 [October 19th, 2022]
- A marriage of virtual reality and physical therapy: Immergo Labs emerges from its UCSC roots - Lookout Santa Cruz - October 19th, 2022 [October 19th, 2022]
- Senior living residents explore the world through virtual reality - Rocky Mountain PBS - October 19th, 2022 [October 19th, 2022]
- Orthopedic Residency Program and PrecisionOS Team up For Exclusive 3-Year Collaboration: Virtual Reality Development and Curriculum Integration - PR... - October 19th, 2022 [October 19th, 2022]
- 6 Insightful Construction XR Cases to Read in 2022 - XR Today - October 19th, 2022 [October 19th, 2022]
- Video footage emerges from an MMA event that put virtual reality users into the thick of the fighting action - Yahoo News - October 19th, 2022 [October 19th, 2022]
- Carmack: Theres a bunch that Im grumpy about in virtual reality - Ars Technica - October 19th, 2022 [October 19th, 2022]
- The Peripheral Co-Creator on the Shows Line Between Reality & Virtual Reality - TV Insider - October 19th, 2022 [October 19th, 2022]
- Clarkson's Dr. Dana Barry Gave a Presentation at International Conference in Italy - Clarkson University News - October 19th, 2022 [October 19th, 2022]
- Apple VR glove with finger- and hand-tracking patent hints at companion accessories for the "Reality Pro" headset - Notebookcheck.net - October 19th, 2022 [October 19th, 2022]
- Virtual reality spectacle of 2010 sockeye run returns to Shuswap Salmon Arm Observer - Salmon Arm Observer - October 19th, 2022 [October 19th, 2022]
- The virtual reality push brings together Microsoft, Meta and the art of war - Stuff - October 19th, 2022 [October 19th, 2022]
- What Is Virtual Reality All About? - Oculus VR - September 20th, 2022 [September 20th, 2022]
- What Is Virtual Reality? | Tech & Learning - September 20th, 2022 [September 20th, 2022]
- 10 Advantages And Disadvantages Of Virtual Reality (2022) - September 20th, 2022 [September 20th, 2022]
- Insights on the Tactile Virtual Reality Global Market Research Report to 2027 - Growing Advancements in Virtual Reality Products & Rising... - September 20th, 2022 [September 20th, 2022]