Today the US Patent & Trademark Office published a patent application from Apple that relates to a future version of Apple Pencil with a huge twist. On one hand the invention relates to delivering tactile feedback to a future version of Apple Pencil. On the other hand, Apple's engineers describe how an Apple Pencil could be used with a mixed reality headset for both augmented and virtual reality. Users will be able to write on virtual paper or have this Apple Pencil used as another tool interacting with virtual objects.
Haptics are used on a touch display found on an iPhone or iPad to convey a variety of different information to a user, such as information regarding one or more touch inputs that a user has provided such as alerts or status of the electronic device or one or more applications.
However, when using Apple Pencil, as it is today, the user misses out on feeling the particular haptic related to alerts and so forth.
In accordance with Apple's invention, improvements to Apple Pencil will be able to produce shear forces that act on a user to provide unique tactile sensations. For example, a shear device can be included at a grip region of a stylus to provide shear sensations at the user's hand (e.g., fingers). The sensation can be targeted directly to the user's hand, rather than generally across the entire stylus.
Augmented and Virtual Reality Application
Apple's patent takes an interesting twist at one point and notes that "Shear forces can be used with a virtual reality, augmented reality, or mixed reality system to simulate tactile sensations of interacting with physical objects even when no such objects are present.
More specifically, the invention at this point veers off and doesn't limit 'Shear Forces' to a future version of Apple Pencil.
Apple specifically states that "A touch-based input device in accordance with embodiments disclosed herein can include any device that is held, worn, or contacted by a user for providing input and/or receiving feedback. The touch-based input device can be used alone or in conjunction with another device."
For the sake of simplicity, "another device" is shown to be an iPad in patent FIG. 1 below as Apple's patent returns to non-virtual applications until we get to patent FIG. 6 further below.
In patent FIG. 2, Apple points us to the grip area of this future Apple Pencil and focuses on where the 'Shear' tactile forces will be delivered. In this one design, the shear feedback is delivered by vertical types of strips.
Beyond tactile haptic feedback, Apple notes that shear forces include unaligned forces that urge one part of the user's hand (e.g., finger) in one direction and another part of the user's hand in an opposite direction.
The shear feedback components include sliding elements #220 shown further below in patent FIG. 11. Apple clarifies that the grip region could actually take up 10% to 90% of the surface of the grip area, even though the patent figures limit the size of the shear feedback components/elements.
In patent FIG. 5 we see that this future version of the Apple Pencil will include an accelerometer #170, a gyroscope #172, and/or a compass #174. During use, the accelerometer can track and record acceleration of Apple Pencil. Acceleration can be measured in a three-dimensional (x, y, and z) coordinate system.
The haptic device #178 can be implemented as any suitable device configured to provide force feedback, vibratory feedback, tactile sensations, and the like.
Free Space Application
In Apple's patent FIG. 6 below Apple Pencil can be used to provide feedback during use in free space. For example, this future version of Apple Pencil can simulate tactile sensations that correspond to a user experience in a mixed reality system.
As shown in FIG. 6, Apple Pencil will be operated by a user in a manner that moves the Pencil through space. Each point in space corresponds to a point within a virtual environment of a mixed reality system.
A region #198 of space can correspond to a region of the virtual environment. For example, a virtual object can be rendered in the virtual environment and displayed or otherwise output for reference by the user. The display can include a headset, a head-up display, and/or an optical head-mounted display in communication with Apple Pencil.
A surface, volume, interior, or other portion of the virtual object can correspond to the location of the region #198 in space. Accordingly, when the future Apple Pencil is moved to the region in space, it is understood by the user to be positioned at the virtual object in the virtual environment.
In Apple's patent FIG. 9 above we're able to see the bridge segments #212 in the sliding elements #220 can be evenly distributed along a length of the user grip region. For example, the user grip region can present radial and/or bilateral symmetry.
In Apple's patent FIGS. 10 and 11 above, the future Apple Pencil can be used to provide feedback during use in free space. As shown in FIG. 10, the future Apple Pencil could be operated by a user in a manner that rotates through space.
Apple's patent application that was published today by the U.S. Patent Office was filed back in Q4 2018. Considering that this is a patent application, the timing of such a product to market is unknown at this time.
Apple's Inventors
Benjamin Jackson: Failure Analysis Engineer - iPad Tech Ops
David Bloom: Sr. Quality Assurance Manager in Portland, Oregon
Steven Taylor: Mechanical Design Engineer
Go here to see the original:
- Virtual Reality - Virtual Reality - December 9th, 2016 [December 9th, 2016]
- VR, Virtual Reality | Virtalis - December 11th, 2016 [December 11th, 2016]
- Road to VR - Virtual Reality News - December 13th, 2016 [December 13th, 2016]
- VR Porn, Exclusive Virtual Reality Sex Videos. WankzVR - December 19th, 2016 [December 19th, 2016]
- Immersion (virtual reality) - Wikipedia - December 22nd, 2016 [December 22nd, 2016]
- Virtual Reality - Setting the Record Straight One Post at ... - December 22nd, 2016 [December 22nd, 2016]
- NBA broadcasts games in virtual reality with NextVR ... - December 28th, 2016 [December 28th, 2016]
- Virtual Reality | Future | Fandom powered by Wikia - December 28th, 2016 [December 28th, 2016]
- Virtual reality, one year out: What went right, what didn't ... - December 28th, 2016 [December 28th, 2016]
- Virtual Reality Toronto - VRTO - Virtual Reality, Toronto - January 9th, 2017 [January 9th, 2017]
- Virtual Reality - January 24th, 2017 [January 24th, 2017]
- Virtual reality headset - Wikipedia - January 27th, 2017 [January 27th, 2017]
- Tribemix Virtual Reality for Dementia Care - YouTube - January 27th, 2017 [January 27th, 2017]
- Virtual Reality Has Arrived in the Art World. Now What? - New York Times - February 7th, 2017 [February 7th, 2017]
- Virtual Reality Experience College Swim Meet - SwimSwam - February 7th, 2017 [February 7th, 2017]
- Explore The Amazon With This Stunning 360 Virtual Reality Video - IFLScience - February 7th, 2017 [February 7th, 2017]
- Adobe's Path To Entering The Virtual Reality Story - Forbes - February 7th, 2017 [February 7th, 2017]
- Europe's virtual reality sector has grown to nearly 300 companies - VentureBeat - February 7th, 2017 [February 7th, 2017]
- Virtual reality to recount Maori life - Otago Daily Times - February 8th, 2017 [February 8th, 2017]
- John Wall: 'I really thought I was gonna die' - ESPN.co.uk - February 8th, 2017 [February 8th, 2017]
- Travel Brands and Meeting Planners Are Using Virtual Reality to Drive Conversion - Skift - February 8th, 2017 [February 8th, 2017]
- Sportsnet to launch 360 Virtual Reality for six NHL games - Sportsnet.ca - February 8th, 2017 [February 8th, 2017]
- In the burgeoning world of virtual reality, storytelling is both cutting-edge and old-fashioned - Los Angeles Times - February 8th, 2017 [February 8th, 2017]
- What parents need to know about virtual reality - Deseret News - February 8th, 2017 [February 8th, 2017]
- 8 Real Success Tips From Women Building The Future With Virtual Reality - Forbes - February 8th, 2017 [February 8th, 2017]
- Virtual reality: prepare for the revolution - AndroidPIT - February 9th, 2017 [February 9th, 2017]
- Prospects Dimming For Facebook's Oculus Rift Virtual-Reality Headsets - Investor's Business Daily - February 9th, 2017 [February 9th, 2017]
- WATCH: YouTube Stars Create Amazing La La Land-Inspired Virtual-Reality Music Video - PEOPLE.com - February 9th, 2017 [February 9th, 2017]
- Virtual Reality addiction threat prompts cautious approach as VR nears 'smartphone-like' take-off - ABC Online - February 9th, 2017 [February 9th, 2017]
- Archaeology turns to virtual reality with Lithodomos start-up - The Sydney Morning Herald - February 9th, 2017 [February 9th, 2017]
- The Future Of Virtual Reality: Mobile VR Platforms In A Battle - Forbes - February 9th, 2017 [February 9th, 2017]
- Virtual reality is boring compared to Beyonc's actual reality - A.V. Club (blog) - February 9th, 2017 [February 9th, 2017]
- Google wants to use Chrome to suck you into virtual reality - CNET - February 9th, 2017 [February 9th, 2017]
- Ringling showcase offers glimpse of virtual reality in Sarasota - Sarasota Herald-Tribune - February 11th, 2017 [February 11th, 2017]
- Allegheny County teachers see how to open world to classrooms with virtual reality - Tribune-Review - February 11th, 2017 [February 11th, 2017]
- Virtual Reality Could Be Your Next Prescription - DesignNews - February 11th, 2017 [February 11th, 2017]
- DACC & Virgin Galactic team up to explore virtual reality - Las Cruces Sun-News - February 11th, 2017 [February 11th, 2017]
- Boulder International Film Festival diving into virtual reality with 2017 slate - Boulder Daily Camera - February 11th, 2017 [February 11th, 2017]
- 'JAEPO 2017' Shows That Japanese Arcade Gaming Is Beginning To Incorporate Virtual Reality - Forbes - February 11th, 2017 [February 11th, 2017]
- Wichita-created virtual reality could take jurors into crime scenes ... - Wichita Eagle - February 11th, 2017 [February 11th, 2017]
- Why this analyst is warning investors to steer clear of virtual reality in 2017 - L.A. Biz - February 12th, 2017 [February 12th, 2017]
- See the Newcastle virtual reality room helping children with autism overcome their phobias - ChronicleLive - February 12th, 2017 [February 12th, 2017]
- See This Famous Masterpiece Recreated in Virtual Reality - UploadVR - February 12th, 2017 [February 12th, 2017]
- Samsung Goes for a New Look in Virtual Reality at New York Fashion Week - Fortune - February 14th, 2017 [February 14th, 2017]
- Virtual reality has a growing impact on college football - FOXSports.com - February 14th, 2017 [February 14th, 2017]
- This company is opening a virtual reality multiplex this fall - Mashable - February 14th, 2017 [February 14th, 2017]
- Personalized virtual reality displays match eyesight | Stanford News - Stanford University News - February 14th, 2017 [February 14th, 2017]
- Virtual reality weather add-ons let you feel the sun and wind - New Scientist - February 14th, 2017 [February 14th, 2017]
- Virtual reality is still too isolating to be 'the next big thing' in tech - CBC.ca - February 14th, 2017 [February 14th, 2017]
- Miyamoto Still Has Doubts About Virtual Reality - GameSpot - February 14th, 2017 [February 14th, 2017]
- Virtual reality industry 'in need of a jump-start,' Imax CEO says at ... - Los Angeles Times - February 16th, 2017 [February 16th, 2017]
- Doctors use virtual reality to prepare children for scans - BBC News - February 16th, 2017 [February 16th, 2017]
- Gamers get a taste of virtual reality with new Missoula arcade - KTVH - February 16th, 2017 [February 16th, 2017]
- Revised orca shows, new virtual-reality swim with whales and new ... - Los Angeles Times - February 16th, 2017 [February 16th, 2017]
- Transcend VR Sues Virtual-Reality Investor Mike Rothenberg - Wall Street Journal (subscription) - February 16th, 2017 [February 16th, 2017]
- This Powerful Gaming Executive Isn't Worried If Virtual Reality Fails - Fortune - February 16th, 2017 [February 16th, 2017]
- Virtual reality helps visitors transport back in time to ancient Roman ... - CBS News - February 16th, 2017 [February 16th, 2017]
- Boise-area entrepreneurs see opportunity in virtual reality amid real-world perils - Idaho Statesman - February 17th, 2017 [February 17th, 2017]
- 3D printing, virtual reality used to bring dinosaur to 'life' - The Hindu - February 17th, 2017 [February 17th, 2017]
- Virtual reality startup Relax VR wants to relieve stress in corporate environments - ZDNet - February 17th, 2017 [February 17th, 2017]
- Escape with Virtual Reality, travelogue at Way - Sentinel-Tribune - February 18th, 2017 [February 18th, 2017]
- VR1: Idaho's first virtual reality arcade opens in Eagle - KBOI-TV - February 18th, 2017 [February 18th, 2017]
- NBA launches virtual reality app with Google Daydream - USA TODAY - February 18th, 2017 [February 18th, 2017]
- Virtual Reality Is Bringing These Lost Worlds Back to Life - NBCNews.com - February 18th, 2017 [February 18th, 2017]
- Now You Can Feel Wind and Temperature While in Virtual Reality - Futurism - February 18th, 2017 [February 18th, 2017]
- IMAX's first virtual reality arcade is here and it is beautiful - The Verge - The Verge - February 18th, 2017 [February 18th, 2017]
- VR Sales Numbers Are Wet Blanket on Adoption Hopes - Fortune - February 20th, 2017 [February 20th, 2017]
- Virtual Reality to Make MRIs Less Scary - Financial Tribune - February 20th, 2017 [February 20th, 2017]
- How Valve is Taking a Nintendo-like Approach to Virtual Reality - UploadVR - February 20th, 2017 [February 20th, 2017]
- Inside an IMAX virtual reality arcade - VentureBeat - February 20th, 2017 [February 20th, 2017]
- With New Invention, Virtual Reality's Potential for Magic Gets Real - New York Times - February 20th, 2017 [February 20th, 2017]
- What Ubisoft learned from its first virtual reality games - VentureBeat - February 20th, 2017 [February 20th, 2017]
- Virtual Reality Was a Flop in 2016. Will 2017 Be the Year It Takes Off? - Motley Fool - February 20th, 2017 [February 20th, 2017]
- Virtual reality to take a decade as users chicken out, study shows - The Australian Financial Review - February 20th, 2017 [February 20th, 2017]
- Virtual reality is making surgery simpler - Deniz Ergurel (subscription) (blog) - February 20th, 2017 [February 20th, 2017]
- Virtual reality has potential to add new dimensions to marketing - Business Day (registration) - February 21st, 2017 [February 21st, 2017]
- Virtual reality will take 'maybe a decade' to become mainstream - Mashable - February 21st, 2017 [February 21st, 2017]
- #MadeInRVAJr: Hanover siblings want to bring virtual reality to your event - WRIC - February 22nd, 2017 [February 22nd, 2017]
- Making Virtual Reality More Accessible - VideoInk - February 22nd, 2017 [February 22nd, 2017]
- New virtual reality lounge opening in the Capital Region - NEWS10 ABC - February 22nd, 2017 [February 22nd, 2017]