What you need to know
-Apple is experimenting with ideas on how to best manage augmented and virtual reality experiences. – Device patent refers to wearable finger controllers. – Smart, the design leaves the fingertips free to use touchscreens.
Apple has long been rumored to be making a virtual reality/augmented reality device, and while the summer software show didn’t unveil the goods last week, a newly discovered patent shows Apple continues to explore the mixed reality space.
The Apple VR/AR headset may not have been unveiled at WWDC 2022, but the patent (number 11,360,558, filed with the United States Patent Office) reveals a series of devices worn around a user’s fingers like thimbles, designed to enable virtual and mixed control reality experiences, and interact with objects within them.
The patent describes arrays of one-dimensional and two-dimensional sensor elements, using capacitive touch sensors, cameras and sensors that track optical and ultrasonic data. Force and inertial motion can also be tracked. Essentially you have something that allows a wearer to control elements in 3D space – the VR or AR equivalent of a pointer or mouse, similar to VR controllers we currently see with Oculus/Meta Quest devices, but reduced to their essential elements to fit on a user’s finger so that gestures, movements and inputs can be tracked through the air.
The patent specifically refers to viewing this content through a ‘head-mounted device’, with haptic feedback to help a wearer understand a sense of presence in the digital world and through its interactions. It also describes how a connected AR system can better recognize real-world objects through their interaction and proximity to the finger-worn devices, and as such overlay different information about physical objects through a headset or similar display based on their rotation and angle. Interestingly, the patent leaves the wearer’s fingertip exposed – presumably to keep them free to interact with a touchscreen device like his best iPhonesor the keyboard of a MacBook, which anchors a wearer to their real devices.
Pointing a finger at the future
Apple is about to enter a crowded AR/VR space, with plenty of competition from the likes of Meta, Sony PlayStation, Snap and other companies betting bigger on the coming importance of the metaverse. Innovation will be the key to getting noticed, so it’s good to see Apple experimenting with fresh ideas.
It’s interesting to see Apple also considering finger-based devices, as Meta, for example, is working hard to make hands-free hand and finger tracking the ultimate norm, using a headset’s built-in camera and space sensors to identify numbers and hands and to have followed accordingly.
But it’s a hit and miss at times, and no system to date has managed to provide hands-free operation with the same level of accuracy that a held device, be that wand or controller, can provide. Apple’s patent here seems to be a concession between both solutions – small enough to be unobtrusive for an immersive mixed reality experience, but with enough sensors and technology to magnify what a person’s fingers can reach.
As always, a patent isn’t a guarantee that a device will ever get out of the idea stage — it’s just to protect Apple’s intellectual property. But it’s an encouraging sign of what Apple may be cooking up for its augmented and virtual future.
PSA: Send your iMessages of 4th of July with fireworks this year
This year, try to remember to use iMessage’s cool Send with Firecrackers feature whenever you wish friends and family a happy Independence Day. You’ve probably thought about doing it afterwards, when it’s too late. This year is the year you remember!
Decorate your dorm with the coolest things
Dorms can be simple, so you’ll want to cover it with as much cool stuff as you can – here are our picks for the essentials!