Facebook offered a glimpse of its plans for a new augmented reality interface, based on technology from CTRL-Labs, the startup it acquired in 2019. In a video, it shows bracelets that use electromyography (EMG) to translate subtle neural signals into actions – how to type, slide or play games like an archery simulator. The bands also offer tactile feedback, creating a system that is more responsive than basic hand tracking options.
Facebook Reality Labs published a blog post detailing their work on a prototype of the bracelets. In their simplest form, bands would track basic gestures that Facebook calls “clicks”, which should be reliable and easy to perform. They are a bit like the Microsoft HoloLens multifaceted “air tap” gesture, but tracked with the nerve signals that pass through your arms, instead of visual sensors mounted on a headset.
The bands can theoretically do a lot more, however. For example, they can track the nerve signals that your brain sends to your fingers as you type, so that you can type on a virtual keyboard without physical buttons. And, unlike a regular keyboard, bands can slowly adapt to the way you type – so they can “learn” the way your fingers move when you’re making common typos and then correct them automatically and capture what you probably intended to type. .
That would be a big change in the way most people interact with computers, but, conceptually, it’s not really a big update on how CTRL-Labs described their work years ago. In fact, the final possibilities for EMG bracelets are much more mind-boggling: eventually, you could perform the same typing-style gestures thought about moving your hands instead of actually moving them. Facebook wants to further optimize user interactions with artificial intelligence and augmented reality glasses, which it announced it was working on last year.
Even in their simplest iterations, these controllers would offer an interface that you could use all the time instead of picking up and holding, like today’s Oculus Touch VR controllers. The effect may be similar to offers from smaller startups, such as the Mudra Band, which detects gestures using an Apple Watch bracelet.
A major new addition is haptics. Facebook says it is implementing several prototypes that can provide subtle feedback using different methods. One of them, the “Bellowband”, has eight pneumatic bellows placed around each wrist. They can be inflated or deflated in patterns that produce different sensations. Another is the “Tasbi”, which uses vibrating actuators and a “new wrist tightening mechanism”. When they are paired with visual feedback from an AR headset, they can offer a lot of information through a simple and intuitive interface.
Facebook insists that although the band reads neural signals, “this is not like mind reading”. Here’s how it explains the concept:
You have many thoughts and choose to act on only a few of them. When this happens, your brain sends signals to your hands and fingers telling them to move in specific ways to perform actions such as typing and sliding. It’s about decoding these signals on your wrist – the actions you’ve already decided to take – and translating them into digital commands for your device.
CTRL-Labs still characterized this technology as a brain-computer interface, but it is in sharp contrast to technologies such as Elon Musk’s Neuralink – which reads neural activity directly from the brain through an implant. Implants have unique uses, especially for people with paralysis or amputated limbs, whose bodies simply cannot send nerve signals to a bracelet. But Facebook CEO Mark Zuckerberg recently criticized implants as a short-term consumer technology, saying that “we don’t think people will want to have their heads pierced to use virtual or augmented reality.” Nor do bracelets have the exact same factor of frightening the privacy of something that reads your thoughts at the source.
That said, the bands will almost certainly be collecting a lot of data. This can include incredibly subtle variations in typing patterns; general levels of body tension; and any biometric information captured by fitness tracking sensors, augmented reality glasses and other technologies that can be integrated into the bands. (Facebook Reality Labs notes that it has a “neuroethics program” that examines the privacy, security and protection implications of AR and neural interface technology.)
Like most wearable technologies, EMG bands offer an intimate look at how our bodies move – and while it doesn’t sound as scary as a band that reads your thoughts, it still requires a lot of confidence.