Facebook finally explains its mysterious new wearable device on the wrist

It first appeared on March 9 as a tweet on the timeline of Andrew Bosworth, the little corner of the Internet that offers a rare glimpse into the mind of a Facebook executive today. Bosworth, who heads Facebook’s augmented and virtual reality research labs, has just shared a blog post describing the company’s 10-year vision for the future of human-computer interaction. So, in a follow-up tweet, he shared a photo of a wearable device not yet seen. Facebook’s vision for the future of computer interaction would apparently involve tying something that looks like an iPod Mini to your wrist.

Facebook already has our social experience and some of the most popular messaging apps in the world – for better or worse. Whenever the company immerses itself in the hardware, be it a very good virtual reality headset or a video chat device that follows your every move, it gets noticed. And it not only arouses intrigue, but also questions: why does Facebook want to have this new computing paradigm?

In that case, the unanswered questions are less about the hardware itself and more about the research behind it – and whether the new interactions that Facebook envisions will only deepen our ties with Facebook. (Answer: probably.) At a media briefing earlier this week, Facebook executives and researchers offered an overview of this technology. In simpler terms, Facebook has been testing new computing inputs using a sensor wearable wrist.

It is an electromyography device, which means that it translates the electrical signals from the motor nerves into digital commands. When it’s on your wrist, you can just move your fingers in space to control virtual inputs, whether you’re using a virtual reality headset or interacting with the real world. You can also “train” you to feel the intent of your fingers, so that the actions happen even when your hands are completely still.

This wrist accessory has no name. It’s just a concept, and there are different versions of it, some of which include tactile feedback. Bosworth says it could take five to ten years for the technology to become widely available.

All of this is linked to Facebook’s plans for virtual and augmented reality, technologies that can sometimes leave the user with a distinct lack of agency when it comes to their hands. Put on a VR headset and your hands will disappear completely. By picking up a pair of hand controllers, you can play or grab virtual objects, but then you lose the ability to take notes or draw accurately. Some AR or “mixed reality” headsets like Microsoft’s HoloLens have cameras that track spatial gestures, so you can use certain hand signals and the headset will interpret those signals … which sometimes works. So Facebook has been using this wearable EMG in its virtual reality lab to see if such a device can allow for more accurate hand-computer interactions.

But Facebook has visions for this pulse technology in addition to AR and VR, says Bosworth. “If you really had access to an interface that would allow you to type or use a mouse – without having to physically type or use a mouse, you could use it anywhere.” The keyboard is an excellent example, he says; this wrist computer is just another means of intentional input, except that you can carry it anywhere.

Bosworth also suggested the kitchen microwave as a use case – although he clarified that Facebook is not, in fact, building a microwave. The home appliance interfaces are all different, so why not program a device like this to understand, simply, when you want to cook something for 10 minutes at medium power?

In the virtual demo that Facebook gave earlier this week, a player was shown wearing his wrist device and controlling a character in a rudimentary video game on a flat screen, all without having to move his fingers. These types of demos tend to (with pardon the pun) a gesture towards mind-reading technology, which Bosworth insisted not. In that case, he said, the mind is generating signals identical to those that would make the thumb move, but the thumb is not moving. The device is recording an espresso intention to move the thumb. “We don’t know what’s going on in the brain, which is full of thoughts, ideas and notions. We don’t know what happens until someone sends a signal over the wire. “

Bosworth also emphasized that this wearable wrist is different from the invasive implants that were used in a 2019 brain-computer interface study that Facebook worked with at the University of California at San Francisco; and it’s different from Elon Musk’s Neuralink, a wireless implant that could theoretically allow people to send neuroelectric signals from their brains directly to digital devices. In other words, Facebook isn’t reading our minds, even though it already knows a lot about what’s going on in our heads.

The researchers say there is still a lot of work to be done in the area of ​​using EMG sensors as virtual input devices. Accuracy is a major challenge. Chris Harrison, director of the Future Interfaces Group at the Human-Computer Interaction Lab at Carnegie Mellon University, points out that the nerves of each human being are slightly different, as are the shapes of our arms and wrists. “There is always a calibration process that must happen with any muscle detection system or BCI system. It really depends on where the computational intelligence is, ”says Harrison.

And even with tactile feedback embedded in these devices, as Facebook is doing with some of its prototypes, there is a risk of visuo-haptic incompatibilities, where the user’s visual experience – whether in AR, VR or real space – does not correlate with the haptic response. These points of friction can make these interactions between humans and computers seem frustrating unreal.

Even though Facebook can overcome these obstacles in its research labs, there is still the question of why Facebook – largely a software company – wants to have this new computing paradigm. And should we trust that? This extremely powerful technology company that has a history of sharing user data in “exchange for other things equally or more valuable”, as Fred Vogelstein of WIRED wrote in 2018? A more recent report from the MIT Technology Review highlights how a Facebook team brought together to tackle “responsible AI” has been hampered by leadership’s relentless pursuit of growth.

Facebook executives said this week that these new human-computer interaction devices will perform as much computing as possible “on the device”, meaning that information is not shared with the cloud; but Bosworth does not commit to the amount of data that can ultimately be shared with Facebook or how that data will be used. The whole thing is a prototype, so there is nothing substantive to separate yet, he says.

“Sometimes, these companies have lots of money big enough to basically invest in these huge R&D projects, and they’ll lose those things if it means they can be the first in the future,” says Michelle Richardson, director of e Privacy Project at non-profit organization Center for Democracy and Technology. “But with companies of any size, any product, once built, is so difficult to reshape. So anything that can start the conversation about this before the devices are built is a good thing. “

Bosworth says Facebook wants to lead this next paradigm shift in computing because the company sees technology like this as fundamental to connecting people. In fact, the past year has shown us the importance of connecting – in feeling myself as if in person, says Bosworth. He also seems to believe that he can gain the necessary trust by not “surprising” customers. “You say what you do, set expectations and meet those expectations over time,” he says. “Trust arrives on foot and leaves on horseback”. Pink AR glasses, activated.

This story originally appeared on wired.com.