A recipe for the success of Apple VR headsets

Almost all major technology companies have dreamed of smart glasses in recent years. From Echo Frames from Amazon to Facebook’s plans for augmented reality, to Microsoft’s HoloLens and continuous reporting of Apple AR Plans – there seems to be a lot of cooking. But in the short term, the path to Tony Stark’s magic glasses is likely to be through VR. This is exactly what Mark Gurman of Bloomberg reports as Apple’s next step: the pair of advanced virtual reality glasses which can be, in a way, like the popular Oculus Quest.

Apple has never had a virtual reality headset (or glasses) before and stands out in the landscape with an ever-growing interest in augmented reality. Apple also bought the VR company NextVR last May. Meanwhile, Google has made its own VR headset for years and has even acquired VR software companies behind applications like Tilt Brush and Job Simulator. But Google has moved away from VR ever since, focusing on AR.

The VR hardware is already here and is surprisingly good. Oculus Quest 2 is one of my favorite game consoles and is easy to set up and dive into. But VR everywhere else is still experiencing growing pains. PC VR has fantastic games and some excellent headphones, but they are still clunky, wired and require very specific hardware to run (and still look glued to Windows, rather than organic). The new PlayStation 5 currently treats the PlayStation VR, originally released in 2016, as an afterthought.

What is needed now is not just better technology: it is deep support in applications and phones, so that these headsets appear to be communicating with everything else in their configuration.

Most virtual reality headsets seem to be stuck on their own islands: specific games, applications and almost no cross-communication between computers or phones, or anything else. They are surprisingly removed from the mobile device ecosystem – smartwatches seem more connected to my life than any virtual reality headset. For many arts and experiences that VR does best, this disconnect works well. It is like an isolation tank worn on the head. But it is a very bad choice to make VR any type of tool you want to use on a daily basis.

Virtual reality has always been like headphones for my eyes, a way to expand the vision around me and immerse myself in something more completely. The headphones are plug-and-play and work with what you need. Virtual reality headsets don’t do that. But maybe they could. Perhaps that is exactly what Apple – and even Google – should enable next.

AirPods Max for your eyes: a fantastic display to wear on your head

The recent movements of Apple headphones are already clear. The AirPods Max, with their spatial audio and very high price, they seek design, immersion and fidelity. The same will be the case with any VR headset. VR screens are getting better and better and are now sharp enough to barely see any pixels. But they are still no better than the phone screens, monitors and TVs that we use all the time. There is only one virtual reality headset I’ve ever seen, from Varjo, that looked “level of retina” enough to be as good or better than a giant monitor. Knowing Apple, it seems that looking for “the best screen” or promising that would be a big goal.

VR headsets must work with iOS (and Android) phones

Yes, phones have already had a phase of virtual reality glasses. But those accessories from the mid-2010s were basically plastic lenses that you would attach to your phone and were extremely basic compared to what devices like the Oculus Quest can do.

But now, VR headsets like Quest barely speak to phones and instead run their own applications. Facebook’s walled garden of content, which requires a Facebook account, needs to work better with other devices if it’s a better home office tool, something Facebook is planning. Facebook can get there, but cooperation from Apple (iOS, Mac) and Google (Android, Chrome) is required. Even Facebook’s relationship with Windows, when it comes to running Oculus apps, seems divided.

Phones and tablets are already super powerful and could function as complete computers with VR headsets as peripherals. Apple could extend iOS on headphones and bridge the iPhone, iPad and even the clock to match the experiences.

Google started exploring VR with its Daydream platform, then backed off. Even if Google doesn’t create its own VR ecosystem, it needs Android to work better with headsets that want to interact.

Apple chips could work on larger VR headsets first and discover smaller glasses later

Qualcomm’s recent changes in recent years, paving the way for headphones and lightweight glasses to connect to phones and eventually transmit 5G graphics, look like what the future may bring. Qualcomm’s processors are inside almost all VR and AR headsets at the moment: Oculus Quest 2, Microsoft HoloLens 2, the next glasses from Vuzix.

Apple’s clear path involves developing or adapting its own extremely powerful phone and Mac chips to the headphone shape and doing, in a way, what Qualcomm has been doing. Or something potentially more advanced.

But the smartglasses that make AR are, according to almost every company I talked to, years away. A VR headset that plays with AR effects, combining reality with cameras that can see the outside world, is a clear intermediate step.

Dealing is already here: it could augment virtual reality headsets and also mix reality

Apple has already placed handle sensors on its iPads and professional iPhones, using depth detection to map and “merge” a room to create a 3D map of what is there, so that virtual objects can be layered. Headphones like the Oculus Quest 2 can scan room obstacles to some extent with basic external cameras, but the handle can do this much more accurately and possibly more quickly.

A next professional VR headset made by Finnish company Varjo has a handle, allowing it to also scan the world and bring real-life objects to VR.

What I really want is a collision detection system that can prevent me from hitting my hands on a wall or table, but that I would have to deal with in order to function constantly. I’m not sure if Apple’s deal could do that without a major blow to battery life.

Would Apple discover a better virtual reality controller?

Oculus Quest 2 has great controllers, but they look more like the gamepad on a video game console – buttons, analog levers, triggers. This is good for games, not great for turning a virtual reality headset into a high-end computer. Would Apple use hand tracking and air gestures, as Quest and Microsoft HoloLens already have, or would it develop some kind of used banner (like an Apple Watch) that could help?

It’s one of my biggest doubts for an Apple VR headset, because I can’t imagine what they can do.

Something that can switch between standalone, phone, iPad or Mac perfectly

Much like AirPods, or the continuity between Apple devices for AirDrop and file and link sharing, a VR headset should work on anything nearby. Oculus Quest comes closer to this by working alone or with a Windows PC when connected with a USB-C cable.

The more Apple and Google allow headsets to work with their devices, the better

Even more than an Apple VR / AR headset, I want the next versions of iOS and Android (and MacOS, and even ChromeOS) to allow these things to connect. Without that glue, virtual reality headsets are destined to always look like strange toys. I already use VR for much more than I expected. But if I’m ever going to use it for anything else, or if Apple wants to position its hardware as a pro-creative tool, it needs to work well with everything else.