July 13, 2023
With last month’s announcements of Meta Quest 3 and Apple’s first mixed reality headset, now is a good time to get candid about the UX of XR. While the user experience has dramatically improved since “modern efforts” to build XR devices began over a decade ago, it still has a long way to go.
Don’t get me wrong: Today’s XR headsets are useful. The value of immersive technologies for the workforce is undeniable, but office workers are not about to replace their laptops with Apple Vision Pro.
I’ll focus mainly on VR: What Big Tech has failed to convey is that virtual reality is really hard, perhaps the hardest technology challenge of our time. Presentation videos like the one at WWDC set unrealistic expectations like all-day use, contributing to a cycle of hype and disappointment that overshadows what the devices can do today.
A TALL ORDER
The good news is that the time between each new generation of VR headsets is shrinking, which means VR is advancing quickly. The bad news is that we underestimated the challenges. While there has been no lack of investment, virtual reality has proved a lot harder than even the most innovative, financially endowed, and passionate companies expected.
When it comes to the ideal VR headset, we’re asking for a lot: A wireless, standalone device that’s comfortable and lightweight, produces very little heat, lasts all day, and is capable of displaying video game console-level graphics without inducing nausea–all for around $300. The headset must also be spatially aware and continuously and precisely track our eye, head, hand, and body movements to accurately render images.
Matthew Ball writes of a VR MVP or Minimal Viable Product that:
- Has an 8K display and 120 Hz refresh rate
- Includes at least a dozen cameras
- Weighs less than 500 grams, and
- Costs under $500
The upcoming Meta Quest 3 would almost fit the bill if not for a resolution of just “better-than-4k.” Pimax offers an 8K gaming headset but it costs over $500 and requires a powerful PC. You get the idea…There is no single VR headset today that can “do everything.”
SHORTCOMINGS
Many of the shortcomings of VR headsets are due to the intimate nature of the form factor. You wear a VR headset on your face and head, it inevitably generates heat and cuts you off from your surroundings, and the screen is very close to your eyes–all sources of discomfort.
Not only is every person’s head different, making it really hard to design a universally acceptable headset, but VR challenges normal human vision. Let’s talk about interpupillary distance (IPD) and the vergence-accommodation conflict (VAC), two complex design challenges for VR developers and sources of eye fatigue in VR.
IPD refers to the distance between the center of your left and right pupils. This value differs from person to person, impacting both the sharpness of the image and your perception of depth in VR. Experts stress the importance of accurately measuring your IPD to avoid distortion and eye strain in VR. Headsets typically have several IPD presets and/or a dial or other means to adjust the IPD. Eye tracking enables auto-IPD adjustment which should theoretically make it a non-issue, but even so devices like Meta Quest 2 accommodate only a range of adult IPDs and women are at a disadvantage on this front.
VAC is more difficult to solve: Vergence and accommodation refer to mechanisms by which our eyes focus on a single object. Normally, both quickly and simultaneously adjust according to the same perceived distance of the object. In VR, you’re staring at a screen that’s just inches away from your eyes while trying to focus on a point in the simulated world that’s further away. Vergence and accommodation no longer match, which can make close-up virtual objects blurry and cause eye strain.
VR headsets can also be nauseating. VR sickness affects anywhere between 20-95% of users depending on the study. Symptoms include nausea, headaches, and cold sweats. It's a form of cybersickness, which occurs when your eyes see movement but your vestibular system - responsible for balance and spatial awareness - senses your body at rest. One source of cybersickness in VR is lag: The higher the refresh rate or the more frequent the headset’s display updates with new images, the smoother the experience. While higher refresh rates reduce sensory mismatch, even the fastest chips today introduce a tiny bit of lag. It may be imperceptible to the naked eye, but the brain can still pick up on it.
To be clear, the above are mainly ergonomic and optical challenges. Complicating matters, especially size and price, are all the (expensive) components that need to fit inside the headset.
GAME OF TRADE-OFFS
VR hardware design is essentially a game of trade-offs. To greatly oversimplify things: For every improvement in a VR headset, there’s a corresponding technical drawback somewhere else. This is because factors like resolution, display, tracking, weight, performance, efficiency, fit, price, etc. are closely related such that improving one often necessitates compromising another.
PPD and FOV provide an example: A higher PPD (pixels per degree or pixel density) typically results in a lower field of view (provided the resolution is fixed). The higher the PPD, the greater the clarity, and the wider the FOV the more immersive the experience, but you can’t have both in one compact, affordable headset.
*For reference, 20/20 vision is around 60 PPD and most VR headsets today have 20-30 PPD. The high-end Varjo XR-3 has the highest PPD and widest FOV, but it’s also tethered and costs around $6,500 just for the headset itself.
Lenses are another example: Meta introduced pancake lenses with the Quest Pro, allowing for a slimmer headset (reduced size) compared to the more commonly used Fresnel lenses. On the downside, pancake lenses absorb more light, requiring a brighter display which drains battery faster. This is why the latest device can seem like a step back compared to its predecessor depending on the use case, what you value most in the overall VR experience, and certain subsets of the population (ex. those who wear prescription glasses or have an IPD under the lower limit of most headsets.)
The fact remains: VR still requires leaps in optics, batteries, processors, etc. to successfully simulate physics, match human vision, and trick the human mind all in a small, fashionable standalone headset.
THE CASE FOR MODULARITY
Some people believe modularity or even individual customization is the key to a better VR experience. In February, Bigscreen VR unveiled an entertainment-focused device it touts as “the world’s smallest VR headset.” Bigscreen Beyond is custom-built to the shape of your face. The company uses a scan of your face performed with an iPhone XR or above to 3D print a foam cushion that matches the contours of your face. The lenses are also customized for your exact IPD. But customizing a piece of hardware takes time and Bigscreen Beyond isn’t standalone.
Bespoke also limits shareability, which is why in June, Bigscreen announced that Beyond will be available in 18 different IPD sizes to enable sharing. The company also promised an improved FOV “except for those with a 70-72mm IPD due to physical size limitations.”
Modularity isn’t new: Razer offers an Adjustable Head Strap System and Facial Interface as alternatives to the standard straps and face pads of the Meta Quest 2. The HTC Vive XR Elite has a removable battery pack to make the headset more lightweight as well as quick-charge and hot-swap features so you can use it without interruption. The strap or Head Band for Apple Vision Pro will come in three sizes, too, for a more custom fit.
While modularity can be a competitive differentiator - having options for additional flexibility and comfort is a plus - it’s still “one-size-fits-most.” On the other hand, a build-your-own-headset model is impractical for mass production and a roadblock to mainstream use. Considering Apple is pitching its upcoming headset more like a general-use computer, modular accessories may be the only way forward.
ADJUSTING EXPECTATIONS
VR is imperfect. Nevertheless, VR headsets are being used today for industrial design and engineering, workplace safety training, and even surgery. The user experience issues are not insurmountable. I have every confidence that engineers and researchers will overcome them in time. Headsets will continue to get better and better. Nevertheless, we need to adjust our expectations for upcoming XR devices and better market VR for short bursts of use.
Consider the following: The first smartphone hit the market in 1994 and the first iPhone arrived in 2007, yet not until 2014 did half of Americans own a smartphone while it took the global population until 2020 to hit 50% ownership. Palmer Luckey designed the first Oculus Rift prototype in 2010. That means we’re at the first iPhone ‘stage’ of VR headsets (13 years into the timeline) and have another seven years to go before half of Americans may own a VR headset.
VR isn’t “behind;” it’s actually right on track.
Image source: Vecteezy
