Spatial computing promises to transform the way we interact with our devices as computing goes truly 3D, with early signs of change already underway. While Apple‘s upcoming Vision Pro is bringing new excitement to this space, gaming-focused VR (Virtual Reality) headsets from companies including Meta, Sony, and Pico have sold millions of units, and AR (Augmented Reality) glasses from Vuzix, Microsoft, and more have found a valuable place in industry.
Events in 2023 led IDTechEx to identify two important narratives it expects to unfold over the next decade of the XR market: virtual reality’s impact will broaden as it feels closer to “real reality”, and the endgame for Big Tech firms involved in XR is to mark their place in a future market of AR devices designed to replace the smartphone.
IDTechEx’s breakdown of types of spatial computing device.
Making VR more immersive
Key expectations for spatial optics and display technologies in 2034.
In August 2023, Meta showed off its latest Butterscotch prototype VR headset. This demonstrated solutions to several issues holding back VR from feeling real, giving a glimpse of the experiences these devices may offer by the end of the decade and the hardware required to do so. Meta is not unique in tackling these problems, but the prototype is unique in combining solutions in one device.
Problem 1: Pixels need to disappear
Butterscotch’s display system approaches retinal resolution, which is the pixel density at which most people’s eyes cannot distinguish individual pixels. This reduces the perception that the user is looking at screens, eliminating the “screen door effect” of the visible pixel grid. Butterscotch manages 56 pixels per degree (PPD) across the user’s field of view (FoV), close to the 60PPD heuristic commonly suggested, although the downside is that the FoV has been approximately halved compared to most other VR headsets.
A production-ready system would need a higher field of view, but this requires incredibly high-resolution screens and massive computing demands. One solution, employed in Finnish professional-market VR specialist Varjo’s XR-3 headset, is to use a higher resolution foveal microdisplay in the center of the vision, where the eye resolves the most detail, and use a lower resolution display for the rest of the visual field. Whilst this sets Varjo’s image quality apart from the rest of the market, the tradeoff is the resulting added weight, complexity, and cost.
Varjo’s headsets are priced at the highest end of the market, but it is possible that this approach will trickle down over the next decade as technology matures. The simpler approach will likely be a move towards large but higher pixel density microdisplay technologies in VR, with foveated rendering taking place at the software level, saving on computing power but not the number of pixels needed for the display. The IDTechEx report “Displays for Virtual, Augmented and Mixed Reality 2024-2034: Forecasts, Technologies, Markets” outlines the expectation that OLED-on-silicon displays, which are more commonly used in camera viewfinders today, will replace the LCDs used today to become the prevalent VR display technology in the 2030s, with VR-specific OLED-on-silicon releases from companies including BOE setting the stage for this already in 2023.
Problem 2: 3D display systems mess with our heads
Butterscotch can also correct for the vergence-accommodation conflict: the mismatch between the fixed focal (accommodation) distance of a binocular stereoscopic display system and the varied distances of objects from the viewer (vergence) it simulates by parallax. This conflict can increase problems with motion sickness felt by users of XR headsets and decrease immersion since the eye is missing important visual cues for depth. Butterscotch solves this problem by tracking where the eyes are looking using IR cameras, tweaking the scene rendering, and, most importantly, adjusting dynamically tunable lenses to focus the headset’s displays to match vergence and accommodation distances.
The lenses used in Butterscotch are moved physically by motors to vary focus, but Meta has also demonstrated geometric phase lens arrays that could be used for the same purpose. These lenses switch from a positive to a negative diopter depending on the handedness of circularly polarized input light: stack these lenses with switchable waveplates, and an extremely compact, dynamically tunable focal length lens with no moving parts and near-zero latency can be produced. Although image quality issues remain, it seems more likely that Meta would take this approach in a commercial headset. IDTechEx’s report “Optics for Virtual, Augmented and Mixed Reality 2024-2034: Technologies, Players and Markets” forecast these lenses to become an extremely important technology for the VR market, with commercial introduction likely occurring towards the end of the 2020s. Notably, other players, including Valve and Apple, have proven interest in the technology through patents and investments, so Meta is hardly the only giant working in this area.
The shape of VR devices in 2034
A similar combination of features to those shown off in Butterscotch, which also include further innovations, including dynamic distortion correction based on the headset’s eye-tracking capabilities, will likely be required to boost immersion and comfort in VR. This is particularly true as interest develops in the use of VR headsets as a replacement for today’s workstations, since these factors are extremely important when working with text for long periods. This offers a true chance to broaden the utility of VR devices, not to mention their market size, but getting to this point is an uphill struggle. Meta spent $7.7 billion on its spatial computing-focused Reality Labs division in H1 2023 alone – of course, not all of this was spent on VR hardware development, but it remains indicative of the scale of R&D work required to make XR devices effective replacements for today’s computing systems.
Big Tech firms are setting the stage for AR
Whilst Meta established itself as an XR hardware player early on, 2023 was the year when Apple finally made its first long-anticipated move here with its announcement of the Vision Pro. This headset, which IDTechEx classes as a VR headset with focus on MR passthrough, is an impressive device, but it is likely just the beginning of the spatial computing game for Apple. The endgame prize for Big Tech players in the XR market is likely establishing see-through AR devices as an eventual replacement for the smartphone, with passthrough MR, where cameras on a VR headset pass a view of the real world to the user, offering a useful steppingstone along this path.
AR vs. VR (including passthrough MR devices) share of the XR market over the next 10 years. By 2034, AR is expected to have overtaken VR.
Big Tech companies have been investing in AR software content for years, delivered through smartphones: Google‘s ARCore and Apple’s ARKit have been highly influential in lowering the barriers to development here, with Amazon using the latter to develop its earliest AR e-commerce efforts in 2017. Smartphones are the only way that most of the public are likely to have experienced augmented or mixed reality experiences, but building software and content libraries now offers a chance to expand to dedicated AR/MR devices as they become more widely used.
Passthrough MR as a stepping stone technology
In 2023, even lower-cost VR devices are routinely including passthrough MR features, which is beginning to democratize more immersive AR/MR experiences: for example, the Pico 4 includes color passthrough to help users orientate themselves within the room. This headset launched at EUR 429 and is a close competitor to Meta’s Quest 2: notably, Pico is owned by TikTok’s parent company ByteDance. It appears that Chinese tech giants have just as much interest in establishing themselves as XR players as their American counterparts. IDTechEx’s report “Virtual Reality & Augmented Reality Headsets 2023-2033: Technologies, Players, and Markets” outlines the different approaches being taken by the players here whilst giving a comprehensive overview of the XR market.
Passthrough MR offers a commercially ready way to deliver captivating MR experiences, but social acceptability is a significant problem since even the most compact and elegant VR device will always conceal the eyes behind displays. The Vision Pro uses an outward-facing display to help address this, but even the most refined implementation of this system is likely to feel, at best, like a significant barrier from the wearer of the headset. The other significant issue is that these devices are fail-deadly and so could not be worn whilst driving, walking, in industrial settings, and so on since they are effectively blindfolds if something goes wrong.
Seethrough AR doesn’t suffer from these problems, offering the potential to replace smartphones as an all-day wearable access point to computing. However, so far, this approach has proven tough to get right, even for big players like Microsoft. These glasses should leave the eyes open to the real world: the crucial components used to project images to the eyes while leaving this view clear, which are known as optical combiners, have arguably offered the greatest technical challenge to AR success, as outlined in IDTechEx’s report “Optics for Virtual, Augmented and Mixed Reality 2024-2034: Technologies, Players and Markets“.
Optics offer the greatest roadblock to AR development
Waveguide combiners, which have generally looked like the combiner solution of choice for this since they can offer a relatively wide FoV while looking like normal spectacle lenses, have proved a challenging combiner technology to get right. Tech and social media companies alike have interests in this area, with Apple, Microsoft, and Snap all acquiring firms with interests in waveguide design. One major problem is the lack of optical efficiency (nits out to the eye for lumens in from the display) of many designs, increasing demands on display brightness, and therefore battery capacity and thermal management: solving this would make compact, all-day wearable AR devices more feasible.
Potential increased use of more efficient reflective waveguides over their diffractive counterparts that are dominant today is one solution here, but diffractive waveguide manufacturers counter that the manufacturability of reflective waveguides is inferior. A solution to this (extremely controversial) manufacturability problem could be a move towards molded plastic reflective waveguides. This has the additional benefit of better mechanical properties for near-eye use compared to glass substrates but has the downside of limiting FoV, since this depends on the refractive index of a waveguide material, and polymers are inferior to glass here.
The outlook for AR in 2034
Waveguides are just one area of technological focus for Big Tech players with an interest in AR devices, not to mention myriad smaller players. The challenges remain significant, but the reward at the end could be huge: AR devices offer the potential for the most seamless access to digital content yet, offering even greater time in front of our eyes to the gatekeepers of this content. It seems unlikely that most of us will be wearing AR glasses instead of carrying a smartphone in 2034, but the early adopters will probably be beginning to make the switch by this point.
Further insight
For in-depth analysis of the technology, players, and markets behind XR optics, displays, headsets and accessories, please see IDTechEx’s XR portfolio. IDTechEx’s report, “Displays for Virtual, Augmented and Mixed Reality 2024-2034: Forecasts, Technologies, Markets“, offers detailed analysis of the spatial computing display landscape including granular ten-year market forecasts and assessments of the potential for success of the technologies covered. “Optics for Virtual, Augmented and Mixed Reality 2024-2034: Technologies, Players and Markets” not only forecasts the adoption of twenty individual optical technologies, but also the material demands these technologies create. “Virtual Reality & Augmented Reality Headsets 2023-2033: Technologies, Players, and Markets” forecasts and predicts technology adoption of displays, optics, eye tracking and haptics in addition to headsets and accessories, as well as offering a data-based review of the evolution of XR device specifications and pricing. All reports feature multiple interview-based company profiles from key industry players in addition to SWOT analysis, readiness level assessments, and much more.
This article is from “Technology Innovations Outlook 2024-2034”, a complimentary magazine of analyst-written articles by IDTechEx providing insights into a number of areas of technology innovation, assessing the landscape now and giving you the outlook for the next decade. You can read the magazine in full at www.IDTechEx.com/Magazine.
Sam Dale
Senior Technology Analyst, IDTechEx
About IDTechEx
IDTechEx guides your strategic business decisions through its Research, Subscription and Consultancy products, helping you profit from emerging technologies. For more information, contact [email protected] or visit www.IDTechEx.com.
