Meta VR headset prototypes aim to pass Visual Turing Test

With an eye on the future of the Metaverse, CEO Mark Zuckerberg has unveiled a slew of Meta VR headset prototypes as the tech giant works towards making virtual reality “as vivid and realistic as the physical world”.

Immersive VR experiences are set to be the cornerstone of the Metaverse – Facebook parent company Meta’s attempt to bring the concept of cyberspace to life as a shared 3D virtual world, home to a virtual community, economy and marketplace.

Meta’s Metaverse is not expected to go mainstream for another five to 10 years, but it is far from a pipe dream. Meta already has 10,000 people working on the Metaverse and will spend $US10 billion on the project this year.

As part of that work, engineers at Meta’s Reality Labs have developed a range of Meta VR headset prototypes. They are working towards the goal of passing the Visual Turing Test, where virtual reality is practically indistinguishable from the real world, Zuckerberg said in Reality Labs’ recent “Inside the Lab” roundtable.

“It’s not going to be that long until we can create scenes in basically perfect fidelity,” Zuckerberg said. “Displays that match the full capacity of human vision are going to unlock some really important things.”

“The first is a realistic sense of presence, and that’s the feeling of being with someone or in some place as if you’re physically there. And given our focus on helping people connect, you can see why this is such a big deal.”

Zuckerberg reiterated plans to ship a high-end VR headset this year, codenamed Project Cambria. It will support both VR and AR, thanks to external cameras passing video to the internal displays. Project Cambria will also ship with eye tracking, a key feature for future Meta headsets. 

Visual Turing Test challenges

Looking beyond Project Cambria, building the perfect VR headset requires mastering four key concepts: high resolution, high dynamic range, variable focal depth and the ability to compensate for optical distortions.

Recreating 20/20 vision across the full human field of view would require greater than 8K resolution, but this can be scaled down by allowing for the fact that peripheral vision does not require as high resolution, according to Zuckerberg and Michael Abrash, Chief Scientist of Meta’s Reality Labs

Zuckerberg showed off the “Butterscotch” VR headset prototype which can display near retinal resolution – allowing the wearer to read the bottom line of an eye test in VR. Achieving this feat required halving the field of view of Meta’s current Quest 2 headset.

Meanwhile, improving the high dynamic range of VR headsets will deliver more realistic brightness, shadows, contrast and colour depth. 

Meta’s “Starburst” HDR prototype can produce up to 20,000 nits of brightness – a vast leap from the Quest 2’s 100 nits and brighter than high-end televisions – in an effort to underpin HDR. With exposed wires, and cooling fans strapped above each eyepiece, the prototype is too large and unwieldy to wear and must be held like binoculars.

Looking to the future, the use of eye tracking will allow VR headsets to allocate resolution to where the wearing is looking, while variable focal depth will allow wearers to easily focus on nearby and far away objects, as well as fix optical distortions inherent in current lenses. 

The “Mirror Lake” prototype is still at the concept stage, but promises to offer mechanical varifocal lenses and eye tracking, with the benefit of holographic lenses. The presentation also included Half Dome 3, the latest in a series of VR headset prototypes which can shift focal planes depending on where the wearer is looking.

Once these individual challenges are overcome with Meta VR headset prototypes, the next challenge is combining all these technological breakthroughs into compact, lightweight VR headsets which can run on batteries for extended periods.

Zuckerberg unveiled the fully-functional Holocake 2 prototype, Meta’s thinnest VR headset yet, which must be tethered to a PC. It is the successor to a 2020 design built on holographic optics, a light-bending technique which allows an almost flat panel to replace the traditional thick refractive lens. 

More Meta and VR news at GadgetGuy.