Virtual Reality (VR) has come a long way, and the tech world is looking forward for a brighter, and a more virtual world.
But virtual doesn't mean that it should look "virtual."
As tech advances, virtual worlds are expected to at least resemble the real world, in which elements inside it, including the interactions, aren't distinguishable from the real world.
And according to Meta's founder and CEO, Mark Zuckerberg, VR headsets that can do this should pass the "virsual Turing Test."
Inside of Meta’s Reality Labs Research division, the company uses the term "visual Turing Test" to represent the bar that needs to be met to convince users' visual system that what’s inside the headset is actually real.
The concept is borrowed from the "Turing Test' concept, which denotes the point at which a human can tell the difference between another human and an artificial intelligence.
The Four Challenges
To pass this visual Turing Test, Zuckerberg and Meta's Reality Labs Chief Scientist Michael Abrash outlined four key visual challenges that VR headsets need to solve before the they can pass the visual Turing Test:
- Varifocal: the ability to focus on arbitrary depths of the virtual scene, with both essential focus functions of the eyes (vergence and accommodation).
- Distortion: lenses inherently distort the light that passes through them, often creating artifacts like color separation and pupil swim that make the existence of the lens obvious.
- Retina resolution: having enough resolution in the display to meet or exceed the resolving power of the human eye, such that there’s no evidence of underlying pixels.
- HDR: also known as high dynamic range, which describes the range of darkness and brightness that we experience in the real world (which almost no display today can properly emulate)..
If these four points can be fulfilled, VR headsets should be able to transport users to a virtual world that isn't distinguishable from reality.
By passing all the points, science fiction can be brought to reality.
The biggest challenge is to trick the eyes into believing something they shouldn't
The thing is, human eyes can adapt to the problem of focusing, through their unique ways to changing the shape of the lenses. The eyes cannot be tricked, if VR optics use solid lenses that don’t move or change shape.
"Our eyes are pretty remarkable. And that they can, they can pick up all kinds of subtle cues when it comes to depth and location," said Zuckerberg. "And when the distance between you and an object doesn’t match the focusing distance, it can throw you off, and it feels weird and your eyes try to focus but you can’t quite get it right. And that can lead to blurring and be tiring."
This is where retinal resolution display plays its role.
VR headsets should also be able to change the focal depth to match where users' are looking by moving the lenses around dynamically, kind of like how autofocus works on on cameras, Zuckerberg said.
If the display can do all these in HDR-capable projection in at least 60 pixels per degree at all distances, from near to far in focus, VR headsets should be able to trick the eyes a lot better.
"Making 3D displays that are as vivid and realistic as the physical world is going to require solving some fundamental challenges," Zuckerberg said.
"There are things about how we physically perceive things, how our brains and our eyes process visual signals and how our brains interpret them to construct a model of the world. Some of the stuff gets pretty deep."
And this vistual Turing Test, considered the holy grail of VR display research, can test VR headsets' capabilities to match the full capacity of human vision.
If VR headsets can pass the four points, they should be able to create a realistic sense of presence, or the feeling that an animated experience is immersive enough to make users feel like they are physically there.
"You all can probably imagine what that would be like if someone in your family who lives far away, or someone who you’re collaborating with on a project or, or even an artist that you like would feel like if you’re right there physically together. And that’s really the sense of presence that I’m talking about," Zuckerberg said.
When this happens, Zuckerberg said that realistic displays should open up a new form of art and individual expression.
Users should be able to express themselves in as immersive and realistic way powerful, and that will be very powerful, he said.
While passing all four, there are challenges that also need to be solved.
For example, there should be a proper balance between high-quality VR headsets, with costs, battery life, and weight.
All that to ensure VR headsets can operate longer, and are comfortable enough for prolonged usage.
If any one of these vectors falls short, it degrades the feeling of immersion.
The Turing Test is named after Alan Turing, the mathematician who led a team of cryptanalysts who broke the Germans’ notorious Enigma code, helping the British turn the tide of World War II.
Turing created the Turing Test in 1950 to determine how long it would take a human to figure out they were talking to a computer before figuring it out.
“What’s important here is the human experience rather than technical measurements. And it’s a test that no VR technology can pass today,” Abrash said in the press briefing. “VR already created this presence of being in virtual places in a genuinely convincing way. It’s not yet at the level where anyone would wonder whether what they’re looking at is real or virtual.”
"We’re in the middle of a big step forward towards realism. I don’t think it’s going to be that long until we can create scenes with basically perfect fidelity,” Zuckerberg said.
"Only instead of just looking at a scene, you’re going to be able to feel like you’re in it, experiencing things that you’d otherwise not get a chance to experience."
"That feeling, the richness of his experience, the type of expression and the type of culture around that. That’s one of the reasons why realism matters too. Current VR systems can only give you a sense that you’re in another place. It’s hard to really describe with words. You know how profound that is. You need to experience it for yourself and I imagine a lot of you have, but we still have a long way to go to get to this level of visual realism."