As is his tradition at Oculus Connect, Michael Abrash, Oculus' Chief Scientist, closed out the opening keynote of the event with his thoughts on the near and distant future of VR. If there’s anyone in the industry that can give us an educated guess as to where the industry is headed, Abrash can.

Throughout the nearly four-year development cycle of the Rift, Oculus continuously refined the hardware and improved its form and function along the way. Now that the Rift is a shipping product, Oculus is looking to the future and exploring what virtual reality hardware could become in the next few years.

Earlier in the Oculus Connect 3 opening keynote, Facebook CEO Mark Zuckerberg revealed a sneak peek at Project Santa Cruz, a tether-free Rift that features inside-out tracking and is powered by a mobile SoC. Inside-out tracking and tether-free VR are obvious “next steps” for VR hardware development, but Abrash expounded on some of the less obvious features that next generation VR could and should have.

“VR five years from now will make VR from today look like something out of prehistory.”

Today’s top-tier virtual reality hardware is nothing short of incredible. The Oculus Rift and the HTC Vive are capable of tricking your mind into believing that you’re in a whole new world if you let it, but if you’re determined, it’s not hard to find subtle cues (such as imperfect visuals) that pull you back to reality. The VR industry has a long way to go before our HMDs become indistinguishable from the real world.

According to Abrash, the human visual system is capable of perceiving 120 pixels/degree over the span of at least 220 degrees. The current Rift hardware features two 1200x1080 panels, which provide roughly 15 pixels/degree over a roughly 90-degree field of view (FOV). The current Rift, then, has miles to go before it can max out the human visual system. Abrash believes that even in five years’ time, VR will still be a long way reproducing the equivalent of 20/20 human vision.

Abrash expects that by 2021, the technology will be available to build an HMD that offers 4K x 4K per eye, which would bring the pixel density up to 30 pixels/ degree. But that's still a far cry from what the human eye is capable of perceiving. Abrash expects that having a wider FOV will be a more compelling experience than simply increasing the pixel density and suggested that 140 degrees is a realistic expectation. Abrash noted that he doesn’t believe that it’s possible to adapt Fresnel lenses to work well beyond 100-degree FOV, so he expects that someone will develop a new lens technology to get past that problem.

We have to imagine that Starbreeze would likely disagree with Abrash about the lenses, because the StarVR HMD offers 210-degree FOV with custom Fresnel lenses.

“It may be necessary to redesign the entire graphics rendering pipeline.”

Today, VR demands the most out of your computer. Your graphics card must push the scene at 90fps to provide a smooth experience. Most current GPUs can handle the workload, but nothing available today is capable of rendering VR content at 4K x 4K per eye at 90fps. Graphics processing capabilities must increase by an order of magnitude so that VR hardware can advance. Foveated rendering will likely play an integral role in rendering VR content on such high-resolution displays, but Abrash speculated that GPU manufacturers might have to go back to the drawing board and come up with a new approach to the graphics pipeline--especially if they hope to make the HMD wireless.

Abrash said that he once thought the problem simply required some “solid engineering,” but he now understands that it’s a bigger problem than he imagined. Eye tracking technology is capable of tracking pupils reliably enough to add life to an avatar, but “tracking required for foveated rendering is not a solved problem at all.”

Foveated rendering relies on extremely precise pupil tracking. Anything less than perfect tracking could result in massive degradation of visual quality, because only the portion of your screen that should be visible gets rendered clearly. Abrash said that current eye tracking technology isn’t capable of tracking your pupil accurately enough to properly implement foveated rendering, primarily because of the vast variety of pupil sizes, the fact that your pupils change in shape and size, and because your pupil’s shape changes when the position of your eye changes.

Abrash is unsure what it will take to surpass the hurdles in eye tracking, but he feels that the problems will be solved within the next five years--and they must be, because “great eye tracking is essential for VR” in the future, he said. He did, though, concede that foveated rendering is the least likely of all of his predictions to come true.

Some Things Won’t Change

VR HMDs will change dramatically in the next five years, but some things probably won’t change. Oculus Touch is just around the corner; pre-orders for the hand controllers open on October 10, and Oculus will start shipping the hardware on December 6. Abrash expects that Touch will remain the “state of the art” input device for VR for years to come. He postulated that it “could become the mouse of VR” and be around for the next 40 years.

It’s Abrash’s belief that your real hands are the only suitable replacement for Touch, but we’re a long way from being able to replicate haptic and kinematic feedback without a hand-held device. That sort of technology isn’t even being worked on yet and is likely a problem for a future generation to solve.

Don’t expect to be using your bare hands as your primary input device, but you can expect to use your hands for basic actions. Abrash believes that your hands will be tracked and rendered along with your avatar. You’ll be able to use hand gestures to interact with simple applications, but complex games will require a physical input device.

Further Refinement

Oculus spent a great deal of time and money on the design of the Rift HMD to ensure that it's as comfortable as possible, but that's not stopping Oculus from making the hardware even better. Abrash said that you can expect future Rift hardware to be lighter and feature better weight distribution than the current generation. He also expects that it will have a solution for prescription lenses.

Abrash shied away from speculating about the software side of the VR industry, though he did share one software idea that he "expects to be using" in five years. He called the concept "Augmented VR," which is a version of mixed reality that lets you pick and chose what you want to see. You'd be able to bring any element from the real world into the digital world and vice versa. And each person would be represented by a virtual human avatar.

Abrash isn't planning to build the software himself, but he threw the idea out there in hopes that someone will make it.

“These Are The Good Old Days”

The world of technology is changing rapidly, whether you like it or not. You may believe that virtual reality is just a passing craze, but the truth is, we’re just at the early stages of this revolutionary technology. The first round of hardware is incredible, but it's by no means perfect. If you’re not satisfied with today’s VR hardware, sit back and wait a few years. These early years will be looked back upon as "the good old days," but rest assured, the best is yet to come.