The Limits of Virtual Reality

There’s no “better” reality than the one we’re in

Immersive, virtual reality has been in popular fiction for several years, but how feasible is it?

Plenty of us dream of the day when we can don a helmet or plug a cord into the backs of our craniums and enter a digital world of imagination. The concept has been bounced around in various forms of entertainment media, often with sinister implications, such as in The Matrix or Sword Art Online. Nevertheless, the idea has captivated audiences, and most people find the prospect of it rather cool.

Only recently, however, have we entered into what could be considered a true era of virtual reality, with hardware like the Oculus Rift and games like VRChat being released. They currently offer a primitive first step into this genre of technology, coming nowhere close to the kind of complexity and depth as their fictional counterparts. Many wonder: How long will it take to get to that point? A better question is: Can it even be done?

I am not versed in computer science; I majored in political science. My “experience” in coding goes little beyond having to use mark-up to format text and pages as needed. Even so, the logical implications of hyper-realistic VR are not lost on me. The limits of VR appear to be eminently physical, owing to what could be described as inefficient creation. I mean “creation” in a somewhat biblical sense, to mean “the making of a reality.” The inefficiency comes from the fact that, whatever reality we try to create, we have to do so within the confines of our current reality.

In the sandbox game Minecraft, which could be considered a form of virtual reality, people have shown that they are able to create very complex things with the introduction of redstone, which enabled the in-game invention of circuits. This video shows someone who made an entire, working calculator. It is quite impressive, but you can see the difference between it and an actual calculator. Why is the Minecraft calculator worse than a real one? It comes back to inefficient creation.

To make the point easy to digest, imagine that, in the real world, the code behind a functioning calculator is the word itself: “CALCULATOR.” Now imagine that Minecraft, being simpler than the real world, only codes using the letters A and B (essentially binary coding). You would have to use combinations of the two to code for other letters.

A would still be A, and B would still be B, but C would be AA. D would be AB, and E would be BA, with the binary alphabet progressing as BB, AAA, AAB, ABA, and so on all the way to Z, which would be BABB. “CALCULATOR,” in this hypothetical Minecraft code, would thus be spelled “AAABABAAABBABABAABABAAAAAABB.”

Which one looks more efficient? Obviously the original “CALCULATOR” does. The real calculator just exists, but the Minecraft calculator has to exist through the extra layer of Minecraft’s programming and data, which which is what simply exists in our world. That is inefficient creation at work.

Some might argue that updating the game to include calculators as actual items (and not forcing players to improvise them haphazardly as seen in the video) would solve the problem. In this context, it would, but that is essentially sneaking the code for a calculator into the code of the game. It’s not the same as manipulating the existing code to create what you want.

Imagine that the Universe is an ideal computer. By that, we mean that it is able to code reality with perfect efficiency. As an example, it would encode something fundamental to existence, like 1 electron, with just 1 electron. Implicitly, it would encode 5 elections with only 5 electrons. Being an ideal computer, the Universe therefore encodes at a 1:1 ratio.

The quintessential problem we have is that, unlike with Minecraft, we cannot update the Universe’s code. We are not gods; we cannot create new energy and matter, nor can we rewrite the laws of science. We can only use what is already here — we have to manipulate the Universe’s code — and we do not have the ability to encode at a 1:1 ratio. If we want to create code for an electron, we are going to need more than one electron’s worth of assets (inefficiency).

Remember the example that I brought up with “CALCULATOR.” Now, replace the letters in this scenario with all the atoms and other particles that make up a calculator (and how many trillions of those there are), and think of how much bigger the AABABABA soup to encode all of that gets at that moment.

This means that any virtual reality we create will, by necessity, be smaller and/or simpler than our own, and it does not matter how technologically advanced we become.

If we wanted to create a virtual scale model of the Earth that completely replicated the properties of every particle within it, then we would need more matter and energy than currently exists on the Earth to code all of that, and it is going to be a long time before we have more than a planet’s worth of assets that we can just spend on recreation.

Some trade-offs will have to be made for VR to work. If we want realistic visuals that can be processed in real-time, then we have to code summaries of physics, rather than the full details. Some of this we can pull off seamlessly. For example, human eyes are content to see things in 144 frames per second. We do not need the visual processing to be much better than for it to be fit for our leisure. The same probably goes for the rest of our senses in some manner, or for things that we do not need to observe in order for the game to be fun (i.e. plate tectonics). The product only has to be convincing, not completely identical to reality.

In any event, taking shortcuts in physics will inevitably result in aberrant behavior of objects in that reality, but that’s the way to conserve assets.

Well — we could also constrain the scale to, say, that of a town instead of a whole planet. You might want epic-sized virtual realities that look as believable as The Matrix, but what is more likely is that VRs of that size will continue to look similar to VRChat, or at least less real than the smaller-scale projects.

That does not mean that VR will not be very fun in the future. It just means that we need to enter this next era with modest expectations. As much as VR seeks to serve as an escape from the burdens of the physical world, it is irrevocably a product of that very same world.