In Back to the Future Part II, Marty McFly gets a video call.
When the film came out in 1989, that was part of its vision of 2015, alongside hoverboards, flying cars and self-lacing shoes. Today, a video call is so ordinary that most of us are more interested in finding an excuse not to answer one.
Technology has a habit of doing this. Something looks impossible, then futuristic, then expensive, and eventually nobody thinks twice about it.
Spatial computing may be somewhere in the middle of that journey. Science fiction has been playing with the idea for decades. The Matrix took it to the extreme, with characters entering entirely constructed environments that could become a training room, a city street or almost anything else.
We’re obviously not at Matrix levels of immersion, but the basic idea doesn’t seem quite so ridiculous anymore. VR can replace the room around you with a digital environment. Mixed reality can do the opposite, keeping the real world visible while adding digital objects to it. Headsets can track your hands, understand your surroundings and use spatial audio to make sounds seem as though they’re coming from different places around you. We aren’t waiting for this technology to be invented anymore.
We’re working out what to do with it.
Gaming Got There First
Gaming has an obvious reason to experiment with spatial technology.
A conventional game asks you to look through a screen into another world. VR puts that world around you.
Turning your head to see what’s behind you feels different from moving a camera with a thumbstick. Reaching towards something is different from pressing a button to pick it up.
Mixed reality takes things in another direction by bringing the game into your surroundings. Your floor can become part of the playing area, your furniture can become an obstacle and virtual objects can appear to occupy the same room as you.
That doesn’t mean televisions and controllers are going anywhere. Sitting on a sofa remains an extremely comfortable way to play a game.
The Technology Has Caught Up
Virtual reality itself isn’t particularly new. The difficult part has always been making it good enough that people actually want to use it.
You need high-resolution displays close enough to your eyes that the image still looks convincing. Cameras and sensors have to track your head and hands with very little delay. The device needs enough processing power to create a 3D environment while constantly updating it as you move.
Mixed reality adds another problem. The headset needs to understand the real world around you. That means recognising walls, floors, furniture and other objects so digital content can be placed in the room without immediately ruining the illusion.
Then there is spatial audio, which changes as you turn your head so a sound can appear to stay in the same place. Eye tracking can tell the device where you’re looking, while increasingly accurate hand tracking means controllers aren’t always necessary.
All of this has to happen quickly. If you turn your head and the image takes too long to follow, spatial computing stops feeling very spatial and starts making you feel sick.
Modern hardware is getting much better at combining all of these things in a single device. Displays have improved, processors have become smaller and more powerful, and cameras and sensors can increasingly understand both the person wearing the headset and the room around them.
Spatial computing isn’t simply about putting a screen closer to your eyes. It’s about making digital things behave as though they occupy the same space you do.
What Actually Deserves to Leave the Screen?
Not everything becomes better because it floats in front of your face. Nobody has been staring at the calculator on their phone wishing it would occupy the entire living room. An email doesn’t necessarily become more exciting because it’s hovering above the kitchen table.
Entertainment has a better case because sometimes presence is part of the experience.
Gaming is the obvious example, but there are others. Sports could put viewers closer to the action. A concert could give someone at home a viewpoint they couldn’t get from an ordinary seat. Fitness apps can put instructions into the same space in which you’re exercising.
Live interactive entertainment has something to gain too. Babestation, for example, already combines live video with interaction between viewers and performers. Spatial technology wouldn’t need to change that basic idea; it could simply make the experience feel more immediate.
The same applies to a musician performing live, friends meeting in a virtual space or spectators watching an event together from different parts of the world.
What Happens Next?
There is still a fairly large object standing between spatial computing and mass adoption.
You’re supposed to wear it on your head.
Headsets have improved enormously, but comfort matters. So do weight, battery life, price and whether people actually want to put something on their face every time they play a game or watch something.
Glasses could solve part of that problem. If spatial technology eventually fits into something closer to ordinary eyewear, using it becomes much less of an event.
But that creates another problem.
For spatial computing to understand the world around you, it needs cameras and sensors looking at that world. In your own living room that may not seem particularly strange. Walk into a café wearing the same technology and suddenly the device may be seeing tables, screens and people who never agreed to be part of your digital fantasy.
That raises questions about privacy, recording and what information the device actually keeps. A pair of spatial glasses might look far less intrusive than a VR headset while potentially collecting much more information about everything around you.
These are difficult problems, but they’re also a sign of how far spatial computing has already come. We’re no longer asking whether a computer can understand the room around us, track our movements or place digital objects into the real world. We’re asking how much it should be allowed to see, how comfortable it is to wear and whether it’s useful enough to justify the price.
Those aren’t science-fiction questions anymore. They’re the sort of problems when things get real.
