When you consider it, the retina doesn’t give us all that much to work with. Two 30mm discs of around 100 million photoreceptor cells each. Not bad on the resolution front – but only a measly 5 per cent of these are colour sensitive and the whole thing is crawling with blood vessels and punctured by a 1.5mm hole where the optic nerve enters the eyeball. So how do we get from these highly compromised little light sensors to the colourful, immersive, and undisrupted view of a bustling city street?

Well if a single slogan could sum up the recent history of vision science it might be this: there’s more to sight than meets the eye.

Over the past decade, a framework called Predictive Processing has begun to help us unravel those additional ingredients. According to this account, our brain continuously tracks patterns in its sensory input, to develop an internal model that generates predictions of how this will change over time and with action. Perception is the process of updating this model in light (or touch, or sound, or smell) of the difference between predicted and actual stimulation. What we actually experience is not the raw input, but rather the end product of a relentless, and entirely subconscious, negotiation between our internally-generated prior expectations and currently incoming stimulation. The senses, it turns out, are less conveyers of information, than constrainers of imagination.

This idea of our brain as geared first and foremost to the task of predictive modelling has surprising scope. In particular it provides a strikingly integrated explanation for a diversity of hallucinations, illusions and other puzzling psychological phenomena, that have often been treated as unrelated quirks of our cognitive systems. In the resurgent field of psychedelics research, for instance, an increasingly popular proposal suggests that such drugs achieve their experience-altering effects by breaking down our established internal model, allowing our expectations to run wild and amplify random patterns in the sensory stream as the brain attempts to regain a hold on some sort of predictive structure.

Such a story may not only explain the similarities between the action of LSD and psilocybin, but also how it’s possible to achieve similar effects by doing nothing more than facing a strobe light with your eyes closed. Doing so bombards the visual system with a stream of random noise, and within as little as 10 minutes many people begin to experience hallucinations that range from multicoloured mandalas to extremely realistic scenes as their brain strives to impose familiar structure on the chaos. A few report the experience of total ego-dissolution and loss of the sense of space or time, that drug users often describe as a feature of particularly transformative trips.

Simply placing yourself in an unfamiliar environment can also produce strange perceptual phenomenon by forcing the brain to reformat its internal model – consider the difficulty of making sense of the size and distance of a building in the desert, or a boat on the ocean. As humanity expands into space and leaves Earth’s familiar regularities behind, it will be not only our physiological but also our perceptual systems that have to readapt.

We needn’t leave Earth to begin exploring ways of re-structuring conscious experience, however. VR already provides the opportunity to deliberately re-engineer it. This could serve as perceptual pre-training for the exploration of alien environments. It has already been used to change how we perceive ourselves. Watching an object being moved in sync with the intended movements of your hand or foot can cause you to perceive that object as part of your body – even if the object bears only a weak resemblance to a normal human limb. In VR, this has been exploited to move a person’s sense of location towards another body entirely, or to create the sensation of having a number of additional limbs. Outside of VR it could be used to encourage the incorporation of transformative physical prosthetics, helping overcome resistance to their unfamiliar form.

The world around us, and the structure of our bodies, are not fixed facts but the product of a skilful, dynamic collaboration between our expectations and our environmental constraints. In the future they may take a form we aren’t even capable of seeing yet. 

Kathryn Nave

A contributing editor at WIRED, Kathryn Nave is currently completing a PhD on the nature of conscious experience in the predictive brain at the University of Edinburgh.

Perceptual pre-training for alien environments

© Vollebak 2020

Founded in 2016, Vollebak uses science and technology to make the future of clothing happen faster. In our first four years we’ve made the world’s first Graphene Jacket using the only material in the world with a Nobel Prize, released 100 Year clothing designed to outlive you, created a Plant and Algae T Shirt grown in forests and bioreactors that turns into worm food, and designed the first jacket for deep space travel. You can find out more about us at vollebak.com.

Sign up to see the future first: