People often talk about the mind’s eye: the ability to both look at something in the world and to close your eyes and picture it. Varun Wadia, a neuroscientist at Cedars-Sinai and Caltech, notes that visual imagination helps us conjure faces, navigate with mental maps and infer unseen parts of objects.
In a study published in Science, Wadia and colleagues show that imagining an object and actually seeing it activate the same neurons and use the same neural code. Kalanit Grill-Spector of Stanford, who was not part of the work, says this neural-level demonstration is novel. Thomas Naselaris of the University of Minnesota adds that the finding helps explain how imagination can fill in missing parts of three-dimensional objects or combine familiar elements into new images, such as a unicorn.
The team recorded single-neuron activity in 16 epilepsy patients who already had electrodes implanted for clinical reasons. While participants viewed hundreds of images spanning faces, animals, plants, words and small objects, researchers monitored more than 700 neurons per person in the ventral temporal cortex, a region tied to object recognition. They tracked both which neurons fired and their firing rates, allowing reconstruction of the neural code associated with each image.
In a second stage, participants closed their eyes and imagined objects they had just seen. Approximately 40 percent of the neurons that responded during perception reactivated during imagination, and they did so with similar strength. The overlap was sufficient for the researchers to decode which object a subject was imagining and to recover features such as size, viewpoint and whether the item was typically found indoors or outdoors, according to Ueli Rutishauser of Cedars-Sinai and Caltech.
These results extend previous brain-imaging studies that suggested overlapping circuits for seeing and imagining by demonstrating the phenomenon at the level of individual neurons, beyond the spatial resolution of fMRI. The work also builds on prior research into how visual systems encode faces and objects, including studies by Doris Tsao.
The paper has practical and theoretical implications. Better knowledge of the shared neural code could inform computational models of vision, improve understanding of vision disorders and guide development of visual prosthetics. It also raises new questions, such as the neural basis of aphantasia, a condition in which people report no voluntary visual imagery. Rutishauser recounts meeting a researcher who says they see nothing when they close their eyes. Scientists suspect that people with aphantasia may rely more on words or abstract concepts than visual representations, but resolving that will require direct neuronal investigations.