Brain Encoding and Decoding

Brain encoding and decoding techniques offer powerful tools for accessing and interpreting the mental processes underlying perception and cognition. These approaches leverage advanced computational models to encode predictions of brain activity based on external stimuli and decode cognitive information from patterns of brain activity.

Encoding focuses on predicting patterns of neural activity based on external sensory inputs or cognitive tasks. By using neuroimaging data such as fMRI, EEG, or MEG, researchers develop models that simulate how the brain responds to specific stimuli—visual, auditory, or otherwise. Encoding models help reveal how the brain organizes and integrates information, offering insights into neural mechanisms such as feature detection, attention, and memory encoding.

Decoding, often referred to as “neural mind reading,” takes the reverse approach: reconstructing cognitive or perceptual information from patterns of brain activity. These techniques decode neural signals to identify both simple perceptual features, such as orientations or colors, and complex stimuli, such as objects, scenes, or dynamic visual inputs like movies.

The integration of encoding and decoding, powered by machine learning, has significantly contributed to expanding our understanding of cognition. By linking brain activity with external stimuli and cognitive processes, these methods offer a comprehensive framework for investigating perception, memory, decision-making, and other cognitive functions. Recent advances in AI have further enhanced the capabilities of encoders and decoders, allowing neural activity to be mapped to continuous text, visual representations, and semantic concepts. For instance, a recent study leveraged brain decoding to explore cortical representations of objects (Nguyen et al., 2024).

Image from Nishimoto et al., Current Biology, 2011