Evidence for Optimal Integration of Visual Feature Representations across Saccades

We explore the visual world through saccadic eye movements, but saccades also present a challenge to visual processing by shifting externally stable objects from one retinal location to another. The brain could solve this problem in two ways: by overwriting preceding input and starting afresh with each fixation or by maintaining a representation of presaccadic visual features in working memory and updating it with new information from the remapped location. Crucially, when multiple objects are present in a scene the planning of eye movements profoundly affects the precision of their working memory representations, transferring limited memory resources from fixation toward the saccade target. Here we show that when humans make saccades, it results in an update of not just the precision of representations but also their contents. When multiple item colors are shifted imperceptibly during a saccade the perceived colors are found to fall between presaccadic and postsaccadic values, with the weight given to each input varying continuously with item location, and fixed relative to saccade parameters. Increasing sensory uncertainty, by adding color noise, biases updating toward the more reliable input, which is consistent with an optimal integration of presaccadic working memory with a postsaccadic updating signal. We recover this update signal and show it to be tightly focused on the vicinity of the saccade target. These results reveal how the nervous system accumulates detailed visual information from multiple views of the same object or scene.