The nature of the visual representations involved in eye movements when walking down the street

In this paper we set out to answer two questions. The first aims to discover whether saccades are driven by low level image features (as suggested by a number of recent and influential models), a position we call image salience, or whether they are driven by the meaning and reward associated with the world, a position we call world salience. The second question concerns the reference frame in which the eye movements are planned. To answer these questions, we recorded six videos, using a head mounted camera, with the viewer walking down a popular shopping street in Bristol. As well as showing these videos to our participants, we also showed spatially and temporally filtered versions of them. We found that, at a coarse spatial scale, subjects viewed similar locations in the image, irrespective of the filtering, and that fixation distributions found when viewing videos with similar filtering were no more alike than if the filtering varied widely. Using a novel mixture modelling technique, we also showed that the most important reference frame was world-centred rather than head or body-based. This was confirmed by a second experiment where the fixation distributions to identical videos was systematically changed by using a swivelling tent that only altered subjects' perception of the gravitational vertical. We conclude that eye movements should not be understood in terms of image salience, or even information maximization, but in terms of the more flexible concept of reward maximization.