Physical inference of falling objects involves simulation of occluded trajectories in early visual areas

Humans possess an intuitive understanding of the environment's physical properties and dynamics, which allows them to predict the outcomes of physical scenarios and successfully interact with the physical world. This predictive ability is thought to rely on mental simulations and has been shown to involve frontoparietal areas. Here, we investigate whether such mental simulations may be accompanied by visual imagery of the predicted physical scene. We designed an intuitive physical inference task requiring participants to infer the parabolic trajectory of an occluded ball falling in accordance with Newtonian physics. Participants underwent fMRI while (i) performing the physical inference task alternately with a visually matched control task, and (ii) passively observing falling balls depicting the trajectories that had to be inferred during the physical inference task. We found that performing the physical inference task activates early visual areas together with a frontoparietal network when compared with the control task. Using multivariate pattern analysis, we show that these regions contain information specific to the trajectory of the occluded ball (i.e., fall direction), despite the absence of visual inputs. Using a cross-classification approach, we further show that in early visual areas, trajectory-specific activity patterns evoked by the physical inference task resemble those evoked by the passive observation of falling balls. Together, our findings suggest that participants simulated the ball trajectory when solving the task, and that the outcome of these simulations may be represented in form of the perceivable sensory consequences in early visual areas.

Automated summary

This study used fMRI to reveal that humans engage in visual imagery when making physical predictions, and that the frontoparietal areas of the brain are involved in this process.