期刊
PERSPECTIVES ON PSYCHOLOGICAL SCIENCE
卷 18, 期 5, 页码 1136-1159出版社
SAGE PUBLICATIONS LTD
DOI: 10.1177/17456916221141422
关键词
language; interaction; multimodality; gestalt perception; binding; segregation
Natural human interaction requires us to process and produce various signals, such as speech, gestures, and facial expressions. This article introduces a framework called interactionally embedded, affordance-driven gestalt perception to explain how humans efficiently process multimodal signals. It discusses empirical evidence of gestalt perception principles in unimodal phenomena and proposes a framework where high-level gestalt predictions are updated by sensory input. The article also provides testable predictions from this framework to understand complex, multimodal behaviors in natural social interaction.
Natural human interaction requires us to produce and process many different signals, including speech, hand and head gestures, and facial expressions. These communicative signals, which occur in a variety of temporal relations with each other (e.g., parallel or temporally misaligned), must be rapidly processed as a coherent message by the receiver. In this contribution, we introduce the notion of interactionally embedded, affordance-driven gestalt perception as a framework that can explain how this rapid processing of multimodal signals is achieved as efficiently as it is. We discuss empirical evidence showing how basic principles of gestalt perception can explain some aspects of unimodal phenomena such as verbal language processing and visual scene perception but require additional features to explain multimodal human communication. We propose a framework in which high-level gestalt predictions are continuously updated by incoming sensory input, such as unfolding speech and visual signals. We outline the constituent processes that shape high-level gestalt perception and their role in perceiving relevance and pragnanz. Finally, we provide testable predictions that arise from this multimodal interactionally embedded gestalt-perception framework. This review and framework therefore provide a theoretically motivated account of how we may understand the highly complex, multimodal behaviors inherent in natural social interaction.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据