4.6 Review

Goals, usefulness and abstraction in value-based choice

相关参考文献

注意:仅列出部分参考文献,下载原文获取全部文献信息。
Article Neurosciences

Planning in the brain

Marcelo G. Mattar et al.

Summary: Recent research reviews neural and behavioral data on how the brain plans in sequential decision-making tasks. A taxonomy of planning algorithms is created to analyze the brain's representation of future states, suggesting a convergence with specific planning algorithms. Further research is needed to address detailed algorithmic questions.

NEURON (2022)

Article Automation & Control Systems

A practical guide to multi-objective reinforcement learning and planning

Conor F. Hayes et al.

Summary: This paper serves as a guide for applying multi-objective methods to difficult decision-making problems, addressing the issue of oversimplification in current research and providing insights for researchers and practitioners familiar with single-objective reinforcement learning and planning methods.

AUTONOMOUS AGENTS AND MULTI-AGENT SYSTEMS (2022)

Article Biochemistry & Molecular Biology

Sequential sampling from memory underlies action selection during abstract decision-making

S. Shushruth et al.

Summary: The study of perceptual decision-making in monkeys has provided insights into how sensory evidence is integrated for decision-making. When monkeys are aware of the motor actions related to their decisions, the integration of evidence is carried out by neurons involved in the selection of these actions. However, it is unclear how monkeys make decisions when they are unaware of the required actions, which is referred to as abstract decisions. This study trained monkeys to associate the direction of motion with the color of two targets and found that monkeys postponed decision formation until the targets were revealed. Neurons in the parietal association area represented the integration of evidence leading to a choice, but the samples of evidence must have been retrieved from short-term memory since the stimulus was no longer visible.

CURRENT BIOLOGY (2022)

Article Multidisciplinary Sciences

People construct simplified mental representations to plan

Mark K. Ho et al.

Summary: One of the most striking features of human cognition is the ability to plan. This research challenges the traditional view that task representations are fixed, proposing that they can be controlled and simplified through online cognitive control. The study demonstrates that strategically perceiving and conceiving problems facilitates the effective use of limited cognitive resources.

NATURE (2022)

Review Neurosciences

Taking stock of value in the orbitofrontal cortex

Eric B. Knudsen et al.

Summary: Previous studies suggested that the orbitofrontal cortex (OFC) plays a central role in optimal decision-making by signaling values associated with different choices. However, new data indicate that the OFC may have a broader role in cognition by representing cognitive maps that guide behavior.

NATURE REVIEWS NEUROSCIENCE (2022)

Article Behavioral Sciences

Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making

R. Fromer et al.

Summary: Research on value-based decision making and cognitive control, although seemingly investigating different problems, have provided parallel insights into how people select thoughts and actions. Recent work highlights the role of decision-making research in informing our understanding of cognitive control, while also emphasizing the contributions of cognitive control research to decision-making.

NEUROSCIENCE AND BIOBEHAVIORAL REVIEWS (2022)

Review Behavioral Sciences

The computational roots of positivity and confirmation biases in reinforcement learning

Stefano Palminteri et al.

Summary: Humans tend to integrate new information subjectively, giving more weight to outcomes with positive affective value and evidence that confirms their prior beliefs. Previous studies believed that these biases were specific to "high-level" belief updates, but our findings provide evidence against this claim. The learning rates in reinforcement learning tasks show the same characteristic asymmetry across different contexts and species, indicating that belief and value updating processes share key computational principles and distortions.

TRENDS IN COGNITIVE SCIENCES (2022)

Review Behavioral Sciences

Abstract task representations for inference and control

Avinash R. Vaidya et al.

Summary: The article discusses the relationship between different brain networks in representing abstract task information and proposes the hypothesis that the differences in abstract task representations lie primarily in format rather than content.

TRENDS IN COGNITIVE SCIENCES (2022)

Article Multidisciplinary Sciences

Higher-dimensional neural representations predict better episodic memory

Jintao Sheng et al.

Summary: Episodic memory allows humans to encode and retrieve information about experiences. The human brain maintains a high-dimensional representation of faces to support accurate episodic memory. Greater dimensionality is associated with better memory performance. Frontoparietal activities can increase dimensionality and reduce correlations of local neural responses.

SCIENCE ADVANCES (2022)

Article Neurosciences

Neural mechanisms of credit assignment for inferred relationships in a structured world

Phillip P. Witkowski et al.

Summary: Animals use abstract representations of a task's structure to support accelerated learning and flexible behavior. This study investigates how humans infer specific choice-outcome associations through structured knowledge and finds that the medial prefrontal cortex efficiently represents hierarchically related associations and assigns credit for both experienced and inferred outcomes. Furthermore, the mPFC and lateral orbitofrontal cortex track the current position in a latent association space, supporting flexible learning and inference in structured environments.

NEURON (2022)

Article Computer Science, Cybernetics

From internal models toward metacognitive AI

Mitsuo Kawato et al.

Summary: Based on a review of literature from the past 20 years, a computational neuroscience model of metacognition is proposed, utilizing generative-inverse model pairs and a cognitive reality monitoring network (CRMN) in the prefrontal cortex. This model allows for the selection, learning, and monitoring of modules in perception, action, and reinforcement learning based on mismatches between generative and inverse models, as well as reward prediction errors.

BIOLOGICAL CYBERNETICS (2021)

Article Neurosciences

Ventromedial Prefrontal Cortex Drives the Prioritization of Self-Associated Stimuli in Working Memory

Shouhang Yin et al.

Summary: Humans tend to prioritize processing self-related information in working memory, with the ventromedial prefrontal cortex playing a causal role in driving this self-bias. Transcranial direct current stimulation can modulate this self-prioritization effect.

JOURNAL OF NEUROSCIENCE (2021)

Article Multidisciplinary Sciences

Activation and disruption of a neural mechanism for novel choice in monkeys

Alessandro Bongioanni et al.

Summary: The study found that the primate medial frontal cortex plays a key role in creatively inferring the value of new options in novel decision-making scenarios. Functional magnetic resonance imaging revealed that MFC carries out different comparison processes for unfamiliar and familiar options, using a multidimensional representation scheme resembling that of grid cells.

NATURE (2021)

Article Multidisciplinary Sciences

Sources of confidence in value-based choice

Jeroen Brus et al.

Summary: The study reveals that confidence in value-based decisions is influenced by endogenous attentional effort and downstream noise in the comparison process, rather than by fluctuations in the precision of value encoding.

NATURE COMMUNICATIONS (2021)

Article Psychology, Biological

Multi-task reinforcement learning in humans

Momchil S. Tomov et al.

Summary: The study shows that participants in complex environments tend to map previously learned strategies to new scenarios, with a strategy that combines successor features and generalized policy iteration predicting behavior best.

NATURE HUMAN BEHAVIOUR (2021)

Article Multidisciplinary Sciences

Shared mechanisms underlie the control of working memory and attention

Matthew F. Panichello et al.

Summary: Cognitive control guides behavior by controlling what, when, and how information is represented in the brain. Prefrontal cortex acts as a domain-general controller for both selection and attention, while parietal and visual cortex represent attention and selection independently. Selection and attention facilitate behavior by enhancing and transforming the representation of selected memory or attended stimulus.

NATURE (2021)

Article Multidisciplinary Sciences

How usefulness shapes neural representations during goal-directed behavior

G. Castegnetti et al.

Summary: The study found that regions in the prefrontal cortex can reshape their representations of value for perceptually identical items based on different goals, supporting generalization across contexts. This indicates that value cognition is not just about rewards, but can be flexibly adjusted with changing goals.

SCIENCE ADVANCES (2021)

Article Behavioral Sciences

The dimensionality of neural representations for control

David Badre et al.

Summary: Cognitive control theory focuses on the control representation that influences outputs based on contextual factors. The dimensionality of neural representations plays a crucial role in neural computation. The importance of the prefrontal cortex in control representations has been emphasized in recent neuroscience research.

CURRENT OPINION IN BEHAVIORAL SCIENCES (2021)

Article Biochemistry & Molecular Biology

Hippocampal neurons construct a map of an abstract value space

Eric B. Knudsen et al.

Summary: The hippocampus plays a critical role in encoding cognitive maps and value spaces. Research shows that in primates, hippocampal neurons encode abstract value spaces through value place fields, similar to how rodent place neurons encode paths through physical space. Value place fields remap when cues change, but also become increasingly correlated across contexts, allowing for generalized maps.
Article Biology

The role of ventromedial prefrontal cortex in reward valuation and future thinking during intertemporal choice

Elisa Ciaramelli et al.

Summary: The study revealed that patients with lesions to the ventromedial prefrontal cortex (vmPFC) exhibited steeper delay discounting compared to healthy controls, and it was not influenced by reward magnitude. However, cues for episodic future thinking (EFT) were able to reduce delay discounting in both vmPFC patients and controls.
Article Biology

Value signals guide abstraction during learning

Aurelio Cortese et al.

Summary: The human brain excels at constructing and using abstractions based on the valuation of sensory features, with high-value abstract representations guiding behavior and leading to better choices and higher confidence. The brain area computing value signals prioritizes latent task elements during abstraction, predicting a causal role for valuation. Neural reinforcement tagging in the sensory cortex evokes abstraction-based decisions, providing a novel interpretation of value as a key factor in forging abstract representations.
Article Psychology, Biological

Aesthetic preference for art can be predicted from a mixture of low- and high-level visual features

Kiyohito Iigaya et al.

Summary: The study found that aesthetic preferences for visual art can be predicted by a model that combines low-level and high-level image features, and that a convolutional neural network trained only on object recognition naturally encodes these features.

NATURE HUMAN BEHAVIOUR (2021)

Article Behavioral Sciences

The Case Against Economic Values in the Orbitofrontal Cortex (or Anywhere Else in the Brain)

Benjamin Y. Hayden et al.

Summary: This article presents an alternative view that economic value is not directly represented in the brain and used for choice by default. This view is influenced by economic and behavioral economics literature on value concepts, as well as recent neural and behavioral findings on how animals and humans learn to make choices.

BEHAVIORAL NEUROSCIENCE (2021)

Article Biochemistry & Molecular Biology

Reward Association Enhances Stimulus-Specific Representations in Primary Visual Cortex

Julia U. Henschke et al.

CURRENT BIOLOGY (2020)

Article Biochemistry & Molecular Biology

The Geometry of Abstraction in the Hippocampus and Prefrontal Cortex

Silvia Bernardi et al.

Article Multidisciplinary Sciences

Orbitofrontal control of visual cortex gain promotes visual associative learning

Dechen Liu et al.

NATURE COMMUNICATIONS (2020)

Article Biochemical Research Methods

Electrical coupling controls dimensionality and chaotic firing of inferior olive neurons

Huu Hoang et al.

PLOS COMPUTATIONAL BIOLOGY (2020)

Article Multidisciplinary Sciences

Value-guided remapping of sensory cortex by lateral orbitofrontal cortex

Abhishek Banerjee et al.

NATURE (2020)

Article Multidisciplinary Sciences

Unconscious reinforcement learning of hidden brain states supported by confidence

Aurelio Cortese et al.

NATURE COMMUNICATIONS (2020)

Article Multidisciplinary Sciences

Computational evidence for hierarchically structured reinforcement learning in humans

Maria K. Eckstein et al.

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA (2020)

Article Neurosciences

The neural and cognitive architecture for learning from a small sample

Aurelio Cortese et al.

CURRENT OPINION IN NEUROBIOLOGY (2019)

Article Multidisciplinary Sciences

Hierarchical reasoning by neural circuits in the frontal cortex

Morteza Sarafyazd et al.

SCIENCE (2019)

Review Behavioral Sciences

Reinforcement Learning, Fast and Slow

Matthew Botvinick et al.

TRENDS IN COGNITIVE SCIENCES (2019)

Article Biochemistry & Molecular Biology

Retrosplenial Cortical Representations of Space and Future Goal Locations Develop with Learning

Adam M. P. Miller et al.

CURRENT BIOLOGY (2019)

Article Multidisciplinary Sciences

Sequential replay of nonspatial task states in the human hippocampus

Nicolas W. Schuck et al.

SCIENCE (2019)

Article Neurosciences

Learning task-state representations

Yael Niv

NATURE NEUROSCIENCE (2019)

Review Behavioral Sciences

Where Does Value Come From?

Keno Juechems et al.

TRENDS IN COGNITIVE SCIENCES (2019)

Article Multidisciplinary Sciences

Frontal cortex neuron types categorically encode single decision variables

Junya Hirokawa et al.

NATURE (2019)

Article Psychology, Multidisciplinary

Gaze Amplifies Value in Decision Making

Stephanie M. Smith et al.

PSYCHOLOGICAL SCIENCE (2019)

Review Psychology, Biological

Dynamics of cognitive control: Theoretical bases, paradigms, and a view for the future

Gabriele Gratton et al.

PSYCHOPHYSIOLOGY (2018)

Article Biochemistry & Molecular Biology

Prefrontal Control of Visual Distraction

Joshua D. Cosman et al.

CURRENT BIOLOGY (2018)

Article Neurosciences

Neural mediators of changes of mind about perceptual decisions

Stephen M. Fleming et al.

NATURE NEUROSCIENCE (2018)

Article Neurosciences

Prefrontal cortex as a meta-reinforcement learning system

Jane X. Wang et al.

NATURE NEUROSCIENCE (2018)

Article Biochemistry & Molecular Biology

Confirmation Bias through Selective Overweighting of Choice-Consistent Evidence

Bharath Chandra Talluri et al.

CURRENT BIOLOGY (2018)

Editorial Material Behavioral Sciences

Stuck in the Present? Constraints on Children's Episodic Prospection

Simona Ghetti et al.

TRENDS IN COGNITIVE SCIENCES (2018)

Review Neurosciences

What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior

Timothy E. J. Behrens et al.

NEURON (2018)

Article Neurosciences

Social Information Is Integrated into Value and Confidence Judgments According to Its Reliability

Benedetto De Martino et al.

JOURNAL OF NEUROSCIENCE (2017)

Review Neurosciences

Managing competing goals - a key role for the frontopolar cortex

Farshad Alizadeh Mansouri et al.

NATURE REVIEWS NEUROSCIENCE (2017)

Review Behavioral Sciences

Neurobiology of Schemas and Schema-Mediated Memory

Asaf Gilboa et al.

TRENDS IN COGNITIVE SCIENCES (2017)

Article Psychology, Biological

Explicit representation of confidence informs future value-based decisions

Tomas Folke et al.

NATURE HUMAN BEHAVIOUR (2017)

Article Psychology, Biological

The successor representation in human reinforcement learning

I. Momennejad et al.

NATURE HUMAN BEHAVIOUR (2017)

Review Neurosciences

Why neurons mix: high dimensionality for higher cognition

Stefano Fusi et al.

CURRENT OPINION IN NEUROBIOLOGY (2016)

Article Multidisciplinary Sciences

Mastering the game of Go with deep neural networks and tree search

David Silver et al.

NATURE (2016)

Review Neurosciences

Decision Making and Sequential Sampling from Memory

Michael N. Shadlen et al.

NEURON (2016)

Article Neurosciences

Signatures of a Statistical Computation in the Human Sense of Confidence

Joshua I. Sanders et al.

NEURON (2016)

Article Multidisciplinary Sciences

Human-level control through deep reinforcement learning

Volodymyr Mnih et al.

NATURE (2015)

Article Neurosciences

Automatic integration of confidence in the brain valuation signal

Mael Lebreton et al.

NATURE NEUROSCIENCE (2015)

Article Neurosciences

Medial Prefrontal Cortex Predicts Internally Driven Strategy Shifts

Nicolas W. Schuck et al.

NEURON (2015)

Article Automation & Control Systems

Multiobjective Reinforcement Learning: A Comprehensive Overview

Chunming Liu et al.

IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS (2015)

Article Neurosciences

Schema Representation in Patients with Ventromedial PFC Lesions

Vanessa E. Ghosh et al.

JOURNAL OF NEUROSCIENCE (2014)

Review Behavioral Sciences

What is a memory schema? A historical perspective on current neuroscience literature

Vanessa E. Ghosh et al.

NEUROPSYCHOLOGIA (2014)

Article Multidisciplinary Sciences

Foundations of human reasoning in the prefrontal cortex

Mael Donoso et al.

SCIENCE (2014)

Article Psychology, Experimental

Eye Tracking and Pupillometry Are Indicators of Dissociable Latent Decision Processes

James F. Cavanagh et al.

JOURNAL OF EXPERIMENTAL PSYCHOLOGY-GENERAL (2014)

Article Psychology, Multidisciplinary

Two systems drive attention to rewards

Christopher K. Kovach et al.

FRONTIERS IN PSYCHOLOGY (2014)

Article Multidisciplinary Sciences

Context-dependent computation by recurrent dynamics in prefrontal cortex

Valerio Mante et al.

NATURE (2013)

Article Multidisciplinary Sciences

The importance of mixed selectivity in complex cognitive tasks

Mattia Rigotti et al.

NATURE (2013)

Article Neurosciences

Confidence in value-based choice

Benedetto De Martino et al.

NATURE NEUROSCIENCE (2013)

Article Neurosciences

Online evaluation of novel choices by simultaneous representation of multiple memories

Helen C. Barron et al.

NATURE NEUROSCIENCE (2013)

Article Multidisciplinary Sciences

High-order feature-based mixture models of classification learning predict individual learning curves and enable personalized teaching

Yarden Cohen et al.

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA (2013)

Article Biochemistry & Molecular Biology

Shaping of Object Representations in the Human Medial Temporal Lobe Based on Temporal Regularities

Anna C. Schapiro et al.

CURRENT BIOLOGY (2012)

Review Neurosciences

Category representation and generalization in the prefrontal cortex

Xiaochuan Pan et al.

EUROPEAN JOURNAL OF NEUROSCIENCE (2012)

Article Computer Science, Artificial Intelligence

MOSAIC for Multiple-Reward Environments

Norikazu Sugimoto et al.

NEURAL COMPUTATION (2012)

Article Multidisciplinary Sciences

Lesion mapping of cognitive control and value-based decision making in the prefrontal cortex

Jan Glaescher et al.

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA (2012)

Article Biochemistry & Molecular Biology

Reasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making

Anne Collins et al.

PLOS BIOLOGY (2012)

Article Economics

Modeling the Change of Paradigm: Non-Bayesian Reactions to Unexpected News

Pietro Ortoleva

AMERICAN ECONOMIC REVIEW (2012)

Review Psychology

A Taxonomy of External and Internal Attention

Marvin M. Chun et al.

ANNUAL REVIEW OF PSYCHOLOGY, VOL 62 (2011)

Article Multidisciplinary Sciences

Control of visual cortical signals by prefrontal dopamine

Behrad Noudoost et al.

NATURE (2011)

Article Multidisciplinary Sciences

Value-driven attentional capture

Brian A. Anderson et al.

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA (2011)

Article Multidisciplinary Sciences

Multialternative drift-diffusion model predicts the relationship between visual fixations and choice in value-based decisions

Ian Krajbich et al.

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA (2011)

Article Multidisciplinary Sciences

Schema-Dependent Gene Activation and Memory Encoding in Neocortex

Dorothy Tse et al.

SCIENCE (2011)

Article Neurosciences

Visual fixations and the computation and comparison of value in simple choice

Ian Krajbich et al.

NATURE NEUROSCIENCE (2010)

Article Neurosciences

Neural Evidence of Statistical Learning: Efficient Detection of Visual Regularities Without Awareness

Nicholas B. Turk-Browne et al.

JOURNAL OF COGNITIVE NEUROSCIENCE (2009)

Review Neurosciences

A framework for studying the neurobiology of value-based decision making

Antonio Rangel et al.

NATURE REVIEWS NEUROSCIENCE (2008)

Article Multidisciplinary Sciences

Activation of the prefrontal cortex in the human visual aesthetic perception

CJ Cela-Conde et al.

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA (2004)

Article Multidisciplinary Sciences

General conditions for predictivity in learning theory

T Poggio et al.

NATURE (2004)

Article Multidisciplinary Sciences

Generalization in vision and motor control

T Poggio et al.

NATURE (2004)

Article Biochemistry & Molecular Biology

Feature-based attention increases the selectivity of population responses in primate visual cortex

JC Martinez-Trujillo et al.

CURRENT BIOLOGY (2004)

Article Computer Science, Artificial Intelligence

Meta-learning in Reinforcement Learning

N Schweighofer et al.

NEURAL NETWORKS (2003)

Article Computer Science, Artificial Intelligence

Multiple model-based reinforcement learning

K Doya et al.

NEURAL COMPUTATION (2002)

Article Computer Science, Artificial Intelligence

Metalearning and neuromodulation

K Doya

NEURAL NETWORKS (2002)

Article Computer Science, Artificial Intelligence

MOSAIC model for sensorimotor learning and control

M Haruno et al.

NEURAL COMPUTATION (2001)

Article Multidisciplinary Sciences

Single neurons in prefrontal cortex encode abstract rules

JD Wallis et al.

NATURE (2001)

Article Multidisciplinary Sciences

Categorical representation of visual stimuli in the primate prefrontal cortex

DJ Freedman et al.

SCIENCE (2001)

Article Neurosciences

Prefrontal modulation of visual processing in humans

F Barceló et al.

NATURE NEUROSCIENCE (2000)