arrow
返回

Distinct value computations support rapid sequential decisions

delete2023-11-21
delete1
delete
OA
AI
A
Andrew Mah
S
Shannon Sandra Schiereck
V
Veronica Bossio
C
Christine M. Constantinople *
DOI:10.1038/s41467-023-43250-xdelete
delete原文链接
delete分享
delete收藏
查看原文
摘要

摘要

En 中文
The value of the environment determines animals' motivational states and sets expectations for error-based learning1-3. How are values computed? Reinforcement learning systems can store or cache values of states or actions that are learned from experience, or they can compute values using a model of the environment to simulate possible futures3. These value computations have distinct trade-offs, and a central question is how neural systems decide which computations to use or whether/how to combine them4-8. Here we show that rats use distinct value computations for sequential decisions within single trials. We used high-throughput training to collect statistically powerful datasets from 291 rats performing a temporal wagering task with hidden reward states. Rats adjusted how quickly they initiated trials and how long they waited for rewards across states, balancing effort and time costs against expected rewards. Statistical modeling revealed that animals computed the value of the environment differently when initiating trials versus when deciding how long to wait for rewards, even though these decisions were only seconds apart. Moreover, value estimates interacted via a dynamic learning rate. Our results reveal how distinct value computations interact on rapid timescales, and demonstrate the power of using high-throughput training to understand rich, cognitive behaviors. How animals determine the value of the environment for motivation and error-based learning remains unclear. Here, the authors found that rats use multiple distinct algorithms to compute the value of the environment for rapid sequential actions on single trials.
Keyword:
ORBITOFRONTAL CORTEX
REWARD
MODEL
REPRESENTATION
INFORMATION
INFERENCE
SENSITIVITY
ADAPTATION
VALUATION
HABENULA
AI总结

AI总结

对已上传原文的论文进行重点信息的提取,主要内容包括:简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。

期刊

Nature Communications 封面图
Nature Communications
IF:
15.7
论文数:
9.6W
被引数:
126.4W

机构

N
New York University
学者数:
1.6K
论文数: 767
被引数: 3.2W
引用论文

引用论文

Inference-Based Decisions in a Hidden State Foraging Task: Differential Contributions of Prefrontal Cortical Areas
errNEURON
IF15
err2020-04-01
err40
errOAAI
errVertechi, Pietro; Lottem, Eran; Sarra, Dario; Godinho, Beatriz; Treves, Isaac; Quendera, Tiago; Lohuis, Matthijs Nicolai Oude; Mainen, Zachary F.
err分享
err收藏
Orbitofrontal Cortex as a Cognitive Map of Task Space
errNEURON
IF15
err2014-01-01
err630
errOAAI
errWilson, Robert C.; Takahashi, Yuji K.; Schoenbaum, Geoffrey; Niv, Yael
err分享
err收藏
Model-Based Influences on Humans' Choices and Striatal Prediction Errors基于模型对人类选择和纹状体预测误差的影响
errNEURON
IF15
err2011-03-01
err1.2K
errOAAI
errDaw, Nathaniel D.; Gershman, Samuel J.; Seymour, Ben; Dayan, Peter; Dolan, Raymond J.
err分享
err收藏
DNA binding by the ETS domain
err1996-11-01
err0
errOAAI
errBarbara J. Graves; Marc E. Gillespie; Lawrence P. Mclntosh
err分享
err收藏
Multiple Timescales of Memory in Lateral Habenula and Dopamine Neurons
errNEURON
IF15
err2010-08-01
err77
errOAAI
errBromberg-Martin, Ethan S.; Matsumoto, Masayuki; Nakahara, Hiroyuki; Hikosaka, Okihide
err分享
err收藏
The obligately lichenicolous genus Lichenoconium represents a novel lineage in the Dothideomycetes
err2011-02-01
err0
PREAI
errJames D. Lawrey; Paul Diederich; Matthew P. Nelsen; Masoumeh Sikaroodi; Patrick M. Gillevet; A. Maarten Brand; Pieter van den Boom
err分享
err收藏
Information Processing in Decision- Making Systems
err2012-04-09
err144
errOAAI
errvan der Meer, Matthijs; Kurth-Nelson, Zeb; Redish, A. David
err分享
err收藏
学者 查看更多内容