返回
Distinct value computations support rapid sequential decisions
DOI:10.1038/s41467-023-43250-x.png)
摘要
En 中文
The value of the environment determines animals' motivational states and sets expectations for error-based learning1-3. How are values computed? Reinforcement learning systems can store or cache values of states or actions that are learned from experience, or they can compute values using a model of the environment to simulate possible futures3. These value computations have distinct trade-offs, and a central question is how neural systems decide which computations to use or whether/how to combine them4-8. Here we show that rats use distinct value computations for sequential decisions within single trials. We used high-throughput training to collect statistically powerful datasets from 291 rats performing a temporal wagering task with hidden reward states. Rats adjusted how quickly they initiated trials and how long they waited for rewards across states, balancing effort and time costs against expected rewards. Statistical modeling revealed that animals computed the value of the environment differently when initiating trials versus when deciding how long to wait for rewards, even though these decisions were only seconds apart. Moreover, value estimates interacted via a dynamic learning rate. Our results reveal how distinct value computations interact on rapid timescales, and demonstrate the power of using high-throughput training to understand rich, cognitive behaviors. How animals determine the value of the environment for motivation and error-based learning remains unclear. Here, the authors found that rats use multiple distinct algorithms to compute the value of the environment for rapid sequential actions on single trials.
Keyword:
ORBITOFRONTAL CORTEX
REWARD
MODEL
REPRESENTATION
INFORMATION
INFERENCE
SENSITIVITY
ADAPTATION
VALUATION
HABENULA
AI总结
对已上传原文的论文进行重点信息的提取,主要内容包括:简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
15.7
论文数:
9.6W
被引数:
126.4W
机构
引用论文
Inference-Based Decisions in a Hidden State Foraging Task: Differential Contributions of Prefrontal Cortical Areas
NEURON
IF15
Model-Based Influences on Humans' Choices and Striatal Prediction Errors基于模型对人类选择和纹状体预测误差的影响
NEURON
IF15

