.png){kind=icon}
.png){kind=icon}
Return
Temporal encoding in deep reinforcement learning agents
.png){kind=icon}
OA
AI
DOI：10.1038/s41598-023-49847-y.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Neuroscientists have observed both cells in the brain that fire at specific points in time, known as time cells, and cells whose activity steadily increases or decreases over time, known as ramping cells. It is speculated that time and ramping cells support temporal computations in the brain and carry mnemonic information. However, due to the limitations in animal experiments, it is difficult to determine how these cells really contribute to behavior. Here, we show that time cells and ramping cells naturally emerge in the recurrent neural networks of deep reinforcement learning models performing simulated interval timing and working memory tasks, which have learned to estimate expected rewards in the future. We show that these cells do indeed carry information about time and items stored in working memory, but they contribute to behavior in large part by providing a dynamic representation on which policy can be computed. Moreover, the information that they do carry depends on both the task demands and the variables provided to the models. Our results suggest that time cells and ramping cells could contribute to temporal and mnemonic calculations, but the way in which they do so may be complex and unintuitive to human observers.
Keywords:
TIME CELLS
HIPPOCAMPAL-NEURONS
PARIETAL CORTEX
ELAPSED TIME
SEQUENCES
PLACE
REPRESENTATION
INFORMATION
MECHANISM
MEMORY
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.