.png){kind=icon}
Return
Model selection in reinforcement learning
.png){kind=icon}
OA
AI
DOI：10.1007/s10994-011-5254-7.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
We consider the problem of model selection in the batch (offline, non-interactive) reinforcement learning setting when the goal is to find an action-value function with the smallest Bellman error among a countable set of candidates functions. We propose a complexity regularization-based model selection algorithm, BERMIN, and prove that it enjoys an oracle-like property: the estimator's error differs from that of an oracle, who selects the candidate with the minimum Bellman error, by only a constant factor and a small remainder term that vanishes at a parametric rate as the number of samples increases. As an application, we consider a problem when the true action-value function belongs to an unknown member of a nested sequence of function spaces. We show that under some additional technical conditions BERMIN leads to a procedure whose rate of convergence, up to a constant factor, matches that of an oracle who knows which of the nested function spaces the true action-value function belongs to, i.e., the procedure achieves adaptivity.
Keywords:
Reinforcement learning
Model selection
Complexity regularization
Adaptivity
Offline learning
Off-policy learning
Finite-sample bounds
Journal
IF:
2.9
Papers:
2.7K
Citations:
3.4W
Organization
Cited Papers
Performance and Emissions Comparison between Biomethane and Natural Gas Fuel in Passenger VehiclesOAAI
OAAIShareSaveConsideraciones sobre el ensayo de ANSTETT y el comportamiento de los cementos frente a sulfatosOAAI
OAAIShareSaveAn Analytical Model Based on Strain Localisation for the Study of Size‐Scale and Slenderness Effects in Uniaxial Compression Tests.png)
Strain
OAAI
StrainIF0
OAAIShareSaveShareSaveShareSaveDispersion and Beating of Bacterial Cellulose and their Influence on Paper PropertiesBioResourcesOAAI
BioResourcesIF0
OAAIShareSaveShareSave
View more