.png){kind=icon}
返回
Multitime scale Markov decision processes
.png){kind=icon}
OA
AI
DOI：10.1109/TAC.2003.812782.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
This paper proposes a simple analytical model called M time scale Markov decision process (MMDPs) for hierarchically structured sequential decision making processes, where decisions in each level in the M-level hierarchy are made in M different discrete time scales. In this model, the state-space and the control-space of each level in the hierarchy are nonoverlapping with those of the other levels, respectively, and the hierarchy is structured in a pyramid sense such that a decision made at level m (slower time scale) state and/or the state will affect the evolutionary decision making process of the lower level m + 1 (faster time scale) until a new decision is made at the higher level but the lower level decisions themselves do not affect the transition dynamics of higher levels. The performance produced by the lower level decisions will affect the higher level decisions. A hierarchical objective function is defined such that the, finite-horizon value of following a (nonstationary) policy at level m + 1 over a decision epoch of level m plus an immediate reward at level m is the single-step reward for the decision making process at level m. From this we define multi-level optimal value function and derive multi-level optimality equation. We discuss how to solve MMDPs exactly and study some approximation methods, along with heuristic, sampling-based schemes, to solve MMDPs.
Keyword:
hierarchical control
Markov decision process (MDP)
multitime scale
rolling horizon
期刊
IF:
7
论文数:
1.3W
被引数:
6.7W
机构
暂无机构信息
引用论文
.png){kind=icon}
Critical Evaluation and Thermodynamic Optimization of the Li2O-Al2O3 and Li2O-MgO-Al2O3 Systems
PREAI
分享收藏
查看更多内容