.png){kind=icon}
.png){kind=icon}
Return
Numerical method for solving impulse control problems in partially observed piecewise deterministic Markov processes
.png){kind=icon}
PRE
AI
DOI：10.1017/apr.2025.10044.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Designing efficient and rigorous numerical methods for sequential decision-making under uncertainty is a difficult problem that arises in many applications frameworks. In this paper we focus on the numerical solution of a subclass of impulse control problems for the piecewise deterministic Markov process (PDMP) when the jump times are hidden. We first state the problem as a partially observed Markov decision process (POMDP) on a continuous state space and with controlled transition kernels corresponding to some specific skeleton chains of the PDMP. We then proceed to build a numerically tractable approximation of the POMDP by tailor-made discretizations of the state spaces. The main difficulty in evaluating the discretization error comes from the possible random jumps of the PDMP between consecutive epochs of the POMDP and requires special care. Finally, we discuss the practical construction of discretization grids and illustrate our method on simulations.
Keywords:
Continuous time Markov process
dynamic programming
hidden process
numerical approximation
partially observed Markov decision process
Journal
A
IF:
1.2
Papers:
41
Citations:
0