Compactly Restrictable Metric Policy Optimization Problems

doi:10.1109/TAC.2022.3217269

Compactly Restrictable Metric Policy Optimization Problems

2023-05-01

1

OA

AI

V

Victor D. Dorobantu *

K

Kamyar Azizzadenesheli

Y

Yisong Yue

DOI：10.1109/TAC.2022.3217269

Original

Original request for help

Save

Abstract

En 中文

We study policy optimization problems for deterministic Markov decision processes (MDPs) with metric state and action spaces, which we refer to as metric policy optimization problems (MPOPs). Our goal is to establish theoretical results on the well-posedness of MPOPs that can characterize practically relevant continuous control systems. To do so, we define a special class of MPOPs called compactly restrictable MPOPs (CR-MPOPs), which are flexible enough to capture the complex behavior of robotic systems but specific enough to admit solutions using dynamic programming methods such as value iteration. We show how to arrive at CR-MPOPs using forward-invariance. We further show that our theoretical results on CR-MPOPs can be used to characterize feedback linearizable control affine systems.

Keywords:

Optimization

Aerospace electronics

Extraterrestrial measurements

Control systems

Robots

Reinforcement learning

Markov processes

Continuous Markov decision processes (MDPs)

optimal control

physical systems

reinforcement learning

sampled-data

selection theorems

value iteration

Compactly Restrictable Metric Policy Optimization Problems

Abstract

Journal

Organization