Temporal logic motion control using actor-critic methods

doi:10.1177/0278364915581505

Temporal logic motion control using actor-critic methods

2015-05-26

23

OA

AI

Ioannis Ch. Paschalidis *

C

Călin Belta

DOI：10.1177/0278364915581505

Original

Save

View PDF

Abstract

En 中文

This paper considers the problem of deploying a robot from a specification given as a temporal logic statement about some properties satisfied by the regions of a large, partitioned environment. We assume that the robot has noisy sensors and actuators and model its motion through the regions of the environment as a Markov decision process (MDP). The robot control problem becomes finding the control policy which maximizes the probability of satisfying the temporal logic task on the MDP. For a large environment, obtaining transition probabilities for each state-action pair, as well as solving the necessary optimization problem for the optimal policy, are computationally intensive. To address these issues, we propose an approximate dynamic programming framework based on a least-squares temporal difference learning method of the actor-critic type. This framework operates on sample paths of the robot and optimizes a randomized control policy with respect to a small set of parameters. The transition probabilities are obtained only when needed. Simulations confirm that convergence of the parameters translates to an approximately optimal policy.

Keywords:

Motion planning

Markov decision processes

dynamic programming

actor-critic methods

Temporal logic motion control using actor-critic methods

Abstract

AI Summary

Journal

Organization