ePrints

Verification of General Markov Decision Processes by Approximate Similarity Relations and Policy Refinement

Lookup NU author(s): Dr Sadegh SoudjaniORCiD

Downloads

• Accepted version [.pdf]

Licence

This is the authors' accepted manuscript of an article that has been published in its final definitive form by Society for Industrial and Applied Mathematics, 2017.

For re-use rights please refer to the publisher's terms and conditions.

Abstract

In this work we introduce new approximate similarity relations that are shown to be key for policy (or control) synthesis over general Markov decision processes. The models of interest are discrete-time Markov decision processes, endowed with uncountably-infinite state spaces and metric output (or observation) spaces. The new relations, underpinned by the use of metrics, allow in particular for a useful trade-off between deviations over probability distributions on states, and distances between model outputs. We show that the new probabilistic similarity relations, inspired by a notion of simulation developed for finite-state models, can be effectively employed over general Markov decision processes for verification purposes, and specifically for control refinement from abstract models.

Publication metadata

Author(s): Haesaert S, Esmaeil Zadeh Soudjani S, Abate A

Publication type: Article

Publication status: Published

Journal: SIAM Journal on Control and Optimization

Year: 2017

Volume: 55

Issue: 4

Pages: 2333-2367

Online publication date: 03/08/2017

Acceptance date: 14/03/2017

Date deposited: 07/11/2019

ISSN (print): 0363-0129

ISSN (electronic): 1095-7138

Publisher: Society for Industrial and Applied Mathematics

URL: https://doi.org/10.1137/16M1079397

DOI: 10.1137/16M1079397

Altmetrics

Share