Trading Performance for Stability in Markov Decision Processes

D 2013

Trading Performance for Stability in Markov Decision Processes

BRÁZDIL, Tomáš, Krishnendu CHATTERJEE, Vojtěch FOREJT and Antonín KUČERA

Basic information

Original name

Trading Performance for Stability in Markov Decision Processes

Authors

BRÁZDIL, Tomáš (203 Czech Republic, belonging to the institution), Krishnendu CHATTERJEE (356 India), Vojtěch FOREJT (203 Czech Republic, belonging to the institution) and Antonín KUČERA (203 Czech Republic, guarantor, belonging to the institution)

Edition

London, Proceedings of 28th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS 2013), p. 331-340, 10 pp. 2013

Publisher

IEEE Computer Society

Other information

Language

English

Type of outcome

Stať ve sborníku

Field of Study

10201 Computer sciences, information science, bioinformatics

Country of publisher

United States of America

Confidentiality degree

není předmětem státního či obchodního tajemství

Publication form

storage medium (CD, DVD, flash disk)

RIV identification code

RIV/00216224:14330/13:00066541

Organization unit

Faculty of Informatics

ISBN

978-1-4799-0413-6

ISSN

DOI

http://dx.doi.org/10.1109/LICS.2013.39

UT WoS

000326815000038

Keywords (in Czech)

Markovovy rozhodovací procesy; optimalizace

Keywords in English

Markov decision processes; optimization

Abstract

V originále

We study the complexity of central controller synthesis problems for finite-state Markov decision processes, where the objective is to optimize both the expected mean-payoff performance of the system and its stability. We argue that the basic theoretical notion of expressing the stability in terms of the variance of the mean-payoff (called global variance in our paper) is not always sufficient, since it ignores possible instabilities on respective runs. For this reason we propose alernative definitions of stability, which we call local and hybrid variance, and which express how rewards on each run deviate from the run's own mean-payoff and from the expected mean-payoff, respectively.

Links

GPP202/12/P612, research and development project

Name: Formální verifikace stochastických systémů s reálným časem (Acronym: Formální verifikace stochastických systémů s reáln)

Investor: Czech Science Foundation

Citovat

BRÁZDIL, Tomáš, Krishnendu CHATTERJEE, Vojtěch FOREJT and Antonín KUČERA. Trading Performance for Stability in Markov Decision Processes. In Proceedings of 28th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS 2013). London: IEEE Computer Society, 2013, p. 331-340. ISBN 978-1-4799-0413-6. Available from: https://dx.doi.org/10.1109/LICS.2013.39.

@inproceedings{1130508,
   author = {Brázdil, Tomáš and Chatterjee, Krishnendu and Forejt, Vojtěch and Kučera, Antonín},
   address = {London},
   booktitle = {Proceedings of 28th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS 2013)},
   doi = {http://dx.doi.org/10.1109/LICS.2013.39},
   keywords = {Markov decision processes; optimization},
   howpublished = {paměťový nosič},
   language = {eng},
   location = {London},
   isbn = {978-1-4799-0413-6},
   pages = {331-340},
   publisher = {IEEE Computer Society},
   title = {Trading Performance for Stability in Markov Decision Processes},
   year = {2013}
}

TY  - JOUR
ID  - 1130508
AU  - Brázdil, Tomáš - Chatterjee, Krishnendu - Forejt, Vojtěch - Kučera, Antonín
PY  - 2013
TI  - Trading Performance for Stability in Markov Decision Processes
PB  - IEEE Computer Society
CY  - London
SN  - 9781479904136
KW  - Markov decision processes
KW  - optimization
N2  - We study the complexity of central controller synthesis problems for finite-state Markov decision processes, where the objective is to optimize both the expected mean-payoff performance of the system and its stability. We argue that the basic theoretical notion of expressing the stability in terms of the variance of the mean-payoff (called global variance in our paper) is not always sufficient, since it ignores possible instabilities on respective runs. For this reason we propose alernative definitions of stability, which we call local and hybrid variance, and which express how rewards on each run deviate from the run's own mean-payoff and from the expected mean-payoff, respectively.
ER  -

BRÁZDIL, Tomáš, Krishnendu CHATTERJEE, Vojtěch FOREJT and Antonín KUČERA. Trading Performance for Stability in Markov Decision Processes. In \textit{Proceedings of 28th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS 2013)}. London: IEEE Computer Society, 2013, p.~331-340. ISBN~978-1-4799-0413-6. Available from: https://dx.doi.org/10.1109/LICS.2013.39.

Detailed Information on Publication Record