Trading performance for stability in Markov decision processes

J 2017

Trading performance for stability in Markov decision processes

BRÁZDIL, Tomáš, Krishnendu CHATTERJEE, Vojtěch FOREJT and Antonín KUČERA

Basic information

Original name

Trading performance for stability in Markov decision processes

Authors

BRÁZDIL, Tomáš (203 Czech Republic, belonging to the institution), Krishnendu CHATTERJEE (40 Austria), Vojtěch FOREJT (203 Czech Republic, belonging to the institution) and Antonín KUČERA (203 Czech Republic, guarantor, belonging to the institution)

Edition

Journal of Computer and System Sciences, SAN DIEGO, Elsevier, 2017, 0022-0000

Other information

Language

English

Type of outcome

Článek v odborném periodiku

Field of Study

10201 Computer sciences, information science, bioinformatics

Country of publisher

United States of America

Confidentiality degree

není předmětem státního či obchodního tajemství

Impact factor

Impact factor: 1.497

RIV identification code

RIV/00216224:14330/17:00094587

Organization unit

Faculty of Informatics

DOI

http://dx.doi.org/10.1016/j.jcss.2016.09.009

UT WoS

000388430000011

Keywords in English

Markov decision processes; Mean payoff; Stability; Stochastic systems; Controller synthesis

Abstract

V originále

We study controller synthesis problems for finite-state Markov decision processes, where the objective is to optimize the expected mean-payoff performance and stability (also known as variability in the literature). We argue that the basic notion of expressing the stability using the statistical variance of the mean payoff is sometimes insufficient, and propose an alternative definition. We show that a strategy ensuring both the expected mean payoff and the variance below given bounds requires randomization and memory, under both the above definitions. We then show that the problem of finding such a strategy can be expressed as a set of constraints.

Links

GA15-17564S, research and development project

Name: Teorie her jako prostředek pro formální analýzu a verifikaci počítačových systémů

Investor: Czech Science Foundation

Citovat

BRÁZDIL, Tomáš, Krishnendu CHATTERJEE, Vojtěch FOREJT and Antonín KUČERA. Trading performance for stability in Markov decision processes. Journal of Computer and System Sciences. SAN DIEGO: Elsevier, 2017, vol. 84, No 2017, p. 144-170. ISSN 0022-0000. Available from: https://dx.doi.org/10.1016/j.jcss.2016.09.009.

@article{1366042,
   author = {Brázdil, Tomáš and Chatterjee, Krishnendu and Forejt, Vojtěch and Kučera, Antonín},
   article_location = {SAN DIEGO},
   article_number = {2017},
   doi = {http://dx.doi.org/10.1016/j.jcss.2016.09.009},
   keywords = {Markov decision processes; Mean payoff; Stability; Stochastic systems; Controller synthesis},
   language = {eng},
   issn = {0022-0000},
   journal = {Journal of Computer and System Sciences},
   title = {Trading performance for stability in Markov decision processes},
   volume = {84},
   year = {2017}
}

TY  - JOUR
ID  - 1366042
AU  - Brázdil, Tomáš - Chatterjee, Krishnendu - Forejt, Vojtěch - Kučera, Antonín
PY  - 2017
TI  - Trading performance for stability in Markov decision processes
JF  - Journal of Computer and System Sciences
VL  - 84
IS  - 2017
SP  - 144-170
EP  - 144-170
PB  - Elsevier
SN  - 00220000
KW  - Markov decision processes
KW  - Mean payoff
KW  - Stability
KW  - Stochastic systems
KW  - Controller synthesis
N2  - We study controller synthesis problems for finite-state Markov decision processes, where the objective is to optimize the expected mean-payoff performance and stability (also known as variability in the literature). We argue that the basic notion of expressing the stability using the statistical variance of the mean payoff is sometimes insufficient, and propose an alternative definition. We show that a strategy ensuring both the expected mean payoff and the variance below given bounds requires randomization and memory, under both the above definitions. We then show that the problem of finding such a strategy can be expressed as a set of constraints.
ER  -

BRÁZDIL, Tomáš, Krishnendu CHATTERJEE, Vojtěch FOREJT and Antonín KUČERA. Trading performance for stability in Markov decision processes. \textit{Journal of Computer and System Sciences}. SAN DIEGO: Elsevier, 2017, vol.~84, No~2017, p.~144-170. ISSN~0022-0000. Available from: https://dx.doi.org/10.1016/j.jcss.2016.09.009.

Detailed Information on Publication Record