Multiple-Environment Markov Decision Processes: Efficient
Analysis and Applications

D 2020

Multiple-Environment Markov Decision Processes: Efficient Analysis and Applications

CHATTERJEE, Krishnendu; Martin CHMELÍK; Deep KARKHANIS; Petr NOVOTNÝ; Amélie ROYER et. al.

Základní údaje

Originální název

Multiple-Environment Markov Decision Processes: Efficient Analysis and Applications

Autoři

CHATTERJEE, Krishnendu; Martin CHMELÍK; Deep KARKHANIS; Petr NOVOTNÝ a Amélie ROYER

Vydání

Palo Alto, Proceedings of the International Conference on Automated Planning and Scheduling, od s. 48-56, 9 s. 2020

Nakladatel

AAAI Press

Další údaje

Jazyk

angličtina

Typ výsledku

Stať ve sborníku

Obor

10200 1.2 Computer and information sciences

Stát vydavatele

Spojené státy

Utajení

není předmětem státního či obchodního tajemství

Forma vydání

elektronická verze "online"

Odkazy

URL

Kód RIV

RIV/00216224:14330/20:00114616

Organizační jednotka

Fakulta informatiky

ISBN

978-1-57735-824-4

ISSN

EID Scopus

2-s2.0-85088499875

Klíčová slova anglicky

decision making; Markov decision processes; contextual recommendations

Štítky

core_A, firank_1, formela-dec

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 29. 4. 2021 08:12, RNDr. Pavel Šmerk, Ph.D.

Anotace

V originále

Multiple-environment Markov decision processes (MEMDPs) are MDPs equipped with not one, but multiple probabilistic transition functions, which represent the various possible unknown environments. While the previous research on MEMDPs focused on theoretical properties for long-run average payoff, we study them with discounted-sum payoff and focus on their practical advantages and applications. MEMDPs can be viewed as a special case of Partially observable and Mixed observability MDPs: the state of the system is perfectly observable, but not the environment. We show that the specific structure of MEMDPs allows for more efficient algorithmic analysis, in particular for faster belief updates. We demonstrate the applicability of MEMDPs in several domains. In particular, we formalize the sequential decision-making approach to contextual recommendation systems as MEMDPs and substantially improve over the previous MDP approach.

Návaznosti

GA19-15134Y, interní kód MU

Název: Verifikace a analýza pravděpodobnostních programů

Investor: Grantová agentura ČR, Verifikace a analýza pravděpodobnostních programů

GJ19-15134Y, projekt VaV

Název: Verifikace a analýza pravděpodobnostních programů

Citovat

CHATTERJEE, Krishnendu; Martin CHMELÍK; Deep KARKHANIS; Petr NOVOTNÝ a Amélie ROYER. Multiple-Environment Markov Decision Processes: Efficient Analysis and Applications. Online. In Proceedings of the International Conference on Automated Planning and Scheduling. Palo Alto: AAAI Press, 2020, s. 48-56. ISBN 978-1-57735-824-4.

@inproceedings{1718276,
   author = {Chatterjee, Krishnendu and Chmelík, Martin and Karkhanis, Deep and Novotný, Petr and Royer, Amélie},
   address = {Palo Alto},
   booktitle = {Proceedings of the International Conference on Automated Planning and Scheduling},
   keywords = {decision making; Markov decision processes; contextual recommendations},
   howpublished = {elektronická verze "online"},
   language = {eng},
   location = {Palo Alto},
   isbn = {978-1-57735-824-4},
   pages = {48-56},
   publisher = {AAAI Press},
   title = {Multiple-Environment Markov Decision Processes: Efficient Analysis and Applications},
   url = {https://ojs.aaai.org//index.php/ICAPS/article/view/6644},
   year = {2020}
}

TY  - CONF
ID  - 1718276
AU  - Chatterjee, Krishnendu - Chmelík, Martin - Karkhanis, Deep - Novotný, Petr - Royer, Amélie
PY  - 2020
TI  - Multiple-Environment Markov Decision Processes: Efficient Analysis and Applications
PB  - AAAI Press
CY  - Palo Alto
SN  - 9781577358244
KW  - decision making
KW  - Markov decision processes
KW  - contextual recommendations
UR  - https://ojs.aaai.org//index.php/ICAPS/article/view/6644
L2  - https://ojs.aaai.org//index.php/ICAPS/article/view/6644
N2  - Multiple-environment Markov decision processes (MEMDPs) are MDPs equipped with not one, but multiple probabilistic transition functions, which represent the various possible unknown environments. While the previous research on MEMDPs focused on theoretical properties for long-run average payoff, we study them with discounted-sum payoff and focus on their practical advantages and applications. MEMDPs can be viewed as a special case of Partially observable and Mixed observability MDPs: the state of the system is perfectly observable, but not the environment. We show that the specific structure of MEMDPs allows for more efficient algorithmic analysis, in particular for faster belief updates. We demonstrate the applicability of MEMDPs in several domains. In particular, we formalize the sequential decision-making approach to contextual recommendation systems as MEMDPs and substantially improve over the previous MDP approach.
ER  -

CHATTERJEE, Krishnendu; Martin CHMELÍK; Deep KARKHANIS; Petr NOVOTNÝ a Amélie ROYER. Multiple-Environment Markov Decision Processes: Efficient Analysis and Applications. Online. In \textit{Proceedings of the International Conference on Automated Planning and Scheduling}. Palo Alto: AAAI Press, 2020, s.~48-56. ISBN~978-1-57735-824-4.

Přehled o publikaci