Optimizing Expectation with Guarantees in POMDPs

D 2017

Optimizing Expectation with Guarantees in POMDPs

CHATTERJEE, Krishnendu; Petr NOVOTNÝ; Guillermo A. PÉREZ; Jean-Francois RASKIN; Djordje ŽIKELIĆ et al.

Základní údaje

Originální název

Optimizing Expectation with Guarantees in POMDPs

Autoři

CHATTERJEE, Krishnendu; Petr NOVOTNÝ; Guillermo A. PÉREZ; Jean-Francois RASKIN a Djordje ŽIKELIĆ

Vydání

Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI), od s. 3725--3732, 2017

Nakladatel

AAAI Press

Další údaje

Typ výsledku

Stať ve sborníku

Utajení

není předmětem státního či obchodního tajemství

Forma vydání

elektronická verze "online"

Odkazy

URL

Označené pro přenos do RIV

EID Scopus

2-s2.0-85030468679

Klíčová slova anglicky

Partially-observable Markov decision processes; Discounted payoff; Probabilistic planning; Verification

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 26. 9. 2019 09:44, doc. RNDr. Petr Novotný, Ph.D.

Anotace

V originále

A standard objective in partially-observable Markov decision processes (POMDPs) is to find a policy that maximizes the expected discounted-sum payoff. However, such policies may still permit unlikely but highly undesirable outcomes, which is problematic especially in safety-critical applications. Recently, there has been a surge of interest in POMDPs where the goal is to maximize the probability to ensure that the payoff is at least a given threshold, but these approaches do not consider any optimization beyond satisfying this threshold constraint. In this work we go beyond both the “expectation” and “threshold” approaches and consider a “guaranteed payoff optimization (GPO)” problem for POMDPs, where we are given a threshold t and the objective is to find a policy σ such that a) each possible outcome of σ yields a discounted-sum payoff of at least t, and b) the expected discounted-sum payoff of σ is optimal (or near-optimal) among all policies satisfying a). We present a practical approach to tackle the GPO problem and evaluate it on standard POMDP benchmarks.

Citovat

CHATTERJEE, Krishnendu; Petr NOVOTNÝ; Guillermo A. PÉREZ; Jean-Francois RASKIN a Djordje ŽIKELIĆ. Optimizing Expectation with Guarantees in POMDPs. Online. In Satinder P. Singh and Shaul Markovitch. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI). AAAI Press, 2017, s. 3725--3732.

@inproceedings{1562480,
   author = {Chatterjee, Krishnendu and Novotný, Petr and Pérez, Guillermo A. and Raskin, JeanandFrancois and Žikelić, Djordje},
   booktitle = {Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI)},
   editor = {Satinder P. Singh and Shaul Markovitch},
   keywords = {Partially-observable Markov decision processes; Discounted payoff; Probabilistic planning; Verification},
   howpublished = {elektronická verze "online"},
   pages = {3725--3732},
   publisher = {AAAI Press},
   title = {Optimizing Expectation with Guarantees in POMDPs},
   url = {http://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14354},
   year = {2017}
}

TY  - CONF
ID  - 1562480
AU  - Chatterjee, Krishnendu - Novotný, Petr - Pérez, Guillermo A. - Raskin, Jean-Francois - Žikelić, Djordje
PY  - 2017
TI  - Optimizing Expectation with Guarantees in POMDPs
PB  - AAAI Press
KW  - Partially-observable Markov decision processes
KW  - Discounted payoff
KW  - Probabilistic planning
KW  - Verification
UR  - http://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14354
N2  - A standard objective in partially-observable Markov decision processes (POMDPs) is to find a policy that maximizes the expected discounted-sum payoff. However, such policies may still permit unlikely but highly undesirable outcomes, which is problematic especially in safety-critical applications. Recently, there has been a surge of interest in POMDPs where the goal is to maximize the probability to ensure that the payoff is at least a given threshold, but these approaches do not consider any optimization beyond satisfying this threshold constraint. In this work we go beyond both the “expectation” and “threshold” approaches and consider a “guaranteed payoff optimization (GPO)” problem for POMDPs, where we are given a threshold t and the objective is to find a policy σ such that a) each possible outcome of σ yields a discounted-sum payoff of at least t, and b) the expected discounted-sum payoff of σ is optimal (or near-optimal) among all policies satisfying a). We present a practical approach to tackle the GPO problem and evaluate it on standard POMDP benchmarks.
ER  -

CHATTERJEE, Krishnendu; Petr NOVOTNÝ; Guillermo A. PÉREZ; Jean-Francois RASKIN a Djordje ŽIKELI$\backslash$'C. Optimizing Expectation with Guarantees in POMDPs. Online. In Satinder P. Singh and Shaul Markovitch. \textit{Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI)}. AAAI Press, 2017, s.~3725--3732.

Přehled o publikaci