Multi-objective Discounted Reward Verification in Graphs and
MDPs

D 2013

Multi-objective Discounted Reward Verification in Graphs and MDPs

CHATTERJEE, Krishnendu, Vojtěch FOREJT a Dominik WOJTCZAK

Základní údaje

Originální název

Multi-objective Discounted Reward Verification in Graphs and MDPs

Autoři

CHATTERJEE, Krishnendu (356 Indie), Vojtěch FOREJT (203 Česká republika, garant, domácí) a Dominik WOJTCZAK (616 Polsko)

Vydání

Berlin, Heidelberg, Logic for Programming, Artificial Intelligence, and Reasoning, od s. 228-242, 15 s. 2013

Nakladatel

Springer

Další údaje

Jazyk

angličtina

Typ výsledku

Stať ve sborníku

Obor

10201 Computer sciences, information science, bioinformatics

Stát vydavatele

Německo

Utajení

není předmětem státního či obchodního tajemství

Forma vydání

tištěná verze "print"

Impakt faktor

Impact factor: 0.402 v roce 2005

Kód RIV

RIV/00216224:14330/13:00072859

Organizační jednotka

Fakulta informatiky

ISBN

978-3-642-45220-8

ISSN

DOI

http://dx.doi.org/10.1007/978-3-642-45221-5_17

Klíčová slova anglicky

multi-objective verification; markov decision processes; graphs

Štítky

core_A, firank_A

Změněno: 29. 4. 2014 20:09, RNDr. Pavel Šmerk, Ph.D.

Anotace

V originále

We study the problem of achieving a given value in Markov decision processes (MDPs) with several independent discounted reward objectives. We consider a generalised version of discounted reward objectives, in which the amount of discounting depends on the states visited and on the objective. This definition extends the usual definition of discounted reward, and allows to capture the systems in which the value of different commodities diminish at different and variable rates. We establish results for two prominent subclasses of the problem, namely state-discount models where the discount factors are only dependent on the state of the MDP (and independent of the objective), and reward-discount models where they are only dependent on the objective (but not on the state of the MDP). For the state-discount models we use a straightforward reduction to expected total reward and show that the problem whether a value is achievable can be solved in polynomial time. For the reward-discount model we show that memory and randomisation of the strategies are required, but nevertheless that the problem is decidable and it is sufficient to consider strategies which after a certain number of steps behave in a memoryless way. For the general case, we show that when restricted to graphs (i.e. MDPs with no randomisation), pure strategies and discount factors of the form 1/n where n is an integer, the problem is in PSPACE and finite memory suffices for achieving a given value. We also show that when the discount factors are not of the form 1/n, the memory required by a strategy can be infinite.

Návaznosti

LG13010, projekt VaV

Název: Zastoupení ČR v European Research Consortium for Informatics and Mathematics (Akronym: ERCIM-CZ)

Investor: Ministerstvo školství, mládeže a tělovýchovy ČR, Zastoupení ČR v European Research Consortium for Informatics and Mathematics

MUNI/33/IP1/2013, interní kód MU

Název: Podpora perspektivních výzkumných týmů Fakulty informatiky a vynikajících vědeckých pracovníků z jiných institucí působících na Fakultě informatiky (Akronym: PVT-VVPZ)

Investor: Masarykova univerzita, Podpora perspektivních výzkumných týmů Fakulty informatiky a vynikajících vědeckých pracovníků z jiných institucí působících na Fakultě informatiky

Citovat

CHATTERJEE, Krishnendu, Vojtěch FOREJT a Dominik WOJTCZAK. Multi-objective Discounted Reward Verification in Graphs and MDPs. In Kenneth L. McMillan and Aart Middeldorp and Andrei Voronkov. Logic for Programming, Artificial Intelligence, and Reasoning. Berlin, Heidelberg: Springer, 2013, s. 228-242. ISBN 978-3-642-45220-8. Dostupné z: https://dx.doi.org/10.1007/978-3-642-45221-5_17.

@inproceedings{1174049,
   author = {Chatterjee, Krishnendu and Forejt, Vojtěch and Wojtczak, Dominik},
   address = {Berlin, Heidelberg},
   booktitle = {Logic for Programming, Artificial Intelligence, and Reasoning},
   doi = {http://dx.doi.org/10.1007/978-3-642-45221-5_17},
   editor = {Kenneth L. McMillan and Aart Middeldorp and Andrei Voronkov},
   keywords = {multi-objective verification; markov decision processes; graphs},
   howpublished = {tištěná verze "print"},
   language = {eng},
   location = {Berlin, Heidelberg},
   isbn = {978-3-642-45220-8},
   pages = {228-242},
   publisher = {Springer},
   title = {Multi-objective Discounted Reward Verification in Graphs and MDPs},
   year = {2013}
}

TY  - JOUR
ID  - 1174049
AU  - Chatterjee, Krishnendu - Forejt, Vojtěch - Wojtczak, Dominik
PY  - 2013
TI  - Multi-objective Discounted Reward Verification in Graphs and MDPs
PB  - Springer
CY  - Berlin, Heidelberg
SN  - 9783642452208
KW  - multi-objective verification
KW  - markov decision processes
KW  - graphs
N2  - We study the problem of achieving a given value in Markov decision processes (MDPs) with several independent discounted reward objectives. We consider a generalised version of discounted reward objectives, in which the amount of discounting depends on the states visited and on the objective. This definition extends the usual definition of discounted reward, and allows to capture the systems in which the value of different commodities diminish at different and variable rates. We establish results for two prominent subclasses of the problem, namely state-discount models where the discount factors are only dependent on the state of the MDP (and independent of the objective), and reward-discount models where they are only dependent on the objective (but not on the state of the MDP). For the state-discount models we use a straightforward reduction to expected total reward and show that the problem whether a value is achievable can be solved in polynomial time. For the reward-discount model we show that memory and randomisation of the strategies are required, but nevertheless that the problem is decidable and it is sufficient to consider strategies which after a certain number of steps behave in a memoryless way. For the general case, we show that when restricted to graphs (i.e. MDPs with no randomisation), pure strategies and discount factors of the form 1/n where n is an integer, the problem is in PSPACE and finite memory suffices for achieving a given value. We also show that when the discount factors are not of the form 1/n, the memory required by a strategy can be infinite.
ER  -

CHATTERJEE, Krishnendu, Vojtěch FOREJT a Dominik WOJTCZAK. Multi-objective Discounted Reward Verification in Graphs and MDPs. In Kenneth L. McMillan and Aart Middeldorp and Andrei Voronkov. \textit{Logic for Programming, Artificial Intelligence, and Reasoning}. Berlin, Heidelberg: Springer, 2013, s.~228-242. ISBN~978-3-642-45220-8. Dostupné z: https://dx.doi.org/10.1007/978-3-642-45221-5\_{}17.

Podrobný výpis o publikaci