Shielding in Resource-Constrained Goal POMDPs

D 2023

Shielding in Resource-Constrained Goal POMDPs

AJDARÓW, Michal, Šimon BRLEJ a Petr NOVOTNÝ

Základní údaje

Originální název

Shielding in Resource-Constrained Goal POMDPs

Autoři

AJDARÓW, Michal (203 Česká republika, domácí), Šimon BRLEJ (703 Slovensko, domácí) a Petr NOVOTNÝ (203 Česká republika, domácí)

Vydání

Washington, DC, USA, Proceedings of the 37th AAAI Conference on Artificial Intelligence, od s. 14674-14682, 9 s. 2023

Nakladatel

AAAI Press

Další údaje

Jazyk

angličtina

Typ výsledku

Stať ve sborníku

Obor

10200 1.2 Computer and information sciences

Stát vydavatele

Spojené státy

Utajení

není předmětem státního či obchodního tajemství

Forma vydání

elektronická verze "online"

Odkazy

URL

Kód RIV

RIV/00216224:14330/23:00131270

Organizační jednotka

Fakulta informatiky

ISBN

978-1-57735-880-0

ISSN

DOI

http://dx.doi.org/10.1609/aaai.v37i12.26715

Klíčová slova anglicky

decision making; Markov decision processes; controller synthesis; resource constraints; shielding

Štítky

core_A, firank_1

Změněno: 7. 4. 2024 23:07, RNDr. Pavel Šmerk, Ph.D.

Anotace

V originále

We consider partially observable Markov decision processes (POMDPs) modeling an agent that needs a supply of a certain resource (e.g., electricity stored in batteries) to operate correctly. The resource is consumed by the agent's actions and can be replenished only in certain states. The agent aims to minimize the expected cost of reaching some goal while preventing resource exhaustion, a problem we call resource-constrained goal optimization (RSGO). We take a two-step approach to the RSGO problem. First, using formal methods techniques, we design an algorithm computing a shield for a given scenario: a procedure that observes the agent and prevents it from using actions that might eventually lead to resource exhaustion. Second, we augment the POMCP heuristic search algorithm for POMDP planning with our shields to obtain an algorithm solving the RSGO problem. We implement our algorithm and present experiments showing its applicability to benchmarks from the literature.

Návaznosti

GA21-24711S, projekt VaV

Název: Efektivní analýza a optimalizace pravděpodobnostních systémů a her (Akronym: Efektivní analýza a optimalizace pravděpodobnostní)

Investor: Grantová agentura ČR, Efektivní analýza a optimalizace pravděpodobnostních systémů a her

MUNI/A/1433/2022, interní kód MU

Název: Zapojení studentů Fakulty informatiky do mezinárodní vědecké komunity 23

Investor: Masarykova univerzita, Zapojení studentů Fakulty informatiky do mezinárodní vědecké komunity 23

Citovat

AJDARÓW, Michal, Šimon BRLEJ a Petr NOVOTNÝ. Shielding in Resource-Constrained Goal POMDPs. Online. In Brian Williams, Yiling Chen, Jennifer Neville. Proceedings of the 37th AAAI Conference on Artificial Intelligence. Washington, DC, USA: AAAI Press, 2023, s. 14674-14682. ISBN 978-1-57735-880-0. Dostupné z: https://dx.doi.org/10.1609/aaai.v37i12.26715.

@inproceedings{2298218,
   author = {Ajdarów, Michal and Brlej, Šimon and Novotný, Petr},
   address = {Washington, DC, USA},
   booktitle = {Proceedings of the 37th AAAI Conference on Artificial Intelligence},
   doi = {http://dx.doi.org/10.1609/aaai.v37i12.26715},
   editor = {Brian Williams, Yiling Chen, Jennifer Neville},
   keywords = {decision making; Markov decision processes; controller synthesis; resource constraints; shielding},
   howpublished = {elektronická verze "online"},
   language = {eng},
   location = {Washington, DC, USA},
   isbn = {978-1-57735-880-0},
   pages = {14674-14682},
   publisher = {AAAI Press},
   title = {Shielding in Resource-Constrained Goal POMDPs},
   url = {https://ojs.aaai.org/index.php/AAAI/article/view/26715},
   year = {2023}
}

TY  - JOUR
ID  - 2298218
AU  - Ajdarów, Michal - Brlej, Šimon - Novotný, Petr
PY  - 2023
TI  - Shielding in Resource-Constrained Goal POMDPs
PB  - AAAI Press
CY  - Washington, DC, USA
SN  - 9781577358800
KW  - decision making
KW  - Markov decision processes
KW  - controller synthesis
KW  - resource constraints
KW  - shielding
UR  - https://ojs.aaai.org/index.php/AAAI/article/view/26715
N2  - We consider partially observable Markov decision processes (POMDPs) modeling an agent that needs a supply of a certain resource (e.g., electricity stored in batteries) to operate correctly. The resource is consumed by the agent's actions and can be replenished only in certain states. The agent aims to minimize the expected cost of reaching some goal while preventing resource exhaustion, a problem we call resource-constrained goal optimization (RSGO). We take a two-step approach to the RSGO problem. First, using formal methods techniques, we design an algorithm computing a shield for a given scenario: a procedure that observes the agent and prevents it from using actions that might eventually lead to resource exhaustion. Second, we augment the POMCP heuristic search algorithm for POMDP planning with our shields to obtain an algorithm solving the RSGO problem. We implement our algorithm and present experiments showing its applicability to benchmarks from the literature.
ER  -

AJDARÓW, Michal, Šimon BRLEJ a Petr NOVOTNÝ. Shielding in Resource-Constrained Goal POMDPs. Online. In Brian Williams, Yiling Chen, Jennifer Neville. \textit{Proceedings of the 37th AAAI Conference on Artificial Intelligence}. Washington, DC, USA: AAAI Press, 2023, s.~14674-14682. ISBN~978-1-57735-880-0. Dostupné z: https://dx.doi.org/10.1609/aaai.v37i12.26715.

Podrobný výpis o publikaci