Závěrečná práce: Bc. Martin Kurečka: Pareto Front Estimation in Risk-Constrained Markov Decision Processes
Diplomová práce
Pareto Front Estimation in Risk-Constrained Markov Decision Processes
Anotace
Předmětem práce je optimalizace Markovovských rozhodovacích procesů s omezením (CMDP) pomocí metody MCTS. Cílem CMDP je nalézt strategii maximalizující odměnu při současném udržení penalizace pod předem danou hranicí. Náš přínos je dvojí. Za prvé podrobně zkoumáme stávající algoritmy založené na MCTS (jmenovitě CC-POMCP a RAMCP) a demonstrujeme koncepční problémy v jejich návrhu. Za druhé představujeme …více
Abstract
The topic of the thesis is the optimisation of constrained Markov decision processes (CMDPs) using Monte Carlo tree search. CMDPs aim to find a control policy maximising a reward signal while keeping a penalty signal below a specified threshold. Our contribution is two-fold. Firstly, we provide a detailed examination of existing MCTS-based algorithms, CC-POMCP and RAMCP, and reveal and demonstrate …více
Zadání práce
The expected contents of the thesis are the following:
- Extension of existing approaches to risk-aware MDPs with the capability to estimate risk-utility tradeoffs and to use these estimate to increase the quality of the algorithms' decision making.
- Implementation of the new algorithm in a stand-alone working tool, including an interface to work with user-defined MDP models.
- Thorough experimental evaluation of the new approach. This will include a comparison with the previous RAMCP/RAlph approached and with the Lagrangian-based method by Lee at al.
- The text part of the thesis will contain the thorough explanation of the new approach as well as the necessary explanation of the previous approaches on which the new ones build. Necessary technical preliminaries will be included. The thesis will discuss relevant related work from the area of risk-constrained planning and learning as well as a presentation and discussion of the experimental results.
18. 12. 2023 10:37, doc. RNDr. Petr Novotný, Ph.D., učo 172743
Práce na příbuzné téma
Seznam prací, které mají shodná klíčová slova.
-
Ablation Study for Risk-Aware Planners
Bc. Matěj Dvořák -
Experimental Evaluation of Risk-Averse Planners
Bc. Martin Bendel -
Pareto Frontier Estimation in Offline Safe Reinforcement Learning
Mgr. Václav Nevyhoštěný -
Sampling Methods for Risk-Averse MDP Solvers
Mgr. Václav Nevyhoštěný -
Risk-Aware Planners for Partially Observable Environments
Bc. Ondřej Zugárek -
Black-Box Hyperparameter Tuning for Risk-Constrained Reinforcement Learning Algorithm
Mgr. Ján Petrák -
Reinforcement Learning for Efficient Attack Agents Training
Ing. Glenn Fischer -
Revisiting Uncertainty Quantification for Offline Reinforcement Learning
Mgr. Vít Unčovský




