The goal of the thesis is to investigate the design choices made in select algorithms for offline reinforcement learning (RL) utilizing the technique of uncertainty quantification. The analysis will focus on how these design choices affect the stability of training and the performance of the trained policies. The experimental evaluation will be performed over standard offline RL benchmarks as well as toy tasks tailor-made to showcase strengths and shortcomings of the particular methods. The student will also try to extend the algorithms with some recently proposed heuristics. The text part of the thesis will contain a description of the analyzed methods and a thorough presentation and discussion of the experimental results.