Context:
Patient records are a goldmine for various data analytics and machine learning applications aiming at gaining insights that could lead to more efficient treatments, inform innovative clinical decision support systems and enable patient empowerment. However, a lot of information in patient records is "locked" in the form of unstructured text that is not readily amenable to machine processing. The latest advances in the language models technology can be used for tackling this challenge. However, few such models exist for relatively under-resourced languages like Czech, and there are no such models at all trained specifically for the medical domain in the Czech context, which is a gap this work will address.

Goals:
- Study the state of the art in general-purpose language models trained on English texts.
- Review language models specifically trained on biomedical text. Focus on both self-supervised learning stage and consequent fine-tuning techniques.
- Implement a bespoke language model trained on Czech (bio)medical content (e.g. patient records) in a self-supervised manner.
- Work with clinical expert(s) on defining a specific unmet clinical need that could be used for task-specific fine-tuning of the self-supervised model (e.g. extraction of named entities such as symptoms, procedures or medications from unstructured health records).
- Explore approaches that could support the results of the developed model with explanations (getting inspired for instance by LIME or gradient-based attribution methods).
- Validate the result with the assistance of the clinical expert.
- Write up the results in a thesis form.

Requirements:
- Keen interest in the topic.
- Knowledge of and previous hands-on experience with natural language processing.
- While the thesis can be written and defended in Czech, its elaboration and presentation in English will be supported enthusiastically (the results may be disseminated to and used by partners in ongoing or future EU projects).
- Monthly (or more frequent, if needed) progress review meetings with the supervisor will be expected.
- The student(s) will also be expected to develop and document any related code using the FI MU Gitlab and ICS SensitiveCloud platforms, and (if applicable) re-use and interact with other related projects there.