Závěrečná práce: Bc. David Rusnák: DevOps and MLOps Pipelines in a Secure Kubernetes Cluster
Diplomová práce
DevOps and MLOps Pipelines in a Secure Kubernetes Cluster
Anotace
Práca sa zameriava na bezpečné a škálovateľné spracovanie lekárskych údajov, čím prispieva k poslaniu projektu AICope zameraného na správu zdravotníckych údajov orientovanú na pacienta. Systém využívajúci Kubernetes a Apache Airflow je nasadený v prostredí Sensitive Cloud. Tento systém predstavuje zásadný krok smerom k futuristickému stavu, ktorý zabezpečuje efektívny prístup k informáciám, prezentuje bezprecedentné poznatky a zachováva dôveru v bezpečné spracovanie údajov.
Abstract
The thesis focuses on the secure and scalable processing of medical data, contributing to the AICope project's mission for patient-centric healthcare data management. A system utilizing Kubernetes and Apache Airflow is deployed in the Sensitive Cloud environment. This system represents a crucial step towards an futuristic state, ensuring efficient information access, presenting unprecedented insights, and maintaining confidence in secure data handling.
Zadání práce
Patient records are a goldmine for various data analytics and machine learning applications aiming at gaining insights that could lead to more efficient treatments, inform innovative clinical decision support systems and enable patient empowerment. Yet these data are very sensitive and thus they cannot be processed as is. Pseudonymisation (replacing direct patient identifiers with anonymous IDs) can help, but such data still cannot be processed on standard computational infrastructures that are potentially open to attacks, while the secure infrastructure of the data providers (i.e., hospitals) typically does not scale to allow for deployment of modern analytical (e.g., machine learning) pipelines. This problem is further exacerbated in case of multi-site studies, involving multiple clinical data providers where federated analytics may be the only way to gain insights from the separate data silos. Infrastructures with tightly controlled access and clearly defined data transfer and processing protocols with corresponding security guarantees that still support the state of the art analytics, either on-site or federated, are therefore required.
Kubernetes (also K8s for short), a system for automated deployment of containerised applications can enable such infrastructures. And this is the problem the thesis will look into - how to use K8s in a secure HPC environment to deploy advanced, possibly federated analytical pipelines on sensitive patient data, following the state-of-the-art approaches in DevOps and MLOps.
Goals:
- Augment a prototype architecture for deploying machine learning models (including federated ones) and various auxiliary services using a K8s infrastructure in the SensitiveCloud environment maintained by the Institute of Computer Science at MU.
- This will involve contributing to new data and workflow pipelines using for instance the Apache Airflow framework, and implementing a proof-of-concept logging module for reproducibility of the analytical pipelines executed in the environment.
- Validate the work by its deployment as a part of the AIcope research project infrastructure.
- Write up the results in a thesis form.
Requirements:
- Keen interest in the topic.
- At least a minimal knowledge and previous hands-on experience with containerisation, DevOps and MLOps.
- While the thesis can be written and defended in Czech, its elaboration and presentation in English will be supported enthusiastically (the results may be disseminated to and used by partners in EU projects).
- Monthly (or more frequent, if needed) thesis progress review meetings with the supervisor will be expected. Other than that, there will be meetings with broader teams of researchers and developers involved in research projects associated with this topic.
- The student(s) will also be expected to develop and document any related code using the FI MU Gitlab platform, and (if applicable) re-use and interact with other related projects there.
18. 12. 2023 13:45, doc. Mgr. Bc. Vít Nováček, PhD, učo 4049
Konzultant
Práce na příbuzné téma
Seznam prací, které mají shodná klíčová slova.
-
Automated Infrastructure Deployment on Public and Private Cloud as a Part of Continuous Integration Process
Mgr. Šimon Obetko -
Logging and Monitoring for Kubernetes Infrastructure
Mgr. Tomáš Zvoník -
Návrh digitální infrastruktury pro společnost Lakmoos
Bc. Adam Žitník, učo 536367 -
Deployment of Automated Patrolling Strategy Calculation
Ing. David Svatý -
Implementation of a Deployment Method to Kubernetes for Electronic Laboratory Notebook Software
Bc. Drahomír Malota -
Orchestration tool for the modular computing subsystem of the ANALYZA platform
Mgr. Richard Kyčerka -
Kontajnerizácia pre vysokú dostupnosť v PostgreSQL
Mgr. Matúš Špik -
Microservices for Situational Awareness and Security Operations
Ing. Daniel Timko




