Using Kubernetes in Academic Environment : Problems and
Approaches

D 2023

Using Kubernetes in Academic Environment : Problems and Approaches

SPIŠAKOVÁ, Viktória, Dalibor KLUSÁČEK a Lukáš HEJTMÁNEK

Základní údaje

Originální název

Using Kubernetes in Academic Environment : Problems and Approaches

Autoři

SPIŠAKOVÁ, Viktória (703 Slovensko, domácí), Dalibor KLUSÁČEK (203 Česká republika) a Lukáš HEJTMÁNEK (203 Česká republika, domácí)

Vydání

1. vyd. Cham (Switzerland), Job Scheduling Strategies for Parallel Processing, od s. 235-253, 19 s. 2023

Nakladatel

Springer

Další údaje

Jazyk

angličtina

Typ výsledku

Stať ve sborníku

Obor

10200 1.2 Computer and information sciences

Stát vydavatele

Švýcarsko

Utajení

není předmětem státního či obchodního tajemství

Forma vydání

tištěná verze "print"

Odkazy

Finální článek Preprint

Impakt faktor

Impact factor: 0.402 v roce 2005

Kód RIV

RIV/00216224:14610/23:00130019

Organizační jednotka

Ústav výpočetní techniky

ISBN

978-3-031-22697-7

ISSN

DOI

http://dx.doi.org/10.1007/978-3-031-22698-4_12

UT WoS

000972597400013

Klíčová slova anglicky

cloud;HPC;scheduling;Kubernetes;resource management

Štítky

rivok

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 8. 4. 2024 09:20, RNDr. Pavel Šmerk, Ph.D.

Anotace

V originále

In this work, we discuss our experience when utilizing the Kubernetes orchestrator (K8s) to efficiently allocate resources in a heterogeneous and dynamic academic environment. In the commercial world, the "pay per use" model is a strong regulating factor for efficient resource usage. In the academic environment, resources are usually provided "for free" to the end-users, thus they often lack a clear motivation to plan their use efficiently. In this paper, we show three major sources of inefficiencies. One is the users' requirement to have interactive computing environments, where the users need resources for their application as soon as possible. Users do not appreciate waiting for interactive environments, but constantly keeping some resources available for interactive tasks is inefficient. The second phenomenon is observable in both interactive and batch workloads; users tend to overestimate necessary limits for their computations, thus wasting resources. Finally, Kubernetes does not support fair-sharing functionality (dynamic user priorities) which hampers the efforts when developing a fair scheme for Pod/job scheduling and/or eviction. We discuss various approaches to deal with these problems such as scavenger jobs, placeholder jobs, Kubernetes-specific resource allocation policies, separate clusters, priority classes, and novel hybrid cloud approach. We also show that all these proposals open interesting scheduling-related questions that are hard to answer with existing Kubernetes tools and policies. Last but not least, we provide a real workload trace from our installation to the scheduling community which captures these phenomena.

Návaznosti

LM2018140, projekt VaV

Název: e-Infrastruktura CZ (Akronym: e-INFRA CZ)

Investor: Ministerstvo školství, mládeže a tělovýchovy ČR, e-Infrastruktura CZ

Citovat

SPIŠAKOVÁ, Viktória, Dalibor KLUSÁČEK a Lukáš HEJTMÁNEK. Using Kubernetes in Academic Environment : Problems and Approaches. In Dalibor Klusáček, Julita Corbalán, Gonzalo P. Rodrigo. Job Scheduling Strategies for Parallel Processing. 1. vyd. Cham (Switzerland): Springer, 2023, s. 235-253. ISBN 978-3-031-22697-7. Dostupné z: https://dx.doi.org/10.1007/978-3-031-22698-4_12.

@inproceedings{2218837,
   author = {Spišaková, Viktória and Klusáček, Dalibor and Hejtmánek, Lukáš},
   address = {Cham (Switzerland)},
   booktitle = {Job Scheduling Strategies for Parallel Processing},
   doi = {http://dx.doi.org/10.1007/978-3-031-22698-4_12},
   edition = {1},
   editor = {Dalibor Klusáček, Julita Corbalán, Gonzalo P. Rodrigo},
   keywords = {cloud;HPC;scheduling;Kubernetes;resource management},
   howpublished = {tištěná verze "print"},
   language = {eng},
   location = {Cham (Switzerland)},
   isbn = {978-3-031-22697-7},
   pages = {235-253},
   publisher = {Springer},
   title = {Using Kubernetes in Academic Environment : Problems and Approaches},
   url = {https://link.springer.com/chapter/10.1007/978-3-031-22698-4_12},
   year = {2023}
}

TY  - JOUR
ID  - 2218837
AU  - Spišaková, Viktória - Klusáček, Dalibor - Hejtmánek, Lukáš
PY  - 2023
TI  - Using Kubernetes in Academic Environment : Problems and Approaches
PB  - Springer
CY  - Cham (Switzerland)
SN  - 9783031226977
KW  - cloud;HPC;scheduling;Kubernetes;resource management
UR  - https://link.springer.com/chapter/10.1007/978-3-031-22698-4_12
N2  - In this work, we discuss our experience when utilizing the Kubernetes orchestrator (K8s) to efficiently allocate resources in a heterogeneous and dynamic academic environment. In the commercial world, the "pay per use" model is a strong regulating factor for efficient resource usage. In the academic environment, resources are usually provided "for free" to the end-users, thus they often lack a clear motivation to plan their use efficiently. In this paper, we show three major sources of inefficiencies. One is the users' requirement to have interactive computing environments, where the users need resources for their application as soon as possible. Users do not appreciate waiting for interactive environments, but constantly keeping some resources available for interactive tasks is inefficient. The second phenomenon is observable in both interactive and batch workloads; users tend to overestimate necessary limits for their computations, thus wasting resources. Finally, Kubernetes does not support fair-sharing functionality (dynamic user priorities) which hampers the efforts when developing a fair scheme for Pod/job scheduling and/or eviction. We discuss various approaches to deal with these problems such as scavenger jobs, placeholder jobs, Kubernetes-specific resource allocation policies, separate clusters, priority classes, and novel hybrid cloud approach. We also show that all these proposals open interesting scheduling-related questions that are hard to answer with existing Kubernetes tools and policies. Last but not least, we provide a real workload trace from our installation to the scheduling community which captures these phenomena.
ER  -

SPIŠAKOVÁ, Viktória, Dalibor KLUSÁČEK a Lukáš HEJTMÁNEK. Using Kubernetes in Academic Environment : Problems and Approaches. In Dalibor Klusáček, Julita Corbalán, Gonzalo P. Rodrigo. \textit{Job Scheduling Strategies for Parallel Processing}. 1. vyd. Cham (Switzerland): Springer, 2023, s.~235-253. ISBN~978-3-031-22697-7. Dostupné z: https://dx.doi.org/10.1007/978-3-031-22698-4\_{}12.

Podrobný výpis o publikaci