Nextflow in Bioinformatics: Executors Performance Comparison
Using Genomics Data

J 2023

Nextflow in Bioinformatics: Executors Performance Comparison Using Genomics Data

SPIŠAKOVÁ, Viktória, Lukáš HEJTMÁNEK a Jakub HYNŠT

Základní údaje

Originální název

Nextflow in Bioinformatics: Executors Performance Comparison Using Genomics Data

Autoři

SPIŠAKOVÁ, Viktória (703 Slovensko, domácí), Lukáš HEJTMÁNEK (203 Česká republika, domácí) a Jakub HYNŠT (203 Česká republika, domácí)

Vydání

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, NETHERLANDS, ELSEVIER, 2023, 0167-739X

Další údaje

Jazyk

angličtina

Typ výsledku

Článek v odborném periodiku

Obor

10201 Computer sciences, information science, bioinformatics

Stát vydavatele

Nizozemské království

Utajení

není předmětem státního či obchodního tajemství

Odkazy

URL

Impakt faktor

Impact factor: 7.500 v roce 2022

Kód RIV

RIV/00216224:14610/23:00130181

Organizační jednotka

Ústav výpočetní techniky

DOI

http://dx.doi.org/10.1016/j.future.2023.01.009

UT WoS

000926828200001

Klíčová slova anglicky

Kubernetes;HPC;Cloud;Performance comparison;Genomics;Nextflow;Big data

Štítky

rivok

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 20. 3. 2024 15:39, Mgr. Alena Mokrá

Anotace

V originále

Processing big data is a computationally demanding task which has usually been fulfilled by HPC batch systems. These complex systems pose a challenge to scientists due to their cumbersome nature and changing environment. The scientists often lack deeper informatics understanding and experiment reproducibility is increasingly becoming a hard request on the research validity. A new computational paradigm — containers — are meant to contain all dependencies and persist the state which help reproducibility. They have gained a lot of popularity in the informatics community but HPC community remains skeptical and doubts that container platforms are appropriate for demanding tasks or that such infrastructure can reach significant performance. In this paper, we observe the performance of various infrastructure types (HPC, Kubernetes, local) on a Sarek Nextflow bioinformatics workflow with real life genomics data of various sizes. We analyze obtained workload trace and discuss pros and cons of utilized infrastructures. We also show some approaches perform better in terms of available resources but others are more suitable for diversified workflows. Based on the results, we provide recommendations for life science groups which plan to analyze data in large scale.

Návaznosti

EF16_026/0008448, projekt VaV

Název: Analýza českých genomů pro teranostiku

LM2018140, projekt VaV

Název: e-Infrastruktura CZ (Akronym: e-INFRA CZ)

Investor: Ministerstvo školství, mládeže a tělovýchovy ČR, e-Infrastruktura CZ

Citovat

SPIŠAKOVÁ, Viktória, Lukáš HEJTMÁNEK a Jakub HYNŠT. Nextflow in Bioinformatics: Executors Performance Comparison Using Genomics Data. FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE. NETHERLANDS: ELSEVIER, 2023, roč. 142, May 2023, s. 328-339. ISSN 0167-739X. Dostupné z: https://dx.doi.org/10.1016/j.future.2023.01.009.

@article{2246279,
   author = {Spišaková, Viktória and Hejtmánek, Lukáš and Hynšt, Jakub},
   article_location = {NETHERLANDS},
   article_number = {May 2023},
   doi = {http://dx.doi.org/10.1016/j.future.2023.01.009},
   keywords = {Kubernetes;HPC;Cloud;Performance comparison;Genomics;Nextflow;Big data},
   language = {eng},
   issn = {0167-739X},
   journal = {FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE},
   title = {Nextflow in Bioinformatics: Executors Performance Comparison Using Genomics Data},
   url = {https://www.sciencedirect.com/journal/future-generation-computer-systems/special-issue/10KV6BLSVBL},
   volume = {142},
   year = {2023}
}

TY  - JOUR
ID  - 2246279
AU  - Spišaková, Viktória - Hejtmánek, Lukáš - Hynšt, Jakub
PY  - 2023
TI  - Nextflow in Bioinformatics: Executors Performance Comparison Using Genomics Data
JF  - FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE
VL  - 142
IS  - May 2023
SP  - 328-339
EP  - 328-339
PB  - ELSEVIER
SN  - 0167739X
KW  - Kubernetes;HPC;Cloud;Performance comparison;Genomics;Nextflow;Big data
UR  - https://www.sciencedirect.com/journal/future-generation-computer-systems/special-issue/10KV6BLSVBL
N2  - Processing big data is a computationally demanding task which has usually been fulfilled by HPC batch systems. These complex systems pose a challenge to scientists due to their cumbersome nature and changing environment. The scientists often lack deeper informatics understanding and experiment reproducibility is increasingly becoming a hard request on the research validity. A new computational paradigm — containers — are meant to contain all dependencies and persist the state which help reproducibility. They have gained a lot of popularity in the informatics community but HPC community remains skeptical and doubts that container platforms are appropriate for demanding tasks or that such infrastructure can reach significant performance. In this paper, we observe the performance of various infrastructure types (HPC, Kubernetes, local) on a Sarek Nextflow bioinformatics workflow with real life genomics data of various sizes. We analyze obtained workload trace and discuss pros and cons of utilized infrastructures. We also show some approaches perform better in terms of available resources but others are more suitable for diversified workflows. Based on the results, we provide recommendations for life science groups which plan to analyze data in large scale.
ER  -

SPIŠAKOVÁ, Viktória, Lukáš HEJTMÁNEK a Jakub HYNŠT. Nextflow in Bioinformatics: Executors Performance Comparison Using Genomics Data. \textit{FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE}. NETHERLANDS: ELSEVIER, 2023, roč.~142, May 2023, s.~328-339. ISSN~0167-739X. Dostupné z: https://dx.doi.org/10.1016/j.future.2023.01.009.

Podrobný výpis o publikaci