Comparison of Independent Analyses of Identical Image Sets
Reveals Significant Analyst-to-Analyst Variability

J 2025

Comparison of Independent Analyses of Identical Image Sets Reveals Significant Analyst-to-Analyst Variability

PENGO, Thomas; Kristopher E KUBOW; Angel MANCEBO; David SVOBODA; Richard W COLE et al.

Základní údaje

Originální název

Comparison of Independent Analyses of Identical Image Sets Reveals Significant Analyst-to-Analyst Variability

Autoři

PENGO, Thomas; Kristopher E KUBOW; Angel MANCEBO; David SVOBODA a Richard W COLE

Vydání

Journal of Biomolecular Techniques, Association of Biomolecular Resource Facilities, 2025, 1524-0215

Další údaje

Jazyk

angličtina

Typ výsledku

Článek v odborném periodiku

Obor

10600 1.6 Biological sciences

Stát vydavatele

Spojené státy

Utajení

není předmětem státního či obchodního tajemství

Odkazy

URL

Označené pro přenos do RIV

Ano

Organizační jednotka

Fakulta informatiky

DOI

https://doi.org/10.7171/3fc1f5fe.60f1999c

Klíčová slova anglicky

reproducibility; reliability; variability; image analysis; image processing; bioimage analysis; image analysis challenge; inter-analyst variation

Štítky

cbia-web

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 7. 10. 2025 14:59, doc. RNDr. David Svoboda, Ph.D.

Anotace

V originále

The Light Microscopy Research Group, a research group with the Association of Biomolecular Resource Facilities, organized a global study where participants were given artificially generated images of various specimens with various signal-to-noise and object proximity levels. Users were tasked with segmenting the images and providing measured metrics as part of the study. Rather than ranking algorithms, our goal was to study the sources of variability of the results across participants given the same task. This study highlights that substantial variability can exist between independent analyses of identical datasets, even when the analysis problem is relatively straightforward and the analysts are experienced. These findings further support the need for data and analysis methods that follow the findable, accessible, interoperable, reusable (FAIR) principles.

Návaznosti

EF18_046/0016045, projekt VaV

Název: Modernizace národní infrastruktury pro biologické a medicínské zobrazování Czech-BioImaging

LM2023050, projekt VaV

Název: Národní infrastruktura pro biologické a medicínské zobrazování

Investor: Ministerstvo školství, mládeže a tělovýchovy ČR, Czech BioImaging: Národní výzkumná infrastruktura pro biologické a medicínské zobrazování

Citovat

PENGO, Thomas; Kristopher E KUBOW; Angel MANCEBO; David SVOBODA a Richard W COLE. Comparison of Independent Analyses of Identical Image Sets Reveals Significant Analyst-to-Analyst Variability. Journal of Biomolecular Techniques. Association of Biomolecular Resource Facilities, 2025, roč. 36, č. 2, s. 1-30. ISSN 1524-0215. Dostupné z: https://doi.org/10.7171/3fc1f5fe.60f1999c.

@article{2524219,
   author = {Pengo, Thomas and Kubow, Kristopher E and Mancebo, Angel and Svoboda, David and Cole, Richard W},
   article_number = {2},
   doi = {https://doi.org/10.7171/3fc1f5fe.60f1999c},
   keywords = {reproducibility; reliability; variability; image analysis; image processing; bioimage analysis; image analysis challenge; inter-analyst variation},
   language = {eng},
   issn = {1524-0215},
   journal = {Journal of Biomolecular Techniques},
   title = {Comparison of Independent Analyses of Identical Image Sets Reveals Significant Analyst-to-Analyst Variability},
   url = {https://jbt.pubpub.org/pub/4de36i6y},
   volume = {36},
   year = {2025}
}

TY  - JOUR
ID  - 2524219
AU  - Pengo, Thomas - Kubow, Kristopher E - Mancebo, Angel - Svoboda, David - Cole, Richard W
PY  - 2025
TI  - Comparison of Independent Analyses of Identical Image Sets Reveals Significant Analyst-to-Analyst Variability
JF  - Journal of Biomolecular Techniques
VL  - 36
IS  - 2
SP  - 1-30
EP  - 1-30
PB  - Association of Biomolecular Resource Facilities
SN  - 15240215
KW  - reproducibility
KW  - reliability
KW  - variability
KW  - image analysis
KW  - image processing
KW  - bioimage analysis
KW  - image analysis challenge
KW  - inter-analyst variation
UR  - https://jbt.pubpub.org/pub/4de36i6y
N2  - The Light Microscopy Research Group, a research group with the Association of Biomolecular Resource Facilities, organized a global study where participants were given artificially generated images of various specimens with various signal-to-noise and object proximity levels. Users were tasked with segmenting the images and providing measured metrics as part of the study. Rather than ranking algorithms, our goal was to study the sources of variability of the results across participants given the same task. This study highlights that substantial variability can exist between independent analyses of identical datasets, even when the analysis problem is relatively straightforward and the analysts are experienced. These findings further support the need for data and analysis methods that follow the findable, accessible, interoperable, reusable (FAIR) principles.
ER  -

PENGO, Thomas; Kristopher E KUBOW; Angel MANCEBO; David SVOBODA a Richard W COLE. Comparison of Independent Analyses of Identical Image Sets Reveals Significant Analyst-to-Analyst Variability. \textit{Journal of Biomolecular Techniques}. Association of Biomolecular Resource Facilities, 2025, roč.~36, č.~2, s.~1-30. ISSN~1524-0215. Dostupné z: https://doi.org/10.7171/3fc1f5fe.60f1999c.

Přehled o publikaci