Assessing Artificial Intelligence and Radiologist Performance
in Musculoskeletal Fracture Detection: Multi-Reader, Multi-Case
Study

D 2025

Assessing Artificial Intelligence and Radiologist Performance in Musculoskeletal Fracture Detection: Multi-Reader, Multi-Case Study

DANDÁR, Jakub; Simon KLÍČNÍK; Zdeněk STRAKA a Daniel KVAK

Základní údaje

Originální název

Assessing Artificial Intelligence and Radiologist Performance in Musculoskeletal Fracture Detection: Multi-Reader, Multi-Case Study

Autoři

DANDÁR, Jakub; Simon KLÍČNÍK; Zdeněk STRAKA a Daniel KVAK

Vydání

1. vyd. Barcelona, Spain, Proceedings of the 1st International Conference on AI in Medicine and Healthcare (AiMH' 2025), od s. 6-13, 8 s. 2025

Nakladatel

IFSA Publishing

Další údaje

Jazyk

angličtina

Typ výsledku

Stať ve sborníku

Stát vydavatele

Španělsko

Utajení

není předmětem státního či obchodního tajemství

Forma vydání

elektronická verze "online"

Označené pro přenos do RIV

ISBN

978-84-09-71190-1

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 13. 7. 2026 14:33, Mgr. Daniel Kvak

Anotace

V originále

Fracture detection on musculoskeletal (MSK) radiographs is critical for both emergency and routine care, yet diagnostic errors remain common due to high workloads and limited radiological expertise. This study evaluates the diagnostic performance of an artificial intelligence (AI) system (Carebot AI Bones 1.8.10; Carebot s.r.o.) in detecting fractures on MSK X-rays, comparing its performance to six radiologists of varying experience levels in a blinded multi-reader, multi-case (MRMC) study. A total of 489 radiographs were retrospectively analyzed from routine clinical practice, with ground truth established for 448 images through consensus among three experienced radiologists. Diagnostic performance was assessed using sensitivity (Se), specificity (Sp), positive (PLR), and negative likelihood ratio (NLR), with statistical analysis including McNemar’s test and Holm’s method for multiple comparisons. The AI system achieved a Se of 0.921 (95% CI: 0.846–0.961) and Sp of 0.897 (0.861–0.924). Radiologists’ Se ranged from 0.663 to 0.933, and Sp ranged from 0.916 to 0.989. The AI demonstrated consistent high sensitivity across body parts, particularly for elbow and hand/wrist fractures, often exceeding radiologists’ performance. Specificity was slightly lower but acceptable, supporting AI’s potential as a complementary diagnostic tool. These findings highlight the clinical utility of AI in MSK fracture detection, particularly in settings with limited resources or high diagnostic workloads. Future research should validate these results in larger, multicentric studies to ensure broader generalizability and evaluate AI integration in real-world workflows.

Citovat

DANDÁR, Jakub; Simon KLÍČNÍK; Zdeněk STRAKA a Daniel KVAK. Assessing Artificial Intelligence and Radiologist Performance in Musculoskeletal Fracture Detection: Multi-Reader, Multi-Case Study. Online. In Prof., Dr. Sergey Y. Yurish. Proceedings of the 1st International Conference on AI in Medicine and Healthcare (AiMH' 2025). 1. vyd. Barcelona, Spain: IFSA Publishing, 2025, s. 6-13. ISBN 978-84-09-71190-1.

@inproceedings{2582692,
   author = {Dandár, Jakub and Klíčník, Simon and Straka, Zdeněk and Kvak, Daniel},
   address = {Barcelona, Spain},
   booktitle = {Proceedings of the 1st International Conference on AI in Medicine and Healthcare (AiMH' 2025)},
   edition = {1},
   editor = {Prof., Dr. Sergey Y. Yurish},
   howpublished = {elektronická verze "online"},
   language = {eng},
   location = {Barcelona, Spain},
   isbn = {978-84-09-71190-1},
   pages = {6-13},
   publisher = {IFSA Publishing},
   title = {Assessing Artificial Intelligence and Radiologist Performance in Musculoskeletal Fracture Detection: Multi-Reader, Multi-Case Study},
   year = {2025}
}

TY  - CONF
ID  - 2582692
AU  - Dandár, Jakub - Klíčník, Simon - Straka, Zdeněk - Kvak, Daniel
PY  - 2025
TI  - Assessing Artificial Intelligence and Radiologist Performance in Musculoskeletal Fracture Detection: Multi-Reader, Multi-Case Study
PB  - IFSA Publishing
CY  - Barcelona, Spain
SN  - 9788409711901
N2  - Fracture detection on musculoskeletal (MSK) radiographs is critical for both emergency and routine care, yet diagnostic errors remain common due to high workloads and limited radiological expertise. This study evaluates the diagnostic performance of an artificial intelligence (AI) system (Carebot AI Bones 1.8.10; Carebot s.r.o.) in detecting fractures on MSK X-rays, comparing its performance to six radiologists of varying experience levels in a blinded multi-reader, multi-case (MRMC) study. A total of 489 radiographs were retrospectively analyzed from routine clinical practice, with ground truth established for 448 images through consensus among three experienced radiologists. Diagnostic performance was assessed using sensitivity (Se), specificity (Sp), positive (PLR), and negative likelihood ratio (NLR), with statistical analysis including McNemar’s test and Holm’s method for multiple comparisons. The AI system achieved a Se of 0.921 (95% CI: 0.846–0.961) and Sp of 0.897 (0.861–0.924). Radiologists’ Se ranged from 0.663 to 0.933, and Sp ranged from 0.916 to 0.989. The AI demonstrated consistent high sensitivity across body parts, particularly for elbow and hand/wrist fractures, often exceeding radiologists’ performance. Specificity was slightly lower but acceptable, supporting AI’s potential as a complementary diagnostic tool. These findings highlight the clinical utility of AI in MSK fracture detection, particularly in settings with limited resources or high diagnostic workloads. Future research should validate these results in larger, multicentric studies to ensure broader generalizability and evaluate AI integration in real-world workflows.
ER  -

DANDÁR, Jakub; Simon KLÍČNÍK; Zdeněk STRAKA a Daniel KVAK. Assessing Artificial Intelligence and Radiologist Performance in Musculoskeletal Fracture Detection: Multi-Reader, Multi-Case Study. Online. In Prof., Dr. Sergey Y. Yurish. \textit{Proceedings of the 1st International Conference on AI in Medicine and Healthcare (AiMH' 2025)}. 1. vyd. Barcelona, Spain: IFSA Publishing, 2025, s.~6-13. ISBN~978-84-09-71190-1.

Přehled o publikaci