Automation bias in medicine: The influence of automated
diagnoses on interpreter accuracy and uncertainty when reading
electrocardiograms

J 2018

Automation bias in medicine: The influence of automated diagnoses on interpreter accuracy and uncertainty when reading electrocardiograms

BOND, Raymond R., Tomáš NOVOTNÝ, Irena ANDRŠOVÁ, Lumír KOC, Martina ŠIŠÁKOVÁ et. al.

Základní údaje

Originální název

Automation bias in medicine: The influence of automated diagnoses on interpreter accuracy and uncertainty when reading electrocardiograms

Autoři

BOND, Raymond R. (826 Velká Británie a Severní Irsko, garant), Tomáš NOVOTNÝ (203 Česká republika, domácí), Irena ANDRŠOVÁ (203 Česká republika, domácí), Lumír KOC (203 Česká republika, domácí), Martina ŠIŠÁKOVÁ (203 Česká republika, domácí), Dewar FINLAY (826 Velká Británie a Severní Irsko), Daniel GULDENRING (826 Velká Británie a Severní Irsko), James MCLAUGHLIN (826 Velká Británie a Severní Irsko), Aaron PEACE (826 Velká Británie a Severní Irsko), Victoria MCGILLIGAN (826 Velká Británie a Severní Irsko), Stephen J. LESLIE (826 Velká Británie a Severní Irsko), Hui WANG (826 Velká Británie a Severní Irsko) a Marek MALIK (826 Velká Británie a Severní Irsko)

Vydání

JOURNAL OF ELECTROCARDIOLOGY, PHILADELPHIA, CHURCHILL LIVINGSTONE INC MEDICAL PUBLISHERS, 2018, 0022-0736

Další údaje

Jazyk

angličtina

Typ výsledku

Článek v odborném periodiku

Obor

30201 Cardiac and Cardiovascular systems

Stát vydavatele

Spojené státy

Utajení

není předmětem státního či obchodního tajemství

Impakt faktor

Impact factor: 1.166

Kód RIV

RIV/00216224:14110/18:00105777

Organizační jednotka

Lékařská fakulta

DOI

http://dx.doi.org/10.1016/j.jelectrocard.2018.08.007

UT WoS

000454674000002

Klíčová slova anglicky

Electrocardiogram

Štítky

14110211, rivok

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 9. 2. 2019 21:01, Soňa Böhmová

Anotace

V originále

Introduction: Interpretation of the 12-lead Electrocardiogram (ECG) is normally assisted with an automated diagnosis (AD), which can facilitate an 'automation bias' where interpreters can be anchored. In this paper, we studied, 1) the effect of an incorrect AD on interpretation accuracy and interpreter confidence (a proxy for uncertainty), and 2) whether confidence and other interpreter features can predict interpretation accuracy using machine learning. Methods: This study analysed 9000 ECG interpretations from cardiology and non-cardiology fellows (CFs and non-CFs). One third of the ECGs involved no ADs, one third with ADs (half as incorrect) and one third had multiple ADs. Interpretations were scored and interpreter confidence was recorded for each interpretation and subsequently standardised using sigma scaling. Spearman coefficients were used for correlation analysis and C5.0 decision trees were used for predicting interpretation accuracy using basic interpreter features such as confidence, age, experience and designation. Results: Interpretation accuracies achieved by CFs and non-CFs dropped by 43.20% and 58.95% respectively when an incorrect AD was presented (p < 0.001). Overall correlation between scaled confidence and interpretation accuracy was higher amongst CFs. However, correlation between confidence and interpretation accuracy decreased for both groups when an incorrect AD was presented. We found that an incorrect AD disturbs the reliability of interpreter confidence in predicting accuracy. An incorrect AD has a greater effect on the confidence of non-CFs (although this is not statistically significant it is close to the threshold, p = 0.065). The best C5.0 decision tree achieved an accuracy rate of 64.67% (p < 0.001), however this is only 6.56% greater than the no information-rate. Conclusion: Incorrect ADs reduce the interpreter's diagnostic accuracy indicating an automation bias. Non-CFs tend to agree more with the ADs in comparison to CFs, hence less expert physicians are more effected by automation bias. Incorrect ADs reduce the interpreter's confidence and also reduces the predictive power of confidence for predicting accuracy (even more so for non-CFs). Whilst a statistically significant model was developed, it is difficult to predict interpretation accuracy using machine learning on basic features such as interpreter confidence, age, reader experience and designation. (C) 2018 Elsevier Inc. All rights reserved.

Citovat

BOND, Raymond R., Tomáš NOVOTNÝ, Irena ANDRŠOVÁ, Lumír KOC, Martina ŠIŠÁKOVÁ, Dewar FINLAY, Daniel GULDENRING, James MCLAUGHLIN, Aaron PEACE, Victoria MCGILLIGAN, Stephen J. LESLIE, Hui WANG a Marek MALIK. Automation bias in medicine: The influence of automated diagnoses on interpreter accuracy and uncertainty when reading electrocardiograms. JOURNAL OF ELECTROCARDIOLOGY. PHILADELPHIA: CHURCHILL LIVINGSTONE INC MEDICAL PUBLISHERS, 2018, roč. 51, č. 6, s. "S6"-"S11", 6 s. ISSN 0022-0736. Dostupné z: https://dx.doi.org/10.1016/j.jelectrocard.2018.08.007.

@article{1489081,
   author = {Bond, Raymond R. and Novotný, Tomáš and Andršová, Irena and Koc, Lumír and Šišáková, Martina and Finlay, Dewar and Guldenring, Daniel and McLaughlin, James and Peace, Aaron and McGilligan, Victoria and Leslie, Stephen J. and Wang, Hui and Malik, Marek},
   article_location = {PHILADELPHIA},
   article_number = {6},
   doi = {http://dx.doi.org/10.1016/j.jelectrocard.2018.08.007},
   keywords = {Electrocardiogram},
   language = {eng},
   issn = {0022-0736},
   journal = {JOURNAL OF ELECTROCARDIOLOGY},
   title = {Automation bias in medicine: The influence of automated diagnoses on interpreter accuracy and uncertainty when reading electrocardiograms},
   volume = {51},
   year = {2018}
}

TY  - JOUR
ID  - 1489081
AU  - Bond, Raymond R. - Novotný, Tomáš - Andršová, Irena - Koc, Lumír - Šišáková, Martina - Finlay, Dewar - Guldenring, Daniel - McLaughlin, James - Peace, Aaron - McGilligan, Victoria - Leslie, Stephen J. - Wang, Hui - Malik, Marek
PY  - 2018
TI  - Automation bias in medicine: The influence of automated diagnoses on interpreter accuracy and uncertainty when reading electrocardiograms
JF  - JOURNAL OF ELECTROCARDIOLOGY
VL  - 51
IS  - 6
SP  - "S6"-"S11"
EP  - "S6"-"S11"
PB  - CHURCHILL LIVINGSTONE INC MEDICAL PUBLISHERS
SN  - 00220736
KW  - Electrocardiogram
N2  - Introduction: Interpretation of the 12-lead Electrocardiogram (ECG) is normally assisted with an automated diagnosis (AD), which can facilitate an 'automation bias' where interpreters can be anchored. In this paper, we studied, 1) the effect of an incorrect AD on interpretation accuracy and interpreter confidence (a proxy for uncertainty), and 2) whether confidence and other interpreter features can predict interpretation accuracy using machine learning. Methods: This study analysed 9000 ECG interpretations from cardiology and non-cardiology fellows (CFs and non-CFs). One third of the ECGs involved no ADs, one third with ADs (half as incorrect) and one third had multiple ADs. Interpretations were scored and interpreter confidence was recorded for each interpretation and subsequently standardised using sigma scaling. Spearman coefficients were used for correlation analysis and C5.0 decision trees were used for predicting interpretation accuracy using basic interpreter features such as confidence, age, experience and designation. Results: Interpretation accuracies achieved by CFs and non-CFs dropped by 43.20% and 58.95% respectively when an incorrect AD was presented (p < 0.001). Overall correlation between scaled confidence and interpretation accuracy was higher amongst CFs. However, correlation between confidence and interpretation accuracy decreased for both groups when an incorrect AD was presented. We found that an incorrect AD disturbs the reliability of interpreter confidence in predicting accuracy. An incorrect AD has a greater effect on the confidence of non-CFs (although this is not statistically significant it is close to the threshold, p = 0.065). The best C5.0 decision tree achieved an accuracy rate of 64.67% (p < 0.001), however this is only 6.56% greater than the no information-rate. Conclusion: Incorrect ADs reduce the interpreter's diagnostic accuracy indicating an automation bias. Non-CFs tend to agree more with the ADs in comparison to CFs, hence less expert physicians are more effected by automation bias. Incorrect ADs reduce the interpreter's confidence and also reduces the predictive power of confidence for predicting accuracy (even more so for non-CFs). Whilst a statistically significant model was developed, it is difficult to predict interpretation accuracy using machine learning on basic features such as interpreter confidence, age, reader experience and designation. (C) 2018 Elsevier Inc. All rights reserved.
ER  -

BOND, Raymond R., Tomáš NOVOTNÝ, Irena ANDRŠOVÁ, Lumír KOC, Martina ŠIŠÁKOVÁ, Dewar FINLAY, Daniel GULDENRING, James MCLAUGHLIN, Aaron PEACE, Victoria MCGILLIGAN, Stephen J. LESLIE, Hui WANG a Marek MALIK. Automation bias in medicine: The influence of automated diagnoses on interpreter accuracy and uncertainty when reading electrocardiograms. \textit{JOURNAL OF ELECTROCARDIOLOGY}. PHILADELPHIA: CHURCHILL LIVINGSTONE INC MEDICAL PUBLISHERS, 2018, roč.~51, č.~6, s.~''S6''-''S11'', 6 s. ISSN~0022-0736. Dostupné z: https://dx.doi.org/10.1016/j.jelectrocard.2018.08.007.

Podrobný výpis o publikaci