Anomaly Detection Algorithm for Real-World Data and Evidence in
Clinical Research: Implementation, Evaluation, and Validation
Study

J 2021

Anomaly Detection Algorithm for Real-World Data and Evidence in Clinical Research: Implementation, Evaluation, and Validation Study

CHUROVÁ, Vendula, Roman VYŠKOVSKÝ, Kateřina MARŠÁLOVÁ, David KUDLÁČEK, Daniel SCHWARZ et. al.

Základní údaje

Originální název

Anomaly Detection Algorithm for Real-World Data and Evidence in Clinical Research: Implementation, Evaluation, and Validation Study

Autoři

CHUROVÁ, Vendula (203 Česká republika, domácí), Roman VYŠKOVSKÝ (203 Česká republika, domácí), Kateřina MARŠÁLOVÁ (203 Česká republika), David KUDLÁČEK (203 Česká republika) a Daniel SCHWARZ (203 Česká republika, garant, domácí)

Vydání

JMIR MEDICAL INFORMATICS, TORONTO, JMIR PUBLICATIONS, INC, 2021, 2291-9694

Další údaje

Jazyk

angličtina

Typ výsledku

Článek v odborném periodiku

Obor

10201 Computer sciences, information science, bioinformatics

Stát vydavatele

Kanada

Utajení

není předmětem státního či obchodního tajemství

Odkazy

URL

Impakt faktor

Impact factor: 3.228

Kód RIV

RIV/00216224:14110/21:00120123

Organizační jednotka

Lékařská fakulta

DOI

http://dx.doi.org/10.2196/27172

UT WoS

000656664300014

Klíčová slova anglicky

clinical research data; real-world evidence; registry database; data quality; EDC system; anomaly detection

Štítky

14119612, rivok

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 16. 5. 2022 09:00, Mgr. Tereza Miškechová

Anotace

V originále

Background: Statistical analysis, which has become an integral part of evidence-based medicine, relies heavily on data quality that is of critical importance in modern clinical research. Input data are not only at risk of being falsified or fabricated, but also at risk of being mishandled by investigators. Objective: The urgent need to assure the highest data quality possible has led to the implementation of various auditing strategies designed to monitor clinical trials and detect errors of different origin that frequently occur in the field. The objective of this study was to describe a machine learning-based algorithm to detect anomalous patterns in data created as a consequence of carelessness, systematic error, or intentionally by entering fabricated values. Methods: A particular electronic data capture (EDC) system, which is used for data management in clinical registries, is presented including its architecture and data structure. This EDC system features an algorithm based on machine learning designed to detect anomalous patterns in quantitative data. The detection algorithm combines clustering with a series of 7 distance metrics that serve to determine the strength of an anomaly. For the detection process, the thresholds and combinations of the metrics were used and the detection performance was evaluated and validated in the experiments involving simulated anomalous data and real-world data. Results: Five different clinical registries related to neuroscience were presented-all of them running in the given EDC system. Two of the registries were selected for the evaluation experiments and served also to validate the detection performance on an independent data set. The best performing combination of the distance metrics was that of Canberra, Manhattan, and Mahalanobis, whereas Cosine and Chebyshev metrics had been excluded from further analysis due to the lowest performance when used as single distance metric-based classifiers. Conclusions: The experimental results demonstrate that the algorithm is universal in nature, and as such may be implemented in other EDC systems, and is capable of anomalous data detection with a sensitivity exceeding 85%.

Návaznosti

NV17-33136A, projekt VaV

Název: Neurominer: odhalování skrytých vzorů v datech ze zobrazování mozku

Citovat

CHUROVÁ, Vendula, Roman VYŠKOVSKÝ, Kateřina MARŠÁLOVÁ, David KUDLÁČEK a Daniel SCHWARZ. Anomaly Detection Algorithm for Real-World Data and Evidence in Clinical Research: Implementation, Evaluation, and Validation Study. JMIR MEDICAL INFORMATICS. TORONTO: JMIR PUBLICATIONS, INC, 2021, roč. 9, č. 5, s. 1-17. ISSN 2291-9694. Dostupné z: https://dx.doi.org/10.2196/27172.

@article{1787337,
   author = {Churová, Vendula and Vyškovský, Roman and Maršálová, Kateřina and Kudláček, David and Schwarz, Daniel},
   article_location = {TORONTO},
   article_number = {5},
   doi = {http://dx.doi.org/10.2196/27172},
   keywords = {clinical research data; real-world evidence; registry database; data quality; EDC system; anomaly detection},
   language = {eng},
   issn = {2291-9694},
   journal = {JMIR MEDICAL INFORMATICS},
   title = {Anomaly Detection Algorithm for Real-World Data and Evidence in Clinical Research: Implementation, Evaluation, and Validation Study},
   url = {https://medinform.jmir.org/2021/5/e27172},
   volume = {9},
   year = {2021}
}

TY  - JOUR
ID  - 1787337
AU  - Churová, Vendula - Vyškovský, Roman - Maršálová, Kateřina - Kudláček, David - Schwarz, Daniel
PY  - 2021
TI  - Anomaly Detection Algorithm for Real-World Data and Evidence in Clinical Research: Implementation, Evaluation, and Validation Study
JF  - JMIR MEDICAL INFORMATICS
VL  - 9
IS  - 5
SP  - 1-17
EP  - 1-17
PB  - JMIR PUBLICATIONS, INC
SN  - 22919694
KW  - clinical research data
KW  - real-world evidence
KW  - registry database
KW  - data quality
KW  - EDC system
KW  - anomaly detection
UR  - https://medinform.jmir.org/2021/5/e27172
N2  - Background: Statistical analysis, which has become an integral part of evidence-based medicine, relies heavily on data quality that is of critical importance in modern clinical research. Input data are not only at risk of being falsified or fabricated, but also at risk of being mishandled by investigators. Objective: The urgent need to assure the highest data quality possible has led to the implementation of various auditing strategies designed to monitor clinical trials and detect errors of different origin that frequently occur in the field. The objective of this study was to describe a machine learning-based algorithm to detect anomalous patterns in data created as a consequence of carelessness, systematic error, or intentionally by entering fabricated values. Methods: A particular electronic data capture (EDC) system, which is used for data management in clinical registries, is presented including its architecture and data structure. This EDC system features an algorithm based on machine learning designed to detect anomalous patterns in quantitative data. The detection algorithm combines clustering with a series of 7 distance metrics that serve to determine the strength of an anomaly. For the detection process, the thresholds and combinations of the metrics were used and the detection performance was evaluated and validated in the experiments involving simulated anomalous data and real-world data. Results: Five different clinical registries related to neuroscience were presented-all of them running in the given EDC system. Two of the registries were selected for the evaluation experiments and served also to validate the detection performance on an independent data set. The best performing combination of the distance metrics was that of Canberra, Manhattan, and Mahalanobis, whereas Cosine and Chebyshev metrics had been excluded from further analysis due to the lowest performance when used as single distance metric-based classifiers. Conclusions: The experimental results demonstrate that the algorithm is universal in nature, and as such may be implemented in other EDC systems, and is capable of anomalous data detection with a sensitivity exceeding 85%.
ER  -

CHUROVÁ, Vendula, Roman VYŠKOVSKÝ, Kateřina MARŠÁLOVÁ, David KUDLÁČEK a Daniel SCHWARZ. Anomaly Detection Algorithm for Real-World Data and Evidence in Clinical Research: Implementation, Evaluation, and Validation Study. \textit{JMIR MEDICAL INFORMATICS}. TORONTO: JMIR PUBLICATIONS, INC, 2021, roč.~9, č.~5, s.~1-17. ISSN~2291-9694. Dostupné z: https://dx.doi.org/10.2196/27172.

Podrobný výpis o publikaci