Pitfalls in users' evaluation of algorithms for text-based
similarity detection in medical education

D 2018

Pitfalls in users' evaluation of algorithms for text-based similarity detection in medical education

ŠČAVNICKÝ, Jakub, Matěj KAROLYI, Petra RŮŽIČKOVÁ, Andrea POKORNÁ, Hana HARAZIM et. al.

Basic information

Original name

Pitfalls in users' evaluation of algorithms for text-based similarity detection in medical education

Authors

ŠČAVNICKÝ, Jakub (703 Slovakia, belonging to the institution), Matěj KAROLYI (203 Czech Republic, belonging to the institution), Petra RŮŽIČKOVÁ (203 Czech Republic, belonging to the institution), Andrea POKORNÁ (203 Czech Republic, belonging to the institution), Hana HARAZIM (703 Slovakia, belonging to the institution), Petr ŠTOURAČ (203 Czech Republic, belonging to the institution) and Martin KOMENDA (203 Czech Republic, guarantor, belonging to the institution)

Edition

New York, PROCEEDINGS OF THE 2018 FEDERATED CONFERENCE ON COMPUTER SCIENCE AND INFORMATION SYSTEMS (FEDCSIS), p. 109-116, 8 pp. 2018

Publisher

IEEE

Other information

Language

English

Type of outcome

Stať ve sborníku

Field of Study

10201 Computer sciences, information science, bioinformatics

Country of publisher

United States of America

Confidentiality degree

není předmětem státního či obchodního tajemství

Publication form

electronic version available online

References:

URL

RIV identification code

RIV/00216224:14110/18:00104404

Organization unit

Faculty of Medicine

ISBN

978-83-949419-5-6

ISSN

UT WoS

000454652300017

Keywords in English

Correlation; education; medical diagnostic imaging; databases; tools; automobiles

Abstract

V originále

This paper introduces a user evaluation of several approaches for an automated similarity detection between study materials and curriculum description in the field of medical and healthcare education. Our objective is to present an effective methodology of getting relevant feedback from medical students and teachers. Two various data sets (electronic study materials represented by interactive educational algorithms on the AKUTNE.CZ platform and the curriculum of the General Medicine study programme) are processed. For the purposes of this work, text similarity between two data sets is expressed lexically, i.e. character-based (n-gram) similarity as well as term-based similarity methods are used. We present the comparison of five selected approaches to similarity calculation as well as an objective discussion covering our experience with and pitfalls of user evaluation.

Links

CZ.02.2.67/0.0/0.0/16_016/0002416, interní kód MU

Name: Strategické investice Masarykovy univerzity do vzdělávání SIMU+

Investor: Ministry of Education, Youth and Sports of the CR, Priority axis 2: Development of universities and human resources for research and development

CZ.02.2.69/0.0/0.0/16_015/0002418, interní kód MU

Name: Masarykova univerzita 4.0 (Acronym: MUNI 4.0)

Investor: Ministry of Education, Youth and Sports of the CR, Priority axis 2: Development of universities and human resources for research and development

MUNI/A/1339/2016, interní kód MU

Name: MERGER: detekce vazeb mezi informačními systémy pro mapování kurikula a pro virtuální pacienty (Acronym: MERGER)

Investor: Masaryk University, Category A

Citovat

ŠČAVNICKÝ, Jakub, Matěj KAROLYI, Petra RŮŽIČKOVÁ, Andrea POKORNÁ, Hana HARAZIM, Petr ŠTOURAČ and Martin KOMENDA. Pitfalls in users' evaluation of algorithms for text-based similarity detection in medical education. Online. In Maria Ganzha, Leszek Maciaszek, Marcin Paprzycki. PROCEEDINGS OF THE 2018 FEDERATED CONFERENCE ON COMPUTER SCIENCE AND INFORMATION SYSTEMS (FEDCSIS). New York: IEEE, 2018, p. 109-116. ISBN 978-83-949419-5-6.

@inproceedings{1465136,
   author = {Ščavnický, Jakub and Karolyi, Matěj and Růžičková, Petra and Pokorná, Andrea and Harazim, Hana and Štourač, Petr and Komenda, Martin},
   address = {New York},
   booktitle = {PROCEEDINGS OF THE 2018 FEDERATED CONFERENCE ON COMPUTER SCIENCE AND INFORMATION SYSTEMS (FEDCSIS)},
   editor = {Maria Ganzha, Leszek Maciaszek, Marcin Paprzycki},
   keywords = {Correlation; education; medical diagnostic imaging; databases; tools; automobiles},
   howpublished = {elektronická verze "online"},
   language = {eng},
   location = {New York},
   isbn = {978-83-949419-5-6},
   pages = {109-116},
   publisher = {IEEE},
   title = {Pitfalls in users' evaluation of algorithms for text-based similarity detection in medical education},
   url = {https://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=8490018},
   year = {2018}
}

TY  - JOUR
ID  - 1465136
AU  - Ščavnický, Jakub - Karolyi, Matěj - Růžičková, Petra - Pokorná, Andrea - Harazim, Hana - Štourač, Petr - Komenda, Martin
PY  - 2018
TI  - Pitfalls in users' evaluation of algorithms for text-based similarity detection in medical education
PB  - IEEE
CY  - New York
SN  - 9788394941956
KW  - Correlation
KW  - education
KW  - medical diagnostic imaging
KW  - databases
KW  - tools
KW  - automobiles
UR  - https://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=8490018
L2  - https://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=8490018
N2  - This paper introduces a user evaluation of several approaches for an automated similarity detection between study materials and curriculum description in the field of medical and healthcare education. Our objective is to present an effective methodology of getting relevant feedback from medical students and teachers. Two various data sets (electronic study materials represented by interactive educational algorithms on the AKUTNE.CZ platform and the curriculum of the General Medicine study programme) are processed. For the purposes of this work, text similarity between two data sets is expressed lexically, i.e. character-based (n-gram) similarity as well as term-based similarity methods are used. We present the comparison of five selected approaches to similarity calculation as well as an objective discussion covering our experience with and pitfalls of user evaluation.
ER  -

ŠČAVNICKÝ, Jakub, Matěj KAROLYI, Petra RŮŽIČKOVÁ, Andrea POKORNÁ, Hana HARAZIM, Petr ŠTOURAČ and Martin KOMENDA. Pitfalls in users' evaluation of algorithms for text-based similarity detection in medical education. Online. In Maria Ganzha, Leszek Maciaszek, Marcin Paprzycki. \textit{PROCEEDINGS OF THE 2018 FEDERATED CONFERENCE ON COMPUTER SCIENCE AND INFORMATION SYSTEMS (FEDCSIS)}. New York: IEEE, 2018, p.~109-116. ISBN~978-83-949419-5-6.

Detailed Information on Publication Record