D 2017

Measuring predictive performance of user models: The details matter

PELÁNEK, Radek

Basic information

Original name

Measuring predictive performance of user models: The details matter

Authors

PELÁNEK, Radek (203 Czech Republic, guarantor, belonging to the institution)

Edition

USA, Adjunct Publication of the 25th Conference on User Modeling, Adaptation and Personalization, p. 197-201, 5 pp. 2017

Publisher

ACM

Other information

Language

English

Type of outcome

Stať ve sborníku

Field of Study

10201 Computer sciences, information science, bioinformatics

Country of publisher

United States of America

Confidentiality degree

není předmětem státního či obchodního tajemství

Publication form

electronic version available online

RIV identification code

RIV/00216224:14330/17:00100554

Organization unit

Faculty of Informatics

ISBN

978-1-4503-5067-9

DOI

UT WoS

000850443800038

Keywords in English

AUC; evaluation; metrics; predictive accuracy; RMSE; student modeling
Změněno: 25/10/2024 16:26, Mgr. Natálie Hílek

Abstract

V originále

Evaluation of user modeling techniques is often based on the predictive accuracy of models. The quantification of predictive accuracy is done using performance metrics. We show that the choice of a performance metric is important and that even details of metric computation matter. We analyze in detail two commonly used metrics (AUC, RMSE) in the context of student modeling. We discuss different approaches to their computation (global, averaging across skill, averaging across students) and show that these methods have different properties. An analysis of recent research papers shows that the reported descriptions of metric computation are often insufficient. To make research conclusions valid and reproducible, researchers need to pay more attention to the choice of performance metrics and they need to describe more explicitly details of their computation