D 2017

Measuring predictive performance of user models: The details matter

PELÁNEK, Radek

Základní údaje

Originální název

Measuring predictive performance of user models: The details matter

Autoři

PELÁNEK, Radek (203 Česká republika, garant, domácí)

Vydání

USA, Adjunct Publication of the 25th Conference on User Modeling, Adaptation and Personalization, od s. 197-201, 5 s. 2017

Nakladatel

ACM

Další údaje

Jazyk

angličtina

Typ výsledku

Stať ve sborníku

Obor

10201 Computer sciences, information science, bioinformatics

Stát vydavatele

Spojené státy

Utajení

není předmětem státního či obchodního tajemství

Forma vydání

elektronická verze "online"

Kód RIV

RIV/00216224:14330/17:00100554

Organizační jednotka

Fakulta informatiky

ISBN

978-1-4503-5067-9

DOI

UT WoS

000850443800038

Klíčová slova anglicky

AUC; evaluation; metrics; predictive accuracy; RMSE; student modeling
Změněno: 25. 10. 2024 16:26, Mgr. Natálie Hílek

Anotace

V originále

Evaluation of user modeling techniques is often based on the predictive accuracy of models. The quantification of predictive accuracy is done using performance metrics. We show that the choice of a performance metric is important and that even details of metric computation matter. We analyze in detail two commonly used metrics (AUC, RMSE) in the context of student modeling. We discuss different approaches to their computation (global, averaging across skill, averaging across students) and show that these methods have different properties. An analysis of recent research papers shows that the reported descriptions of metric computation are often insufficient. To make research conclusions valid and reproducible, researchers need to pay more attention to the choice of performance metrics and they need to describe more explicitly details of their computation