Better Model, Worse Predictions: The Dangers in Student Model
Comparisons

D 2021

Better Model, Worse Predictions: The Dangers in Student Model Comparisons

ČECHÁK, Jaroslav and Radek PELÁNEK

Basic information

Original name

Better Model, Worse Predictions: The Dangers in Student Model Comparisons

Authors

ČECHÁK, Jaroslav (203 Czech Republic, belonging to the institution) and Radek PELÁNEK (203 Czech Republic, belonging to the institution)

Edition

Cham, International Conference on Artificial Intelligence in Education, p. 500-511, 12 pp. 2021

Publisher

Springer

Other information

Language

English

Type of outcome

Proceedings paper

Field of Study

10201 Computer sciences, information science, bioinformatics

Country of publisher

Switzerland

Confidentiality degree

is not subject to a state or trade secret

Publication form

printed version "print"

Impact factor

Impact factor: 0.402 in 2005

RIV identification code

RIV/00216224:14330/21:00121881

Organization unit

Faculty of Informatics

ISBN

978-3-030-78291-7

ISSN

DOI

https://doi.org/10.1007/978-3-030-78292-4_40

UT WoS

000885021300040

EID Scopus

2-s2.0-85126447678

Keywords in English

Additive factor model; Student modeling; Simulation; Model comparison

Abstract

In the original language

The additive factor model is a widely used tool for analyzing educational data, yet it is often used as an off-the-shelf solution without considering implementation details. A common practice is to compare multiple additive factor models, choose the one with the best predictive accuracy, and interpret the parameters of the model as evidence of student learning. In this work, we use simulated data to show that in certain situations, this approach can lead to misleading results. Specifically, we show how student skill distribution affects estimates of other model parameters.

Links

MUNI/A/1549/2020, interní kód MU

Name: Zapojení studentů Fakulty informatiky do mezinárodní vědecké komunity 21 (Acronym: SKOMU)

Investor: Masaryk University

MUNI/A/1573/2020, interní kód MU

Name: Aplikovaný výzkum: vyhledávání, analýza a vizualizace rozsáhlých dat, zpracování přirozeného jazyka, umělá inteligence pro analýzu biomedicínských obrazů.

Investor: Masaryk University

Cite

ČECHÁK, Jaroslav and Radek PELÁNEK. Better Model, Worse Predictions: The Dangers in Student Model Comparisons. In Roll, Ido and McNamara, Danielle and Sosnovsky, Sergey and Luckin, Rose and Dimitrova, Vania. International Conference on Artificial Intelligence in Education. Cham: Springer, 2021, p. 500-511. ISBN 978-3-030-78291-7. Available from: https://doi.org/10.1007/978-3-030-78292-4_40.

@inproceedings{1780791,
   author = {Čechák, Jaroslav and Pelánek, Radek},
   address = {Cham},
   booktitle = {International Conference on Artificial Intelligence in Education},
   doi = {https://doi.org/10.1007/978-3-030-78292-4_40},
   editor = {Roll, Ido and McNamara, Danielle and Sosnovsky, Sergey and Luckin, Rose and Dimitrova, Vania},
   keywords = {Additive factor model; Student modeling; Simulation; Model comparison},
   howpublished = {tištěná verze "print"},
   language = {eng},
   location = {Cham},
   isbn = {978-3-030-78291-7},
   pages = {500-511},
   publisher = {Springer},
   title = {Better Model, Worse Predictions: The Dangers in Student Model Comparisons},
   year = {2021}
}

TY  - CONF
ID  - 1780791
AU  - Čechák, Jaroslav - Pelánek, Radek
PY  - 2021
TI  - Better Model, Worse Predictions: The Dangers in Student Model Comparisons
PB  - Springer
CY  - Cham
SN  - 9783030782917
KW  - Additive factor model
KW  - Student modeling
KW  - Simulation
KW  - Model comparison
N2  - The additive factor model is a widely used tool for analyzing educational data, yet it is often used as an off-the-shelf solution without considering implementation details. A common practice is to compare multiple additive factor models, choose the one with the best predictive accuracy, and interpret the parameters of the model as evidence of student learning. In this work, we use simulated data to show that in certain situations, this approach can lead to misleading results. Specifically, we show how student skill distribution affects estimates of other model parameters.
ER  -

ČECHÁK, Jaroslav and Radek PELÁNEK. Better Model, Worse Predictions: The Dangers in Student Model Comparisons. In Roll, Ido and McNamara, Danielle and Sosnovsky, Sergey and Luckin, Rose and Dimitrova, Vania. \textit{International Conference on Artificial Intelligence in Education}. Cham: Springer, 2021, p.~500-511. ISBN~978-3-030-78291-7. Available from: https://doi.org/10.1007/978-3-030-78292-4\_{}40.

Přehled o publikaci