Validity and Reliability of Student Models for Problem-Solving
Activities

D 2021

Validity and Reliability of Student Models for Problem-Solving Activities

EFFENBERGER, Tomáš and Radek PELÁNEK

Basic information

Original name

Validity and Reliability of Student Models for Problem-Solving Activities

Authors

EFFENBERGER, Tomáš (203 Czech Republic, guarantor, belonging to the institution) and Radek PELÁNEK (203 Czech Republic, belonging to the institution)

Edition

New York, NY, USA, Proceedings of the 11th International Conference on Learning Analytics and Knowledge, p. 1-11, 11 pp. 2021

Publisher

Association for Computing Machinery

Other information

Language

English

Type of outcome

Stať ve sborníku

Field of Study

10201 Computer sciences, information science, bioinformatics

Country of publisher

United States of America

Confidentiality degree

není předmětem státního či obchodního tajemství

Publication form

electronic version available online

References:

URL

RIV identification code

RIV/00216224:14330/21:00121402

Organization unit

Faculty of Informatics

ISBN

978-1-4503-8935-8

DOI

http://dx.doi.org/10.1145/3448139.3448140

UT WoS

000883342500001

Keywords in English

student modeling; skills; difficulties; validity; reliability; performance measures; problem solving; introductory programming

Abstract

V originále

Student models are typically evaluated through predicting the correctness of the next answer. This approach is insufficient in the problem-solving context, especially for student models that use performance data beyond binary correctness. We propose more comprehensive methods for validating student models and illustrate them in the context of introductory programming. We demonstrate the insufficiency of the next answer correctness prediction task, as it is neither able to reveal low validity of student models that use just binary correctness, nor does it show increased validity of models that use other performance data. The key message is that the prevalent usage of the next answer correctness for validating student models and binary correctness as the only input to the models is not always warranted and limits the progress in learning analytics.

Links

MUNI/A/1549/2020, interní kód MU

Name: Zapojení studentů Fakulty informatiky do mezinárodní vědecké komunity 21 (Acronym: SKOMU)

Investor: Masaryk University

Citovat

EFFENBERGER, Tomáš and Radek PELÁNEK. Validity and Reliability of Student Models for Problem-Solving Activities. Online. In Proceedings of the 11th International Conference on Learning Analytics and Knowledge. New York, NY, USA: Association for Computing Machinery, 2021, p. 1-11. ISBN 978-1-4503-8935-8. Available from: https://dx.doi.org/10.1145/3448139.3448140.

@inproceedings{1759856,
   author = {Effenberger, Tomáš and Pelánek, Radek},
   address = {New York, NY, USA},
   booktitle = {Proceedings of the 11th International Conference on Learning Analytics and Knowledge},
   doi = {http://dx.doi.org/10.1145/3448139.3448140},
   keywords = {student modeling; skills; difficulties; validity; reliability; performance measures; problem solving; introductory programming},
   howpublished = {elektronická verze "online"},
   language = {eng},
   location = {New York, NY, USA},
   isbn = {978-1-4503-8935-8},
   pages = {1-11},
   publisher = {Association for Computing Machinery},
   title = {Validity and Reliability of Student Models for Problem-Solving Activities},
   url = {https://dl.acm.org/doi/10.1145/3448139.3448140},
   year = {2021}
}

TY  - JOUR
ID  - 1759856
AU  - Effenberger, Tomáš - Pelánek, Radek
PY  - 2021
TI  - Validity and Reliability of Student Models for Problem-Solving Activities
PB  - Association for Computing Machinery
CY  - New York, NY, USA
SN  - 9781450389358
KW  - student modeling
KW  - skills
KW  - difficulties
KW  - validity
KW  - reliability
KW  - performance measures
KW  - problem solving
KW  - introductory programming
UR  - https://dl.acm.org/doi/10.1145/3448139.3448140
N2  - Student models are typically evaluated through predicting the correctness of the next answer. This approach is insufficient in the problem-solving context, especially for student models that use performance data beyond binary correctness. We propose more comprehensive methods for validating student models and illustrate them in the context of introductory programming. We demonstrate the insufficiency of the next answer correctness prediction task, as it is neither able to reveal low validity of student models that use just binary correctness, nor does it show increased validity of models that use other performance data. The key message is that the prevalent usage of the next answer correctness for validating student models and binary correctness as the only input to the models is not always warranted and limits the progress in learning analytics.
ER  -

EFFENBERGER, Tomáš and Radek PELÁNEK. Validity and Reliability of Student Models for Problem-Solving Activities. Online. In \textit{Proceedings of the 11th International Conference on Learning Analytics and Knowledge}. New York, NY, USA: Association for Computing Machinery, 2021, p.~1-11. ISBN~978-1-4503-8935-8. Available from: https://dx.doi.org/10.1145/3448139.3448140.

Detailed Information on Publication Record