Evaluating Algorithmic Bias in Models for Predicting Academic
Performance of Filipino Students

a 2024

Evaluating Algorithmic Bias in Models for Predicting Academic Performance of Filipino Students

ŠVÁBENSKÝ, Valdemar; Mélina VERGER; Maria Mercedes T. RODRIGO; Clarence James G. MONTEROZO; Ryan S. BAKER et al.

Základní údaje

Originální název

Evaluating Algorithmic Bias in Models for Predicting Academic Performance of Filipino Students

Autoři

ŠVÁBENSKÝ, Valdemar ; Mélina VERGER; Maria Mercedes T. RODRIGO; Clarence James G. MONTEROZO; Ryan S. BAKER; Miguel Zenon Nicanor L. SAAVEDRA; Sébastien LALLÉ a Atsushi SHIMADA

Vydání

17th International Conference on Educational Data Mining (EDM 2024), 2024

Další údaje

Jazyk

angličtina

Typ výsledku

Konferenční abstrakt

Obor

10201 Computer sciences, information science, bioinformatics

Stát vydavatele

Spojené státy

Utajení

není předmětem státního či obchodního tajemství

Odkazy

Published version in the proceedings

Označené pro přenos do RIV

DOI

https://doi.org/10.5281/zenodo.12729936

Klíčová slova anglicky

fairness; online learning; distance education; performance prediction; educational data mining; learning analytics

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 15. 7. 2024 06:36, RNDr. Valdemar Švábenský, Ph.D.

Anotace

V originále

Algorithmic bias is a major issue in machine learning models in educational contexts. However, it has not yet been studied thoroughly in Asian learning contexts, and only limited work has considered algorithmic bias based on regional (sub-national) background. As a step towards addressing this gap, this paper examines the population of 5,986 students at a large university in the Philippines, investigating algorithmic bias based on students' regional background. The university used the Canvas learning management system (LMS) in its online courses across a broad range of domains. Over the period of three semesters, we collected 48.7 million log records of the students' activity in Canvas. We used these logs to train binary classification models that predict student grades from the LMS activity. The best-performing model reached AUC of 0.75 and weighted F1-score of 0.79. Subsequently, we examined the data for bias based on students' region. Evaluation using three metrics: AUC, weighted F1-score, and MADD showed consistent results across all demographic groups. Thus, no unfairness was observed against a particular student group in the grade predictions.

Přiložené soubory

2024-EDM-evaluating-algorithmic-bias-models-predicting-academic-performance-filipino-students-poster.pdf

2024-EDM-evaluating-algorithmic-bias-models-predicting-academic-performance-filipino-students-paper.pdf

Citovat

ŠVÁBENSKÝ, Valdemar; Mélina VERGER; Maria Mercedes T. RODRIGO; Clarence James G. MONTEROZO; Ryan S. BAKER; Miguel Zenon Nicanor L. SAAVEDRA; Sébastien LALLÉ a Atsushi SHIMADA. Evaluating Algorithmic Bias in Models for Predicting Academic Performance of Filipino Students. In 17th International Conference on Educational Data Mining (EDM 2024). 2024. Dostupné z: https://doi.org/10.5281/zenodo.12729936.

@proceedings{2401897,
   author = {Švábenský, Valdemar and Verger, Mélina and Rodrigo, Maria Mercedes T. and Monterozo, Clarence James G. and Baker, Ryan S. and Saavedra, Miguel Zenon Nicanor L. and Lallé, Sébastien and Shimada, Atsushi},
   booktitle = {17th International Conference on Educational Data Mining (EDM 2024)},
   doi = {https://doi.org/10.5281/zenodo.12729936},
   keywords = {fairness; online learning; distance education; performance prediction; educational data mining; learning analytics},
   language = {eng},
   title = {Evaluating Algorithmic Bias in Models for Predicting Academic Performance of Filipino Students},
   url = {https://educationaldatamining.org/edm2024/proceedings/2024.EDM-posters.82/2024.EDM-posters.82.pdf},
   year = {2024}
}

TY  - CONF
ID  - 2401897
AU  - Švábenský, Valdemar - Verger, Mélina - Rodrigo, Maria Mercedes T. - Monterozo, Clarence James G. - Baker, Ryan S. - Saavedra, Miguel Zenon Nicanor L. - Lallé, Sébastien - Shimada, Atsushi
PY  - 2024
TI  - Evaluating Algorithmic Bias in Models for Predicting Academic Performance of Filipino Students
KW  - fairness
KW  - online learning
KW  - distance education
KW  - performance prediction
KW  - educational data mining
KW  - learning analytics
UR  - https://educationaldatamining.org/edm2024/proceedings/2024.EDM-posters.82/2024.EDM-posters.82.pdf
N2  - Algorithmic bias is a major issue in machine learning models in educational contexts. However, it has not yet been studied thoroughly in Asian learning contexts, and only limited work has considered algorithmic bias based on regional (sub-national) background. As a step towards addressing this gap, this paper examines the population of 5,986 students at a large university in the Philippines, investigating algorithmic bias based on students' regional background. The university used the Canvas learning management system (LMS) in its online courses across a broad range of domains. Over the period of three semesters, we collected 48.7 million log records of the students' activity in Canvas. We used these logs to train binary classification models that predict student grades from the LMS activity. The best-performing model reached AUC of 0.75 and weighted F1-score of 0.79. Subsequently, we examined the data for bias based on students' region. Evaluation using three metrics: AUC, weighted F1-score, and MADD showed consistent results across all demographic groups. Thus, no unfairness was observed against a particular student group in the grade predictions.
ER  -

ŠVÁBENSKÝ, Valdemar; Mélina VERGER; Maria Mercedes T. RODRIGO; Clarence James G. MONTEROZO; Ryan S. BAKER; Miguel Zenon Nicanor L. SAAVEDRA; Sébastien LALLÉ a Atsushi SHIMADA. Evaluating Algorithmic Bias in Models for Predicting Academic Performance of Filipino Students. In \textit{17th International Conference on Educational Data Mining (EDM 2024)}. 2024. Dostupné z: https://doi.org/10.5281/zenodo.12729936.

Přehled o publikaci