Applications of deep language models for reflective writings

J 2023

Applications of deep language models for reflective writings

NEHYBA, Jan a Michal ŠTEFÁNIK

Základní údaje

Originální název

Applications of deep language models for reflective writings

Autoři

NEHYBA, Jan a Michal ŠTEFÁNIK

Vydání

Education and Information Technologies, UNITED STATES, SPRINGER, 2023, 1360-2357

Další údaje

Jazyk

angličtina

Typ výsledku

Článek v odborném periodiku

Obor

50301 Education, general; including training, pedagogy, didactics [and education systems]

Stát vydavatele

Spojené státy

Utajení

není předmětem státního či obchodního tajemství

Odkazy

URL

Impakt faktor

Impact factor: 4.800

Označené pro přenos do RIV

Ano

Kód RIV

RIV/00216224:14330/23:00129992

Organizační jednotka

Fakulta informatiky

Klíčová slova anglicky

Deep learning; Natural language processing; Reflection dataset; Reflection classification; Analyses of reflective journals; Generalized linear mixed models

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 22. 2. 2024 15:58, Mgr. Daniela Marcollová

Anotace

V originále

Social sciences expose many cognitively complex, highly qualified, or fuzzy problems, whose resolution relies primarily on expert judgement rather than automated systems. One of such instances that we study in this work is a reflection analysis in the writings of student teachers. We share a hands-on experience on how these challenges can be successfully tackled in data collection for machine learning. Based on the novel deep learning architectures pre-trained for a general language understanding, we can reach an accuracy ranging from 76.56–79.37% on low-confidence samples to 97.56–100% on high confidence cases. We open-source all our resources and models, and use the models to analyse previously ungrounded hypotheses on reflection of university students. Our work provides a toolset for objective measurements of reflection in higher education writings, applicable in more than 100 other languages worldwide with a loss in accuracy measured between 0–4.2% Thanks to the outstanding accuracy of the deep models, the presented toolset allows for previously unavailable applications, such as providing semi-automated student feedback or measuring an effect of systematic changes in the educational process via the students’ response.

Návaznosti

MUNI/A/1339/2022, interní kód MU

Název: Rozvoj technik pro zpracování dat pro podporu vyhledávání, analýz a vizualizací rozsáhlých datových souborů s využitím umělé inteligence

Investor: Masarykova univerzita, Rozvoj technik pro zpracování dat pro podporu vyhledávání, analýz a vizualizací rozsáhlých datových souborů s využitím umělé inteligence

Citovat

NEHYBA, Jan a Michal ŠTEFÁNIK. Applications of deep language models for reflective writings. Education and Information Technologies. UNITED STATES: SPRINGER, 2023, roč. 28, č. 3, s. 2961-2999. ISSN 1360-2357. Dostupné z: https://doi.org/10.1007/s10639-022-11254-7.

@article{1876027,
   author = {Nehyba, Jan and Štefánik, Michal},
   article_location = {UNITED STATES},
   article_number = {3},
   doi = {https://doi.org/10.1007/s10639-022-11254-7},
   keywords = {Deep learning; Natural language processing; Reflection dataset; Reflection classification; Analyses of reflective journals; Generalized linear mixed models},
   language = {eng},
   issn = {1360-2357},
   journal = {Education and Information Technologies},
   title = {Applications of deep language models for reflective writings},
   url = {https://link.springer.com/article/10.1007/s10639-022-11254-7},
   volume = {28},
   year = {2023}
}

TY  - JOUR
ID  - 1876027
AU  - Nehyba, Jan - Štefánik, Michal
PY  - 2023
TI  - Applications of deep language models for reflective writings
JF  - Education and Information Technologies
VL  - 28
IS  - 3
SP  - 2961-2999
EP  - 2961-2999
PB  - SPRINGER
SN  - 13602357
KW  - Deep learning
KW  - Natural language processing
KW  - Reflection dataset
KW  - Reflection classification
KW  - Analyses of reflective journals
KW  - Generalized linear mixed models
UR  - https://link.springer.com/article/10.1007/s10639-022-11254-7
N2  - Social sciences expose many cognitively complex, highly qualified, or fuzzy problems, whose resolution relies primarily on expert judgement rather than automated systems. One of such instances that we study in this work is a reflection analysis in the writings of student teachers. We share a hands-on experience on how these challenges can be successfully tackled in data collection for machine learning. Based on the novel deep learning architectures pre-trained for a general language understanding, we can reach an accuracy ranging from 76.56–79.37% on low-confidence samples to 97.56–100% on high confidence cases. We open-source all our resources and models, and use the models to analyse previously ungrounded hypotheses on reflection of university students. Our work provides a toolset for objective measurements of reflection in higher education writings, applicable in more than 100 other languages worldwide with a loss in accuracy measured between 0–4.2% Thanks to the outstanding accuracy of the deep models, the presented toolset allows for previously unavailable applications, such as providing semi-automated student feedback or measuring an effect of systematic changes in the educational process via the students’ response.
ER  -

NEHYBA, Jan a Michal ŠTEFÁNIK. Applications of deep language models for reflective writings. \textit{Education and Information Technologies}. UNITED STATES: SPRINGER, 2023, roč.~28, č.~3, s.~2961-2999. ISSN~1360-2357. Dostupné z: https://doi.org/10.1007/s10639-022-11254-7.

Přehled o publikaci