CÍGLER, Hynek, Michal JABŮREK and Jan ŠIRŮČEK. Reanalyzing the DISMAS Test Data: Comparing IRT and CTT Based Estimates of the Error of Measurement. In The 9th Conference of The International Test Commission : Global and Local Challenges for Best Practices in Assessment. 2014. ISBN 978-84-9860-997-4.
Other formats:   BibTeX LaTeX RIS
Basic information
Original name Reanalyzing the DISMAS Test Data: Comparing IRT and CTT Based Estimates of the Error of Measurement
Authors CÍGLER, Hynek (203 Czech Republic, guarantor, belonging to the institution), Michal JABŮREK (203 Czech Republic, belonging to the institution) and Jan ŠIRŮČEK (203 Czech Republic, belonging to the institution).
Edition The 9th Conference of The International Test Commission : Global and Local Challenges for Best Practices in Assessment, 2014.
Other information
Original language English
Type of outcome Conference abstract
Field of Study 50100 5.1 Psychology and cognitive sciences
Country of publisher Spain
Confidentiality degree is not subject to a state or trade secret
WWW Book of Abstract
RIV identification code RIV/00216224:14230/14:00076015
Organization unit Faculty of Social Studies
ISBN 978-84-9860-997-4
Keywords (in Czech) DISMAS; IRT; DIF; diagnostika matematických schopností
Keywords in English DISMAS; IRT; DIF; assessment of math ability
Tags International impact
Changed by Changed by: Mgr. Michal Jabůrek, Ph.D., učo 167235. Changed: 21/7/2014 16:39.
Abstract
DISMAS test (Traspe & Skalkova, 2013) is an original Czech assessment tool for the diagnosis of the structure of math disability in children in grades 1–5. It consists of five main areas divided into 14 unidimensional subtests and it was developed and standardized using classical test theory. Authors verified validity. Subtests’ correlations with WISC-III were low, but significant, differences between normal and clinical population were high and significant for all grades. Regardless the high validity, reliability seemed to be poor – Cronbach alpha ranged from .44 to .95 on subtest level, on the test level from .69 to .76 depending on the grade level. The aim of this paper is to show why. We questioned the adequacy of CTT and internal consistency as a method for estimating reliability in this case. That is why we applied simple Rasch model and Partial Credit Rasch model to reanalyze the data. Consequently, the latent trait estimates as well as their error of measurement were transformed back to the raw score scale. We compared IRT and CTT based errors of measurement on the same scale for all observed raw scores according to the distribution of the standardization sample. In most subtests the error of estimate based on CTT was smaller around the average score but bigger in the extremes, especially at the low end of the scale (DISMAS is designed to test disability, so it has a high ceiling effect). We conclude that IRT analysis produces more accurate estimate for children with math disability, for whom the test is designed, so in this case Rasch model seems to be more appropriate method to estimate confidence intervals than CTT. Finally, we discuss other sources of relatively low reliability.
PrintDisplayed: 31/7/2024 03:17