The Usefulness of the CEFR in the Investigation of Test
Versions Content Equivalence

D 2018

The Usefulness of the CEFR in the Investigation of Test Versions Content Equivalence

HULEŠOVÁ, Martina

Základní údaje

Originální název

The Usefulness of the CEFR in the Investigation of Test Versions Content Equivalence

Autoři

HULEŠOVÁ, Martina (203 Česká republika, garant, domácí)

Vydání

Bologna, PROCEEDINGS OF THE 11TH INNOVATION IN LANGUAGE LEARNING INTERNATIONAL CONFERENCE, od s. 12-16, 5 s. 2018

Nakladatel

Filodiritto Publisher

Další údaje

Jazyk

angličtina

Typ výsledku

Stať ve sborníku

Obor

60202 Specific languages

Stát vydavatele

Itálie

Utajení

není předmětem státního či obchodního tajemství

Forma vydání

tištěná verze "print"

Kód RIV

RIV/00216224:14410/18:00112616

Organizační jednotka

Pedagogická fakulta

ISBN

978-88-85813-21-2

ISSN

UT WoS

000471904400001

Klíčová slova anglicky

expert judgement; CEFR descriptors; content and construct equivalence; test versions equivalence

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 7. 4. 2020 13:28, Dana Nesnídalová

Anotace

V originále

This paper presents one part of the PhD research realized within the broader framework of test versions equivalence in high-stakes testing context, particularly in the Slovak upper-secondary school leaving exam in English at B1 level (Maturita). The objectives of the research project are to investigate the extent of equivalence of the test versions used between 2012 and 2015 and, on the basis of the results, to propose what processes could be implemented in the test development with the objective to reach test version equivalence. In this paper, we focus on the use of the CEFR as a tool for the investigation of content and construct equivalence as the Maturita exam claims to be linked to the B1 CEFR level. Content structure analysis using expert judgement and item-descriptor matching method were conducted and the agreement coefficients were calculated. Preliminary findings indicate that CEFR descriptors can be problematic for describing the test content and construct at a discrete, detailed level, as the descriptors differ in terms of completeness, structure and specificity level. The use of CEFR-based descriptive models is also problematized by the fact that the characteristics of test items are seen as the result of the interaction among test takers' proficiency, design of the item, expert judges' characteristics and their internalization of the judgement task. The key findings of the analysis and the usefulness of the CEFR for this purpose will be discussed in light of the whole research project and possible further steps will be presented.

Návaznosti

MUNI/A/0881/2017, interní kód MU

Název: Aktev VII. (Akronym: Aktev VII.)

Investor: Masarykova univerzita, Aktev VII., DO R. 2020_Kategorie A - Specifický výzkum - Studentské výzkumné projekty

Citovat

HULEŠOVÁ, Martina. The Usefulness of the CEFR in the Investigation of Test Versions Content Equivalence. In PROCEEDINGS OF THE 11TH INNOVATION IN LANGUAGE LEARNING INTERNATIONAL CONFERENCE. Bologna: Filodiritto Publisher, 2018, s. 12-16. ISBN 978-88-85813-21-2.

@inproceedings{1606997,
   author = {Hulešová, Martina},
   address = {Bologna},
   booktitle = {PROCEEDINGS OF THE 11TH INNOVATION IN LANGUAGE LEARNING INTERNATIONAL CONFERENCE},
   keywords = {expert judgement; CEFR descriptors; content and construct equivalence; test versions equivalence},
   howpublished = {tištěná verze "print"},
   language = {eng},
   location = {Bologna},
   isbn = {978-88-85813-21-2},
   pages = {12-16},
   publisher = {Filodiritto Publisher},
   title = {The Usefulness of the CEFR in the Investigation of Test Versions Content Equivalence},
   year = {2018}
}

TY  - JOUR
ID  - 1606997
AU  - Hulešová, Martina
PY  - 2018
TI  - The Usefulness of the CEFR in the Investigation of Test Versions Content Equivalence
PB  - Filodiritto Publisher
CY  - Bologna
SN  - 9788885813212
KW  - expert judgement
KW  - CEFR descriptors
KW  - content and construct equivalence
KW  - test versions equivalence
N2  - This paper presents one part of the PhD research realized within the broader framework of test versions equivalence in high-stakes testing context, particularly in the Slovak upper-secondary school leaving exam in English at B1 level (Maturita). The objectives of the research project are to investigate the extent of equivalence of the test versions used between 2012 and 2015 and, on the basis of the results, to propose what processes could be implemented in the test development with the objective to reach test version equivalence. In this paper, we focus on the use of the CEFR as a tool for the investigation of content and construct equivalence as the Maturita exam claims to be linked to the B1 CEFR level. Content structure analysis using expert judgement and item-descriptor matching method were conducted and the agreement coefficients were calculated. Preliminary findings indicate that CEFR descriptors can be problematic for describing the test content and construct at a discrete, detailed level, as the descriptors differ in terms of completeness, structure and specificity level. The use of CEFR-based descriptive models is also problematized by the fact that the characteristics of test items are seen as the result of the interaction among test takers' proficiency, design of the item, expert judges' characteristics and their internalization of the judgement task. The key findings of the analysis and the usefulness of the CEFR for this purpose will be discussed in light of the whole research project and possible further steps will be presented.
ER  -

HULEŠOVÁ, Martina. The Usefulness of the CEFR in the Investigation of Test Versions Content Equivalence. In \textit{PROCEEDINGS OF THE 11TH INNOVATION IN LANGUAGE LEARNING INTERNATIONAL CONFERENCE}. Bologna: Filodiritto Publisher, 2018, s.~12-16. ISBN~978-88-85813-21-2.

Podrobný výpis o publikaci