1 12th CercleS ConferenceTesting and assessment at the tertiary level Pitfalls and challenges of relating testing and assessment to the CEFR at the tertiary level Blanka Pojslová Masaryk University Language Centre Brno, Czech Republic 12th International CercleS Conference 6 – 8 September 2012, LSE London, UK 2 12th CercleS ConferenceTesting and assessment at the tertiary level CercleS 2012 Who we are What our testing situation is like What we are trying to achieve in the field of testing What sources we are drawing on Where we are in terms of relating our tests to the CEFR What is meant by a small-scale approach vs. large-scale approach 3 12th CercleS ConferenceTesting and assessment at the tertiary level Who we are Language Centre (LC) of Masaryk University (MU), Brno, Czech Republic LSP and LAP courses provider for MU Test provider for LSP and LAP courses at MU Masaryk University Language Education System The process of standardisation of language teaching and assessment in relation to the CEFR (2006) 4 12th CercleS ConferenceTesting and assessment at the tertiary level What our testing situation is like LC department at the Faculty of Languages taught and targeted levels Economics (11 tests x parallel versions) English / German / French – B1 / B2 / C1 Spanish – B2 / B1 Law (11 tests x parallel versions) English / German – B1 / B2 French / Spanish – B2 + Latin Social Studies and Arts (8 tests x parallel versions) English / German / French / Spanish – B2 (LAP) Science (8 tests x parallel versions) English / German / French / Spanish – B1 / B2 Education (18 tests x parallel versions) English / German – B1 / B2 Medicine (12 tests x parallel versions) English – B2 Czech – B1 + Latin Sports Studies (18 tests x parallel versions) English – B1 / B2 Informatics (1test x parallel versions) English – B2 5 12th CercleS ConferenceTesting and assessment at the tertiary level What we are trying to achieve External standardisation – to the CEFR Internal standardisation unified assessment criteria unified format of our final / end-of-course tests 6 12th CercleS ConferenceTesting and assessment at the tertiary level What sources we are drawing on in this pursuit The Common European Framework of Reference for Languages (CEFR) What is assessed How to interpret performance How comparison can be made A Manual for Relating Language Examinations to the CEFR Familiarisation Specification Standardisation Training and Benchmarking Standard Setting Validation Manual for Language Test Development and Examining Guidance for good practice in test development 7 12th CercleS ConferenceTesting and assessment at the tertiary level Where we are in terms of relating our tests to CEFR Basic requirement of relating tests to the CEFR Low awareness of good practice in language test (LT) development Teacher = tester Assessment criteria differ and also their application Two EU-funded projects 8 12th CercleS ConferenceTesting and assessment at the tertiary level Small-scale approach: COMPACT Model final tests developed in compliance with good practice in LT development at the tertiary level Small testing team Time management, On-line testing, etc. Assessment criteria – speaking Familiarisation seminars 9 12th CercleS ConferenceTesting and assessment at the tertiary level Large-scale approach: IMPACT Raise awareness in LT development among all the LC teachers = testers Have tests demonstrating their own validity & ready for linking them to the CEFR Series of LT seminars Practical application Awareness-raising Standard Setting seminar Small scale approach – model Standard Setting 10 12th CercleS ConferenceTesting and assessment at the tertiary level Challenges Resources Panel of judges Variety of languages at our LC 11 12th CercleS ConferenceTesting and assessment at the tertiary level To conclude... First steps may be modest, but the aim is to help examination providers to work within a structure, so that later work can build on what has been done before. (Manual:p.9) By targeting this distant goal we are learning a lot and our students are being treated in a far more fair way and benefiting from this process even now 12 12th CercleS ConferenceTesting and assessment at the tertiary level Thank you … … for your attention!