An Integrated Mixed Methods Research Design:
Example of the Project „Foreign Language Learning Strategies and Achievement: Analysis of Strategy
Clusters and Sequences“
Katerina Vlckova
Institute of Research in School Education
Masaryk-University, Brno, Czech Rep.
vlckova@ped.muni.cz
Mixed Methods Workshop
GESIS, Mannheim
9. 5. 2014
This paper was funded by Czech Science Foundation – Project Nr. GAP407/12/0432
Foreign Language Learning Strategies and Achievement: Analysis of Strategy Clusters and Sequences.
Research question
What are the typical clusters und sequences of foreign language (English) learning strategies for the 4 language skills (reading,
writing, listening, speaking) in the connection to achievement at the end of lower secondary compulsory education?
Aim
1) to introduce the concept of strategy sequences and
clusters as theoretical construct,
2) to find out the typical sequences and clusters of foreign
language learning strategies and their relation to
achievement in language situations and tasks,
3) to specify the relation of strategies and achievement in
foreign language at the end of lower secondary (i.e.
compulsory) education.
Gap in the current state of the art
What are the typical strategies and their sequences and clusters
per language skills ? How are they related to achievement?
Theoretical framework
The project is based on an analysis of problems of foreign language learning and
instruction in the Czech Republic. Many pupils do not reach the expected language
level even after many years of learning. The reason is not only the quality of
instruction but also the learning strategies of pupils. More than two thirds of pupils
reported not to be sure how to learn, and showed deficits in learning skills and
strategy use.
The project is based on the current state of learning strategies research and theory
which show that strategies do not exist separately but in sequences and clusters that
have to be described and analysed in terms of their orchestration and quality of use.
The current research also shows a problem of a weak relation between learning
strategies and achievement and suggests therefore to measure strategies at tasks.
Validity threats
We don´t measure learning strategies, but test taking strategies , or precisely
learner strategies => it is usually measured like this, learning strategies are
mirrored in the test taking strategies.
Low information value of templates data. => To use strategy questionnaire for
triangulation. Later, to do interviews with 3 low and 3 high achievers.
Large amount of data for QL analysis of templates (10 per a pupil). => Detailed
analysis only of low and high achievers with complete data.
Risk for data quality = big amount of pupil´s time (2,5 lessons for tasks with
templates, each questionnaire 10 minutes). => Teachers will get results, they got
some presents or money.
Scales of questionnaires have only 3 points. => They are used only as context
information, they are in their original form.
It could (but rarely) happen that the questionnaires about strategies and styles
were distributed before templates, so it could structure the pupils answers in
templates. => We collected this information from teachers.
Data are not anonymous – because we need to be able to interconnect all the
questionnaires and pupils do the tasks during more lessons. => The most
problematic parts (with high social desirability effect) are administrated by
researches.
Methods
10 language tasks from standardised KET test (listening, writing,
reading, speaking), each with a template (pupils 'description of their
strategies at a task)
Standardized questionnaires (learning and achievement motivation,
attitudes towards school subject, learning strategies, teacher´s
strategy support questionnaire)
Unstandardized questionnaire (learning styles, demographical items)
Sample and sampling
Probability sampling of lower secondary schools from South Moravian
region. Full sampling of classes and pupils at the level of schools.
Administrators: researchers and than teachers.
16 schools, 462 pupils at the end of lower secondary education
Data analysis
On the basis of score from the language test tasks and strategy
questionnaire and motivation and style questionnaire… construction
of a typology of learners or just main direction: low and high
achievers.
Qualitative analysis of templates regarding strategy clusters and
sequences only for low and high achievers and their subgroups.
Connecting QN and QL data.
Research questions
1. What are the typical strategy sequences and clusters for different language tasks and situations in the four
basic language skills (reading, writing, listening, and speaking)? What learning strategies and techniques
constitute them?
2. Are there any differences in these strategy clusters and sequences? (based on pupils´ general achievement
level, level of language competence, gender, learning and achievement motivation, attitude towards
language learning, learning style dimensions, level of strategy awareness, and teacher’s class instruction)
3. How is this combination and orchestration of strategies and techniques related to achievement in the
language task with respect to the above mentioned variables?
4. Can we build a typology of learners based on their strategy use and their strategy clusters/sequences? Can
the typology be interpreted in the context of learning styles and cognitive styles theories?
5. Can we distinguish strategies on the basis of their level of effectiveness, i.e. connection to achievement?
ÚKOL 1 (čtení a psaní) Jméno: ……………..………………….
Třída: ……………..………………….
Co která informační cedule (A-H) sděluje (1-5)?
Příklad:
0 This is not a good place to leave your bags. Odpověď: H
Odpovědi napiš sem:
1 ……… 2 ……… 3……… 4……… 5………
Na tomto úkolu jsem pracoval/a zhruba _____ minut.
Language task from test KET
(reading and writing)
Template – pupils write down how they were solving the task (sequences are supported by the design)