Has the COIL achieved the intended objectives and outcomes? The evidence that students have improved their competences can be obtained from data gathered through students’ self-assessment. Keep in mind that this self-reporting is based on subjectivity and can be valued only as students’ perceived, rather than measured, improvement of competences. Don’t forget to gather data from instructors/teachers as well to learn about the impact and issues from their perspectives. If a more evidence-based approach is required, consider triangulating quantitative with the qualitative data.

High-quality and research-based impact reports include evaluation of the impact of virtual exchange on initial teacher education by Evaluate Group (2019), Erasmus+ Virtual Exchange Impact Report 2020 (Helm and Velden, 2021), or SUNY COIL Stevens Initiative Assessment Final Report (Guth and Helm, 2017). The Evaluate Group (2019) and Erasmus+ Virtual Exchange Impact Report 2020 both evaluate varieties of virtual exchange, including COIL.

The Evaluate Group report involved 25 virtual exchange projects at 34 teacher training institutions from 16 countries and 1,018 students of initial teacher education. The report uses quantitative and qualitative research instruments and analyzes and interprets data on students’ digital, intercultural, and linguistic competence development as well as on the learning experiences of both students and teacher trainers (2019, p. 3). The report particularly evaluates the impact of virtual exchange on student teachers’ digital pedagogical competence, intercultural competence, and foreign language learning. It also researches the experiences of the teachers in the virtual exchanges (ibid, p. 10). The applied research methods and research questions can be recommended if you endeavor an in-depth and longitudinal research of the impact of your COIL. More valuable for your purposes might be the findings in the report that can help you in designing your own COIL and address possible challenges:

• students’ intercultural issues in virtual exchange, including ways of engaging with difference (ibid, pp. 30-34)
• students’ issues and benefits referring to technology use (ibid, pp. 45-50)
• challenges and problems encountered by the teachers in the virtual exchanges (ibid, pp. 66-72) and what the teachers consider important for successful virtual exchange (ibid, pp. 73-78)

Erasmus+ Virtual Exchange Report 2020 (Helm and Velden, 2021) specifies models of virtual exchange (pp. 28-30) and asks a series of inspiring research questions (ibid, pp. 32-34) to evaluate participants attitudinal change, perceived improvement in the 21st century skills, improvement in global competences and activation (ibid, p. 11). Particularly the report’s definition and research of 21st century skills and global competences (ibid, p. 22) is valuable for precising and evaluating COIL. Activation, that is students’ determination to be active and make a change as a result of online collaboration (ibid, p. 64), is also considered a desirable outcome of COIL and should be a part of any COIL evaluation. The research methods include the pre- and post- virtual exchange questionnaire to monitor respondents’ progress (ibid, p. 34) and interviews and focus groups to understand what factors led to students’ improvement (ibid, p. 37). Multiple students’ testimonies illustrate the report’s findings and offer suggestions for successful COIL.

SUNY COIL Stevens Initiative Assessment Final Report (Guth and Helm, 2017) evaluates COILs exclusively and is closest to our goal of evaluating the impact of COIL. This COIL Guide draws on the Report’s definition of the 21 century skills [ch 4] (ibid, pp. 39-40) and is likewise inspired by some of the report’s findings: for example, creating opportunities for students to engage in less formal interactions in COIL that will allow them to learn more about one another (ibid, p. 44), how to address communication issues and the use of technology (ibid, p. 48), and why and how to turn COIL to reflective experiential learning (ibid, p. 49).

The Report utilizes COIL Assessment Model based on sets of reflective questions for students to answer before, during and at the end of their COIL experience (ibid, p. 11). The questions gather largely qualitative data that are analyzed by comprehensive coding of the data and identification of themes (for more see ibid, pp. 15-16). The pre-, mid-, and post-COIL questions inspired this COIL guide and the author’s own evaluation of students work in COIL as well and are therefore quoted verbatim here:

Pre-COIL Prompts

It is recommended that this group of prompts be assigned before COIL introductions and ice-breakers, but after the students have received the course syllabus and the local class has met at least once.

1. The following are the Student Learning Objectives (SLOs) for this COIL collaboration. Do you understand them and how are you ready to work to achieve them? Please comment on each. (50-100 words) *Teachers post your SLOs within this question*
2. How do you think your interaction with students from another country might impact what you learn in this course? (30-50 words)
3. How do you think the way you see and understand the world might change by connecting with students in another country? (30-50 words)
4. How would you describe your cultural background? (30-50 words)
5. What do you want from this COIL experience? (30-50 words)

(Guth and Helm, 2017, p. 10)

Mid-COIL Prompts

Purpose: This set of prompts seeks to draw out the experiences that students are gaining during the COIL Module. It is recommended that these be assigned at the module midpoint. Please encourage your students to add specific evidence and examples where possible.

1 Provide two or more observations about your interactions with your partner(s) as you work with them in the online environment. (30-50 words)

2. Describe how your course has been impacted by connecting with a class from another country. (30-50 words)
3. Now that you have connected with students from another country, how are your views of your partner(s) culture changing? (30-50 words)
4. How do you think that your cultural background impacts the way in which you interpret course content and interact with your partner(s)? (50-100 words)
5. What surprises you about the COIL experience? (30-50 words)

(ibid.)

Post-COIL Prompts

Purpose: This set of prompts seeks to draw out the skills and knowledge your students have gained, what perceptions have changed, and what insights they may have as a result of this experience.

1. Do you believe that you have achieved the student learning objectives of this course? Why or why not? Be specific. (50-100 words)

*Teachers post your SLOs within this question*

2. What was the most important thing you learned from this collaborative course? (30-50 words)
3. Please describe how doing this course collaboratively with international partner(s) impacted your learning experience? (30-50 words)
4. Given your online interactions with students from another country, describe any key changes that occurred in how you view the world? (30-50 words)
5. How did your cultural background influence your understanding of this course material and shape your interaction with your partner(s)? (50-100 words)
6. At this point, in what way might this course and subject, as taught at your university, be a reflection of your culture? (30-50 words)
7. Was there any aspect of this COIL-enhanced course that was stressful in any way? If so, please describe this challenge and what you learned from it? (30-50 words)
8. What do you want to explore further as a result of this connection to students from another country? (50-100 words)

(ibid., pp. 10-11)

Tips for sustaining and promoting COILs:

• Award COIL instructors and students digital certificates to add to resumes

  

  ×
• Publicize COIL collaborations and their impacts on teachers and students.
• Invite instructors and students to tell the story of their COIL collaboration during Erasmus staff week, etc. (Deardorff, 2023, pp. 269-270).