• 1.What is the Peer Assessment for?

    The Peer Assessment is an application in the IS MU implementing the pedagogical evaluation process, where students evaluate each other. This process is known as "peer assessment" or less precisely "peer review" in the literature.

    The basic difference between the above-mentioned terms lies in the choice of assessors, i.e. the persons responsible for the assessment of the submitted file or files and the elaboration of the evaluation. While in the peer assessment process it is typical that the submitters and assessors are students and often come from the same group of people, in the peer review, the assessors are carefully selected from among experts in the field to review the submitted scientific work in an qualified and impartial manner, ensuring a high degree of relevance. The basic mechanism for collecting and distributing files and assessments is the same for both of the use cases.

  • 2.Why use the Peer Assessment in teaching?

    There are a number of reasons to include peer assessment in teaching. The following list names at least some of them:

    • When preparing their assessments, students can enhance their critical thinking, their ability to objectively assess someone else's work and to be critical in a constructive manner.
    • Students get feedback not only from their teacher, the authority, but also from their colleagues - their criticism can be accepted by students more easily.
    • Assessors can also be students of a completely different course or those from a higher year, for whom it can be part of the preparation for future pedagogical career or the opportunity to improve their other professional competencies.
    • The application allows to specify an individual person as an assessor, therefore the teacher can distribute the assessment of files among his/hers doctoral students or colleagues who do not teach the course.
    • The teacher can easily obtain a larger number of assessments of any number of files, as the number of potential assessors increases with the increasing number of students.
    If students assess other students, they are actively involved in the educational process. Creating a quality assessment naturally requires quality knowledge of the area, which can lead them to further education. When they assess files submitted by their colleagues, they have the opportunity to take a look into ideas of other students who have worked on the same assignment.

  • 3.Where do I find the Peer Assessment in the IS MU?

    The Peer Assessment is a course application, i.e. it is linked to a specific course or courses. The teacher can find it in the Teacher's Notebook.

    Information System Teacher's Notebook [courses] Administrator [in the Peer Assessment section ]

    Students can find it in the Student agenda, Teaching tab.

    Information System Student Teaching Peer Assessment

    A person who is an assessor and at the same time not a student or teacher can find the application on the title page of the IS MU in the E-learning section.

    Information System Peer assessment

  • 4.How do I set up a peer assessment?

    In the application Peer Assessment

    Information System Teacher [course] Administrator [in the Peer assessment section ]
    you need to click on the "Create new" option. The procedure for creating a new peer assessment is divided into five points (or steps):

    1. Name of the peer assessment: Filling in basic data such as name, instructions for submitters, date of publication (i.e. date when the peer assessment is made available to the submitters in the IS MU, not when distribution of files or assessment starts), the settings of sending information e-mails, specification wheather identity of submitters and/or assessors is to be shown, etc.

    2. Submission of files: Creating a new homework vault or selecting from existing ones. When creating a new homework vault, it is needed to set up the date until which students can submit their files. Only those students who have uploaded at least one file into at least one of the selected homework vaults are considered to be submitters.

    3. Selecting assessors: In the simplest case just turn on the "Submitters are assessors" option. Alternatively, assessors can be specified as students of courses or seminar groups, or individually using učo. The teacher can only choose from courses he/she is entitled to. The IS will remember the settings and it will evaluate actual persons during the distribution of files to the assessors. The peer assessment can therefore be set up at the very beginning of the semester.

      If a student submits more than one file (even into the several homework vaults), the assessor will be given all files from this student. In other words, assessors will get "assigned" submitters, not the individual files.

    4. Assessment attribute settings: The teacher has to specify the beginning and end of the assessment and how many times is one submitter to be assessed, or how many files is one assessor to evaluate. In the Adjustment of peer assessment distribution section it is possible to set up the assessment of teacher's files (see question no. 12). All of the settings have to be achieveble at the time of distribution, otherwise the distribution is going to fail. In this step, you can also set a later results publishing date than is the date of assessment end. If the results are transferred to notebooks, they will be made available to students on this date.

    5. Creating the assessment form: The teacher has advanced WYSIWYG (informal abbreviation of "what you see is what you get") assessment form editor. Here, the teacher can specify a number of questions, which then lead the assessor during the process of assessment. Questions can have one or more possible answers, free text, grades and points. For each question, the teacher can determine whether its value is to be transferred to the submitter's notebook.

  • 5.Is it possible to use already created or closed homework vaults?

    Yes, it is. When setting up a peer review, it is possible to create one or more new homework vaults or use the existing ones (even the ones already closed). When using homework vault that already exists and is open for submitting files, it is necessary to ensure the correct setting of deadlines for submission in relation to the beginning of the assessment.

    When using already closed (and therefore filled with files) homework vault, it is advisable to cancel sending the e-mail to the submitters feature (setting in the first step of creating peer assessment), which in this case, is no longer useful.

    Students who submitted their first file only after the distribution of files has taken place (i.e. after the assessment beginning date) are not considered to be submitters. If students are allowed to change their files even after the assessment beginning date, the assessor will always be offered the version of the file that is current at the moment he/she decides to assess it.

  • 6.How will students find out result of their assessments?

    After the results publishing date, submitters will find how they were assessed both in the Peer Assessment application (the role of the submitter) as well as in their notebooks, provided the teacher has set up for the results to be transferred into them.

  • 7.Who can be an assessor of the student's files?

    It can be almost anyone who is in the Information System MU active and logged in. Therefore, the application for the assessors can be also accessed from the title page outside of the student or teacher agenda. Persons who are neither students of the course nor of the seminar group have to be specified using their učo.

  • 8.I don't want students to know who they are being assessed by and who they are assessing

    It is possible to hide the identity of the assessors and of the submitters. The pseudonymity set up will ensure that nowhere in the application the name and ID number of the assessor/submitter is written and the files submitted are renamed so that they cannot be linked to the file name in the homework vault. However, the Peer Assessment application does not delete metadata from the submitted files nor does it check their content. Therefore, if a submitter directly signs his or her work or leaves metadata in the file (e.g., the author of a document in Word), the assessor can still retrieve the identity of the submitter despite the pseudonymity set.

    Teachers always know the identity of submitters and assessors.

    Pseudonymity is set when a new peer assessment is created:

    The Information system Teacher [selected course] (Peer Assessment) Administrator Create new

    or when editing a peer assessment:

    The Information system Teacher [selected course] (Peer Assessment) Administrator Edit

    1 Pseudonymity options are in the extension settings, you need to click on "more".

    2 Individual pseudonymity options to choose from.

  • 9.As the teacher, is it possible for me to see how submitters or assessors are fulfilling their responsibilities?

    Yes, it is. After the distribution of files takes place, the "Overviews" option is made available for your role of administrator. It is displayed after clicking on the peer assessment tile. Lists of submitters and assessors can be found here. It displays summary information on how submitters and assessors fulfill their formal duties given by their roles. These lists can be exported by the teacher and processed further. The teacher can also see, for example, links to the assessments themselves, submitted files and totals of points awarded and/or received.

    The "Submitters" and "Assessors" options can also be used to track how they assess or how they are assessed in read-only mode. All of these options offer the same data, only presented in a different way. The teacher can always see the identity of the assessors and submitters in here, regardless of the setting of the peer assessment and teacher's potential role in it.

    The teacher cannot change the assessment instead of the assessor. However, if the results are transferred to notebooks, e.g. for the purpose of calculating the average and the final classification, the teacher can modify these notebooks directly at his/hers discretion.

  • 10.Is it possible to delete a peer assessment already available to students?

    Yes, it is. The "Delete" option is displayed after clicking on the peer assessment tile (as the administrator). Any erroneous or ongoing peer assessments can be deleted in this way. However, they cannot be deleted after the end of the assessment period. If the peer assessment is already made available to the submitters or if the assessment period is already in progress, then submitters and/or assessors will receive an information e-mail after its cancelation.

    Canceled peer assessments can be found via the "Make a copy" option and then on the "Unsaved" tab. Settings from deleted peer assessments can thus be used as a basis (i.e. template) to create a new one. Similarily, various unsaved versions of peer assessments can be found in here as well.

  • 11.Is it possible not to publish the results of the peer assessment to students?

    No, it is not. Once the assessment period has ended, the peer assessment can no longer be deleted. Publication of results can only be temporarily postponed. However, this setting can only be changed for upcoming or ongoing peer assessments.

  • 12.I want to create the peer assessment in a language other than Czech. How do I set up peer assessment in English, for example?

    There is difference whether all students should have their peer assessment in one (foreign) language, or whether a part of the students should have the same peer assessment in Czech and part in English.

    If all students are to have one language version of the peer assessment, whether in English or in another language, it is sufficient to fill all of the columns (otherwise intended for Czech) in the required language, i.e. columns intended for the main language of the peer assessment, see question no. 4 above. All of the columns, including the assessment form, can also be filled in in the languages using non-latin alphabet. (The numbers in the distribution priority setting or for assigned points have to always be Arabic.) Assessors may use non-latin alphabets as well.

    If part of the students is to have the same peer assessment in Czech and part in English, depending on the language they have switched their IS into, in the first step of setting up peer assessment it is necessary to click on the "+ EN" icon and to fill in all of the columns in English as well. Assessment form in step 5 also has to be filled in both of the langueges.

    Information e-mails to submitters and assessors are sent in the language they have switched the IS into.

  • 13.Can students be assigned files that the teacher uploads to the homework vault for assessment?

    Yes, if this is set in the peer assessment using the "[] Evaluate teacher's files" and "[] by all assessors" attributes (Step 4 Assessment attribute settings). If neither of these two attributes is ticked, files uploaded by teachers into connected homework vaults are ignored during distribution. Also, the teacher cannot upload the file on behalf of the student into the homework vault (e.g. if student sends a file via e-mail).

    If the attributes are ticked in this way:

    then the teacher who uploads (or submits) at least one file to at least one of the linked homework vaults will be considered to be a regular submitter and his/her files will be randomly assigned to assessor according to the distribution priority setting. In this way, it is possible to test how will a random assessor assesses the teacher's files. At the same time, the teacher can try the way how submitting student sees the peer assessment at different stages.

    If the attributes are ticked in this way:

    then assessors will assess files of submitters as specified in the distribution priority setting. And above that, all of the assessors will assess each file of each teacher uploaded into any of linked homework vaults.

    If only teacher's files are to be assessed, the frequency 0 times needs to be set as the distribution priority.

    In this way, for example, the teacher can make available a model task to students.

    The current features of the application are yet to allow to solve the requirement when the teacher wants to distribute x files among the students so that one student always receives exactly one of the teacher's files (since each assessor is always assigned all files from the teacher).

  • 14.Assessors do not have enough time. Can I extend the period for assessment of files?

    It is possible to change the date on which assessment period will end (in step 4 when setting the assessment attributes). This change can be made during the assessment period itself at latest. I.e. when the assessment period has ended, it is no longer possible to extend it. The date can only be changed to the future one and this change is irreversible. At the same time, the date of publication of results needs to be adjusted.

    Other assessment settings cannot be changed after the assessment period itself starts.

  • 15.Is it possible to create multiple peer assessments with the same settings?

    When creating the peer assessment, the teacher can also create it using the "Make a copy" button. The teacher can then choose from the existing peer assessments of the given course (tab "Proposed") as well as from peer assessments in his/hers other courses (tab "By period"). In the copy created in this way, it is always needed to choose a homework vault (Step 2) and fill in assessment-related dates (Step 4). The "Unsaved" tab allows you to use the settings of deleted or unsaved (i.e. work in progress) peer assessments.

  • 16.Is it possible to copy all peer assessments to a new semester together with study materials at the beginning of new term?

    Yes, it is. If the teacher wants to copy all peer assessment from the previous term (or another course), he/she can use the "Copying and sharing study materials" option.

    Information system Teacher [selected course] Copying and sharing study materials

    The target course has to have empty the Study Materials section, so this option is typically used at the beginning of a new semester. After the copying, peer assessments will always be displayed in the target course as incomplete ones.

  • 17.Is it possible to create a peer assessment that is shared by several courses?

    Any peer assessment is shared among all courses that are selected in the Teacher's Notebook when it is being created or modified.

    These courses can be selected in the Teacher's Notebook via the "other courses" option. Afterwards the teacher ticks all courses to be included into the peer assessment, and confirms with the "Selected" button. All peer assessments created or edited (i.e. clicking on "Edit" option and then saving them) using this selection of courses will be shared among them.

    Sharing of the peer assessment among courses cannot be canceled. If sharing is not desired, it is possible to delete the peer assessment and create a new one (i.e. the copy of it) from its settings. But this is not always necessary. Students of the course (in which the peer assessment is located) are considered to be submitters if they have submitted at least one file to at least one of the linked homework vaults (or have the right to do so) and the assessors are determined according the settings in step 3. The rest of the students do not have access to the peer assessment and they do not receive any information e-mails, they can only see it in their Student agenda, in the Teaching tab.

  • 18.What types of files can be assessed? I need students to assess videos

    For peer assessment, it is possible to use all types of files that can be uploaded into the homework vault in the Study Materials. Peer assessment has no other limitations.

    When assessing a video (or audio recording) created by students, the format they use can be problematic if it cannot by played directly in the browser, or if it can be played only on devices with a certain operating system, etc. As a prevention we recommend asking students to convert the recorded video (or audio) to HTML5 format, which can be also used in case of the peer assessment, where it can be played directly in the browser without the need to download. If necessary, this conversion (but not uploading of the file into the homework vault) can also be done by the teacher.

    The procedure of converting to HTML5 format is described in detail in the Help section here. The conversion can be performed for multiple of ticked files at once. It should be taken into account that the conversion does not have to take place immediately, it may take some time.

Provided you have failed to find the information you were searching for, you can contact us at etech(zavináč/atsign)fi(tečka/dot)muni(tečka/dot)cz.