Brief summary

A peer assessment strategy has been introduced to the module EBU6305 Interactive Media Design and Production where each student submits a prototype of their product to QMPlus for their peers to review and assess. Each student will receive a submission mark which is an averaged of all received marks, as well as an assessment mark awarded by the QMPlus system to reflect the degree of fairness in the marking process. The assessment feedback has been perceived to be useful and the workshop event is successful.

Background & Context

In EBU6305 Interactive Media Design and Production, students need to submit a web-based artefact that involves heavy user interaction, where user is the focus in the design and production process. Due to the time limit, it would not be realistic for students to apply for ethics approval and involve human participants in the testing process. However, another key learning outcome in the module has led to a greater opportunity that combines two aims in one occasion – there is a learning outcome for students to conduct usability analysis independently via a heuristic approach. By having the opportunity to review each other’s work,  students are not only able to provide valuable feedback to the product designer in the perspective of users but also practice independent evaluation.

Peer assessment presents several advantages for the module. First, the evaluation must be based on facts and evidence, and the analysis should avoid subjective opinions and follow evidence-based practice. These disciplinary skills can only be acquired via ample exercises, and the peer assessment workshop of this module has provided students an outlet to practise these skills. Second, peer assessment provides an excellent opportunity for them to conduct independent and unbiased evaluation whilst learning the success and/or mistakes from others. Third, students will get the chance to familiarise even more with assessment culture. In order to properly assess the work of their peers, students need to have a good understanding of the assessment criteria and the assignment task, both of which promote a deeper approach to learning.

Beyond the benefits provided by peer assessment, it also comes with certain challenges. As such, the peer assessment component is kept at a relatively modest 10 marks of the coursework (i.e. weighed as 2.5% of the module assessment) to ensure it does not result in a major impact on students’ final scores. The element of chance usually results in only minimum variations, which will not impact the overall summative result to any meaningful extent. The practical experience that students gained in this process represents an invaluable element throughout course delivery and assessment.

Methodology

The peer assessment underwent the following process:

1. Preparation: The Module Organiser (MO) explains the rationale and the procedure of peer assessment workshop to the students during lectures.
2. Setup Phase: The MO configures all necessary fields of the workshop on QMPlus with instruction for students. Once ready, the workshop can be switched to the Submission Phase:
3. Submission Phase: Each student must make an individual submission of their prototype before a given deadline (i.e. submission deadline). After the submission deadline is passed, the MO switches the workshop to the Assessment Phase.
4. Assessment Phase: During the Assessment Phase, students need to review and assess other other’s work whilst also having their own work being assessed by others. This needs to be completed before the Assessment Deadline.
5. Evaluation Phase: After the Assessment Deadline, the MO switches the workshop to the Evaluation Phase. Marks for each student are calculated and visible on the system.
6. Close of the Workshop. The MO may give conclusive remarks to students about the workshop.

To create a workshop, simply choose “Add an activity or resource” on QMPlus, then choose “Workshop”. There are links at the bottom of this page on how to set up a Workshop in QMplus .

Outcome

Each student receives two marks for the workshop:

• Submission mark: The mark that others have awarded to the student. This is usually calculated as an average mark of all received marks.
• Assessment mark: The mark to judge how the student has assessed others. The built-in algorithm from Moodle is used for the calculation of marks, which is based on a Comparison with the best assessment scheme to work out a hypothetical absolutely fair assessment.

It is worth noting that modules involving group work will need extra attention. Each student needs to make an individual submission in order to participate in peer assessment, which means they may receive different submission marks. If each student from the same group makes identical submissions (as the case of this module), then marks can be manually averaged among all members within the same group to ensure fairness in the evaluation phase.

If by chance a student is assigned with their own work to assess, then students should be made aware that it is neither an advantage nor a disadvantage, but a good opportunity for them further to exercise objective and unbiased evaluation. Again, this should be explained in the briefing lecture prior to the peer assessment.

The result of students’ active involvement in the assessment process has provided the opportunity for them to compare and discuss the coursework, as well as to gain knowledge and develop a greater understanding of the assignment content and assessment process. Some students regarded peer assessment as an incentive to perform, while others saw it as a potentially subjective system that cannot guarantee fairness. The MO should respond to these criticisms with academic guidance: there is a built-in element of chance and everybody faces the same situation. While no system of peer assessment can reliably guarantee that every student receives what they may deem a ‘fair’ judgement, students should regard the process as a unique opportunity to receive wider peer criticism of their work. Students should also not assume any ‘low’ mark they receive is not a true reflection of their work as the assessor may have provided genuine insights that others have missed. Indeed, such a process depicts the reality of working in industry and the real world, where a product will never receive entirely positive reviews. The process allows students to experience this in a realistic manner and gain the opportunity to take on board valid criticisms raised by not only module tutors but the wider peer group.

Key Points for Effective Practice

• Brief students in advance about the purpose, marking, and the process of the peer assessment.
• Give clear instructions to students on the setup page of the workshop for each phase.
• Make sure to switch to the next phase at the end of each phase.

Links to further resources or materials

• Adding a Workshop Activity: https://elearning.qmul.ac.uk/guide/adding-a-workshop-activity/
• Workshop Settings on Moodle: https://docs.moodle.org/38/en/Workshop_settings