Contributor: Dr Heidrun Bien
Michael Parkes (E-Learning Technologist)
Department: Centre for Psychiatry

Brief summary

The E-Learning Production Scheme enabled us to develop a new, engaging, QMplus embedded tool to support our students learning about Systematic Reviews. Divided into sub-topics, students are able to quickly access specific content and directly test their understanding. Each topic contains a video, slides and an optional tool presenting 4-7 interactive quiz questions of various formats, offering automatic feedback and unlimited re-takes. Preliminary feedback has been very positive and the tool is ready to be trialled with our full new cohort of students starting in 17/18.

Background

In the MSc Programme Mental Health, we use QMplus to give students (Onsite and Distance Learning) access to a range of materials to study and revise. For lecture topics, this typically includes presentation slides, pre-recorded videos, readings, worksheets, etc.

Crucially, our students come from diverse backgrounds, not only culturally, and in terms of education systems but also with respect to their pre-knowledge. It is important we get everyone up to speed in the first Module in core topics such as introduction to systematic reviews, quantitative and qualitative methods. The earlier and the better students master these topics, the more they benefit throughout their studies, and the more likely they are to complete the programme successfully.

Learning about research methods can initially feel abstract and dry before it gradually improves once students start using the methods hands-on. To improve this, I was looking for a different way to teach methods, early on allowing and encouraging students to engage with the material more directly. The experience should be motivating, informative, and risk-free (i.e. making available a tool that is optional and unmarked. And it should be easily accessible and, from a lecturer point of view, easily editable/extendable in the future should the need arise.

We selected the core topic Systematic Reviews and decided to break the instructional material into sub- sections, each accompanied by an optional interactive learning tool. A mixture of question types and the option of immediate automatic feedback plus unlimited re-takes should allow students to test their understanding, and learn while actively and playfully engaging with the topic.

Funded by the E-Learning Production Scheme, a first tool (on Systematic Reviews) was to be developed in early 2017, to be made available to the first new cohort of MSc students in Module 1 of the academic year 17/18. The experience of creating this initial tool and, crucially, feedback on its effectiveness from both students and staff could inform potential future initiatives.

Methodology

Four people helped shape this project;

• myself – as lecturer, project applicant and lead, focusing on the academic content and pedagogic approach;
• Jade Mille – who was recruited specifically for this project as E-Learning Student Assistant and who created all quizzes;
• Michael Parkes – as internal E-Learning Technologist who provided ideas, feedback and support;
• Manoj Singh – who provided advice and support as contact person from the E-Learning Unit.

Here is the process we followed –

1. Review original lecture materials on Systematic Reviews & and divide into sub-topics (6).
2. Create a 6-topic template on QMplus , insert graphics (where possible taken from lecture slides)
3. Within each topic, create Moodle ‘Tab Display’(Presentation, Video, Quiz)
4. Divide original lecture video and ppt into segments, insert under respective topic/tab.
5. Study available types of quizzes and setting, consider strengths and weaknesses (e.g., open questions difficult to mark automatically; immediate feedback)
6. Draft questions, create question bank and quizzes
7. Several rounds of testing and improving further (accuracy, usability, experience)
   1. internally (several rounds)
   2. ‘externally’ from a volunteering current student
8. Present project at the SMD E-Learning Showcase (watch the presentation here)
9. In the new academic year (2017/18): Make the tool available to our new cohort of students, collect feedback (from both students and tutors) on experience and learning outcomes

As an MSc student in Computer Science, the project student assistant did not have a prior knowledge of the topic systematic reviews. To be able to come up with a set of draft questions, she needed to not only get some training in the technical side of QMplus embedding of tools, but also get an understanding of the subject area Systematic Reviews, and study the range and advantages/disadvantages of different question types available.

Overall, we spent more time than initially anticipated on developing questions we were happy to use in the final version. For example, multiple choice questions are only as good as the incorrect choices are. Open answer questions, despite being much liked by the student assistant, were eventually excluded because subtle deviances in phrasing would simply create too many false errors in automatic marking.

Outcome

Within the scope of the project, we were able to achieve what we had intended: a now ready-to-be-trialled QMplus embedded tool which presents the core method ‘Systematic Review’ in a more accessible and engaging way. Divided into sub-topics, students will be able to quickly access and revise specific content. Each sub-topic is accompanied by an optional tool presenting 4-7 interactive questions of various formats, offering automatic feedback and unlimited re-takes.

With the start of the upcoming academic year 17/18 we will be able to test the tool properly with a full new cohort of students. Feedback collected so far from colleagues and a volunteering current student was helpful in shaping the project and promising as to its effectiveness. In particular, the use and mix of interactive questions types (e.g. drag and drop matching / ordering / gap fill), the option of immediate feedback and unlimited re-takes seem to be much liked.

Advice for others

I would encourage anyone who is tempted to develop interactive learning tools within QMplus to give it go; especially if you already have some material / a collection of questions to start with. The technical side is not too difficult to acquire, instructions are clear and support is available, and with a good variety of options and settings tools can be designed for various purposes.

As we could not offer the student assistant a computer place within out institute, she worked on the project remotely. As the E-Learning Unit needed the project to be completed within a set time frame tight time, it was important to ensure effective communication (a quick turnaround of emails, weekly signing off of time sheets etc.) and I would advise to pre-schedule a number of meetings.