Introduction

In November 2022, we worked with the Digital Learning Manager for HSS to recruit eight (8) students to work as Student Digital Champions (SDCs) to get their input into some of our learning technologies.

We specifically looked at the following two topics as part of a pilot project:

1. Q-Review captions and transcripts
2. Technology Enhanced Learning (TEL) Student Inductions

Students were selected from a variety of schools within the faculty, as well as from interdisciplinary programmes to ensure that the student voice was broadly representative.

The Student Digital Champions worked for a total of 71 hours, an average of 9 total hours per student, over a 6-month period ending in May 2023.

We learned so much from the students, and plan to recruit more students in the future to get their input into our TEL resources.

 

Mini project 1: Q-Review Captions

Q-Review captions: Background

Q-Review is the University’s lecture capture system. Upon recordings being published, automatic speech recognition (ASR) generated transcripts and captions are made available to the students, typically 90-93% accurate. These transcripts and captions are not reviewed or edited before being released and data is not available to us to determine their use. Captions can be edited in the Q-Review system and roles could be configured so that users only made changes to the content being reviewed (with no access to any other content or data).

Q-Review captions: Methodology

15 courses were selected for a pilot, across a variety of schools and of differing subject matter, with the aim to:

1. review the accuracy of the ASR transcripts/captions
2. determine the effort required for the captions to be made verbatim

Modules where students might benefit from transcripts/captions being made verbatim were chosen – based on the number of students enrolled on the module and those modules where students enrolled were also registered with DDS.

• The Student Digital Champions (SDCs) were trained to edit and publish changes to transcripts and captions.
• They were allocated a number of recordings, with preference given to 1-2 hour lectures over longer content so that a greater number of modules could be looked at.
• Where possible the SDCs were matched to their subject of study, so they would be more familiar with terminology in order to make edits.
• The SDCs recorded the time spent to edit captions, as well as any challenges they faced – whether technical or as a result of the process itself.
• The SDCss sought feedback from their peers on their use of the captioning and transcript technology. They created a survey, which they circulated to students in the faculty and gathered 160 responses.

 

Q-Review captions: Evaluation

Upon the completion of the captioning process and collation of student feedback, the SDCs contributed to an evaluation session in which we reviewed the data, talked through the project methodology and the students provided feedback and shared their experiences on the project as a whole.

Findings

Of the HSS students surveyed, 80% stated that they utilise Q-Review recordings occasionally or frequently and of those that did, 80% stated that they utilise the captions.

 

Only 10% of respondents stated that they use the captions or transcripts for accessibility purposes, with the majority using captions for note taking (67%) or further understanding material / terminology (63%).

81% of respondents commented that they found ASR captions either somewhat or very helpful, 71% said that they found the transcript/caption accuracy as somewhat or very accurate.

The Q-Review captions’ accuracy was identified as being an issue, with the following factors listed as being challenges:

1. Subject matter – significant differences in the volume of edits required, depending on language used in taught content (e.g. Lecture discussing folklore was audible, but context required)
2. Teachers accents – the captions pick up incorrect words
3. Terminology related to the topic/subject is not understood by the captioning tool (and keywords not made available on the slides)
4. Audio not captured sufficiently through the microphone array
5. Low volume levels, which was often a result of the following factors:
   • Teachers not wearing lapel mics
   • Teachers walking away from the podium mic
   • Conversations in the classroom – noisy (ceiling mic installations making it hard to hear as well as to caption)
   • Conversations in the classroom – inaudible (questions or comments from the audience isn’t picked up clearly)

Updating of the captions in the Q-Review system typically took between 1.5-4x the duration of the recording. (e.g. a 1hr recording took between 90-240 mins to make verbatim), with approximately 10,000 words of content per hour of teaching for lectures.

The SDCs reported that they were able to make the edits easily, but that familiarising themselves with the terminology or having to re-listen to parts where audio was not clear, resulted in edits taking longer.

SDCs reported that when editing captions for subjects that they were studying, they found the exercise assisted them with their understanding of the content, however without having attended the lectures they might not have been able to caption the material as completely.

Observations, based on the students feedback and survey results

Captions are now a part of a student’s life – in other areas outside of University as well, including: TikTok, YouTube and other video platforms, with c. 80% using captions by default.

Many students turn captions on by default on Q-Review recordings, even if they don’t have any specific accessibility needs.

Students have come to rely on captions, and this is especially useful when revising for exams or completing assignments – as they can search the captions for keywords and jump to that section of the recording.

 

 

Recommendations:

1. Caption edits to be carried out to make verbatim for modules where there are students with acute accessibility needs
2. Glossary of terms should be provided with lecture slides/notes to aid student familiarity of terms and comprehension of captions
3. Lapel mic / neck mic use to ensure best quality of audio and captions is made available to learners
4. Reminder to students who are attending the lecture in-person to refrain from chatting for lecture recording quality purposes

—

Mini project 2: TEL New Student Inductions

TEL student inductions: Project Background

Currently, the TEL team deliver approximately 40-50 induction sessions for new students during Welcome Week in early September and January each year. These student induction sessions are attended by over 1,500 new students to QM.

We discuss key TEL technology that students will use during their studies at QM, some of which include:

1. The online learning environment, QMplus – how to locate their modules and complete any online coursework or assessment activities
2. How to access lecture recordings via Q-Review, and how this works at QM
3. How to access Zoom or Ms Teams for online classes

These inductions are 45-60 minutes in a lab on campus. There is also an online QMplus for students course area, where students can revisit what was discussed at the induction and go through additional information.

The induction sessions have been developed and delivered by members of the TEL team in the same general format for over 10 years.

The Task

Student Digital Champions were tasked with:

1. Reviewing the current TEL student inductions, including the online QMplus for students induction area.
2. Investigating other University new student inductions
3. Providing suggestions for improving the student inductions

 

 

Students Survey

The students put together a survey and received 160 reponses.

Key findings:

Attendance

Of the 160 respondents, 59% indicated that they attended a TEL student induction, with the remaining 41% indicating that they either did not attend, or do not remember whether or not they attended (25% and 16% respectively)

Usefulness

When asked whether the induction sessions were useful or not 82% of students replied that yes, the induction talks were useful.

The aspects highlighted as most helpful included the following;

‘Everything was explored by using a step by step process. While they there were showing us the steps, we got a chance to do it at the same time which was practical.’

‘Time to try it out for ourselves with staff available to support’

‘visual examples and good explanations’

‘the recording feature as well as the transcript option seemed most useful to me’

‘How helpful both instructors were, they reassured me because I was very confused about how it works’

‘Simplicity of the induction talk to not overwhelm the audience.’

As for the reasons why students did not find the induction talks useful, there were the following comments;

‘It was a long session for a straightforward process’

‘Whilst it was helpful, there was a lot to remember and quite a bit of the layout, etc on QMPlus, etc in the tutorial did not match what was on my homepage, QMPlus etc.

‘Pretty self-explanatory on the website itself’

‘It felt rushed due to the limited time slot the session ran, and I had to learn to navigate through different sections of QMPlus as I was doing the course’

‘I do not take in information well that way.’

 

Difficulties

Students were asked to indicate if they had any difficulties with the the learning technologies at QM.

The results were as shown:

Positively, the most popular answer was no difficulties but as seen in the above chart, the aspect that most struggled with was accessing grades and feedback via QMPlus, closely followed by navigating QMplus.

Other learning technologies that students cited as having difficulties with included:

1. MySis
2. The wifi
3. Module pages being difficult to navigate
4. Printing services
5. QReview- incorrect transcription
6. Attendance records for each course
7. Reading material difficult to access in VLE books through QMplus
8. Library website is confusing
9. Difficulties over which username/password to use for different areas
10. Accessing MySis account
11. Issues trying to download Alternative Formats of pdfs/documents uploaded to QM+

Where students have gone to find help

• Most students cited their friends as the primary place they went to get help
• Many students also raised a ticket via the IT Helpdesk or asked a tutor

 

Students were asked to make suggestions to help to improve student induction to QMplus and other learning technologies. the following was feedback left in the open comments:

1. “Make induction talks available online (or recordings of the talks)”
2. “Have super simple way of having instructions online. Maybe YouTube video and then have links on QM or emailed to us so we can have access”
3. “- Easy email access to pdf file with the instructions on how to navigate QMplus”
4. “i think accessing feedback should be easier because at times, marks will say they’re available when in reality they’re not. i think it would be easier if marks always appeared in one place, i know sometimes in law, marks appear in qmplus, turnitin, or merely in a turnitin comment which can be confusing. i also know a few of my friends have had some issues with submission can which can be very stressful sometimes.”
5. “Please bring back the assignment tab where you could see all your assignments and their due dates in one section.”
6. “Make it a mandatory process to ensure all students are informed about the features of the platform”
7. “Make it more user friendly The style of QMplus that was used in academic years 20/21 and 21/22 was more user friendly and more practical because of the menu and the tabs which you could customise”
8. “Make the interface less cluttered and easier to navigate. Right now, too many information is displayed and the design is not intuitive at all.”
9. “Live chat feature”
10. “Maybe it should be more practical rather than listening to tutor.”
11. “Create a custom colour change or tint of background for dyslexic pupils so that its easier to read than the current format”

 

Conclusion and Next Steps

Due to the success of the project and the feedback received from the student digital champions about the TEL student inductions, the TEL Team have now secured funding to recruit 3 students to be a part of a summer project to revamp the TEL student inductions. We will recruit a further 5 students, making a total of 8, to co-deliver the TEL student inductions to new students in September 2023.

This will be the first time that students have been involved with co-creating the student induction content for TEL applications, and the first time that they will be tasked with delivering the TEL student inductions. New QM students will be able to hear first-hand from more senior students about our technologies along with some tips such as ‘what they wish they had known when they were new QM students’.

3 students have been recruited and will begin work on updating TEL student induction in June 2023.

---

This report was written by:

• Alysa Bramble (TEL Training and Support Manager)
• Richard Chantler (HSS Digital Learning Manager)
• Ella Hirmasto (Student Digital Champion Team Leader)