During the academic year 2012/13, QMplus was introduced into all three academic faculties at Queen Mary University of London under a project called QMplus phase I. This phase saw QMplus used within all taught modules in the Faculty of Humanities and Social Sciences (HSS), all postgraduate taught and distance programmes in the School of Medicine and Dentistry (SMD), and all taught programmes in the Institute of Dentistry.
In the academic year 2013/14, QMplus was introduced into the remaining schools and departments of the college including all schools in the Faculty of Science & Engineering, the 5 year Undergraduate Medical programme (MBBS) and the newly created Joint Programme at Nanchang University in China. This second part of the project was called QMplus phase II. Thus, for the first time since the introduction of a Virtual Learning Environment (VLE) to Queen Mary all undergraduate and taught postgraduate students were able to access an online module /course to complement their face-to-face teaching. This had been the ambition of the strategic plan 2010-15.
The purpose of carrying out the 2013/14 student survey was to gauge what are arguably the most important measures of the effectiveness of a new system:
- What do the students who have been using it to study think of it?
- Has it helped enrich and enable their learning?
- Have we got the balance of design and usability right?
- Do students feel that their experience has improved by using the tool?
- Have the changes made after the first year of operation enhanced student learning?
This survey was administered over six weeks from mid-May to the end of June 2014. In total we received 780 responses which compares to 542 responses in the 2012/13 QMplus survey. This represents a response rate of around 4.5% of the total eligible headcount[1].
You can read the full text of the Executive summary on this page or download the Full Report and the Executive Summary by following the links below.
- Download a copy of the full report: QMplus 2013_14 Student Survey [PDF]
- Download a copy of the Executive Summary: QMplus 2013/14 Student Survey: Executive Summary [PDF]
- The QMplus 2012/13 Student Survey
Key Findings
Orientation and access
The results in this section demonstrate that the balance of approaches we have employed for orientating students to the learning environment is about right. On the whole self-directed methods have proved most useful, with students learning as they go along or accessing handouts or online material available through QMplus itself. This year we have seen an increase in the development of handouts on QMplus at School level and these have proved popular too.
63% of respondents (485) are accessing QMplus at least once a day up from 58% in 2012/13. Of those, 37% are accessing QMplus more than once a day (285 respondents) up 6% from the previous year. We also see that the vast majority of students are accessing QMplus at least once a week (93%).
The results of this year’s survey provide a wealth of information about how, when and where students are accessing our VLE. We learn that mobile device usage has grown compared to last year particularly for off-campus access. It is no longer possible to make assumptions about standard access methods and standard browsers or operating systems as the landscape is so diverse and varies depending on context. Statistics across all device types showed that more than 50% of respondents are using mobile devices on campus, at home and on the move.
The move to mobile devices to access the VLE can also be viewed as a great pedagogical opportunity for educators. Mobile devices can more easily be integrated into teaching experiences whether in labs, seminar rooms or out in the field and offer a host of tools that can be employed by students when working on coursework or developing e-portfolios.
More detail can be found in section 7 of the report.
Supporting student learning
Teaching materials
There is an overall trend of increased awareness across all categories of teaching material i.e. more respondents had encountered them or knew the y were there in this survey compared to the last.
Perhaps the most important message coming out of this section is that the core teaching materials made available through QMplus i.e. lecture notes and slides, reading lists, links to external websites and access to videos of lectures (Q-Review) are all seen as useful by students, in fact in the case of notes and slides 67% found them very useful.
There were, however, a lot of comments in the optional questions about the consistency, quality and provision of those teaching materials.
Respondents did not rate links to multimedia content to be as useful as the other teaching materials investigated, though many respondents had not encountered these kinds of content at all. This may be due to the fact that curating a set of multimedia materials and linking them to key areas of the curriculum takes time and energy, it may be due to the lack of quality external resources worth linking to or it might be a question of awareness of the ease with which resources such as this can be included in QMplus. Many students told us how useful Q-Review was (“Q-Review is very handy”) though we also received a substantial number of comments about reliability and quality of the provision, as well as a desire for more of it, and making its availability more prominent in the module layout.
More detail can be found in section 8.2 of the report .
Assessment and Feedback Tools, Methods and Approaches
Assessment and feedback are two of the most important aspects of the student learning experience and if we can get them right in QMplus then we can make a huge impact on the way students perceive their experience at Queen Mary. There are lots of variables that make up the lifecycle of assessment and the VLE is only one part of it. Every school, department and institute has their own processes and finding alignment can be tricky.
Assignment submission in QMplus proved useful for most, but many students commented about the lack of assignment receipts, the mixture of buttons on screen or the lack of information about what to do if something went wrong.
Assessment information is rated as most useful in this section, but students are also looking for this to be more up to date, they would like to have links to worked solutions or past exam papers and they would like that consistently across modules.
Grades and feedback were widely commented on too. In this section they have been rated as useful for the majority of students but what they are also asking for is school wide decisions about where the grades are going to live: QMplus? MYSIS? Bespoke systems? They are also asking for the grades to be more accessible on the module homepage. As far as feedback is concerned there is a call for it to be provided, provided more consistently and for students to be notified when it is available.
Given the complexity of assessment, we recommend that we do not carry out any significant development of assessment and feedback tools in QMplus prior to the college carrying out a full examination of the requirements of assessment and whether current systems for supporting those processes are fit for purpose. When we understand what is required in a clear and consistent fashion we should align our e-learning systems to match those needs.
More detail can be found in section 8.3 of the report.
Groups & Portfolios
We are still not seeing as much use of the Groups & Portfolios area (Mahara) as we might have expected. In fact the survey results suggest that it is the part of the system that students are requested to use the least in their studies. This may be due to lack of awareness of its potential to support teaching and learning and it may also be the fault of the software itself i.e. lacking particular functionality.
More detail can be found in section 8.4 of the report.
Collaborative tools
Like the Groups & Portfolios area, the collaborative tools in QMplus are not being used as widely as we might have thought. Building collaborative activities in QMplus that work can take considerable time and effort. When they work they can be particularly enjoyable for all involved. A persistent group of respondents simply do not know about some tools like blogs, wikis, messaging and peer review capabilities.
Given this, it is pleasing to see that forums are being used and perceived as useful by 37% of respondents, but it may also be considered somewhat surprising that forum use is so low. Asynchronous discussion is one of the bedrocks of blended teaching viewed by many as a relatively easy way to extend the classroom conversation into the virtual space, include unheard voices in the classroom, and allow for more thoughtful response to questions.
Students should be free to choose whatever communication channel they like to chat and email friends. However, there may be specific learning contexts where communication between classmates, group members or between a student and their personal tutor or module lead should be done in a password protected environment that is close to the materials being discussed, free from advertising or more ‘private’. For this reason we must ensure that the tools provided are fit for purpose and we should continue to inform users of their presence and how to use them.
More detail can be found in section 8.5 of the report.
Technical support
The results in this section overall suggest that QMplus is becoming more reliable and that fewer respondents are seeking out help from any of the support channels we investigated. When they do need help, the informal channels of asking friends, colleagues or their module convenor, as well as the online help are all preferred and seen as being useful. When help is required and the preferred channels are not available, or they cannot resolve the problem themselves, then the more formal support provided by the Helpdesk is sought with face-to-face or email support considered to be the most useful, compared with the Helpdesk telephone line.
More detail can be found in section 9 of the report.
Look & Feel
The results from this section do suggest that in terms of usefulness the various components that make up the design of most module templates rate highly. In fact, the responses received in this section rate the features higher in terms of usefulness than any other question we asked. However the ability to use QMplus on mobile devices was a persistent issue with difficulties in downloading files and viewing content expressed by many.
Landing pages were a new innovation for QMplus in the 2013/14 academic year. This survey is the first opportunity to evaluate their effectiveness in delivering a School-level ‘identity’ and information area to the otherwise ‘institutional’ and ‘faculty’ look of QMplus.
Clearly some students struggled to see the usefulness of the landing page and would have preferred a direct route to their learning materials. This was compounded by the fact that for many the page never seemed to change and therefore felt redundant. Other students were more enthusiastic but wanted to see richer, more stimulating and up-to-date content including such things as news and information about what is happening in the school, careers advice and job support etc.
More detail can be found in section 10 of the report
Usability
It is pleasing to see that students continue to find it easy to use the core QMplus functionality required on a daily basis i.e. logging in, accessing and downloading files, getting news and announcements etc. In fact only two of the features we explored (Accessing grades and feedback and navigating around) were difficult for more than 14% of respondents. That said, four of the features and activities that we asked about had not been tried at all by 75% of respondents and this was unchanged from the first year of QMplus use.
On a more positive note, progress has been made in a number of areas and more students are encountering a richer variety of learning materials in QMplus including Q-Review lectures, quizzes and audio and video content.
More detail can be found in section 11 of the report
Key Recommendations & Completed actions
Key Recommendations
- Consider integrating a standardised question about QMplus into Module evaluations across the college.
- Develop an online ‘Welcome pack’ for students.
- Find out more about how and why students are using their mobile devices by running focus groups possibly through the student union.
- Publicise the availability and potential value of the QMplus Archive to staff and students.
- Showcase best practice in the use of video and audio in teaching, particularly within QMplus.
- Implement a college-wide evaluation of assessment & feedback with a view to aligning QMplus tools more effectively.
- Profile the benefits of using QMplus Groups & Portfolios (Mahara) for such things as reflecting in an e-portfolio, sharing work with colleagues and supporting group work and interaction.
- Develop a good practice guide for staff in the use of Discussion forums.
- Surface KPIs for the IT Helpdesk based on speed and quality of responses.
- Provide best practice guidelines on designing QMplus module/course areas.
- Communicate survey results to the timetable team in IT services.
- Enhance and expand our guidance and support around advanced features of QMplus including the development of peer assessment and use of discussion forums.
- On the next upgrade work with our theme designers to ensure navigation, usability and prominence is given to those features deemed important by QMplus users.
Completed Actions
- Bespoke workshop or training sessions now being offered to schools, tailored to specific needs.
- QMplus Phase III project is working on making the Mahara software mobile friendly delivery June 2015.
- The ‘Module info block’ is now easier to edit and now takes contact information from the QMplus profile of the module convenor automatically where it exists.
- Two video recording booths (Mile-End and Whitechapel) to be launched in spring 2015 enabling easy production of high quality content.
- Currently developing enhanced integration of grades between QMplus and MySIS to be implemented in March release to QMplus.
- Developed a suite of video and text-based support materials for students showing them how to take advantage of the key features of Mahara.
- The new ELU website has a guide for staff on enhancing communication.
- Live online support being piloted in Spring 2015.
- Implementing a ‘mobile friendly Mahara’ theme as part of QMplus phase III (June 2015).
- An optional news ‘ticker’ allowing schools to promote local news and the capability of adding a Facebook group feed was added to the functionality in December 2014.
- Development of a 4E framework to assist in the development of e-learning around the college.
- Book a Learning Technologist scheme launched.
Conclusion
The 2013/14 QMplus survey has provided an extremely valuable and informative snapshot of the primary tool provided for e-learning at Queen Mary and as such will help guide its development over the forthcoming year.
780 students not only completed the mandatory questions but also spent a considerable amount of time providing detailed commentary, including an incredible 607 comments on aspects of QMplus they used on mobile devices. We had clearly touched a nerve! It is heartening to see that level of engagement so late in the year which suggests that just as students have told us they are using these tools on a daily basis, they also feel passionately about them as we do and are keen to work with us on ensuring that they are fit for purpose.
Students have commented for two years in a row now that they would like to see better and more consistent use of QMplus across their modules. Whilst we must celebrate the achievements we have made in extending VLE usage to all corners of Queen Mary teaching, we must keep our eye on the way the system is being used with a view to maintaining standards and enhancing what we do.
Improved Awareness
The most pleasing trend that has emerged from this year’s survey is the clear and significant increase in awareness of the tools and functionality available within QMplus compared to the first year of its rollout. Of particular note is the increase in awareness of the tools within the Mahara – Groups and Portfolios area. On the other hand, almost 50% of survey respondents had not used an online discussion forum, often considered a basic blended learning tool, so we still have some work to do in demonstrating to academic staff ways in which the system can be used to enhance learning and teaching.
Consistency
It’s one thing to have awareness of tools and functionality but a greater leap to actually engage with these tools and incorporate them into teaching and learning. In the survey this year, students indicated that they didn’t just want to see stimulating online modules but they also wanted those modules to be designed and organised in consistent ways. If you are going to use Q-Review then use it all the time and make sure the lectures have been captured properly and are properly signposted in the course; if you are going to post grades and feedback via digital channels then post them for all assessments across all modules in the same way and let the students know at the beginning of the year the approach you are going to take.
The 2013/14 survey has revealed that students are happy with QMplus when it is delivering course content but less so when it is being used as a communication tool. Emails received from announcement areas in QMplus are seen as useful but messaging, groups and other ‘social’ tools are either not known about or perceived as clunky or not fit for purpose.
There was a cry from the trenches that consistency in schools and policies are lacking:
“Not all lecturers are using the system fully. e.g. some lecturers put assessment details, revision slides and lots of other useful information on there whilst others put up a bare minimum of lecture slides.”
The survey has shown us how valuable it is to have a school strategy in place for using QMplus and to publicise this strategy as widely as possible. This strategy might be prescriptive about the types of content required in a course area, the layout of learning materials, provision of grades and feedback and the configuration of assignments. Once a strategy is in place then it can be demonstrated to students across all levels at the beginning of the semester so that expectations are understood by all parties.
Continuous Improvement
Students have indicated that we should not stand still, and we haven’t. Since the 2013/14 survey closed, we have been working hard on improving and expanding the e-learning offerings to the college. This year as part of the QMplus Phase III project, we are working to provide an improved communication platform that meets the needs of students when they require the privacy and security that a college-supported system can bring. The lecture capture system (Q-Review) was expanded and upgraded in the summer of 2014, bringing HTML5 streaming capability to mobile and international audiences as well as better and more comprehensive analytics. Simultaneously, HSS has adopted an opt-out policy for the use of Q-Review across all schools with a subsequent leap in the volume of lectures being captured. Undergraduate medicine was already doing this and S & E may follow in 2015.
Video streaming capability is being further expanded in the near future with the launch of a college-wide media server (QMplus Media). This will be an effective but easy-to-use platform for staff and students alike to upload and store video content through QMplus. We are also launching two purpose-built video recording studios for staff to capture high quality video and/or handwriting for incorporation into their online teaching.
Looking Forwards
As we continue to enhance QMplus we need to ensure that all staff involved with using the system are kept up to date with changes and enhancements. Many staff will only have engaged in formal training during the migration into QMplus, and for some that is almost 3 years ago; a lot has changed in 3 years. The E-Learning Unit must continue to provide opportunities for basic training in the use of the tools within QMplus as well as the more advanced capabilities and we must put particular focus on the opportunities these tools can provide to extend and enhance the excellent teaching that is already taking place.
Finally, an important approach will be to work with faculty e-learning user groups and School administration and management teams to help develop local strategies that support disciplinary pedagogical needs as well as school-wide priorities. At QMUL we can and should continue to enhance our e-learning and teaching through both ambitious strategies (e.g. SETLA[2] and ITTP[3]) and practical implementations of the spirit of these strategies. The E-learning Unit has chosen a 4E enhancement framework that will help to stimulate conversations across the college in this area. With this framework, continued engagement with the QMUL community, and a commitment to improving the technical infrastructure, we will deliver QMplus as the effective online learning environment for all Queen Mary students.
[1] Data source: Queen Mary Strategic Planning Office statistics: http://qm-web.planning.qmul.ac.uk/FactsAndFigures/StudentStatistics/index.html
[2] Student Experience, Learning, Teaching and Assessment Strategy
[3] IT Transformation Programme
- Download a copy of the full report: QMplus 2013_14 Student Survey [PDF]
- Download a copy of the Executive Summary: QMplus 2013/14 Student Survey: Executive Summary [PDF]
- The QMplus 2012/13 Student Survey
