Initial consultation event: Timing and publication
-
Upload
higher-education-funding-council-for-england -
Category
Education
-
view
290 -
download
1
Transcript of Initial consultation event: Timing and publication
A feedback survey for taught postgraduatesWorkshop 2: Timing and publication
Catherine Cameron
Senior HE Policy Adviser, HEFCE
Richard Puttock
Head of Data and Management Information, HEFCE
Future Inn, Bristol City Centre
21 September 2017
Session Outline
1. Survey coverage (10 mins)
2. Survey timing (20 mins)
3. Data linking (10 mins)
4. Publication levels (20 mins)
Survey coverage
Initial starting point was whether there were any groups of PGT students who should be excluded
Our proposal
All students on PGT courses which lead to a qualification
Excepting those on integrated Masters
Questions or comments?
Survey timing
Key consideration is to provide representative and meaningful information
Options
• Single survey window
• Post completion survey
• Multi-point survey
• Rolling survey
Survey timing
End date analysis
Whilst June/July and September/October cover most students, there are differences for:
• Particular providers
• Some students on unstructured courses
• Some students studying particular subjects
End month Percentage of students
Jan 3%
Feb 2%
Mar 3%
Apr 2%
May 4%
Jun 12%
Jul 16%
Aug 5%
Sep 34%
Oct 10%
Nov 6%
Dec 3%
Survey timing
Conclusion
• A multi-point or rolling survey is the only way to achieve responses which reflect the majority of a students’ academic experience and a sufficiently high response rate.
• Running the survey at four points in the year should be sufficient to capture a significant majority of the PGT population close to the end of their period of study.
Survey timing - questions
Your questions ….
Our questions
a. What are the challenges for institutions in delivering a multi-point survey?
b. Is there anything we haven’t considered that we should have?
Data linking
• In order to be able to analyse and represent survey responses by student characteristics we must be able to link back to individualised student data
• This will also allow us to link to other datasets: NSS, Graduate Outcomes for analysis
• Data futures – will introduce in-year data collection and potentially allow us to generate target lists
Questions or comments?
Publication
• Intend to consult on some elements of publication approach now and some following successful piloting
• The key things that we think that we will need to consider are:
• Publication routes
• Publication thresholds
• Publication level
• Breakdown of responses
• Benchmarking of responses
• Detailed approach to presentation of information
Publication
Publication routes
Envisage publishing outcomes in similar way to NSS:
• Publishing responses on the Office for Students (OfS) website
• Publishing on the central information source to support decision-making being provided by the Office for Students and the other funding bodies (assuming continuation)
• Publishing an open dataset so that other information providers can make use of this information
• Making responses available to providers through a secure system.
Publication
Publication thresholds
Propose to adopt same thresholds as used for publication of NSS
• At least 10 students
• 50 per cent of sample
Question
Our analysis is based on these thresholds and we propose not to consult on them. Are there any concerns about this?
Publication
Potential coverage
However
Expected gaps for some particular types of provision, particularly at FECs, due to very small cohort sizes
Publication level Estimated coverage (70% response rate)
Estimated number of students included
Course 19% 67%
HECoS CAH level 3 57% 94%
HECoS CAH level 2 70% 98%
Publication
Breakdown of responses
We also need to consider the course characteristics by which the survey responses should be broken down to best meet its purposes
For example, mode of study.
But …. we must be careful not to fragment to an extent where we cannot publish data
Questions
• Are there ways in which is would be essential to break down the data to make it meaningful?
• Are there ways in which it would be desirable to break it down?
PublicationBenchmarking
• We will consider benchmarking as part of the second stage of developing proposals.
• HEFCE is currently leading a fundamental review of the approach to benchmarking used in the UK HE Performance Indicators and the TEF to ensure it meets best practice.
• This will influence our approach to benchmarking of survey outcomes.
Other questions or comments?