Measuring Teaching Quality

Careers Registration and

Learning Gain

14 July 2016

Dr. Bob Gilworth - Director of The Careers Group, University of London

& AGCAS Director of Research

2

Careers Registration

- Employability-related questions included in student registration data collection

- Completed by all new and all re-enrolling students at the start of each year

- Pioneered at University of Leeds in 2012

- Both cross-sectional and longitudinal

- Already scaled and sustainable

3

Career Decidedness (CD)

Readiness to engage with career management- Decide, Plan, Compete

Please select the statement which best represents your current careers position: I am not ready to start thinking about my career yet (Decide) I have no career ideas yet but want to start thinking (Decide) I have some ideas about my career & am ready to start planning (Decide) I have a career in mind & intend to gain relevant work experience (Plan) I know what I want to do but not sure how to get there (Plan) I want to spend a year gaining experience (Plan) I am ready to apply for graduate level / professional opportunities (Compete) I am ready to apply for further study (Compete) I have been applying for opportunities & have not been successful (Compete) I have a job, further study or my own business plan confirmed (Other)

4

Employability Experience (EE)

I have gained work experience through (select all that apply):

a placement year during my degree

a summer internship with an organisation

a vacation internship (not summer) with an organisation

work shadowing

a short placement as part of a University module (e.g. 10 or 20 credit module)

part time work alongside my studies

a holiday job

volunteering

a position of responsibility in a club or society

full time work prior to my course (two years or less)

full time work prior to my course (more than two years)

self-employment / running my own business

I have no work experience to date

5

Benefits of CR

- Current: Live data vs DLHE data six months after graduation- Comprehensive: Extent and nature of students’ work

experience vs limited information from formal placements- Consistent: Same data on all students vs patchy data gathered

through departments- Connected: Links employability data to the student record- Comparable:

• Individual progress (starting points and trajectory) — what support students’ need and when

• Institutional impact —cohorts, departments, backgrounds— where to target scarce resources

6

Visualising the ‘journey’All students

Decide

Plan

Compete

Other

1st year

Decide

Plan

Compete

Other

2nd year

Decide

Plan

Compete

Other

Final year

Decide

Plan

Compete

Other

7

Project partnersInstitution Status

University of Bristol Implemented 2014

University of Exeter Implemented 2014

Goldsmiths, University of London Implemented 2015

King’s College London Implemented 2015

Lancaster University Implemented 2014

Liverpool John Moores University Implemented 2014

Queen Mary, University of London Aiming for implementation 2017

Royal Veterinary College Negotiating implementation, other measures

St George’s University of London Implemented 2015 (variations)

St Mary’s University Pilot from 2015, full implementation 2017

School of Oriental and African Studies Implemented 2015 (non-mandatory)

University College London Aiming for implementation 2016

University of Edinburgh Pilot in 2014, wider roll-out 2015

Ulster University Implemented 2015

8

Primary Research Question

Does Careers Registration provide a viablemethod for obtaining data that provides anindication of learning gain related to work

readiness?

Correlating CR data with existing measures ofwork related learning gain

9

Secondary Research Questions

- What’s the best way to implement CR?

• Statement wording

- Can we use CR to highlight employability needs of different groups?

• Links to WP factors

- Can we use CR to evaluate different employability activities?

• Stakeholder understanding of data

10

Suggested success criteria

1. Three cross-sectional analyses and three partial longitudinal analyses of CR data across three institutions

2. One comparison of CR data with DLHE, one smaller-cohort comparison with other measures of employability, one comparison of CR data with other data such as NSS scores, retention rates or academic record

3. An evaluation of the effectiveness of data collection and the quality of the data and monitoring roll-out of CR within three new institutions

4. An examination of the usefulness of CR data for informing student engagement, needs of specific student groups and collaboration with academic departments and employers

5. Recommendations for the implementation of CR in institutions and a standardised methodology

11

The TEF

Source: Fig 2. TEF assessment framework conceptual model

12

The TEFStudent Outcomes and Learning Gain

Indicative list of additional evidence

Aspect Possible examples of evidence

Student Outcomes and Learning Gain

• Learning gain and distance-travelled by students• Evidence of longer-term employment outcomes and progression

of graduates including into highly-skilled employment• Evidence and impact of initiatives aimed at maximising graduate

employability• Extent of student involvement in enterprise and

entrepreneurship• Impact of initiatives aimed at closing gaps in development,

attainment and progression for students from different backgrounds, in particular those from disadvantaged backgrounds or those who are at greater risk of not achieving positive outcomes.

Source: TEF Technical Consultation for Year Two, May 2016