A consistent approach to the evaluation of teaching:a tool for ‘Raising the Bar’

Professor Suzanne CholertonPVC (Learning and Teaching)

What’s included

• Overview of Newcastle University

• Module evaluation in context

• Process and outcomes

• Challenges for staff and students

• Engaging staff and students

• What have we learned?

• What’s next?

3

Newcastle University

• Vision

• We aim to be a world-class, research-intensive university, to deliver teaching and facilitate learning of the highest quality, and to play a leading role in the economic, social and cultural development of the North East of England.

• Structure

• 3 Faculties; 24 Schools; 11 Research Institutes

• Students

• UG - 16,000 (+1000 at overseas campuses)

• PG – 6000 (4000 Taught)

• Staff

• 5300 (+120 at overseas campuses)

Module Evaluation in Context

• Project 2012

– Institutional response to the significant changes in HE

– What do I get in return for my £9k fee?

• Drivers

– Quality assurance of provision

– Reward and recognition

– Students as partners

• Approach

– Consistency

– The use of a single system

– A common set of questions (including staff-specific)

– Migration away from paper to online

– Academic Unit ownership

4

Module evaluation - the process

• 2011/12:

– Development of core questions including staff specific

– Began to re-consider policy Purchase of Evasys

• 2012/13:

– All modules evaluated using core questions

– Piloted in eight schools in Semester 1 and 14 in Semester 2

– Introduced wider range of question sets

– Evaluation of core questions and system by staff though Evasys

• 2013/14:

– EvaSys use a requirement for all modules

– Included staff specific questions

– All campuses involved

– All evaluation online

– Additional questions sets and module specific question facility available

5

6

Fieldtrip Core

Summary

Staff specific

Module specific

3. Fieldtrip modules

Design, Studio or

Performance Core

Summary

Staff specific

Module specific

4. Design, Studio or

performance modules

Module Evaluation Question Structures

Supervision focussed

Clinical CBPS Core

Summary

Staff specific

Module specific

5. Clinical CBPS modules

Independent study Core

Summary

Module specific

2. Independent

Study modules

Supervision focussed

Staff specific

Supervision focussed

Core

Summary

Staff specific

Seminar core

Module specific

Practical sessions core

School staff specific

1. Most modules

School or Subject specific

Group/ Team work core

Compulsory set of centrally owned questions

Optional set of centrally owned questions

School can choose whether to write and use these questions

Module evaluation - the process continued

• 2014/15:

– 1068 modules evaluated in Semester 1

– 695 staff contributors

– 105,387 individual evaluations

– 52,774 responses = average response rate 50% (48% in 2013/14)

Success is dependent upon winning hearts and minds

Challenges for staff

• Change management– System– Access and data use– Standardisation

• Staff concerns about use of online rather than paper evaluations. Common worries include:– Response rates– Electronic interface encourages students

to write inappropriate comments– Which students respond

9

Engaging staff throughout the process

• Involvement of academic and administrative staff at all stages (including sceptics)

• Broad consultation• Treating concerns sensitively • Explain the rationale and

demonstrate the benefits• Work with academic units and

individuals to address issues• Provision of supporting materials

for academic and administrative staff for their use and for use with students

Module evaluation resources

11

Challenges for students

• Barriers to student understanding and engagement with evaluation process- Concerns about anonymity- Lack of knowledge about how feedback is used- ‘Survey fatigue’

• Skills gap: lack of guidance/support for students about how to give constructive feedback

12

Engaging students: central support

• Invitation emails

• ‘You Said, We Did’ webpage

• Video interviews

• Staff Student Committees

• ‘Thinking About My Evaluation’ campaign week, in collaboration with NUSU

13

Online resources

14

Thinking About My Evaluation Campaign

15

Engaging students: good practice in Schools

• Induction lectures

• Shout outs in teaching sessions

• Staff photos on VLE

• Student newsletters

• Discussion forums and text messaging

16

What have we learned?

• That investing in the system gets you only part of the way

• More about culture change than system implementation

• Getting policy right is crucial• Impossible to get core questions agreed by all• Efforts need to be sustained• Collecting staff specific feedback presents

challenges and opportunities• Students need to see their feedback is being

considered

What’s next?

• Continue to support schools with implementation and use of outputs

• Benchmarking and interpretation of scores• Continued work with EvaSys on information

flow and reporting• Increased use of data in promotions and

annual PDR• The use of Evasys for consistency in stage

evaluation • MyImpact (T)

18

THANK YOU

19