Peer feedback in formative assessment to aid learning:

25
Peer feedback in formative assessment to aid learning: Annie Cushing Angela Hall Olwyn Westwood Centre for Medical Education s CETL 2008

description

Peer feedback in formative assessment to aid learning:. Annie Cushing Angela Hall Olwyn Westwood Centre for Medical Education. s. CETL 2008. s. CETL 2008. Drivers. Professional Skill – Multisource Feedback as part of professional practice NSS – students want feedback - PowerPoint PPT Presentation

Transcript of Peer feedback in formative assessment to aid learning:

Page 1: Peer feedback in formative assessment to aid learning:

Peer feedback in formative assessment to aid learning:

Annie CushingAngela Hall

Olwyn Westwood

Centre for Medical Educations

CETL 2008

Page 2: Peer feedback in formative assessment to aid learning:

Drivers

• Professional Skill – Multisource Feedback as part of professional practice

• NSS – students want feedback

• Formative feedback as a support for learning

• Large student numbers

• Proximity of practice/feedback/practice

s

CETL 2008

Page 3: Peer feedback in formative assessment to aid learning:

Study 1 in Nov 2008 Graduate Entry Students: Year 1: n=93

Medical & Nursing [Adult, Child, Mental Health]

• 3 Stations• Based on PEBL cases• 5-minute station with 3-min feedback• Actors play patient/carer

Page 4: Peer feedback in formative assessment to aid learning:

Study 1: Nov 2008 ( 78 attended = 84%)

Students:• Generate the marking criteria*• Undertake all OSCE stations as ‘candidates’• Undertake ‘examiner’ role in 1 station.• Actors give verbal feedback

Evaluation by:

Questionnaire*: 20 statement items Likert response scale Focus Group: nominal group technique.

* Adapted from FAST project Open Univ/Sheffield Hallam (Brown et al 2003)

Page 5: Peer feedback in formative assessment to aid learning:

Q1. Being the examiner was helpful in providing clear

instructions on what was expected of me(N=77)

All students (n = 77)

-100.0 -80.0 -60.0 -40.0 -20.0 0.0 20.0 40.0 60.0 80.0 100.0

Strongly disagree

Disagree

Agree

Strongly agree

Page 6: Peer feedback in formative assessment to aid learning:

Q5 The feedback was rapid and helpfulN=78

All students (n = 78)

-100.0 -80.0 -60.0 -40.0 -20.0 0.0 20.0 40.0 60.0 80.0 100.0

Strongly disagree

Disagree

Agree

Strongly agree

Page 7: Peer feedback in formative assessment to aid learning:

Q13. I found it difficult giving feedback to my colleagues. (p <0.05)

Female students (n = 52)

-100.0 -80.0 -60.0 -40.0 -20.0 0.0 20.0 40.0 60.0 80.0 100.0

Strongly disagree

Disagree

Agree

Strongly agree

Male students (n = 26)

-100.0 -80.0 -60.0 -40.0 -20.0 0.0 20.0 40.0 60.0 80.0 100.0

Strongly disagree

Disagree

Agree

Strongly agree

Page 8: Peer feedback in formative assessment to aid learning:

Q20. I felt that I was able to be honest in my feedback

Female students (n = 52)

-100.0 -80.0 -60.0 -40.0 -20.0 0.0 20.0 40.0 60.0 80.0 100.0

Strongly disagree

Disagree

Agree

Strongly agree

Male students (n = 26)

-100.0 -80.0 -60.0 -40.0 -20.0 0.0 20.0 40.0 60.0 80.0 100.0

Strongly disagree

Disagree

Agree

Strongly agree

Page 9: Peer feedback in formative assessment to aid learning:

Focus Group (7) Helpful to Learning

Seeing examples of:• good and bad communications (content &

delivery) (5 votes)

• differences between people in the same situation (5 votes)

• others’ behaviour helps you understand what you should do yourself (3 votes)

Page 10: Peer feedback in formative assessment to aid learning:

Unhelpful to Learning

• Checklist is too long (7 votes)

• Peers are ‘amateurs’, both at the skills being role-played and at feedback (5 votes)

• Some people are not brave enough in giving critical feedback (4 votes)

Page 11: Peer feedback in formative assessment to aid learning:

Anxiety about giving Negative Feedback

“I thought, ‘You really should never say that in front of somebody’, and I didn’t, to be honest, have the guts to tell them.”

“You want to say something positive but also you want to get across what they need to improve. ………. But it can be quite difficult, I think, because if you’re peers, it's harder than if you're a tutor.”

Page 12: Peer feedback in formative assessment to aid learning:

Feedback Sargeant J et al (2008) J Contin Educ Health Prof

Three inter-related factors for acceptance of

negative feedback:

1. Credibility – specificity, personal observation, explicit standards of performance

2. Emotion- when inconsistent with self perceptions of being a ‘good doctor’

3. Reflection – a good mediator of assimilation, acceptance and application

Page 13: Peer feedback in formative assessment to aid learning:

Incorporating Feedback

• “I thought I'd done something quite well, and someone pointed out that actually I could have done it this way and it might have been better. And then seeing someone else do it, I thought, ‘Oh well, yeah, that’s a much better way of doing it’. But it hadn't even occurred to me that I could improve on the way I was doing it. And I don't think it's until someone points it out to you in quite a specific way that you think, ‘Well actually, yeah, I’ll try that’. And I've done it since, and it is better.”

Page 14: Peer feedback in formative assessment to aid learning:

Learning Transfer

“With a patient who wanted a long conversation, I had to draw on the OSCE session. ‘OK, how do I make sure that I'm actually quite interested in what you're saying now?’ Because I've got three other patients that I really need to deal with.’”

(Nurse)

Page 15: Peer feedback in formative assessment to aid learning:

Study 2: May 2009Changes in response to Study 1

Teaching on ‘Giving Feedback’ (video CETL).

What would you give feedback on? - generate a marking scheme

How does the facilitator do with her feedback?- generate principles of feedback

Behavioural, specific, balanced +ve with areas to improve.

Page 16: Peer feedback in formative assessment to aid learning:

Changes in Study 2In response to Study 1

• More feedback time:

4-mins interview and 4-mins feedback• Actor to give written rather than verbal feedback

Empathy

Clarity• Cases to be circulated in advance.

Page 17: Peer feedback in formative assessment to aid learning:

Feedback

Page 18: Peer feedback in formative assessment to aid learning:

Responses to Questionnaire Statements [1]

-80.0

-60.0

-40.0

-20.0

0.0

20.0

40.0

60.0

80.0

100.0

Being theexaminer

w as helpfulin providing

clearinstructions

on w hatw as

expected ofme

With thisexercise

there w aslots of

feedback

I w ouldlearn better

if thefeedbackhad beenmore in-depth

Thefeedbackw as rapidand helpful

Feedbackmainly tells

me howw ell I did

compared toothers

Feedbackhelps me tounderstand

issuesbetter

Feedbackhelps me to

improve

Receivingfeedbackhelped meunderstandw hy I gotthe mark I

did

I found thefeedbackfrom thesimulated

patienthelpful

I listencarefully tofeedback

I w as ableto act upon

thefeedback I

got insubsequent

stations

Feedbackstimulatesme to go

back overmaterial

covered onother areas

of thecourse

I learnt newthings

w hilst beingan examiner

I felt that Iw as able tobe honest in

myfeedback

Strongly disagree Disagree Agree Strongly agree

(n = 48)

(n = 48)

(n = 48)

(n =48)

(n = 48)

(n = 48)

(n = 48)

(n = 48) (n = 48)

(n = 48)

(n = 48)

(n = 48)

(n = 48)

(n = 48)

Page 19: Peer feedback in formative assessment to aid learning:

Responses to Questionnaire Statements [2]

-80.0

-60.0

-40.0

-20.0

0.0

20.0

40.0

60.0

When I got things w rong ormisunderstood, I did notreceive much guidance

The feedback I received w asnot useful

I did not understand some ofthe feedback

I w ould prefer feedback fromtutors rather than my

colleagues

I found it diff icult givingfeedback to my colleagues

Feedback is not likely to bemuch help for the future

performances

Strongly disagree Disagree Agree Strongly agree

(n = 48)(n = 48)

(n = 48)

(n = 48) (n = 48)

(n = 48)

Page 20: Peer feedback in formative assessment to aid learning:

Free Text Comments

“I understand it was part of the research but it definitely would have been valuable to have been given verbal feedback from the actor as well as written”

“Really good – nice to see that we have improved since the last one, both in interviewing and giving feedback to colleagues”

“Peer feedback: I think it is very helpful, although we are relatively at the same level so limited in depth of feedback”

“Love the OSCE sessions with the actors. They are invaluable and a great learning tool”.

Page 21: Peer feedback in formative assessment to aid learning:

I

Introduced into Curriculum’08 MBBS 2009-10

Core Curriculum in Year 3• Both Communication (3) and Clinical (3) stations:

• Faculty supported, supervised and guided sessions on peer feedback engage learners actively

• Learners need training on giving constructive feedback

• Learners value these sessions

• Resourcing – cost effectiveness

Page 22: Peer feedback in formative assessment to aid learning:

Spin-offs

• www.bbc.co.uk/health

• BSc Med EdEffect of intervention on student performance in end of year

summative OSCE marker station.

Sample:

GEP yr 1 cohort (formative OSCE)

Graduates in Year 2

Page 23: Peer feedback in formative assessment to aid learning:

Thank you

Any Questions?

Page 24: Peer feedback in formative assessment to aid learning:

ReferencesSargeant J et al. 2007 Challenges in multi-source feedback: Intended

and unintended outcomes. Med Educ. 41:583-591Nicol & MacFarlane (2004) Rethinking Formative Assessment in HE: a

theoretical model and seven principles of good feedback practice

Boud, D. (2000). Sustainable assessment: rethinking assessment for the learning society. Studies in Continuing Education. 22 (2), 151-167.

Kurtz S, Silverman J. 2005. Teaching & Learning Communication Skills in Medicine. Radcliffe Pub Oxford. (Calgary-Cambridge Guide)

Weaver M. (2006) Do students value feedback? Student perceptions of tutors’ written responses, Assessment and Evaluation in Higher Education 31(3) pp. 379-394

Bing-You R.G., Paterson, P., Levine, M.A. (1997) Feedback falling on deaf ears: residents’ receptivity to feedback tempered by sender credibility, Medical Teacher, 19, pp. 40-44

Page 25: Peer feedback in formative assessment to aid learning:

Cost effectiveness

120 students All got chance to:• Examine 6 peers• Practice 6 stations• Be a patient in 2 stations

Tutors = 8‘Bodies’ = 12Actor patient = 4