SO WE DID IT! DID IT WORK? HOW DO WE KNOW?A PRESENTATION ABOUT METHODS OF EVALUATION USED FOR IPL AT UEA
BY DR SUSANNE LINDQVIST 6TH JUNE 2016
“Brilliant! Really enjoyed it and learned lots – thank you!”
“The topic was not relevant. One member of our group did not do, or say , anything.
It was a complete waste of my time and I learned nothing.”
“I didn’t realise the value at the time, but now as I have qualified I appreciate
how valuable it was”
“My group did really well, but many of the others didn’t”
“Interprofessional education occurs when two or more professions
learn with, from and about each other in order to improve
collaboration and the quality of care”
CAIPE, 2002
INTERPROFESSIONAL EDUCATION
CYCLE OF IPL DEVELOPMENT
▪ IDEA
▪ INVOLVING STAKEHOLDERS
▪ DEVELOPMENT
▪ PILOT
▪ EVALUATION OF PILOT
▪ FURTHER DEVELOPMENT OF PILOT
▪ FULL SCALE INTERVENTION
▪ EVALUATION OF FULL SCALE INTERVENTION
▪ ONGOING DEVELOPMENT
▪ NEW IDEA!
INTERVENTION
PILOT
IDEA
TODAY’S PRESENTATION WILL FOCUS ON EVALUATION
▪ IDEA
▪ INVOLVING STAKEHOLDERS
▪ DEVELOPMENT
▪ PILOT
▪ EVALUATION
▪ DEVELOPMENT
▪ FULL SCALE INTERVENTION
▪ EVALUATION
▪ ONGOING DEVELOPMENT
▪ NEW IDEA!
BRIEF OVERVIEW OF PROFESSIONS INVOLVED IN PRE-REGISTRATION IPL AT UEAIPL at UEA involves ~2500 students from 12 different courses:
Operating Department Practice
Paramedic Science
Physiotherapy
Pharmacy
Medicine
Nursing
Occupational Therapy
Speech and Language Therapy
Midwifery
Social Work
Education
Clinical Psychology
BRIEF OVERVIEW OF IPL AT UEA
IPL1: Teamworking - how individual behaviours and communication impact on team effectiveness, performance and care delivery
IPL3: Learning and working with the service user - working together with service users to explore best collaborative practice
IPL2: Interprofessional Teamworking - what different professionals contribute to the care journey and how they work together to provide high quality care, that is person-centred, compassionate and safe
IPL4: Dealing with complex and sensitive issues - listening to people with lived experiences and to those who care for them
IPL5: Interprofessional Clinial Skills – practice working side-by-side to clarify roles and responsibilities
IPL1 and IPL2 are compulsory; IPL3 and IPL4 are voluntary, IPL5 is compulsory for some, but not others…
EVALUATION OF LEARNING OUTCOMES
Kirkpatrick levels of learning (learner’s reaction, modification in attitudes, knowledge and skills, changes in behaviour, changes in practice, benefit to patients/service users)
Kirkpatrick, 1996
Bloom’s taxonomy of learning (knowledge, comprehension, evaluation, analysis, synthesis, application)
Blooms Taxonomy: Bloom, 1956; Anderson et al., 2001
Millers pyramid (knows, knows how, shows how, does)
Millers Pyramid: Miller, 1990
Mapping LOs to existing learning theories
EVALUATION CONSIDERING THE 3P MODEL TO MAXIMISE OUTPUT
Presage(Input)
Process(IPL intervention)
Product(Output)
Biggs, 2002; Frye & Hemmer, 2012
FEEDBACK FORMS
▪ How well the intervention as a whole helped them achieve the learning outcomes
▪ What they specifically learned
▪ How useful different elements of the intervention were
▪ How supportive their facilitator was
▪ Ideas of how to further develop the intervention
We ask students to feed forward to us about:
IPL1 – HOW HELPFUL WAS IPL1 IN HELPING YOU ACHIEVE THE FOLLOWING LEARNING OUTCOMES?
Analysis of IPL1 Student Evaluation 2014/5 and 2015/6
2014/15 (respondents: 750) 2015/16 (respondents: 864)
Learning Outcomes Helpful/Very Helpful Not very or
at all helpful
Helpful/Very Helpful Not very or
at all helpful
Constructively discuss and
identify how different
behaviours and personality
styles can impact on a team’s
ability to function effectively
63% 16% 74% 10%
Recognise the effect of
Interprofessional teamwork
on integrated and high quality
care that is person-centred,
compassionate and safe
66% 15% 83% 4%
Recognise the effect of your
own professional role and
responsibilities and of
respecting those of other
professions when delivering
integrated care as part of an
Interprofessional team
51% 9% 78% 8%
1 %
3 %
13 %
43 %
40 %
Recognise the effect of interprofessional teamwork on the delivery of integrated and high quality care
Not at all helpful
Not very helpful
Neither helpful norunhelpful
Helpful
Very helpful
2 %
6 %
14 %
49 %
29 %
Recognise the importance of your own professional role and responsibilities and of respecting those of other professions when delivering integrated care
Not at all helpful
Not very helpful
Neither helpful norunhelpful
Helpful
Very helpful
2 %
8 %
16 %
53 %
21 %
Identify behaviours and personality styles that influence a team's ability to function effectively
Not at all helpful
Not very helpful
Neither helpful norunhelpfulHelpful
Very helpful
IPL1 – HOW HELPFUL WAS IPL1 IN HELPING YOU ACHIEVE THE FOLLOWING LEARNING OUTCOMES?
Analysis of IPL2 Student Evaluation 2014/5 and 2015/6
2014/15 (respondents: 419) 2015/16 (respondents: 539)
Learning Outcomes Helpful/Very Helpful Not very or at all helpful Helpful/Very Helpful Not very or at all helpful
Describe the role and responsibilities of your own and other professions in providing integrated and high quality care that is person-centred, compassionate and safe
85% 4% 77% 9%
Demonstrate awareness of common
challenges associated with
interprofessional teamworking and
approaches to overcoming these in
order to provide integrated care
85% 4% 82% 6%
Demonstrate some key teamwork skills
and behaviours that enhance care
81% 6% 76% 9%
Recognise the impact of
interprofessional teamworking on care
from the service user perspective
88% 5% 79% 8%
Appraise and constructively assess the
performance of your peers
72% 7% 70% 9%
IPL2 – HOW HELPFUL WAS IPL2 IN HELPING YOU ACHIEVE THE FOLLOWING LEARNING OUTCOMES?
Not at all helpful2 %
Not very helpful7 %
Neither helpful nor unhelpful
14 %
Helpful55 %
Very helpful22 %
Describe the role and responsibilities of your own and other professions in providing integrated and
high quality care that is person-centred, compassionate and safe
Not at all helpful2 %
Not very helpful4 %
Neither helpful nor unhelpful
12 %
Helpful57 %
Very helpful25 %
Demonstrate awareness of common challenges associated with interprofessional teamworking and approaches to overcoming these in order to provide
integrated care
IPL2 – HOW HELPFUL WAS IPL2 IN HELPING YOU ACHIEVE THE FOLLOWING LEARNING OUTCOMES?
IPL2Please tell what specifically you have learnt from IPL2 that was helpful to your development as a
future collaborative practitioner?
“… I thought I may face prejudice from some of the nurses and paramedics, like with my group last year.
However the group of individuals I was with this year were great, easy to communicate with and got the work
done. There was mutual understanding and respect amongst us all and no judgement. It was good to see that
the minority of hostile individuals I met last year aren't reflective of the majority of lovely future health care
professionals I've seen.”
“I found the workshops particularly helpful, especially when exploring other healthcare roles and engaging with
other students from different professions. On reflection I feel that the sessions made me more aware of the
importance of collaborative working. The sessions have opened my eyes to how things can go badly when
professionals do not communicate effectively and how this can impact on patient experiences and outcomes.”
“The idea of interprofessional working for the benefit of the service user is hindered by the attitudes of
individual professionals. IPL2 helped me develop ways in which to constructively deal with these individuals.”
“… As a student, it was disappointing to hear so much negativity directed towards my field. Hopefully the process will have taught other students that not all doctors are egotistical. It made me think about how I can try to overcome prejudices about doctors when working in a team.”
IPL2Please comment on the input from the Service User/s in your session
Hugh - Session A
“I really enjoyed having the service user, it gave a real life perspective on issues that arise with patient care and
made you think about how to avoid it. I thought having first hand accounts was really useful and hit home.”
“Found the service user to be very informative and encouraging.”
Hugh - Session C
“I found our service user more of a hindrance, I personally found him very judgmental and the feedback he gave
was not very reassuring or relevant to our courses if anything he made people feel embarrassed about their
presentation, their skills as a professional and/or appearance.”
“I appreciated hearing his perspective in the first session, however I think he misunderstood his role. In the
second session he gave feedback that was not really relevant and somewhat missed the point of the IPL session.
I think sometimes even the facilitator picked up that he was not very helpful and sometimes a bit
condescending.”
FEEDBACK IMPROVES FROM IPL1 TO IPL5…
▪ IPL3 and IPL4 is voluntary and involves more service user interaction ▪ IPL5 is focussing around clinical skills
So, do we need IPL1 and IPL2?...
ATTITUDES TO OWN AND OTHER HEALTHCARE PROFESSIONS
2002-2012
▪ Students arrive with views of their own professions and views of others
▪ Doctors are seen as the least “caring” and nurses the most “caring” profession
▪ Students who completed IPL1 in the past - when it was 7 weeks long - developed positive attitudes
▪ Students who did not complete IPL1 (acting control) showed no change or developed more negative views by simply spending 7 weeks in their course and no IPL
▪ Students’ views reverted back to baseline by the time they completed their course
2013 onwards…
▪ Today, students do not develop more positive attitudes after IPL1
▪ They do however appear to develop more positive attitudes after IPL2 now than before, and since we introduced IPL5 the positive shift in attitudes after IPL2 seems to be sustained in their last year.
So, do we need IPL1?...
ACTING VERSUS REACTING…
INTERVENTION
PILOT
IDEA
▪ Always strive to develop from Good to Great.▪ Look ahead, but check behind.▪ Agree way forward.▪ Try make a difference, if not to all, to most.▪ Involve stakeholders – in further development and appraisal of progress.▪ The ownership is on everyone – the learner too.
References alluded to in this presentation
Miller, G.E. The assessment of clinical skills/competence/performance. Acaddemic Medicine, 1990; 65: 563–67.
Anderson, L.W., Krathwohl, D.R., Airasian, P.W., Cruikshank, K.A., Mayer, R.E., Pintrich, P.R., Raths, J. & Wittrock, M.C. A Taxonomy for Learning, Teaching, and Assessing: A revision of Bloom's Taxonomy of Educational Objectives. 2001, New York: Pearson, Allyn & Bacon
Bloom, B.S. (Ed.). Engelhart, M.D., Furst, E.J., Hill, W.H., Krathwohl, D.R. Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. 1956, New York: David McKay Co Inc
Biggs, J.B. Aligning the curriculum to promote good learning LTSN Generic Centre. 2002
Anderson, E., Smith, R., & Hammick, M. Evaluating an interprofessional education curriculum : A theory-informed approach. Medical Teacher, 2015; Early Online 1-10.
Frye, A. & Hemmer, P. Program evaluation models and related theories: AMEE Guide No. 67. Medical Teacher, 2012; 34:e288–e2299.
Kirkpatrick, D. Revisiting Kirkpatrick’s four level model. Training and Development, 1996; 1: 54–59.
Freeman S, Wright A & Lindqvist S. ‘Facilitator Training for educators involved in interprofessional learning’. Journal of
Interprofessional Care, 2010; 24(4):375-385.
CAIPE, Centre for the Advancement of Interprofessional Education – a definition. 2002 + Introducing InterprofessionalEducation, 2012 http://caipe.org.uk/
Hawkes G, Nunney I & Lindqvist S. Caring for attitudes as a means of caring for patients – improving medical, pharmacyand nursing students’ attitudes to each other's professions by engaging them in interprofessional learning. MedicalTeacher, 2013; Web paper e1-e.
Top Related