Between Scylla and Charybdis: Navigating the discourses of ......language of Taylor’s Today, we...
Transcript of Between Scylla and Charybdis: Navigating the discourses of ......language of Taylor’s Today, we...
Between Scylla and Charybdis:
Between Scylla and Charybdis: y
Navigating the discourses of
yNavigating the discourses of discourses of competence discourses of competence
Brian Hodges MD, PhD, FRCPCProfessor, Scientist and Director
Wilson Centre for Research in Education
Brian Hodges MD, PhD, FRCPCProfessor, Scientist and Director
Wilson Centre for Research in EducationWilson Centre for Research in EducationUniversity of Toronto
Wilson Centre for Research in EducationUniversity of Toronto
Medical education discourseMedical education discourse
As every medical professor knows you just find a
As every medical professor knows you just find a knows, you just find a patient with some fractures and escort a gaggle of students to the bedside for
knows, you just find a patient with some fractures and escort a gaggle of students to the bedside for students to the bedside for a good long stare.
A l h ff i
students to the bedside for a good long stare.
A l h ff i As long as the sufferer is not prodded too hard, or denied a clear view of the
l h l l
As long as the sufferer is not prodded too hard, or denied a clear view of the
l h l ltelevision, no ethical rules are violated.television, no ethical rules are violated.
Economist, M h 24 2007March 24, 2007
Is this how you talk about Is this how you talk about Is this how you talk about medical education?
Is this how you talk about medical education?
We all have models (discourses) about medical education and competenceWe all have models (discourses) about medical education and competenceThe discourses we use frame the way we see and talk about what competence i
The discourses we use frame the way we see and talk about what competence iisThese discourses change in time and l
isThese discourses change in time and lplaceplace
Encountering different Encountering different Encountering different discourses by travelingEncountering different discourses by traveling
My interest in discourses took me on a 6 year voyage to real places (France, My interest in discourses took me on a 6 year voyage to real places (France, China, Ethiopia, Israel, Jordan, Pakistan…) and virtual locations (books, internet documents) where
China, Ethiopia, Israel, Jordan, Pakistan…) and virtual locations (books, internet documents) where internet, documents) where competence is discussed and debatedinternet, documents) where competence is discussed and debated
I am going to take you on an imaginary version of that voyageI am going to take you on an imaginary version of that voyageversion of that voyageversion of that voyage
h hh hThis is our shipThis is our ship
The HippocratesHippocrates
Harrison’sOur map*Our map*
Harrison s Island
Miller’s Island
Cronbach’s Island
Taylor’s Island
Schön’s Island
*Conceptual destinations
Our guide booksOur guide books
Our dictionaries of language
Our dictionaries of language language language
Harrison’s IslandHarrison’s IslandHarrison s IslandKnowledge discourse
Miller’s Island
Harrison s IslandKnowledge discourse
Miller’s IslandMiller s IslandPerformance discourse
Cronbach’s Island
Miller s IslandPerformance discourse
Cronbach’s IslandCronbach s IslandPsychometric discourse
Taylor’s Island
Cronbach s IslandPsychometric discourse
Taylor’s IslandTaylor s IslandProduction discourse
Schön's Island
Taylor s IslandProduction discourse
Schön's IslandSchön s IslandReflection discourse
Schön s IslandReflection discourse
Our guide to culture and h
Our guide to culture and harchitecturearchitecture
Harrison’sHarrison s Island
Miller’s Island
Cronbach’s Island
Taylor’s Island
Schön’s Island
Harrison’s IslandHarrison’s Island
Copyright ©2000 BMJ Publishing Group Ltd.
Medical school architecture Medical school architecture Medical school architecture Medical school architecture Medical school architecture Medical school architecture on Harrison’s Islandon Harrison’s Island
Medical school architecture Medical school architecture on Harrison’s Islandon Harrison’s Island
Harrison’s IslandHarrison’s IslandKnowledge Knowledge discoursediscourse
Facts, foundational knowledge, basic science, first Facts, foundational knowledge, basic science, first principles, fund of knowledge, classic text books, principles, fund of knowledge, classic text books, classic articlesclassic articles
Roles and Roles and activitiesactivities
Teacher’s: Teacher’s: Lecturers, source of wisdomLecturers, source of wisdomStudent’s: Student’s: Memorize, reproduceMemorize, reproduce
Tests: MultipleTests: Multiple--choice examschoice exams
Believe in…Believe in… Knowledge accumulationKnowledge accumulation
ReligionReligion Cognitive psychologyCognitive psychology
Warnings about this Warnings about this islandisland
The preoccupation with doing well on standardized tests has conditioned the way young people …think. They have better-developed cognitive
The preoccupation with doing well on standardized tests has conditioned the way young people …think. They have better-developed cognitive y p gabilities to recognized random facts than to construct patterns or think systematically.
Jacques Barzun, 1988
y p gabilities to recognized random facts than to construct patterns or think systematically.
Jacques Barzun, 1988q ,
Student: Madam Madam –– do you have higher do you have higher conjugated or unconjugated conjugated or unconjugated
q ,
Student: Madam Madam –– do you have higher do you have higher conjugated or unconjugated conjugated or unconjugated conjugated or unconjugated conjugated or unconjugated bilirubin?bilirubin?
Harrison’s Island Medical Student
conjugated or unconjugated conjugated or unconjugated bilirubin?bilirubin?
Harrison’s Island Medical Student
Harrison’sHarrison s Island
Miller’s Miller’s IslandIsland
Cronbach’s Island
Taylor’s Island
Schön’s Island
Miller’s IslandMiller’s Island
Copyright ©2000 BMJ Publishing Group Ltd.
Medical School Architecture on Medical School Architecture on
Miller’s IslandMiller’s Island
Medical Student
Simulated Patient
Interaction
Physician Examiner
Medical classroom
Miller’s IslandMiller’s IslandPerformancePerformance Simulations simulated patient feedbackSimulations simulated patient feedbackPerformance Performance discoursediscourse
Simulations, simulated patient, feedback, Simulations, simulated patient, feedback, performance, skills, skills centre, OSCE, multiple performance, skills, skills centre, OSCE, multiple observationsobservations
Roles and Roles and activitiesactivities
Teacher’s:Teacher’s: Demonstrating and observing skills Demonstrating and observing skills Student’s: Student’s: Performing for observers Performing for observers Tests: PerformanceTests: Performance--based assessment based assessment
(simulation, OSCE) (simulation, OSCE)
Believe in…Believe in… Observation of performanceObservation of performance
ReligionReligion BehaviourismBehaviourism
The way they talk on The way they talk on The way they talk on The way they talk on The way they talk on The way they talk on Miller’s IslandMiller’s Island
The way they talk on The way they talk on Miller’s IslandMiller’s Island
In many places they would ask students to write an essay on the origin of the word shoelace or give them a multiple choice
In many places they would ask students to write an essay on the origin of the word shoelace or give them a multiple choice shoelace, or give them a multiple choice question on the design of shoelaces or even ask them to describe the steps in tying a h l
shoelace, or give them a multiple choice question on the design of shoelaces or even ask them to describe the steps in tying a h l shoelace.
Whereas really the only way of doing it is
shoelace.
Whereas really the only way of doing it is Whereas really the only way of doing it is showing you know how to tie a shoelace.
Ronald Harden 2005
Whereas really the only way of doing it is showing you know how to tie a shoelace.
Ronald Harden 2005
Warnings about Warnings about gMiller’s Island
gMiller’s Island
Exclusive reliance on a pedagogical approach of simulation training may be encouraging students to become “simulation doctors” who act out a good
Exclusive reliance on a pedagogical approach of simulation training may be encouraging students to become “simulation doctors” who act out a good grelationship with their patients but have no authentic connection with them
Hanna and Fins (2006)
grelationship with their patients but have no authentic connection with them
Hanna and Fins (2006)Hanna and Fins (2006)
Student: Oh that must be hard for you…wow that t b h d f h th t t b ll
Hanna and Fins (2006)
Student: Oh that must be hard for you…wow that t b h d f h th t t b ll must be hard for you… oh, yes that must be really
hard for youPatient: Can you stop saying that?!
must be hard for you… oh, yes that must be really hard for you
Patient: Can you stop saying that?!Medical Student, Miller’s IslandMedical Student, Miller’s Island
Harrison’sHarrison s Island
Miller’s Island
Cronbach’s Cronbach’s IslandIsland
Taylor’s Island
Schön’s Island
Cronbach's IslandCronbach's IslandCronbach's IslandCronbach's Island
Medical school architecture on Medical school architecture on
Cronbach’s IslandCronbach’s Island
Medical Student
Standardized Patient
Interaction
Ph i i
Interaction
Psychometricians
PhysicianExaminer OSCE
Station
Clinical skills testing centre
The way they talk on The way they talk on The way they talk on Cronbach’s island
The way they talk on Cronbach’s island
Knowledge of educational products and educational purposes must become quantitative, take the form of measurement
Knowledge of educational products and educational purposes must become quantitative, take the form of measurement of measurement.
Education is one form of human engineering and will
of measurement.
Education is one form of human engineering and will g gprofit by measurement of human nature and achievement as mechanical and electrical engineering have profited by using the foot-pound,
g gprofit by measurement of human nature and achievement as mechanical and electrical engineering have profited by using the foot-pound, e g ee g a e p o ted by us g t e oot pou d,calorie, volt and ampere.
Thorndike, 1922
e g ee g a e p o ted by us g t e oot pou d,calorie, volt and ampere.
Thorndike, 1922
Cronbach’s IslandCronbach’s IslandPsychometricPsychometric reliability validity generalizability datareliability validity generalizability dataPsychometric Psychometric discoursediscourse
reliability, validity, generalizability, data, reliability, validity, generalizability, data, psychometrician, candidate, checklist, itempsychometrician, candidate, checklist, item--banking, banking, cutcut--point, standardizationpoint, standardization
Roles and Roles and activitiesactivities
Teacher’s:Teacher’s: Preparing students for examsPreparing students for examsStudent’s:Student’s: Strategies to max scores on exams Strategies to max scores on exams Tests: Standardized, summative exams, Tests: Standardized, summative exams,
Believe in…Believe in… The conversion of human behaviour to numbers The conversion of human behaviour to numbers
ReligionReligion PsychometricsPsychometrics
Learning the Learning the language of Cronbach’s language of Cronbach’s Cronbach s Cronbach s
In the 1980s many people moved from the older Miller’s and Harrison’s Island to Cronbach’s and learned its language of psychometrics
In the 1980s many people moved from the older Miller’s and Harrison’s Island to Cronbach’s and learned its language of psychometricslearned its language of psychometrics
The standardized-patient technique …has many of the same
learned its language of psychometrics
The standardized-patient technique …has many of the same p q yadvantages of the multiple-choice question
It is a standardized item, can be given in multiples, and can be
p q yadvantages of the multiple-choice question
It is a standardized item, can be given in multiples, and can be It is a standardized item, can be given in multiples, and can be scored in reliable and valid ways.
Howard Barrows, formerly of Miller’s Island 1993
It is a standardized item, can be given in multiples, and can be scored in reliable and valid ways.
Howard Barrows, formerly of Miller’s Island 1993
Warning: There are severe Warning: There are severe paradigmatic storms around
Cronbach’s islandparadigmatic storms around
Cronbach’s islandCronbach s islandCronbach s island
H i Bi d BHarrison Bird Brown
Warnings about Warnings about gCronbach’s island
gCronbach’s island
In a standardized environment, it becomes a problem when [a patient] doesn’t look the same as everybody else -Diana Tabak 2006
In a standardized environment, it becomes a problem when [a patient] doesn’t look the same as everybody else -Diana Tabak 2006
I have heard enough anecdotes about the shotgun behaviour induced by checklists to shift the
I have heard enough anecdotes about the shotgun behaviour induced by checklists to shift the behaviour induced by checklists to shift the burden of proof onto the advocates of this strategy -Geoff Norman 2005
behaviour induced by checklists to shift the burden of proof onto the advocates of this strategy -Geoff Norman 2005
I think the problem is that as we’ve tried to get more valid and more reliable statistically, we’ve
I think the problem is that as we’ve tried to get more valid and more reliable statistically, we’ve actually come away from what we were really trying to do in the first place. -Jean Ker 2005actually come away from what we were really trying to do in the first place. -Jean Ker 2005
From these choppy waters, …From these choppy waters, …From these choppy waters, …From these choppy waters, …
We now follow the route that some took from the older islands to two We now follow the route that some took from the older islands to two neighboring IslandsOff the coasts of Taylor’s and neighboring IslandsOff the coasts of Taylor’s and Off the coasts of Taylor s and Schön's Islands, there are dangers!Off the coasts of Taylor s and Schön's Islands, there are dangers!
Caught between the Caught between the S ll d Ch bdiS ll d Ch bdiScylla and CharybdisScylla and Charybdis
Harrison’sHarrison s Island
Miller’s Island
Cronbach’s Island
Taylor’s Island
Schön’s Island
Taylor’s IslandTaylor’s IslandTaylor s IslandTaylor s Island
Michael Wolf, Mother Jones July/August2005July/August2005
Medical School Architecture Medical School Architecture on Taylor’s Islandon Taylor’s Island
C did t St d di d
Physician Examiners
Candidate StandardizedPatient
Interaction
h i iOSCE Station
Management
Psychometricians
OSCE testingOSCE testinginstitution
Taylor’s IslandTaylor’s IslandProductionProduction Cost, production, finished product, accountability,Cost, production, finished product, accountability,Production Production discoursediscourse
Cost, production, finished product, accountability, Cost, production, finished product, accountability, outcomes, efficiencyoutcomes, efficiency
R l dR l d T h ’T h ’ M t d l bM t d l bRoles and Roles and activitiesactivities
Teacher’s:Teacher’s: Management and labourManagement and labour
Student role: Raw material or productStudent role: Raw material or product
Tests: Mass Testing quality assuranceTests: Mass Testing quality assuranceTests: Mass Testing, quality assuranceTests: Mass Testing, quality assurance
Believe Believe Productivity and efficiencyProductivity and efficiencyin…in…
ReligionReligion Production/CapitalismProduction/Capitalism
The way they talk The way they talk The way they talk on Taylor’s IslandThe way they talk on Taylor’s Island
The corporate order, with its assembly-line techniques job differentiation and
The corporate order, with its assembly-line techniques job differentiation and line techniques, job differentiation, and increased organizational size, demanded a different type of office
line techniques, job differentiation, and increased organizational size, demanded a different type of office demanded a different type of office space and a more regulated and regimented flow of time
demanded a different type of office space and a more regulated and regimented flow of time
Kwolek-Folland (1994) Kwolek-Folland (1994)
Learning the Learning the Learning the language of Taylor’s
Learning the language of Taylor’s
Today, we have reached new heights in the science of producing and testing medical students.
Today, we have reached new heights in the science of producing and testing medical students.students.
We run the operation using standardized and efficient p ocess
students.
We run the operation using standardized and efficient p ocessefficient process.
As a result we are in a much better position to
efficient process.
As a result we are in a much better position to passure the public that we are graduating a product with the skills for safe and effective practice.
passure the public that we are graduating a product with the skills for safe and effective practice.practice.practice.
Beware the Scylla lurking in lurking in the waters f T l ’ of Taylor’s
Island!Athena (2007) Women in Greek Mythsin Greek Myths
The siren call of d d h
The siren call of d d hproductivity and the
death of feedbackproductivity and the death of feedbackdeath of feedbackdeath of feedback
In a production model, the many In a production model, the many p , yassessments of “quality” are summative, for external bodies, and
p , yassessments of “quality” are summative, for external bodies, and there is no feedback to students.there is no feedback to students.
Health professionals live in an “examined society” in which constant surveillance and testing locates the responsibility for their
Health professionals live in an “examined society” in which constant surveillance and testing locates the responsibility for their testing locates the responsibility for their competence externally to the individual.testing locates the responsibility for their competence externally to the individual.
External monitoring of External monitoring of External monitoring of quality may help to:
External monitoring of quality may help to:
Discover and remove a few very low Discover and remove a few very low performers (although the eventual fail rate on most health professional examinations is near zero)
performers (although the eventual fail rate on most health professional examinations is near zero)near zero)Reassure the public that the profession is monitoring knowledge and skills of the next
ti
near zero)Reassure the public that the profession is monitoring knowledge and skills of the next
tigenerationStandard setting across the country and internationally
generationStandard setting across the country and internationallyinternationallyinternationally
But some say this But some say this culture of production
is dangerousculture of production
is dangerousis dangerousis dangerous
I di d l i I di d l i It distorts student learning Produces no feedback to advance learning L d t diti ti d t ti
It distorts student learning Produces no feedback to advance learning L d t diti ti d t tiLeads to commoditization and contracting-out of examinations, test preparation and even teaching, with significant increases in
Leads to commoditization and contracting-out of examinations, test preparation and even teaching, with significant increases in e e teac g, t s g ca t c easescosts for students and professionalse e teac g, t s g ca t c easescosts for students and professionals
If you create a sausage factory - you will get sausages.
If you create a sausage factory - you will get sausages.
Harrison’sHarrison s Island
Miller’s Island
Cronbach’s Island
Taylor’s Island
Schön’s Schön’s IslandIsland
Schön's IslandSchön's Island
Medical school architecture Medical school architecture on Schön's Islandon Schön's Island
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Medical Student
Teacher/Examiner
The way they talk on The way they talk on The way they talk on Schön's Island
The way they talk on Schön's Island
The concept of learner as a mere The concept of learner as a mere The concept of learner as a mere processor of information has been replaced by the image of a self-
The concept of learner as a mere processor of information has been replaced by the image of a self-replaced by the image of a selfmotivated, self-directed problem solver.
replaced by the image of a selfmotivated, self-directed problem solver.
Ministry of Education 1980Ministry of Education 1980
Schön's IslandSchön's IslandSchön s IslandSchön s IslandReflection Reflection discoursediscourse
reflection, selfreflection, self--directed learning, learning contracts, directed learning, learning contracts, portfolios, adult learnerportfolios, adult learnerdiscoursediscourse portfolios, adult learnerportfolios, adult learner
Roles and Roles and ti ititi iti
Teacher: Teacher: Guide/Mentor/ConfessorGuide/Mentor/ConfessorSt d t lSt d t l R fl ti lfR fl ti lf l il iactivitiesactivities Student role: Student role: Reflecting, selfReflecting, self--analysisanalysisMeasures: PortfolioMeasures: Portfolio
Believe Believe in…in…
SelfSelf--assessment, selfassessment, self--direction, selfdirection, self--reflectionreflection
ReligionReligion SelfSelf--actualization, psychoanalysisactualization, psychoanalysis
The ‘trinity’ of reflective The ‘trinity’ of reflective The trinity of reflective technologies
The trinity of reflective technologies
Self-assessment, self-direction and self-regulationSelf-assessment, self-direction and self-regulationAdult learning theory and the idea that the locus for control of competence is i l
Adult learning theory and the idea that the locus for control of competence is i l internal Different technologies such as:
tf li fl ti di i l b k
internal Different technologies such as:
tf li fl ti di i l b k portfolios, reflective diaries, logbooks, self-directed web-based modules portfolios, reflective diaries, logbooks, self-directed web-based modules
But beware!
But beware! beware! You can beware! You can vanish into the vanish into the into the
Charybdis!into the
Charybdis!Athena (2007) Women in Greek Myths, http://www.paleothea.com/index.html
Warning’s about Warning’s about Schön's IslandSchön's Island
In a review of many studies on self-assessment, “There is a subset of clinicians who appear, either by training or personality unable to judge
In a review of many studies on self-assessment, “There is a subset of clinicians who appear, either by training or personality unable to judge by training or personality, unable to judge themselves”
Dave Davis (2006)
by training or personality, unable to judge themselves”
Dave Davis (2006)
In adopting self-directed methods for maintenance of competence, “Regulators appear quite
In adopting self-directed methods for maintenance of competence, “Regulators appear quite p g pp qunconcerned about the lack of coherence between what is being monitored ‘at a distance’ and the actual professional knowledge (needed) to function skilfully and competently”
p g pp qunconcerned about the lack of coherence between what is being monitored ‘at a distance’ and the actual professional knowledge (needed) to function skilfully and competently”to function skilfully and competently
Nelson and Purkis (2004)
to function skilfully and competently
Nelson and Purkis (2004)
The archetype of the The archetype of the ypinternally, self-
regulating professional
ypinternally, self-
regulating professionalregulating professionalregulating professional
Reflects regularly on daily practiceSelf-assesses gaps in knowledge or skillReflects regularly on daily practiceSelf-assesses gaps in knowledge or skillSeeks opportunities to redress gapsInvests energy to learn (or relearn)I t k l d i t ti
Seeks opportunities to redress gapsInvests energy to learn (or relearn)I t k l d i t tiIncorporates new knowledge into practiceIncorporates new knowledge into practice
Handfield-Jones 2002
But if we read the But if we read the But if we read the literature…
But if we read the literature…
We look for gaps? “S lf t ti ” f ti f lf fl ti
We look for gaps? “S lf t ti ” f ti f lf fl ti“Self-protective” function of self-reflection
We find gaps when we look? The ineffecti eness of self assessment
“Self-protective” function of self-reflection
We find gaps when we look? The ineffecti eness of self assessmentThe ineffectiveness of self-assessment
We try to address gaps through learning?We study what we are already good at
The ineffectiveness of self-assessment
We try to address gaps through learning?We study what we are already good atWe study what we are already good at
We incorporate new information into practice?
We study what we are already good at
We incorporate new information into practice?practice?
Ineffectiveness of CME for changing practicepractice?
Ineffectiveness of CME for changing practice
A repeated patternA repeated patternA repeated patternA repeated pattern
8090
100
50607080
rcen
tile
Actual ScoresPerceived Scores
10203040Pe
r
Perceived (After)
010
BottomQuartile
2ndQuartile
3rdQuartile
TopQuartile
Kruger and Dunning 1999, Hodges et al 2000
h fl h hh fl h hThe flaw in the theoryThe flaw in the theory
Theories of adult learning focus on the reasons why people DO learnTheories of adult learning focus on the reasons why people DO learn
Little or no research on why people DON’T learnLittle or no research on why people DON’T learn
It is impossible to make people understand h i i f i i k l d
It is impossible to make people understand h i i f i i k l d their ignorance, for it requires knowledge to
perceive it; and therefore, he that can perceive it, hath it not
their ignorance, for it requires knowledge to perceive it; and therefore, he that can perceive it, hath it not
Jeremy TaylorJeremy Taylor
The voyage The voyage homehome
Traveler’s diseases you Traveler’s diseases you Traveler s diseases you don’t want to bring homeTraveler s diseases you
don’t want to bring home
U bl d li d k l dU bl d li d k l dUnusable decontextualized knowledgeDeficient or fake interpersonal skills Homogenization/suppression of diversity
Unusable decontextualized knowledgeDeficient or fake interpersonal skills Homogenization/suppression of diversityHomogenization/suppression of diversityCommoditization of education technology and rising costs to students and professionalsI i h t f t h t d t l ti hi
Homogenization/suppression of diversityCommoditization of education technology and rising costs to students and professionalsI i h t f t h t d t l ti hi Impoverishment of teacher-student relationship and the loss of feedbackReflection without substance
Impoverishment of teacher-student relationship and the loss of feedbackReflection without substance
Lessons from Harrison’s Lessons from Harrison’s Lessons from Harrison s and Miller’s Islands
Lessons from Harrison s and Miller’s Islands
Knowledge is important -but try to avoid teaching and testing “pure”
Knowledge is important -but try to avoid teaching and testing “pure” and testing “pure” knowledge - integrate with skills and real context
and testing “pure” knowledge - integrate with skills and real context
Skills are important by must be integrated with Skills are important by must be integrated with ust be teg ated tknowledgeWhen using simulations ensure transfer to
ust be teg ated tknowledgeWhen using simulations ensure transfer to ensure transfer to ‘authentic’ clinical settingsensure transfer to ‘authentic’ clinical settings
Lessons from Cronbach’s Lessons from Cronbach’s IslandIsland
Highly standardized psychometric methods can be helpful, but they have
Highly standardized psychometric methods can be helpful, but they have be helpful, but they have some serious side-effects
Commit to di e sit in
be helpful, but they have some serious side-effects
Commit to di e sit in Commit to diversity in teaching/testing, reduce rigid standardization -real lif i “ ”
Commit to diversity in teaching/testing, reduce rigid standardization -real lif i “ ”life is “messy”
Minimize checklists and
life is “messy”
Minimize checklists and Minimize checklists and “reductionistic” measuresMinimize checklists and “reductionistic” measures
Lessons from Taylor’s IslandLessons from Taylor’s IslandLessons from Taylor s IslandLessons from Taylor s IslandDon’t created a big teaching Don’t created a big teaching g gand testing factory that does only end-of-line quality assurance
g gand testing factory that does only end-of-line quality assurance
Use small-scale teaching and frequent local assessment Use small-scale teaching and frequent local assessment frequent, local assessment whenever possible versus “one-shot” mass assessment
frequent, local assessment whenever possible versus “one-shot” mass assessment
Avoid mechanization, labour-management, manufacturing Avoid mechanization, labour-management, manufacturing g , gmodels that dehumanize medical education
g , gmodels that dehumanize medical education
Lessons from Schön's IslandLessons from Schön's IslandLessons from Schön s IslandLessons from Schön s IslandUse reflection carefully: Enough
t l i t t t Use reflection carefully: Enough
t l i t t t external input to correct inaccuracies of self-judgment; not so much it deforms learning
external input to correct inaccuracies of self-judgment; not so much it deforms learning
Think of teacher as coach who can help a student (or a MD in Think of teacher as coach who can help a student (or a MD in can help a student (or a MD in practice) be sure they know “when to slow down” or “when to look it up”
can help a student (or a MD in practice) be sure they know “when to slow down” or “when to look it up”to look it up
Use “self-selected external t ” d f ll
to look it up
Use “self-selected external t ” d f ll assessments” and follow-up on
learning plansassessments” and follow-up on learning plans
I hope you have enjoyed I hope you have enjoyed I hope you have enjoyed the trip!
I hope you have enjoyed the trip!
BibliographyBibliographyAilia Athena (2007) Women in Greek Myths, http://www.paleothea.com/index.htmlBarzun J. (1988) Multiple Choice flunks out. New York Times, October 11. Cathell DW. (1890) Book on the physician himself. FA Davis, Philadelphia and London.Davis DA MA Thomson AD Oxman and RB Haynes (1995) Changing physician
Ailia Athena (2007) Women in Greek Myths, http://www.paleothea.com/index.htmlBarzun J. (1988) Multiple Choice flunks out. New York Times, October 11. Cathell DW. (1890) Book on the physician himself. FA Davis, Philadelphia and London.Davis DA MA Thomson AD Oxman and RB Haynes (1995) Changing physician Davis DA, MA Thomson, AD Oxman, and RB Haynes. (1995). Changing physician
performance: A systematic review of the effect of continuing medical education strategies. JAMA274(9) (6 Sept.): 700–5.
Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. (2006) Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA.. 296(9):1094-102.
Davis DA, MA Thomson, AD Oxman, and RB Haynes. (1995). Changing physician performance: A systematic review of the effect of continuing medical education strategies. JAMA274(9) (6 Sept.): 700–5.
Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. (2006) Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA.. 296(9):1094-102.y ( )
Eva KW. (2003) On the generality of specificity. Medical Education. 37, 587-588.Eva KW, Cunnington JP, Reiter HI, Keane DR and Norman GR. (2004) How can I know what
I don’t know? Advances in Health Sciences Education. 9, 211-24.Eva KW and G. Regehr. (2005). Self-assessment in the health professions: A reformulation
and research agenda Academic Medicine 80(10 Suppl): S46–54
y ( )Eva KW. (2003) On the generality of specificity. Medical Education. 37, 587-588.Eva KW, Cunnington JP, Reiter HI, Keane DR and Norman GR. (2004) How can I know what
I don’t know? Advances in Health Sciences Education. 9, 211-24.Eva KW and G. Regehr. (2005). Self-assessment in the health professions: A reformulation
and research agenda Academic Medicine 80(10 Suppl): S46–54and research agenda. Academic Medicine 80(10 Suppl): S46 54.Eva K and Regehr G. (2007). Knowing when to look it up: A new conception of self-
assessment ability. Academic Medicine, 82(10) October 2007 Supplement, In press.Foucault M. (1969) The Archaeology of Knowledge and the Discourse on Language.
Pantheon Books: New York.Foucault M (1965) Madness and Civilization; A History of Insanity in the Age of Reason
and research agenda. Academic Medicine 80(10 Suppl): S46 54.Eva K and Regehr G. (2007). Knowing when to look it up: A new conception of self-
assessment ability. Academic Medicine, 82(10) October 2007 Supplement, In press.Foucault M. (1969) The Archaeology of Knowledge and the Discourse on Language.
Pantheon Books: New York.Foucault M (1965) Madness and Civilization; A History of Insanity in the Age of Reason Foucault M. (1965) Madness and Civilization; A History of Insanity in the Age of Reason.
Random House: New York, USA.Foucault, M. (1995). Discipline and punish: The birth of the prison. (A. M. Sheridan,
Trans.). New York: Vintage Books. (Original work published in 1975).Hanna M and Fins JJ. (2006) Why simulation training ought to be complemented by
experiential and humanist learning Academic Medicine 81(3):265-270
Foucault M. (1965) Madness and Civilization; A History of Insanity in the Age of Reason. Random House: New York, USA.
Foucault, M. (1995). Discipline and punish: The birth of the prison. (A. M. Sheridan, Trans.). New York: Vintage Books. (Original work published in 1975).
Hanna M and Fins JJ. (2006) Why simulation training ought to be complemented by experiential and humanist learning Academic Medicine 81(3):265-270experiential and humanist learning. Academic Medicine. 81(3):265 270.experiential and humanist learning. Academic Medicine. 81(3):265 270.
Harden RM and Gleason FA. (1979) Assessment of clinical competence using an objective structured clinical examination. Medical Education. 13:41-47.
Harden RM and Gleason FA. (1979) Assessment of clinical competence using an objective structured clinical examination. Medical Education. 13:41-47.
Harden R. (2005) Interview. Recorded at Dundee Scotland. January 2005. Unpublished.
Hodges B, Regehr G, McNaughton N, Tiberius R, Hanson M. (1999) OSCE checklists do not capture increasing levels of expertise. A d i M di i 74(10) 64 69
Harden R. (2005) Interview. Recorded at Dundee Scotland. January 2005. Unpublished.
Hodges B, Regehr G, McNaughton N, Tiberius R, Hanson M. (1999) OSCE checklists do not capture increasing levels of expertise. A d i M di i 74(10) 64 69 Academic Medicine, 74(10):64-69
Hodges B, Regehr G, Martin D. (2001) Difficulties in recognizing one’s own incompetence: Doctors who are unskilled and unaware of it. Academic Medicine. 76(10 Suppl):S87-9.
Academic Medicine, 74(10):64-69 Hodges B, Regehr G, Martin D. (2001) Difficulties in recognizing
one’s own incompetence: Doctors who are unskilled and unaware of it. Academic Medicine. 76(10 Suppl):S87-9.
Hodges B. (2003a) OSCE! Variations on a theme by Harden. Medical Education. 37 (12):1134-1140.
Hodges B. (2003b) Validity and the OSCE. Medical Teacher. 25(3), 250-254.
Hodges B. (2003a) OSCE! Variations on a theme by Harden. Medical Education. 37 (12):1134-1140.
Hodges B. (2003b) Validity and the OSCE. Medical Teacher. 25(3), 250-254.
Hodges B. (2004) Medical student bodies and the pedagogy of self-reflection, self-assessment and self-regulation. Journal of Curriculum Theory. 20(2), 41-51.
Hodges B. (2005a) The birth and world-wide adoption of the OSCE: A d d i hi t i l t d A i ti f M di l
Hodges B. (2004) Medical student bodies and the pedagogy of self-reflection, self-assessment and self-regulation. Journal of Curriculum Theory. 20(2), 41-51.
Hodges B. (2005a) The birth and world-wide adoption of the OSCE: A d d i hi t i l t d A i ti f M di l An advanced socio-historical study. Association for Medical Education in Europe, Amsterdam.
Hodges B. (2005b) The many and conflicting histories of medical education in Canada and the United States: An introduction to the paradigm wars Medical Education 39:613 621
An advanced socio-historical study. Association for Medical Education in Europe, Amsterdam.
Hodges B. (2005b) The many and conflicting histories of medical education in Canada and the United States: An introduction to the paradigm wars Medical Education 39:613 621paradigm wars. Medical Education. 39:613-621.paradigm wars. Medical Education. 39:613-621.
Kasper DL (Ed). (2005) Harrison's principles of internal medicine.16th ed. New York: McGraw-Hill.
Miller G (1990) The Assessment of Clinical Skills/Competence/Performance
Kasper DL (Ed). (2005) Harrison's principles of internal medicine.16th ed. New York: McGraw-Hill.
Miller G (1990) The Assessment of Clinical Skills/Competence/Performance Miller G. (1990) The Assessment of Clinical Skills/Competence/Performance. Academic Medicine. 65(9):S63-S67.
Moulton CA, Regehr G, Mylopolous M &MacRae H. (2007) Slowing down when you should: A new model of expert judgment. Academic Medicine, 82(10), October Supplment, In press.
Miller G. (1990) The Assessment of Clinical Skills/Competence/Performance. Academic Medicine. 65(9):S63-S67.
Moulton CA, Regehr G, Mylopolous M &MacRae H. (2007) Slowing down when you should: A new model of expert judgment. Academic Medicine, 82(10), October Supplment, In press.
Nelson S and Purkis ME. (2004) Mandatory reflection: the Canadian reconstitution of the competence nurse. Nursing Inquiry. 11(4):247-257.
Norman GR, van der Vleuten CP, De Graaff E. (1991) Pitfalls in the pursuit of objectivity: issues of validity, efficiency and acceptability. Medical Education. 25 (2):119-26.
Nelson S and Purkis ME. (2004) Mandatory reflection: the Canadian reconstitution of the competence nurse. Nursing Inquiry. 11(4):247-257.
Norman GR, van der Vleuten CP, De Graaff E. (1991) Pitfalls in the pursuit of objectivity: issues of validity, efficiency and acceptability. Medical Education. 25 (2):119-26.Education. 25 (2):119 26.
Norman GR. (1999). The adult learner: A mythical species. Academic Medicine 74(8): 886–9.
Norman G. (2005) Editorial--checklists vs. ratings, the illusion of objectivity, the demise of skills and the debasement of evidence. Adv Health Sci Educ Theory Pract 10 (1):1 3
Education. 25 (2):119 26.Norman GR. (1999). The adult learner: A mythical species. Academic
Medicine 74(8): 886–9.Norman G. (2005) Editorial--checklists vs. ratings, the illusion of objectivity,
the demise of skills and the debasement of evidence. Adv Health Sci Educ Theory Pract 10 (1):1 3 Theory Pract. 10 (1):1-3
Ontario Ministry of Education. (1980) Issues and Directions. Government of Ontario, Toronto.
Royal College of Physicians and Surgeons of Canada (RCPSC). (2006) Maintenance of Competence Program Guide.
Theory Pract. 10 (1):1-3 Ontario Ministry of Education. (1980) Issues and Directions. Government of
Ontario, Toronto.Royal College of Physicians and Surgeons of Canada (RCPSC). (2006)
Maintenance of Competence Program Guide. a te a ce o Co pete ce og a Gu dehttp://rcpsc.medical.org/main_e.php. Accessed September 1, 2006.
Schon DA. (1987). Educating the reflective practitioner: Toward a new design for teaching and learning in the professions. San Francisco, CA: Jossey-Bass.
a te a ce o Co pete ce og a Gu dehttp://rcpsc.medical.org/main_e.php. Accessed September 1, 2006.
Schon DA. (1987). Educating the reflective practitioner: Toward a new design for teaching and learning in the professions. San Francisco, CA: Jossey-Bass.
Segouin C and Hodges B. (2005) Educating Physicians in France and Canada: Are the differences based on evidence
Segouin C and Hodges B. (2005) Educating Physicians in France and Canada: Are the differences based on evidence or history? Medical Education 39:1205-1212.
Shorter, E. (1985). Doctors and their patients: A social history. New York: Simon and Schuster.
Schurwirth LWT and van der Vleuten CPM (2006) A plea for
or history? Medical Education 39:1205-1212.Shorter, E. (1985). Doctors and their patients: A social
history. New York: Simon and Schuster.Schurwirth LWT and van der Vleuten CPM (2006) A plea for Schurwirth LWT and van der Vleuten CPM. (2006) A plea for
new psychometric models in educational assessment. Medical Education. 40:296-300
van de Vleuten CPM. (2000) Validity of final examinations in d d t di l t i i B iti h M di l J l
Schurwirth LWT and van der Vleuten CPM. (2006) A plea for new psychometric models in educational assessment. Medical Education. 40:296-300
van de Vleuten CPM. (2000) Validity of final examinations in d d t di l t i i B iti h M di l J l undergraduate medical training. British Medical Journal.
321:1`217-1219.van der Vleuten CPM, Norman GR, De Graaff E. (1991) Pitfalls
in the pursuit of objectivity: issues of reliability. Medical
undergraduate medical training. British Medical Journal. 321:1`217-1219.
van der Vleuten CPM, Norman GR, De Graaff E. (1991) Pitfalls in the pursuit of objectivity: issues of reliability. Medical p j y yEducation. 25(2):110-8.
Wallace P. (1997) Following the treads of innovation: The history of standardized patients in medical education. Caduceus. Vol. 13(2), 5-28.
p j y yEducation. 25(2):110-8.
Wallace P. (1997) Following the treads of innovation: The history of standardized patients in medical education. Caduceus. Vol. 13(2), 5-28.Caduceus. Vol. 13(2), 5 28.Caduceus. Vol. 13(2), 5 28.