The Irkutsk State Medical Academy for Postgraduate Education (Tempus IV 159328) STEERING COMMITTEE...
-
Upload
stephen-sherman -
Category
Documents
-
view
216 -
download
2
Transcript of The Irkutsk State Medical Academy for Postgraduate Education (Tempus IV 159328) STEERING COMMITTEE...
The Irkutsk State Medical Academyfor Postgraduate Education
(Tempus IV 159328)
STEERING COMMITTEE MEETING &
INTERNATIONAL FINAL CONFERENCE
Evaluation of Medical Student: Knowledge, know-how
and life skills
Pr Sylvain MeurisFaculté de Médecine
Université Libre de Bruxelles
Objective :To Reach Professional Competence !
Is the habitual and judicious use of:
Communication
Knowledge
Evidence-based decision-making
Technical Skills
Emotions
Values and reflection to improve the health of the individual patient and the community
Who Assess and What Should Be Assessed ?
Transmission of knowledge, know-how & life skills between teacher and student
Who : Teacher - During the training process
Academic steeplechase (Education Minister)!
Peers - End of the training process Accreditation Council for Graduate Medical Education
(Health Minister)
What : Numerous and Cumulative Competencies :
Knowledge + Practice-based learning and improvement + Interpersonal and communication skills + Ethics & Professionalism + System-based practice + Patient care,…
Is Assessment Necessary ?
No formal assessment before the Renaissance in Europe Chinese invention (VI century) to
Select Imperial officers
Imported by Portuguese Jesuits to Select Representatives for the Portuguese Colonial
Empire
Widely adopted in European administrations and Universities !!!
Probably useful
Is Assessment Necessary ?
A useful tool to : Train students and Assume graduation responsibility
Two questions: WHY do we need to assess ? WHAT do we want to measure ?
Is Assessment Necessary ?
Answers depends on the position
Teacher verify if the minimum level of “knowledge” is reached Learning strategy
Student prepare examination to reach the minimum level to be authorized to continue… Strategy to pass
WHY do we assess ?
To ensure safety of patientsour responsibility to the public
To achieve of a minimum standardresponsibility
to the candidate and
to the University !
In principle…
WHY do we assess ?
To ensure competence
As a means of academic competition
In practice: the scope…
WHY do we assess ?
Formative: to give feedback and advice When the cook tastes the soup, that's
formative
Summative: to grade When the guests taste the soup, that's
summative
Qualificative: to graduate
In practice: the purpose
When should we assess ?
WHEN should we assess ?
Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7.
Knows
Shows how
Knows how
Does
Pro
fess
iona
l aut
hent
icity
Written, Oral orComputer based assessment
Performance or hands on assessment
WHEN should we assess ?
C+
Pass/Fail
A
A-
85% F
S
Unsatisfactory
67% DC
B
93%
4/5
7/10
14/20
Certified
Qualified
Assessment is continuous (Life-Long-Learning)
HOW do we assess ? Assessment
The process of measuring something with the purpose of assigning a numerical value.
Scoring The procedure of assigning a numerical value to
assessment task. Evaluation
The process of determining the worth of something in relation to established benchmarks using assessment information.
Assessment Concerns Validity -- Is the test assessing what’s
intended? Are test items based on stated objectives? Are test items properly constructed?
Difficulty -- Are questions too easy or too hard? (e.g., 30% to 70% of students should answer a given item correctly)
Discriminability -- Are the performance on individual test items positively correlated with overall student performances? (e.g., only best students do well on most difficult questions)
Evaluation Types
Criterion-referenced evaluation -- student performance is assessed against a set of predetermined standards
Norm-referenced evaluation -- student performance is assessed relative to the other students
Criterion-Referenced Eval’s
Based on a predetermined (minimum) set of criteria.
For instance, 90% and up = A 80% to 89.99% = B 70% to 79.99% = C 60% to 69.99% = D 59.99% and below = F
Criterion-Referenced Eval’s
Pros:- Sets minimum performance expectations.
- Demonstrate what students can and cannot do in relation to important content-area standards
Cons:- Some times it’s hard to know just where to set boundary conditions
- Lack of comparison data with other students and/or schools.
Norm-referenced Evaluation Based upon the assumption of a
standard normal (Gaussian) distribution with n > 30.
Employs the z score: A = top 10% (z > +1.28) B = next 20% (+0.53 < z < +1.28) C = central 40% (-0.53 < z < +0.53) D = next 20% (-1.28 < z < -0.53) F = bottom 10% (z < -1.28)
ECTS = European Credit Transfer System
Norm-referenced Evaluation Pros:
- Ensures a “spread” between top and bottom of the class for clear grade setting- Shows student performance relative to group
Cons:
Top and bottom performances can sometimes be very close
Dispenses with absolute criteria for performance
Being above average does not necessarily imply “A” performance
In a group with great performance, some will be ensured an “F.”
Criterion and Norm Evaluation Norm-Referenced
Ensures a competitive classroom atmosphere
Assumes a standard normal distribution
Small-group statistics a problem
Assumes “this” class like all others
Criterion-ReferencedAllows for a cooperative classroom atmosphere
No assumptions about form of distribution
Small-group statistics not a problem
Difficult to know just where to set criteria
Academic Grading !
In Sweden
1 Country = 1 Grading Scale
Need to be harmonized
WHAT do we measure ?
To test not only presence of knowledge
…but also the application of knowledge
In principle…
Model of clinical assessment
Certification of competence - pass / fail a state (and legal) requirement
Grading in rank orderfor employment / placement purposes
A competition for the award of a prize
In principle…a three-fold aim
Model of clinical assessment
Measurement of:
adequacy of basic clinical skills ability to interpret clinical findings facility of communication in practical
settings ability to think analytically about diagnosis ability to discuss management logically
In practice…
Practical steps for assessment At least 6 encounters with different clinical
situations (Internal Medicine, Surgery, Pediatrics, Ob-Gyn,…)
At least 2 examiners at every encounter, each examiner giving an individual assessment
Highly structured examination and detailed assessment of skills
Examiners from other Universities for process evaluation and quality control
Set-points for Testing
Attitude to patient
Actual examination skills
Presentation of findings
Clinical judgment
Clinical examination of patients
Methodology-related Issues
Reliability
Validity
Practicality
Transparency
Organisational Issues
Resources available
Student numbers
Patient availability
Invasive procedures
Scheduling
Role of Grading
Is there need for grading ?
Selection process for employment
Selection for postgraduate training
Quality of assessment method and performance of candidates
Final question :What happens to candidates who fail ?
Review of performance – a formative exercise
Counselling at a personal level
Specific attention and individual training
Repeat assessment after a period of time
In conclusionI. Assessment of student (a future peer) is a
continuous and complex process
II. Assessment of clinical skills is Teacher, Medical School, National and Culture-Dependent
III. There is place for widespread application of agreed standard methods to assess
- clinical competences in core curricula
- additional competences essential to individual practice
IV. Teachers must give example of what are
Knowledge, Know-How and Life-Skills