Dr. Ching Ling

Prof(Addl)

Dept. of Obs & Gyn

ObjectivesDefine OSCE/OSPE

Difference between OSCE and Conventional exams

Realize the circumstances that necessitated introduction of OSCE

Advantages and Disadvantages of OSCE

Planning an OSCE/OSPE

Traditional Long CaseOne hour with patient

History taking and examination not observed

Interrogated for 20 minutes

No structured questioning or marking by examiners

Problems with Oral ExamsLack of Validity

Lack of Reliability

Lack of Objectivity

Lack of ValidityTeachers of opinion it tests recall and not depth of knowledge. Clinical competencies not measured.

Students of opinion that scores depend on whether they are examined at beginning or end of session. Ridiculing students answer is not uncommon

Lack of ReliabilityInter examiner reliability is low

Evans et al showed that marks awarded correlated positively with no. of words used and speaking time of students and negatively with that of the examiner

The oral marks did not show any correlation between the same students marks on an objective test on the same content material

Lack of ObjectivityLack of clarity in questioning and questions of variable levels of difficulty are common.

Since questions posed to candidates are not uniform and there are no guidelines for marking, scores given to candidates are arbitrary

Sometimes the examiner expect a predetermined correct answer and are unwilling to consider the candidate’s answer as a possible alternative

Problem With Oral ExamVariability – Student, Examiner, Patient are the three

Variables.

Patient Variability:

The disease he is suffering from

His temperament

His attitude towards his disease

The process of examination

Problem With Oral ExamExaminer Variable:

Marks awarded by one examiner differs from another for the same performance

Marks awarded by same examiner, for similar competence, on same day varies

Hence the patient and examiner variable should be reduced to a minimum

Problem With Oral ExamCompetencies not examined:

One tests only the product not the process since marks are awarded based on overall impression candidate makes on the examiner

Little effort made to evaluate various components of clinical examination

Clinical Competencies Not

Examined 1.To obtain detailed and relevant history

2.Carry out systemic physical examination

3.Reach provisional diagnosis

4.Select appropriate investigations

5.Interpret the results of the investigations

6.Recommend and institute appropriate treatment

7.Impart education to the patient

8.Communicate effectively especially when patient suffering from diseases like cancer

Problem With Oral ExamDifficulty in Conducting Clinical Examination:

A large number of patients and examiners would be required for a given number of students to improve outcome of clinical examination

Each candidate would have to be given a large number of cases

A lot of time required

OSCE/OSPEHarden and Gleeson in 1979 originally described it.

Now used in the quantitative assessment of competence in the field of undergraduate and postgraduate medical education

Definition

A method of clinical/practical examination where predetermined decisions are made on the competencies to be tested and checklists incorporating important evaluable skills are prepared

MethodologyCandidates rotate sequentially around a series of

structured cases located in ‘stations’

Specific tasks have to be performed in each station(history taking, examination of a patient or practical skill) An observer observes and marks on a checklist or rating scale

Some stations candidate has to write answers to questions. X rays, specimen or instrument may be kept. These stations does not have an examiner or observer

MethodologyStations are of various types

� Procedural station

� Linked station

� Must pass station

� Rest station

The Exam Day1 Registration

2Orientation

3Escorting to exam position

4Station instruction time

5The encounter

6Post encounter period

7Exam ends/Escorting to dismissal area

Station ProfileInstruction to students:

� Patient profile

� Time duration to do the task

� An observer, using a checklist, will assess your performance while you interact with the patient

Station ProfileBriefing to the observer:

What the student is required to assess

Observe the students using the following checklist, without any probing or interference from you

Station ProfileBriefing to the Patient:

Do not give more information than asked for

Try to answer the questions as clearly as possible

Try to be consistent with all the students as they interview you

Real patientsConsistency- must give same history each time

Can fall sick

Develop new signs/ lose old ones

Can get tired (10 students a day)

Simulated patientsConsistency- reliability

Training

Briefing

Database of actors

Scenarios in advance

Practice with each other and with examiner

ASSESSMENTThe students answer sheets and the examiners checklists are marked according to previously agreed scheme by all the examiners

The fundamental principle being every candidate completes the same assignments in the same amount of time and is marked according to a structured marking schedule

Scoring with ChecklistAdvantages:

� Helps examiner know what station designers are looking for

� Helps examiners be objective

� Facilitates use of non expert examiners

Disadvantages:

� Rewards primarily process/thoroughness

� May not sufficiently reward exceptional candidates

� Ignores examiners expertise

Global ScoringExcellent: Perfectly performed clinical task with no omission; no unnecessary procedure done

Proficient: Performed clinical task satisfactorily with minor mistakes

Average: Performed the task at an acceptable standard with some mistakes and unnecessary procedure

Poor: Failed to carry out the task. Lot of mistakes and misconception

Global ScoringAdvantages:

� Utilizes expertise of examiners

� Examiners are in a position to make a judgement about the performance

Disadvantages:

� Examiners have to be expert examiners i.e. trained

� Examiners have to be familiar with expected standards for the level of the test

OSCEObjective: Examiners use a checklist for evaluating trainees

Structured: Trainee sees the same problem and performs the same tasks in the same time frame

Clinical: The tasks are representative of those faced in real clinical situations

OSPEBasic Medical Sciences

A multi station, multi task process of assessment

Reflect the objectives – construct validity

Understand and apply theory

Test practical skills

Developing clinical skills – related to Basic Medical Sciences

Advantages of OSCE/OSPEReliable

1. Consistent scoring by examiners according to predetermined criteria

2.Candidates performs tasks across clinical, practical and communication skill domains- more reliable picture of candidates overall competence

3. Each candidate examined by number of different examiners at different stations, so multiple independent observations are collated. Individual examiner bias is thus attenuated

Advantages of OSCE/OSPE� Valid, Reliable, Objective

� Consistent examination scenarios for students

� Uniform marking scheme for examiners

� Generates feedback for both learners and teachers

� More students assessed at a given time

� Students clinical and interactive competencies examined

Advantages of OSCE/OSPE� Examiners are required only at certain stations

� Junior examiners may be appointed for other stations as checklists are provided

� Reproducible

� Can be used with large number of students

� The variable of the examiner and the patient are to a large extent removed

� Activity within the Department which promotes teamwork

Disadvantages of OSCE/OSPE� Requires an extensive amount of organizing

� Expensive in terms of manpower, resource and time

� Students skill assessed in compartments

� Observer fatigue

� Needs space

� Training needed

Educational ImpactStudents focus on their assessments rather than the learning objectives of the course

There is a danger in the use of detailed checklists as this may encourage students to memorize the steps in a checklist rather than learn and practice the skill.

To Sum Up

OSCE can assess clinical, communication and practical skills but are still situated in the context of an examination setting. To assess doctors in the context of their professional practice requires the use of different formats in the workplace

OSCE

OSCE has proved to be so effective that it is now being adopted in disciplines other than medicine like dentistry, nursing, midwifery, pharmacy and even engineering and law

Objective Structured Long

Examination Record (OSLER)Developed by Gleeson in 1997 to improve objectivity, reliability and validity of long case examination.

History taking: 1. Pace and clarity of presentation

2. Communication process

3. Systematic approach

4. Establishment of case facts

OSLER Physical Examination: 5. Systematic approach

6. Examination technique

7.Correct physical findings

Management: 8. Appropriate investigations

9. Appropriate management

10. Clinical acumen

OSLEREach item graded as P+ (Very Good)

P (Pass)

P-(Below Pass)

Final grade for each item is decided by conferring with co-examiners and so is an overall grade. All candidates are assessed on the same 10 items by the examiners over 20-30 minutes.

THANK YOU