meu workshop Osce ospe an introduction

MEU WORKSHOP

OSCE / OSPE

An introduction

OSPE/ OSCE – an introduction

Session Objectives

At the end of this session, you must know

• WHAT the abbreviation “OSCE/OSPE” stands for

• HOW an OSPE/OSCE is organized

• The STRENGTHS/ WEAKNESSES of the

OSCE/OSPE system

EVALUATION of educational inputs

Evaluation consists of finding out, or measuring

the extent to which educational objectives have

been attained

Objectives

Educational objectives in the medical setting

usually aim at improving ability in THREE areas

• COGNITIVE, aiming at increasing

KNOWLEDGE, also the ability to SOLVE a

PROBLEM using the INTELLECT

• AFFECTIVE, aiming at the ability to develop,

use and convey the right ATTITUDE in a given

setting

Objectives

• PSYCHOMOTOR, aiming at enhancing the

SKILLS of the student in order that he or she

should be able to DO, or PERFORM a TASK

Evaluation

EVALUATION of an objective in the COGNITIVE

domain is done by measuring the change in the

KNOWLEDGE of the student

This is done by administering

• Essay questions,

• MCQs,

• Short notes ……

in the THEORY examination

Evaluation

EVALUATION of an objective in the

PSYCHOMOTOR domain is done by measuring

the SKILL of the student, in his ability to perform

a given TASK

Evaluation

TASKS in a medical setting can be of various

types, but are generally

A. Tasks related to clinical or practical

PROCEDURES and EXAMINATIONS, like

drawing blood from a patient, or examining for

the presence of hepatomegaly, or performing a

surgical operation, or staining a slide

B. Tasks related to COMMUNICATION,

like taking a history or counselling a patient

Evaluation

C. Tasks related to INTERPRETATION and

ANALYSIS, like reading and interpreting an

X Ray picture, or a laboratory report – partly

knowledge based, partly skill based

Evaluation

These abilities are generally assessed in the

PRACTICAL or CLINICAL examination

This afternoon’s session

will focus on this

Evaluation

As it exists currently, the PRACTICAL or

CLINICAL examination in a medical college

usually comprises bedside clinical examinations,

laboratory based examinations and other forms of

test and exercises

We will now OBSERVE a typical bedside clinical

examination of two students

Evaluation

Watch the ensuing role play closely and answer

some questions after it is over…...

Evaluation

The role play…..

Evaluation

In the examination you witnessed, the aspects of

the objective being tested (probably) were

a. Ability to take a history (Communication/

Knowledge)

b. Ability to check for the presence of icterus

(Skill/ Knowledge)

Evaluation

The QUESTIONS which arise are:

Q1. If the student HAS ATTAINED the objective,

is there a good chance that this test/exam/method

will recognize this ?

Is the method VALID ?

Evaluation

Q2. If two students have attained the objective to

the same extent, does the test/exam/method give

the SAME RESULT on repeated measurement ?

Is the method RELIABLE ?

Evaluation

Q3. Is it ONLY the extent of attainment of

objective that is being measured, or do other things

influence the result of the test /exam /method ?

Is the method OBJECTIVE ?

Evaluation

Since the answer to these questions, most of the

time, is variable, there is a need for an alternate

test/exam/system which is VALID, RELIABLE

and OBJECTIVE

Presenting….

The

Objective O

Structured S

Practical / Clinical P/C

Examination E

OSCE/ OSPE

What is an OBJECTIVE, STRUCTURED

CLINICAL EXAMINATION ?

Let us understand the answer to this question in

the following steps:

OSCE/ OSPE

STEP ONE - Clarity on OBJECTIVES

Since an evaluation is a measurement of the extent

to which an objective has been attained, it follows

that we must have clear OBJECTIVES in order to

have clear EVALUATIONS

Example of an objective:

The Student must be able to diagnose and treat a

case of Hypothyroidism

“The Student must be able to diagnose and treat a case of

Hypothyroidism”

STEP TWO – Identify the PRACTICAL aspects

of the objective

If the objective is to be attained, the student needs

to have KNOWLEDGE, which will be tested in

the THEORY examination.


Hypothyroidism”

STEP TWO – Identify the PRACTICAL aspects

of the objective

But in the domain of SKILLS required, we ask

the question

“If this objective is to be attained, what are the

PRACTICAL/CLINICAL tasks the student must

be able to do ?”


Hypothyroidism”

Using the same example..

In diagnosing and treating hypothyroidism, the

student must be able to do the following tasks

a. Take history from a person who presents

with symptoms suggesting hypothyroidism

b. Perform a clinical examination, including

interpreting response to the ankle jerk

c. Investigate appropriately to confirm the

diagnosis

d. Counsel the patient regarding appropriate

treatment

among other tasks…….

“The Student must be able to diagnose and treat a case of Hypothyroidism”

STEP THREE – Select which of the tasks will be

assessed in the PRACTICAL/ CLINICAL

examination.

In the example, out of the four practical tasks,

the task

“Recording response to the ankle jerk”

may be considered suitable for assessment



Hypothyroidism”

STEP FOUR – Taking each selected task, break

the task down into its constituent units

In the example of “Recording response to the

ankle jerk”, examiners may agree upon the

following:

“ If the student is able to record ankle jerk

response correctly, then he/she must be able to do

the following :


Hypothyroidism”

• Introduce herself, explain the procedure to the

patient, take his consent

• Ask the person to lie down, making sure that the

lower limb is exposed below the knee,

•Position the lower limb, flexed at the knee, everted

and slightly dorsi-flexed at the foot

• Expose the achilles tendon

• With one hand, slightly dorsi-flex the foot


Hypothyroidism”

• Strike the achilles tendon with the rubber

hammer

• Interpret the response appropriately

• Take leave of the patient, thanking him for his co

operation


Hypothyroidism”

This list constitutes the sub components, or

CHECKLIST for assessment of that particular

task.

Every broad task, like

- Recording temperature, or

- Performing a peripheral smear of blood from a

patient suspected to be having malaria,

can be broken down into a series of steps which

make up the task, when correctly performed


Hypothyroidism”

STEP FIVE – Assign scores (weights) for each of

the sub components of the checklist,

In our “response to ankle jerk..” example, if the

student introduces himself and takes consent, he

gets some marks, if he then positions the foot, he

gets some marks.. and so on

Certain sub tasks may be considered more

important than others, and are given a greater

score (weight)


Hypothyroidism”

For example

making sure that the “ Position of the lower limb,

flexed at the knee, everted and slightly dorsi-

flexed at the foot ”

may be considered more important than

“Expose the achilles tendon”


Hypothyroidism”

Hence “:position …” may be given a maximum

score of 5, while “expose the …” may be given

a score of 2

That is, if the student positions the lower limb

correctly, he is given 5, or 4.5

If he exposes the tendon correctly before

striking it, he is given a score of 2

So, the checklist may look like this ….


Task – Record Ankle jerk Time - 3 minutes Marks - 20

Specific sub tasks Total marks Obtained

• Introduce yourself, explain the procedure to 3

the patient, take his consent

• Ask the person to lie down, making sure 5

that the knee is flexed , foot is everted and slightly

dorsi-flexed

• Expose the achilles tendon 2

• With one hand, slightly dorsi-flex the foot 2

• Strike the achilles tendon with the rubber 2

hammer appropriately

• Record the response appropriately in the 5

scoring form provided

• Take leave of the patient, thanking him 1

for his co-operation

OSCE/ OSPE

STEP SIX – Set up “Stations” for assessment of

tasks

A “station” is the site at which the student is

actually assessed on a particular ability

Stations are of different types, depending on the

nature of the task to be assessed

a. History Taking stations are sites where the history

taking ability of a student is assessed. eg “This

patient complains of abdominal pain. Take a

history pertaining to abdominal pain”

OSCE/ OSPE

b. Procedure, or Practical stations are sites where the

student’s ability to perform a clinical procedure or

examination is assessed, eg “Record ankle jerk

response”

c. Communication stations are sites where the

communication ability of a student is assessed e.g

“This is a mother of a three year old child with

diarrhea. Advise her regarding use of ORT for her

child”

OSCE/ OSPE

d. Response stations are sites where the

interpretative ability of a student is assessed e.g

“ Interpret this Chest X ray of a 40 year old patient

with acute dyspnea and state 3 reasons for your

answer.”

Sometimes, REST stations are inserted in between

two task-oriented stations to give students a chance

to organise their thoughts, and to have a cup of

coffee !

OSCE/ OSPE

Stations must have the following items:

a. Instruction to the candidate, for example “Record

the ankle jerk response for this patient”

b. Materials - Table/chair/couch and other materials

as required for the task eg knee hammer

OSCE/ OSPE

c. A patient.

If the task is “Take the temperature of this 1 – year

old child”, where the emphasis is more on

performance and less on interpretation, then there

is no need to have an actual patient with fever,

though it would be good to have one.

OSCE/ OSPE

“SIMULATED” patients can be used.

Tasks related to communication and history taking

ability are also suited for the use of simulated

patients.

Simulated patients must be trained / informed of

the nature of the procedure prior to the examination

OSCE/ OSPE

However, tasks like “examine this persons

respiratory system” done in order to see if the

student can recognise an abnormal clinical finding

– eg rhonchi , or crepitations, will necessitate the

use of REAL patients with clinical problems.

OSCE/ OSPE

d. An Assessor. If the station is a procedure, history

or communication station then the EXAMINER

must be the assessor, since direct observation must

be done.

But in RESPONSE stations, there is generally no

need for an observer. If required, an assistant can be

present to ensure that the instruction is understood,

and to ensure that the exercise proceeds smoothly.


Hypothyroidism”

The TIME allotted for each task is generally

uniform.

Some tasks may require shorter periods of time,

but in general, the time allotted for each task is

fixed as the time required for the task requiring

most time.

OSCE/ OSPE

In Summary, the steps in understanding an

OSCE/OSPE are:

1. Have a set of CLEAR OBJECTIVES, and select

those which are to be assessed

2. Identify the PRACTICAL aspects of the objective,

in terms of the tasks which the student must be able

to do if the objective is to be attained

3. Of the tasks, select the TASK for assessment in

the current examination

4. Break the task down into SUB-TASKS

OSCE/ OSPE

Summary

5. Assign SCORES (WEIGHTS) for each sub task,

the total marks, and the TIME required for the task

When this has been done for all tasks/skills which

will be assessed,

6. Set up STATIONS for the OSCE / OSPE, ensuring

that each station is complete in all respects

- Instruction

- Materials

- Patient/ Simulated patient

- Assessor

Station 1 Station 2 Station 3 Station 4 Station 5

Station 10 Station 9 Station 8 Station 7 Station 6

Rest History Response Procedure Response

Response Commu -

nicationResponse Rest Procedure

OSCE/ OSPE Stations

Observer/

assessor

Decide on OBJECTIVE to be

assessed

Decide on PRACTICAL ASPECTS

of the objective

Select task which will be assessed in

today’s OSCE/OSPE

Break task down into subtasks, assign

scores, make checklist

Decide on resources required

Make station

Diagnose and treat

Hypertension

Take History, Record Blood

Pressure, Counsel reg. non-

drug interventions

Counselling reg. non-drug

interventions

Greet, assess, classify,

reinforce, listen,

summarise...

Well lit station, two chairs,

simulated patient, observer..

One communication

station…

SUMMARY

OSCE/ OSPE

We are now ready to observe a demonstration of an

OSCE/OSPE.

We will observe a demonstration of FOUR types of

assessments – History taking, Procedure,

Communication and Response

You will work in FOUR groups

OSCE/ OSPE

Group ONE

Assume that the educational objective being

assessed is

“The student must be able to diagnose and treat a

person with Coronary Heart Disease”

Practical aspects of this objective ?

Take History, examine CVS, Interpret ECG,

investigate, treat, counsel, follow-up

OSCE/ OSPE

Group One – History taking

For today’s demonstration, the task chosen will be

“This patient comes with a complaint of chest pain.

Take a history of chest pain”

OSCE/ OSPEGroup One

You will have to do the following:

A. Break the task down and prepare a checklist for

assessment, with scores - like the template shown

B. State all your requirements for a station, and

prepare a simulated patient if required

C. As the whole group observes, perform the

assessment of a student – you need to prepare a

simulated student too !

D. You will then make a presentation of the checklist

which you prepared

OSCE/ OSPE


• Introduce yourself, explain the procedure to 3

the patient, take his consent

• Ask the person to lie down, making sure 5

that the lower limb is everted and slightly

dorsi-flexed

• Expose the achilles tendon 2

• With one hand, slightly dorsi-flex the foot 2

• Strike the achilles tendon with the rubber 2

hammer appropriately

• Record the response appropriately in the 5

scoring form provided

• Take leave of the patient, thanking him 1

for his co-operation

Task – Record Ankle jerk Time - 3 minutes Marks - 20

OSCE/ OSPE


•

•

•

•

•

•

•

Task – Take H/o Chest Pain Time ………. Marks…….

OSCE/ OSPE

Group TWO


assessed is

“The student must be able to diagnose and manage a

child with diarrhea”


Take History, examination, investigate if necessary,

counsel regarding ORT use, treat, follow-up…..

OSCE/ OSPE

Group Two - Communication


“This is the parent of a 1-year old child with

diarrhea. Advise him/her regarding the use of ORT

in diarrhea”

OSCE/ OSPE

Group Two



assessment, with scores




assessment of the (simulated) student

D. You will then make a presentation of the checklist

which you prepared

OSCE/ OSPE


•

•

•

•

•

•

•

Task – Counsel re ORT use Time ………. Marks…….

OSCE/ OSPE

Group THREE


assessed is

“The student must be able to identify and correctly

interpret risk factors for Cardiovascular Disease in

adults”


Take History, examination, ask for appropriate tests,

interpret test results, counsel patient at high risk…..

OSCE/ OSPE

Group Three - Response


“This is the blood test result of a 40-year old

executive who came for a “master health check”.

The only complaint was occasional tiredness, worse

over the past 6 months. Read the report carefully

and answer the questions which follow..”

OSCE/ OSPE

Group Three - Response

Mr X Age - 40 years Sex – Male

Hemoglobin

Fasting Blood sugar

Post Prandial blood sugar

Total Cholesterol

LDL Cholesterol

HDL Cholesterol

OSCE/ OSPE

Group Three




B. State all your requirements for a station

Make a presentation of this to the larger group

OSCE/ OSPE

Group FOUR


assessed is

“The student must be able to diagnose and manage a

person with Hypertension”


Take History, record blood pressure, examination,

ask for tests if required, counsel reg non drug/drug

therapy…..

OSCE/ OSPE

Group Four - Procedure


“This patient has been complaining of giddiness.

Record his/her blood pressure”

OSCE/ OSPE

Group Four







assessment of the (simulated) student

D. You could then make a presentation of the

checklist which you prepared

OSCE/ OSPE


•

•

•

•

•

•

•

Task – Record Blood Pressure Time ………. Marks…….

OSCE/ OSPE

DEMONSTRATION……..

OSCE/ OSPE

At the end of the demonstration, note the resources

required for the exercise

1. Stations - a large hall with cubicles ?

2. Materials - equipment/ bed/ chair/ table/ X ray

films, other situation specific material

3. Personnel

- Examiners who will directly observe,

- Patients who need to be kept motivated,

- Simulated patients who need to be trained,

- Assistants

OSCE/ OSPE

4. Evaluation sheets for examiners to mark on, and

for students to write on, if required.

5. Students instruction sheets

6. Time keepers and bells

……………….

OSCE/ OSPE

But..

What did you feel about the ADVANTAGES of

OSCE/OSPE ?

About the VALIDITY of the exercise ?

The RELIABILITY of OSE ?

Its OBJECTIVITY ?

The FAIRNESS to the student ?

The RANGE OF ABILITY which can be tested ?

OSCE/ OSPE

For 4 students, S1 to S4,

who were tested on 4 tasks T1 to T4,

a summary table of results of the OSE

may be as follows:

OSCE/ OSPE

Summary Mark sheet

TASK S1 S2 S3 S4 Total Max

History 14 15 13 13 55 80 Response 15 18 19 18 70 80 Procedure 18 17 16 19 70 80 Communication 13 12 10 11 46 80 TOTAL 60 62 58 61 Max Obtainable 80 80 80 80

Conclusions Student S2 got the maximum marks Task – communication performance was the worst – do we need to do something about this ?

OSCE/ OSPE

What are the DISADVANTAGES of OSEs ?

1. Involves lots of planning

- Listing skills/knowledge to be tested

- Preparing checklists of sub skills

- Obtaining suitable material, cases, personnel

-Preparation of answer keys for stations

2. Tends to compartmentalise the skills tested,

rather than testing the entire range of skills at one

sitting

OSCE/ OSPE

•Requires time and effort on the part of the examiners, during the examination

•The “Murphy Index” (Number of things that can go wrong) is higher than for conventional examinations

…. BUT….

OSPE/OSCE

DEFINITELY worth thinking about at your

department,

and more importantly….

the RGUHS is planning to implement OSCE/

OSPE as part of its practical examination

system !!

Session Objectives

At the end of this session, you must know

• WHAT the abbreviation “OSPE” stands for

• HOW an OSPE/OSCE is organized

• The STRENGTHS/ WEAKNESSES of the

OSCE/OSPE system

Thank You !

meu workshop Osce ospe an introduction

Education

Transcript of meu workshop Osce ospe an introduction