USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for...
Transcript of USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for...
![Page 1: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/1.jpg)
USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ASSESSMENT OF NAS MILESTONES CAROLYN STALVEY, MD ERIK W. BLACK, PHD NICOLE M. PARADISE BLACK, MD MED
![Page 2: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/2.jpg)
OVERVIEW
![Page 3: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/3.jpg)
Interns by Department
Anesthesiology (n=12)
Emergency Med (n=8)
Family Medicine (n=10)
Internal Medicine (n=40)
Categorical (n=24)
Preliminary (n=16)
Neurosurgery (n=3)
Obstetrics and Gynecology (n=3)
Otolaryngology (n=2)
Orthopedics (n=4)
Pediatrics (n=17)
Psychiatry (n=8)
Surgery (n=13)
Urology (n=3)
Interns tested: N = 123
![Page 4: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/4.jpg)
Stations Details Categories Post-Encounter
Chest Pain unstable angina with SP
Hx, PE, Communication Differential and Read EKG
SOB Pulmonary embolism with SP
Hx, PE, Communication SOAP note
RLQ Pain Ectopic pregnancy with SP
Hx, PE, Communication Handoff note
Mock Page Answer page, Infected PICC line with SN
Interprof Comm, Hx, Clinical Reasoning
![Page 5: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/5.jpg)
Stations Details Categories Post-Encounter
Discharge Pneumonia Discharge instruction with SP
Education, Communication
Discharge Summary, CXR
Medical Error Disclosure medication error to SP
Education, Safety, Communication
Documentation note
Sterile Gown & Glove Nurse/grader in room Safety
Informed Consent Blood transfusion, with SP
Education, Communication
![Page 6: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/6.jpg)
Test Blueprint - Categories
Case Name History Physical Exam Patient Safety Patient Education
InterProfession Communication
Clinical Decision Making
Communication Skills
Chest Pain X X X X X Gowning and
Gloving X Informed Consent X X
Medical Error X X X Mock Page X X X Discharge/ Pneumonia X X
RLQ Pain X X X X Shortness of
Breath X X X X
![Page 7: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/7.jpg)
Test Blueprint – Notes, other skills
Case Name SOAP Note Discharge Note Documentation of Error Handoff Note EKG
Interpretation X-ray
Interpretation
Chest Pain X Medical Error X
Discharge/ Pneumonia X X
RLQ Pain X Shortness of
Breath X
![Page 8: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/8.jpg)
Evaluators
Standardized patients/nurses History Physical Exam Communication items
OSCE Committee Notes EKG CXR
![Page 9: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/9.jpg)
Milestones Addressed - IM
Case Name IM Milestones
Chest Pain 2MK: Knowledge of core content; 1PC: Diagnostic tests; 1PC Historical Data Gathering; 1PC Physical Examination; 4ICS Communicate Effectively;
Gowning and Gloving 6SBP: Recognizes systems error and advocated for system improvement
Informed Consent 1PC: Patient Management; 4ICS Communicate Effectively; 4ICS: Demonstrate Compassion and Respect
Medical Error 5P: Adhere to Basic Ethical Principles; 4ICS: Communicate Effectively
Mock Page 4ICS Interprofessional Team;1PC Clinical Decision Making; 1PC: Patient Management
Discharge/ Pneumonia
2MK: Knowledge of core content; 4ICS Transition of Care, Health Records; 1PC Clinical Reasoning
RLQ Pain 2MK: Knowledge of core content; 4ICS Transition of Care, Health Records; 1PC Historical Data Gathering; 1PC Physical Examination
Shortness of Breath 2MK: Knowledge of core content; 4ICS Transition of Care, Health Records; 1PC Historical Data Gathering; 1PC Physical Examination
![Page 10: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/10.jpg)
Milestone Alignment
High frequency Milestones (IM) 1. Works effectively within an Interprofessional
Team (9) 2. Improves via Self Assessment (7) 3. Diagnostic Tests (6) 4. Participates in the education of all members
of health care team (6) 5. Communicates Effectively (6) 6. Demonstrates Compassion and Respect to
Patients (6)
![Page 11: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/11.jpg)
PERFORMANCE SUMMARY
![Page 12: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/12.jpg)
Performance by Category
Category Items MEAN SD
Patient Safety 15 87.99 22.50
Communication Skills 84 81.78 9.93
History 40 76.76 15.80
Physical Examination 30 74.43 21.00
Patient Education 29 73.44 19.40
Interprofessional Communication 2 60.97 26.80
Clinical Decision Making 7 45.12 25.00
![Page 13: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/13.jpg)
Post Encounter
Case Post Encounter Mean SD
Chest Pain Differential Diagnosis 82.16 20.61
SOB SOAP Note 77.04 17.75
Pneumonia Interpret X-ray 75.10 43.19
RLQ Pain Handoff Note 67.56 16.21
Pneumonia Discharge Note 65.75 18.72
Error Documentation of Error 57.56 22.42
Chest Pain Interpret EKG 49.96 25.61
![Page 14: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/14.jpg)
HCAHPS Items
Case Name Patient Education
Communication Skills
Chest Pain
X
Gowning and Gloving
Informed Consent X
Medical Error X
Mock Page
Pneumonia X
X
RLQ Pain
X
Shortness of Breath
X
1. Description symptoms to look for post discharge
2. Description of medication side effects
3. Physician listened 4. Physician
explained 5. Physician respected
![Page 15: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/15.jpg)
HCAHPS
Item (Scale: 1 – 5) Mean Physician listened 3.67
Physician explained 3.60
Physician respected 3.84
Item (Yes/No) Frequency The resident told me about possible side effects of diarrhea or loose stool. 93%
The resident told me to seek medical attention if I have chills or fever higher than 101.5. 40%
The resident told me to contact my primary care doctor if I get worse. 37% The resident told me to seek medical attention if I have problems breathing. 33%
The resident told me to seek medical attention for chest pain. 25%
The resident told me to seek medical attention if my cough gets worse. 21%
![Page 16: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/16.jpg)
General Areas of Weakness
Discharge Instructions symptoms to watch for smoking cessation (done 76%)
Sterile Gown and Glove Nearly all contaminated themselves at some point
Informed Consent Often don't mention it is ok to refuse transfusion
(done 80%) Allergic reactions (done 87%, 59%)
![Page 17: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/17.jpg)
General Areas of Weakness
Handoffs What needs to be followed up What to do if..
Pulmonary exam (65%) Medical Error
Many don't ask if patient understood (done 44%) Neglecting to say it will be investigated and will get
back to you (done 73%)
![Page 18: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/18.jpg)
UF vs. Non-UF Graduates
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Informed Consent
Medical Error
RLQ Pain Chest Pain Gown and Glove
Shortness of Breath
(PE)
Pneumonia Mock Page
Mea
n Sc
ore
UF
Non-UF
p =.02
p < .01
![Page 19: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/19.jpg)
UF vs. Non-UF Graduates
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Mea
n Sc
ore
UF
Non-UF
p = .05
![Page 20: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/20.jpg)
PD Reactions
Program directors described the OSCE as a positive experience
Helpful method of evaluating residents and identifying issues
A strong majority expressed interest in a retest (particularly primary care)
![Page 21: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/21.jpg)
Next Steps
Continuous improvement and evaluation of OSCE 1 Remediation? OSCE 2 Continue to increase alignment with Milestones
![Page 22: USING A COMPREHENSIVE INTERN OSCE FOR THE INITIAL ...€¦ · using a comprehensive intern osce for the initial assessment of nas milestones carolyn stalvey, md erik w. black, phd](https://reader034.fdocuments.us/reader034/viewer/2022042804/5f583e2f528a9d5cc421dead/html5/thumbnails/22.jpg)
Thank you