Post on 25-Feb-2016
description
Implementing Milestones:Historical context, competency based medical
education, and outcomes
August 6, 2013Felix Ankel, MD
http://www.youtube.com/watch?v=Qy1dpLmJTrY&feature=related
All,We recently had a case where an intern, when asked by the nurse taking care of a patient with a NSTEMI, said it was okay for the patient to go to the bathroom off monitor. You guessed it; the patient subsequently was found in arrest in the bathroom. In our reactionary culture, some of our faculty are advocating for all decisions to go through the 3rd year resident and to make sure the intern doesn’t make decisions that adversly affect patient care. So, the questions for the collective:
• Do you have a formal statement about what interns are allowed to do?
• If not formally, do you have a process of what interns are allowed to do?
• Are the nurses the “determiners” of who to go to?
www.wordle.net
Macrotrends
Hierarachies
Story
Function
Individual Experts
Design
Autonomy, Mastery, Purpose
Competency
Wisdom of crowds
Networks
Argument
Carrots and Sticks
Knowledge
MilestonesPC1: Emergency StabilizationPC2: Performance of Focused History & Physical ExamPC3: Diagnostic StudiesPC4: DiagnosisPC5: PharmacotherapyPC6: Observation and ReassessmentPC7: DispositionPC8: Multi-tasking (Task-switching)
MilestonesPC9: General Approach to Procedures
PC10: Airway Management
PC11: Anesthesia and Acute Pain Management
PC12: Other Procedures: Ultrasound (Diagnostic / Procedural)
PC13: Other Procedures: Wound Management
PC14: Other Procedures: Vascular Access
Milestones• MK - Medical Knowledge• PROF1 - Professional values• PROF2 - Accountability• ICS1 - Patient Centered Communication• ICS2 - Team Management • PBLI - Practice Based Performance Improvement• SBP1 - Patient Safety• SBP2 - Systems-based Management• SBP3 - Technology
Dreyfus model of skill acquisition
Dreyfus model of skill acquisition
Dreyfus model of skill acquisition
Next Accreditation System
16
Continuous Oversight &
Improvement Emphasis
Milestone Reporting
(semi-annually)
Case LogsResident &
Faculty OpinionsProgram &
Institutional Information
Overall Milestone Project Goal
17
Obtain outcome measures (i.e., milestones of competency
development) to use as evidence of programs’ educational effectiveness
“Early Adopters”
Emergency Medicine
Internal Medicine
NeurosurgeryOrthop
edics
Pediatrics
Radiology
Urology
Emergency Medicine Firsts• Developed and completed Milestones in 5 months
– Approved by ABEM BOD 1/2012, and by RRC-EM 2/2012
• Based on Milestone progress, EM invited into NAS trial rollout July 2013
• Milestones are truly along a continuum of end of medical school to certification standards– Only specialty to take ABMS certification standards and apply to Milestones
• Milestones are based on extensive survey data related to ABEM certification standards
• Only specialty allowed to make revisions in program requirements– Only specialty to integrate the Milestones into proposed program requirement
changes
Uniformity of Milestone Reporting• 5 levels of proficiency
– Novice, Beginner, Competent, Proficient, Expert• Level 4 - The ABEM certification standard
– By definition where an individual should be at time of graduation
• Level 5 - Attained after practice• Narrative anchors• Based on knowledge, skills and abilities (KSAs)• Balance between
– Deconstruction (microtasks)– Integration (complex performance)
What are the characteristics of a highly functioning CBME system?
Carraccio C, Wolfshtal SD, Englander R, Ferentz K, Martin C. 2002. Shifting paradigms: from Flexner to competencies. Acad Med 77(5):361-367. p 362.
Structure- and process-based
Competency-Based
Driving Force: Curriculum Content Outcome
Driving Force: Process Teacher Learner
Path of learning Hierarchical Non-hierarchical
Responsibility for content Teacher Student and teacher
Goal for educational encounter Knowledge acquisition Knowledge application
Program completion Fixed time Variable time
Carraccio C, Wolfshtal SD, Englander R, Ferentz K, Martin C. 2002. Shifting paradigms: from Flexner to competencies. Acad Med 77(5):361-367. p 362.
Structure- and process-based
Competency-Based
Driving Force: Curriculum Content Outcome
Driving Force: Process Teacher Learner
Path of learning Hierarchical Non-hierarchical
Responsibility for content Teacher Student and teacher
Goal for educational encounter Knowledge acquisition Knowledge application
Program completion Fixed time Variable time
Carraccio C, Wolfshtal SD, Englander R, Ferentz K, Martin C. 2002. Shifting paradigms: from Flexner to competencies. Acad Med 77(5):361-367. p 362.
Structure- and process-based
Competency-Based
Driving Force: Curriculum Content Outcome
Driving Force: Process Teacher Learner
Path of learning Hierarchical Non-hierarchical
Responsibility for content Teacher Student and teacher
Goal for educational encounter Knowledge acquisition Knowledge application
Program completion Fixed time Variable time
Carraccio C, Wolfshtal SD, Englander R, Ferentz K, Martin C. 2002. Shifting paradigms: from Flexner to competencies. Acad Med 77(5):361-367. p 362.
Structure- and process-based
Competency-Based
Driving Force: Curriculum Content Outcome
Driving Force: Process Teacher Learner
Path of learning Hierarchical Non-hierarchical
Responsibility for content Teacher Student and teacher
Goal for educational encounter Knowledge acquisition Knowledge application
Program completion Fixed time Variable time
Microsystem success and assessment system correlates
Microsystem success characteristic
Assessment system correlates
Information and information technology
Portfolio, preferably electronic
Leadership of microsystem Clerkship and program directorsMacrosystem support of microsystem
Support and resources from department chair and institution
Patient focus Appropriate clinical experiences; measuring patient experience
Staff focus Faculty development in assessment; involvement of non-physicians in assessment
Interdependence of care team Working in interdisciplinary teams; teamwork competence
Process improvement Continuous quality improvement of assessment methods and training tools
Education and training Competency-based; developmental clinical experiences; milestones and benchmarks
Performance results Outcomes of training; at minimum, competence needed to advance to next stage
• EM used as pilot specialty in NAS– Began July, 2013
• Integration of Milestones into EM Program Requirements– A first!
• Development of assessment methodology– Specialty-wide implementation of assessment methods?
JMTF?
Next Steps
50
Next Accreditation System
Reliabilities across methodsslide created by Cees van der Vleuten 2012
TestingTime inHours
1
2
4
8
MCQ1
0.62
0.76
0.93
0.93
Case-BasedShort
Essay2
0.68
0.73
0.84
0.82
PMP1
0.36
0.53
0.69
0.82
OralExam3
0.50
0.69
0.82
0.90
LongCase4
0.60
0.75
0.86
0.90
OSCE5
0.47
0.64
0.78
0.88
1Norcini et al., 19852Stalenhoef-Halling et al., 19903Swanson, 1987
4Wass et al., 20015Petrusa, 2002
Macrotrends• From hierarchies to networks (CoP)• From individual experts to wisdom of crowds (CCC)• From knowledge to competency (CBME)• From carrots and sticks to autonomy, mastery, and
purpose (Dreyfus and Dreyfus)• From function to design (CLER visit)• From argument to story (narrative anchor)