ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B....
-
Upload
linette-george -
Category
Documents
-
view
213 -
download
0
Transcript of ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B....
![Page 1: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/1.jpg)
ADVANCES IN GME: Mastering Accreditation, Learner
Assessment, and the Learning Environment
Robert B. Baron, MD MSAssociate Dean, Graduate and Continuing Medical Education
Designated Institutional Official (DIO)
UCSF
![Page 2: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/2.jpg)
Disclosure:
No relevant financial relationships or
conflicts of interest
![Page 3: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/3.jpg)
Today’s Agenda
Understanding ACGME Accreditation
Understanding the Clinical Learning Environment Review (CLER)
Best Practices in Learner Assessment: Drs. Hung, Rosenbluth, Coffa
![Page 4: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/4.jpg)
Accreditation Challenges
Extra work, risk of poor outcome
Opportunity to identify and build on assets and strengthen
weaknesses
![Page 5: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/5.jpg)
Definitions
Accreditation: whether a residency or fellowship is in substantial compliance with established educational standards. Responsibility of the ACGME and its Residency Review Committees (RRCs)
Certification: whether a individual physician has met the requirements of a particular specialty. Responsibility of the member boards of the American Board of Medical Specialties (ABMS).
![Page 6: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/6.jpg)
Accreditation Basics
Each Sponsoring Institution is accredited by the ACGME
At UCSF the Sponsoring Institution is the School of Medicine (at many other institutions--including ½ of medical schools-- it is the “Teaching Hospital”)
Each Program is accredited by the ACGME
![Page 7: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/7.jpg)
2014 – 2015 UCSF Demographics
1,471 Trainees931 Residents279 ACGME/ABMS Fellows244 Non-ACGME Fellows17 Non-MD Trainees
176 Programs26 Residencies60 ACGME/ABMS Fellowships84 Non-ACGME Fellowships6 Non-MD Training
![Page 8: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/8.jpg)
27 ACGME Residency Review Committees (RRC)
Two major responsibilities: Develop and approve training
standards
Review and accredit residency and fellowship programs
Remember: they are us!
![Page 9: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/9.jpg)
ACCREDITATION RESOURCES Office of GME staff and faculty
Other UCSF program directors and coordinators
Other program directors in your specialty.
Attend regional and national meetings
Call the RRC staff (but call us first to discuss)
![Page 10: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/10.jpg)
Why Did ACGME Create a New (“Next”) Accreditation System (NAS)?
• Reduce the burden of accreditation
• Free good programs to innovate
• Assist poor programs to improve
• Realize the promise of Outcomes Project
• Provide public accountability for GME outcomes
![Page 11: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/11.jpg)
NAS Big Picture• Less prescriptive program requirements
that promote curricular innovation
• Continuous accreditation model
• Annual monitoring of programs based on performance indicators/outcomes
• Holding sponsoring institutions responsible for oversight of educational and clinical systems
![Page 12: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/12.jpg)
![Page 13: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/13.jpg)
The Building Blocks of the Next Accreditation System
Program Self Study Visits – 10 years Program Self Study Visits – 10 years
Institutional Self Study Visits– 10 years Institutional Self Study Visits– 10 years
Additional Site Visits as NeededAdditional Site Visits as Needed
Continuous RRC Oversight and Accreditation Core Program Oversight of Subs
Sponsor Oversight for All
Continuous RRC Oversight and Accreditation Core Program Oversight of Subs
Sponsor Oversight for All
CLER Visits every 18 monthsCLER Visits every 18 months
![Page 14: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/14.jpg)
How is the burden reduced?
• No Program Information Forms (PIFs)• Scheduled program visits from ACGME every 10 years• Focused site visits when “issues” are identified• Formal mid-cycle internal reviews no longer required• Most data elements used in NAS are already in place
in ADS
• Streamlined ADS annual update– Removed 33 questions– 14 questions simplified – Faculty CVs removed (except for Program Director)– 11 multiple choice or yes/no questions added
![Page 15: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/15.jpg)
Categorized Program Requirements
Core Detail OutcomeStatements that define structure, resource, or process elements essential to every program
Statements that describe specific structure, resource or process for achieving compliance with a Core Requirement
Statements that specify expected measurable or observable attributes (knowledge, abilities, skills, or attitudes) of trainees at key stages of their education
Program requirements are now categorized as core, detail, and outcome.
![Page 16: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/16.jpg)
Categorization of Program Requirements: Reduce Burden + Promote Innovation
•Why is this important?– Programs in good standing can innovate –
not asked whether adhering to detailed PR
– But: detailed PR do not go away. PDs will not need to demonstrate compliance with these PRs, unless it becomes evident that a particular outcome or core PR is not being met
![Page 17: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/17.jpg)
Read your requirements…
![Page 18: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/18.jpg)
![Page 19: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/19.jpg)
10-year Self Study
• Addresses:
1) Citations, areas for improvement, other information from ACGME
2) Strengths and areas for improvement identified by
• Annual Program Evaluation (APE)• Other program/institutional sources• Compliance with core requirements, faculty
development, etc
Data from entire period will be used
![Page 20: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/20.jpg)
Outcome Data for Annual Review
• Program attrition• Program changes• Scholarly activity (faculty and trainees)• Board pass rate (from Boards)• Clinical experience (case logs, survey data)• Resident survey• Faculty Survey• Milestones• (CLER visit data)
![Page 21: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/21.jpg)
![Page 22: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/22.jpg)
2014 ACGME Resident/Fellow and Faculty Surveys
Overall evaluation of the programResident/Fellow
Survey Faculty Survey
Very Positive 66% 89%
Positive 27% 10%
Neutral 6% 1%
Negative <<1% 0%
Very Negative <<1% 0%
![Page 23: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/23.jpg)
UCSF Areas for Improvement 2014 Aggregated Survey
• Duty hours 93%• Confidential evaluations 81%• Use evaluations to improve 72%• Feedback after assignments 61%• Education compromised 66%• Data about practice habits 48%• Transition care if fatigued 78%• Raise concerns without fear 82%
![Page 24: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/24.jpg)
![Page 25: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/25.jpg)
RRC Letters of Notification
• Citations• Levied by RRC without a site visit• Linked to program requirements• Reviewed annually by RRC• Reviewed during site visits• Removed (quickly) based on progress report, site visit,
new annual data. Older ones removed after two years
• Areas for Improvement (AFI)• Annual data raises an issue. “General concerns”• May be given by staff• Not linked to program requirement• No response required• Slate remains clean-based on each year’s submissions• Not the same as citations
![Page 26: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/26.jpg)
![Page 27: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/27.jpg)
Special Reviews
Replaces time-based, formal mid-cycle Internal Review
Three types: Initial review (prior to first site visit) Periodic review (1-2 years prior to first self study) Special review (programs with relative
underperformance as reviewed by GMEC) ACGME Update ACGME resident, fellow and faculty surveys ACGME RRC notifications - especially site visits UCSF Duty hour reports (UCSF) UCSF Program Directors Annual Update (UCSF)
![Page 28: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/28.jpg)
Subspecialty (Fellowship) Programs
• In NAS: Core residency and subspecialty programs reviewed together
• Self study visits will assess both together
• Letters of Notification will include both
• Assures that core residency and subspecialty programs will use resources effectively
![Page 29: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/29.jpg)
New ACGME Program Requirement
The specialty-specific Milestones must be used as one of the tools to ensure residents are able to practice core professional activities without supervision upon completion of the program. (Core)
![Page 30: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/30.jpg)
ACGME Reporting Milestones (example Internal Medicine)
![Page 31: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/31.jpg)
Milestones Defined
• Milestones are NOT an evaluation tool. Milestones are a reporting instrument.
• The Clinical Competence Committee (CCC) of each program will review assessment data.
• The CCC will take data and apply them to the milestones to mark the progress of a resident.
![Page 32: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/32.jpg)
Milestones defined
Meaningful, measurable markers of progression of competence
– What abilities does the trainee possess at a given stage?
– What can the trainee be entrusted with?
![Page 33: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/33.jpg)
![Page 34: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/34.jpg)
Learner Assessment Skeleton
• In-training exam (or other knowledge tests)• End of rotation assessments (global
assessment—fewer, more focused)• Direct observations (CEX, on-the-fly, check
lists, procedures, etc)• Multi-source feedback (self, peers, students,
other professional staff, patients)• Learner portfolio (Clinical experience-case
logs, etc), conference presentations, QI work, scholarship, teaching, reflection, learning plans, etc)
![Page 35: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/35.jpg)
Milestone
![Page 36: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/36.jpg)
Neurology
![Page 37: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/37.jpg)
Milestones Related to Competencies (and Subcompetencies)
![Page 38: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/38.jpg)
Entrustable Professional Activities (EPAs)
Define important clinical activities
Link competencies/milestones
Include professional judgment of competence by clinicians
![Page 39: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/39.jpg)
EPA defined
A core unit of work reflecting a responsibility that should only be entrusted upon someone with adequate competencies Ole ten Cate, Medical Teacher 2010;32:669-675
![Page 40: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/40.jpg)
person-descriptors
knowledge, skills, attitudes, values
• content expertise• collaboration ability• communication ability• management ability• professional attitude• scholarly habits
work-descriptors
essential parts of professional practice
• discharge patients• counsel patients• design treatment plans• lead family meetings• perform paracenteses• resuscitate if needed
Competencies EPAs
Competencies versus EPAs
![Page 41: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/41.jpg)
EPA Examples
Caring for an acute stroke patient
Discharging a patient from the hospital and preventing readmissions
Conducting a family meeting about withdrawal of support
Driving a car at night (in the rain, on the freeway)
![Page 42: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/42.jpg)
The Competency-EPA Framework
Medical Know
Communication
Patient Care
Professionalism
PBLI
SBP
++
+
+
+
+
++
++
+
+
+
++
++
++
++
+
++
+
++
EPA1 EPA2 EPA3 EPA4 EPA5
![Page 43: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/43.jpg)
Clinical Competence Committee (CCC)
• Each program/program director will be required to form a Clinical Competence Committee (CCC)
• Composition: minimum three faculty; also OK non-physician members and senior residents
• Review all evaluations• Report milestones to ACGME• Recommend to PD re promotion remediation
dismissal.
![Page 44: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/44.jpg)
CCC Processes
• Consensus-based recommendations
• Respect personal privacy
• Objective, behavior-based assessments
• Summary minutes taken by program coordinator
• Various trainee review strategies will work
• Identify areas of CCC and program weakness for annual review
![Page 45: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/45.jpg)
Program Evaluation Committee
• Required for each residency and fellowship
• Must have a written description
• Appointed by Program Director (PD)
• Oversee curriculum development and program evaluation (APE)
• PD may be chair or appoint chair
• Two faculty and one resident or fellow
• Must meet (at least) annually
![Page 46: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/46.jpg)
Program Evaluation Committee
• Review and revise goals and objectives
• Address areas of ACGME non-compliance
• Review program using evaluations of faculty, residents, and others
• Write an Annual Program review (APE), with 3-5 action items
• Track: resident performance, faculty performance, graduate performance (including Boards), program quality and progress on previous years action plans
![Page 47: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/47.jpg)
![Page 48: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/48.jpg)
![Page 49: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/49.jpg)
CLER Visit – December 2 – 4, 2014
• Team of four visitors• Met with:
– Senior leadership– 70 residents and fellows– 70 program directors– 70 teaching faculty– Walking rounds of 30 clinical areas
• Spoke to residents, fellows, nurses, techs, etc.• Observed three end-of-shift hand-offs
![Page 50: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/50.jpg)
Findings: Patient Safety
* On walking rounds, knowledge of terminology and principles varied** 44% of those, reported the event; 13% relied on a nurse to report; 31% relied on a
physician supervisor; 11% didn’t submit a report
Residents/Fellows
Program Directors
Faculty
Knew UCSF Medical Center patient safety priorities 55% 82% 83%
Received formal education/training about patient safety
92%*
Believed UCSF Medical Center provides a safe, non-punitive environment for reporting errors, near misses, and unsafe conditions
90%
Experienced an adverse event or near miss 75%**
Believed less than half of trainees have reported a patient safety event using the IR system
86% 84%
Opportunity to participate in an RCA 41% 82% 70%
![Page 51: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/51.jpg)
Findings: Healthcare Quality
Residents/Fellows
Program Directors
Faculty
Knew UCSF Medical Center healthcare QI priorities 59% 67% 75%
Engaged with Medical Center leadership in developing and advancing quality strategy
12%
Participated in a QI activity directed by Medical Center administration
78%
Participated in a QI project of their own design or one designed by their program/department
88%
Residents/fellows have access to organized systems for collecting/analyzing data for the purpose of QI
63% 83% 72%
![Page 52: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/52.jpg)
Findings: Transitions in Care
• During walking rounds:– Nurses and trainees expressed concerns about patient transfers from one
level of care to another– Observed hand-offs varied in use of templates, style of template, and
format/level of information detail relayed– Faculty present in only one observed hand-off
Residents/Fellows
Program Directors
Faculty
Knew UCSF Medical Center priorities for improving transitions of care
62% 87% 73%
Use standardized process for sign-off and transfer of patient care during change of duty
83%
Use written templates of patient information to facilitate hand-off process
65%
Use standardized processes for transfers of patients between floors/units
53%
Use standardized processes for transfers from inpatient to outpatient care
57%
![Page 53: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/53.jpg)
Findings: SupervisionResidents/Fellows
Program Directors
Faculty
Residents/fellows always know what they are allowed to do with and without direct supervision
97% 100% 100%
Been placed in a situation or witnessed one of their peers in a situation at UCSF Medical Center where they believed there was inadequate supervision
22%
Have an objective way of knowing which procedures a particular resident/fellow is allowed to perform with or without direct supervision
93% 92%
In the past year, had to manage an issue of resident/fellow supervision that resulted in a patient safety event
27%
Perception of patients’ awareness of the different roles of residents/fellows and attending physicians
22% 56% 23%
![Page 54: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/54.jpg)
Findings: Duty Hours, Fatigue Management, and Mitigation
Residents/Fellows
Program Directors
Faculty
Received education on fatigue management and mitigation
90% 92% 73%
Scenario: Maximally fatigues resident two hours before end of shift; what would you (or residents in general) do?
37% power through
10% 24%
Underreporting of moonlighting time by residents and fellows
20%
Recalled a patient safety event related to trainee fatigue
8%
![Page 55: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/55.jpg)
Findings: ProfessionalismResidents/Fellows
Program Directors
Faculty
Received education on various professionalism topics during orientation
78%
Received education on various professionalism topics throughout training
74%
Believe UCSF Medical Center provides a supportive, non-punitive environment for bringing forward concerns regarding honesty in reporting
88%
While at UCSF Medical Center, there was at least one occasion where pressure was felt to compromise integrity to satisfy an authority figure
20%
Documented a history or physical finding in a patient chart they did not personally elicit
64%
Believe the majority of residents/fellows have documented a history or physical finding in a patient chart they did not personally elicit
51% 48%
![Page 56: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/56.jpg)
Summary: UCSF CLER Opportunities
• Continued work on MD incident reporting• Better feedback and dissemination of IR results • Increase participation in RCA’s• Greater engagement of housestaff in QI strategy• More analysis and dissemination of clinical
outcomes in vulnerable populations• More standardization of handoffs (all clinicians)• Better fatigue mitigation• Enhanced EHR professionalism
![Page 57: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/57.jpg)
Summary ACGME NAS
• The NAS started July 2013
• More work early. Less burden long term?
• Greater opportunity for innovation for high functioning programs
• Better learner assessment and outcome measurement
• Much higher expectations re learner engagement in clinical environment.
• Ten year cycles and self studies (PDSA, SWOT, etc)
• Greater public accountability
![Page 58: ADVANCES IN GME: Mastering Accreditation, Learner Assessment, and the Learning Environment Robert B. Baron, MD MS Associate Dean, Graduate and Continuing.](https://reader035.fdocuments.us/reader035/viewer/2022062804/56649ee45503460f94bf27cd/html5/thumbnails/58.jpg)
Keeping an “E” in GME
Meet your program requirements, but be innovative Collaborate at UCSF and nationally Work hard on your annual program evaluation and
continuous improvement processes Support our residents and fellows