Post on 25-Dec-2015
J. L. Marsh, MD
Chair - Orthopaedic RRC
Director of the ABOSCarroll B. Larson Chair
Residency Program Director
University of Iowa Hospitals and Clinics
Iowa City Iowa
NAS - (and other new program requirements)
Overview of what you will need to do!
Next Accreditation System (NAS) Milestones Procedural minimums Continuous data assessment Surgical skills training Greater institutional oversight No PIF, less and different site visits
We are about to experience tremendous changes to the oversight of GME
The financial implications of these
changes are uncertain at this
time
Traditional Program Review Program review scheduled PIF prepared and sent to ACGME and SV SV – 1-2 days RRC review
PIF and SVR
Board pass rates, Resident Survey, Case log data RRC actions
Initial or continued accreditation with citations 1-5 year cycle Progress report Propose probation
SV and PIF are key portions of program review
What is NAS???
• Program accreditation system without:– Mandated site visits– PIF’s– 100% mandatory PR’s– Direct resident interviews
• Program accreditation system with:– Annual or semi annual data review– Performance metrics within the data elements– Emphasis on outcomes– Focused site visits driven by data metrics– More opportunity for program innovation– Data charts, graphs and data flows!!!
Notice the word “data” is used 5
times!!
Notice the “PIF and site visit”
are going away
NAS is not all about Milestones
Milestones will be a work in progress but will not be a part of program accreditation for several years
But Milestones are one of the new requirements where increased effort and cost for programs maybe necessary and they need to start now!!
NAS represents a substantial change in program oversight
Change in focus / function of RRC More educational Less regulatory
PD’s empowered to innovate & create an excellent programCore vs. detail requirements
Improved tools for program review without a PIF Focused reviews triggered by parameters set by the RRC
The data for Program Review by RRC Trended & weighted performance metrics
– Program data– Resident and faculty scholarship (new template)– Clinical experience (enhanced case logs)– Resident Survey (new questions)– Core Faculty Survey (new) – Semi-annual Resident Evaluation
• Milestones (new)• Clinical Competency Committee (new)
– Rolling Board pass rates (Parts I & II)– Program Self-Study (new) Site Visit (every 10 years)
Many of theseare outcomes, many of them are new!
Performance thresholds based on data elements
• Weighting of data elements will provide screening criteria• RRC annual review and action only if necessary• Potential actions include:
Initial or Continued Accreditation Request more information from program Request Site visit (focused or full) on short timeline Continued Accreditation with Warning Probation
• “If there is a problem get in there and fix it”• “If the data is good…….leave them alone…innovation”
Absent – SVR and PIF document
Program Requirement Changes
• Common & Specialty Specific (no change)• Core: requirement must be met as specified; if not,
program can be cited• Detail: programs will not be assessed for compliance with
these requirements if they demonstrate good educational outcomes. These are mandatory for new programs & those that failed to meet outcomes expectations (on Probation or Continued Accreditation with Warning)
• Outcome – Some data elements are based on these
NAS Program Review
Each program reviewed at least annually
NAS is a continuous accreditation process
– Review of annually submitted data
– Supplemented by:
• Reports of self-study visits every ten years
• Progress reports (when requested)
• Reports of site visits (as necessary)
Annual Review of Data (Oct. – Nov.)
Options Available Prior to January RRC Meeting• Focused Site Visit
• Full Site Visit
• Request Clarification or Progress Report
• Send Material out to RRC for ReviewHighlighting the Problem(s) for Peer Decision
• Move to Consent Agenda
Proposed workflow prior to RRC meeting
• Minimal notification given
• Minimal document preparation expected
• Team of site visitors
• Specific program area(s) investigated as instructed by
the RRC
NAS: Focused Site Visit
• Application for new program
• At the end of the initial accreditation period
• RRC identifies broad issues / concerns
• Other serious conditions or situations identified by the
RRC
NAS: Full Site Visit
Programs (CA) meet all established performance indicator thresholds (40%) – letter from ED Continued accreditation with no RRC review
Programs (CA) fail to meet 1 established performance indicator (30%) - letter from ED Continued Accreditation but notes need for improvement - indicates the deficiency
Programs that fail to meet 2 established performanceindicator thresholds but not “High Stakes” indicators (20%) – ED reviews program for trends – if first time event letter from ED Continued Accreditation notes problems no further RRC review
Theoretical Work Flow – Consent agenda (90%)
Programs fail to meet 3 -5 established performance indicator thresholds (7%) – Two RRC reviewers assigned
Programs fail 6-9 performance indicator thresholds (3%) – Assigned a focused or full site visit
Theoretical Work Flow – RRC review
For NAS you need to do two things!
• Do well on the performance metrics
• Appoint a new committee to oversee Milestones and develop a plan to
evaluate them
The data for Program Review by RRC
Trended & weighted performance metrics– Program data– Resident and faculty scholarship (new template)– Clinical experience (enhanced case logs)– Resident Survey (new questions)– Core Faculty Survey (new) – Semi-annual Resident Evaluation
• Milestones (new)• Clinical Competency Committee (new)
– Rolling Board pass rates (Parts I & II)– Program Self-Study (new) Site Visit (every 10 years)
Many of theseare outcomes, many of them are new!
Maintain a consistent solid
performance on all of these!!
Clinical Competency Committee
Semi Annual Review of Data to assign Milestones
• CCC - Faculty time and input necessary for these individual resident evaluations which are the Milestones
• There may also be a PEC committee. – program evaluation
Milestones5 level assessments of resident knowledge,
skills, attitudes, and other attributes of performance in the six competencies in a
developmental framework from less to more advanced. They are designed to demonstrate
program outcomes by assessing resident progress through the competencies measured
in the milestone framework!
Milestones: Medical Knowledge & Patient Care
• ACL• Ankle Arthritis• Ankle Fracture• Carpal Tunnel• Degenerative Spine• Diabetic Foot• Diaphyseal Femur & Tibia
Fracture• Distal Radius Fracture• Adult Elbow Fracture• Hip & Knee Osteoarthritis
• Hip Fracture• Metastatic Bone Lesion• Meniscal Tear• Pediatric Septic Hip• Rotator Cuff Injury• Pediatric Supracondylar
Humerus FractureSmall slices of clinical care – a biopsy of resident performance!
Operationalizing Milestone reporting?
• The faculty, PD and PC time and effort to accomplish this remain uncertain
• Therefore the tradeoff for absence of SV’s and PIF’s remains uncertain
• In my opinion they are good assessments which will make a more uniform national standard to assess resident competence
There are other non NAS requirements that will have financial implications
for your department
• 6 months of PGY 1 ortho
• Mandated surgical skills training through simulation
PG-1 Year Changes 2013-2014
• ABOS certification rules developed from results of a CORD survey
• ACGME/RRC accreditation rules developed from ABOS
6 months of orthopaedic surgery
Basic surgical skills training
Good news – they are the same!
PG-1 Year Changes 2013-2014More time on orthopaedics!
So orthopaedic PGY 1’s will be on ortho for 6 months instead of 3 months
In our program we have 6 PGY 1’s so effectively this is a junior level 1.5 FTE
How much of a cost advantage for a department is this?
6 months of orthopaedics
Basic surgical skills requirements (core)
A curriculum with goals and objectives
Assessment metrics
A dedicated space for the skills training
Training in basic skills required of residents for emergency care and to prepare residents for future participation in surgical procedures
This is what is required!What will that cost?
Results of a 2011 National Orthopaedic Program Director and Resident Survey – Karam and Marsh JBJS 2012
Only 50% of residency programs have a skills lab and program.
There is high interest among PD’s in a skills curriculum.
Most PD’s have little knowledge of the budget for skills training or the cost of a skills lab
Cost is a challenge to expansion of skills programs
*Percentages may not total 100% because respondents were allowed to choose more than one answer.
Lack of funding
Barrier to skills program
Members of the ABOS (AOA/CORD and AAOS) Surgical Skills Task Force
• J. Lawrence Marsh, MD – Chair (ABOS)
• James E. Carpenter, MD (ABOS)• Shepard R. Hurwitz, MD (ABOS)• Michelle A. James, MD (ABOS)• Joel T. Jeffries, MD (AOA/CORD)• David F. Martin, MD (ABOS)• Peter M. Murray, MD (ABOS)• Bradford O. Parsons, MD (AAOS)• Robert A. Pedowitz, MD, Ph.D. Co-
Chair (AAOS)
• Brian C. Toolan, MD (AAOS)• Ann E. Van Heest, MD
(AOA/CORD)• M. Daniel Wongworawat, MD
(AAOS)
1. Sterile technique and operating room set up
2. Knot tying & suturing
3. Microsurgical suturing
4. Soft tissue handling techniques
5. Casting and splinting
6. Traction
7. Compartment syndrome
8. Bone handling techniques
9. Fluoroscopy
10. K-wire techniques
11. Basic techniques in ORIF
12. Principles and techniques of fracture reduction
13. External fixation
14. Basic Arthroscopy skills
15. Basics of Arthoplasty
16. Joint injection
17. Patient Safety
Modules (ABOS skills taskforce modules)
Modules should include:Evaluation and assessment strategies
• Guided practice until performance within time standards
• Video of performance with blinded review by expert faculty with “pass” or “needs more practice”
• OR performance ONLY after verification
Summary and Conclusions Resident satisfaction was high.
A dedicated month of surgical simulation has potential to change the paradigm of skills training for junior residents.
Considerable time invested in the planning and execution but faculty members were eager to contribute.
The greatest expense was for cadaveric specimens. With better planning more cost effective simulations, this expense could be reduced.
Overal Summary and Conclusions
NAS will take more time/effort and more yearly costLess demanding in the year of a site visitThe balance is hard to knowOther changes to PR at PGY 1 will affect finances
Less availability for all night callMore ortho time for PGY 1’s (3 vs 6 months)
Surgical simulation will cost moneyMore or less depending….This investment may be worthwhile