High Quality Residency Education and Patient Safety are Directly Related

David T. Harrington, MDProgram Director, General Surgical Residency Program

Associate Professor of Surgery

Residency Work Hours

• Resident fatigue – clinical errors• Residency work hours surrogate for patient safety• Effects of work hour rules - mixed

– Patient Safety• Lack of definitive effect• More cross-coverage and hand-offs

– Residency Education• Improved resident well-being, less “burn-out”• Decreased operative experience (FA, TA, continuity)• Increased choice of fellowships• Decreased medical student teaching

Combined Surgical Group

3

3.2

3.4

3.6

3.8

4

4.2

4.4

4.6

4.8

5

-3 -2 -1 1 2

% M

orta

lity

V MajorMajorMinorV MinorNon-teach

KG Volpp et al, Mortality among hospitalized Medicare beneficiaries in the first 2 years….., JAMA 2007

Residency Work Hours

• Resident fatigue – clinical errors• Residency work hours surrogate for patient safety• Effects of work hour rules - mixed

– Patient Safety• Lack of definitive effect• More cross-coverage and hand-offs

– Residency Education• Improved resident well-being, less “burn-out”• Decreased operative experience (FA, TA, continuity)• Increased choice of fellowships• Decreased medical student teaching

Patient Safety and Surgical Quality

• Traditional M&M• Process

– Joint Commission– SCIP– Centers of Excellence

• Trauma - ASC/COT• Bariatric - ASBS• Cancer, Orthopedic, Cardiac

• Performance– NSQIP– UHC– CMS – MSDRG and P4P

NSQIP Observed to Expected (O/E) Ratio• Represents the hospital’s outcomes compared to the other ACS NSQIP hospitals, adjusted for inter-hospital differences in patients’ characteristics, comorbidities, and preoperative laboratory values

LOW OUTLIER: If the upper bound of the O/E confidence interval is <1.0, the hospital’s outcomes are statistically better than expected. Thus, the hospital’s outcomes are “Exemplary.”

ACS NSQIP Hospital ID Number

HIGH OUTLIER: If the lower bound of the O/E ratio is >1.0, the hospital’s outcomes are statistically worse than expected. Thus, the hospital’s outcomes “Need Improvement.”

AS EXPECTED

Surgical Residency TrainingOutcomes

• Process– RRC accreditation

• Performance– ABSITE scores– Pass rates on QE & CE– Fellowship placement– Successful practice

1st Time Pass Rate 2004 - 2009

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100

CE

QE

Hypothesis

Programs which deliver high quality care as measured by:participation in NSQIP

verification as a trauma center

high rates of compliance in SCIP

would have a sound educational programs evidenced by:high pass rates on QE and CE

Methods & Materials

• NSQIP participation – NSQIP annual report• Trauma Center verification –

www.facs.org/trauma/verified

• SCIP compliance for laparoscopic cholecystectomy: SCIP1, 2, 3, VTE1, VTE2 – www.hospitalcompare.hhs.gov

• QE and CE – https://home.absurgery.org/default.jsp?prog_passreport

Results

• NSQIP– 85 (34%) participated

• Trauma Center Verification– 82 (32.8%) verified trauma centers– 65 (26%) were Level 1 trauma centers

• SCIP– All hospitals had data available

• ABS Examinations – 1st time passage rate– QE 83.9 +/- 13.1– CE 84.5 +/- 11.1– Combined 80.0 +/- 16.2

NSQIP – Yes NSQIP - No p-value

Qualifying Exam (average)

87.9 81.8 < 0.001

Certifying Exam (average)

85.8 83.3 0.100

Combined (average) 77.2 70.1 0.001

NSQIP Participation & QE/CE

Trauma CenterYes

Trauma CenterNo

p-value

Qualifying Exam (average)

87.4 83.0 0.038

Certifying Exam (average)

88.6 83.9 0.006

Combined (average) 78.8 71.3 0.003

Trauma Center Verification & QE/CE

Level 1 – Yes Level 1 - No p-value

Qualifying Exam (average)

88.4 83.1 0.012

Certifying Exam (average)

89.1 84.4 0.005

Combined (average) 80.1 71.9 < 0.001

Level 1 Verification & QE/CE

CE2007 vs SCIP1

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100

Certifying Exam

SCIP

1

p = 0.30

CE2007 vs SCIP2

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100

Certifying Exam

SCIP

1

p = 0.11

CE2007 vs SCIP3

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100

Certifying Exam

SCIP

1

p = 0.01

CE2007 vs VTE 1

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100

Certifying Exam

SCIP

1

p = 0.18

CE2007 vs VTE 2

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100

Certifying Exam

SCIP

1

p = 0.092

Hypothesis

Programs which deliver high quality care as measured by:participation in NSQIP

verification as a trauma center

high rates of compliance in SCIP

would have a sound educational programs evidenced by:high pass rates on QE and CE

Excellent Patient Safety = Excellent Education

• RRC site visit– Trauma center verification– Improving SCIP compliance– Improving NSQIP performance

• Teaching hospitals are quintessential quality model– New physicians– Ascending levels of responsibility

• 6 core competencies– System-based practice– Practice-based learning

• Multi-disciplinary– RRC, NSQIP, Trauma Verification, SCIP compliance

Shortcomings

• QE/CE pass rates ≠ quality of training• Study has no relevance to medical training• Participation in NSQIP ≠ higher quality of

patient care

NSQIP: Changes in Morbidity 2005-2007

Conclusions

• Hospitals that participate in NSQIP have a higher quality of surgical resident education than hospitals that do not.

• Hospitals that obtain Trauma Center verification have a higher quality of surgical resident education than hospitals that do not.

• Hospitals that obtain Level 1 Trauma Center verification have a higher quality of surgical resident education than hospitals that do not.

High Quality Residency Education and Patient Safety are Directly Related

David T. Harrington, MDProgram Director, General Surgical Residency Program

Associate Professor of Surgery