Validation of Mayo Clinic Risk Adjustment Model for In-Hospital Mortality following

Percutaneous Coronary Interventions using the National Cardiovascular Data Registry

Validation of Mayo Clinic Risk Adjustment Model for In-Hospital Mortality following

Percutaneous Coronary Interventions using the National Cardiovascular Data Registry

Mandeep Singh; Eric D. Peterson*; Sarah Milford-Beland*; John S. Rumsfeld,# John A. Spertus**

Mayo Clinic, Rochester, DCRI* (S.M-B, E.P.), Mid America Heart Institute** (J.A.S.), Denver VA

Medical Center# (J.S.R.)

Mandeep Singh; Eric D. Peterson*; Sarah Milford-Beland*; John S. Rumsfeld,# John A. Spertus**

Mayo Clinic, Rochester, DCRI* (S.M-B, E.P.), Mid America Heart Institute** (J.A.S.), Denver VA

Medical Center# (J.S.R.)

No Financial Disclosure or Conflict of Interest

BACKGROUNDBACKGROUND

• Predictive models can assist patients and clinicians in decision-making and informed consent.

• Existing PCI risk models include angiographic variables limiting routine clinical use.

• Mayo Clinic Risk Score (MCRS) for in-hospital mortality is based on pre-procedural clinical and non-invasive assessment.

• MCRS can potentially serve as a risk assessment aid to patients/physicians before coronary angiography for PCI.

• Predictive models can assist patients and clinicians in decision-making and informed consent.

• Existing PCI risk models include angiographic variables limiting routine clinical use.

• Mayo Clinic Risk Score (MCRS) for in-hospital mortality is based on pre-procedural clinical and non-invasive assessment.

• MCRS can potentially serve as a risk assessment aid to patients/physicians before coronary angiography for PCI.

BackgroundBackground

• External validation of the MCRS is lacking

• The NCDR cath-PCI registry presents an ideal opportunity to validate the MCRS

• Study population: Index PCI for 309,351 patients in NCDR participating hospital between January 2004 and March 2006.

• Outcome: In-hospital mortality during the hospital admission following PCI.

• External validation of the MCRS is lacking

• The NCDR cath-PCI registry presents an ideal opportunity to validate the MCRS

• Study population: Index PCI for 309,351 patients in NCDR participating hospital between January 2004 and March 2006.

• Outcome: In-hospital mortality during the hospital admission following PCI.

CP1246788-1

Risk scoreRisk score

Points Score

Age (yr) See below ____

Creatinine (mg/dL) See below ____

LV ejection See below ____fraction (%)

Preprocedural shock 9 ____

MI within 24 hours 4 ____

CHF on presentation 3 ____(without AMIor shock)

Peripheral 2 ____vascular disease

Total score ____

Points Score

Age (yr) See below ____

Creatinine (mg/dL) See below ____

LV ejection See below ____fraction (%)

Preprocedural shock 9 ____

MI within 24 hours 4 ____

CHF on presentation 3 ____(without AMIor shock)

Peripheral 2 ____vascular disease

Total score ____

Est

imat

ed r

isk

of

dea

th (

%)

Est

imat

ed r

isk

of

dea

th (

%)

0 5 10 15 20 25

80706050403020

10

5432

1

0.5

0.1

20 30 40 50 60 70 80 90 1 2 3 4 5 6 7 8 9 10 11 0 20 40 60 80

Age (yr)Age (yr) Creatinine (mg/dL)Creatinine (mg/dL) LV ejection fraction (%)LV ejection fraction (%)

22 11 00 11 22 33 44 55 11 0011 3322 44 55 66 44 33 22 11 00

Mayo Clinic Risk Score (MCRS)Mayo Clinic Risk Score (MCRS)MortalityMortality

Mayo Clinic Risk Score (MCRS)Mayo Clinic Risk Score (MCRS)MortalityMortality

C-index=0.90

Statistical MethodsStatistical Methods

• Using the MCRS equation, predicted probabilities of death were calculated for each patient in the NCDR population.

• Patients with the same predicted mortality score were grouped together, and within each group, the observed (O) mortality rate was calculated.

• The O vs. E (expected) mortality rates for these groups were plotted and we used H-L method for calibration

• Model discrimination was assessed using ROC, or c-statistic, for the entire population and within pre-specified subgroups.

• Using the MCRS equation, predicted probabilities of death were calculated for each patient in the NCDR population.

• Patients with the same predicted mortality score were grouped together, and within each group, the observed (O) mortality rate was calculated.

• The O vs. E (expected) mortality rates for these groups were plotted and we used H-L method for calibration

• Model discrimination was assessed using ROC, or c-statistic, for the entire population and within pre-specified subgroups.

Statistical Methods (Cont.)Statistical Methods (Cont.)

• The analysis was refined to include recalibration of the MCRS equation using the ACC population

• For this recalibrated model, patients with the same predicted mortality score were again grouped together.

• O vs. E mortality rates were plotted.

• Calibration: Hosmer-Lemeshow method.

• Internal validation of the new model using NCDR PCI patients April 2006, March 2007.

• The analysis was refined to include recalibration of the MCRS equation using the ACC population

• For this recalibrated model, patients with the same predicted mortality score were again grouped together.

• O vs. E mortality rates were plotted.

• Calibration: Hosmer-Lemeshow method.

• Internal validation of the new model using NCDR PCI patients April 2006, March 2007.

Patient Characteristics by In-Hospital Mortality in the NCDR Patient Characteristics by In-Hospital Mortality in the NCDR Variable Number (%) Mortality p

Age

<60Y 114,844 (37.12) 0.60 <0.0001

≥80Y 34383 (11.11) 3.22

Congestive heart failure

Yes 27003 (8.73) 5.25 <0.0001

No 282,321 (91.26) 0.84

Acute Myocardial infarction

Yes 68116 (22.02) 3.44 <0.0001

No   241,128 (77.95) 0.60

Peripheral vascular disease

Yes 36568 (11.82) 2.18 <0.0001

No 272,768 (88.17) 1.10

Cardiogenic shock

Yes 6314 (2.04) 24.83 <0.0001

No 303,007 (97.95) 0.73

Renal failure

Yes 16323 (5.28) 3.89 <0.0001

No 293,012 (94.72) 1.08

Variable Number (%) Mortality p

Age

<60Y 114,844 (37.12) 0.60 <0.0001

≥80Y 34383 (11.11) 3.22

Congestive heart failure

Yes 27003 (8.73) 5.25 <0.0001

No 282,321 (91.26) 0.84

Acute Myocardial infarction

Yes 68116 (22.02) 3.44 <0.0001

No   241,128 (77.95) 0.60

Peripheral vascular disease

Yes 36568 (11.82) 2.18 <0.0001

No 272,768 (88.17) 1.10

Cardiogenic shock

Yes 6314 (2.04) 24.83 <0.0001

No 303,007 (97.95) 0.73

Renal failure

Yes 16323 (5.28) 3.89 <0.0001

No 293,012 (94.72) 1.08

02468

101214161820

0 3 6 9 12 15 18 21 24

MCRS

Frequency of the Risk, based on the MCRS of Patients Undergoing PCI

%

Risk Score

Fre

qu

en

cy (

%)

Discrimination of the MCRSDiscrimination of the MCRSDiscrimination of the MCRSDiscrimination of the MCRS

CP1246782-7

Group N MCRS (Min- Max) C-indexOverall 309,351 0-25 0.884

Shock/ AMI 69920 4-25 0.873

Age <40 5627 1-21 0.938

Age 65+ 151517 0-25 0.858

CHF 27003 3-25 0.82

Creatinine <0.7 10491 1-20 0.797

Creatinine >1.2 66839 1-25 0.875

Multivessel Dx 150579 0-25 0.87

Female 104110 0-24 0.872

Diabetes 98081 0-24 0.878

Group N MCRS (Min- Max) C-indexOverall 309,351 0-25 0.884

Shock/ AMI 69920 4-25 0.873

Age <40 5627 1-21 0.938

Age 65+ 151517 0-25 0.858

CHF 27003 3-25 0.82

Creatinine <0.7 10491 1-20 0.797

Creatinine >1.2 66839 1-25 0.875

Multivessel Dx 150579 0-25 0.87

Female 104110 0-24 0.872

Diabetes 98081 0-24 0.878

Observed versus expected in-hospital mortality using the original MCRS prediction equation

Observed versus expected in-hospital mortality using the original MCRS prediction equation

Obs

erve

d M

orta

lity

(%)

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

55%

60%

65%

70%

Predicted Mortality (%)

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70%

O vs. E in-hospital mortality with recalibrated quadratic MCRS, internal validation sample (433,045)

O vs. E in-hospital mortality with recalibrated quadratic MCRS, internal validation sample (433,045)

Ob

serv

ed M

orta

lity

(%

)

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

55%

60%

65%

70%

Predicted Mortality (%)

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70%

O=5,177; E=5,310 deaths (difference 2.5 per 100)

c index= 0.885

Summary and ConclusionsSummary and Conclusions

• External validation of the MCRS using NCDR confirms its broader applicability.

• The MCRS has high discrimination for in-hospital mortality using 7 clinical/non-invasive variables.

• Most variables can be obtained at the time of first visit.

• This may help the operator to individualize the risk of procedural death from PCI, and to counsel patients at the time of PCI.

• External validation of the new, recalibrated MCRS model is, however, required.

• External validation of the MCRS using NCDR confirms its broader applicability.

• The MCRS has high discrimination for in-hospital mortality using 7 clinical/non-invasive variables.

• Most variables can be obtained at the time of first visit.

• This may help the operator to individualize the risk of procedural death from PCI, and to counsel patients at the time of PCI.

• External validation of the new, recalibrated MCRS model is, however, required.