Risk Prediction in Clinical Practice

Joann G. Elmore MD, MPH

SCCA June 11, 2014

Overview

1. Risk prediction models need to work well for individual patients to be helpful in individual counseling.

2. Both clinicians and patients (and the media) must understand the numeric information resulting from risk prediction models in order to use them effectively.

All PatientsAll Patients

Risk Prediction Models

Ultra Low RiskUltra Low Risk Medium RiskMedium Risk High RiskHigh Risk

All PatientsAll Patients

Risk Prediction Models

Ultra Low RiskUltra Low Risk Medium RiskMedium Risk High RiskHigh Risk

All PatientsAll Patients

Ultra Low RiskUltra Low Risk Medium RiskMedium Risk High RiskHigh Risk

Risk Prediction Models

http://www.cancer.gov/bcrisktool/

Validation of Gail Model at the Population Level

The Gail model works well at the population level, with calibration calculated at

0.94 (B Rockhill, et al JNCI 2001)

1.01 (WE Barlow et al JNCI 2006)

1.03 (JA Tice et al Ann Intern Med 2008)

Validation of Gail ModelPopulation vs. Individual Risk

Population of women

0% to 100%

Individual woman

Either 0% or 100%

Ideal ModelEstimated Five-year Risk of Breast Cancer

Excellent Discrimination (c-statistic=1.0) (All Women with Risk > "X" Value get Cancer)

Women

with

Cancer

Womenwith noCancerProportion

of Sample

Low Risk X High Risk

Gail ModelEstimated Five-year Risk of Breast Cancer

C-statistic 0.58

80,755

50,000

100,000

Women with Cancer

Women with no Cancer

1.67% cutoff

Gail Model Example Estimated Five-year Risk of Breast Cancer

Gail Risk Value

For every 47 women classified as high-risk only 1 will subsequently be diagnosed with breast cancer (Sensitivity 0.44; Specificity 0.66)

Performance of Gail Model using a 5-year cancer risk of 1.67%

Clinical Experience

“Doctor, what is my risk of getting breast cancer?”

41-year-old patient

What is a 41-year-old woman’s risk of breast cancer over

the next 5 years?

A 41-year-old white woman whose mother had breast cancer, who had one prior breast biopsy with atypical hyperplasia, who was age 40 at first live birth. What is her risk of a breast cancer diagnosis in the next 5 years?

A. Less than 4%B. 10%C. 20%D. 50%

What is a 41-year-old woman’s risk of breast cancer over

the next 5 years?

Radiologists’ Estimates of the 5-year risk of a breast cancer diagnosis.

“A 41-year old white woman...”

Calculated risk

Egger, et al., Med Dec Making, 2005

Numeric Literacy

Example Question

High School Diploma or

less

Post- graduate degree

How many heads in 1,000 coin flips?

62% 86%

Convert 1% to # of patients in 1,000

60% 82%

LM Schwartz and S Woloshin 2000;G Gigerenzer et al, 2008

Convert 1 in 1,000 to a percentage

23% 27%

Gigerenzer G etal., Psychol Sci Public

Interest 2007

Variation of Lumpectomy Rates by Hospital Referral Region (2003-04) Dartmouth Atlas

Effect of Nancy Reagan’s Mastectomy

Nattinger et al. JAMA 1998

Screening for Cancer(www.areyoudense.org)

Treatment Decisions(e.g., Adjuvant!)

Referral to Hospice(e.g., estimating 6-mo cut-off)

The Doctor (2014)

www.myopennotes.org

Conclusions1. Risk prediction models work well in

populations, but not (yet) for individual patients.

2. Implementation into clinical practice and

communication about risks is challenging.

3. The impact and value of these models in clinical practice needs to be studied.

Thank you

The Doctor by Sir Luke Fildes (1887)