Fundamental issues

Alyna T. Chien MD, MSThe University of Chicago

Harvard Quality ColloquiumAugust 20, 2008

Goal

• Illustrate how the first pediatric public reporting effort is facing inherent challenges to pediatric quality measurement

CHARTCalifornia Hospital Assessment and Reporting

Taskforce

Inherent challenges

• Perspective• What to measure• Evidence base• Sample sizes

Inherent challenges

• Perspective– Consumers– Providers

• Free standing children’s hospitals• Community hospitals

– Purchasers

Inherent challenges

• What to measure– Structure, Process, Outcome– Safety, Effectiveness, Efficiency, Equity, Patient-centeredness,

Patient Safety, Timeliness

Inherent challenges

• Evidence base– Proper endpoints?

Inherent challenges

• Sample size– 1/5th of the adult population– Lower disease prevalence– Significant proportion of care provided in adult contexts– Significant proportion of care provided in community contexts

300M adults~50M children (~20% of population)

Max ‘volume’~500,000 newborns in California

271 of 442 hospitals with pediatric services~1800 newborns per hospital

~180 non-newborn admissions per hospital

Strategy

• Perspective Diverse Workgroup• What to measure Diverse portfolio• Patchier evidence base Diverse portfolio• Patchier evidence base Invest in development• Small sample size “Functional” measures?• Small sample size “Structural” measures?• Small sample size Aggregation methods?• Small sample size Invest in development

Tool

Does it apply to children?prevalence

costIs it evidence based?

What dimensions does it measure?Portfolio diversification

Strategy

• Systematic approach

• Portfolio diversification

PMD Tool Quickie

• Does it apply to children? – prevalence

• Which ones?• Impact?

– mortality– disease burden

– cost• Evidence base?• Dimension of quality?

• Portfolio diversification

ALL AGES

Annual discharges U.S. (n)

Annual expenditures (millions)

3,968,346 Liveborn Liveborn $ 10,461

147,782 Pneumonia Appendicitis $ 1,101

139,528 Acute bronchitis Pneumonia $ 1,056

138,574 Asthma Mood disorders $ 974

99,308 Fluid and Electrolyte disorders Fracture of the lower limb $ 965

95,201 Mood disorders Acute bronchitis $ 893

73,455 Appendicitis Asthma $ 796

47,789 Epilepsy, convulsions Epilepsy, convulsions $ 463

38,004 Urinary tract infections Fluid and electrolyte disorders $ 355

34,550 Intestinal infections Maintenance chemotherapy,

radiotherapy $ 306

30,676 Viral infections Urinary tract infections $ 261

29,692 Trauma to the perineum and vulva Viral infections $ 157

26,860Hemolytic jaundice and perinatal jaundice

Trauma to the perineum and vulva $ 154

19,436Normal delivery without complications Intestinal infections $ 151

18,217 Noninfectious gastroenteritis Substance-related and alcohol

disorders $ 148

16,782 Fracture of the upper limb Fracture of the upper limb $ 145

13,884 Fracture of the lower limb ADHD and disruptive behavior

disorders $ 103

13,726Maintenance chemotherapy, radiotherapy Early or threatened labor $ 102

12,897 Early or threatened labor Intracranial injury $ 101

9,522 Diabetes mellitus with complications Hemolytic jaundice and

perinatal jaundice $ 98

8,249Substance-related and alcohol disorders

Normal delivery without complications $ 95

7,255 Poisoning by other medications Complication of device, implant $ 76

6,679ADHD and disruptive behavior disorders

Diabetes mellitus with complications $ 74

6,217Fetal distress and abnormal forces of labor Noninfectious gastroenteritis $ 60

6,152 Abnormal fetal heart rate Fetal distress and abnormal

forces of labor $ 53

5,202 Umbilical cord complication Poisoning by other medications $ 52

4,735 Abdominal pain Abnormal fetal heart rate $ 46

4,384

Hypertension complicating pregnancy, childbirth, and the puerperium

Hypertension complicating pregnancy, childbirth, and the

puerperium $ 38

3,478 Skin infections Excess amniotic fluid and other

problems of amniotic cavity $ 28

3,328 Intracranial injury Umbilical cord complication $ 28

3,325Excess amniotic fluid and other problems of amniotic cavity Abdominal pain $ 25

2,471 Complication of device, implant Skin infections $ 21

2,442 Fever of unknown origin Fever of unknown origin $ 13

811 Sickle cell anemia Sickle cell anemia $ 9

536 Skin disorders Skin disorders $ 3

5,039,493 19,411

CHART Starter SetApplies to children

Population?

Impact

Evidence

Quality dimension

Exclusive breastfeeding rate

VolumeNormal newborns

Sort of Process/OutcomeEffectiveness

NICU nosocomial infection rate

Premies/at-risk newborns

Modifiable by provider

Process/OutcomeSafety

Antibiotics for appendicitis

VolumeAll ages

Not really Process

CHART Future Directions

• Perspective• What to measure• Evidence base• Sample size

Summary

• Pediatric public reporting in its infancy

• Lots of room for improvement

ThanksCHART Core• R. Adams Dudley, MD MBA• Mitzi Dean, PhD• Ted Karrison, MS • James Anderson, PhD

Peds Workgroup• Diana Dooley (CCHA)• Erin Givens (CSCC)• Jeff Gould (Stanford)• Greg Janos (Sutter)• Tom Klitzner (UCLA)• Paul Kurtin (UC San Diego)• Paul Sharef (UCLA)

Extra slides

Public reporting: adult history

• Florence Nightengale• Ernest Codman

• Mortality after Coronary Artery Bypass Grafts

• Medicare

Public reporting: mechanisms?

• Free market mechanism:

Comparative info about healthcare quality

payors (employers, health plans, and patients)

choose higher quality providers

financial rewards will flow to better performers

(and away from poorer ones)

• “Self-improvement” mechanism:

Comparative info about healthcare quality

providers (hospitals, medical groups, individual physicians)

better awareness of quality issues

quality regulation cost containment quality improvement

Do “consumers” act on the information?

• Free market mechanism: Comparative info about healthcare quality payors (employers, health plans, and patients) choose higher quality providers financial rewards will flow to better performers (and away from poorer ones)

• “Self-improvement” mechanism: Comparative info about healthcare quality providers (hospitals, medical groups, individual physicians) better awareness of quality issues quality regulation cost containment quality improvement

YES and NO

Marshall MN et. al, JAMA 2000

Public reporting: mechanisms for exclusive

breastfeeding• Volume• Data availability

• Low prevalence of conditions• Appropriate processes?• Appropriate outcomes?

Public reporting: mechanisms for exclusive

breastfeeding• Prenatal information/decision-making

– Family/Friends– Healthcare providers– Social agencies (e.g. WIC)

• Immediate post-natal period– Family/Friends– Healthcare providers Hospital “critical period”?

•Hospital-based supports•A “critical period”?

Public reporting: mechanisms for exclusive

breastfeeding• Prenatal information/decision-making

– Family/Friends– Healthcare providers– Social agencies (e.g. WIC)

• Immediate post-natal period– Family/Friends– Healthcare providers Hospital “critical period”?

•Hospital-based supports•A “critical period”?

Public reporting: mechanisms for exclusive

breastfeeding• Prenatal information/decision-making

– Family/Friends– Healthcare providers– Social agencies (e.g. WIC)

• Immediate post-natal period– Family/Friends– Healthcare providers

•Hospital-based supports•A “critical period”?

Performance – three approaches

For CHART’s exclusive breastfeeding measure:

Zh = Ÿh – Ýh . VAR (Ÿh – Ýh)

where:• Ÿh is mean exclusive breastfeeding rate for each hospital• Ýh is mean exclusive breastfeeding rate for all newborns

(statewide average)• No other individual level adjustments• Hospitals admitting <30 newborns annually are excluded

Background

American healthcare lacks quality and equity

Public reporting is one strategy aimed at improving the quality of that care

• “Free market” mechanism– consumers/payors

• “Self-improvement” mechanisms– providers, individuals and organizations

• Evidence equivocal

Public reporting’s role in eliminating (or exacerbating) racial/ethnic disparities is unknown

“Free market” “Self-imprvmnt” Consumers / payors Providers

Consequence:• Improve Want to go somewhere Want to be viewed

as “equitable” “equitable”

• Worsen Promotes segregation

• Status quo

Unique opportunity to explore public reporting’s potential role in presenting racial/ethnic disparities

Exclusive breastfeeding rates in California hospitals:• Enough patients

– 10% of all American newborns born in California (500,000/year)

• Enough hospitals– 283 are licensed to provide pediatric services

• Good racial/ethnic data (as currently available)

– standardized collection– complete– variation expected

This goal of this paper is to explore three approaches to incorporating information on racial/ethnic disparities into hospital public reporting:1. Adjusting expected performance for

race/ethnicity

2. Stratifying performance by race/ethnicity

3. Developing a ‘disparity’ score

Hypotheses:

1. Hospital performance/rankings will change depending on how race/ethnicity is incorporated into performance methodology

2. Each methodology will have ‘pros’ and ‘cons’

Methods

Study Design

Cross-sectional

Primary independent variables: - Different performance measurement methodologies:

1. Proportional2. Stratified3. + ‘Disparity’ score

Primary dependent variables: - Changes in hospital rank

Performance – traditional methodIn general:

Zh = Ÿh – Ýh . VAR (Ÿh – Ýh)Where:• Zh is standardized performance• Ÿh is observed mean performance including adjustments• Ýh is expected mean performance including adjustments• Each adjusted for patient characteristics

Conventionally:• Hospitals with <30 observations are excluded• Performance can be estimated using standard frequentist

approaches• Or using Bayesian ones

Performance – alternative approaches

Traditional ZhT = ŸhT – Ýall / VAR (ŸhT–Ýall)

Proportional ZhP = ŸhP – Ýall / VAR (ŸhP–Ýall)

Stratified ZhWh = ŸhWh – ÝallWh / VAR (ŸhWh–ÝhWh)ZhAA = ŸhAA – ÝallAA / VAR (ŸhAA–ÝhAA)ZhHi = ŸhHi – ÝallHi / VAR (ŸhHi–ÝhHi)ZhAs = ŸhAs – ÝallAs / VAR (ŸhAs–ÝhAs)ZhOt = ŸhOt – ÝallOt / VAR (ŸhOt–ÝhOt)

‘Disparity’ score

=0 if ZhWh = ZhAA = ZhHi = ZhAs = ZhOt

Data Source

California Department of Public Health Center for Family HealthGenetic Disease Screening ProgramNewborn Screening Data 2006• Mandated statewide screening program• Established 1966• Screening rate 99%

Data Collection

As part of the Newborn Screen, all providers are required to answer the following questions:

“All feeding since birth: (check only one box)

[]Breast only []Formula only []Breast & Formula[]TPN/Hyperal []Other. (SPECIFY):__________________________________________________”

“Race/ethnicity: (check all that apply)

[]White []Hispanic []Black []Chinese []Japanese[]Korean []Cambodian []Laotian []Vietnamese []Filipino[]Asian Indian []Middle Eastern []Native American[]Samoan []Other (SPECIFY):________________________________________________________”

Data Quality - Missing

2006

Number of newborns• 492,587 in dataset – cross-checked with Vital Statistics

Breastfeeding status• Indicated 97.2%• Missing 2.8%

Race/ethnicity noted• Indicated 97.4%• Missing 2.6%

Data Quality - Validity

2006

Breastfeeding status GDSP ?NSLAH• Exclusive ~40%• Any ~90%

Race/ethnicity noted GDSP ?Census• White• African American• Hispanic• Asian• Multi• Other

PRELIMINARY RESULTS

271 eligible hospitals174 participating in CHART

PRELIMINARY RESULTS

California 2006      

  Newborns Exclusive Breastfeeding Rate

  n mean sd

Total 492,587 0.427 0.495

White 125,136 0.637 0.481

African America 24,018 0.330 0.470

Hispanic 261,456 0.321 0.467

Asian/PI 45,566 0.440 0.496

Other/Multi 23,353 0.500 0.500

Missing 13,058 0.526 0.499

Additional Analyses

Impact of dropping hospitals with <30 observations, particularly with the Stratified approach

Treatment of missing values - zeros - dropped

Frequentist versus Bayesian approach