Methodological Issues in Producing CDC Data for the

National Healthcare Quality and Disparities Reports

Suzanne E. Proctor, MSPHRichard J. Klein, MPH

NCHS Data User’s ConferenceJuly 12, 2004

Background

Preliminary set of measures for the 1st National Healthcare

Quality Report (NHQR) and National Healthcare Disparities

Report (NHDR) presented to NCHS in summer 2002

High degree of overlap between measures in two reports (same Quality of Care measures)

Majority of requested measures were Healthy People 2010 measures

• Measures already gone through consensus process • Established national data source• Measures widely recognized & used at Federal & State levels

Background

Inclusion of HP2010 measures simplified the project

• Operational definitions developed

• Technical issues (e.g., age adjustment) resolved

• SAS/SUDAAN programs written

• Some baseline & update data available for most measures

However…

• Definitions for some measures differed slightly from HP2010

• Population subgroup data and cross-tabulations requested beyond what is

regularly programmed for HP2010

• Additional analytic issues arose during the production of data

Scope of NCHS Contribution

Summary of Measures Provided for the 2003-2004 Reports

NHQR only

NHDR only

Both reports

HP2010 measure (55) 18 7 30

Modified HP2010 measure (13) 4 0 9

Not a HP2010 measure (39) 6 31 2

TOTAL MEASURES: 107 28 38 41

Summary of Measures Provided for the 2003-2004 Reports

National & State data

National data only

State data only

HP2010 measure (55) 23 28 4

Modified HP2010 measure (13) 2 10 1

Not a HP2010 measure (39) 0 39 0

TOTAL MEASURES: 107 25 77 5

Distribution of Measures by Healthy People 2010 Focus Area

6

33

18

243

44

108

Number of measures (both reports)

3. Cancer5. Diabetes

16. Maternal, Infant & Child Health14. Immunization & Infectious Diseases

13. HIV12. Heart Disease & Stoke

18. Mental Health

1. Access to Quality Health Services

21. Oral Health24. Respiratory Diseases

Not a HP2010 measure

Distribution of Measures by National Data Source

11

4555

66

710

1519

23

Behavioral Risk Factor Surveillance System

National Ambulatory Medical Care Surveys

National Immunization Survey

National Health Interview Survey

National Hospital Discharge Survey

National Vital Statistics System

HIV/AIDS Reporting System

National Nosocomial Infections Surveillance System

Number of measures (both reports)

Adult Spectrum of Disease Surveillance Project

Children with Special Healthcare Needs

National Health and Nutrition Examination Survey

National Home and Hospice Care Survey

National Nursing Home Survey

Types of Measures Provided

Selected NHQR measures

Effectiveness

• Percent of women (40 and over) who report they had a mammogram in the past 2 years

• Hospital admissions for pediatric asthma per 10,000 population under 18 years

• HIV-infection deaths per 100,000 population

Safety

• Central line-associated bloodstream infection in Intensive Care Unit patients

Selected NHQR measures

Timeliness

• Percent of persons who have a usual source of medical care, by place of care

• Percent of patients who wait 1 or more hours to be seen by a physician, by urgency of needed care

Patient Centeredness

(No measures provided)

Selected NHDR measures

Quality of Care

• Effectiveness: Hospital admissions for lower extremityamputations in patients with diabetes per 1,000 population

• Safety: (No measures provided)

• Timeliness: Percent of persons in fair or poor health whohave a specific source of on-going care

• Patient centeredness: (No measures provided)

Selected NHDR measures

Access to Care

• Entry barriers: Percent of persons < 65 years with public health insurance only

• Structural barriers: (No measures provided)

• Patients’ perceptions: (No measures provided)

• Use: Emergency department visits per population

Depth of Data Provided

Data Templates

Specific tabulations requested by AHRQ differed forthe NHQR and NHDR

Typical NHQR Data Template for Population-Based Objectives

Total Total

AgeAge RaceRace 

American Indian or Alaska Native Asian or Pacific Islander

Asian Native Hawaiian or Other Pacific Islander Black or African American White

Hispanic origin and raceHispanic origin and raceHispanic or Latino Not Hispanic or Latino Black or African American White  

GenderGender Female Male

Socioeconomic StatusSocioeconomic Status 

Family Income Level Poor Near poor Middle/high income

oror

Education Level Less than high school High school graduate At least some college

NHDR Data Templates for Population-Based Objectives

Race Race x NHQR variables

EthnicityEthnicity x NHQR variables

Socioeconomic statusSocioeconomic status x NHQR variables

Data Templates

Specific tabulations requested by AHRQ differed forthe NHQR and NHDR

Data quality/small number issues limited ability toprovide data for some cross-tabulations in the

NHDR

Availability of data to complete templates varied bydata source

• SES and health insurance information not available from all

data sources

• Collection & tabulation of race/ethnic data differed by data source

Tracking Trends

Tracking Trends – Key Issues

Change in Federal standards for collecting and tabulating race/ethnicity data

Change in Census population denominators

National/State data comparability

Periodicity of data collection

Sample size and data quality issues

Population Denominators

For the 2003 reports, NCHS provided 1999 and 2000 data from NVSS

Denominators for these data were post-censal estimates (based on the 1990 Census)

For the 2004 reports, NCHS reran the 1999 and 2000 NVSS data using 2000 census data and

revised 1999 inter-censal estimates

National/State Data Comparability

National and State data provided for 25 measures

In general, the national and State data werecomparable

• NVSS national and State data based on same source reports • NIS national and State data based on same source reports• NHIS (national) and BRFSS (state) surveys – compare cautiously

However, not all of the BRFSS questions were asked by all States each year

• Some BRFSS questions contained in optional modules

Periodicity of Data

For most measures, NCHS was able to provide a new data point for each data year shown

However …

• Some surveys are fielded on a periodic basis (e.g. NNHS)

• Questions used to track some measures were not included inthe survey each year (e.g. NHIS, BRFSS)

• For some measures, needed to combine 2 or more years ofdata to obtain reliable estimates (e.g. NHANES)

Periodicity of Data - Example

NHIS measure:

12-12. Percent of adults (18 years and over) who have had a blood pressure measurement within the past 2 years and can state whether it was normal or high

Data available in 1998, 2003

BRFSS measure:

12-15. Percent of adults (18 years and over) who have had their blood cholesterol checked within the past 5 years

2001: 50 States + D.C.2002: 12 States + D.C.

Sample Size/Data Quality Issues

Data for smaller subpopulation groups may be statistically unreliable due to small sample size

Even when the subpopulation groups size is sufficient for univariate analysis, cross-tabulation with other variables in

the template can spread the data too thin

For some data sources, the sample design does not produce representative estimates for some sub-population

groups

Large number of missing responses

DSU – data are suppressed

Summary

Reports provided an opportunity for intra-agency and cross-agency collaboration on two major Federal

public health reports

Reports provided a rich source of additional data for HP2010 (e.g. Progress Reviews)

Excellent example of how the existing Federal data infrastructure can be readily used for in-depth

examination of critical public health issues