VA Quality Measurement: Is There a “Halo Effect?”

Steven M.Asch MD MPH

VA Sepulveda HSR&D COE

RAND Health

Geffen School of Medicine at UCLA

VA Performance Measurement

• Dates back to 1995• ~30 mostly chart-based process measures,

varying evidence support• Facility-based primary care sampling• Feedback, public reporting• Regional managers have financial

incentives

VA Improves 12/13 Leading Indicators 1995- 2000

0

10

2030

4050

6070

8090

100

Hgba1c Flu vac CRC Screen ASA MI

19951996199819992000

Jha NEJM 348:22 2003

VA Beats Medicare 12-1 in 2000

01020

3040

5060

708090

100

Mammo Pneumovac DM eye

MedicareVA

Jha NEJM 348:22 2003

What About Overall Quality?

• Leading indicator systems – Focus QI efforts, but– Vulnerable to gaming, resource

misallocation

• What about overall quality? • How does VA overall quality

compare to the community?

Overall Process Quality Measured by QATools

• Literature reviews conducted to develop evidence-explicit indicators

• 5 Delphi expert panels convened to evaluate recommended indicators

• 348 clinically detailed process indicators for 26 clinical areas

• Example: Patients with pre-existing coronary disease with an LDL > 130mg/dl should begin diet or drug therapy within 3 months.

Example Clinical Areas General Medical Conditions

• BPH (4)• Depression (14)• Diabetes (13)• Dyspepsia/PUD (8)• Headache (21)• Low back pain (6)• Orthopedic conditions (10)• Preventive care (35)• STDs (26)

Oncologic Conditions • Colorectal cancer (12)• CA pain and palliation (3)• Prostate cancer (6)

Cardiopulmonary Conditions• Asthma (25)• Congestive heart failure (36)• COPD (20)• Coronary artery disease (37)• Hyperlipidemia (7)• Hypertension (26)• Pneumonia (5)• Stroke/TIA (10)

Study Sites

•SEATTLE

ORANGE COUNTY

PHOENIX

LITTLE ROCK

INDIANAPOLIS

• CLEVELAND•

GREENVILLE

MIAMI•

NEWARK•

• BOSTON

• SYRACUSE

• • •

•

•

LANSING•

VISN 22

VISN 11

% Receiving Indicated Care

53

67*

58

73*

44

64*

5653

0

10

20

30

40

50

60

70

80

Overall Chronic Prev Acute

CommVA

% Receiving Indicated Care

5969

63

81*

52

68*

0

10

20

30

40

50

60

70

80

90

COPD HTN DM

CommVA

Is There A “Halo Effect?”

• Does the VA advantage exist only in specific processes subject to VA performance management?

• Does it extend to clinically related conditions or areas?

• Does it extend to clinically unrelated areas?

P. Gaugin-”Self Portrait with a Halo”

3 Mutually Exclusive Summary Scores

In VA measurement set (26 indicators) Not in VA measurement set, but in same

condition (178 indicators) Neither (167 indicators)

Adjusted % Receiving Indicated Care

53

67*

42

69*

58

71*

51 54

0

10

20

30

40

50

60

70

80

Overall VAConditions

Comm

VA

VAMeasures

Neither

Conclusions

• VA is improving on leading indicators

• VA beats Medicare on leading indicators

• VA beats community on broader measures

• “Halo effect” of performance measurement diminishes with clinical distance

Wins: Performance measurement

Losses: ?????

Ties: Acute care, eye exams, COPD

Errors: Hoping for too big a halo

Questions

Adjusted RR of Indicated Care

1.22*

2.04*

2.74*

0.00

0.25

0.50

0.75

1.00

1.25

1.50

1.75

2.00

VA PerformanceMeasures

VA PerformanceMeasure Conditions

Non-VA PerformanceMeasure Conditions

Sampling: Men>35 with 2 Visits/yr

Community VA

Source Random digit dial

Admin data

Response rate 48% 97%

N 992 597

Study period 1/98-12/99 1/98-12/99

Sites 12 cities 26 facilities