NEJM 065990

8/18/2019 NEJM 065990

http://slidepdf.com/reader/full/nejm-065990 1/10

n engl j med 357;2 www.nejm.org july 12, 2007 181

s pe c i a l r e p o r t

Th e n e w e n g l a n d j o u r n a l o f m e d i c i n e

Quality of Primary Care in England

with the Introduction of Pay for PerformanceStephen Campbell, Ph.D., David Reeves, Ph.D., Evangelos Kontopantelis, Ph.D.,

Elizabeth Middleton, M.Sc., Bonnie Sibbald, Ph.D., and Martin Roland, D.M.

In 2004, the United Kingdom committed £1.8 bil-lion ($3.2 billion) to a new pay-for-performancecontract for family practitioners.1 During the first year, the levels of achievement exceeded those an-ticipated by the government, with an average of83.4% of the available incentive payments claimed.2 However, the quality of care in English family

practices had already begun to improve in responseto a wide range of initiatives,3-6 including nation-al standards for the treatment of major chronicdiseases and a national system of inspection (Ta-

ble 1). Family practitioners already had some ex-perience with f inancial incentives from the lim-ited use of incentive programs that were initiatedin 1990.7,8 It is therefore unclear whether the highlevels of quality attained after the pay-for-perfor-mance contract was introduced in 2004 reflectimprovements that were already under way or whether existing trends toward improvement wereaccelerated. The effect of the incentive programmust be understood in the context of the com-prehensive quality-improvement strategy within which the contract was introduced.

This report presents data from a longitudinalcohort study that measured the quality of care ina representative sample of primary care practicesin England at two time points (1998 and 2003)

before the pay-for-performance program was in-troduced and at one time point (2005) after itsintroduction. A validated set of criteria was usedto assess quality in the management of threechronic conditions: asthma, coronary heart dis-ease, and type 2 diabetes. Because some clinicalindicators — the measures of the quality of clini-

cal care — were not rewarded with financial pay-ments in the 2004 pay-for-performance program,the study design also permitted a comparison oftrends in the quality of care for indicators for which financial incentives were provided and forthose for which they were not provided in themanagement of these three conditions.

Methods

In 1998, we measured the quality of care in a strat-ified, random sample of 60 primary care practicesin six geographic areas of England. These practices were nationally representative in terms of size, whether the practice was approved for residencytraining, and the sociodemographic characteris-tics of their populations.9 We followed up 42 ofthese practices in 2003 and 2005. The reduction inthe number of practices was due partly to attri-tion and partly to the retirement of solo physi-

Table 1. Examples of Key Initiatives in the Broad National Quality-Improvement Strategy.

National standards for the treatment of major chronic diseases, such as the National Service Frameworks for coronary

heart disease (1999) and diabetes (2003)

Contractual requirement for practitioners to undertake a clinical audit (initially a requirement in the 1990 contract)

Financial incentives for cervical cytologic testing and immunization (early 1990s)

Widespread use of audit and feedback by the Primary Care Trusts

Release of comparative data for quality of care to practitioners (common) and the public (rare) by the Primary CareTrusts

Annual appraisal of all primary care physicians (by the Primary Care Trusts and including discussion of some audit data)

National system of inspection and monitoring of performance (by the Healthcare Commission)

The New England Journal of Medicine

Downloaded from nejm.org on April 20, 2016. For personal use only. No other uses without permission.

Copyright © 2007 Massachusetts Medical Society. All rights reserved.

8/18/2019 NEJM 065990


Th e n e w e n g l a n d j o u r n a l o f medicine

n engl j med 357;2 www.nejm.org july 12, 2007182

cians and the closing of other practices. The 42practices for which data were available for thelongitudinal analysis were still nationally repre-sentative in terms of socioeconomic status, butsolo practitioners were underrepresented.4 How-ever, these 42 practices have values close to thenational averages for socioeconomic status, pop-

ulation density, and type of housing of the pa-tient population, and their performance was alsotypical of English family practices during the first year of the pay-for-performance program.2 Theanalysis was restricted to the 42 practices for which data were available for all three time points(1998, 2003, and 2005). The research protocol wasapproved by the ethics committee of the multi-center Manchester National Health Service.

Data Collection

Trained research staff extracted the data to assess

the quality of clinical care for the categories ofcoronary heart disease (15 clinical indicators),asthma (12 clinical indicators), and type 2 dia-betes (21 clinical indicators). Data were collectedfrom both computerized and handwritten medi-

cal records with the use of evidence-based reviewcriteria10,11 developed with the RAND–UCLA ap-propriateness method.12 Patients with these threeconditions were randomly selected from lists ofthose receiving the relevant drugs (see the Sup-plementary Appendix, available with the full textof this article at www.nejm.org) according to re-

peat prescriptions within the previous 6 months,and separate samples of patients treated in 1998,2003, and 2005 were selected. In 1998, for twopractices, there were no eligible patients who hadcoronary heart disease because of the young ageof the patient population, so for that time point,the results for this condition are based on only40 practices. Data were collected for up to 20 pa-tients for each of the three conditions in eachpractice in 1998 (some small practices did nothave 20 patients for each of the conditions) andfor up to 12 patients for each condition in each

practice in 2003 and 2005. Data were collectedfor a total of 2300 patients in 1998, for 1495 pa-tients in 2003, and for 1482 patients in 2005.These data are presented as a pooled analysisacross practices.

Although the study did not include conditionsthat were not rewarded with f inancial incentivesin the pay-for-performance program, there wereclinical indicators for coronary heart disease,asthma, and type 2 diabetes for which financialincentives were not provided in 2004. We com-pared 30 indicators for which financial incentives were provided with 17 indicators for which finan-cial incentives were not provided. In this analysis, we excluded three clinical indicators for whichthis distinction was unclear — that is, it was notclear whether there were financial incentives provided for the indicator at all three time points.

Statistical Analysis

An overall score for the quality of care was com-puted for each patient included for 1998, 2003,and 2005. For each patient with asthma, coronary

heart disease, or type 2 diabetes, the score wascomputed as a ratio: the number of clinical indi-cators for which appropriate care was provided,divided by the number of indicators relevant tothat patient. Expressed as a percentage, this scorerepresents the percentage of “necessary care”10 provided to each patient, within a range from 0 to100. We adopted this measure for consistency withour previous investigation of this sample.4 Scoresfor the quality of care at the practice level were

90

M e a n P r a c t i c e - Q u a l i t y S c o r e ( % )

80

85

70

60

55

75

65

01998 1999 2000 2001 2002 2003 2004 2005

CHD

Asthma

Diabetes

Year

l

Figure 1. Mean Scores for Clinical Quality at the Practice Level for Coronary

Heart Disease, Asthma, and Type 2 Diabetes, 1998 to 2005.

The quality of care for coronary heart disease (CHD), asthma, and type 2

diabetes was improving between 1998 and 2003, before the introduction ofpay for performance. The rate of improvement in quality of care increased

significantly for diabetes and asthma between 2003 and 2005, after the in-

troduction of pay for performance; the rate for coronary heart disease,which was increasing most rapidly before pay for performance, continued

at the same rate after pay for performance was introduced.




8/18/2019 NEJM 065990


special report


computed as the simple average of the scores forindividual patients within each practice.

We also performed analyses for individualclinical indicators. Data were available for at leastfive patients in a practice for each indicator ana-lyzed and for geographic areas where the numberof practices meeting this criterion was at least 10.

In 1998, data were collected for indicators re-quiring an activity to be undertaken annually onthe basis of data recorded during the previous14 months, whereas in 2003 and 2005, data werecollected for indicators requiring an activity to beundertaken annually on the basis of data record-ed during the previous 15 months, in line withthe pay-for-performance contract. For consis-tency, we generated 15-month versions of the clini-cal indicators for 1998 from our original data foruse in the present analysis.

We compared scores for observed quality in

2005 with the scores predicted on the basis ofthe trend between 1998 and 2003. Practice scoresfor individual indicators, computed as a percent-age of patients receiving appropriate care as theindicator, were subject to a ceiling effect of 100%.In calculating the expected scores for 2005, it wasinappropriate to use a simple linear model, be-cause such a model would fail to account forceiling effects (i.e., some predicted scores wouldhave exceeded 100% if the previous linear trendhad been extrapolated from 2003 to 2005). Probitand logit models are most commonly used tomodel binary data. We adopted the logit modela priori for the current analysis, calculating ex-pected values for 2005 on the basis of the logitcurve that the scores from 1998 to 2003 followedand extrapolating this curve to 2005. After per-

forming the analysis, we computed probit predic-tions for comparative purposes and found theseall to be within 1 percentage point of their logitequivalent. For quality-of-care scores between 20and 80%, the logit and probit curves are essen-tially linear.

For each practice, we therefore computed a

predicted score for 2005 using a logit projectionfrom 1998 and 2003. We computed predicted val-ues for overall scores and for individual clinicalindicators. The predicted scores were then com-pared with the actual scores for the practices in2005. However, because of floor and ceiling ef-fects, the differences between actual and predict-ed scores are not equivalent across the scale: thedifference between an observed score of 54 anda predicted score of 50 does not have the sameimport as the difference between a score of 99 anda score of 95. To adjust for this difference, ob-

served and predicted scores were converted intotheir logit equivalents before the analysis. Underthe transformation, a proportion, P, is trans-formed into a log odds, as Logit(P) = ln[P ÷ (1 − P)].Since Logit(P) cannot be computed where the value of P is 0 or 1, we used the empirical logit,Logit(P) = ln[(P + 0.5 ÷ n) ÷ (1 − P + 0.5 ÷ n)], in thesecases, where n is the number of observations over which P is calculated.13 The logit transformationmaps the scale of 0 to 100% to a scale of lessthan infinity to infinity.13 The transformationtherefore “stretches” the scores at the extremes, which increases the effect on the results of theanalysis of practices for which scores are closeto the f loor or the ceiling.

The transformed observed and predicted scoresfor 2005 were then compared by means of a

Table 2. Changes in Mean Scores at the Practice Level for Quality of Care for Coronary Heart Disease, Type 2 Diabetes,and Asthma, 1998 to 2005.*

VariableCoronary Heart

Disease Diabetes Asthma

Mean score for 1998 — % 58.6 61.6 60.2

Mean score for 2003 — % 76.2 70.4 70.3

Mean score for 2005 — % 85.0 81.4 84.3

Mean predicted score for 2005 (logit model) — % 80.7 73.2 72.3

Mean difference between transformed observed scoreand predicted score for 2005 — % (95% CI)

0.22 (−0.02 to 0.45) 0.68 (0.27 to 1.1) 0.44 (0.27 to 0.62)

P value 0.07 0.002 <0.001

* CI denotes confidence interval. P values are for the comparison between transformed observed and predicted scoresfor 2005.




8/18/2019 NEJM 065990




T a b l e 3 .

S c o r e s f o r Q u a l i t y o f C a r e a t t h e P r a c t i c e L e v e l i n 1 9 9 8 ,

2 0 0 3 , a n d

2 0 0 5 . *

C o n d i t i o n

C l i n i c a l I n d i c a t o r

M e a n

S c o r e †

M e a n D i f f e r

e n c e

b e t w e e n T r a n s

f o r m e d

O b s e r v e d S c o

r e a n d

P r e d i c t e d S c o r e

i n 2 0 0 5

( 9 5 %

C I ) ‡

P V a l u e

1 9 9 8

2 0

0 3

2 0 0 5

p e r

c e n t

C o r o n a r y h e a r t

d i s e a s e

F r e q u e n c y o r p a

t t e r n o f a n g i n a a t t a c k s r e c o r d e d d u r i n g t h e

p r e v i o u s 1 5 m o

4 3 . 4

6 0 . 3

7 0 . 1

0 . 2 2 ( − 0 . 4 2 t o

0 . 8 6 )

0 . 4 9

B l o o d p r e s s u r e

r e c o r d e d d u r i n g t h e p r e v i o u s 1 5 m o

8 7 . 0

9 5 . 3

9 8 . 8

0 . 2 6 ( 0 . 0 2 t o 0

. 5 1 )

0 . 0 4

E x e r c i s e c a p a c i t y r e c o r d e d d u r i n g t h e p r e v i o u s 1 5 m o

3 6 . 5

6 6 . 3

7 2 . 8

0 . 2 1 ( − 0 . 5 0 t o

0 . 9 1 )

0 . 5 5

C h o l e s t e r o l l e v e

l r e c o r d e d d u r i n g t h e p r e v i o u s 5 y r

6 3 . 5

8 9 . 7

9 8 . 7

0 . 5 4 ( 0 . 2 6 t o 0

. 8 2 )

< 0 . 0 0 1

D i e t a r y a d v i c e r e c o r d e d d u r i n g t h e p r e v i o u s 5 y r

5 8 . 8

7 5 . 7

8 7 . 4

0 . 6 4 ( − 0 . 0 6 t o

1 . 3 4 )

0 . 0 7

S m o k i n g s t a t u s

r e c o r d e d d u r i n g t h e p r e v i o u s 5 y r

8 5 . 1

8 6 . 6

9 8 . 3

0 . 8 7 ( 0 . 4 7 t o 1

. 2 7 )

< 0 . 0 0 1

R e f e r r a l t o s p e c i a l i s t f o r e x e r c i s e s t r e s s t e s t i n g o r a s s e s s m e n t ( E C G ) e v e r

r e c o r d e d

7 6 . 2

8 8 . 2

8 9 . 2

− 0 . 0 1 ( − 0 . 4 0 t o

0 . 3 8 )

0 . 9 5

P r e s c r i p t i o n o r a d v i c e t o t a k e a s p i r i n r e c o r d e d u n l e s s r e c o r

d o f c o n t r a i n d i -

c a t i o n o r i n t

o l e r a n c e

7 6 . 2

8 0 . 7

9 2 . 5

0 . 7 1 ( 0 . 2 7 t o 1

. 1 5 )

0 . 0 0 2

B l o o d p r e s s u r e

c o n t r o l l e d t o ≤ 1 5 0 / 9 0 m m H g

4 7 . 9

7 2 . 2

8 1 . 5

0 . 1 5 ( − 0 . 2 3 t o

0 . 5 2 )

0 . 4 4

S e r u m c h o l e s t e r o l c o n t r o l l e d t o 1 9 0 µ g / d l

1 6 . 9

6 0 . 7

7 2 . 9

0 . 1 0 ( − 0 . 4 1 t o

0 . 6 1 )

0 . 6 9

C o n d i t i o n a l

f a c t o r s

S m o k i n g a d v i c e

t o s m o k e r s r e c o r d e d d u r i n g t h e p r e v i o u s 5

y r

6 6 . 5

7 6 . 8

9 9 . 1

N A

N A

W e i g h t a d v i c e f o r o v e r w e i g h t p a t i e n t s r e c o r d e d d u r i n g t h e

p r e v i o u s 5 y r

6 3 . 7

7 8 . 8

8 2 . 5

0 . 2 8 ( − 0 . 7 2 t o

1 . 2 9 )

0 . 5 6

A c t i o n t a k e n o n

b l o o d p r e s s u r e i f s y s t o l i c p r e s s u r e > 1 6 0 m

m H g ,

o r i f > 1 4 0 m

m H g a n d t o t a l s e r u m c h o l e s t e r o l > 1 9 0 m g

/ d l

3 4 . 3

4 4 . 4

5 8 . 3

N A

N A

A c t i o n t a k e n i f c

h o l e s t e r o l > 1 9 0 m g / d l i n p a t i e n t a g e < 7 0 y r

5 3 . 8

7 3 . 6

7 9 . 2

N A

N A

B e t a - b l o c k e r p r e s c r i b e d a s m a i n t e n a n c e t h e r a p y i f s o l e t h e r a p y

4 7 . 4

5 8 . 1

7 9 . 5

N A

N A

A s t h m a

N o r m a l o r p r e d i c t e d p e a k e x p i r a t o r y f l o w o r r e c o r d o f d i f f i c

u l t y u s i n g m e t e r

r e c o r d e d d u

r i n g t h e p r e v i o u s 5 y r

7 6 . 1

8 5 . 7

9 1 . 6

0 . 2 8 ( − 0 . 0 7 t o

0 . 6 4 )

0 . 1 2

D a i l y , n o c t u r n a l , o r a c t i v i t y - l i m i t i n g s y m p t o m s r e c o r d e d d u

r i n g t h e p r e v i -

o u s 1 5 m o

4 3 . 7

5 8 . 1

7 6 . 5

0 . 9 1 ( 0 . 3 1 t o 1

. 5 2 )

0 . 0 0 4



8 2 . 6

8 4 . 3

9 6 . 0

0 . 5 9 ( 0 . 1 6 t o 1

. 0 1 )

0 . 0 0 8

I n h a l e r t e c h n i q u

e r e c o r d e d d u r i n g t h e p r e v i o u s 5 y r

5 3 . 1

5 8 . 7

7 6 . 0

0 . 7 1 ( 0 . 1 3 t o 1

. 2 8 )

0 . 0 2




8/18/2019 NEJM 065990


special report



f a c t o r s

S m o k i n g a d v i c e

t o s m o k e r s r e c o r d e d d u r i n g t h e p r e v i o u s 5

y r

6 4 . 2

8 6

. 6

9 6 . 6

N A

N A

S e l f - m a n a g e m e n t p l a n f o r p a t i e n t s r e c e i v i n g h i g h - d o s e c o r t i c o s t e r o i d s o r

t h o s e w h o h a d i n p a t i e n t t r e a t m e n t f o r a s t h m a r e c o r d e d

d u r i n g t h e p r e -

v i o u s 5 y r

2 2 . 5

5 6

. 0

7 7 . 8

N A

N A

S h o r t - a c t i n g b r o

n c h o d i l a t o r s p r e s c r i b e d f o r u s e b e f o r e e x e r

c i s e f o r p a t i e n t s

w i t h e x e r c i s e

- i n d u c e d b r o n c h o s p a s m r e c o r d e d d u r i n g t h e p r e v i o u s 5 y r

4 0 . 1

7 4

. 2

9 0 . 9

N A

N A

P e a k e x p i r a t o r y f l o w d u r i n g a c o n s u l t a t i o n f o r a n e x a c e r b a t i o n r e c o r d e d f o r

m o s t r e c e n t

e p i s o d e

1 0 0

1 0 0

1 0 0

N A

N A

O r a l c o r t i c o s t e r o i d s p r e s c r i b e d i f p e a k e x p i r a t o r y f l o w < 6 0 %

o f n o r m a l

r e c o r d e d f o r

m o s t r e c e n t e p i s o d e

6 1 . 7

7 3

. 6

9 3 . 6

N A

N A

A c t i o n t a k e n i f p

a t i e n t s h a d n o c t u r n a l s y m p t o m s r e c o r d e d d u r i n g t h e p r e v i -

o u s 5 y r

3 0 . 6

8 0

. 8

1 0 0

N A

N A

A c t i o n t a k e n i f p

a t i e n t s h a d a c t i v i t y - l i m i t i n g s y m p t o m s r e c o

r d e d d u r i n g t h e

p r e v i o u s 5 y r

5 0 . 0

0

0

N A

N A

R e f e r r a l t o r e s p i r a t o r y p h y s i c i a n i f o r a l s t e r o i d s w e r e u s e d i n m a i n t e n a n c e

t r e a t m e n t , r e c o r d e d d u r i n g t h e p r e v i o u s 5 y r

7 6 . 1

8 5

. 7

9 1 . 6

N A

N A

T y p e 2 d i a b e t e s G l y c a t e d h e m o g

l o b i n r e c o r d e d d u r i n g t h e p r e v i o u s 1 5 m o

8 7 . 1

9 3

. 1

9 9 . 4

0 . 5 4 ( 0 . 2 9 t o 0

. 8 0 )

< 0 . 0 0 1

V i s u a l e x a m i n a t i o n o f f e e t r e c o r d e d d u r i n g t h e p r e v i o u s 1 5

m o

5 7 . 4

6 9

. 6

8 8 . 0

0 . 9 9 ( 0 . 4 5 t o 1

. 5 3 )

0 . 0 0 1

P e r i p h e r a l p u l s e

s o r v i b r a t i o n s e n s e r e c o r d e d d u r i n g t h e p r e v i o u s 1 5 m o

6 0 . 0

6 2

. 9

9 0 . 2

1 . 5 8 ( 0 . 9 8 t o 2

. 1 7 )

< 0 . 0 0 1

S e r u m c r e a t i n i n

e r e c o r d e d d u r i n g t h e p r e v i o u s 1 5 m o

7 9 . 6

8 9

. 9

9 6 . 4

0 . 3 3 ( − 0 . 0 8 t o

0 . 7 3 )

0 . 1 1

U r i n e p r o t e i n u r i a r e c o r d e d d u r i n g t h e p r e v i o u s 1 5 m o

6 6 . 3

7 4

. 8

8 2 . 8

0 . 4 3 ( − 0 . 1 2 t o

0 . 9 8 )

0 . 1 2

E x a m i n a t i o n o f f u n d i o r v i s u a l a c u i t y r e c o r d e d d u r i n g t h e p r e v i o u s 1 5 m o

6 9 . 4

7 2

. 2

8 2 . 7

0 . 5 8 ( 0 . 1 0 t o 1

. 0 6 )

0 . 0 2

W e i g h t r e c o r d e d

d u r i n g t h e p r e v i o u s 1 5 m o

8 0 . 2

8 6

. 5

9 7 . 2

0 . 6 0 ( 0 . 2 0 t o 1

. 0 1 )

0 . 0 0 5

B l o o d p r e s s u r e r e c o r d e d d u r i n g t h e p r e v i o u s 1 5 m o

9 2 . 6

9 5

. 8

9 9 . 0

0 . 1 5 ( − 0 . 1 0 t o

0 . 4 0 )

0 . 2 2



8 6 . 5

8 8

. 5

9 8 . 4

0 . 5 8 ( 0 . 1 3 t o 1

. 0 3 )

0 . 0 1

S e r u m c h o l e s t e r o l r e c o r d e d d u r i n g t h e p r e v i o u s 5 y r

7 5 . 1

9 7

. 6

9 9 . 4

0 . 0 5 ( − 0 . 0 8 t o

0 . 1 8 )

0 . 4 7

B l o o d p r e s s u r e c o n t r o l l e d t o ≤ 1 4 0 / 8 5 m m H g ( r e c o r d e d i n

p r e v i o u s 1 5 m o )

2 1 . 8

3 5

. 4

4 9 . 0

0 . 4 9 ( 0 . 0 4 t o 0

. 9 4 )

0 . 0 3

T o t a l s e r u m c h o

l e s t e r o l c o n t r o l l e d t o ≤ 1 9 0 m g / d l ( r e c o r d e d i n p r e v i o u s 5 y r )

2 1 . 8

5 2

. 0

7 2 . 5

0 . 4 2 ( 0 . 0 3 t o 0

. 8 1 )

0 . 0 3

G l y c a t e d h e m o g

l o b i n c o n t r o l l e d t o ≤ 7 . 4 % ( r e c o r d e d i n p r e v i o u s 1 5 m o )

3 7 . 8

3 9

. 8

5 0 . 6

0 . 4 0 ( 0 t o 0 . 8 1

)

0 . 0 5




8/18/2019 NEJM 065990





f a c t o r s

D o c u m e n t a t i o n

o f p a t i e n t e d u c a t i o n i f d i a b e t e s d i a g n o s e d

< 5 y r r e c o r d e d

d u r i n g t h e p

r e v i o u s 5 y r

8 4 . 8

8 7 . 7

1 0 0

0 . 2 9 ( − 0 . 1 4 t o

0 . 7 3 )

0 . 1 7 §

A d v i c e g i v e n t o

s m o k e r s r e c o r d e d d u r i n g t h e p r e v i o u s 5 y r

7 0 . 0

7 8 . 3

9 5 . 2

N A

N A

R e f e r r a l t o a s p e

c i a l i s t i f c r e a t i n i n e > 2 . 2 6 µ g / d l r e c o r d e d d u

r i n g t h e p r e v i -

o u s 5 y r

1 0 0

1 0 0

1 0 0

N A

N A

I n p a t i e n t s a g e < 8 0 y r , t r e a t m e n t o f f e r e d i f a v e r a g e o f t h e t h

r e e p r e v i o u s

b l o o d - p r e s s u r e r e a d i n g s w a s d i a s t o l i c > 1 0 0 o r s y s t o l i c > 1 5 0 a n d d i a -

s t o l i c > 9 0 m

m H g

5 3 . 6

9 0 . 5

6 6 . 7

N A

N A

I n p a t i e n t s t r e a t e d f o r h y p e r t e n s i o n , i f p r o t e i n u r i a p r e s e n t ,

p a t i e n t s s h o u l d

r e c e i v e A C E

i n h i b i t o r

6 8 . 8

1 0 0

7 5 . 0

N A

N A

I n p a t i e n t s s t a r t i n g A C E i n h i b i t o r , m e a s u r e m e n t o f c r e a t i n i n e a n d p o t a s s i -

u m w i t h i n 1

m o b e f o r e t r e a t m e n t s t a r t e d r e c o r d e d d u r i n g t h e p r e v i o u s

5 y r

3 8 . 1

2 9 . 9

3 8 . 8

N A

N A

I n p a t i e n t s r e c e i v i n g s u l f o n y l u r e a , h y p o g l y c e m i a s y m p t o m s r e c o r d e d d u r -

i n g t h e p r e v i o u s 1 5 m o

1 8 . 2

8 . 1

7 . 8

− 0 . 1 2 ( − 0 . 5 7 t o

0 . 3 3 )

0 . 6 0

I n p a t i e n t s ≤ 7 0

y r o f a g e , i f p r e v i o u s g l y c a t e d h e m o g l o b i n m

e a s u r e d w a s

> 9 % , p a t i e n

t s o f f e r e d a t h e r a p e u t i c i n t e r v e n t i o n t o i m p

r o v e g l y c e m i c

c o n t r o l

8 1 . 3

7 1 . 9

6 6 . 0

N A

N A

* C o n d i t i o n a l i n d i c a t o r s a r e i n d i c

a t o r s t h a t d i d n o t a p p l y t o a l l p a t i e n t s . T o

c o n v e r t v a l u e s f o r c h o l e s t e r o l t o m i l l i m o l e s p e r l i t e r , m u l t i p l y b y 0 . 0 2 6 . T o c o n v e r t v a l u e s f o r c r e a t i n i n e t o

m i c r o m o l e s p e r l i t e r , m u l t i p l y b

y 8 8 . 4 . C I d e n o t e s c o n f i d e n c e i n t e r v a l , E C G

e l e c t r o c a r d i o g r a p h y , N A n o t a n a l y z e d ( a n

a l y s i s w a s p e r f o r m e d o n l y w h e n t h e r e w e r

e a t l e a s t 1 0 p r a c t i c e s

w i t h a m i n i m u m

o f f i v e p a t i e n t s e a c h t h a t p r o v i d e d d a t a f o r a n i n d i c a t o r a

t e a c h o f t h e t h r e e t i m e p o i n t s ) , a n d A C E a n g i o t e n s i n - c o n v e r t i n g e n z y m e .

† T h e r e s u l t s a r e b a s e d o n a l l p r a

c t i c e s f o r w h i c h d a t a w e r e a v a i l a b l e .

‡ T h e r e s u l t s a r e b a s e d o n p r a c t i c e s w i t h a t l e a s t f i v e p a t i e n t s w h o h a d t h e c o n d i t i o n a t e a c h o f t h e t h r e e t i m e p o i n t s .

S t a t i s t i c a l t e s t s w e r e n o t p e r f o r m e d f o r g e o g r a p h i c a r e a s i n w h i c h

f e w e r t h a n 1 0 p r a c t i c e s m e t t h i s c r i t e r i o n .

§ R e s u l t s b e c a m e s i g n i f i c a n t ( P =

0 . 0 0 4 ) u s i n g t h e b o o t s t r a p m e t h o d .




8/18/2019 NEJM 065990


special report


matched-pairs two-tailed t-test. In view of therelatively small number of practices included inthe analysis and the distributional assumptionsmade by this test, a bootstrap procedure based on1000 bootstrap samples was used to confirm thesignificance of the tests.

Although the logit model is appropriate to an

analysis of individual binary indicators, the over-all scores for quality used here may not conformto the logit curve as the scores approach theceiling. Therefore we repeated the analysis, usinga linear model applied to the untransformedscores. The linear model makes no adjustmentfor ceiling effects, except that we did not allowpredictions greater than 100%. Hence the resultsof the sensitivity analysis are likely to be conservative.

For comparisons of clinical indicators for whichfinancial incentives were provided with those for

which they were not provided, we derived observedand predicted scores for clinical quality at thepractice level separately for the groups of indi-cators in the management of each condition for which financial incentives were provided and forthose indicators for which they were not provid-ed. The logit transformation was then applied tothese scores. A matched-pairs t-test was used tocompare the difference between the observedscores for indicators for which financial incen-tives were provided and those for which financialincentives were not provided with the predicteddifference. The significance of the test was con-firmed with the use of the bootstrap procedure.

Results

The quality of care in the categories of coronaryheart disease, asthma, and type 2 diabetes im-proved between 2003 and 2005, continuing theearlier trend (Fig. 1). However, the increase in therate of improvement between 2003 and 2005 wassignificant for asthma (P<0.001) and diabetes

(P = 0.002) (Table 2). Scores for coronary heartdisease also increased, but the change in the rateof improvement was not significant (P = 0.07). Sim-ilarly, the sensitivity analysis performed with theuse of the more conservative linear model showedsignificant increases in the rate of improvementfor asthma and diabetes, as compared with therates for coronary heart disease.

On the basis of the conservative linear model,the annual rate of increase in the quality of care

between 2003 and 2005 for diabetes was fasterthan the annual rate between 1998 and 2003 in34 practices and was slower in 8 practices (sig-nificantly so for 13 and 0 practices, respectively;P<0.05). For asthma, the annual rate of improve-ment in the quality of care was unchanged be-tween 1998 and 2003 in 1 practice, faster in 28

practices, and slower in 13 practices (significant-ly so for 10 and 2 practices, respectively; P<0.05).For coronary heart disease, although the annualrate of improvement between 1998 and 2003 wasfaster in 23 practices, it was slower in 17 practic-es (significantly so for 7 and 5 practices, respec-tively; P<0.05).

Observed scores for clinical quality for 1998,2003, and 2005 and the predicted score for 2005for each indicator in each condition are shown inTable 3. The table includes examples of changesin individual clinical indicators that are likely to

be particularly important for improving patientoutcomes, such as control of cholesterol and bloodpressure in the management of coronary heartdisease.14

We then compared changes in the quality ofcare between clinical indicators for which finan-cial incentives were provided in 2004 and thosefor which financial incentives were not provided.The quality of performance for indicators withincentives in all three conditions was substan-tially higher at all three time points than forthose without incentives. However, in all condi-tions, the rate of improvement between 2003 and2005 for clinical indicators for which financialincentives were provided, as compared with thosefor which they were not, did not differ significant-ly from the rate predicted on the basis of the trendbetween 1998 and 2003 (Table 4).

Discussion

Although the quality of care in the categories ofasthma, coronary heart disease, and type 2 dia-

betes was improving before the introduction ofthe 2004 contract, our results suggest that theintroduction of pay for performance was associ-ated with a modest acceleration in improvementfor two of these three conditions: diabetes andasthma. In most of the 42 practices for whichdata were available, the annual improvement forboth was accelerated. The results are based oncare reported in the medical records but not nec-essarily on care provided, and it is a common




8/18/2019 NEJM 065990




criticism of pay-for-performance programs thattheir main effect is to promote better recordingof care rather than better care. However, the pan-els used to develop the indicators judged that to

provide good care, it was necessary both to provide the care and to record the processes and in-termediate outcomes assessed in terms of the in-dicators we used.10

Because some details of the new contract werepublicized before the contract was introduced in2004, it is possible that some practices were pre-paring for the incentives during our second roundof data collection in 2003. If this is true, wemight have overestimated improvements in qual-ity made before pay for performance was intro-duced,4 so that the results reported here couldrepresent a conservative estimate of the actual im-provement resulting from pay for performance.

The key question is whether the increased rateof improvement in quality of care after the newcontract was introduced can be attributed to payfor performance or to other factors. The pay-for-performance program was the only major nationalpolicy implemented in primary care in Englandin 2004 that targeted the types of care processesevaluated in this study. However, since practices were observed at only two time points before the

introduction of pay for performance, we were un-able to determine whether the rate of improvementhad already accelerated as a result of earlier butstill ongoing initiatives. No control group couldbe recruited, because financial incentives wereapplied simultaneously across the whole of theUnited Kingdom. A final concern is that the pa-tients included in the study were selected on thebasis of the presence or absence of treatment withrelevant drugs, and those who were untreated or

who did not comply with treatment were excludedfrom the analysis. This selection bias could haveresulted in overestimation of the quality of careat all three time points, although the trends inquality should have been unaffected.

The study focuses on three chronic conditions— asthma, coronary heart disease, and type 2

diabetes — for which financial incentives wereprovided under the pay-for-performance programand which had also been subject to considerablequality-improvement activity in the United King-dom as part of a national quality-improvementstrategy. The finding of a significant increase inthe rate of improvement for asthma and diabe-tes but not for coronary heart disease may re-flect the fact that in 2003 scores for quality forcoronary heart disease were already higher thanthose for the other two conditions. Coronaryheart disease had been a particular target of ear-

lier quality-improvement initiatives, with 98% ofthe Primary Care Trusts reporting coronary heartdisease initiatives in 2001 and 2002.15

The finding of no significant difference inthe rate of improvement between clinical indica-tors for which financial incentives were provid-ed and those for which they were not providedsuggests that the pay-for-performance programmay not necessarily have been responsible for theacceleration in improvement that we found be-tween 2003 and 2005. However, the study was notdesigned or powered for this analysis, and thebroad confidence limits for many of the clinicalindicators shown in Table 3 reflect the uncer-tainty associated with the small sample availablefor the analysis. In addition, there may have beena “halo effect,” as a result of which some indi-cators for which financial incentives were notprovided may have been indirectly rewarded. Forexample, the clinical indicator “control of totalserum cholesterol in coronary heart disease to190 mg per deciliter (5 mmol per liter) or less,” which in the 2004 contract became an indicator

for which a financial incentive was provided, islikely to have influenced performance on “evi-dence of action being taken if cholesterol wasraised,” a clinical indicator for which a financialincentive was not specifically provided. Improve-ments may therefore have spilled over onto oth-er aspects of care that were not subject to per-formance monitoring. This effect has previouslybeen noted in the Department of Veterans Af-fairs quality-improvement programs.16 The study

Table 4. Mean Difference in Improvement for Indicatorswith and without Incentives.*

CategoryMean Difference

(95% CI) P Value

Coronary heartdisease

0.53 (−0.01 to 1.08) 0.054

Asthma 0.03 (−0.45 to 0.51) 0.904Type 2 diabetes 0.08 (−0.32 to 0.49) 0.682

* The mean difference is the amount by which the ob-served difference between transformed overall scoresfor clinical indicators for which financial incentiveswere or were not provided exceeded the predicted dif-ference. CI denotes confidence interval.




8/18/2019 NEJM 065990


special report


was not able to assess what is perhaps a moreimportant question, namely, what the effect offinancial incentives was on care for conditionsfor which no financial incentives were providedat all.

The introduction of the pay-for-performanceprogram has been associated with a general

trend in the National Health Service away fromplacing implicit trust in health care profession-als and toward more active monitoring of theirperformance than before the program was in-troduced.17 Financial incentives are most likelyto be an effective means of influencing profes-sional behavior when performance targets andrewards are aligned to the values of the staff be-ing rewarded.18,19 Professional motivation alonemay not be sufficient to improve the quality ofcare, especially when physicians have to makefinancial investments in their practices — for

example, by employing more staff to achievegains in quality. Sustained improvement in qual-ity of care, which involves a range of health careproviders (e.g., physicians, nurses, and adminis-trative staff), requires a combination of otherfactors, including clear goals, good teamwork,and effective leadership.20

Owing to the inherent limitations of ourstudy design and data, it was not possible to de-termine whether improvements in quality of careresulted only from the pay-for-performance pro-gram; our f indings are consistent with previous work, which suggested that f inancial incentivescan change professional behavior21,22 and thatpatients receive higher-quality care in geograph-ic areas where performance measures and mon-itoring have been established.16 However, there arealso potential, unintended consequences of suchschemes.1,23 These include the possible neglectof geographic areas where financial incentives forimprovements in care are not provided and of“myopia” (the pursuit of short-term targets atthe expense of legitimate long-term objectives) or

“misrepresentation” (deliberate manipulation ofdata so that reported behavior differs from actualbehavior).24 In addition, external incentives maycrowd out motivation — the desire to do a task well for its own sake.25,26 In the United Kingdom,family practitioners have predicted that amongthe adverse consequences of financial incentivesmay be a reduction in the continuity of care,fragmentation of care as a result of specialization within practices, and neglect of conditions for

which financial incentives are not provided.27 Despite these concerns, overall job satisfactionamong family physicians was higher in 2004than in 2001.28 Moreover, a recent report fromthe United States suggests that targeted quality-improvement programs have not resulted in adeterioration in the quality of care in untargeted

disease areas.29 Our results generally supportthe view of the Institute of Medicine that pay-for-performance programs can make a usefulcontribution to improving quality,30 particularly when such programs are part of a comprehensivequality-improvement program.31

The size of the gains in quality in relation tothe costs of pay for performance remains a po-litical issue in the United Kingdom,32 and thegovernment now accepts that it paid more thanit had expected to pay for the improvements inperformance.33 The proportion of practice income

taken as profit by general practitioners appearsto have increased after the new contract was in-troduced, suggesting that gains in quality couldhave been achieved at a lower cost. For the years2006 through 2007, the pay-for-performanceframework has been amended to introduce high-er payment thresholds, new targets, and new dis-ease areas34 without increasing physicians’ max-imum available income from incentive payments.Physicians in the United Kingdom may now needto work harder or employ more staff to earn thesame rewards that they had received before 2006.

Supported by the U.K. Department of Health. The views pre-sented here are those of the authors and not necessarily of theU.K. Department of Health.

Dr. Roland reports serving as an academic advisor to the gov-ernment and the British Medical Association negotiating teamsduring the development of the United Kingdom pay-for-perfor-mance scheme during 2001 and 2002. No other potential con-flict of interest relevant to this article was reported.

From the National Primary Care Research and DevelopmentCentre, University of Manchester, Manchester, United King-dom. Address reprint requests to Dr. Campbell at the NationalPrimary Care Research and Development Centre, Universityof Manchester, Oxford Rd., Manchester M13 9PL, UnitedKingdom.

Roland M. Linking physician pay to quality of care: a majorexperiment in the United Kingdom. N Engl J Med 2004;351:1448-54.

Doran T, Fullwood C, Gravelle H, et al. Pay-for-performanceprograms in family practices in the United Kingdom. N Engl JMed 2006;355:375-84.

Campbell S, Steiner A, Robison J, Webb D, Raven A, RolandM. Is the quality of care in general medical practice improving?Results of a longitudinal observational study. Br J Gen Pract2003;53:298-304.

Campbell SM, Roland MO, Middleton E, Reeves D. Improve-ments in the quality of clinical care in English general practice

1.

2.

3.

4.




8/18/2019 NEJM 065990


special report


1998-2003: longitudinal observational study. BMJ 2005;331:1121-3.

Campbell S, Wilkin D, Roland M. Primary care groups: im-proving quality of care through clinical governance. BMJ 2001;322:1580-2.

Campbell SM, Roland MO. The experience of the UnitedKingdom. In: Garcia-Pena C, Munoz O, Duran L, Vazquez F, eds.Family medicine at the dawn of the 21st century. Mexico City:CAMS, 2005:391-416.

Baker D, Middleton E. Cervical screening and health in-equalit y in England in the 1990s. J Epidemiol Community Health2003;57:417-23.

Middleton E, Baker D. Comparison of social distr ibution ofimmunisation with measles, mumps, and rubella vaccine, En-gland, 1991-2001. BMJ 2003;326:854.

Campbell SM, Hann M, Hacker J, et al. Identifying predic-tors of high quality care in English general practice: observa-tional study. BMJ 2001;323:784-7.

Campbell SM, Roland MO, Shekelle PG, Cantril l JA, BuetowSA, Cragg DK. Development of review criteria for assessing thequality of management of stable angina, adult asthma and non-insulin dependent diabetes mellitus in general practice. QualHealth Care 1999;8:6-15.

Campbell SM, Hann M, Hacker J, Durie A, Thapar A, RolandMO. Quality assessment for three common conditions in pri-

mary care: validity and reliability of review criteria developed byexpert panels for angina, asthma and type 2 diabetes. Qual SafHealth Care 2002;11:125-30.

Brook RH, Chassin MR, Fink A, Solomon DH, Kosecoff J,Park RE. A method for the detailed assessment of the appropri-ateness of medical technologies. Int J Technol Assess HealthCare 1986;2:53-63.

Collett D. Modelling binary data. London: Chapman andHall, 1991.

McElduff P, Lyratzopoulos G, Edwards R, Heller RF, ShekelleP, Roland M. Will changes in primary care improve health out-comes? Modelling the impact of financial incentives introducedto improve quality of care in the UK. Qual Saf Health Care2004;13:191-7.

Campbell S, Wilkin D. Clinical governance. In: Wilkin D,Coleman A, Dowling B, Smith K, eds. The National Tracker Sur-

vey of primary care groups and trusts 2001/2002: taking respon-sibility? Manchester, United Kingdom: University of Manchester,National Primary Care Research and Development Centre, Uni- versity of Manchester, 2002. (Accessed June 21, 2007, at http:// www.npcrdc.ac.uk/Publications/TRACKER_REPORT_2002.pdf.)

Asch SM, McGlynn EA, Hogan MM, et al. Comparison ofquality of care for patients in the Veterans Health Administra-tion and patients in a national sample. Ann Intern Med 2004;141:938-45.

Checkland K, Marshall M, Harrison S. Re-thinking account-ability: trust versus confidence in medical practice. Qual SafHealth Care 2004;13:130-5.

Marshall M, Smith P. Rewarding results: using financial in-

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

17.

18.

centives to improve quality. Qual Saf Health Care 2003;12:397-8.

Spooner A, Chapple A, Roland M. What makes British gen-eral practitioners take part in a quality improvement scheme? J Health Serv Res Policy 2001;6:145-50.

Campbell S, Steiner A, Robison J, et al. Do Personal MedicalServices contracts improve quality of care? A multi-method eval-uation. J Health Serv Res Policy 2005;10:31-9.

Gosden T, Forland F, Kristiansen IS, et al. Impact of pay-

ment system on behaviour of primary care physicians: a system-atic review. J Health Serv Res Policy 2001;6:44-55.Epstein AM, Lee TH, Hamel MB. Paying physicians for high-

quality care. N Engl J Med 2004;350:406-10.McGlynn EA. Intended and unintended consequences: what

should we real ly worry about? Med Care 2007;45:3-5.Smith P. On the unintended consequences of publishing per-

formance data in the public sector. Int J Publ Admin 1995;18:277-310.

Deci EL, Koestner R, Ryan RM. A meta-analytic review ofexperiments examining the effects of extrinsic rewards on in-trinsic motivation. Psychol Bull 1999;125:627-68.

Gagné M, Deci EL. Self-determination theory and work mo-tivation. J Organ Behav 2005;26:331-62.

Roland MO, Campbell SM, Bailey N, Whalley D, Sibbald B.Financial incentives to improve the quality of primary care in

the UK: predicting the consequences of change. Prim HealthCare Res Dev 2006;7:18-26.

Whalley D, Bojke C, Gravelle H, Sibbald B. GP job satisfac-tion in view of contract reform: a national survey. Br J Gen Pract2006;56:87-92.

Ganz DA, Wenger NS, Roth CP, et al. The effect of a qualityimprovement initiative on the quality of other aspects of healthcare: the law of unintended consequences? Med Care 2007;45:8-18.

Fisher ES, Davis K. Pay for performance — recommendationsof the Institute of Medicine. N Engl J Med 2006 (Web only). (Avail-able at http://content.nejm.org/cgi/content/full/NEJMp068216/DC1.)

Committee on Redesigning Health Insurance PerformanceMeasures, Payment, and Performance Improvement Programs.Rewarding provider performance: aligning incentives in Medi-

care. Washington, DC: National Academies Press, 2007. (Ac-cessed June 21, 2007, at http://www.nap.edu/catalog/11723.html.)

Galvin R. Pay-for-performance: too much of a good thing?A conversation with Martin Roland. Health Aff (Millwood)2006;25:w412-w419.

GP pay raise ‘was mistake.’ BBC News Health. (Accessed June21, 2007, at http://news.bbc.co.uk/player/nol/newsid_6280000/newsid_6280000/6280077.stm?bw=bb&mp=rm.)

British Medical Association. Quality and outcomes frame- work guidance: summary of indicators — clinical domain. (Ac-cessed June 21, 2007, at http://www.bma.org.uk/ap.nsf/Content/qof06~summclinical#SecondaryPreventionofCoronary.)Copyright © 2007 Massachusetts Medical Society.

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.

29.

30.

31.

32.

33.

34.



NEJM 065990

Documents

Transcript of NEJM 065990