Measuring health system performance: Problems and ...Accept complexity Make selective use of...
Transcript of Measuring health system performance: Problems and ...Accept complexity Make selective use of...
![Page 1: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/1.jpg)
Measuring health system performance: Problems and Opportunities in the Era of Assessment and Accountability
C. David Naylor*,Karey Iron† and Kiren Handa†
*Faculty of Medicine, University of Torontoand
† Institute for Clinical Evaluative Sciences
![Page 2: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/2.jpg)
Health Care Reform: No Magic BulletsHealth Care Reform: No Magic Bullets
• Incrementalism rules
• Bottom up : improvements in services, measurement–driven
• Top down : policy adjustments, experience/evidence-based
![Page 3: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/3.jpg)
The Health Care Decision-Making TriadThe Health Care Decision-Making Triad
EVIDENCEEVIDENCEEVIDENCE
VALUES or CULTUREVALUES or CULTUREVALUES or CULTURE
CIRCUMSTANCES or CONTEXTCIRCUMSTANCES or CONTEXTCIRCUMSTANCES or CONTEXT
![Page 4: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/4.jpg)
Some Drivers of the MeasurementSome Drivers of the MeasurementMovementMovement
• Fiscal/Management Imperatives• Public Expectations• Ubiquity of Variations and Errors• Rising Stakes: Post-Genomic Technology
![Page 5: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/5.jpg)
![Page 6: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/6.jpg)
Average life expectancy at birth in 13 OECDAverage life expectancy at birth in 13 OECDcountries, 1960-96countries, 1960-96
60
65
70
75
80
85
19601964
19681972
19761980
19841988
19921996
Life
exp
ecta
ncy
at b
irth
Males
Females
WHO 00193
Hurst J. Bulletin of the World Health Organization, 2000, 78 (6)
![Page 7: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/7.jpg)
EXAM QUESTION:EXAM QUESTION:
Health care has only modest effects onHealth care has only modest effects onpopulation health status.population health status.
Should measures of health systemShould measures of health systemperformance include population healthperformance include population healthIndicators as outcomes or as confounders?Indicators as outcomes or as confounders?
PLEASE ANSWER IN 200 WORDS OR LESS…PLEASE ANSWER IN 200 WORDS OR LESS…
![Page 8: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/8.jpg)
Population Health: Major DeterminantsPopulation Health: Major Determinants
Health StatusHealth StatusHealth Status
Health ServicesSystem
Health ServicesHealth ServicesSystemSystem
EnvironmentalEnvironmentalEnvironmental
LifestyleLifestyleLifestyle
GeneticEndowment
GeneticGeneticEndowmentEndowment
Productivity& Wealth
ProductivityProductivity& Wealth& WealthSocioeconomicSocioeconomicSocioeconomic
![Page 9: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/9.jpg)
Measurement Priorities for ProgramsMeasurement Priorities for Programsand Interventionsand Interventions
• Strong evidence for impacts on healthstatus
• High Costs or high risks
• High variation
![Page 10: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/10.jpg)
AMI Patients across income quintileAMI Patients across income quintilein Ontario, Canadain Ontario, Canada
1 2 3 4 5Neighborhood median income quintile
base
line
fact
ors
Nagemale
P < 0.001
Male = 63.1%
Source: Alter et al. 1999
![Page 11: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/11.jpg)
0.75
0.8
0.85
0.9
0.95
1
0 91 182 273 364
Time from Acute Myocardial Infarction (days)
Prop
ortio
n of
pat
ient
s su
rviv
ing
0.75
0.8
0.85
0.9
0.95
1
0 91 182 273 364
Time from Acute Myocardial Infarction (days)
Prop
ortio
n of
pat
ient
s su
rviv
ing
LowestLowestquintilequintile
HighestHighestquintilequintile
Log-rank test; p<0.001Log-rank test; p<0.001
Survival post-AMI by SESSurvival post-AMI by SES
Source: Alter et al. 1999
![Page 12: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/12.jpg)
Accept complexity
Make selective use of composite measures
Different measures to inform differentstakeholders
![Page 13: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/13.jpg)
�
�
��
��
�
92
94
9593
96
97
98
Number alive
Num
ber d
i ed
Example of Shewhart’s control chart to describe process variation
Mortality ofwomen aged 65 + in2 UK towns,1992-98
� Mortality by year ofDr. Harold Shipman’swomen patients
Source: Mohammed MA et al. Lancet 2001
![Page 14: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/14.jpg)
Statistical Significance versusStatistical Significance versusPolicy RelevancePolicy Relevance
• Population health impact
• Budgetary impact
• ‘Identifiable victims’
• Font-size of headlines!
![Page 15: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/15.jpg)
Focus on assembling accurate,reliable, and relevant data
![Page 16: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/16.jpg)
Talk with different audiencesabout their data needs and
concerns before(re-) designing information
systems
![Page 17: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/17.jpg)
Standardize and fill the ‘black holes’ inStandardize and fill the ‘black holes’ inhealth info-structurehealth info-structure
• Standardize definitions of inputs & outputs
• Better characterization of providers, patients,processes, and outcomes
• Address ‘black holes’: I.e. waiting lists, primary/ambulatory care, long-term and rehabservices
![Page 18: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/18.jpg)
Supplement existing data on aproject-specific and time-limitedbasis with clear analytical goals.
![Page 19: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/19.jpg)
Acquire data to follow patients throughepisodes of illness and
across the care continuum
Integrated data can help reducedis-integration of health services
![Page 20: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/20.jpg)
Take advantage of technology andupgrade info-structure
as necessary
![Page 21: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/21.jpg)
HOW…to achieve IMPROVEMENTHOW…to achieve IMPROVEMENT
ALIGN:• Information systems• Performance measures• Reporting systems• Organizational/Professional culture• Implementation mechanisms
![Page 22: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/22.jpg)
0%
20%
40%
60%
80%
100%
Less than 10 minutes Longer than 20 minutes
Australia Canada New Zealand United Kingdom United States
Length of Most Recent Doctor Visit
43%33%
43%
65%
30%
13%23%
18% 15%
33%
1998 Commonwealth Fund International Policy Health Survey
![Page 23: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/23.jpg)
Respondents reporting time theirdoctor spent with them was too short
23%
14%
14%
15%
13%
0% 20% 40% 60% 80% 100%
United States
UnitedKingdom
New Zealand
Canada
Australia
23%
14%
14%
15%
13%
0% 20% 40% 60% 80% 100%
United States
UnitedKingdom
New Zealand
Canada
Australia
1998 Commonwealth Fund International Policy Health Survey
![Page 24: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/24.jpg)
0
5
10
15
20
Limited Moderate Severe
Threat to life from Coronary Blockages
Age 20-64Age 65-74Age ≥≥≥≥ 75
Rel
ativ
e C
AB
G ra
te (N
Y to
ON
)Coronary surgery: OntarioCoronary surgery: Ontario vs vs New York New York
7.3
10.8
16.8
0.8 1.22.2 2.0 2.5
4.5
![Page 25: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/25.jpg)
Percentages of Senior Citizens receivingCardiac Procedures after Heart Attacks in
the United States and Ontario, Canada,1991
* The coronary angiography rates do not include procedures performedon an outpatient basis in either country
ProcedureUnited States
Ontario, Canada Relative Rate
United States
Ontario, Canada Relative Rate
Coronary angiography* 35 6 5 37 10 4PTCA 12 1 8 13 3 5CABG 10 1 8 12 3 4CABG + PTCA 22 3 8 25 6 4
ProcedureUnited States
Ontario, Canada Relative Rate
United States
Ontario, Canada Relative Rate
Coronary angiography* 35 6 5 37 10 4PTCA 12 1 8 13 3 5CABG 10 1 8 12 3 4CABG + PTCA 22 3 8 25 6 4
![Page 26: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/26.jpg)
Tu et al. 1997. NEJM 336 (21):1500-1505
![Page 27: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/27.jpg)
EQUOL GUSTOEQUOL GUSTO
Domains AssessedQuality of LifeFunctional Status
Employment Status/ role functioningSymptoms (chest pain, dyspnea)
Psychological well-beingTime trade-off (QALY)General Health Status
Economics/Resource ConsumptionHospitalizations
Cardiac catheterizationRevascularizationOutpatient care
Source: Mark et al NEJM 1994;331:1130-5
![Page 28: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/28.jpg)
Diminishing Marginal Returnsof Health Care
Resources for Health Care
Hea
lth S
tatu
s
AA
BB
CC
![Page 29: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/29.jpg)
![Page 30: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/30.jpg)
Rating of zerostars for a dozenBritish hospitals
Medical PostOctober 16, 2001
![Page 31: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/31.jpg)
“There has always been a convenient excuse when there is aproblem, but these tables explode the myth once and for all.”
Alan MilburnHealth Secretary
“I worry that strident reporting of a hospital’s weaknesses candent the confidence of the community in its local hospital,adversely affect recruitment and further damage the morale ofclinical staff.”
Peter HawkerChairman, BMA Consultants’ Committee
Medical Post, October 16, 2001
![Page 32: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/32.jpg)
HOW…to achieve IMPROVEMENTHOW…to achieve IMPROVEMENT
ALIGN:• Information systems• Performance measures• Reporting systems• Organizational/Professional culture• Implementation mechanisms
![Page 33: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/33.jpg)
Regulatory Admin/Professional
Market-based
EconomicNon-economic
IncentivesIncentives
MechanismsMechanisms
ActorsActorsConsumersPurchasers/FundersProfessionals/Managers
![Page 34: Measuring health system performance: Problems and ...Accept complexity Make selective use of composite measures ... Integrated data can help reduce dis-integration of health services.](https://reader034.fdocuments.us/reader034/viewer/2022051921/600eba73a66a8544f7376f4c/html5/thumbnails/34.jpg)
Measurement and ManagementAssessment and AccountabilityOnwards and Upwards!