Measuring Health System Performance Lecture 3 › fileadmin › a38331600 › 2015.lectures … ·...
Transcript of Measuring Health System Performance Lecture 3 › fileadmin › a38331600 › 2015.lectures … ·...
Measuring Health System Performance
Lecture 3
Health Service OutcomesHealth Service Outcomes
Reinhard Busse, Prof. Dr. med. MPHDepartment of Health Care Management
Berlin University of Technology/
(WHO Collaborating Centre for Health Systems Research and Management)
European Observatory on Health Systems and Policies
Outline
• Concept and background
– Population health vs. Health service indicators
– Health service outcomes = quality?
• Dimensions of quality
• Outcome vs process
• Different type of Health Service Outcome indicators• Different type of Health Service Outcome indicators
– Mortality indicators (general, condition-specific)
– Patient Safety Indicators
– Readmissions
– Ambulatory-care sensitive hospitalisations
– Patient Reported Outcome Measures
– Process indicators
Population health indicators (denominator: population)
Life expectancy/
overall mortality
Health-adjusted
life expectancy
YLD
Tracer: Condition-
specific mortality (e.g.
AMI, breast cancer)
YLL
Amenable/ avoidable
mortality (group of tracers)
Ambulatory-care
sensitive
hospitalisations
Health service indicators
Condition-specific
inpatient mortality
Condition-
specific 5-year
survival (e.g.
breast cancer)
Specific
(tracer)
(denominator: patients)
hospitalisations
Infant mortality
Hospital
mortality
Hospital
readmissions
Patient
safety
indicators
inpatient mortality
(e.g. AMI)
Condition-specific
processes
Patient-reported
outcomes (function,
quality-of-life)
Attributability to
health care provider
generic
„... quality of care is that component of the difference
between efficacy and effectiveness that can be attributed to
care providers, taking account of the environment in which
they work.“
(Brook RH & Lohr KN. Efficacy, effectiveness, variations, and quality. Medical
Health service performance = quality?
• no uniform definition of quality; an often quoted definition is:
(Brook RH & Lohr KN. Efficacy, effectiveness, variations, and quality. Medical
Care 1985; 25: 710-722)
• This definition implies focus on effectiveness (health) as outcome
• OECD definition is wider as it also includes patient safety and
patient-centeredness (which we summarise under patient experience)
Most commonly used quality dimensionsDimension Definition
Safety The degree to which health care processes avoid, prevent
and ameliorate adverse outcomes or injures that stem
from the processes of health care itself
Effectiveness The degree of achieving desirable outcomes, given the
correct provision of evidence-based healthcare services to
all who could benefit but not to those who would not
benefit. Includes appropriateness of care.benefit. Includes appropriateness of care.
Responsiveness How a system treats people to meet their legitimate non-
health expectations
Accessibility The ease with which services are reached
Equity The extent to which a system deals fairly with all concerned
Efficiency The system’s optimal use of available resources to yield
maximum benefits or results.
Source: Kelley, E & Hurst, J. (2006) Health Care Quality Indicators Project: Conceptual
Frameowrk Paper. OECD Working Paper No. 23
Elements of quality and potential
problems
Elements of Quality Care Type of Quality Problem
People get the care they need Underuse
People need the care they get Overuse
Provided safely ErrorProvided safely Error
Timely Delays
Patient-centered Unresponsive
Delivered equitably Disparities
Delivered efficiently Waste
IOM, Crossing the Quality Chasm (2001)
Population
health status
(need)
Other sectors
Nutrition/ agriculture
Environment
Personnel sufficient
and well qualified?
Institutions of high standards?
Technologies effective?
Coverage &
needs-based,
equitable
access?
Patients satisfied,
services safe and
of high quality?
Health care
outcome:
satisfaction,
complications
etc.Structures and
organisation
Patients
Process
Health
gain/
Outcome
Human
resources
Technologies
Financial
resources
Fair and sustainable funding?
Utilization responsive,
appropriate, coordinated …?
Health care system
Re
sou
rce
cre
ati
on
ad
eq
ua
te?
How much?
How equitable?
Are the services delivered efficiently?
Donabedian Quality Assurance Model
Structure:Material Resources
Operational CharacteristicsOrganizational Characteristics
Process:Clinical Care
Policy and ProcedureAdherence to standardsOrganizational Characteristics Adherence to standards
Outcome:Health status of patients
Clinical measures
Suitability of condition-specific mortality
(and amenable mortality) for performance assessment:
the example of AMI mortality in England, 2002-2010
Smolina et al (BMJ, 2012) http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3266430/
Authors ask: how much of this decline is due to a fall in incidence and how much to declines in case fatality?
Contribution of average annual trends in event
rate and case fatality rate to average annual trend
in mortality for AMI by region, 2002-10, England
From a policy From a policy
maker’s perspective:
why is distinguishing
between these
causes important?
Mortality measurement: a valid
indicator for quality of care?
• Increased publication of risk adjusted
hospital mortality rates
• What is the relationship between
mortality rates and quality of care?
• Systematic Review of evidence (2007)• Systematic Review of evidence (2007)
– A positive correlation between better
quality of care and risk adjusted mortality
was found in under half the relationships
• Shahian et al. (2010)
– Risk adjustment method can produce
substantially different results
Methodological Challenges
Reliability of Data
• mortality statistics, registries, administrative data-bases,
electronic health records, survey data
Validity of IndicatorsValidity of Indicators
• -face validity
• content validity
• construct validity
• criterium validity
Hospital Standardised Mortality Ratio (HSMR)
Summary-level In-Hospital Mortality Indicator (SHMI)
Admissions
and deaths included
• All in-hospital deaths
• For the 56 diagnostic groups
(e.g. CCS = clinical
classification software or DRGs)
• All in-hospital deaths and
deaths 30d post discharge
• For all diagnosis super groups
except births, stillbirths
• Age group
• Diagnosis/procedure subgroup
• Comorbidities
• Age group
• Diagnosis
• Comorbidities
14
Risk adjustment
• Comorbidities
• Admission method
• Gender
• Palliative care coding
• Deprivation
• Number of previous emergency
admissions
• Discharge year
• Month of admission
• Source of admission
• Comorbidities
• Admission method
• Gender
Dutch hospital standardised mortality ratios
2001-03 (standardised for age, sex, urgency/readmission, LOS within 50 CCS
groups leading to 80% all deaths, excluding small hospitals and those with poor
data recording, using year 2000 standard)
100
120
140
HS
MR
s (9
5%
CIs
) 200
1-2
003
0
20
40
60
80
96
35
68
14
83
81
51
25
89
50
103 3
52
44
85 5
78
36
12
100
72
94
13
104
65
33
34
95
101
39
93
82
79
23
61
47
37
20
87
97
45
31
107
19
98
54
102
Hospital number (assigned by BJ)
HS
MR
s (9
5%
CIs
) 200
1-2
003
Mortality for particular conditions?
• Select conditions where quality of care is associated with
mortality (AMI, Stroke, Hip)
• Challenges:
– sample size
(ex. Dimick et al. 2004 – studies 7 operations that the Agency for Healthcare (ex. Dimick et al. 2004 – studies 7 operations that the Agency for Healthcare
Research and Quality in the US recommended surgical mortality as a quality
indicator – found that only for 1 (CABG surgery) was sample size large
enough to make quality assessments)
– generalizability of results
(Challenges include: International comparisons of mortality, timing,
measurement)
Source: Dimick et al. (2004) Surgical Mortality as an Indicator of Hospital Quality: The Problem with Small Sample Size. JAMA 292(7):847-851.
Patient Safety Indicators
• Common: infection rates (MRSA, C-difficile)
• Death among surgical inpatients with serious treatable
complications
• Adverse post-op outcomes (i.e. pressure ulcer, hip fracture,
hemorrhage etc)hemorrhage etc)
• Composite - Complication/patient safety for selected
indicators
• Never events, non events (wrong site surgery, wrong
prosthesis, etc)
• Medication errors
http://www.qualityindicators.ahrq.gov/Downloads/Modules/PSI/V41/TechSpecs/PSI%2004%20Death%20among%20Surgical%20Inpatients.pdfhttp://www.england.nhs.uk/wp-content/uploads/2013/12/nev-ev-list-1314-clar.pdf
Patient Safety Indicators
Readmission indicators:
a valid health services outcome measure?
• Various studies suggest Readmissions may not be always indicative of poor quality
• McClellan & Staiger (1999), Papanicolas & Mcguire (2011) – found some conditions had negative correlations between mortality and readmissions – USA and UK samples.
• Laudicella et al. (2013) hospitals’ performance in readmissions is • Laudicella et al. (2013) hospitals’ performance in readmissions is determined in part by their difference in the quality of care and in part by their difference in the share of unobservably sicker patients. (UK sample)
• Fischer et al. (2011): Systematic review of readmission indicators identified only 21 out of 486 studies addressed validity of indicator when using it as an outcome measure.
– Little consensus over time-frame, type of readmission and case-mix adjustment applied.
Ambulatory Care Sensitive Conditions:good ambulatory care should prevent or reduce hospitalisations
What are ACSCs?
Chronic conditions that include congestive heart failure, diabetes, asthma, angina, epilepsy and hypertension.
Actively managing patients Actively managing patients with ACS conditions – through vaccination; better self-management, disease-management or case-management; or lifestyle interventions – prevents acute exacerbations and reduces the need for emergency hospital admission.
http://www.dartmouthatlas.org/data/topic/topic.aspx?cat=25
ACSCs: comparison of Canadian provinces
But is it a valid indicator?
Trends in ACSCs – is primary care in the
UK improving?
Bardsley et al (2012) Is secondary preventive care improving? Observational study of 10-year
trends in emergency admissions for conditions amenable to ambulatory care
http://bmjopen.bmj.com/content/3/1/e002007?cpetoc
Patient Reported Outcome Measures
(PROMs)
• Instruments which gain information about health, illness and
the effects of health care interventions from the perspective
of the patient (Fitzpatrick et al, 1997)
• Four types of PROM:
– Generic– Generic
– Utility
– Disease specific
– Individual
Generic proms are applicable to the
widest possible range of health problems
Utility instruments assign utilities to
respondent’s health states – the EQ5D
Disease-specific proms are tailored to the
specific disease for which they are intended
Individual level proms are tailored to the
specific disease for which they are intended
PROM results in England 2013/14
Participation and
Coverage: There have
been 128,759 PROMs-
eligible procedures
carried out in hospitals
and 98,695 pre-operative
questionnaires returned
so far, a participation rate so far, a participation rate
of 76.7%.
For the 98,695 pre-
operative questionnaires
returned, 44,460 post-
operative questionnaires
were sent out, of which
11,423 have been
returned so far - a return
rate of 25.7% (73.3%).
Yet, how effective are PROMs?
• Fitzpatrick (2009) assess PROMs with respect to 7 criteria:– Reliability
– Validity
– Responsiveness
– Precision
– Interpretability
– Acceptability– Acceptability
– Feasibility
• PROMs are increasingly being used as performance instruments
• Hold much promise – but realising their potential requires: – Credible data collection, instruments and analysis
– Good reporting so that information is useful for different users and for different types of decisions
– Recognitions of limitations
– An open mind to this type of information
Outcome vs Process Measures
The case for measuring outcomes of care
• Central indicator of the success of health care
• Essential for determining “what works” in health care
• Can nurture innovation
The case for measuring processes of care
• Certain aspects of process (such as waiting
times or patient experience) are often valued
by patients
• Certain processes are known to be
associated with desired health outcomes.
• Measuring outcomes can be difficult, costly
and takes a long time -
• Can nurture innovation
• Are universal and do not become easily obsolete
• Clinical attention is focused on securing improved health rather than ‘checklists of activities’
• Harder to manipulate than process measures
Measuring outcomes can be difficult, costly
and takes a long time -
• Process measures are almost instantaneous
and can be acted on quickly.
• Process measures are usually readily
attributable to the provider of care and so
more easily interpretable (as opposed to
outcome measures which display a lot of
random noise).
• It is easier to devise incentive schemes
associated with process rather than
outcomes.
Process Measures
• Process Measures (waiting times) vs Clinical Process Measures (Measuring blood pressure for Hypertensive patients)
• Advantages: fast to collect, easier to attribute • Advantages: fast to collect, easier to attribute directly to health services, reflect compliance with good practice.
• Disadvantages: may be less relevant when considered alone, not always applicable, may become dated.
Developing Clinical Process Measures
• Selecting topics
• Reviewing clinical evidence
• Identifying quality indicators
• Constructing process measures
• Creating scoring methods
Process indicators (examples)
Measure Set Measures
Acute Myocardial Infarction (AMI)
■ Aspirin on Arrival
■ Aspirin Prescribed at Discharge
■ ACEI or ARB for LVSD
■ Beta Blocker Prescribed at Discharge
■ Beta Blocker on Arrival
■ Thrombolytic agent received within 30 minutes of hospital arrival
■ Percutaneous Coronary Intervention within 120 minutes of hospital arrival
■ Adult Smoking Cessation Advice/Counseling
■ LVF Assessment ■ Discharge Instructions
Heart Failure (HF)
■ LVF Assessment
■ ACEI for LVSD
■ Discharge Instructions
■ Adult Smoking Cessation Advice/Counseling
Pneumonia (PN)
■ Initial antibiotic received within 4 hours of hospital arrival
■ Oxygenation Assessment
■ Pneumoccoccal Screening and/or Vaccination
■ Blood Cultures
■ Adult Smoking Cessation Advice/Counseling
■ Appropriate initial antibiotic selection
Surgical Infection Prevention (SIP)
■ Prophylactic antibiotic received within 1 hour prior surgical incision
■ Prophylactic antibiotic discontinued within 24 hours after surgical infection
Patient Safety Indicators (PSI)
■ Postoperative Septicemia
■ Postoperative PE/DVT
■ Infection due to medical care
■ OB trauma without instruments
Interpretation issues: what measures of
quality can and cannot tell you
• MOST indicators require further investigation or
validation before one can be confident that it
indicates ‘good’ or ‘bad’ quality.
• Often our assessment depends on where an Often our assessment depends on where an
organization/physician/unit is placed in relation to
others rather than to an absolute standard, but this
can also be influenced by OTHER factors:
– Data issues, differences in clinical practice, external
factors, random variation, all of the above