1 04/19/23 07:38 LifePaths

On Measuring Outcomes and Productivity in Canada’s

Health Care Sector first principles and basic definitions the UK Atkinson Report, the System of

National Accounts (SNA), and the “standard” approach to measuring (health sector) productivity

empirical nuggets and “awkward facts” alternative and better approaches

Michael Wolfson

Statistics Canada

NB – More detail can be seen in the “notes view”; still draft, please do not circulate without permission

2 04/19/23 07:38 LifePaths

First Principles - I

Population health is the fundamental objective Health outcomes relate to / depend on health

interventions• i.e. social activities, whether deliberate or

inadvertent Health care is one kind of intervention which often contributes to health• but not always, and certainly not solely

3 04/19/23 07:38 LifePaths

First Principles - II

Doing more with less is a good thing• i.e. being more efficient or more

productive is beneficial People and care providers are

heterogeneous• so summing or averaging to produce

overall indices can produce misleading results

4 04/19/23 07:38 LifePaths

Definition (by Construction) - Population Health I

ask everyone (or a sample thereof) a structured set of questions (or do an exam) to assess each person’s health status• i.e. a profile for each person (n.b. gives micro

detail) construct an index for each person, based on

their health profile• e.g. McMaster Health Utility Index, or QALY

average over people (perhaps age-standardized)

5 04/19/23 07:38 LifePaths

Definition - Population Health II

or combine individual-level summary health indices with life table (mortality rates) to measure Health-Adjusted Life Expectancy (HALE)

6 04/19/23 07:38 LifePaths

“Cause – Deleted” Changes in Life Expectancy (LE) and Health-Adjusted Life

Expectancy (HALE)

2.4

1

0

0.5

0.4

0.4

0.3

0.1

0

0.7

1.8

0.7

0.6

0.8

0.3

0.4

0.3

0

0

0.4

00.511.522.5

IHD

Lung cancer

Breast cancer

Stroke

COPD

Colorectal cancer

Diabetes

Melanoma

Osteoarthritis

Mental disorders

Men

Women

Source: Manuel et al, ICES and Health Canada, NPHS

HALELE

2.2

0.9

0.7

0.5

0.4

0.5

0.2

1

0.9

1.5

0.6

0.5

0.7

0.5

0.3

0.4

0.1

2.4

1.1

0 0.5 1 1.5 2 2.5

Men

Women

7 04/19/23 07:38 LifePaths

Definition - Health Outcome

health status “before”

health status “after”

health intervention

other factors

health outcome change in health status attributable to a health intervention

(for an individual)

8 04/19/23 07:38 LifePaths

Charles Wright on Vancouver Cataracts pre- and post-surgery patient self-completed questionnaires “31% of patients booked for cataract surgery report a visual

function score of 91 points or more on a scale of 100. … “These data tend to confirm the observation that cataract

surgery is now occurring in many patients with minor degrees of self-reported visual disability. …

“The overall results are positive, but 27% of patients show either no change or deterioration of VFA (Visual Function Assessment) score after the operation.”

9 04/19/23 07:38 LifePaths

Relative Risks of “Preventive” Tamoxifen0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5

(9.27)

(Fisher et. al., J National Cancer Institute, 1999)

Breast Ca

Fractures

CHD

Cataracts

Stroke

Deep Vein Thromb

Endometrial Ca

Pulmonary Emb

10 04/19/23 07:38 LifePaths

-0.04

-0.02

0

0.02

0.04

0.06

0.08

0.1

0.12

1.66 2.08 2.49 2.91 3.32 3.74 4.15

5-yr Predicted Risk / Proportion of Women Affected

Ch

an

ge

in

Lif

e E

xp

ec

tan

cy

Simulated Change in Life Expectancy for Canadian Women for Alternative Scenarios of Preventive Tamoxifen

(95% CIs)

42.3% 24.6% 16.2% 9.0% 4.0% 2.2% 1.7%

(Will et. al., British J Cancer, 2001)

11 04/19/23 07:38 LifePaths

Definition – Health Care, per the Evans and Stoddart “Plumbing Diagram”

“bottom line”

“thermostat”

12 04/19/23 07:38 LifePaths

(Tu et al on Coronary Surgery)

n.b. virtually no differences in one year survival; but no data on differences in health-related QoL

e.g. almost 17x, with no

benefits?

13 04/19/23 07:38 LifePaths

0

2

4

6

8

10

12

14

16

18

0 20 40 60 80 100 120

Mortality Rate(%, age 65-70)

Career Earnings and Deathfor 500,000 Canadian Men

top quintile

(Career Earnings and Death)

Average Earnings (age 45-64, 1988 $000s)

Source: Wolfson et al., Gerontology, 1993

14 04/19/23 07:38 LifePaths

Definition – Productivity (General)

productivity level “output” / “input” productivity growth growth in outputs - growth in

inputs i.e. getting more output for given inputs, or getting the

same output from fewer inputs n.b. in common parlance

• no presumption that everything has to be measured in $$$• indeed, usual thoughts are in physical units (e.g. patients seen

or cataracts done per day)

15 04/19/23 07:38 LifePaths

Definition – Productivity (business school)

let me compare myself to another firm, typically a competitor

is she producing her widgets at lower unit costs than me?

i.e. benchmarking for individual product lines

16 04/19/23 07:38 LifePaths

Definition – Productivity (“standard” economics and SNA)

the economy has myriad productive agents (firms)• each of whom uses inputs = total capital services + total labour

services (factors of production)• to produce outputs (goods and services) summing to GDP

everything is measured in $ -- with the total being (conceptually) the sum of unit prices x quantities• but over time, prices (p’s) change, and this is not “real” • and quantities (q’s) change e.g. in terms of “quality”

to measure productivity, time series of outputs and inputs are constructed• taking out “pure” price changes, and• adjusting for improvements in quality• so that productivity = output – sum { inputs }

17 04/19/23 07:38 LifePaths

Definition – Health Sector Productivity (“standard” economics and SNA)

“standard” economists and National Accountants want to treat “health care” as an industry, with “health care services” its outputs, analogous to private sector industries

“firms” in the health sector are divided (mainly) into hospitals, nursing homes, and providers of ambulatory care (OECD SHA)• n.b. no concept of “regional health authority” or “local health

integration network”

ideal concept for “outputs” is care for “episodes of illness”, though DRGs in practice (OECD SHA)• n.b. no concept of “continuum of care”, nor chronic illness, nor

recognition of co-morbidities

18 04/19/23 07:38 LifePaths

Public Sector Challenge to “Economists’ Productivity”

???“Profits”

Inputs

Commercial Sector

Public Sector

Outputs

19 04/19/23 07:38 LifePaths

UK Office of National Statistics (ONS) and their “Productivity Paradox”

UK Labour Government massively increased spending in health care (and education) starting in the late 1990s

the ONS had been dutifully measuring public sector productivity, using SNA / economists’ concepts

productivity declined (unfortunately) ONS (2004) asked Sir Tony Atkinson what to do (and many jurisdictions are considering the Atkinson

report recommendations, as well as gradually adopting the OECD’s System of Health Accounts)

20 04/19/23 07:38 LifePaths

ONS, Atkinson, and Productivity mandate from ONS National Statistician:

• “To advance methodologies for the measurement of government output, productivity and associated price indices” (OK)

• “in the context of the National Accounts” (Oh oh!) question: why not first pose issue in general and then only secondarily ask whether SNA is an

appropriate framework, and if not what would be?

21 04/19/23 07:38 LifePaths

Atkinson Report – Analysis I SNA data are the essential foundation

• for macroeconomic management• and as an indicator of social welfare

UK context per Bank of England (May 04)• CPI up 10% from 1997Q1 to 2003Q4• nominal government spending up 62%• ONS measure of real public sector output up 14% (Huh?)

GDP as welfare measure• more $ on (e.g.) health care treatments increases welfare –

certainly if appropriate and effective, but “asymmetric information”

22 04/19/23 07:38 LifePaths

Atkinson Report – Analysis II “National Accounts are not a substitute for

performance indicators” (para 1.27) “It is not necessarily the case that even a crude

measure of government output is preferable to an index based on total cost.” (para 2.25) – i.e. the conventional way of doing the SNA

measurement of quality change (e.g. improvements in methods and technology) is a major challenge

the UK, as part of the EU, is bound to measure SNA according to international standards (but the US and Canada, so far, have ignored these)

23 04/19/23 07:38 LifePaths

Atkinson Report – Selected Recommendations

the SNA should measure government non-market output (e.g. health care services) using a procedure parallel to that of the market sector (para 4.7)

use the “treatment” or GP visit as the canonical “output” of the health care sector

weight different kinds of treatment by their costs try to adjust for quality change – ideally by moving from

treatments to “care pathways”, and connecting care to health outcomes• n.b. sounds good, but feasibility?

also treat shorter waits as improved quality• n.b. nothing on appropriateness, or “watchful waiting”

24 04/19/23 07:38 LifePaths

“Technology Adoption from Hybrid Corn to

Beta Blockers”, Skinner and Staiger,

NBER, 2005

corn

corn corn

tractors

computers beta blockers

(Skinner I)

25 04/19/23 07:38 LifePaths

(education and) social capital

beta blockers

beta blockers

adjusted 1 yr mortality rate

$$$(Skinner II)

adjusted 1 yr mortality rate

(education and) social capital

adjusted 1 yr mortality rate

26 04/19/23 07:38 LifePaths

(Skinner III)

(Source: Skinner, Staiger, Fisher; Medical Technology, 2006)

27 04/19/23 07:38 LifePaths

“Wall of Ignorance”

28 04/19/23 07:38 LifePaths

Heart Attack Patients, 2000: Treatment and One Year Mortality Rates for Large Canadian Health Regions

0

5

10

15

20

25

0 10 20 30 40 50 60

percent revascularized within 30 days

percent dead

within one year Alberta

Quebec

British ColumbiaOntario

29 04/19/23 07:38 LifePaths

E. A. Codman and W.E. Deming Codman: early 1900s Boston surgeon famous for “End Results Cards” – to keep

track of surgical patients and follow them up one year later to • observe outcomes

• systematically learn from experience

100 years later: not yet implemented in health care

Deming: post WW II concern with product quality in manufacturing

father of field of statistical process quality control

50 years later: not yet implemented in health care

30 04/19/23 07:38 LifePaths

Underlying Person-Oriented Information (POI) for Heart Attack / Revascularization Analysis

Heart Attack (AMI)Treatment (revascularization = bypass or angioplasty)Death

one year observation window

one year follow-up window(excluded)

31 04/19/23 07:38 LifePaths

Heart Attack Patients, 2000: Treatment and One Year Mortality Rates for Large Canadian Health Regions

0

5

10

15

20

25

0 10 20 30 40 50 60

percent revascularized within 30 days

percent dead

within one year Alberta

Quebec

British ColumbiaOntario

32 04/19/23 07:38 LifePaths

Heart Attack Patients, 2000: Treatment and One Year Mortality Rates for Large Canadian Health Regions

0

5

10

15

20

25

0 10 20 30 40 50 60

percent revascularized within 30 days

percent dead

within one year

better (less intervention, & better survival)

Alberta

Quebec

British ColumbiaOntario

(more output ???)

33 04/19/23 07:38 LifePaths

Hospital 65+ Patient Co-morbidity

CHFHigh BP CPD Diab's Ca

RA etc. Psych Deprn

number (000's) 111 237 128 125 101 16 20 30

pct of all 16.4 35.0 18.9 18.5 14.9 2.3 3.0 4.5

cond'n only (%) 23.7 37.7 28.0 22.8 47.8 27.7 26.0 24.0

cond'n +1 37.1 37.6 38.0 41.9 31.0 36.3 35.1 35.0

cond'n +2 27.5 18.4 23.7 25.2 15.2 23.5 24.6 25.6

cond'n +3 9.9 5.4 8.7 8.5 4.8 9.6 10.6 11.6

based on 676,508 hospital inpatient discharges across 10 provinces in 2001/2

34 04/19/23 07:38 LifePaths

Health Care Outputs or Health Outcomes?

SNA approach: health care inputs health care outputs (i.e. “treatments”)• leave for others to figure out connections from health care outputs

health outcomes (para 7.27, OECD SHA)

public policy priority: what (broad) allocation of resources produces the most “health gain” (i.e. increase in population health) – inputs outcomes• SNA approach is helpful on inputs and costs• though focus on aggregation distracts from “benchmarking”, i.e. “firm”

level analyses• and SNA compulsion to create an artificial concept of “output” is useless

for this purpose

35 04/19/23 07:38 LifePaths

Platitudes?

You can’t manage what you can’t measure

You get what you measure

“Don’t ask how many (health care) events per pound; ask how much health per pound.” D. Berwick, BMJ 2005

36 04/19/23 07:38 LifePaths

(Ontario Framework)

Ontario Health Scorecard

production and use of evidence

sensible allocation

access / waits

continuity of care

health status

clinical results

healthy living

spending / resources

sustainability / equity

37 04/19/23 07:38 LifePaths

Vision – Coherent, Integrated Statistical System

Broad Summary Indicators

Regional Indicators / Planning Info

Facility Management Information / Unit Costs

Basic Encounter Data / Health Surveys

Health Accounts / Simulation Models