Disability weights measurement in the Global Burden of Disease Study 2010
-
Upload
institute-for-health-metrics-and-evaluation-university-of-washington -
Category
Health & Medicine
-
view
107 -
download
0
description
Transcript of Disability weights measurement in the Global Burden of Disease Study 2010
Disability weights measurement in the Global Burden of Disease Study 2010
Joshua A Salomon
Harvard School of Public Health
Global Health Metrics and Evaluation Conference
June 18, 2013Supported by funding from the Bill and Melinda Gates Foundation
Salomon – Disability weights - 2 2
Global Burden of Disease Study
• Global Burden of Disease (GBD) Study aims to measure impact of disease and injury in terms of losses in population health
• GBD 1990 study launched in 1991, updated by WHO in 2000s• GBD 2010 study, undertaken from 2007-2012, provides first
comprehensive overhaul since 1996
• GBD quantifies • Magnitude of different health problems in units of disability-adjusted
life years (DALYs)• Overall population health in units of healthy life expectancy (HALE)
Salomon – Disability weights - 3 3
Disability weights
• Disability weights provide the bridge between mortality and non-fatal outcomes in DALYs and in healthy life expectancy
• To measure health impact of non-fatal outcomes, GBD needs weights for all unique sequelae, which capture the major health consequences of all of the causes in the study
• Disability weights quantify severity of outcomes as percentage reductions from perfect health, which are multiplied by years lived in each sequela to give years lived with disability (YLD) e.g., If weight for blindness were 0.20, then 5 years lived with
blindness would be equivalent to dying one year prematurely
Salomon – Disability weights - 4
Intervention 2:Prevent 1y of deafness for
2000 individuals
Intervention 1:Extend life by 1y in
1000 healthy individuals
Disability weights in the 1996 GBD revision
Person trade-off: which would you choose?
• Expert panel used ‘person trade-off’ to assign values to 22 indicator conditions
Salomon – Disability weights - 5
1
Disability weights
2 3 4 5 6 7
Class 1:• Vitiligo on face
Class 4:• Below-knee amputation• Deafness
Class 7:• Active psychosis• Quadriplegia
Disability weights in the 1996 GBD revision
• Expert panel used ‘person trade-off’ to assign values to 22 indicator conditions
• These 22 conditions used as operational definitions of 7 disability classes
Salomon – Disability weights - 6
1
Disability weights
2 3 4 5 6 7
Rheumatoid arthritis cases
Average disability weight=0.2*0.07 + 0.4*0.18 + 0.4*0.30=0.21
Disability weights in the 1996 GBD revision
• Expert panel used ‘person trade-off’ to assign values to 22 indicator conditions
• These 22 conditions used as operational definitions of 7 disability classes
• Remaining conditions allocated across classes to compute average weights
Salomon – Disability weights - 7 7
Disability weights measurement study goals
• Derive weights for all 220 health states capturing nonfatal outcomes from 291 disease and injury causes in GBD 2010
• Address criticisms of previous approaches by: Focusing on valuations from community respondents… … in a diverse range of settings … using suitable measurement methods
• Specific research aims Develop valid and reliable data collection tools for population-
based surveys Empirical examination of variation in weights
Salomon – Disability weights - 8
• Background• Study design and methods• Key findings• Interpretations, limitations, conclusions
Disability weights measurement in GBD 2010
Salomon – Disability weights - 9 9
Study components
• Population-based household surveys• Face-to-face interviews in Tanzania,
Bangladesh, Indonesia, Peru• Telephone interview in random sample of
US households• Focus on paired comparisons for 108
health-states• Key objectives include comparative analysis
across diverse settings and benchmarking Internet survey against community samples
• Open-access Internet surveys• Available in English, Spanish and Mandarin• Key objectives are to fill in gaps with
remaining sequelae and to anchor scale for paired comparison responses
Bangladesh
Tanzania
Salomon – Disability weights - 10
Web survey
Web survey included 16,328 respondents from 167 countries
Salomon – Disability weights - 11
Web survey included 16,328 respondents from 167 countries
Web survey
1 - 9
10 - 49
50 - 99
100 - 499
500+
Salomon – Disability weights - 12 12
Measurement methods: paired comparisons
• Primary mode of eliciting responses is paired comparison• Respondents hear (or read) two descriptions of hypothetical people,
each with a randomly selected condition• Respondents indicate which person is healthier
• Paired comparison questions chosen for relative ease of comprehension, administration and analysis• Literacy and numeracy not essential• Health comparisons not tied to external “calibrators” such as risk• Appealing intuitive basis and established strategies for analysis
Salomon – Disability weights - 13 13
Framing paired comparisons
• Basis for all comparison are lay descriptions of sequelae, which highlight major functional consequences and symptoms associated with each sequela
• Must be brief: restricted to <35 words based on pretest results• Must use simple, non-clinical vocabulary
• Prologue to paired comparison questions orients respondents to focus on functioning
A person’s health may limit how well parts of his body or his mind work. As a result, some people are not able to do all of the things in life that others may do, and some people are more severely limited than others.
I am going to ask you a series of questions about different health problems. In each question I will describe two different people … Imagine they have the same number of years left to live, and will experience the health problems that I describe for the rest of their lives. I will ask you to tell me which person you think is healthier overall, in terms of having fewer physical or mental limitations on what they can do in life…
Salomon – Disability weights - 14
Paired comparison example
The first person has vision problems that make it difficult to see and recognize faces of family or friends across a room.
The second person has severe back and leg pain, which causes difficulty dressing, sitting, standing, walking, and lifting things. The person sleeps poorly and feels worried.
Imagine that both people will have these problems for the rest of their lives. Who would you say is healthier overall, the first person or the second person?
• Respondents in household surveys each answered 15 of these questions, with pairs of sequelae drawn at random from the universe of 108 × 108 possible pairs
Salomon – Disability weights - 15 15
Analyzing paired comparison data
Bangladesh Tanzania• Simple ordering of
outcomes based on “winning” proportions, as in various websites using paired comparisons to rank large pools of competitors
http://kittenwar.com/
Salomon – Disability weights - 16 16
Analyzing paired comparison data
Bangladesh Tanzania
Won 78% of 511 battles Won 77% of 539 battles
Winningest
Lost 80% of 846 battles Lost 80% of 1872 battles
Losingest
• Simple ordering of outcomes based on “winning” proportions, as in various websites using paired comparisons to rank large pools of competitors
http://kittenwar.com/
Salomon – Disability weights - 17 17
Analyzing paired comparison data
Bangladesh Tanzania• Simple ordering of
outcomes based on “winning” proportions, as in various websites using paired comparisons to rank large pools of competitors
• More sophisticated algorithms incorporate information on the relative strength of the opponent (e.g. Elo rating system in Chess)
• Statistical approaches enable maximum likelihood estimation of underlying scores
Salomon – Disability weights - 18 18
Statistical modeling of paired comparisons data
• Conceptual foundation for statistical modeling of paired comparisons data comes from Thurstone (1927), through Luce (1959), McFadden (1974) and others
• Intuitively, a pair of health states that are similar in severity are likely to produce greater disagreement over which is healthier than a pair of states that are different in severity
• Consider a population in which: 70% of people think depression is worse than blindness 90% of people think blindness is worse than arthritis
• Reasonable to conclude that depression and blindness are nearer on some unobserved scale than blindness and arthritis
Salomon – Disability weights - 19
-4 -2 0 2 4 6 8
blindness
90%
Salomon – Disability weights - 20
-4 -2 0 2 4 6 8
depression blindness
90%70%
Salomon – Disability weights - 21
-4 -2 0 2 4 6 8
depression blindness
90%
Salomon – Disability weights - 22
-4 -2 0 2 4 6 8
depression blindnessarthritis
90%
Salomon – Disability weights - 23
-4 -2 0 2 4 6 8
depression blindnessarthritis
Salomon – Disability weights - 24 24
Statistical modeling of paired comparisons data
• Statistical model formalizes this intuition Each health state i has an unobserved actual health level Xi as perceived by an
individual rater The rater makes a choice in the paired comparison based on which of the
health states is regarded as ‘healthier’State 1 is chosen over State 2 if X1 > X2
• Thurstone model assumes that Xi is normally distributed, implying that the choice probabilities are also normal
• We can model response probabilities using probit regression Requires arbitrary identifying assumptions Assumes that each health state has the same variance around the disability
weight
Salomon – Disability weights - 25
From paired comparisons to disability weights
Analysis of paired comparisons
Analysis of “population health equivalence”
responses
Rescaling of paired comparison results
47
Salomon – Disability weights - 26
From paired comparisons to disability weights
Analysis of paired comparisons
Analysis of “population health equivalence”
responses
Rescaling of paired comparison results
• Pooled data from all household surveys, CATI and web survey
• Estimated probit model with separate variance for each survey
Salomon – Disability weights - 27
From paired comparisons to disability weights
Analysis of paired comparisons
Analysis of “population health equivalence”
responses
Rescaling of paired comparison results
• Some Web survey respondents randomly assigned to “population health equivalence” (PHE) questions, for subset of 30 sequelae
• PHE provides separate estimates for these sequelae that are anchored on (0,1) disability scale
49
Salomon – Disability weights - 28
From paired comparisons to disability weights
Analysis of paired comparisons
Analysis of “population health equivalence”
responses
Rescaling of paired comparison results
• Probit scale mapped to (0,1) disability scale by regressing probit coefficients on PHE anchors
• Non-constant variance accommodated by regression on logit-transformed PHE values, then back-transforming values into (0,1) space
50
Salomon – Disability weights - 29
• Background• Study design and methods• Key findings• Interpretations, limitations, conclusions
Disability weights measurement in GBD 2010
Salomon – Disability weights - 30
Results: paired comparison responses
• Probabilities of responses on paired comparisons (‘Who is healthier?’) summarized in heat maps.
Best Worst
Best
Worst
First health state in pair
Second health state in pair
Salomon – Disability weights - 31
• Probabilities of responses on paired comparisons (‘Who is healthier?’) summarized in heat maps.
High agreement in choices between very healthy vs. unhealthy outcomes (>90%)
… or vice versa (<10%)
Split responses for similar outcomes (~50%)
Results: paired comparison responses
Salomon – Disability weights - 32
Measurement error in paired comparisons
• In the household survey, we assessed test-retest reliability by randomly assigning 20% of respondents to have the same pair repeated as 1st and 15th comparison
• Consistency of responses ranged between 60% and 70%
• Consistency of 2 coin flips would be 50%
• Kappa values range from 20% to 40%
Percent of respondents with consistent retest responses
36
Salomon – Disability weights - 33
• Probabilities of responses on paired comparisons (‘Who is healthier?’) summarized in heat maps.
High agreement in choices between very healthy vs. unhealthy outcomes (>90%)
… or vice versa (<10%)
Split responses for similar outcomes (~50%)
Results: paired comparison responses
Salomon – Disability weights - 34
Results: paired comparisons across surveys
Salomon – Disability weights - 35 35
Results: probit values across surveys
Salomon – Disability weights - 36
Results: comparison of household and web surveys
• Web respondents comprise non-random, highly educated, self-selected sample
Educational attainment in HH & web samples
Salomon – Disability weights - 37
Results: comparison of household and web surveys
Tanzania (N=2,613)
Web (N=3,417)
• Web respondents comprise non-random, highly educated, self-selected sample
• But, response probabilities are virtually indistinguishable from those in household surveys
Salomon – Disability weights - 38
Results: comparison of household and web surveys
• Web respondents comprise non-random, highly educated, self-selected sample
• But, response probabilities are virtually indistinguishable from those in household surveys
• And estimated weights from probit regressions are very highly correlated
Salomon – Disability weights - 39
Results: new disability weights
Salomon – Disability weights - 40
Results: new disability weights
Salomon – Disability weights - 41
Results: new disability weights
Salomon – Disability weights - 42
Results: new disability weights
Salomon – Disability weights - 43
Results: new disability weights
Salomon – Disability weights - 44
Results: comparison to previous disability weights
Salomon – Disability weights - 45
Framing challenge: drug use disorders
…uses heroin daily and has difficulty controlling the habit. When the effects wear off, the person feels severe nausea, agitation, vomiting and fever. The person has a lot of difficulty in daily activities.
Salomon – Disability weights - 46
Framing challenge: drug use disorders
…uses heroin daily and has difficulty controlling the habit. When the effects wear off, the person feels severe nausea, agitation, vomiting and fever. The person has a lot of difficulty in daily activities.
Salomon – Disability weights - 47
Framing experiment: caffeine addiction
Alternative descriptions for caffeine addiction
…drinks several cups of coffee a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches
…takes medication several times a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches
…uses an addictive substance several times a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches
…uses an addictive drug several times a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches
…uses an illegal, addictive drug several times a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches
Salomon – Disability weights - 48
Framing experiment: caffeine addiction
Alternative descriptions for caffeine addictionDisability weights
…drinks several cups of coffee a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches
0.018
…takes medication several times a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches
0.064
…uses an addictive substance several times a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches
0.067
…uses an addictive drug several times a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches
0.136
…uses an illegal, addictive drug several times a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches
0.198
Salomon – Disability weights - 49
…takes daily medication and has difficulty going without it. When the effects wear off, the person feels severe nausea, agitation, vomiting and fever. The person has a lot of difficulty in daily activities.
…uses heroin daily and has difficulty controlling the habit. When the effects wear off, the person feels severe nausea, agitation, vomiting and fever. The person has a lot of difficulty in daily activities.
Framing challenge: drug use disorders
Salomon – Disability weights - 50
…takes daily medication and has difficulty going without it. When the effects wear off, the person feels severe nausea, agitation, vomiting and fever. The person has a lot of difficulty in daily activities.
…uses heroin daily and has difficulty controlling the habit. When the effects wear off, the person feels severe nausea, agitation, vomiting and fever. The person has a lot of difficulty in daily activities.
Framing challenge: drug use disorders
Salomon – Disability weights - 51
• Background• Study design and methods• Key findings• Interpretations, limitations, conclusions
Disability Weights Measurement Study
Salomon – Disability weights - 52
Interpretations
Largest empirical effort to date to measure weights for health outcomes across range of populations shows:• Feasible to collect this sort of information in virtually any population• Simple data collection tools can be combined with straightforward analytic
techniques to yield meaningful weights• Weights appear highly consistent across diverse cultural settings and
respondent characteristics
New disability weights provide critical resource for assessment of burden of disease, healthy life expectancy and intervention cost-effectiveness
Salomon – Disability weights - 53
Limitations
• Selection of countries to provide diversity, not as random sample of world’s population
• Web survey heavily skewed toward North America, Australia, Western Europe, with few respondents from Africa or Middle East
• Responses depend on validity of lay descriptions Are some key consequences omitted? Does inclusion of labels in selected instances (e.g. drug use disorders)
bias disability weights for these conditions?
• Future work should prioritize Exploring use of standardized health-state classifications More empirical data on distribution of sequelae across health-states
Salomon – Disability weights - 54
Lancet 2012; 380: 2129-2143.