Involving the Public in Priority Involving the Public in Priority SettingSetting

A case study in NHS Dumfries and Galloway

Verity Watson1

Andrew Carnon2

Mandy Ryan1 Derek Cox2

1 Health Economics Research Unit, University of Aberdeen2 Directorate of Public Health and Strategic Planning, NHS Dumfries and Galloway

StructureStructure

1. Introduction to priority setting

2. Overview of priority-setting process in NHS Dumfries & Galloway

3. How discrete choice experiments can be used in priority setting

4. Applying the discrete choice experiment in NHS Dumfries & Galloway

5. How successful was the priority-setting process?

How Do We Decide?How Do We Decide?

IntroductionIntroduction

• Options for health care are constrained by limited funding

• Choices imply priorities

• To make priorities explicit requires a priority- setting framework

• Options appraisal:

– identified projects are scored based on pre-defined criteria

– the relative importance of criteria are defined by weights

• Weights can lack transparency and accountability

Portsmouth Scorecard (Sandwell modification)Portsmouth Scorecard (Sandwell modification)

Factor weighting Very low

Low Mid High Top

Evidence < 3 10 20 30 40

Benefit number < 3 10 20 30 40

Cost < 3 10 20 30 40

Acceptability < 3 10 20 30 40

National requirement < 3 10 20 30 40

Addressing inequality <3 5 10 15 20

Wider society benefit <3 5 10 15 20

Only alternative <3 5 10 15 20

Local feeling 0 5 5 7 10

(Austin et al, 2007)

Priority-Setting Principles in Priority-Setting Principles in NHS Dumfries & GallowayNHS Dumfries & Galloway

• Focus on Delivering for Health

• Transparency and rigour of process

• Public involvement

• Acute services CHP/Long term conditions

• Annual event

• Learning process

Priority-Setting CriteriaPriority-Setting Criteria

Ten Criteria were chosen based on Delivering for Health

1. Location of care2. Public consultation while developing project3. Use of latest technology4. Service availability5. Patient involvement in own care6. Management of care7. Evidence of clinical effectiveness8. Health gain9. Risk avoidance10. Priority area

NHS Dumfries & Galloway NHS Dumfries & Galloway ProgrammesProgrammes

• Acute Services• Cancer• CHD/Stroke/Diabetes• Child Health• Corporate• Healthcare-Acquired

Infection

• Learning Disabilities• Local Health

Partnership/Primary Care

• Long Term Conditions• Mental Health• Older People• Public Health

Process in Process in NHS Dumfries & GallowayNHS Dumfries & Galloway • Public involvement event

• 12 programme leads to submit bids

• Panel to take overview

• Bids scored on weighted criteria

• Ranked list of bids produced (Health Intelligence Unit)

• Corporate Management Team decisions

Public involvement event:Public involvement event:

Discrete Choice ExperimentDiscrete Choice Experiment

Applying DCEs to Priority SettingApplying DCEs to Priority Setting

To investigate the relative importance of criteria to public in Dumfries and Galloway

Attributes are the 10 priority setting criteria

Describe the criteria by a number of levels

Define all possible combinations of attributes and levels

Create choice sets

Ask respondents to choose between different hypothetical health services for Dumfries and Galloway.

Attributes and LevelsAttributes and Levels1. Location of care

• at home• at GP• at Local Health Partnership• at D&G Royal Infirmary• outside D&G

2. Public involvement• no consultation• consultation at final stage• consultation at some but not all stages• consultation at all stages

Attributes and LevelsAttributes and Levels3. Use of Technology

• does not use latest technology• uses latest technology• uses cutting edge technology

4. Service availability• office hours only • office hours and outside office hours

5. Patient involvement in own care• decision by health professional• patient shares decision

Attributes and LevelsAttributes and Levels6. Management of care

• group of health professionals not working as a team• an individual health professional• group of health professionals working as team

7. Evidence of clinical effectiveness• no evidence• number of clinical studies• at least one RCT• at least three RCTs

8. Health Gain• small gain to a small number• large gain to a small number• large gain to a large number

Attributes and LevelsAttributes and Levels9. Risk avoidance

• low risk to lower than low risk• medium risk to low risk• medium risk to lower than low risk• high risk to medium risk• high risk to low risk• high risk to lower than low risk

10. Priority area• none• local priority• national priority• local and national priority

Experimental designExperimental design

207,360 possible combinations (51x61x43x33x22)

Used experimental design techniques and reduce to 64 profiles.

These were paired with a mirror image (foldover method)

Ensured orthogonality, minimum overlap and level balance

Respondents were presented with the choice of two health services and asked to choose their most preferred.

Example choiceExample choice A B

Location of care Dumfries and Galloway Royal

Infirmary Outside Dumfries and Galloway

Public consultation in decision making

No consultation Public and Patients were consulted at the final stage

Use of latest technology

Not using the latest technology Using the latest technology

Service Availability Office Hours only Office hours and out-with office

hours Patient’s involvement in own care decision

No opportunity Has the opportunity

Management of care Care is managed by a single

individual A group of professionals working as a

team Evidence of clinical effectiveness

Clinical Studies At least 1 RCT

Health Gain Large Gain to a Small Number

Small Gain to a Large Number Large gain to a large number

Risk Avoidance Reduction from a high risk to a

low risk Reduction from a medium risk to a

low risk Priority Area National priority Local and National Priority

Service A Service B Which service do you prefer?

Sample and SettingSample and Setting

A random sample of 100 members of public from Dumfries and Galloway

Invited to attend a half day meeting

Lunch and refreshments and travel expenses

68 respondents attended.

Sample and SettingSample and Setting

Age of Sample and Age of Sample and Dumfries & Galloway PopulationDumfries & Galloway Population

05

10152025303540

18-35 36 - 50 51 - 65 66+

Age

Per

cen

t

Sample D&G Population

Rurality of Sample and Rurality of Sample and Dumfries & Galloway PopulationDumfries & Galloway Population

05

10152025303540

Largeurban

Otherurban

Accessiblesmalltowns

Remotesmalltowns

Accessiblerural

Remoterural

Rurality

Per

cen

t

Sample D&G Population

Deprivation of Sample and Deprivation of Sample and Dumfries & Galloway PopulationDumfries & Galloway Population

0

10

20

30

40

50

1 2 3 4 5

Scottish Index of Multiple Deprivation Quintiles

Per

cen

t

Sample D&G Population

Advantaged Disadvantaged

AnalysisAnalysis

Responses are dichotomous, =1 for the option chosen and 0 for the option not chosen

Analysis using a logit regression:

Benefit = βiDxl

All criteria are qualitativethese are coded as dummy variablesOne level is omittedCoefficients are interpreted relative to the omitted level

Applying DCEs to Priority SettingApplying DCEs to Priority Setting

Criteria Coefficient/Weight

1. Location of care: At GP’s office 0.2523549*** At local health partnership 0.1182278 At Dumfries and Galloway Royal Infirmary 0.3395357*** Outside Dumfries and Galloway -0.5770746*** At home Comparator 2. Public involvement in decision making: Consultation at the FINAL stage 0.1165849* Consultation at SOME but NOT ALL stages 0.2253108*** Consultation at ALL stages 0.4921577*** No consultation Comparator 3. Use of technology: Uses the latest technology 0.7056297*** Uses cutting edge technology 0.7087127*** Does not use the latest technology Comparator 4. Service Availability: Office hours only -0.5708215*** Office hours and outside office hours Comparator

Applying DCEs to Priority SettingApplying DCEs to Priority SettingCriteria Coefficient/Weight

5. Patient involvement in own care: Decision made by health professional -0.474334*** Patient shares in decision Comparator 6. Management of Care: By a individual health professional 0.1576166*** By a group of health professionals working as a team 0.7264206*** By a group of health professionals NOT working as a team Comparator 7. Evidence of Clinical Effectiveness A number of clinical studies -0.0117717 At least 1 RCT 0.0667422 At least 3 RCT 0.4146794*** No evidence Comparator 8. Health Gain Small gain to a large number or large gain to a small 0.4328322*** Large gain to a large number 0.9661512*** Small gain to a small number Comparator 9. Risk Avoidance Medium risk reducing to a low risk 0.1112207 Medium risk reducing to lower than low risk 0.2056392** High risk reducing to a medium risk 0.0775806 High risk reducing to a low risk 0.1491032 High risk reducing to lower than low risk 0.1032418 Low risk reducing to lower than low risk Comparator 10. Priority Area Local priority 0.2644803*** National priority 0.0711403 Local and national priority 0.1950561*** No priority Comparator

How were the How were the Discrete Choice Experiment Discrete Choice Experiment

results used?results used?

Example: Community/Voluntary Sector Example: Community/Voluntary Sector Health & Wellbeing DatabaseHealth & Wellbeing DatabaseCriterion Level Weighted

score

Location Regional 0.34

Public involvement All stages 0.49

Use of technology Up to date 0.71

Availability All hours 0.57

User involvement in treatment/service

Has opportunity 0.47

Management of care Not applicable -

Evidence of effectiveness Not applicable -

Health gain Large gain to a large number 0.97

Priority area Both local and national 0.19

Weighted bid score =100 x Σ criterion weighted scores

no of criteria scored

53.43

0

10

20

30

40

50

60

1 3 7 10 13 16 19 22 25 28 31 34 37 40 44 46 48 52 54 58 61 64 67 70 73 76 79 82 85 88 91 94

Ranked Order

Wei

gh

ted

Bid

Sco

re

Acute Services CHP/LTC Other Bids

Weighted Bid ScoresWeighted Bid Scores

33.31

37.71

37.83

39.1

42.62

43.28

49.12

0 10 20 30 40 50

Average Bid Score

Corporate

Child Health

Long Term Conditions

CHP/Primary Care

Acute Services

CHD/Stroke/Diabetes

Public Health

Programme Bid ScoresProgramme Bid Scores

Top and Bottom BidsTop and Bottom Bids

Top Scoring Bids

1. DGH capital developments

2. Additional consultant in elderly medicine

3. Expand anaesthetic services

4. Rapid access chest pain service

5. Community/voluntary sector health and wellbeing database

6. Extra IT training staff

Bottom Scoring Bids

90. Set up neurology/genetic clinic

91. Increase paediatric clinics

92. Orthoptic vision screening

93. Expand school nursing

94. Appoint NHS Board business continuity manager

95. Provide extra secretarial input for paediatric consultants

Outputs of Priority-Setting ProcessOutputs of Priority-Setting Process

• Good spread of weighted bid scores

• Public involvement, transparency and accountability intrinsic to process

• Uncertain whether Delivering for Health principles fully reflected in weights (acute services and CHP/ long term conditions)

• Some anomalies (e.g. school nursing, IT training)

• Bid scores used as central aid to decision making, along with other factors (directives, risk, available finance)

How Successful was the Priority-How Successful was the Priority-Setting Process?Setting Process?

Limitations

• Sample size

• Complex concepts for public

• Cost of exercise

• Decision making still difficult

Benefits

• Planned approach to decision making

• Public response positive

• Transparency and defensibility

• Workable process

Organisational Priority-setting DecisionsOrganisational Priority-setting Decisions

Relative Benefits

• Location• Patient involvement• Health gain, etc

Difficulty

• Current state• Availability of skills required• Scale of change

Urgency

• Risk if not done• Quick wins possible• Current versus avoided future costs

Dependencies

• Funding available • Fit with existing projects• Basket of projects