1 EPI235: Epi Methods in HSR April 17, 2007 L5 Program Evaluation with Longitudinal Data 1:...
-
Upload
liliana-williamson -
Category
Documents
-
view
215 -
download
0
Transcript of 1 EPI235: Epi Methods in HSR April 17, 2007 L5 Program Evaluation with Longitudinal Data 1:...
1
EPI235: Epi Methods in HSR
April 17, 2007 L5
Program Evaluation with Longitudinal Data 1: Applications (Dr. Schneeweiss)
Various examples of applications in Health Services Research. Strengths and limitations of time series analysis .
Background reading: •Soumerai SB, Avorn J, Gortmaker S, Ross‑Degnan D. Payment restrictions for prescription drugs in Medicaid: Effects on therapy, cost, and equity. New Engl J Med 1987;317:550‑556.•Soumerai SB, Ross-Degnan D, Avorn J, McLaughlin TJ. Effects of Medicaid drug-payment limits on admission to hospitals and nursing homes. N Engl J Med 1991;325:1072-1077
2
3
4
5
What is Program Evaluation?
6
Program evaluation
What is program evaluation?
Design options:
Randomized trial
Cohort study
Quasi-experiment
What means quasi-experiment?
- implementation vs. evaluation perspective
- unintended randomization
Quasi-experiment = pre-post comparison?
= longitudinal design
= repeated measure design
Program Evaluation
7
8
9
10
11
12
Observational Design: Simple pre-post comparisons
Time
Intervention
Assumptions for causal inference:
1. The pre experience represents the post experience had there been no intervention
Outcome rate
13
Observational Design: Simple pre-post comparisons
Time
Intervention
Threat to causal inference:
Single pre-post estimates are averages of an underlying trend independent of the Intervention
Outcome rate
14
25% cost sharing in Quebec
0.0
0.7
1.4
2.1
# o
f e
ss
en
tia
l dru
gs
pe
r d
ay
ObservedPredicted (--- 95% CI)
11-93 8-94 8-95 8-96 8-97
Elderly (n=70801)
Policy Initiated
Tamblyn, JAMA 2000
15
Observational Design: no concurrent controls
Time
Intervention
Assumptions for causal inference:
1. Close temporal relation
2. Extrapolation of baseline trend is equal to the counterfactual experienceSchneeweiss, Health Policy 2000
16
Fixed ¢50 cost sharing in SC
Nelson, Med Care 1984
17
Observational Design: With concurrent controls
Assumptions for causal inference:
1. …
2. Control trend is equal to the counterfactual experience of intervention group
Time
Intervention
Intervention group
Control group
Schneeweiss, J Clin Epi 2002
18
3 prescription caps in NH:
80%
85%
90%
95%
100%
Baseline Cap After Cap
% o
uts
ide
nu
rsin
g h
om
e
New Jersey
New Hampshire
Soumerai NEJM 1991
19
Longitudinal quasi-experimental evaluation designs
Strength Weaknesses
Controls for between-person
confounding
Immediate causal inference
Intuitive graphs – easy to understand
Violation of basic assumption
Co-intervention or long-term trend
Slow or delayed effect
Regression to the mean
Reactiveness
Correlated data
Within-person confounding?
20
… or randomization?
Time
Intervention
Intervention group
Control group
R
Assumptions for causal inference:
1. Subjects comply with their assigned ‘treatment’ = policy
21
Complex statistical analyses are less convincing for decision makers
22
Formulary delisting in BC
23
More Examples:
24
More Examples:
Prescription drug useSurgical site infectionsContraindicated drug use
25
Medicaid prior authorization: Use
Smalley NEJM 1995
26
Medicaid prior authorization: $$
Smalley NEJM 1995
27
Open squares = utilization indicator (cesarean sections receiving perioperative antibiotic prophylaxis) Open circles = timing indicator (antibiotic within 1 hour of delivery)Solid diamonds = surgical site infection rate after cesarean section
Period 1 was a baseline period. Periods 2 and 3 were successive intervention periods.
Weinberg et al.: Arch Intern Med 2001
Reducing surgical site infections after C-section: Hospital A
28
Open squares = utilization indicator (cesarean sections receiving perioperative antibiotic prophylaxis) Open circles = timing indicator (antibiotic within 1 hour of delivery)Solid diamonds = surgical site infection rate after cesarean section
Period 1 was a baseline period. Periods 2 and 3 were successive intervention periods.
Weinberg et al.: Arch Intern Med 2001
Reducing surgical site infections after C-section: Hospital B
29
The Intervention:
'Dear Doctor' letters concerning interactions between cisapride and a series of drugs.
A letter in 1995 described a risk of prolonged QT intervals and serious ventricular arrhythmia in patients who received macrolide antibiotics and imidazole antifungals in conjunction with cisapride.
A June 1998 letter that expanded the list of contraindicated comedications had wider distribution than an earlier one, was accompanied by substantial Internet and media coverage, and was complemented by an effort to inform large pharmacy dispensing information organizations of the warnings against concurrent use of the named drugs.
FDA Risk Management: Effectiveness of a ‘Dear Doctor’ letter
30
Weatherby LB et al. PharmEpi Drug Safety 2001
31
32
Lecture on LPUs:
Weatherby LB et al. Clin Pharm Ther 2002
33
The Policy Model
34
Policy Model: reference pricing
35
The Clinical Model
36
More on the Clinical Model:
37
Clinical Model: reference pricing
38
Policy Model vs. Clinical Model