Seminar: “Real-life” Research for
“Real-life” Research
Monday 25th June
RIRL
09:30 Welcome and introduction to real-life research Prof David Price
10:00 Practical examples of pragmatic and observational studies including
new models of pragmatic trials Prof David Price
10:45 Utilising databases for observational research Julie von Ziegenweidt
(Database manager)
11:30 Tea and coffee break
11:45 Managing confounding in observational databases. Adjustment,
matching and other techniques
Annie Burden
(Senior Statistician)
12:30 Lunch
13:15 Cardiovascular disease risk and pharmacological smoking cessation
interventions: a retrospective, real-life evaluation
Dr Erika Sims
(Senior Researcher)
14:00 Creating your own database – examples from practice iHARP – a 6
country cross-sectional database
Stan Musgrave
(Senior Research Fellow UEA)
14:45 Tea and coffee break
15:00 Practical group work – design your own project
16:00 Publishing real-life research Prof David Price
Program
Introduction to Real Life Research
Professor David Price
• What is real-life research?
• What are the advantages of real-life research
• How do we conduct real-life studies at RIRL
Introduction to Real-life research
• What is real-life research?
• What are the advantages of real-life research
• How do we conduct real-life studies at RIRL
Introduction to Real-life research
Types of Trial
• Randomised Control Trial
o Patients with no confounding factors allocated at random to
receive different clinical interventions
o Measure efficacy
• Pragmatic Trial
o Randomised trial to compare two or more medical interventions
directly relevant to clinical care or health care delivery and
assess those interventions‟ effectiveness in real-world practice.
• Observational Study
o Case-control and cohort studies
o Retrospective analysis of outcomes to clinical interventions
carried out in normal patient care
What is effectiveness?
Population
Effectiveness
Efficacy x
“real-life” population of patients x
“real-life” management x
Understanding Clinical Effectiveness
What evidence do we really need?
Evidence
Theoretical
Theoretical model provide
rationale
Classical double-blind
double-dummy RCTs
Gold standard, large range of
outcomes.
But not “real-life” patients,
compliance and represent <10%
of patients
Pragmatic trials
More real-life Broader inclusion
criteria Allow normal factors to
occur Usually randomised
Simple outcomes But still consent &
rigorous
Observational Data
Real-life patients Not randomised
Routine data Normal decisions Difficult to ensure
group comparability
Matching of case controls,
adjustment
• What is real-life research?
• What are the advantages of real-life research
• How do we conduct real-life studies at RIRL
Introduction to Real-life research
Randomised Controlled Trials in Asthma as an Example
Travers et al. Thorax 2007
INCLUSION CRITERIA ECOLOGY OF CARE
Lung function 50 to 80% predicted Intense Patient Monitoring
Bronchodilator reversibility Regular inhaler review
Absence of co-morbidity Weekly assessments
Non smokers or ex-smokers <10 pack years
Daily diary of treatment
Symptomatic and regular use of medication
Good inhaler technique
High adherence to medication
Norwegian study of asthma patients to
identify who would be eligible for standard
clinical trials N
um
ber
of
Pati
en
ts All patients
Clinical asthma
FEV 50-85%
Reversibility 12%
No co-morbidity
Pack-year < 10
Reg use of ICS
Symptomatic asthma1.2%
Herland K, Akselsen JP, Skjonsberg OH, Bjermer L. How representative are
clinical study patients with asthma or COPD for a larger "real life" population of
patients with obstructive lung disease? Respir Med. 2005;99:11-9
External validity of RCTs
0
5
10
15
20
25
30
35
40
% indiv
iduals
pote
ntially
meeting
elig
ibili
ty c
rite
ria
References cited in GINA with level A or B evidence grade
Median 4%
Travers et al. Thorax 2007
n=179
Does it matter (even result in bias) if we exclude patients with:
• Poor inhaler technique
• Active rhinitis
• Smokers
• Lower adherence
• Lesser reversibility than 20%
AND
Design the study not like real-life?
Study only lasts 3 months?
Recommendations in guidelines often
muddled by “irrelevant” RCTs
Most inhaler device studies only include those with good inhaler technique!
Studies of real-life inhaler technique
References Inhaler device
1)Erickson et al. MDI
2)Larsen et al. MDI
3)Schmid et al. DPI (Turbuhaler®)
4)Shrestha et al. MDI
5)Thompson et al. MDI
6)van Beerendonk
et al.
MDI (19% ) or DPI
(81%)
82
58 64
79 76
89
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6
Pe
rce
nta
ge o
f p
atie
nts
Reference number
Percentage of patients with inadequate inhaler technique
Reviewed in Duerden M, Price D. Dis Manage Health Outcomes 2001; 9 (2)
Subanalysis of Asthma Patients with Concomitant
Allergic Rhinitis in COMPACT
*Montelukast 10 mg once daily + budesonide 400 µg twice daily; **Budesonide 800 µg twice daily Price DB et al. Allergy 2006
Ch
an
ge
fr
om
bas
elin
e
(L/m
in,
LS
mea
n
SE
M)
p=0.36
Weeks
All patients
Montelukast Provided Greater Improvements in Asthma Patients with
Concomitant Allergic Rhinitis
50
40
30
20
10
0
Montelukast (n=433)*
Budesonide (n=425)**
0 4 8 12 Weeks
Montelukast (n=216)*
Budesonide (n=184)**
0 4 8 12
Patients with rhinitis 50
40
30
20
10
0
p<0.03
Smoking in asthma and effect of inhaled
steroids
Chalmers GW et al. Thorax 2002;57:226–230
Kerstjens HA, et al. Eur Respir J 1993;6(6):868-76.
NS + ICS
NS + PL
S + PL
S + ICS
High dose ICS (FP 500) vs LTRA in light
smokers
• 20-30% of asthma
patients smoke
• Studies of low
dose ICS have
shown little
efficacy
• This study
compared FP 250
bid, montelukast
10mg and placebo
over 6 months
Percentage of Days without Nocturnal Awakenings
Month
0 1 2 3 4 5 6 7
% o
f d
ays w
ith
ou
t n
octu
rna
l a
wa
ke
nin
gs
50
55
60
65
70
75
80
Montelukast
Fluticasone
Placebo
Price D et al ERS 2011
Monteleukast
ICS
Placebo
RCT References
1) Pawels R et al. N Engl J Med 1997
2) Kips J et al. Am J Respir Crit Care 2000
3) Bateman E. Am J Respir Crit Care 2004
4) Papi A et al. Eur Respir J 2007
5) Busse W et al. J Allergy Clin Immunol 2008
Real-world References
1) Partridge Pulm Med 2006
2) De Marco et al. Int Arch Allergy Immunol 2005
3 and 4) Janson et al. Eur Respir J 2001 3=Italy 4=UK
5 and 6) Breekveldt-Postma et al. Pharmaco-epidemiol Drug Saf 2008 5=fixed combination 6=ICS
7) Stallberg et al. Resp Med 2003
8) Adams et al. J Allergy Clin Immunol 2002
9) Corrigan Prim Care Resp J 2011
Real-life adherence in observational studies vs. randomised trials
0
20
40
60
80
100
1 2 3 4 5
Pe
rce
nta
ge o
f P
atie
nts
Randomised Trials 75-125 75-125
89 >95
>80
45
34
17
49
14.1 8.3
34
21
40
0
20
40
60
80
100
1 2 3 4 5 6 7 8 9
Real-world Trials
Long-term adherence in a clinical trial with inhaled steroids in children
Jonasson G, Carlsen KH, Mowinckel P. Arch Dis Child. 2000;83:330-3
Budesonide bid Placebo bid
• What is real-life research
• What are the advantages of real-life research
• How do we conduct real-life studies at RIRL
Introduction to Real-life research
RIRL Study Design
Index prescription date
(IPD) either:
•Initiation of treatment
•Step-up treatment
Baseline period
• matching cohorts
• confounding factor definition Outcome period
• outcome comparison
• adjusted for baseline confounders
0 -12m +12m Treatment option 2
Treatment option 1
• General Practice Research Database (GPRD)
o De-identified longitudinal data
o 500 primary care practices in the UK
o 13 million total records, of which 3.6 million are active
o Broadly representative of UK patient population
• Optimum Patient Care Research Database (OPC-RD)
o Anonymous data extracted from practices during records-
or nurse based clinical reviews
o Electronic records from over 500 000 patients with
respiratory disease
Clinical Datasets
GPRD (now the CPRD) website resources
• UK GP Practice Recruitment
• Identification of eligible patients according to protocol requirements
• Collection of routine data required for CRF direct from electronic patient record
• Pre-filling of CRF using routine data
• Nurse delivered consultations
Optimum Patient Care Research Service
• Benefits to GP
o Minimises impact of participating in study on workload
o Increases likelihood to participate
• Benefits to Client
o Reduces time spent in practice identifying eligible
patients
o Reduces time spent completing CRFs
o Reduces transposition errors incurred during
completion of CRF
Optimum Patient Care Research Service
Examples of Study Design 1
Patients enrolled & consented
into Study by OPC Research
Nurse
GPs Review Eligible Patient List
Invitations to participate sent to
eligible patients by remote
mailing
Extraction of Routine
Data from practice
management system
Identification of patients by
Read Code
Identify eligible patients
Report to GP
Anonymise
Anonymisation /
de-anonymisation
programme
De-anonymise
Export Routine data
into study template
Responses returned to OPC‟s
offices. Appointments made by OPC
Research Nurse to attend study day
at GP practice Routine Data
used to pre-fill
OPC Screening
Check List to aid
completion
alongside CRF OPC Screening Checklist (SC)
completed (including confirmation of
pre-filled data). Data from SC
transcribed into CRF.
• International multi-centre observational study
• Eligible patients identified using data from electronic patient record
• Paper CRF required to be completed at all sites for all data
• Screening Checklist enables completion of 58 of 70 (>80%) fields on CRF with data from electronic patient record
• OPC staff undertake study visits
Examples of Study Design 1
Data prefilled from electronic patient record
Examples of Study Design 2
International multi-centre observational study Eligible patients identified using data from electronic patient record Current therapy data directly exported as ASCII file to CRO for analysis Paper CRF completed for required to be completed at all sites Screening Checklist enables completion of 58 of 70 (>80%) fields on CRF with data from electronic patient record
Extraction of Routine
Data from practice
management system
Identification of patients by
Read Code
Identify eligible patients
Report to GP
GPs undertake patient visits
Anonymise
Anonymisation /
de-anonymisation
programme
De-anonymise
Export Routine data
into study template
OPC Electronic Screening Checklist
(ESC) prefilled; to be confirmed by patient during visit
GPs Review Eligible Patient List
Invitations to participate sent to
eligible patients by remote mailing
Routine Data
used to pre-fill
OPC Electronic
Screening
Check List
(analogous to
eCRF)
Export data for
recruited patients
into ASCII file for
direct entry into
study database
Examples of Study Design 2
Data extracted from routine
patient record into OPC-RD
Prefill Electronic
SCL & Study
Database
Additional data directly into
electronic SCL & Study Database
Research staff do not need access to patient records minimising transposition
errors
Rapid data collection
• 12 European Countries, 163 sites in total
• 8150 patients in total (approximately 50 per site)
The role of OPC
• 10 sites needed in the UK
• Sponsor identified 4 sites
• OPC identified 6 sites
A prospective observational study in
asthma and OPC
• OPC has established a good relationship with a
number of practices across the UK
• Many practices are already research pro-active
and recognise the value of “Real-Life” Research
• Practices often involved in multiple studies with
OPC
• Practice involvement is rewarded by OPC‟s Clinical
Review Service and/ or remuneration
The Advantages of OPC
Practical Examples of pragmatic
and observational studies
Professor David Price
• Recent papers from RIRL
• The ELEVATE trial
• Current Research
Practical Examples of Studies
• Recent papers from RIRL
• The ELEVATE trial
• Current Research
Practical Examples of Studies
Baseline: 1 year
Outcome: 1 year
Index prescription
date (IPD)
Matched 1:1 on baseline
demographics and
disease severity
Fluticasone
5 to 60 years Receiving a first prescription of ICS
J ALLERGY CLIN IMMUNOL SEPTEMBER 2010
0
5
10
15
20
25
30
35
40
45
50
Pe
rce
nta
ge o
f P
atie
nts
Dose of ICS at IPD
Fluticasone
HFA-BDP(Qvar)
HFA-BDP (Qvar)
Odds Ratios/Rate Ratios
0.7 0.8 1 1.2 1.5 2
Asthma control + SABA adjusted
Asthma control + SABA
Asthma control adjusted
Asthma control
Initiation Population: Odds Ratios and
Rate Ratios
*Adjusted for year of index date, acetaminophen, asthma consultations, rhinitis diagnosis, recorded
asthma diagnosis, and cardiac disease diagnosis
SABA: Short acting β-agonist
Price et al. J Allergy Clin Immunol 2010;126:511-8
FP set to 1.00
Greater asthma control for all definitions of
“control” with HFA-BDP (Qvar) compared to FP
95% CI
n = 1319 for both cohorts
Baseline: 1 year
Outcome: 1 year
Index prescription
date (IPD)
Matched 1:1
HFA-BDP (Qvar)
CFC-BDP Patients 5-60 years
Study Period: January 1997 to June 2007
0.7 0.8 1 1.2 1.3
Change in therapy
Exacerbations
Asthma control no therapy change
Asthma control + SABA
Proxy asthma control
Adjusted odds ratios and rate ratios for EF
HFA-BDP compared to CFC-BDP (95% CI)
CFC–BDP set to 1.00
INITIATION POPULATION
0.3 0.5 1 2 3
CFC–BDP set to 1.00
STEP-UP POPULATION
EF HFA-BDP n=2882
CFC-BDP n=8646
EF HFA-BDP n=258
CFC-BDP n=516
Fine-particle BDP gives greater asthma control than the larger particle
CFC-BDP – greater penetration into airways
Baseline: 1 year
Outcome: 1 year
Index prescription
date (IPD) Pressurized metered-dose inhaler (pMDI)
For patient characterisation
For outcome evaluation
Matched
Breath-actuated MDI (BAI)
Dry powder inhaler (DPI)
David Price John Haughney Erika Sims Muzammil Ali Julie von Ziegenweidt Elizabeth V Hillyer Amanda J Lee Alison Chisholm Neil Barnes
BAI and DPI (pMDI set to 1.00)
INITIATION COHORT
STEP-UP COHORT
pMDI (n = 39,746)
BAI (n = 9809)
DPI (n = 6792)
pMDI (n = 6245)
BAI (n = 1388)
DPI (n = 1536)
• Recent papers from RIRL
• The ELEVATE trial
• Current Research
Practical Examples of Studies
• Recruiting patients for study
o Asking GPs to recruit patients required
• Pre-defined database will make it easier in future to
carry out large-scale database studies
• Working with practices allows us to identify patients
for research in current studies
Challenges of the ELEVATE Trial
0 2 10 26 52 78 104
Week Week:
Tailored treatment as indicated by
guidelines
LTRA Ideally no ICS use
ICS Ideally no LTRA use
Baseline
V1 V2 V3 V4 V5 V6 V7
SABA
Randomisation
Study included to show that effectiveness results maybe difference to efficacy results; this study does not advocate the use of LTRA outside of the licence guidelines
CONSORT Diagram
* = Mean (SD), NR = not reported, NA = not applicable, %PPEF= Percent predicted PEF, %PFEV1= Percent predicted FEV1
1- Bateman et al. Can Guideline-defined Asthma Control Be Achieved?: The Gaining Optimal Asthma ControL Study Am. J. Respir. Crit. Care Med. 2004; 170: 836-844. 170. p.836, (2004)
ELEVATE
Step 2; N=306
GOAL
Strata 1; N=1098
Sex (% Female) 51% 57%
Age * 45.8 (16.4) 36.3 (15.6)
Quality of Life (Juniper AQLQ 1, worst, to 7)
4. 74 (1.04) 4.4 (1.00)
Lung Function * 86
%PPEF 77
%PFEV1
Percent reversibility * 8.9% (9.86) 22% (12.2)
Smokers – current 21.9% 9.5%
Drop out rate 4.0% 15.4%
Demographics and drop out rates Comparison to other studies (ELEVATE step 2, GOAL1)
First-Line Controller Therapy Trial Months
First-Line Controller Therapy
Trial
LTRA (n=148) ICS(n=158) Rate Ratio 95% CI LTRA vs. ICS
Mean no. of exacerbations 0.44±0.94 0.35±0.95 1.27 (0.83-1.92)
Secondary Outcome Measures
0
10
20
30
40
50
60
70
80
90
0 1 >1
Pe
rce
nta
ge o
f P
atie
nts
Number of Exacerbations
LTRA(n=148)
ICS(n=158)
LTRA ICS
Rate
%
0
20
40
60
80
100 p=0.11
n=108 n=101
65
92
15
41
62
21
Adherence to Therapy
Sub-group analysis - smokers
-2 0 8 26 52 78 104
Week Week:
Tailored treatment as indicated
by guidelines
LTRA + ICS No LABA use
LABA + ICS No LTRA use
Baseline
V1 V2 V3 V4 V5 V6 V7
ICS +SABA
Randomisation
PE
F (
% o
f p
red
icte
d)
Step-up Therapy Trial
Conclusions
• Whilst efficacy based studies suggest ICS are more
efficacious that LTRAs
• Real-life randomised trial i.e. real patients and real
practice – equivalent clinical outcomes
• Neither option universally effective
• Recent papers from RIRL
• The ELEVATE trial
• Current Research
Practical Examples of Studies
• A real-life trial assessing the safety and effectiveness of Relovair (fluticasone/vilanterol) in COPD and Asthma compared with usual care
• 4000 COPD patients and 5000 asthma patients
• 1 year study duration
Endpoints
• Collected partly from integrated primary and secondary care datasets
• Patient symptoms over time
• Exacerbation of symptoms
• Contact with healthcare professionals for respiratory reasons
• Other medication required to control symptoms
• Exhaled nitric oxide can be used as a marker for
inflammation
• Devices that measure this inflammation can be used
to help with asthma monitoring
• Using proxy measures of outcome control to explore
whether eNO monitoring can help to improve
asthma management
Using eNO to monitor patient asthma
treatment
The eNO study: Full study design
Patients reviewed with FeNO monitoring
Index prescription date
(IPD): initiation of FeNO
monitoring
Baseline period: 1 year Outcome period:
1) 6 months for monitoring to take effect
2) 12 months to measure outcomes
Patients reviewed with no FeNO monitoring
Research Question: Does using eNO to monitor patient asthma improve their asthma outcmes.
Preliminary results: Changes
before and after eNO monitoring
Patients reviewed with FeNO monitoring
Index prescription date
(IPD): initiation of FeNO
monitoring
Baseline period: 1 year Outcome period: 6 months
Patients must have •> 2 years history of asthma (diagnostic code and/or prescription for asthma therapy at least 2 years prior to IPD) •Evidence of current asthma treatment (≥2 asthma prescriptions during baseline + outcome year) •Have at least one year of up-to-standard (UTS) baseline data and at least 6 months of UTS outcome data (following the IPD).
Reasons for using eNO monitoring
Poor ICS adherence (MPR < 80%)
58.0%
Diagnosed not asthma no ICS in outcome period
21.8%
Evaluation of high risk (>1 exacerbation in baseline)
18.3%
General Monitoring
36.4%
Data Source: Optimum Patient Care Research
Database
• Anonymous data extracted from practices for
chronic respiratory service review.
• Two types of data collected
o Routine Clinical Data
o Patient Recorded Data
• OPCRD has been approved by Trent Multi Centre
Research Ethics Committee for clinical research use
Preliminary Results: Patient Demographics
eNO Treatment Group
BMI (kg/m2) 25.7 (23, 30.4)
Female n(%) 196 (52.8)
Smoking Status
Non 260 (70.1)
Current 19 (5.1)
Ex 86 (23.2)
Missing data
6 (1.6)
Frequency distribution for age Mean: 42.26 S.D: 20.77
Preliminary results: Co-morbidities and
Therapies
Co-morbidities* in eNO monitoring group n(%)
Rhinitis 174 (46.9)
GERD 20 (5.4)
Cardiac Disease 6 (1.6)
Medication therapies+ in eNO monitoring group n(%)
Beta Blockers 4 (1.1)
NSAIDS 196 (52.8)
Paracetamol 57 (15.4)
*Defined as Read code for co-morbidity at any time
+Defined as prescriptions received during the 1 year prior to IPD or at IPD
Sub Analysis: Patients increasing their average daily
ICS dose from baseline to outcome by ≥50%
Patients prescribed in eNO Monitoring Group n(%) Mean (SD)
Before eNO monitoring
Following eNO monitoring
Based on 6-month outcome period (Counts scaled to 365 days)
Total Acute Oral steroids 0.26 (0.75) 0.32 (1.13)
LRTI Consultations resulting in script for antibiotics
0.08 (0.29) 0.05 (0.38)
Based on 1 year outcome period
Total Acute Oral steroids 0.21 (0.64) 0.23 (0.63)
LRTI Consultations resulting in script for antibiotics
0.12 (0.35) 0.05 (0.26)
Change in Asthma Severity by BTS Step
0
5
10
15
20
25
30
35
0 1 2 3 4
Pe
rce
nta
ge o
f P
atie
nts
BTS Stage
Baseline
Outcome
Preliminary Results: Change in Control Status
Baseline Patients n(%) Outcome Patients n(%)
Uncontrolled 83 (22.4) 47 (12.7)
Controlled 288 (77.6) 324 (87.3)
Total 371 (100) 371 (100)
77.6 87.3
0
20
40
60
80
100
Baseline Outcome
Patients with fully controlled asthma before and after eNO monitoring
Utilising databases for observational
research
Julie von Ziegenweidt
Database Manager
RIRL
• Julie von Ziegenweidt o Senior Analyst/Data Manager
• Jeremy Brockman o Data Analyst
• Daniel West o Junior Analyst
RiRL Data Analysis Team
• Different types of studies
• Study Design
• Choosing the right data source
• Data Analysis
o Understanding the data
o Data Management
o Extracting the study data
o Quality Checks
o Matching and randomisation
o Study Archive
• Summary
Data Analysis
• Different types of studies
• Study Design
• Choosing the right data source
• Data Analysis
o Understanding the data
o Data Management
o Extracting the study data
o Quality Checks
o Matching and randomisation
o Study Archive
• Summary
Data Analysis
• Observational studies Comparing 2 study periods to determine if the intervention
make a difference?
• ie „to compare the effectiveness of QVAR v FP in Asthma‟
This may be Retrospective or Prospective
• Prevalence studies
o The total number of cases of a disease in a given population at a specific time.
– ie identify proportion of patients with asthma on long term oral steroids in the UK
Different types of studies
• Different types of studies
• Study Design
• Choosing the right data source
• Data Analysis
o Understanding the data
o Data Management
o Extracting the study data
o Quality Checks
o Matching and randomisation
o Study Archive
• Summary
Data Analysis
Study Design
Index date either:
•Initiation of treatment
•Step-up treatment
Baseline period
• matching cohorts
• confounding factor definition Outcome period
• outcome comparison
• adjusted for baseline confounders
0 -12m +12m Treatment option 2
Treatment option 1
• Different types of studies
• Study Design
• Choosing the right data source
• Data Analysis
o Understanding the data
o Data Management
o Extracting the study data
o Quality Checks
o Matching and randomisation
o Study Archive
• Summary
Data Analysis
• Are they representative?
• Do they provide the right data to answer research question?
• Do they provide the data in the way the researcher can work with it?
Retrospective Data sources
• Bespoke database to answer a specific question
• Example: The iHARP database
Prospective data sources
Patient Questionnaire
Consultation with respiratory clinician
and patient
Data from all three sources present in
the database
Review of routine practise data
• Different types of studies
• Study Design
• Choosing the right data source
• Data Analysis
o Understanding the data
o Data Management
o Extracting the study data
o Quality Checks
o Matching and randomisation
o Study Archive
• Summary
Data Analysis
Preparation for analysis
(Study protocol and design)
Study population
(Consort Diagram, Inclusion /Exclusion
criteria)
Matching
(Matching criteria)
Data Analysis
o Coding Practices
– How has the data been sourced? (clinical v
questionnaire)
– Multiple coding environments (primary care v
insurance)
o Coding dictionaries
– ICD9/10, UK Read codes v2/3, or ATC or NCD
o Understanding definition of each column of data
– Patient ID, EventDate etc
Understanding the data
o Code lists – Asthma – read code, read terms.
– ICS inhalers – code, name, strength, device (MDI,BAI,DPI), device types (easi-breathe), doses in pack, drug, form etc.
– Interpretation tables – 2 puffs twice a day = 4 doses a day
o Data validation: – Duplication of entries
• multiple entries on same day for same drug but different prescribing instructions.
o Interpretation of possible mis-diagnosis – sensitivity searches
o Cleaning of data: – Quality of data –
• registration date prior to year of birth
• recorded height for child is over 2m
Data Management
Consort Diagram
Patient on any ICS
Increase ICS dose
• The Consort Diagram is a simple flow diagram to show the progression of patient identification
• Confirmation that patient identification is in line with study design
• For Example:
Change to combination therapy via fixed
combination inhaler
Addition of a LABA to existing ICS
• One row per unique patient
• Perform descriptive stats to ensure correctness of data (ie, means,
medians, SD, perc)
• Data dictionary o descriptions,
o data types (i.e. string or integers),
o indicators of missing data,
o definition of proxies
Data Quality Verification
• In real world studies case-control matching can be necessary to ensure
similarity of patients included in an analysis to minimise outcome
confounding
Matching
Baseline characteristics descriptively compared to identify variances between
cohort severity
Cases are potentially matched to multiple
controls
Random case-control matches
MATCHING
RANDOMISATION
Example of Matching Criteria
MATCHING CRITERIA AND CATEGORIES
• Age: < 12 (exact match) ≥12 (+/- 5 yrs)
• Average Daily SABA Dose 0 – 200, 201 – 400, 401 +
• Sex: male / female
• Last ICS dose prior to IPD 1 –100, 101 – 200, 201 – 400, 401 – 600, 601 – 800, 801 +
• Baseline primary outcome status
• Number of asthma consultations
0,1,2+
Example: The ICS vs Combination Therapy
Study
Preparation for study
Study population
Matching
Outcomes: proxy for asthma control, exacerbations Covariates: Smoking etc. Inclusion/Exclusion criteria: Valid asthma, valid data
RESULT: Valid population of anonymous matched patients
• Study Archiving
o Raw Data
o Extracted Datasets
o Code Lists
o Drug Dictionaries
o Data Dictionary
o Program Scripts
Lastly...
• Exploration and familiarisation of raw patient data essential
• Data validation (check for inconsistencies etc.)
• Set up of Diagnostic and drug lookup tables
• Study design specifies all data required at the outset o ie ensuring outcomes are appropriate for study hypothesis.
• Consort diagram communicates accurately the methodology behind study design.
• Continuous Data validation checks!!
Summary
Managing Confounding in
Observational Databases
Annie Burden
Senior Statistician
RIRL
• Annie Burden
o Senior Statistician
• Vicky Thomas
o Lead Statistician
o SAS
• Francesca Barion
o Lead Health Economist
o STATA
• Muzammil Ali
o Statistician
o SPSS
RiRL Statistics Team Statistics & Health Economics
• RCTs
o Subjects are Randomised
o Variation in baseline characteristics should be random
across treatment groups
• Observational Studies
o We get what we get!
o Variation in the baseline characteristics between
treatment groups may be:
– Systematic
– Random
Introduction
“Any claim coming from an
observational study is most
likely to be wrong”.
S. Stanley Young & Alan Karr
Significance Magazine September 2011 Volume 8 Issue 3
The mis-conception
• Multiple Testing
o Keep asking questions of the data & something will
eventually come up positive;
o E.g. “Women eating breakfast cereals leads to more
boy babies”.
• Bias
o Hidden Confounders
• Multiple Models
o Limitless possible combinations of confounders.
The mis-conception (continued)
• Multiple Testing
o Limited number of Primary Outcomes;
o Outcomes set a priori;
o Interpretation is important...
– Clinician Input (Female diet cannot influence baby‟s gender)
• Bias
o Try to include all potential confounders.
• Multiple Models
o Very careful selection of:
– Potential confounders; and
– Covariates to include in the final model.
The truth
• “A confounding variable is an extraneous variable in a statistical model that correlates (positively or negatively) with both the dependent variable and the independent variable.”
• “... Confounding is a particular challenge.” Wikipedia
• “Confounding – where the estimated association is NOT the same as the true causal effect...”
Thomas Lumley 2005
• “Throwing things into disorder; mixing up; confusing.” The Concise Oxford Dictionary
Confounding - Definitions
Confounding
• Baseline / Characterisation Variables related to the
Outcome (Dependent) Variable
o Predictive Variables;
• Baseline / Characterisation Variables related to the
Treatment (Independent Variable)
o Baseline Differences;
• Check for relationships between the potential
confounders to avoid double-accounting
o Linear relationships through correlation coefficients;
o Non-linear relationships via modelling / plots.
Potential Confounders
Treatment Group P value
Control Treatment Total
Number of
Exacerbations (ATS
Definition)
N (% non-missing) 194 (100.0) 388 (100.0) 582 (100.0)
0.949 Mean (SD) 0.32 (0.60) 0.32 (0.60) 0.32 (0.60)
Median (IQR) 0 (0, 1) 0 (0, 1) 0 (0, 1)
Total acute Oral
steroids
N (% non-missing) 194 (100.0) 388 (100.0) 582 (100.0)
0.986 Mean (SD) 0.12 (0.35) 0.12 (0.33) 0.12 (0.33)
Median (IQR) 0 (0, 0) 0 (0, 0) 0 (0, 0)
LRTI Consultations
resulting in
prescription for
Antibiotics
N (% non-missing) 194 (100.0) 388 (100.0) 582 (100.0)
0.781 Mean (SD) 0.24 (0.54) 0.22 (0.54) 0.23 (0.54)
Median (IQR) 0 (0, 0) 0 (0, 0) 0 (0, 0)
Baseline Differences
Gender Height Weight BMI (categorised)
Gender
Height 0.54
Weight 0.34 0.55
BMI (categorised) 0.78
Linear Relationships
Spearman’s Correlation Coefficients
Non-linear Relationships
Example Output from SPSS Statistics19 www.spss.com/uk/software/statistics/
2 main options: • Perform comparisons only between observations
that have the same value of the confounder:
o Stratify data by confounder(s)
o Adjust for a confounder(s) in regression (an
approximation to stratification that requires less data).
• Perform comparisons only between groups that
have the same distribution of the confounder:
o Match on the confounder(s)
Removing Confounding
Score Statistics For Type 3 GEE Analysis
Source DF Chi-Square Pr > ChiSq
Treatment 1 9.8 0.0017
Age 1 3.84 0.0501
Acute_Oral_Steroids 2 20.01 <.0001
Non_asthma_Consults 3 7.83 0.0496
ZYear_of_IPD 1 1.69 0.1941
Asthma_A&E 1 24.34 <.0001
Rhinitis_Dx 1 8.93 0.0028
Adjustment
Example Output from SAS v9.3 www.SAS.com/offices/europe/uk/technologies/analytics/statistics/stat/ondex.html
Aim for Parsimony!
• To minimise baseline differences between treatment groups
• Match on:
o Demographic variables (Age, Gender);
o Study-appropriate measures of baseline disease severity, for example:
– Baseline Exacerbations
– Baseline Consultations
– Controller Medication
– Reliever Medication.
• Matching Ratio (e.g.1:1, 1:2, 1:3) to maximise power of statistical tests
Matching
• Minimal adjustment for residual confounders
o Parsimony is Good!
• “Conditional” Models
o Conditional Logistic Regression
o Conditional Poisson Regression
o Conditional Ordinal Logistic Regression
• Matching aims to minimise differences between treatment groups at baseline BUT....
• Need to consider whether matched cohorts are then representative of the full populations.
Matching
• Another option but not as readily understood by reviewers etc.
• Using covariates predictive of outcome, calculate Propensity Score (using Logistic Regression):
= P(Treatment | Covariates)
• Match on Propensity Scores
• Advantages
o Easily Includes Multiple Covariates
• Disadvantages
o Does not necessarily match clinically “similar” patients
Propensity Scoring
Asthma Control Treatment Group
Total Control Active
Controlled n (%) 267 (70.4) 540 (71.2) 807 (71.0)
Uncontrolled n (%) 112 (29.6) 218 (28.8) 330 (29.0)
Total n (%) 379 (100) 758 (100) 1137 (100)
Odds Ratio adjusted for baseline
confounders* (95% CI) 1.00
1.24 (0.92, 1.66)
Example Results ASTHMA CONTROL Defined as: Controlled: the absence of the following during the one-year outcome period: •Asthma-related:
oHospital attendance or admission oA&E attendance, OR oOut of hours consultations, OR oOut-patient department attendance
•GP consultations for lower respiratory tract infection •Prescriptions for acute courses of oral steroids. Uncontrolled: all others.
*Adjusted for: Number of exacerbations (clinical definition) (categorised), Number of non-asthma-related consultations (categorised) and GERD diagnosis &/or therapy (YES/NO).
Logistic Regression Model
Number of Exacerbations (ATS Definition)
Treatment Group Total
Control Active
None n (%) 306 (80.7) 613 (80.9) 919 (80.8)
1 n (%) 57 (15.0) 99 (13.1) 156 (13.7)
2+ n (%) 16 (4.2) 46 (6.1) 62 (5.5)
Total (n) 379 (100) 758 (100) 1137 (100)
Rate Ratio adjusted for baseline
confounders* (95% CI) 1.00
1.04 (0.76, 1.44)
Example Results
EXACERBATIONS (ATS DEFINITION) Defined as the occurrence of: •Asthma-related:
oHospital attendance / admissions OR oA&E attendance
•Use of acute oral steroids.
*Adjusted for: GERD diagnosis &/or therapy (YES/NO), Number of Primary Care Consultations (categorised) and Number of prescriptions for SABA (categorised).
Poisson Regression Model
Adherence to ICS Therapy
Treatment Group
Total Control Active
< 50%n (%) 186 (49.1) 162 (21.4) 348 (30.6)
50-69% n (%) 53 (14.0) 190 (25.1) 243 (21.4)
70-99% n (%) 79 (20.8) 132 (17.4) 211 (18.6)
≥ 100% n (%) 61 (16.1) 274 (36.1) 335 (29.5)
Total n (%) 379 (100) 758 (100) 1137 (100)
Odds Ratio adjusted for baseline
confounders* (95% CI) 1.00
2.79 (2.21, 3.52)
Example Results
ADHERENCE TO ICS THERAPY
*Adjusted for: Age, Asthma diagnosis (YES/NO), Number of Primary Care consultations (categorised), Number of prescriptions for SABA (categorised), spacer device use (YES/NO) and Year of IPD.
Ordinal Regression Model
• Treatment Effect varies with stratum of the 3rd variable
o E.g. Active Drug has different effectiveness relative to
Control depending on smoking status / gender / year
• Effect Modification & Confounding can exist separately
or together:
o Effect modification without confounding
– Adjust & Look at interactions
o Confounding without effect modification
– Adjust / match
o Both confounding and effect modification
– Adjust / match AND
– Look at interactions
Effect Modification
Effect Modification (cont.)
Effect Modification (cont.)
Effect Modification (cont.)
Effect Modification (cont.)
• Exploration of and familiarisation with Data very
important
o Data validation (check for inconsistencies etc.)
o Patient Characterisation & Baseline Differences
between Treatment Groups
o Variables Predictive of Outcome
o Relationships between Variables
• Interpretation of the results / Clinical Input
o Ensure results are sensible
o Ensure adjustments are sensible
Summary
Cardiovascular disease risk and
pharmalogical smoking cessation
interventions: a retrospective, real-
life evaluation Dr. Erika Sims
Senior Researcher
RiRL Research Team
-
-
-
Clinical Research Team RiRL Academic
Professor David Price RiRL Chief Investigator
Professor of Primary Care Respiratory Medicine University of Aberdeen
Dr Erika Sims RiRL Senior Researcher
Honorary Research Fellow University of East Anglia
Dr Stan Musgrave RiRL Research and Medical Data Associate
Senior Research Fellow University of East Anglia
Dr Yolande Cordeaux Medical Researcher Research Fellow; University of Cambridge
• Tobacco dependence is a chronic, relapsing
condition
Smoking cessation interventions
• Tobacco dependence is a chronic, relapsing
condition
• In EU in 2000
o 655,000 deaths attributed to tobacco use
o Societal & healthcare costs €97.7–130.3 billion
Smoking cessation interventions
• Tobacco dependence is a chronic, relapsing
condition
• In EU in 2000
o 655,000 deaths attributed to tobacco use
o Societal & healthcare costs €97.7–130.3 billion
• Two approaches to smoking cessation:
o Smoking Cessation Advice
o Pharmacological support
Smoking cessation interventions
• Tobacco dependence is a chronic, relapsing
condition
• In EU in 2000
o 655,000 deaths attributed to tobacco use
o Societal & healthcare costs €97.7–130.3 billion
• Two approaches to smoking cessation:
o Smoking Cessation Advice
o Pharmacological support
– Nicotine replacement therapy - since 1980‟s
– Bupropion (2000) & Varenicline (2006)
Smoking cessation interventions
Nicotine Replacement Therapy
• Nicotine Replacement Therapy
o Substitutes for nicotine
o nasal sprays, inhalers, gum and tablets, transdermal
patches
Safety of NRT
• Gillies et al 2012; Intensive Care Medicine, in press
o Retrospective case review
o No evidence of „harm associated with NRT, with the ICU model actually trending towards benefit‟.
o n = 423
• Ruiz et al 2012; Nicotine & Tobacco Research, in press
o Prospective study of COPD patients
o „All types of treatments were safe.‟
o n = 472
• Zapawa et al 2011; Addictive Behaviours vol 36: 327
o Systematic Review
o „Persistent (i.e., long-term) use of NRT does not appear harmful‟
Preliminary analyses: NRT vs SC
Adjusted for history of CVD, age and sex:
o 1.44 (95% CI: 1.17–1.79) for CVD
Preliminary analyses: NRT vs SC
• Adjusted for history of CVD, age and sex:
o 1.44 (95% CI: 1.17–1.79) for CVD
• No difference in cardiovascular risk profile :
o body mass index (BMI),
o hyperlipidaemia,
o systolic blood pressure,
o hypertension and
o diabetes.
Preliminary analyses: NRT vs SC
• Adjusted for history of CVD, age and sex:
o 1.44 (95% CI: 1.17–1.79) for CVD
• No difference in cardiovascular risk profile : o body mass index (BMI),
o hyperlipidaemia,
o systolic blood pressure,
o hypertension and
o diabetes.
• But limited analysis.
o population not large enough to draw conclusion on mortality.
o Other confounders?
• Patients exposed to NRT and other smoking
cessation pharmacotherapies are at a higher risk of
CVD compared with patients undertaking quit
attempts unaided by pharmacotherapies.
Hypothesis
• Patients exposed to NRT and other smoking
cessation pharmacotherapies are at a higher risk of
CVD compared with patients undertaking quit
attempts unaided by pharmacotherapies.
• 4 way analysis:
–NRT Smoking Cessation
–Varenicline Advice
–Bupropion
Hypothesis
VS
• Patients exposed to NRT and other smoking
cessation pharmacotherapies are at a higher risk of
CVD compared with patients undertaking quit
attempts unaided by pharmacotherapies.
• 4 way analysis:
–NRT Smoking Cessation
–Varenicline Advice
–Bupropion
Hypothesis
VS
Study Design
Index prescription date
Initiation of NRT
Baseline period
•no CS pharmacological aids Outcome period
• outcome comparison
• adjusted for baseline confounders
0 -12m +12m SC advice
NRT
• Retrospective, matched cohort study using GPRD
2006 - 2009
Study Cohorts
• Exposure Cohort – NRT o No recorded exposure to CS pharmacological aids in the prior year,
o First recorded smoking cessation intervention was NRT (using any of, or a combination of products) at the index date.
Study Cohorts
• Exposure Cohort – NRT o No recorded exposure to CS pharmacological aids in the prior year,
o First recorded smoking cessation intervention was NRT (using any of, or a combination of products) at the index date.
• Comparison Cohort o No recorded smoking cessation attempts using pharmacological aids in the prior year
o First recorded smoking cessation intervention was smoking cessation advice unaided by pharmacological therapies at the IPD and during the outcome periods.
Study Cohorts
• Exposure Cohort – NRT o No recorded exposure to CS pharmacological aids in the prior year,
o First recorded smoking cessation intervention was NRT (using any of, or a combination of products) at the index date.
• Comparison Cohort o No recorded smoking cessation attempts using pharmacological aids in the prior year
o First recorded smoking cessation intervention was smoking cessation advice unaided by pharmacological therapies at the IPD and during the outcome periods.
• All Patients o Aged: 18–75 years
o Current smoker throughout the prior year (any quantity of cigarettes).
o No past history of CVD
o Have at least one year of up-to-standard (UTS) baseline data as defined by GPRD (prior to the IPD) and at least 4 weeks‟ of UTS outcome data (following the IPD) or UTS data up to the time of death if death occurred within the outcome period.
Study Outcomes
• Cardiovascular event during 52-week outcome period:
o Coronary Heart Disease diagnosis
o Coronary Heart Disease related death
o Cerebrovascular disease diagnosis
o Cerebrovascular disease death and No of Days from IPD
o Number of GP consultations
o Hospital attendances for Coronary Heart Disease or
Cerebrovascular disease, (including admission, A&E attendance,
out-of-hours attendance, or Out-Patient Department (OPD)
attendance)
• Demographics
• Co-morbidities
• Therapies
• Clinical Outcomes
• Healthcare utilisation
• As for Baseline Variables
• Death
Baseline Variables
Statistical Analysis
• Baseline Variables
o Descriptive Analysis Means: t-test
Medians: Mann-Whitney U-Test
Proportion: Chi-squared test
• Matched Baseline & Outcome Variables
o Conditional Logistic Regression
• Change from Baseline Analyses
o Unadjusted: Conditional Logistic Regression
o Adjusted: Cox Proportional Hazards Model
• Baseline Variables
• Matching
• Baseline Analysis for Matched
Cohorts
• Outcome Variables
• Outcome Analysis
Results
• Baseline Variables
• Matching
• Baseline Analysis for Matched
Cohorts
• Outcome Variables
• Outcome Analysis
Results
Variables CS Advice NRT p
n 40,799 17,121
Age at IPD (years) Mean (SD) 47.9 (11.6) 46.8 (11.3) <0.001
Height (m) Mean (SD) 1.7 (0.1) 1.7 (0.1) <0.001
Weight (kg) Mean (SD) 76.7 (17.9) 76.0 (17.8) <0.001
BMI (kg/m2) Mean (SD) 26.7 (5.5) 26.6 (5.6) 0.178
Gender (Male) n (%) 18,776 (46.0) 8847 (51.7) <0.001
BMI Category: Underweight n (%) 854 (2.4) 407 (2.7)
0.101 Normal n (%) 14,456 (40.8) 6321 (41.5)
Overweight n (%) 12,169 (34.3) 5135 (33.7)
Obese n (%) 7989 (22.5) 3373 (22.1)
Year of IPD 2006 n (%) 18,005 (44.1) 10,136 (59.2)
<0.001 2007 n (%) 13,788 (33.8) 4901 (28.6)
2008 n (%) 9006 (22.1) 2084 (12.2)
Baseline Variables - demographics
Baseline Variables: co-morbidities
1 Read Code at any time, 2 At any time prior to and including IPD 3 Calculated using the Charlson Comorbidity Index over 1 year prior to & including IPD
0
4
8
12
16
CS Advice
NRT
*
*
*
*
* *
*
* *
% cohort
*p<0.05
Baseline Variables – therapies
0
2
4
6
8CS AdviceNRT%
cohort
*
*
*
*
*p<0.05
Variables CS Advice NRT p
Systolic Blood Pressure Mean (SD) 2496 (6.1) 1034 (6.0) <0.001
Diastolic Blood Pressure Mean (SD) 866 (2.1) 421 (2.5) <0.001
GP Consultations Mean (SD) 52 (0.1) 26 (0.2) <0.001
Total GP Consultations 0 - 2 n (%) 10,280 (25.2) 3095 (18.1)
<0.001 3 - 5 n (%) 11,366 (27.9) 4489 (26.2)
6 - 10 n (%) 9972 (24.4) 4616 (27.0)
11+ n (%) 9181 (22.5) 4921 (28.7)
GP consultations for CHD 1+ n (%) 76 (0.2) 71 (0.4) <0.001
GP Consultations for CerebroVD 1+ n (%) 130 (0.3) 88 (0.5) <0.001
Total OPD Attendance 1+ n (%) 5641 (13.8) 2731 (16.0) <0.001
Baseline Variables – characteristics
• Baseline Variables
• Matching
• Baseline Analysis for Matched
Cohorts
• Outcome Variables
• Outcome Analysis
Results
• To reduce difference between cohorts, cohorts
populations were matched
• 2 SC Advice : 1 NRT
• Patients were matched on:
o Gender
o Diabetes
o Cardiovascular Disease
o Hypertension
Matching
• Baseline Variables
• Matching
• Baseline Analysis for Matched
Cohorts
• Outcome Variables
• Outcome Analysis
Results
Variables CS Advice NRT p
n 33,852 16,926
Age at IPD (years) Mean (SD) 46.96 (11.2) 46.87 (11.3) <0.001
Height (m) Mean (SD) 1.69 (0.1) 1.69 (0.1) 0.839
Weight (kg) Mean (SD) 75.99 (17.9) 76.06 (17.8) 0.785
BMI (kg/m2) Mean (SD) 26.6 (5.6) 26.6 (5.6) 0.835
Gender (Male) n (%) 17,328 (51.2) 8664 (51.2) NA
BMI Category: Underweight n (%) 739 (2.5) 400 (2.7)
0.660 Normal n (%) 12,264 (41.6) 6245 (41.5)
Overweight n (%) 9923 (33.6) 5080 (33.7)
Obese n (%) 6574 (22.3) 3334 (22.1)
Year of IPD 2006 n (%) 14,920 (44.1) 10,019 (59.2)
<0.001 2007 n (%) 11,466 (33.9) 4839 (28.6)
2008 n (%) 7466 (22.1) 2068 (12.2)
Baseline Variables – demographics
Matched
Baseline Variables – co-morbidities
Matched
1 Read Code at any time, 2 At any time prior to and including IPD 3 Calculated using the Charlson Comorbidity Index over 1 year prior to & including IPD
0
4
8
12
16CS Advice
NRT
% matched cohort *
*p<0.05
* * * *
Baseline Variables – therapies
Matched
0
2
4
6
8CS Advice
NRT
% matched cohort
*p<0.05
* *
* *
*
*
Variables CS Advice NRT p
Systolic Blood Pressure Mean (SD) 130.9 (18.7) 130.8 (18.1) 0.018
Diastolic Blood Pressure Mean (SD) 79.5 (11.0) 79.2 (10.6) 0.012
GP Consultations Mean (SD) 7.5 (7.5) 8.8 (8.6) <0.001
Total GP Consultations 0 - 2 n (%) 8352 (24.7) 3077 (18.2)
<0.001 3 - 5 n (%) 9486 (28.0) 4443 (26.2)
6 - 10 n (%) 8371 (24.7) 4565 (27.0)
11+ n (%) 7643 (22.6) 4841 (28.6)
GP consultations for CHD 64 (0.2) 70 (0.4) <0.001
GP Consultations for CerebroVD 117 (0.3) 87 (0.5) 0.004
Total OPD Attendance 4707 (13.9) 2689 (15.9) <0.001
Baseline Variables – characteristics
Matched
• Baseline Variables
• Matching
• Baseline Analysis for Matched
Cohorts
• Outcome Variables
• Outcome Analysis
Results
Outcomes – 52 week outcome
0
0.2
0.4
0.6
0.8
1
Coronary Heart DiseaseDiagnosis
Cerebrovascular DiseaseDiagnosis
All Cause Mortality
CS Advice
NRT
* *
% matched cohort
*p<0.05
*
0
2
4
6
8
10
12CS Advice
NRT
% matched cohort
Outcomes – 52 week outcome
*p<0.05
* *
*
*
*
*
Outcomes – 52 week outcome
Healthcare Utilisation Variables CS Advice NRT p
Total Primary & Secondary Care 1 n (%) Consultations for CHD or Cerebrovascular Disease 2+ n (%)
135 (0.4) 107 (0.6) 0.110
54 (0.2) 49 (0.3)
GP Consultations for CHD n (%) 89 (0.3) 74 (0.4) <0.001
GP Consultations for Cerebrovascular Disease n (%) 94 (0.3) 82 (0.5) 0.065
No significant differences in OPD attendances or hospitalisations for CHD or Cerebrovascular Disease
• Baseline Variables
• Matching
• Baseline Analysis for Matched
Cohorts
• Outcome Variables
• Outcome Analysis
Results
Secondary Outcomes – 52 week Adjusted for Baseline Confounders
Time to first Coronary Heart Disease diagnosis
Time to first CardioVD diagnosis (ex prior Hx)
Time to first Cerebrovascular disease diagnosis
Time to first CerebroVD diagnosis (ex prior Hx)
All Cause Mortality
All Cause Mortality (ex prior Hx)
Primary & Secondary Care Consultations for CVD
Primary & Secondary Care Consultations for CVD (ex prior Hx)
Secondary Outcomes – 52 week Adjusted for Baseline Confounders
Cardiovascular Disease Cerebrovascular Disease
Secondary Outcomes – 52 week Adjusted for Baseline Confounders
Time to first Coronary Heart Disease diagnosis
Time to first CardioVD diagnosis (ex prior Hx)
Time to first Cerebrovascular disease diagnosis
Time to first CerebroVD diagnosis (ex prior Hx)
All Cause Mortality
All Cause Mortality (ex prior Hx)
Primary & Secondary Care Consultations for CVD
Primary & Secondary Care Consultations for CVD (ex prior Hx)
Secondary Outcomes – 52 week Adjusted for Baseline Confounders
Mortality
Secondary Outcomes – 52 week Adjusted for Baseline Confounders
Time to first Coronary Heart Disease diagnosis
Time to first CardioVD diagnosis (ex prior Hx)
Time to first Cerebrovascular disease diagnosis
Time to first CerebroVD diagnosis (ex prior Hx)
All Cause Mortality
All Cause Mortality (ex prior Hx)
Consultations for CHD & CVD
Consultations for CVD (ex prior Hx)
Secondary Outcomes – 52 week Adjusted for Baseline Confounders
Time to first Coronary Heart Disease diagnosis
Time to first CHD diagnosis (ex prior Hx)
Time to first Cerebrovascular Disease diagnosis
Time to first CVD diagnosis (ex prior Hx)
All Cause Mortality
All Cause Mortality (ex prior Hx)
Consultations for CVD
Consultations for CVD (ex prior Hx)
Deaths in 52 week Outcome Period
• Why?
o Predisposing factors?
– Demographics
– Co-morbidities
– Therapies
– Other?
Baseline Characteristics – Deaths in 52 week
Clinical Outcomes
0
20
40
60
80
200
6
200
7
200
8
CO
PD
Angin
a
CC
I S
core
= 1
CC
I S
core
= 2
Beta
-Blo
cker
Antipla
tele
t
Year of IPD Comorbidities Therapies
CS Advice
NRT%
cohort
*p<0.05 *
* *
* * *
* * *
Baseline Characteristics – Deaths in 52 week
Healthcare Utilisation
0
10
20
30
40
50
60
70
0 - 2 3 - 5 6 - 10 11+
Total GP Consultations Total OPDAttendance
CS Advice
NRT% cohort
*p<0.05
* * *
*
*
Conclusions
• NRT is strongly associated with increased risk of
o coronary heart disease
o cerebral vascular disease
o all cause mortality
Increased Risk is independent of prior history
• Death in NRT cohort associated with
o Earlier formulations of NRT
o Higher prevalence of COPD but not Angina
o Prescription of Beta-blockers, anti-platelet therapies
Limitations
• Availability of NRT
o NRT available over the counter in UK
• Data Availability
o Limited to data held in GPRD
o Are we missing other confounding factors?
• Causal Link
o No information on causality
Team Effort
• Data Management Julie von Ziegenweidt et al
o Protocol Design
o GPRD to usable dataset
o Matching
• Statistics Annie Burden et al
o Protocol Design
o Statistical Analysis
• Research Team David Price & Erika Sims
o Protocol Design
o Manuscript
Questions
Creating your own database: examples
from practice iHARP - a 6 country
cross-sectional database
Stanley Musgrave
Senior Research Fellow
University of East Anglia
• 1. Why and What is “iHARP”?
• 2. How has iHARP been set up
o International steering committee
o Mix of CROs and local champions
• 3. What type of information is in the iHARP database
• 4. iHARP data collection - UK: application and use of
questionnaires
• 5. iHARP data collection – Other countries: the website
and use of questionnaires
• 6. Use of the database for research; CRITIKAL
• ??Lessons from iHARP for other databases ??
Outline…
• In the earlier presentations, a “Standard” OPC database has been presented. While it remains the core, it isn‟t the only thing possible.
• Study designs – “The Standard” (retrospective observational), and alternatives: cross-sectional observational and others
• Data sources – “the Standard” (OPCRD and GPRD), and alternatives: supplemental clinical data
• Clinical and geographic settings – “The Standard” (UK general practice) and alternatives:
o UK, Europe and Australia;
o Pulmonologists, GPs, nurses, Respiratory technicians, pharmacists, in their clinical settings
…a quick word about issues that
influence what’s in your database….
• Implementation of the IPCRG’s Helping Asthma
in Real-world Patients (HARP) Strategy (i-HARP)
• A respiratory review service
• A multi-country collaboration
The Background
What and Why is iHARP?
From the “Original” Harp programme
Promote an integrated approach to asthma
management, which focuses on doctor and
patient behaviour and sharing of best
practices
All aspects of real world inhaler efficacy:
handling, airflow, device type, and particle
size, as well as compliance, individual
phenotype factors
Understanding the reasons for poor
asthma control
Haughney J, et al. Respir Med 2008; 102:1681–1693.
Incorrect
diagnosis
Asthma control assessed
Uncontrolled: either current
symptoms or exacerbations
Well controlled
Continue/consider
step-down
Poor
compliance Rhinitis Smoking
Inadequate
or
incorrect
therapy
Other
phenotypes
Virus-
associated
wheeze
Exercise
induced
Low necessity High concerns Mixed devices Increased
inflammation
Produces
steroid
resistance
Not right for
that patient
Need for
more therapy Side effects
Concerns
Intrusiveness
Poor training
Erosion
Smoking Poor
inhaler
technique
Errors with the inhalation manoeuvre
• Dose preparation
• Do not exhale
• MDI – inhalation time is too short so flow is too fast
• DPI – inhale too slow – not enough acceleration
Percentage of patients making one error and the perception of their GPs
Molimard et al J Aerosol Med 2003; 16: 249-254
Perc
enta
ge
Aerolizer Autohaler Diskus pMDI Turbuhaler n= 769 728 894 552 868
Patients making at least 1 error
GP opinion – patient inhaled the right dose
• 30% of patients do not exhale before their
inhalation Molimard et al J Aerosol
Med 2003; 16: 249-254.
• Greater relative lung deposition when exhaling
to RV Hindle et al. Thorax 1993; 48 :607-610
• Alveolar deposition ↑40% for each ↑1L of
inhaled volume Pavia et al. Thorax
1977.; 32: 194-197.
LUNG VOLUME and LUNG DEPOSITION
Inhalation Profiles Asthmatic Adults when using a DPI
0
20
40
60
80
100
120
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
Time (sec)
Inh
alat
ion
Flo
w R
ate
(l/
min
)
PIF: 101-108 L/min
PIF: 58-62 L/min
2.32 (85%)
1.64L (72%)
1.36L (35%)
1.35L (125%)
2.00L (55%)
1.14L (95%)
Inhaled volume (% pred FEV1)
DPI OPERATING PRINCIPLE
Energy Input Energy Carrier Process Result
Inspiratory Force
Flow & Pressure
Deaggregation Fine Particle
Mass
Lactose
Drug
Airflow Generated By Patient’s Inspiratory Effort
Turbulent Energy R Q P =
Integrated enhanced asthma reviews in Real-world Patients
• Patients invited for iHARP Review are:
• age 18+
• Have a current asthma diagnosis/therapy
• Are at BTS/SIGN Asthma step 3 or 4
• Patient filled questionnaire, clinician observation of inhaler use, and inhalation assessment (Spirotrac /AIMS)
• Inhaler handling errors - a priori defined critical errors, • with planned work to refine and examine several items considered likely to be
critical - potentially critical
A multi-country collaboration
◦ UK,
◦ Netherlands, France, Norway, Sweden, Italy, Spain, Australia
So, What is iHARP?
How has iHARP been setup?
iHARP steering committee
• UK: David Price, Henry Chrystyn, John Haughney, Dermot
Ryan, Kevin Gruffydd-Jones
• France: Nicolas Roche, David Costa
• Italy: Federico Lavorini, Alberto Papi, Antonio Infantino
• Spain: Miguel Román Rodríguez
• Germany: J Christian Virchow
• Sweden: Karin Lisspers, Björn Ställberg
• Australia: Sinthia Bosnic-Anticevich
• Norway: Svein Henrichsen
• Netherlands: Thys van der Molen
Country lead Service implementation Practices
Recruited
Spain Miguel Roman Spanish Primary Care Respiratory Group GP network (GRAP).
19
France Nicolas Roche
Managed by College Régional des Généralistes Enseignants in Auvergne. GPs recruited by David Costa and GP network
Pending
Netherlands Thys van der Molen Ellen van Heijst & asthma COPD service network Reviews
starting
Italy Alberto Papi Andrea Romangini & Ricerche Nouveau (CRO), supported by Federico Lavorini and Antonio Infantino’s network of GPs
5 (to increase
to 20 in July)
Spain Miguel Roman GRAP GP network 19
Sweden Bjorn Stallberg Karin Lisspers & network of GPs 1
Norway Svein Hoegh Henrichsen
Beraki Ghezai and network of GPs. July start
Australia Sinthia Bosnic-Anticevich
Implemented in community pharmacies 46
Germany J Christian Virchow - -
iHARP dataset
• Unique international dataset containing: o Clinical records
o Patient reported outcomes
o Clinician reviews
• Comprehensive data on treatment, patient reported outcomes, adherence, smoking status etc.
• Personalised feedback for both patients and healthcare professionals
• Customisation to national standards of practice
• Anonymous database for research purposes (such as the CRITIKAL study)
iHARP in UK: Streamlined process
Patient Questionnaire
Consultation with respiratory clinician
and patient
Patient Feedback -review -therapy
-risk identification -inhaler technique
Review of routine practise data
Patient fills in online
Questionnaire
Review of routine practise data Patient Information
in database
iHARP UK implementation
o Current medication
o Effects of asthma
symptoms on daily life
o Confounding factors
(e.g. smoking, rhinitis)
o Side effects
o Adherence to
medication and
attitudes about
adherence
Questionnaire
- online or in paper form, covering:
Page 1 of 2 Page 2 of 2
iHARP review: internet technology
patient-reported information
Inhalation assessment
Spirotrac sub-group
Inhalation time ≥3 sec for MDI is acceptable; <3 sec is critical
Objective acceleration assessment:
MDI PIF L/min
DPI low resistance IF L/min at 0.4 sec
DPI high resistance IF L/min at 0.4 sec
Acceptable 30 to 90 60+ 90+
potentially critical
<60 <90
Critical <30, or >90 <30 <60
MDI Does not remove Cap Critical Puff 1 - Does not shake before actuation Error Puff 1 - Does not breathe out Potentially Critical Puff 1 - Exhalation into the inhaler Error Puff 1 - Does not hold inhaler upright Critical Puff 1 - Puts inhaler in mouth but does not seal lips Potentially Critical Puff 1 - Does not have head tilted such that chin is slightly upwards Error Puff 1 - Actuation not corresponding to inhalation; actuation before inhalation Critical Puff 1 - Actuation not corresponding to inhalation; actuation is too late Critical Puff 1 - Inhalation is not slow and deep - defined as lasting at least 3 seconds Potentially critical Puff 1 - Failure to actuate Critical Puff 1 - Failure to inhale Critical Puff 1 - Inhalation through the nose Critical Puff 1 - No breath-hold (or for less than 3 seconds) Error Patient coughed during the inhalation data clarification Second dose within 30 seconds Error No Repeat second inhalation Potentially Critical items listed as "Puff 1" repeated for "Puff 2" Patient coughed during the inhalation data clarification After second inhalation - doesn't replace cap Error When asked - patient does not know how to tell that their device is empty Critical Patient has an expired device Potentially critical If on Fostair, ask if they know how long they can use their inhaler after receiving it from the pharmacy - should be less than 20 weeks/5 months.) potentially Critical Patient did not bring their own device to the clinical visit data clarification Does not mention priming when asked: "What do you do when you haven't used your inhaler for 24 hours? (Evohaler 1 week, Fostair 2 weeks)" Potentially Critical Does not mention priming when asked: "What do you do when you use your inhaler for the first time?" Potentially Critical
Feedback: considerations
for therapy and
management, to
the clinician,
who confirms its
appropriateness,
and
communicates to
patient.
Anonymous ID
saved locally
• Patients invited for iHARP Review are:
• age 18+
• Have a current asthma diagnosis/therapy
• Are at BTS/SIGN Asthma step 3 or 4
• Implementation in a variety of clinical settings
• GPs, referral assessment clinics
• Questionnaire, clinician review, handling error
observation & inhalation assessment with AIMS
iHARP “non-UK” implementation
iHARP internationally
Differences in the international process
• Clinical Implementation
o Nurses, GPs and pulmonologists
• Consultation setting
o Referral clinic, primary care site
• Airflow assessment
o AIMS2 rather than spirometry
• iHARP for research
o In the UK iHARP is a service provider; international ethics
approval allows the work to be used for research purposes
iHARP Global website
The Vitalograph Aerosol Inhalation Monitor 2 (AIM 2) is a small desktop device powered by mains batteries and designed to objectively assess patient’s inhaler technique. It has a placebo pMDI and DPI attached and utilizes a system of visual indicator lights to assist someone to assess their own or observe another persons’ ability to use an inhaler.
Flow indicators: Green indicator light confirms inhaler actuated within 3 seconds after starting to breathe in & while inhaling at an inspiratory flow between 10-50 l/min.
Syncronisation indicator: Green indicator light confirms coordination between pMDi
actuation and inspiration is present.
Breath Hold indicator: Acceptable breath-holding times indicated by green light;
Three indicator lights: confirming correct (green) or incorrect (red) technique:
• A different model
• Labnoord referral clinical laboratory, seeing patients
referred by GPs
• Questionnaires – a mix of existing Labnoord form
and a supplemental form for OPCRD,with clinical
assessment.
The Netherlands…
LabNoord Standard iHARP supplemental
Handling errors and AIMS forms and assessment
+ LabNoord routine clinical session data is collected as routine
LabNoord database subset
1. A LabNoord clinic session invitation is sent to patient with these questionnaires:
2. In the LabNoord clinic
Copy, Courier to RIRL; For scanning to Database
Within OPCRD, an iHARP “Netherlands subset” Data available for future use
(Anonymised ... with Unique ID) transfered to OPCRD
(with the Unique ID)
Data entry
• Aim:
o To identify the prognostic (patient- and treatment-related) factors
associated with asthma control in patients receiving maintenance
fixed-dose combination (FDC) inhaled corticosteroid / long-acting
beta2-agonist (ICS/LABA) therapy in primary care.
• Exposures -Patients receiving any of the following inhaled therapies:
– Fluticasone / salmeterol (FP/SAL; Seretide®) via Diskus DPI
– FP/SAL (Seretide®) via MDI
– Budesonide / formoterol (BUD/FOR; Symbicort®) via Turbuhaler (including patients on SMART-use as maintenance and reliever-therapy)
– Beclometasone dipropionate/ formoterol (BDP/FOR; Fostair®) via MDI
Critikal
• UK: David Price, Henry Chrystyn, John Haughney, Dermot
Ryan, Kevin Gruffydd-Jones
• France: Nicolas Roche, David Costa
• Italy: Federico Lavorini, Alberto Papi, Antonio Infantino
• Spain: Miguel Román Rodríguez
• Germany: J Christian Virchow
• Sweden: Karin Lisspers, Björn Ställberg
• Australia: Sinthia Bosnic-Anticevich
• Norway: Svein Henrichsen
• Netherlands: Thys van der Molen
Critikal – the iHARP steering Committee
Country Number of patients
UK and Australia 2500
France
2500
Italy
Germany
Spain
Sweden
Norway
TOTAL 5000
Critikal
(1) Asthma control: ATAQ, GINA
(2) Risk assessment - exacerbations in the prior year
(3) Inhaler technique:
(a) Subjective patient perception.
(b) Subjective acceleration assessment
(c) Objective clinician technique assessment
(Sub-groups only) (d) Objective acceleration assessment
(4) Adherence
(a) Subjective adherence assessment: patient perception
(b) Objective adherence assessment: prescription refills
…
Baseline measurements
• Supported by:
o Research in Real Life
o Grant from Mundipharma
Critikal
• Your research question and protocol
• Selecting and specifying data variables from the
OPCRD
• Time periods, selection and eligibility criteria
• Baseline description, demographics, risk factors…
• Other issues (matching…)
•
So. Creating your own database…
Designing your own project
Practical Group Work
Publishing Real-life Research
Professor David Price
• Displaying The Data
o Statistics
o Study Limitations
• Research Publication
o Real-life research becoming more accepted
o Not just accepted, but asked for!
Challenges of Real-life publications
Displaying real-life research: reviewers are
used to RCT format
RCT Real-life research
A: Initiation or step-up of therapy B: Switching therapy
Kenneth Stanley Circulation. 2007;115:1164-1169
Adjusting for confounding: minimising
baseline differences
• Convincing reviewers of the rigour of real-life
research
• Methods need to emphasise that confounding
factors have been minimised
o Statistical adjustments for baseline differences
o Patient matching
o Adjustments to address residual confounding
• RIRL definitions differ from clinical trial definitions
o Proxy Asthma Control
o Adherence
o Etc.
• Important to ensure that they are still clinically valid
• Current working on definition validation: showing the
validity, and importance, of our outcomes
Validation of Definitions
• ADEPT
o Independent committee that reviews OPC research
studies
o External review of OPC projects
• All trials registered on clinicaltrials.gov – results in
the public sphere
Approval and Validation
• Displaying The Data
o Statistics
o Study Limitations
• Research Publication
o Real-life research becoming more accepted
o Not just accepted, but asked for!
Challenges of Real-life publications
Who said this to The UK Royal College of Physicians?
“Randomised controlled trials (RCTs), long regarded at the 'gold standard‘ of evidence, have been put on an
undeserved pedestal.”
“They should be replaced by a diversity of approaches that involve
analysing the totality of the evidence-base.” Sir Michael Rawlins
Chairman of NICE
National Institute of Health & Clinical Excellence
Providing quality standards for healthcare
Classical RCTs: The Parachute Paradigm
Parachutes reduce the risk of injury after gravitational challenge, but their effectiveness has not been proved with randomised controlled trials.
(Smith GC, Pell JP. BMJ 2003; 327:1459-1461)
Common reviewer objections: RIRL solutions
• The possibility of residual confounding, eg, more knowledgeable / better doctors prescribing Qvar vs. the comparator o Possibilities listed in each study and specific solutions for
each study explored
• Outcome meanings differ from accepted clinical meaning (e.g “asthma control”) o Meanings well defined in paper and validation work
ongoing to justify them
• Confusion about how matching and statistical adjustments work o Full description detailed in methods section
• Patients with COPD are included in asthma studies o Smokers over 60 are excluded and very few diagnosis of
COPD younger than this
• Pragmatic trials designed and conducted to answer important questions facing patients, clinicians, and policymakers.
• Compare medical interventions that are directly relevant to clinical care or health care delivery and strive to assess effectiveness in real-world practice.
• Use broad eligibility criteria to ensure inclusion of patients whose care will be influenced by the trial‟s results.
Journals are starting to see the value of
real-life research
A treatment must be able to show its success rate in the “real-world”
In the USA: Interest is growing in “comparative
effectiveness research”
„„research evaluating and comparing health outcomes and the clinical
effectiveness, risks, and benefits of 2 or more health care interventions,
protocols for treatment, care management, and delivery, procedures,
medical devices, diagnostic tools, pharmaceuticals (including drugs and
biological agents), integrative health practices, and any other strategies or
items being used in the treatment, management, and diagnosis of, or
prevention of illness or injury in, individuals.‟‟
(J Allergy Clin Immunol 2011;127:123-7.)
CER as defined by the Patient Protection and Affordable Care Act
RIRL Research being published across a wide
variety of journals
• NEJM
• J Allergy Clin. Immunol
• CHEST
• J Asthma Allergy
• Respiratory Medicine
• Health Technol. Assess.
• PCRJ
Evidence from many designs: gives a full
picture of value of treatment
Evidence
Theoretical
Theoretical model provide
rationale
Classical double-blind
double-dummy RCTs
Gold standard, large range of
outcomes.
But not “real-life” patients,
compliance and represent <10%
of patients
Pragmatic trials
More real-life Broader inclusion
criteria Allow normal factors to
occur Usually randomised
Simple outcomes But still consent
& rigorous
Observational Data
Real-life patients Not randomised
Routine data Normal decisions Difficult to ensure
group comparability
Matching of case controls,
adjustment
Top Related