Can we teach critical appraisal
‘the GATE approach’
Rod Jackson
University of Auckland, NZ
September 2010
in 30 minutes!
GATE: a Graphic Appraisal Tool for Epidemiology
1 picture, 2 formulas &
3 acronyms
A picture: the GATE frame
the shape of every epidemiological study
©
Participants
Exposure Group Comparison Group
OutcomesTime
P
E C
O
T
The PECOT acronym: the 5 parts of every epidemiological study
All epidemiological studies can be hung on the GATE frame
2 formulas: study analyses
1. Occurrence of disease
= Numerator ÷ Denominator ÷ Time
2. Random error (95% Confidence Interval)
= 1.96 x Standard Error
D
N
P
E C
OT
P
E
C
O
T
Recruitment
Allocation
Maintenance
Measurement of outcomes
blind or
objective
The RAMMbo* acronym: assessing study bias
* Paul Glasziou
Participants
Exposure Group Comparison Group
OutcomesTime
P
E C
O
T
GATE frame picture & PECOT acronym
Describe a study’s design by hanging PECOT on the frame
Participants
Study Setting
Eligible Participants
ParticipantsP
Fill in for
DVT in long-haul flights.Lancet 2001; 357:1485-9
Page 95 in Glasziou et al
ParticipantsStudy Setting: volunteers, UK, ? 1990s
Eligible Participants: no previous DVT, > 50 yrs, planned economy air travel 2 sectors > 8 hours
Participants: 200, mean age 61-62 years
P
DVT in long-haul flights: Lancet 2001;357:1485-9
Exposure & Comparison Groups
Exposure or Intervention Group
(EG)
Comparison or Control Group
(CG)EG CG
Fill in for
DVT in long-haul flights.Lancet 2001; 357:1485-9
Page 95 in Glasziou et al
Exposure & Comparison Groups
Exposure or Intervention Group
(EG):Below knee compression
stockings
Comparison or Control Group
(CG):no stockings
100 100
115 116
Outcomes (O)
Outcomes (O)Oa b
c d
yes
no
‘Dis-ease’
Fill in for
DVT in long-haul flights.Lancet 2001; 357:1485-9
Page 95 in Glasziou et al
Outcomes (O)
Outcomes (O)O
a= 0
b= 12
c d
yes
noDVT
100 100
Time (T)
T
incidence
prevalence
Fill in for
DVT in long-haul flights.Lancet 2001; 357:1485-9
Page 95 in Glasziou et al
Time (T)
T= from take-off to 2 days post
final flight
incidence
Outcome: e.g. death
0 0
Time (T)
T = at post-flight assessment (<48 hrs)
prevalence
Outcome: number with
DVTs 0 12
Study design: GATE frame & PECOT
Participants
Exposure Group Comparison Group
OutcomesTime
P
E C
O
TEvery epidemiological study hangs on the GATE frame
Participants
Exposure Group:stockings
Comparison Group:no stockings
Outcomes:DVT
Time:at post flight assess
231
100 100
DVT in long-haul flights: Lancet 2001;357:1485-9
115 116
Setting: UK volunteers – 479 consideredEligible: > 50 yrs, no DVT, flying > 8 hrs
0 12
2 formulas: study analyses
1. Occurrence of disease
= Numerator ÷ Denominator ÷ Time
2. Random error (95% Confidence Interval)
= 1.96 x Standard Error
D
N
Denominator (Participants)D
N Numerator (Outcomes)
O = N÷D
All epidemiological studies involve measuring the OCCURRENCE of disease
Occurrence = Numerator ÷ Denominator
GATE study analyses
P
EG CG
O
Exposure Group (EG)
Numerator 1: a
Comparison Group (CG)
Overall Denominator
a b
c d
Numerator 2: b
Describe a study’s analyses by hanging the numbers on the frame
Denominator 1: Denominator 2:
Occurrence = N ÷ D
P
EG CG
O
Denominator 1:Exposure Group
EG
Numerator 1:a
Denominator 2:Comparison Group
CG
a b
c d
Numerator 2:b
Exposure Group Occurrence:EGO = a ÷ EG
Comparison Group Occurrence:CGO = b ÷ CG
Calculate EGO & CGO for the outcome ‘DVT’ post long-haul flight
P
EG CG
Oa b
c d
Denominator 1:Exposure Group
EG = 100
Numerator 1:a = 3*
Denominator 2:Comparison Group
CG = 100
Numerator 2:b = 12
EGO = 3/100 people at post flight assessment
CGO = 12/100= people at post flight assessment
* a = 0 but have used 3 to illustrate calculations on next slide
Describing differences between occurrences
Relative difference or Relative Risk = EGO ÷ CGO
Absolute Difference or Risk Difference = EGO - CGO
Number Needed To Treat (NNT) = 1 ÷ RD
Fill in for
DVT in long-haul flights.Lancet 2001; 357:1485-9
Page 95 in Glasziou et al
Describing differences between occurrences
Relative difference or Relative Risk = EGO ÷ CGO
Absolute Difference or Risk Difference = EGO - CGO
Number Needed To Treat (NNT) = 1 ÷ RD
= 3/100 ÷ 12/100 = 3/12 = 0.25
= 3/100 - 12/100 = - 9/12 = - 7.5 /100
= 1 ÷ (- 7.5 /100) = - 100/7.5 = 13.3
Analyses
it’s all about EGO & CGO
P
E C
OT
P
E
C
O
T
Recruitment
Allocation
Maintenance
Measurement of outcomes
blind or
objective
The RAMMbo acronym: assessing study bias
RAMMbo
E C
OT
appropriate Recruitment processes?P
• Study setting & eligibility criteria well described?
e.g. Recruit random/representative sample OR consecutive eligibles OR volunteers from advertisements• Participants representative of eligibles?• Prognostic/risk profile appropriate to
study question?
P
Study setting
Eligible people
RAMMbo
E C
OT
appropriate Recruitment processes?P
• Study setting & eligibility criteria well described?
Recruited volunteers from advertisements; eligibility criteria well described• Participants representative of eligibles?479 considered, 248 excluded - probably• Prognostic/risk profile appropriate to
study question? yes
P
Study setting
Eligible people
EG CG
OT
P
RCT: Allocate randomly by investigators (e.g drugs)
EG CG
OT
P
Cohort: Allocate by measurement (e.g. smoking)
RAMMbo: A is for Allocation
were EG & CG similar at baseline?
EG CG
OT
P
RCT: Allocate randomly by investigators using sealed envelopes
RAMMbo: A is for Allocation
EG & CG similar at baseline except more women in stocking group
RAMMbo
EG CG
OT
good Maintenance?did most participants remain in allocated groups (EG &
CG)
P
Participants &/or investigators blind to exposure (and comparison exposure)?
Compliance high & similar in EG & CG?Contamination low & similar in EG & CG?Co-interventions low & similar in EG & CG?
Completeness of follow-up high & similar in EG & CG?
RAMMbo
100 100
OT
good Maintenance?did most participants remain in allocated groups (EG &
CG)
P
Participants &/or investigators blind to exposure (and comparison exposure)? no
Compliance high & similar in EG & CG? dk*Contamination low & similar in EG & CG? dkCo-interventions low & similar in EG & CG? dk
Completeness of follow-up high & similar in EG & CG? probably ok, lost < 15%* dk = don’t know
115 116
RAMMbo
EG CG
OT
Measurement of outcomesblind or objective?
P
If outcome measurements not Objective (eg. automated or definitive) were investigators blind to exposure (and comparison exposure)
RAMMbo
EG CG
OT
Measurement of outcomesblind or objective?
P
Outcome measurements reasonably objective (ultra-sound)Technician was blind to exposure (although some participants may have had a stocking line)
The 4 (GATE) study biases
P
E C
OT
Recruitment bias
Allocation bias
Maintenance bias
Measurement (of outcomes) bias
P
E C
OT
Recruitment ok
Allocation ok
Maintenance ok
Measurement (of outcomes) ok
DVT in long-haul flights: Lancet 2001;357:1485-9
2 formulas: study analyses
1. Occurrence of disease
= Numerator ÷ Denominator ÷ Time
2. Random error (95% Confidence Interval)
= 1.96 x Standard Error
D
N
2 formulas: study analyses
1. Occurrence of disease
= Numerator ÷ Denominator ÷ Time
2. Random error (95% Confidence Interval)
= 1.96 x Standard Error
EGO = 0%; 95% CI = 0% - 3%
CGO = 10%; 95% CI = 4.8% - 16%
D
N
Excel CATs & paper Gate-lites
There is a GATE for every study designwww.epiq.co.nz
the 3rd acronym:SRs & meta-analyses
• Find appropriate studies?
• Appraise selected studies?
• Include only valid studies?
• Total-up (synthesise) appropriately?
• Heterogeneity adequated addressed?
In the 19th century we made great advances in health through the provision of clean, clear water; in the 21st century
we will make the same advances through clean, clear (systematically
reviewed) information.Muir Gray
Systematic reviews are one of the greatest ideas of modern thought.
They should be celebrated.
Ben Goldacre
Muhammad Shafique Sajid et al. Knee-length graduated compression stockings for thromboprophylaxis in air travellers: A meta-analysis. Int J Angiol. 2008; 17: 119–123.
Nine trials studying participants using KL stockings were analyzed. Forty-six of 1261 participants randomly assigned to the control group developed deep vein thrombosis (DVT), compared with two of 1237 participants (0.16%) in the KL stockings group. …..There was an absolute difference of 3.4% in the incidence of DVT, in favour of KL stockings. The number needed to treat with KL stockings to avoid one case of DVT was 29.4. However, there was significant heterogeneity among trials. The RR for DVT was 0.08 in high-risk participants and 0.14 in low- to medium-risk participants.
GATE can also be used to frame the first 4 steps of EBP
The 4 steps of EBP1. Ask a focused question.
2. Access (systematically search for) epidemiological evidence to help answer question.
3. Appraise evidence found for its validity, effect size, precision (ideally all the relevant evidence)
4. Apply the evidence:
a. amalgamate the valid evidence with other relevant information (patient/community values, clinical/health issues, & policy context) to make a good decision; and
b. Act (implement) the decision in practice
Participants
Exposure Group Comparison Group
OutcomesTime
P
E C
O
T
the PECOT acronym: the 5 parts of every epidemiological study
All epidemiological studies can be hung on the GATE frame
EBP Step 1: ASK - turn your question into 5 parts (PECOT)
1. Participants (the patient problem)
2. Exposure (e.g. a therapy)
3. Comparison (there is always an alternative! - another therapy or no treatment…
4. Outcome (e.g. a disease you want to prevent or manage)
5. Time frame (over which you expect a result)
EBP Step 2: ACCESS the evidence – use PECOT to identify search terms
1. Participants (the patient problem)
2. Exposure (e.g. a therapy)
3. Comparison (there is always an alternative! - another therapy or no treatment…
4. Outcome (e.g. a disease you want to prevent or manage)
5. Time frame (over which you expect a result)
P
E C
OT
P
E
C
O
T
Recruitment
Allocation
Maintenance
Measurement of outcomes
blind or
objective
EBP Step 3: APPRAISE the evidence – using PECOT & RAMMbo
EBP Step 4: APPLY the evidence by: a. AMALGAMATING the relevant information & making
an evidence-based decision:’ the X-factor
©
Epidemiologic evidence
Clinical / health considerations
Policy issues
Patient / community preferences
X-factor: making evidence-based decisions
expertise: ‘putting it all together’ the art of practice
Step 4b
ACT
Step 5: EBP 360°1. Ask a focused question.
2. Access (systematically search for) epidemiological evidence to help answer question.
3. Appraise evidence AND then meta-analyse (systematically review) ALL relevant valid evidence.
4. Apply the best evidence:
a. amalgamate the valid evidence with other relevant information to make a good decision; and
b. ACT on your decision5. AUDIT your practice (i.e. check your actual practice –
‘actions’ - against ‘best’ evidence-based practice)
= EBP + Quality Improvement
Top Related