Course: Research in Biomedicine and Health IIISeminar 4: Critical assessment of evidence

EBM steps◦Step 1: Formulating questions that can be

answered◦Step 2: Finding best evidence◦Step 3: Quick critical assessment of the

evidence◦Step 4: Applying evidence◦Step 5: Assessing effectiveness and

efficiency of the process

3. Critical assessment of evidence

1. Are study results valid?

2. What are the results?

3. Can the results be applied to a concrete patient?

Example: RCTValidity of research:

Is the clinical question clearly formulated?

Was the allocation of participants into groups random?

Was the allocation sequence know to the physician treating patients?

Were all participants analyzed and was it intention to treat analysis?

Were the participants in experimental and control group similar in known prognostic factors?

Were the participants/physicians/assessors blinded to the study groups?

Was the follow-up of participants complete?

Were the data adequately statistically analyzed?

Example: Your neighbor asks if zinc pastils would help her 10 year-old daughter in recovering from a common cold

The researchers state that the study was randomized and describe the randomization method.

Out of 124 children in the experimental group, 7 (6%) terminated the study, and 3 (2%) out of 125 children in the control group

During analysis, all participants were included, regardless whether they quit the study;

Authors provide detailed data on the characteristics of two groups and show that they are similar.

Authors also state that all involved in the study were blinded to the received treatment.

In the end, you are convinced that the strength of evidence in the study is high enough and that the risk of bias is small.

Example: RCTStudy result: Outcome measures

ARR = absolute risk reduction = EER – CER

RRR = relative risk reduction = (CER – EER) / CER = ARR/CER

NNT = number needed to treat = 1/ARR

95% confidence interval (95%CI)

Example: Your neighbor asks if zinc pastilles would help her 10 year-old daughter in recovering from a common cold

School missing(No. children missing)

RRR ARR NNT

PlaceboCER

Zinc pastillesEER

(CER-EER)/CER CER-EER 1/ARR

20.8% 18.5% (20.8%-18.5%)/20.8% = 11.1%

20.8% - 18.5% = 2.3%

1/2.3% = 44 patients

Adverse effects (e.g., poor taste in the mouth; dizinnes; irritation of the oral cavity, tongue and throat, diarrhoea)

RRI (increase in RR)

ARI (increase in AR)

Number needed to harm(NNH)

PlaceboCER

Zinc pastilsEER

(CER-EER)/CER CER-EER 1/ARI

79.8% 88.6% (79.8%-88.6%)/79.8% = 11.0%

79.8% - 88.6% = 8.8%

1/8.8% = 11 patients

What needs to be reported in an article on RCT

• CONSORT statement

http://www.consort-statement.org/

Checklist

Flow diagram

http://www.consort-statement.org

Schulz KF, Altman DG, Moher D, for the CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. Ann Int Med 2010;152 (11):726-32

Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG, for the CONSORT Group. CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trial. BMJ 2010;340:c869

•checklist of essential items that should be included in reports of RCTs• diagram for documenting the flow of participants through a trial

13

Title and abstract1a Identification as a randomised trial in the title 1b Structured summary of trial design, methods,

results, and conclusions (for specific guidance see CONSORT for abstracts)

14

Background and objectives

2a Scientific background and explanation of rationale

2b Specific objectives or hypotheses

15

16

Trial design3a Description of trial design (such as parallel, factorial)

including allocation ratio3b Important changes to methods after trial

commencement (such as eligibility criteria), with reasons

Participants4a Eligibility criteria for participants 4b Settings and locations where the data were collected

17

18

19

Interventions5 The interventions for each group with sufficient details to

allow replication, including how and when they were actually administered

Outcomes6a Completely defined pre-specified primary and

secondary outcome measures, including how and when they were assessed

6b Any changes to trial outcomes after the trial commenced, with reasons

20

21

22

Sample size

7a How sample size was determined 7b When applicable, explanation of any interim

analyses and stopping guidelines

23

24

Randomisation Sequence generation8a Method used to generate the random allocation

sequence8b Type of randomisation; details of any restriction (such

as blocking and block size)

Allocation concealment mechanism9 Mechanism used to implement the random allocation

sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned

25

26

27

Randomisation Implementation10 Who generated the random allocation sequence, who

enrolled participants, and who assigned participants to interventions

Blinding11a If done, who was blinded after assignment to interventions

(for example, participants, care providers, those assessing outcomes) and how

11b If relevant, description of the similarity of interventions

28

29

30

Statistical methods

12a Statistical methods used to compare groups for primary and secondary outcomes

12b Methods for additional analyses, such as subgroup analyses and adjusted analyses

31

32

33

Participant flow (a diagram is strongly recommended)13a For each group, the numbers of participants who

were randomly assigned, received intended treatment, and were analysed for the primary outcome

13b For each group, losses and exclusions after randomisation, together with reasons

Recruitment14a Dates defining the periods of recruitment and follow-

up14b Why the trial ended or was stopped

34

35

36

Baseline data15 A table showing baseline demographic and

clinical characteristics for each group

Numbers analysed16 For each group, number of participants

(denominator) included in each analysis and whether the analysis was by original assigned groups

37

38

39

Outcomes and estimation17a For each primary and secondary outcome,

results for each group, and the estimated effect size and its precision (such as 95% confidence interval)

17b For binary outcomes, presentation of both absolute and relative effect sizes is recommended

CONSORT items and examplesResults

40

41

42

Ancillary analyses18 Results of any other analyses performed,

including subgroup analyses and adjusted analyses, distinguishing pre-specified from exploratory

Harms19 All important harms or unintended effects in

each group (for specific guidance see CONSORT for harms

43

44

45

Limitations20 Trial limitations, addressing sources of potential

bias, imprecision, and, if relevant, multiplicity of analysesGeneralisability21 Generalisability (external validity, applicability) of

the trial findingsInterpretation22 Interpretation consistent with results, balancing

benefits and harms, and considering other relevant evidence

46

47

48

49

Registration23 Registration number and name of trial registry

Protocol24 Where the full trial protocol can be accessed, if

availableFunding25 Sources of funding and other support (such as

supply of drugs), role of funders

50

RCT

Applicability of results to individual patients

Are the participants in the study similar to patients who you want to treat with the intervention in question?

Were all clinically relevant outcomes studied? Can the intervention be administered in the patient’s

setting? Are there other types of treatments? Is the benefit of the intervention greater than

possible risk or price?

4. Clinical application of evidence

• Guidance for clinical practice

• Decision analysis:probability

xSubjective assessment of the condition (utility)

5. Evaluation