How to Teach Statistics in EBM

Rafael Perera

Basic teaching advice

• Know your audience• Know your audience!• Create a knowledge gap• Give a map of the main concepts• Decide which ones to focus on• Use plenty of examples• Let them do the work/thinking

Main Concepts

• Bias and Measurement error• P values and Confidence Intervals• Which Statistical tests are needed and when• Correlation and Association• Models / Regression and alternatives for

Adjustment• Survival Analysis• Meta Analysis • Statistics for Diagnostic Studies

There is a time and place…

Fundamental Equation of Error

• Measure = Truth + Bias + Random Error

Use good study design

Use large

numbers

Researcher

Critically AppraiseDesign

ConfidenceIntervals

andP-values

Reader

true result

Biaslow high

Ra

nd

om

err

or

hig

h

low

Bias versus Random error

Bias and Measurement error

Groups of 3-4 people

1 – subject

2 – measurers

Measurers – measure (twice) and record the head size of the subject. Keep measurements hidden.

Bias and Measurement error

Intra-Observer variability

Measurement error

• Same answer

• Varied by < 0.5 cm

• Varied by < 1cm

• Varied by < 2 cm

• Varied by >2 cm

Bias and Measurement error

Inter-Observer variability

Measurement error

• Same answer

• Varied by < 0.5 cm

• Varied by < 1cm

• Varied by < 2 cm

• Varied by >2 cm

Bias and Measurement error

Bias

Included ears?

Included nose?

Which part of the head?

Other?

Does it matter?

In paediatric practice following meningitis, a head circumference that increases by 7mm in a day will result in urgent head imaging

In obstetrics measurements of the fundal height can vary by up to 5cm (the difference between having a baby delivered early due to IUGR or not when opposite occur)

The question is can you reproduce the test in your setting and will it perform as

well in your setting

Measuring Random error

Most things don’t work!

Two methods of assessing the role of random error

•P-values • (Hypothesis Testing)

– use statistical test to examine the ‘null’ hypothesis

– if p<0.05 then result is statistically significant

•Confidence Intervals(Estimation)

– estimates the range of values that is likely to include the true value

Relationship between p-values and confidence intervalsIf the ‘no effect’ value falls outside the CI then the result is statistically significant

The Steps in Testing a Hypothesis

State the

null hypothesis

H0 Choose the

test statistic

that

summarizes

the data

Based on H0

calculate the probability of

getting the

value of the

test statistic

Interpret the

P-value

Some Statistical tests

• Comparing groups– T-tests (1 or 2 groups, normally distributed)– Chi-squared (2 or more groups, categorical or binary data)– Mann-Whitney U (2 groups, non-normal data)– Log-rank test (2 groups, survival data)– ANOVA (multiple groups, normally distributed)– …

• Tips: – Understand what the hypothesis being tested is– Use the p-value to assess the level of evidence against it– (Experienced) Assess if the test was adequate for the question

and data analysed

Hand outs

1. Incidence/ Prevalence and CI2. Survival analysis3. Regression models / Adjustment4. Linear association / Correlation 5. Confounding / Odds Ratios / Logistic

Regression6. Diagnostic Tests7. Meta-analysis

Reading confidence intervals

Clinically significant

Vitamin X shortens a 5 day cold

Would you take it twice per day if it shortened the cold by:

Clinically significant

Vitamin X shortens a 5 day cold

Would you take it twice per day if it shortened the cold by:

50%

Clinically significant

Vitamin X shortens a 5 day cold

Would you take it twice per day if it shortened the cold by:

50%20%

Clinically significant

Vitamin X shortens a 5 day cold

Would you take it twice per day if it shortened the cold by:

50%20%10%

Clinically significant

Vitamin X shortens a 5 day cold

Would you take it twice per day if it shortened the cold by:

50%20%10%5%

Clinically significant

Vitamin X shortens a 5 day cold

Would you take it twice per day if it shortened the cold by:

50%20%10%5%1%

(a) (b)

(c) (d)

Minimum clinicalImportant difference

No difference

Which are clinically significant?

010

20

Thank you

EXTRAS

Different types of measurements use different types of statistics

• Dichotomous: – Male,female OR infected, non-infected

• Categorical: – Red, green, blue OR

• Ordinal: – Nil, +, ++ of glucose

• Interval:– temperature

STATISTICS

Proportion, Risk

Mode, Proportions

Mode, Median?

Mean, Median

> t wo samples

Independence between two or more variables

Parametric

Non parametric

Between means for

continuous data

Between

distributions

Hypothesis testing and

a ssessing d ifferences

Parametric ANOVA

Sign test for related

samples

Rank sum test for independent samples

Kruskal Wallis

T test difference for related samples

Non parametric

T test for independent

samples

McNemar’s test for

related groups

Between one observed

variable and a theoretical

distribution

X2 test for goodness

of fit

X 2 test for

independence

Two samples

One sample vs. H 0

One sample vs. H 0 Z score equal

proportions

Z score

Between proportions forcategorical data

Flowchart of Statistical Tests for Hypothesis Testing

Flowchart of Statistical Tests for Hypothesis Testing

Between distributions

Between one observed variable and a theoretical distribution

Independence between two or more variables

c2 test for goodness of fit

c2 test for independence

McNemar’s test for related groups

Flowchart of Statistical Tests for Hypothesis Testing

Between means for continuous data

Two samples

t-test independent samples

Rank sum test for independent samples

Sign test for related samples

t-test difference for related samples

ANOVA

Kruskal – Wallis

Parametric

> two samples

Non Parametric

Parametric

Non Parametric

Flowchart of Statistical Tests for Hypothesis Testing

Between proportions for categorical data

One sample vs. H0

Two samples

Z-score

Z-score equal proportions

Summarising proportions One sample: Risk, OddsTwo samples: Relative risk, Odds ratios, Risk differences