Log in to Poll everywhere

www.pollev.com/register

Type in presenter name:

martindachse767

Happy statistics

Martin Dachsel, Acute Medicine Consultant

Medicine will be harder after this

talk!!!!

Black or white

The World of Ana Steele

Qualities of a good clinician for diagnosis and Management

Knowledgable Clinical Decision maker

Medical Schools and

PG Education?

Think, reason and decision making → Probably most critical skill

What does clinical reasoning involve

• Includes: • History

• Examination findings

• Test results

Clinical Uncertainty

Diagnosis and Treatment

Cognitive Biases

Group of people read profile

Equal mix of introvert and extrovert attidudes

What happened?

Recalling more examples

of introversion

Recalling more examples

of extroversion

Confirmation Bias

Cognitive Biases

However

Around 160 cognitive biases described on Wikipedia

Metacognition and cognitive debiasing

• Thinking about thinking → reflection about clinical decision making

• Is used to reduce the impact of biases

• Debiasing examples: • Differential diagnosis → forcing strategy to consider other diagnoses → will work for anchoring, adjustment, search satisficing, premature diagnostic closure

• Mnemonics → protect against memory failures, ensures full range of possibilities is considered → works against availability bias, anchoring and adjustment

It’s all in the history and examinationClinician starts to generate different hypotheses

Can start asking specific questions

Testing of Hypotheses

Physical Examination can narrow things down

Test results narrow differentials further

But diagnostic tests tell us for sure, or?

Test

Results

Pre-test probability / belief

Increase / Decrease likelihood of disease

Mr T, 88 years

Ischaemic bowel (pre-test probability 80%)

Severe abdominal pain, lactate of 12, history of AF

Excellent Test: CT angio (Sensitivity 93%, Specificity 94%) is NEGATIVE

Excluded Ischaemic Bowel? Chance still 20%

Mammography for breast cancer

• Prevalence of breast cancer age 40: 1%

• Sensitivity of mammography: 80%

• Specificity: 90.4%

• What’s the likelihood that someone positively tested has breast cancer?

Epidemiology

Positive Test means Cancer in

7.8%

P(Cancer / +ve Mammogram) = P(+ve Mammogram / Cancer) x P(Cancer)

P(+ve Mammogram)

P(Cancer / +ve Mammogram) = Sensitivity x Prevalence

P(+ve Mammogram)

Cancer

No Cancer

+ ve test

+ ve test

- ve test

- ve test

P( +ve Mamogram) = (0.01 x 0.8) + (0.99 x 0.096) = 0.103

P(Cancer / +ve Mammogram) = P(+ve Mammogram / Cancer) x P(Cancer)

P(+ve Mammogram)

P(Cancer / +ve Mammogram) = Sensitivity x Prevalence

P(+ve Mammogram)

P(Cancer / +ve Mammogram) = 0.8 x 0.01

0.103

P(Cancer / +ve Mammogram) = 0.0776 = 7.76%

P(Cancer / +ve Mammogram) = 0.8 x 0.20

0.2368

P(Cancer / +ve Mammogram) = 0.676 = 67.6%

How about women over 70

Prevalence of breast cancer age 70: 20 %Sensitivity of mammography: 80%Specifity: 90.4%

Ruling In versus Ruling Out test

•Highly sensitive test ?

•Highly Specific Test ?

Diagnostic Tests (Troponin T levels)

Threshold 13

Troponin T 13

ACS No ACS

Trop >13 100 8

Trop <13 0 70

Sensitivity: 100/100 = 1

Specificity: 70/78 = 0.89

PPV: 100/108 = 0.92

NPV: 70/70 = 1

Threshold 30

Threshold 30

Sensitivity: 80/100 = 0.80

Specificity: 85/88 = 0.96

PPV: 80/83 = 0.96

NPV: 85/105 = 0.81

ACS No ACS

Trop >30 80 3

Trop <30 20 85

0

10

20

30

40

50

60

70

80

90

100

No ACS ACS

Threshold 55

Threshold 55

Sensitivity: 50/100 = 0.50

Specificity: 88/88 = 1

PPV: 50/50 = 1

NPV: 88/138 = 0.64

ACS No ACS

Trop >55 50 0

Trop <55 50 88

Highly Sensitive Test (i.e. Troponin level 13)

PPV: 100/108 = 0.92

NPV: 70/70 = 1Good Rule out test

Highly Specific Test (i.e. Troponin level 55)

PPV: 50/50 = 1

NPV: 88/138 = 0.64Good Rule in test

Graphical Representation

• ?

ROC curves

Absolute vs relative risk reduction

Absolute risk reduction: 1%Relative risk reduction: 50%

Sepsis Mortality

•Neighboring trust showed reduction in Sepsis mortality from 16% to 11%

•Automatic call of outreach team when NEWS>3

•Highly celebrated project

Over 3 months

before after

Reviewed patients 430 650Absolute deaths 70 72Mortality rate 16.3% 11.1%

•Work increase for outreach team by 51%•No decrease in Sepsis mortality

The Solution: the NNT

•Number needed to treat = 1/absolute risk reduction

•Answer to how many patient I need to treat to have 1 positive outcome → really good to explain benefit and risks to patients and families

•Coffee example: absolute risk reduction 1% : NNT = 1 / 0.01 = 100

NNT of 5

• New drug to reduce flatulence: NNT of 5

• Good?

• Side effect: 10% suffer fatal MI

• Still good?

NNT of 50

• New Asthma drug reduces hospital admission in 1 in 50 patients

• Good?

• How about harm:• None?

• Brilliant?

• Costs £100.000 per patient, so 1 hospital admission avoidance: £ 5.000.000

NNT of 500

• Novel treatment to avoid heart attacks

• Good?

• No side effects

• Good?

• Free

• Good?

• Exercise 3 x week

NNT

•Look at the whole picture:•NNT•NNH: number needed to harm•Costs / efficiency

Fragility Index

Event No Event

Treatment A 2 98

Treatment B 10 90

Event No Event

Treatment A 3 97

Treatment B 10 90

Fisher’s exact test: p=0.03 Fisher’s exact test: p=0.08

How many events needed to render trial not significant?

Fragility Index: 1

Trial of 56 Randomised ICU trials (Ridgeon et al., 2016): Median Fragility Index: ?

2Fragility Index < loss to follow up in 87%

Dr Nobel vs Dr Repeat

• Dr Nobel aspires for Nobel Prize, studies highly original hypotheses

• Dr Repeat repeats the research of others, looks to reproduce significant results

Dr Noble versus Dr Repeat

• Dr Noble: • 1000 randomised trials, 900 studies: Null Hypothesis is true, 100 studies:

Alternative hypothesis is true• All trials had large amount of patients, type I error 5% (effect shown when

null hypothesis is true) and type II error is 10% (effect not shown when alternative hypothesis is true)

• Dr Repeat:• 1000 trials, however examining previous positive trials, Alternative hypothesis

true in 900 trials, Null hypothesis true in 100 trials• Type I error is 5%, type II error is 10%

HA H0 Total

p<0.05

p>0.05

Total 1000

Dr Nobel

• 1000 randomised trials, 900 studies: Null Hypothesis true, 100 studies: Alternative Hypothesis is true

• All trials had large amount of patients, type I error 5% (effect shown when null hypothesis is true) and type II error is 10% (effect not shown when alternative hypothesis is true)

HA H0 Total

p<0.05

p>0.05

Total 100 900

Total

135

865

1000

H0

45

855

900

HA

90

10

100

Dr Repeat

• 1000 trials, however examining previous positive trials, Alternative hypothesis true in 900 trials, Null hypothesis true in 100 trials

• Type I error is 5%, type II error is 10%

HA H0 Total

p<0.05

p>0.05

Total 900 100 1000

H0

5

95

100

Total

815

185

1000

HA

810

90

900

Who is more trustworthy

•Positive result: Dr Noble vs Dr Repeat

•Negative result: Dr Noble vs Dr Repeat

Lets work it out

• Positive result: • Dr Noble: 135 trials, in 45 (33%) the null hypothesis was true (90/135 = 66%)

• Dr Repeat: 815 trials, in 5 trials the null hypothesis was true (810/815 = 99.4%)

• Negative result: • Dr Noble: 865 trials, in 855 the null hypothesis is true (855/865 = 98.8%)

• Dr Repeat: 185 trials, in 95 the null hypothesis is true (95/185 = 51.3%)

What to make out of this

• When we read a study of a trial we must consider if study belongs to Dr Repeat’s or Dr Noble’s universe

• If we have confidence in null hypothesis (theoretical considerations or other results) → probably Dr Noble’s universe, probably shouldn’t believe in significant p-value

• If we however think the two treatment have different t, we may refer study to Dr Repeat’s universe, and decide it is relevant even with a p value above 0.05

Summary

• Important topic to provide patient centred care

• Bayes Theorem is fun

• Remember the NNT and in particular the NNH

• Rule out rule in tests

Resources

Further Resources