George TH Ellison PhD DSc

Division of Epidemiology and Biostatistics

Leeds Institute of Genetics, Health and Therapeutics

g.t.h.ellison@leeds.ac.uk

Wendy Harrison (Leeds) and Graham Law (Leeds)

Johannes Textor (Utrecht)

Teaching DAGs to support MBChB students design, analyze and critically appraise

clinical research

DAGs help us distinguish between:

- nonparametric theoretical models of causality; and

- optimal parametric statistical models for testing these

DAGs can be used at every stage of quantitative research:

- optimising the number of variables measured (design)

- optimising adjustment for confounding (analysis)

- evaluating published statistical models (critical appraisal)

Why DAGs?

Why teach statistical modeling in MBChB? Most clinical research/audit uses an observational design Most observational research is poorly/implicitly modelled

What is a DAG (Directed Acyclic Graph)? A type of ‘causal path diagram’ with: unidirectional

(‘causal’) arrows linking variables; and no circular paths

DAGitty.net applies algorithms automatically

Challenges facing the application of DAGs Algorithms are tedious and time-consuming to apply DAGs with more than a handful of variables are complex

Cross-tabulation might help as variables

causes one above

caused by one above

no causal relationship

Comparing three ways of drawing DAGs Three one-hour tutorials using three approaches:

(i) ‘graphical’; (ii) ‘cross-tabulation’; and (iii) ‘relational’

Each approach evaluated based on:

- how many variables were included in the DAG

- mediators/confounders correctly identified*

- student feedback on ease of use and interpretation

All participants were third year MBChB students who had

completed a year-long critical appraisal course

The context was a published paper on an accessible topic:

‘determinants of pregnancy-associated weight gain’

Handouts contained 10, 20 and 30 variables:

Group using ‘graphical’ approach:

Group using ‘cross-tabulation’ approach:

Group using ‘cross-tabulation’ approach:

Group using ‘relational’ approach:

Group using ‘relational’ approach:

What do I (now) think the DAG should be?

Focusing on the ‘relational’

None of the students were able to attempt including more

than 10 variables in their cross-tabulation 86% correctly identified covariates that should have been

classified as ‘mediators’ by Harris et al. 1999... Fewer than 5% correctly identified the only covariate that

is likely to have acted as ‘confounder’ (maternal age) A disproportionate use of ‘competing exposure’ as a

classification for covariates that are likely to have been

‘mediators’ suggests students were reluctant to identify

‘exposure’ as a potential/likely/theoretical cause

Focusing on the ‘relational’ Most students found it ‘Difficult’...

Why?

- understanding DAGs and DAG-related terminology

- “Time consuming” debating/agreeing links and directions

- “...so much depends on variation and opinion”

Summary It is feasible to teach DAGs to MBChB students

Most students are capable of distinguishing between

‘confounders’, ‘mediators’ and ‘competing exposures’

‘Cross-tabulation’ and ‘relational’ were slower to apply

but less likely to result in errors

Suggestions for future development:

- include a quiz to strengthen initial knowledge

- (perhaps) avoid group work (at least initially)

- reward recognition of ‘subjective causality’

- explore an approach that involves removing rather than

including causal paths (‘arcs’)