CAMARADES: Bringing evidence to translational medicine

Stanford visit

8 Feb 2011

CAMARADES: Bringing evidence to translational medicine

Starting points

• As a community we thought we knew how to do clinical trials, but when we looked we found lots of confounding by bias

• As a community we think we know how to do animal experiment in stroke, but when we looked we found lots of confounding by bias

• As a community we think we know how to do in vitro biological research

CAMARADES: Bringing evidence to translational medicine

Caveats

• We have looked in domains where we know there is translational failure – it might just be stroke researchers

• Most of our evidence is observational not experimental

• We are not experts in other fields, and experts may know best

• There are broader issues of how we know what we know – hierarchies of evidence

CAMARADES: Bringing evidence to translational medicine

Not just stroke researchers

• EAE modelling of MS• Dopamine agonists in PD• Receptor Ligand Binding Assays

CAMARADES: Bringing evidence to translational medicine

Systematic review in EAE

CAMARADES: Bringing evidence to translational medicine

EAE modelling of MS

CAMARADES: Bringing evidence to translational medicine

Systematic review in PD

CAMARADES: Bringing evidence to translational medicine

Dopamine Agonists in PD

Blinded outcome assessment Composite quality

CAMARADES: Bringing evidence to translational medicine

Receptor Ligand Binding Assays

McPartland et al 2007

CAMARADES: Bringing evidence to translational medicine

Assertions

• Poor study quality is highly prevalent in laboratory research

• The effectiveness of laboratory research could be substantially improved if study quality were improved

• Improved study quality would reduce the risks of translational failure

CAMARADES: Bringing evidence to translational medicine

Is it possible to study this systematically?

• Cannabinoid receptors• NOS transgenics in stroke• Pathophysiology of EAE

CAMARADES: Bringing evidence to translational medicine

NOS transgenics in focal ischaemia

Grey bar represents the 95% confidence limits of the pooled effect size, bar width reflects the number of animals contributing to each analysis and vertical error bars represent 95% confidence intervals.

nexp nani

m

es 95% CI

eNOS -/- 16 308 -17.3 -22.3 to -12.4

iNOS +/-19 320

10.9 8.1 to 13.6

iNOS -/- 17.9 12.2 to 23.5

nNOS -/- 23 402 24.0 13.0 to 35.0

CAMARADES: Bringing evidence to translational medicine

TH1 and TH17 cells in EAE

• Concordance is observed when a decreased number of circulating Th1 or Th17 cells is associated with decreased EAE severity, and when an increase in numbers of circulating Th1 or Th17 cells is associated with increased EAE severity. (a),(b) Concordance (green) and discordance (red) between change in the number of circulating (a) Th1 or (b) Th17 cells and change in the severity of observed EAE. Concordance is greater for the Th17-EAE pathway. (c) “Meta-blot” showing this relationship. Dots on the line represent concordance, those off the line show discordance. The size of each dot reflects the number of studies contributing to that point. Dot colour reflects the presence of factors which might bias the results of that group of experiments (Red: bias likely; Green: bias unlikely). The width of the arrow reflects the number of experiments contributing data to the analysis.

CAMARADES: Bringing evidence to translational medicine

Contrasting strategies

• Random sampling “brute force” attack

• Focused study of particular area of biology

CAMARADES: Bringing evidence to translational medicine

Random Sampling

• Decide what we want to look for• Sample at random (?PMID) against

inclusion and exclusion criteria• Categorise publications and experiments• Extract high level quality metrics• Extract data where possible (quantatative/

qualitative)• Seek evidence for influence of quality

metrics on outcome• Seek evidence for publication bias

CAMARADES: Bringing evidence to translational medicine

..but

• no accepted taxonomy of laboratory research

• would require iterative approach• PMID selection will lead to

incomplete ascertainment

CAMARADES: Bringing evidence to translational medicine

Focussed studyThe pathophysiology of NMDA

activation

• Start with the NMDA receptor as a “seed node”

• Systematically identify evidence for things which modulate NMDA-R activity

• Systematically identify evidence for things which are modulated by NMDA-R activity

• Weigh the evidence (taxonomy, quality, magnitude)

• Identify each “thing” as a secondary node and iterate until a pathway is created

CAMARADES: Bringing evidence to translational medicine

…but

• Number of nodes will increase geometrically with each iteration

• Can the strength of evidence be condensed into only a few dimensions?

CAMARADES: Bringing evidence to translational medicine

Opportunities

• Funding application in progress (7 years, statistician, post docs, multi-centre animal studies)

• IUPHAR database project• AHRQ “Mechanistic Frameworks”

project