Presented by Presented by John Quackenbush, Ph.D.John Quackenbush, Ph.D.

at the at the June 10, 2003June 10, 2003meeting of themeeting of the

Pharmacology Toxicology SubcommitteePharmacology Toxicology Subcommitteeof theof the

Advisory Committee for Pharmaceutical ScienceAdvisory Committee for Pharmaceutical Science

The Experimental Design dictates a good deal of what The Experimental Design dictates a good deal of what you can do with the datayou can do with the data

Good normalization and processing reflects the Good normalization and processing reflects the experimental designexperimental design

The design also facilitates certain comparisons The design also facilitates certain comparisons between samples and provides the statistical power between samples and provides the statistical power you need for assigning confidence limits to individual you need for assigning confidence limits to individual measurementsmeasurements

The design must reflect experimental realityThe design must reflect experimental reality

The most straight-forward designs compare The most straight-forward designs compare expression in two classes of samples to look for expression in two classes of samples to look for patterns that distinguish them.patterns that distinguish them.

The Experimental Design

Sample Pairing for Co-Hybridization ExperimentsSample Pairing for Co-Hybridization Experiments

Direct Comparison with Dye Swap:Direct Comparison with Dye Swap:

AA11

AA11

BB11

BB11

AA22

AA22

BB22

BB22

AA11 BB11 AA22 BB22

Balanced Block Design:Balanced Block Design:

AA33

AA33

BB33

BB33

AA33 BB33 AA44 BB44

AA44

AA44

BB44

BB44

• RNA sample is RNA sample is notnot limiting (e.g. plenty of sample) limiting (e.g. plenty of sample)• Flip dyes account for any gene-dye effectsFlip dyes account for any gene-dye effects

• RNA sample is limitingRNA sample is limiting• Balanced blocking accounts for any gene-dye effectsBalanced blocking accounts for any gene-dye effects

Multiple Sample PairingsMultiple Sample PairingsReference Design (Indirect Reference Design (Indirect Comparison):Comparison):

AA

CC

BB

DD

AA BB CC

RR

DD

AA

CC

BB

EE

FF

DD

Loop Design:Loop Design:

• More than two samples are comparedMore than two samples are compared(e.g. tumor classification, time course)(e.g. tumor classification, time course)

• Flip dyes are not necessary but can be Flip dyes are not necessary but can be done to increase precisiondone to increase precision

• Ratio values are inferred (indirect)Ratio values are inferred (indirect)• Suited for cluster analysis – need common Suited for cluster analysis – need common

referencereference

Loop designLoop design Can provide direct measurementsCan provide direct measurements Give more data on each experimental sample with Give more data on each experimental sample with

the same number of hybsthe same number of hybs Require more RNA per sampleRequire more RNA per sample Can “unwind” with a bad sample or for a gene Can “unwind” with a bad sample or for a gene

with bad datawith bad data

Reference designReference design Easily extensibleEasily extensible Simple interpretation of all resultsSimple interpretation of all results Requires less RNA per sampleRequires less RNA per sample Less sensitive to bad RNA samples and bad arrayLess sensitive to bad RNA samples and bad array

elementselements

Loop vs. Reference DesignsLoop vs. Reference Designs

Parental - stressedParental - stressed

Parental - unstressedParental - unstressed

Derived - stressedDerived - stressed

Derived - unstressedDerived - unstressed

EnvironmentEnvironment

GenotypeGenotype

One Possible Experimental ParadigmOne Possible Experimental Paradigm:: Examining Genotype, Phenotype, and EnvironmentExamining Genotype, Phenotype, and Environment

Reference SampleReference Sample

Assay VariationAssay Variation

Biological replicas are more informative than Biological replicas are more informative than correlated replicas (independent RNA, independent correlated replicas (independent RNA, independent slides)slides)

More replicas are better – higher statistical powerMore replicas are better – higher statistical power

For loops, hybridizations of individual samples should For loops, hybridizations of individual samples should be “balanced” (as many Cy3 as Cy5 labelings)be “balanced” (as many Cy3 as Cy5 labelings)

Self-self hybs add data on reproducibility and can be Self-self hybs add data on reproducibility and can be used to produce error modelsused to produce error models

At a minimum, should use dye swap replicates to At a minimum, should use dye swap replicates to compensate for any dye biases in labeling or detectioncompensate for any dye biases in labeling or detection

Basic Design Principles

How Many Replicates?How Many Replicates?

Where zWhere z/2/2 and z and z are normal percentile values at significance are normal percentile values at significance level level and false negative rate and false negative rate ; parameter ; parameter represents the represents the minimum detectable logminimum detectable log22 ratio; and ratio; and represents the SD of log represents the SD of log ratio values.ratio values.

For For = 0.001 and = 0.001 and = 0.05, then z = 0.05, then z/2/2 = -3.29 and z = -3.29 and z = -1.65. = -1.65.

Assume Assume = 1.0 (2-fold change) and = 1.0 (2-fold change) and = 0.25, = 0.25,

Therefore n = 12 samples (6 query and 6 control).Therefore n = 12 samples (6 query and 6 control).

(Simon et al., (Simon et al., Genetic EpidemiologyGenetic Epidemiology 23: 21-36, 2002) 23: 21-36, 2002)

n = [4(zn = [4(z/2/2 + z + z))22] / [(] / [(/1.4/1.4))22]]