ACMS 20340Statistics for Life Sciences

Chapter 8:Designing Experiments

Fisher’s Experiments

Experiment Terminology

individuals (if human) → subjectsexplanatory variable → factors

In conducting an experiment, we try to keep as many variablesconstant while only changing the designated factor variables.

A treatment is a specific combination of factors applied to thesubjects.

The experiment compares the response to a given treatment toother treatments, no treatment, or a fake treatment.

Experiments versus Observational Studies

Experiments are in many ways preferable to observational studies.

I With an experiment we can study the specific treatments weare interested in.

I With an experiment we can study the combined effects ofmany factors simultaneously.

Corn and chicks: an exampleHow do two specific varieties of genetically modified corn andvarious protein-level diets affect the growth of newborn chicks?

The factors areI corn type: Opaque-2, Floury-2, Normal CornI protein level: 12%, 16%, 20%

There are 9 different treatment groups between the two factors.

Control is important

In medical experiments, there are usually at least two groupsconsidered.

One group serves as a baseline against which we compare thetreatment we are interested in. This is called the control group.

We would rather the subjects didn’t know which group they belongto, so a placebo is usually used.

A placebo is a treatment given to the control group which is fakebut otherwise indistinguishable from the treatment given to theexperimental group.

The placebo effect occurs when subjects feel the treatmentworked just from the psychological impact of having a treatment.

Placebo A-Go-Go

Principles of Experimental Design

Here are some basic principles of experimental design.

I Use a control group to keep lurking variables at bay.

I Randomize as much as possible to head off selection bias.

I Use enough subjects to reduce the effect any one subject hason the results.

The goal is to see a difference between the experimental groupslarge enough to not be a chance variation.

By using enough subjects and assigning enough to each group, theaverage response observed in each group becomes more likely to berepresentative the population with that treatment.

An observed effect which is large enough to occur only rarely bychance is called statistically significant.

How do we know what counts as chance variation? We useprobability (chapter 9).

Types of experimental design

These basic principles for experimental design describerandomized comparative experiments.

A randomized comparative experiment uses both the comparison oftwo or more treatments and chance assignment of subjects totreatments.

Three sub-categories of randomized comparative experimentswhich we will discuss are:

I Completely randomized design

I Matched pairs design

I Block design

Completely randomized design

Each subject in a completely randomized design is assigned one ofany of the possible groups at random.

I We always determine in advance how large each group shouldbe.

The chicks experiment is an example of a completely randomizeddesign.

Gastric freezing: an example

Matched Pairs Design I

Completely randomized designs are useful, but they don’t alwaysgive the most precise results.

Sometimes it helps to match the subjects in various ways.

For instance, a matched pairs design combines matching withrandomization.

A matched pairs design compares only two treatments.

We choose pairs of subjects that are as closely matched aspossible.

I E.g., individuals of the same sex, age, weight, etc.

I E.g., genetically related individuals such as twins or animalsborn in the same litter.

Matched Pairs Design II

We assign one treatment to one member of the pair, and the othertreatment to the other.

How do we decide which member gets which treatment?

CHANCE!

Why do we randomize the order of treatment?

The order of the treatment might influence the response of thesubjects, so we randomize to rule out a possible bias.

The pairs in matched pairs designs need not always consist of twodistinct individuals.

We can consider the same subject twice, applying one treatmentafter the other (in a randomized order, of course).

Drugs: an example

Generics are brand-name drugs manufactured by a differentcompany but with identical active ingredients and properties.

Individuals are given either Brand X or its generic version on agiven day so that drug absorption can be measured.

One week later, each individual receives the other drug to measuredrug absorption.

A difference in absorption extent between Brand X and its genericis then calculated for each individual.

Block design

Closely related to a matched pairs design is a block design.

Here’s an example:

Block design

The idea is to test blocks of individuals.

A block is simply a group of individuals that are known before theexperiment to be similar in some way that is expected to affect theresponse to the treatments.

In a block design, the random assignment of individuals totreatments is carried out separately within each block.

Soybeans: an example

We would like to study theeffects of two different typesof tillage and three differenttypes of pesticides on soybeanyields.

Soybeans: an example

To do so, we first divide thearea we are testing intosmaller blocks.

Soybeans: an example

Next, we subdivide each ofthese blocks into six smallerplots.

Soybeans: an example

Lastly, we randomly applyeach of the six treatments toone of the six plots in eachblock.

Some Cautions

According to the text,

“The logic of a randomized comparative experiment depends onour ability to treat all the subjects identically in every way exceptfor the actual treatments being compared.”

We can do better:

“In performing a randomized comparative experiment, we shoulddo our best to ensure that the subjects are treated identically inevery way except for the actual treatments being compared.”

How can we achieve this?

One way to treat the subjects identically in every way except forthe treatments we give to them is by means of a double-blindexperiment.

In a double-blind experiment, neither the subjects nor the peoplewho administer the experiment know which treatment each subjectis receiving.

Why be this careful?

We don’t want the way the experiment is administered to give riseto some bias.

For instance, a patient’s perception of his or her prognosis can beinfluenced by how he or she interacts with a doctor. This mightcreate or reinforce a placebo effect.

Double-blind experiment: an example

A study was conducted todetermine whether the herbalsupplement ginkgo biloba canhelp alleviate tinnitus, aringing noise in the ears thatdoesn’t have an effectivepharmaceutical treatment.

Double-blind experiment: an example

978 healthy adults with tinnitus were matched by age and sex.Within each pair, one individual received the ginkgo bilobatreatment, while the other was given a placebo.

Moreover, the experiment was double-blind, as neither the subjectsnor the investigators knew who was receiving the ginkgo biloba andwho was receiving the placebo, as the tablets wereindistinguishable.

Double-blind experiment: an example

The tablets were distributed to the subjects in coded bottles, andthe code was revealed only after the experiment had concluded andthe data had been collected.

Other issues: Replication

Convincing evidence usually requires more than just awell-designed experiment that takes into account a number ofpotential sources of bias.

It requires that the study be successfully replicated by theinvestigators as well as investigators from different locations.

The environment of an experiment might influence the outcome ofthat experiment.

Other issues: Realism

“The most serious potential weakness of experiments”:

lack of realism

That is, the subjects or treatment or setting of an experiment maynot realistically duplicate the situations we want to study.

For example, saccharin causes bladder cancer in rats, but it turnsout that this cancer is specific to the rat urinary system and isrelated to saccharin consumption at concentrations higher thanwhat is realistic for human consumption.

Other issues: Ethics

Should we consider as acceptable all experiments that areperformed in the name of science, or for the sake of knowledge, orfor the sake of the public’s greater good?

Is it morally right to include a placebo in a study design whenthere are safe and efficient treatments already available?

Should we continue an experiment if we find that one of thetreatments shows early signs of adverse effects? And what if one ofthe treatments shows early signs of clear superiority?

Is it ever acceptable to leave subjects in the dark about thepurpose and expected results of an experiment?

Nope!

Other issues: Ethics

“The interests of the subject must always prevail over the interestsof science and society.”

I The 1964 Helsinki Declaration of the World MedicalAssociation

All studies funded by the U.S. government must now obey thefollowing principles:

I Those carrying out the study must have an institutionalreview board to protect subjects from possible harm.

I All subjects must give their informed consent in writing beforedata is collected.

I Subjects must be informed about the nature of a study andany risk of harm it may bring.

I All individual data must be kept confidential.

Ethics: Tuskeegee syphilis study

“For 40 years, the U.S. Public Health Services has conducted astudy in which human guinea pigs, not given proper treatment,have died of syphilis and its side effects. The study was conductedto determine from autopsies what the disease does to the humanbody.”

Ethics: Tuskeegee syphilis study

Beginning in 1932, nearly 400 poor black men with syphilis wereenrolled in the study.

Originally intended to be run for six months, the study lasted untila whistle-blower turned to the press in 1972.

The subjects of the study were never told they had syphilis, andwere also never told that they were part of a medical experiment.

Penicillin was known to be a highly effective cure in 1943, andbecame widely available by 1947.

It was not given to any of the subjects in the study.