Chapter 5Chapter 5

Experimental Design

Definitions:1) Observational study -

observe outcomes without imposing any treatment

2) Experiment - actively impose some treatment in order to observe the response

I’ve developed a new rabbit food, Hippity Hop.

Rabbit Food

Makes fur soft & shiny!

Increases energy!100% of

daily vitamins & essential

oils!

Can I just make these claims?

What must I do to make these claims?

Who (what) should I test this on?

What do I test?

NO

Do an experiment

Rabbits

The type of food

3)Experimental unit – the single individual (person, animal, plant, etc.) to which the different treatments are assigned

4) Factor – is the explanatory variable5) Level – a specific value for the factor

6) Response variable – what you measure

7) Treatment – a specific experimental condition applied to the units

I plan to test my new rabbit food.

What are my experimental units?

What is my factor?

What is the response variable?

Rabbits

Type of food

How well they grow

Hippity HopI’ll use my pet rabbit, Lucky!

Since Lucky’s coat is shinier & he has more energy, then Hippity Hop is a better rabbit food!

8) Control group – a group that is used to compare the factor against; can be a placebo or the “old” or current item9) Placebo – a “dummy” treatment that can have no physical effect

Old Food Hippity Hop

Now I’ll use Lucky & my friend’s rabbit, Flash. Lucky gets Hippity Hop food & Flash gets the old

rabbit food.

WOW! Lucky is bigger & shinier so Hippity Hop is better!

WOW! Lucky is bigger & shinier so Hippity Hop is better!

Old Food Hippity Hop

The first five rabbits that I catch will get

Hippity Hop food and the remaining

five will get the old food.The Hippity Hop rabbits

have scored higher so it’s the better food!

Old Food Hippity Hop

Number the rabbits from 1 – 10.

Place the numbers in a hat.

1

2

3

45

6

78

9 10

The first five numbers pulled from the hat will be the rabbits that get

Hippity Hop food.

I evaluated the rabbits & found that the rabbits eating Hippity Hop are better than the old food!

The remaining rabbits get the old food.

39857

10) blinding - method used so that units do not know which treatment they are getting

11) double blind - neither the units nor the evaluator know which treatment a subject received

Principles of Experimental Principles of Experimental DesignDesign

• Control of effects of extraneous variables on the response – by comparing treatment groups to a control group (placebo or “old”)

• Replication of the experiment on many subjects to quantify the natural variation in the experiment

• Randomization – the use of chance to assign subjects to treatments

The ONLY way to show cause & effect is with a well-designed, well-controlled experiment!

The ONLY way to show cause & effect is with a well-designed, well-controlled experiment!!The ONLY way to show

cause & effect is with a well-designed, well-controlled experiment!!!

Example 1: A farm-product manufacturer wants to determine if the yield of a crop is different when the soil is treated with three different types of fertilizers. Fifteen similar plots of land are planted with the same type of seed but are fertilized differently. At the end of the growing season, the mean yield from the sample plots is compared.Experimental units?

Factors?

Levels?

Response variable?

How many treatments?

Plots of land

Type of fertilizer

Fertilizer types A, B, & CYield of crop

3

Example 2: A consumer group wants to test cake pans to see which works the best (bakes evenly). It will test aluminum, glass, and plastic pans in both gas and electric ovens.

Experiment units?

Factors?

Levels?

Response variable?

Number of treatments?

Two factors - type of pan & type of oven

Type of pan has 3 levels (aluminum, glass, & plastic & type of oven has 2 levels (electric & gas)

How evenly the cake bakes

6

Cake batter

Example 3: A farm-product manufacturer wants to determine if the yield of a crop is different when the soil is treated with three different types of fertilizers. Fifteen similar plots of land are planted with the same type of seed but are fertilized differently. At the end of the growing season, the mean yield from the sample plots is compared.Why is the same type of seed used on all 15 plots?What are other potential extraneous variables?Does this experiment have a placebo? Explain

It is part of the controls in the experiment.

Type of soil, amount of water, etc.

NO – a placebo is not needed in this experiment

Experiment Experiment DesignsDesigns

•Completely randomized – all experimental units are allocated at random among all treatments

Treatment group 1

Treatment group 2

Treatment group 3

explanatory

variable

response

variable

Ra n

dom

ass

ignm

e nt

Treatment C

Treatment B

Completely randomized design

Randomly assign experimental units

to treatments

Treatment A

Treatment D

•Randomized block – units are blocked into groups and then randomly assigned to treatments

Ran

dom

ass

ignm

entGroup1

Group2

Treatment 1Treatment 2Treatment 3

Treatment 3Treatment 2Treatment 1

explanatory response

varaible varaible

Treatment B

Randomized block design

Randomly assign experimental units

to treatments

Treatment A

Put into homogeneous groups

Treatment A Treatment B

– match up experimental units according to similar characteristics & randomly assign one to one treatment & the other automatically gets the 2nd treatment

– have each unit do both treatments in random order

– the assignment of treatments is dependent

•Matched pairs - a Matched pairs - a special type of block special type of block designdesign

Pair experimental units according to

specific characteristics.

Next, randomly assign one unit from a pair to Treatment A. The other unit gets Treatment B.

Treatment A Treatment B

This is one way to do a matched pairs design – another way is to have the individual unit do both treatments (as in a taste test).

12) Confounding variable – the effect of the confounding variable on the response cannot be separated from the effects of the explanatory variable (factor)

Example 4: An article from USA Today reports the number of victims of violent crimes per 1000 people. 51 victims have never been married, 42 are divorced or separated, 13 are married, and 8 are widowed.Is this an experiment? Why or why not?

What is a potential confounding variable?

Age – younger people are more at risk to be victims of violent crimes

No, no treatment was imposed on people.

Example 5: Four new word-processing programs are to be compared by measuring the speed with which standard tasks can be completed. One hundred volunteers are randomly assigned to one of the four programs and their speeds are measured.

Is this an experiment? Why or why not?What type of design is this?

Factors? Levels?

Response variable?

Yes, a treatment is imposed.

Completely randomizedone factor: word-processing program with 4 levels speed

Example 6: Suppose that the manufacturer wants to test a new fertilizer against the current one on the market. Ten 2-acre plots of land scattered throughout the county are used. Each plot is subdivided into two subplots, one of which is treated with the current fertilizer, and the other with the new fertilizer. Wheat is planted and the crop yields are measured.What type of design is this? Why use this method?

When does randomization occur?

Matched - pairs design

Randomly assigned treatment to first acre of each two-acre plot

Randomization reduces bias by spreading any uncontrolled confounding variables evenly throughout the treatment groups.

Variability is controlled by sample size. Larger samples produce statistics with less variability.

Blocking also helps reduce variability.

Is there another way to reduce

variability?

High bias & low variability

Low bias & low variability

Low bias & high variability

High bias & high variability