Experimental DesignExperimental Design

By Julie-Anne Spatz and Adam By Julie-Anne Spatz and Adam MacLeodMacLeod

Experiment vs. Observational Experiment vs. Observational StudyStudy

An experiment deliberately imposes a An experiment deliberately imposes a treatment on a group of units or subjects treatment on a group of units or subjects in the interest of observing the response. in the interest of observing the response.

VS.VS.

An observational study involves collecting An observational study involves collecting and analyzing data without changing and analyzing data without changing existing conditions or imposing a existing conditions or imposing a treatment.treatment.

The ExperimentThe Experiment

““This study was designed to examine the This study was designed to examine the effects of carbohydrate-electrolyte effects of carbohydrate-electrolyte ingestion on physical and mental function ingestion on physical and mental function associated with the performance of associated with the performance of intermittent high-intensity (IHI) exercise intermittent high-intensity (IHI) exercise similar to many common competitive similar to many common competitive sporting events.” sporting events.”

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11932585&dopt=Abstractcmd=Retrieve&db=PubMed&list_uids=11932585&dopt=Abstract

In Other Words…In Other Words…

Researchers, RS Welsh, JM Davis, JR Researchers, RS Welsh, JM Davis, JR Burke, and HG Williams, examined Burke, and HG Williams, examined athletes physical performance while athletes physical performance while drinking either Gatorade or water. drinking either Gatorade or water. We will use this experiment to We will use this experiment to explain general terms about explain general terms about experimental design.experimental design.

Units or SubjectsUnits or Subjects

Experimental units: the individuals Experimental units: the individuals on which the experiment is doneon which the experiment is done

Subjects: when the units are human Subjects: when the units are human beingsbeings

The subjects used were 60 male The subjects used were 60 male soccer players.soccer players.

TreatmentsTreatments

Treatment: a specific experimental condition applied to Treatment: a specific experimental condition applied to the unitsthe units

Placebo: a dummy treatment, that can have no physical Placebo: a dummy treatment, that can have no physical effecteffect

Factor: a controlled independent variable (called the Factor: a controlled independent variable (called the explanatory variable), whose levels are set by the explanatory variable), whose levels are set by the experimenterexperimenter

Level: each treatments specific valueLevel: each treatments specific value

The factors in this experiment include the amounts of The factors in this experiment include the amounts of Gatorade and water given to each soccer player every Gatorade and water given to each soccer player every fifteen minutes throughout a soccer game.fifteen minutes throughout a soccer game.

Example of levels:Example of levels:Gatorade: 6 ounces, 8 ounces, 10 ouncesGatorade: 6 ounces, 8 ounces, 10 ouncesWater: 6 ounces, 8 ounces, 10 ouncesWater: 6 ounces, 8 ounces, 10 ounces

VariablesVariables

Control Group: the subjects that are not Control Group: the subjects that are not subjected to the treatmentsubjected to the treatment

Experimental Group: the subjects that are Experimental Group: the subjects that are subjected to the treatmentsubjected to the treatment

Control Group: people received only water Control Group: people received only water (30 people)(30 people)

Experimental Group: people received only Experimental Group: people received only Gatorade (30 people)Gatorade (30 people)

RandomizationRandomization

Randomization: the use of chance to Randomization: the use of chance to divide experimental units into groupsdivide experimental units into groups

To conduct a completely randomized To conduct a completely randomized experiment:experiment:

1. use a Table of Random Digits1. use a Table of Random Digits

2. use the calculator to generate 2. use the calculator to generate random numbers random numbers

Table of Random DigitsTable of Random Digits

Calculator TechniqueCalculator Technique

1.1. Stat Edit highlight L1Stat Edit highlight L1

2.2. Math PRB 5:randInt(Math PRB 5:randInt(

3.3. Type in randInt(lower bound, upper Type in randInt(lower bound, upper bound, amount of integers needed)bound, amount of integers needed)

Stratified Random Sampling vs. Stratified Random Sampling vs. Multistage SamplingMultistage Sampling

Stratified Random Sample: an experimenter would first Stratified Random Sample: an experimenter would first divide the population into groups of similar individuals divide the population into groups of similar individuals called a strata, then choose a separate SRS in each stratum called a strata, then choose a separate SRS in each stratum and combine these SRSs to form the full sampleand combine these SRSs to form the full sample

Multistage Sampling: is a sample where the elements are Multistage Sampling: is a sample where the elements are chosen in more than one stagechosen in more than one stage

Ex. 1: If the experimenters wanted to do a stratified random Ex. 1: If the experimenters wanted to do a stratified random sample, they could do this by putting soccer players in to sample, they could do this by putting soccer players in to different stratas by region of the United States and then different stratas by region of the United States and then conduct the experiment.conduct the experiment.

Ex. 2: If the experimenters wanted to do a multistage Ex. 2: If the experimenters wanted to do a multistage sample, they could do this by choosing the soccer players sample, they could do this by choosing the soccer players at the state level, then the city level, then the county level, at the state level, then the city level, then the county level, and then by league. and then by league.

Randomization Technique UsedRandomization Technique Used

The experimenter used a random The experimenter used a random number table to assign each soccer number table to assign each soccer player a number 1 thru 60. Then, player a number 1 thru 60. Then, the people with odd numbers were the people with odd numbers were put into the control group and the put into the control group and the people with even numbers were put people with even numbers were put into the experimental group.into the experimental group.

Single Blind Experiment vs. Double Single Blind Experiment vs. Double Blind ExperimentBlind Experiment

Single Blind Experiment: the experimenters, but not the Single Blind Experiment: the experimenters, but not the subjects know which treatment a subject receivedsubjects know which treatment a subject received

Double Blind Experiment: the experimenters nor the subjects Double Blind Experiment: the experimenters nor the subjects know which treatment a subject receivedknow which treatment a subject received

This experiment was a single blind experiment because the This experiment was a single blind experiment because the experimenter knew what groups the subjects were put in, but experimenter knew what groups the subjects were put in, but the subjects were not aware. To ensure that the subjects the subjects were not aware. To ensure that the subjects could not tell the difference between the Gatorade and water, could not tell the difference between the Gatorade and water, the experimenters put a type of sweetener into the Gatorade. the experimenters put a type of sweetener into the Gatorade. The sweetener only affected the flavor, but not the amount The sweetener only affected the flavor, but not the amount of electrolytes and carbohydrates.of electrolytes and carbohydrates.

If the experiment were to be a double blind experiment, the If the experiment were to be a double blind experiment, the experimenter would have had to of had a separate person experimenter would have had to of had a separate person assign which groups would receive the Gatorade and water, assign which groups would receive the Gatorade and water, without the experimenters knowledge. Then the researcher without the experimenters knowledge. Then the researcher could take over conducting the experiment.could take over conducting the experiment.

Matched PairsMatched Pairs Matched Pairs: experiment that compares two Matched Pairs: experiment that compares two

treatmentstreatments

Ex. 1: If this experimenter were to use matched Ex. 1: If this experimenter were to use matched pairs, every soccer player would drink water pairs, every soccer player would drink water throughout one game, then Gatorade throughout throughout one game, then Gatorade throughout another, and the experimenter would note which another, and the experimenter would note which one yielded higher performance.one yielded higher performance.

Ex. 2: An experiment is conducted to compare the Ex. 2: An experiment is conducted to compare the taste of a new spaghetti sauce with the taste of a taste of a new spaghetti sauce with the taste of a successful sauce. Each of a number of subjects successful sauce. Each of a number of subjects tastes both sauces in random order and notes tastes both sauces in random order and notes which one tastes better.which one tastes better.

Block DesignBlock Design Block: a group of experimental units or subjects that are similar in Block: a group of experimental units or subjects that are similar in

ways that are expected to affect the response to the treatmentsways that are expected to affect the response to the treatments Block Design: the random assignment of units to treatments is Block Design: the random assignment of units to treatments is

carried out separately within each block.carried out separately within each block.

Ex. 1: If the experimenter thought gender was an issue, the Ex. 1: If the experimenter thought gender was an issue, the experimenter would separate the men from the women, and test experimenter would separate the men from the women, and test the effects of both the Gatorade and the water on the two groups. the effects of both the Gatorade and the water on the two groups.

Ex. 2: If the experimenter thought that age was an issue, the Ex. 2: If the experimenter thought that age was an issue, the experimenter would separate the people into four age groups; one experimenter would separate the people into four age groups; one of people aged 1-15, one of people aged 16-30, one of people of people aged 1-15, one of people aged 16-30, one of people aged 31-45, and one of people aged 46-60 and test each group aged 31-45, and one of people aged 46-60 and test each group with the Gatorade and water. with the Gatorade and water.

Ex. 3: If the experimenter was interested in physical condition, the Ex. 3: If the experimenter was interested in physical condition, the experimenter would separate the subjects into two groups of experimenter would separate the subjects into two groups of strong physical condition and weak physical condition, and then strong physical condition and weak physical condition, and then test the Gatorade and water on each group.test the Gatorade and water on each group.

Compare Results

Group 230 Women

Treatment 1Gatorade

Treatment 2Water

Treatment 1Gatorade

Treatment 2Water

Random Allocation

Group 130 Men

Block Design By Gender

Cautions of Experimental DesignCautions of Experimental Design Hidden Bias: if the units or subjects are dealt Hidden Bias: if the units or subjects are dealt

with any different ways, hidden bias can arise with any different ways, hidden bias can arise (unequal conditions introduce bias)(unequal conditions introduce bias)

Ex. If the researcher were to hand a Ex. If the researcher were to hand a soccer player Gatorade and smile and soccer player Gatorade and smile and hand the other soccer player water with a hand the other soccer player water with a frownfrown

Lack of Realism: units or subjects may not Lack of Realism: units or subjects may not believe in the experiment being conducted, believe in the experiment being conducted, making the results of the experiment less making the results of the experiment less reliablereliable

Ex. If the soccer players already Ex. If the soccer players already believed believed that Gatorade produced that Gatorade produced stronger athletic stronger athletic performance than performance than waterwater

Cautions of Experimental Design Cautions of Experimental Design ContinuedContinued

Undercoverage: not including the whole Undercoverage: not including the whole populationpopulation

Ex. If the researcher only included Ex. If the researcher only included soccer players from the East coast.soccer players from the East coast.

Non-response: Some people may choose Non-response: Some people may choose not to respond to a questionnaire or phone not to respond to a questionnaire or phone survey, decreasing the number of survey, decreasing the number of responsesresponses

Ex. If some soccer players turned Ex. If some soccer players turned down down the opportunity to participate in the opportunity to participate in the the experimentexperiment

Confounding VariablesConfounding Variables

Confounding Variable: is a "hidden" Confounding Variable: is a "hidden" variable that affects the variables in variable that affects the variables in question but is not known, and question but is not known, and distorts the resulting data distorts the resulting data

In this experiment, some In this experiment, some confounding variables include diet, confounding variables include diet, athletic ability, age, weather athletic ability, age, weather conditions, gender, etc.conditions, gender, etc.

BibliographyBibliography

http://helios.bto.ed.ac.uk/bto/statistics/tress2.http://helios.bto.ed.ac.uk/bto/statistics/tress2.html#DESIGN%20OF%20EXPERIMENTShtml#DESIGN%20OF%20EXPERIMENTS

http://www.stats.gla.ac.uk/steps/glossary/http://www.stats.gla.ac.uk/steps/glossary/anova.html#treatmentanova.html#treatment

http://www.sytsma.com/phad530/http://www.sytsma.com/phad530/expdesig.htmlexpdesig.html

http://www.stat.yale.edu/Courses/1997-http://www.stat.yale.edu/Courses/1997-98/101/expdes.htm98/101/expdes.htm

http://www.ncbi.nlm.nih.gov/entrez/http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11932cmd=Retrieve&db=PubMed&list_uids=11932585&dopt=Abstract585&dopt=Abstract