Chapter 6 Experiments

Observational Studies- passive data collection. We observe, record, or measure, but we don’t interfere.

Experiments– involve active data production. Actively intervene by imposing some treatment in order to see what happens.

Subjects- individuals studied in an experiment.

Treatment- any specific experimental condition applied to the subjects. If an experiment has several explanatory variables, a treatment is a combination of specific values of these variables.

Do students who take a course on the Web learn well as those who take the same course in a traditional classroom?

The best way to find out is to assign some students to the classroom and others the Web. That’s an experiment.

If students choose for themselves whether to enroll in a classroom or in an online version of a course. This is not an experiment.

This study simply measured their learning.

Let’s say the students who chose the online course had an average score on a pretest of 40.70, where as the students who chose the classroom setting had a pretest score of 27.64.

It’s hard to compare the two when the online students already have a head start.

The effect of the online versus the classroom is mixed up with influences lurking in the background.

Online vs. Classroom

Lurking Variable- A variable that has an important effect on the relationship in a study but is not one of the explanatory variables studied.

Confounding—two variables are confounded when their effects on a response variable cannot be distinguished from each other. The confounding variable may be either explanatory variable or lurking variables.

In the 1990s, researchers measured the number television sets per person x and the life expectancy y for the world’s nations. There was a high positive correlation: nations with many TV sets had higher life expectancies.

Does television extend life?

What causes obesity in children? Inheritance from parents, overeating, lack of physical activity, and too much television have all been named as explanatory variables.

The results of a study of Mexican American girls aged 9 to 12 years are typical. Measure BMI, a measure of weight relative to height, for both the girls and their mothers. Also measure hours of television, minutes of physical activity and intake of several kinds of food.

Results: the girls’ BMIs were weakly correlated (r=-.18) with physical activity and also with diet and television. The strongest correlation(r=.506) was between the BMI of the daughters and the BMI of their mothers.

Obesity in Mothers and Daughters

Clinical trials- Experiments that study the effectiveness of medical treatments on actual patients.

Placebo- dummy treatment with no active ingredients.

Placebo effect- many patients respond favorably to any treatment, even a placebo.

Ulcers seem to accompany the stress of modern life. “Gastric freezing” is a clever treatment for stomach ulcers. The patient swallows a deflated balloon with tubes attached: then a refrigerated solution is pumped through the balloon for an hour. The idea is that cooling the stomach will reduce its production of acid and so relieve ulcers.

A report claimed that gastric freezing did relieve ulcer pain.

Imposed treatment measure response Gastric Freezing Reduced Pain?

A second clinical trail was done several years later, divided ulcer patients into two groups. One group was treated by gastric freezing as before, the other group received a placebo treatment in which the solution in the balloon was at body temperature.

Results: 34% of the 82 patients in the treatment group improved. But so did 38% of the 78 patients in the placebo group.

A standard experiment on “avoidance conditioning” in rats is as follows. The apparatus is a cage with two compartments separated by a door. One compartment is painted white and is lighted, while the other is black and unlighted. A rat is placed in the black compartment, the door is opened, and a bell rings. If the rat is still in the black compartment after 5 seconds, it is shocked through the metal door until it moves to the white compartment. Since the rat prefers the dark, it will not move to the white compartment until it has been conditioned to avoid the shock. This process is repeated to observe how many trials are required to condition the rat.

What are the subjects? What is the treatment? What is the response variable?

A Basic Experiment

Sickle-cell anemia is an inherited disorder of the red blood cells. It can cause severe pain and many complications. The National Institute of Health carried out a clinical trial of the drug hydroxyurea for treatment of sickle-cell anemia. The subjects were 299 adult patients who had had at least three episodes of pain in the previous year. Half of the subjects received hydroxyurea and the other half received a placebo that looked and tasted the same. All subjects were treated the same, therefore any lurking variables affected both groups equally and should not have caused any differences between their average responses.

Sickle-cell Anemia

The two groups must be similar in all respects before they start taking the medication.

Best way to avoid bias, you should draw a SRS of the subjects. One way is to use Table B.

Things to consider:

Randomized comparative experiment– one that compares just two treatments.

Control Group- the placebo group. This allows us to control the effects of lurking variables.

Control group does not necessarily have to receive a placebo. Clinical trials often compare new treatments with a treatment that is already on the market.

Random assignment produces groups of subjects that should be similar in all respects before we apply the treatments.

A proper comparative design ensures that influences other than the experimental treatments operate equally on all groups.

Therefore, differences in the response variable must be due to the effects of the treatments.

The logic of experimental Design

1. Control – the effects of lurking variables on the response.

2. Randomize-use impersonal chance to assign subjects to treatments.

3. Use enough subjects in each group to reduce chance variation in the results.

Principles of Experimental Design

Can aspirin help prevent heart attacks? The Physicians’ Health Study, a large medical experiment involving 22,071 male physicians, attempted to answer this question. One randomly selected group of 11.037 physicians took an aspirin every second day, while the rest took a placebo. After several years the study found that subjects in the aspirin group had significantly fewer heart attacks than subjects in the placebo group.

Identify the subject Identify the explanatory variable Identify the response variable

Aspirin and heart attacks

It is possible to use a computer to make telephone calls over the internet. How will the cost affect the behavior of users of this service? You will offer the service to all 200 rooms in a college dorm. Some rooms will pay a flat rate. Others will pay higher rates at peak periods and very low rates off-peak. You are interested in the amount and time of use.

Outline the design of an experiment to study the effect of rate structure.

Internet phone calls

An observed effect so large that it would rarely occur by chance is called Statistically Significant. A difference so large would almost never happen just by chance.

Statistically significant

A randomized comparative experiment examines whether a calcium supplement in the diet reduces the blood pressure of healthy men. The subjects receive either a calcium supplement or a placebo for 12 weeks. The researchers conclude that “the blood pressure of the calcium group was significantly lower than that of the placebo group.”

Explain what statistically significant means in the context of this experiment.

Statistically Significance

Does regular church attendance lengthen people’s lives?

Do doctors discriminate against women in treating heart disease?

Does talking on a cell phone while driving increase the risk of having an accident?

We can’t randomly assign people to attend church, or assign heart disease patients to be men or women, or require driving while using a cell phone.

The best data we have about these and many other cause and effect questions come from observational studies.

How to live with observational Studies.

Good studies can be comparative. We can compare random samples of people

who attend or don’t attend religious service regularly.

We compare how doctors treat men and women patients.

We can compare drivers on cell phones with the same drivers when they are not on the phone.

To see the effects of taking a painkiller during pregnancy, we compare women who did so with women who did not. From a large pool of women who did not take the drug, we select individuals who match the drug group in age, education, number of children and other lurking variables. We now have two groups that are similar in all these ways, so that these lurking variables should not affect our comparison of the groups.

Painkillers and pregnancy

Comparison does not eliminate confounding.

A good comparative study measures and

adjusts for confounding variables.

Observational studies suggest that children who watch many hours of television get lower grades in school and are more likely to commit crimes than those who watch less TV. Explain why these studies do not show that watching TV causes these harmful effects. Suggest some lurking variables that may be confounded with heavy TV viewing.

TV harms children

Randomized Comparative Experiment- An experiment that compares just two treatments.

This also assumes all the subjects are treated alike except for the treatments that the experiment is designed to compare.

Any other unequal treatment can cause bias.

Does a new breakfast cereal provide good nutrition? To find out, compare the weight gains of young rats fed the new product and rats fed a standard diet. The rats are randomly assigned to diets and are housed in large racks of cages.

It turns out that rats in upper cages grow a bit faster than rats in bottom cages. If the experimenters put rats fed the new product at the top and those fed the standard diet below.

Would the experiment be bias? What could be a solution?

Mice, Rats, and Rabbits

Medical studies must take special care to show that a new treatment is not just a placebo.

Part of EQUAL TREATMENT FOR ALL SUBJECTS is to be sure that the placebo effect operates on all subjects.

The placebo effect is so strong, it would be foolish to tell subjects in a medical experiment whether they are receiving a new drug or a placebo. Knowing they are getting “just a placebo” might weaken the placebo effect and bias the experiment.

Some bare facts!!!

There are times that even doctors or the medical personnel should not know which treatment the subjects are getting.

Doctors’ expectations change how they interact with patients and even the way they diagnose a condition, if they were to know which treatment the subjects are receiving.

Double-blind experiment- neither the subjects nor the people who work with them know which treatment each subject is receiving.

Only the study’s statistician knows which treatment the subject is receiving.

Double-Blind Experiment

Refusal to participate is a serious problem for medical experiments on treatments for serious diseases

Minorities, women, the poor and the elderly have long been underrepresented in clinical trials. The law now requires representation of women and minorities. Refusal still remains a problem with minorities.

Why do you think there is still refusal? How can we remedy this problem?

Refusals, non-adherers and dropouts

Non-adherers- subjects who participate but don’t follow the experimental treatment.

Can this cause bias?

Dropouts-experiments that continue over an extended period of time suffer from when subject begin the experiment but do not complete it.

Can this cause bias?

Orlistat is a drug that may help reduce obesity by preventing absorption of fat from the foods we eat. The drug was compared with a placebo in a double blind randomized trial.

Researchers started with 1187 obese subjects. They gave a placebo to all the subjects for 4 weeks and dropped the subjects who did not take the pill regularly. There were 892 subjects left.

Researchers randomized the subjects to orlistat or a placebo, along with a weight-loss diet. After a year 576 subjects were still participating.

On the average, the Orlistat group lost 3.15 kg more than the placebo group.

The study kept going for another year, at the end of the second year, 403 subjects were left. Orlistat again beat the placebo, reducing weight on the average of 2.25 kg.

Can we trust the results?

Dropouts in a medical study

We must be sure our findings are statistically significant. They are too strong to often occur by chance.

We need to make sure the treatment, subjects or the environment of our experiment be realistic.

Can we generalize our conclusions?

A psychologist wants to study the effects of failure and frustration on the relationships among members of a work team. She forms a team of students, brings them to the laboratory, and has them play a game that requires teamwork. The game is rigged so that they lose regularly. The psychologist observes the students through a one-way window, and notes the changes in their behavior during an evening of game playing.

Studying frustration

Cars sold in the United States since 1986 have been required to have a high centered brake light in addition to the usual two brake lights at the rear of the vehicle.

This safety requirement was justified by randomized comparative experiments with fleets of rental and business cars. The experiments showed that the third brake light reduced rear-end collisions by as much as 50%.

After almost a decade of actual use, The Insurance Institute found only 5% reduction in rear-end collisions, helpful but much less than the experiments predicted.

What happened??

Center Brake lights

1. Divide the subjects at random into as many groups as there are treatments.

2. Then apply each treatment to one of the groups.

This is considered Completely Randomized Designs.

Experimental Design in the Real World

A fabrics researchers is studying the durability of a fabric under repeated washing. Because durability may depend on the water temperature and the type of cleansing agent used, the researchers decides to investigate the effect of these two explanatory variables.

Variable A: water temperature with three levels: hot (1450F), warm (1000F) and cold(500F).

Variable B: cleansing agent with three levels: regular Tide, low-phosphate Tide, and Ivory Liquid.

The treatment consists of washing a piece of fabric 50 times in a home automatic washer.

Durable Fabric

Interaction- combination effect between two explanatory variables.

Ex: Washing agent and water temperature Ex: Drugs interaction with alcohol.

Matched Pairs Design *compares just two treatments. * choose pairs of subjects that are as

closely matched as possible. * Randomly assign the two treatments to

the subjects in each pair. *sometimes each “pair” is just one subject

who gets both treatments one after the other.

*Randomize the order of the treatments that the subjects receives.

Pepsi wanted to demonstrate that Coke drinkers prefer Pepsi when they taste both colas blind. People who said they were Coke drinkers, tasted both colas from glasses without brand markings and said which they liked better.

This is an example of … The subjects were…

Coke versus Pepsi

Block– a group of experimental subjects that are know before the experiment to be similar in some way that is expected to affect the response to the treatments.

Block Design– the random assignment of subjects to treatments is carried out separately within each block.

This is a form of Control. They control the effects of some outside variables by bringing those variables into the experiment to form the blocks.

Block Design

Women and men respond differently to advertising. An experiment to compare the effectiveness of 3 television commercials for the same product will want to look separately at the reactions of men and women, as well as assess the overall response to the ads.

Men, Women and advertising