3.1 Design of Experiments

Ulrich HoenschMAT210

Rocky Mountain CollegeBillings, MT 59102

Anecdotal Evidence and Available Data

Many U.S. government databases containing available data can befound at www.fedstats.gov.

Observation Versus Experiment

Example A study of child care found that “the more time childrenspent in child care from birth to age 41

2 , the more adults tended torate them as less likely to get along with others, as more assertive,more disobedient, and as aggressive.”This is an observational study – the effects of child care areconfounded (mixed up) with characteristics of families who usechild care.

Experimental Units, Subjects, Treatment

The treatments usually correspond to different combinations of theexplanatory variables (also called factors). Experimental designconcerns itself with how the different levels of the factor areassigned to the experimental units.

Example: Assignment to Treatment Groups

Suppose we have 20 subjects and want to assign them to fourtreatments:

1. Placebo (no effective substance).

2. Aspirin only.

3. Drug only.

4. Aspirin and drug.

Randomized Experimental Design

An experiment is randomized if assignment to different treatmentgroups is left to chance. The concept of chance is introduced intoan experiment by randomly assigning subjects to varioustreatments. We use a random digits table to accomplish this.

Example: Assignment to Treatment Groups – RandomizedDesign

We can use the random digits table (Table B) to randomly assignthe treatments to the 20 subjects. We need 5 = 20/4 subjectswith the same treatment. Start e.g. in line 120.

I The first digit is a 3; give treatment 3 to the first subject.

I The next digit is a 5; ignore it.

I The next digit is a 4; give treatment 4 to the second subject.

I The next couple of digits are not in the range 1-4; ignorethem.

I The next digit is a 2; give treatment 2 to the third subject.

Example: Assignment to Treatment Groups – RandomizedDesign

I The next digit is a 3; give treatment 3 to the 4th subject.

I The next digit is a 9; ignore it.

I The next digit is a 4; give treatment 4 to the 5th subject.

Proceed like this until all subjects are assigned treatments. If onetreatment is used five times, skip the treatment number when itsdigit comes up again.

3 4 2 3 4 2 1 4 2 3 4 3 4 3 2 1 1 1 2 1

Example: Assignment to Treatment Groups – Block Design

Suppose we now have 24 subjects, and 12 are female (red), 12 aremale (black). Now, we want to randomly assign treatments toeach group or block (2 blocks, 12 subjects per block, 12/4 = 3subjects for each treatment in each block).

The gender (levels: female, male) is called a blocking variable.

Example: Assignment to Treatment Groups – Block DesignAgain, for each of the two groups with 12 subjects each, we needto randomly assign treatments so that each of the four treatmentsis used three times (12/4 = 3) within each group. Suppose westart with line 133 of the random digits table.

The assignment for the groups is:

4 4 4 1 1 3 3 3 2 1 2 2

4 2 1 4 1 1 3 2 2 3 3 4

Principles of Experimental Design

Experiments conducted in this way are called randomizedcomparative experiments.

Gold Standard in Clinical Experiments, Lack of Realism

The most valuable clinical experiments are:

I comparative: they have one or more treatment groups and acontrol group;

I randomized: assignment to the groups is random;

I double-blinded: neither the subjects themselves nor themedical personnel who worked with them know whichtreatment a subject had received.

Note that the most serious drawback of experiments is lack ofrealism: a clinical or laboratory setting might not translate out“into the field.”

3.2 Sampling Design

Ulrich HoenschMAT210

Rocky Mountain CollegeBillings, MT 59102

Population and Sample

A population survey in which data from every individual of thepopulation is collected is called a census.An important social science survey is the General Social Survey(GSS). One systematic problem with surveys is that ofnonresponse. The GSS has a nonresponse rate of 30% (orequivalently a response rate of 70%).

Undercoverage and Nonresponse

Example For telephone surveys, undercoverage occurs becausepeople without a listed phone number are left out. Also, if phonecalls are made e.g. weekdays during the morning or afternoon,nonresponse occurs because people who work regular hours cannotbe contacted.

Voluntary Response Sample, Sampling Frame

A sampling frame is a compiled list of the individuals you want totake a sample from. Usually these individuals are assignednumbers. The sampling frame should (but in practice does notusually) capture the entire population.

Simple Random Sample

Example We select five of the numbers 01 to 20 at random. If weuse Table B starting, for example, with line 158, and taking twodigits at a time, we obtain:

16, 17, 08, 06, 12.

Stratified Samples

Example To conduct a survey about teacher education, schooldistricts are first divided into urban, suburban, and rural schooldistricts. Then, a SRS is taken from each stratum.

Other Types of Samples

I For a systematic sample, you select every nth individual inthe sampling frame. For example, in a list of persons with IDsfrom 00 to 99, select 00, 05, 10, 15, . . ..

I In a cluster sample, you first divide the population intogroups (or clusters), select some of these clusters at random,and then take a sample of individuals from each cluster. Forexample, in a county, select 5 election precincts at random,and then select 100 voters in each precinct.

Example: Population

Example: Sampling Frame

60

40

20

0

61

41

21

1

62

42

22

2

63

43

23

3

64

44

24

4

65

45

25

5

66

46

26

6

67

47

27

7

68

48

28

8

69

49

29

9

70

50

30

10

71

51

31

11

72

52

32

12

73

53

33

13

74

54

34

14

75

55

35

15

76

56

36

16

77

57

37

17

78

58

38

18

79

59

39

19

Example: Simple Random Sample (SRS) of Size 10 (useLine 130)

69

5 16

48

17

40 53

64

20

19

Example: Stratified Random Sample of Size 5 + 5 (use line115)

61

4 17

76

3222

47

9

73

45