352 C hap te r Ten

11 .0 • •

• • 10.5

Y 10.0 •

9.5

9.0 L...-___ ----'-____ ---'-____ ---'-____ --'

0 .0 0.5 1.0

X

FIGURE 10.23 Interpreting r = 0 for curvilinear data.

1.5 2 .0

Establishing causation requires solid scientific understanding. Causation cannot be "proven" by statistics alone. Some statistical techniques, such as path analysis, can help determine if the correlations between a number of variables are consistent with causal assumptions. However, these methods are beyond the scope of this book.

Designed Experiments Designed experiments play an important role in quality improvement. While the confi­dence intervals and hypothesis tests previously discussed are limited to rather simple comparisons between one sample and requirements or between two samples, the designed experiments will use ANOVA (analysis of variance) techniques to partition the variation in a response amongst the potential sources of variation. This section will introduce the basic concepts involved and it will contrast the statistically designed experiment with the "one factor at a time" (OFAT) approach used traditionally. Also briefly discussed are the concepts involved in Taguchi methods, statistical methods named after their creator, Dr. Genichi Taguchi.

The Traditional Approach versus Statistically Designed Experiments The traditional approach, which most of us learned in high school science class, is to hold all factors constant except one. When this approach is used we can be sure that the variation is due to a cause and effect relationship or so we are told. However, this approach suffers from a number of problems:

• It usually isn't possible to hold all other variables constant.

• There is no way to account for the effect of joint variation of independent vari­ables, such as interaction.

• There is no way to account for experimental error, including measurement variation.

Analyze Phase 353

The statistically designed experiment usually involves varying two or more vari­ables simultaneously and obtaining multiple measurements under the same experi­mental conditions. The advantage of the statistical approach is threefold:

1. Interactions can be detected and measured. Failure to detect interactions is a major flaw in the OFAT approach.

2. Each value does the work of several values. A properly designed experiment allows you to use the same observation to estimate several different effects. This translates directly to cost savings when using the statistical approach.

3. Experimental error is quantified and used to determine the confidence the experimenter has in his conclusions.

Terminology Much of the early work on the design of experiments involved agricultural studies. The language of experimental design still reflects these origins. The experimental area was literally a piece of ground. A block was a smaller piece of ground with fairly uniform properties. A plot was smaller still and it served as the basic unit of the design. As the plot was planted, fertilized and harvested, it could be split simply by drawing a line. A treatment was actually a treatment, such as the application of fertilizer. Unfortunately for the Six Sigma analyst, these terms are still part of the language of experiments. The analyst must do his or her best to understand quality improvement experimenting using these terms. Natrella (1963) recommends the following:

Experimental area can be thought of as the scope of the planned experiment. For us, a block can be a group of results from a particular operator, or from a particular machine, or on a particular day-any planned natural grouping which should serve to make results from one block more alike than results from different blocks. For us, a treatment is the factor being investigated (material, environmental condition, etc.) in a single fac­tor experiment. In factorial experiments (where several variables are being investigated at the same time) we speak of a treatment combination and we mean the prescribed levels of the factors to be applied to an experimental unit. For us, a yield is a measured result and, happily enough, in chemistry it will sometimes be a yield.

Definitions A designed experiment is an experiment where one or more factors, called independent variables, believed to have an effect on the experimental outcome are identified and manipulated according to a predetermined plan. Data collected from a designed exper­iment can be analyzed statistically to determine the effect of the independent variables, or combinations of more than one independent variable. An experimental plan must also include provisions for dealing with extraneous variables, that is, variables not explicitly identified as independent variables.

Response variable-The variable being investigated, also called the dependent vari­able, sometimes called simply response.

Primary variables-The controllable variables believed most likely to have an effect. These may be quantitative, such as temperature, pressure, or speed, or they may be qualitative such as vendor, production method, and operator.

Background variables-Variables, identified by the designers of the experiment, which may have an effect but either cannot or should not be deliberately manipulated

354 C hap te r Ten

Interaction present

B-High

/ /

/

A-Low

/

/

B-Low

/

A-High

FIGURE 10.24 Illustration of interaction.

No interaction present

B-High

. B-Low

A-Low A-High

or held constant. The effect of background variables can contaminate primary variable effects unless they are properly handled. The most common method of handling back­ground variables is blocking (blocking is described later in this chapter).

Experimental error-In any given experimental situation, a great many variables may be potential sources of variation. So many, in fact, that no experiment could be designed that deals with every possible source of variation explicitly. Those variables that are not considered explicitly are analogous to common causes of variation. They represent the "noise level" of the process and their effects are kept from contaminating the primary variable effects by randomization. Randomization is a term meant to describe a procedure that assigns test units to test conditions in such a way that any given unit has an equal probability of being processed under a given set of test conditions.

Interaction-A condition where the effect of one factor depends on the level of another factor. Interaction is illustrated in Fig. 10.24.

Design Characteristics Good experiments don't just happen, they are a result of careful planning. A good experimental plan depends on (Natrella 1963):

• The purpose of the experiment

• Physical restrictions on the process of taking measurements

• Restrictions imposed by limitations of time, money, material, and personnel

The analyst must explain clearly why the experiment is being done, why the experi­mental treatments were selected, and how the completed experiment will accomplish the stated objectives. The experimental plan should be in writing and it should be endorsed by all key participants. The plan will include a statement of the objectives of the experi­ment, the experimental treatments to be applied, the size of the experiment, the time frame, and a brief discussion of the methods to be used to analyze the results. Two con­cepts are of particular interest to the Six Sigma analyst: replication and randomization.

Replication-The collection of more than one observation for the same set of exper­imental conditions. Replication allows the experimenter to estimate experimental error. If variation exists when all experimental conditions are held constant, the cause must be something other than the variables being controlled by the experimenter. Experimental error can be estimated without replicating the entire experiment. If a process has been

Analyze Phase 355

in statistical control for a period of time, experimental error can be estimated from the control chart. Replication also serves to decrease bias due to uncontrolled factors.

Randomization-In order to eliminate bias from the experiment, variables not specifi­cally controlled as factors should be randomized. This means that allocations of specimens to treatments should be made using some mechanical method of randomization, such as a random numbers table. Randomization also ensures valid estimates of experimental error.

Types of Design Experiments can be designed to meet a wide variety of experimental objectives. A few of the more common types of experimental designs are defined here.

Fixed-effects model: An experimental model where all possible factor levels are studied. For example, if there are three different materials, all three are included in the experiment.

Random-effects model: An experimental model where the levels of factors evalu­ated by the experiment represent a sample of all possible levels. For example, if we have three different materials but only use two materials in the experiment.

Mixed model: An experimental model with both fixed and random effects. Completely randomized design: An experimental plan where the order in which

the experiment is performed is completely random, for example,

Level I

Test Sequence Number

A 7,1,5

B 2,3,6

C 8,4

Randomized-block design: An experimental design is one where the experimental observations are divided into "blocks" according to some criterion. The blocks are filled sequentially, but the order within the block is filled randomly. For example, assume we are conducting a painting test with different materials, material A and material B. We have four test pieces of each materiaL Ideally we would like to clean all of the pieces at the same time to ensure that the cleaning process doesn't have an effect on our results; but what if our test requires that we use a cleaning tank that cleans two test pieces at a time? The tank load then becomes a "blocking factor." We will have four blocks, which might look like this:

Material Tank Load Test Piece Number

A 1 7

B 1

B 2 5

A 2

B 3 3

A 6

B 4 4

A 8

356 C hap te r Ten

Since each material appears exactly once per cleaning tank load we say the design is balanced. The material totals or averages can be compared directly. The reader should be aware that statistical designs exist to handle more complicated "unbalanced designs."

Latin-square designs: Designs where each treatment appears once and only once in each row and column. A Latin-square plan is useful when it is necessary or desirable to allow for two specific sources of nonhomogeneity in the conditions affecting test results. Such designs were originally applied in agricultural experimentation when the two sources of nonhomogeneity were the two directions on the field and the" square" was literally a square piece of ground. Its usage has been extended to many other applica­tions where there are two sources of nonhomogeneity that may affect experimental results-for example, machines, positions, operators, runs, and days. A third variable is then associated with the other two in a prescribed fashion. The use of Latin squares is restricted by two conditions:

1. The number of rows, columns and treatments must all be the same.

2. There must be no interactions between row and column factors.

Natrella (1963, pp. 13-30) provides the following example of a Latin square. Suppose we wish to compare four materials with regard to their wearing qualities. Suppose fur­ther that we have a wear-testing machine which can handle four samples simultane­ously. Two sources of inhomogeneity might be the variations from run to run, and the variation among the four positions on the wear machine. In this situation, a 4 x 4 Latin square will enable us to allow for both sources of inhomogeneity if we can make four runs. The Latin square plan is as in Fig. 10.25 (the four materials are labeled A, B, C, D).

The procedure to be followed in using a given Latin square is as follows:

1. Permute the columns at random

2. Permute the rows at random

3. Assign letters randomly to the treatments

One-Factor ANOVA The following example will be used to illustrate the interpretation of a single factor analysis of variance. With the widespread availability of computers, few people actu­ally perform such complex calculations by hand. The analysis below was performed using Microsoft Excel. Commonly used statistical methods such as regression and ANOVA are included in most high-end spreadsheets.

The coded results in Table 10.3 were obtained from a single factor, completely ran­domized experiment, in which the production outputs of three machines (A, B, and C) were to be compared.

Position number

Run (1 ) (2) (3) (4)

1 A B C D

2 B C D A

3 C D A B

4 D A B C

FIGURE 10.25 A 4 x 4 Latin square.