DOE - Blocking out the Nuisance - Part One.pdf

Published on Quality Digest (http://www.qualitydigest.com)

Home > Content

Blocking Out the Nuisance, Part One

By: Steven Ouellette

You all know what happens when you make assumptions

No, I am not writing about going, “Lalalalala! I can't hear you!” when someone is trying to tell you bad news. In the last couple of articles, we have been exploring how to properly perform research in industry. In this one, we will take a look at how you can handle variables in the real world that just get in the way, but still need to be dealt with. And there is a Really Important Thing partway through this article that will save you a lot of money.

ADVERTISEMENT

So far we have done a good chunk of the research planning. When last we interfaced, we were planning an experiment to see how manipulating different process variables (the “treatments”) would affect the expansion of a complex polymer. But since it is the real world we are working in, we had also identified humidity as a variable that could affect expansion, a variable over which we had little control.

of 7Blocking Out the Nuisance, Part One

9/29/2011http://www.qualitydigest.com/print/16185

Figure 1: The research design process from Design of Experiments in Quality Engineering, [1] by Jeffrey T. Luftig and Victoria S. Jordan (McGraw-Hill, 1998) Click here for larger image. [2]

Now the question was, “How are we going to handle humidity?” We have no plans to air-condition or dehumidify the factory floor, so it is not like we can reduce the humidity. It is a nuisance—it could affect things, but we really can’t control it.

As you might guess, this is not at all an unusual occurrence in the real world of business. So let’s talk about how to handle a situation like that.

Really important thing

So here is one of the things I really wanted to tell you about in the context of research design. Knowing this will save you money and be a big advantage over other researchers, as I promised in my first article on this subject.

There are three ways to control for nuisance variables. The worst one is to make assumptions. You just assume that the known factors won’t have an effect, or if they do it’s a small one, or if it’s not a small one, that it’s just an extrapolation of what you saw in the study.

This is a bad idea.

These known factors might have very large and unexpected effects on your dependent variable, rendering anything you think you are finding in your experiment misleading.

So for you technoids out there longing to do an ANOVA....



Let’s say That Guy Over There is trying to increase the critical performance measure “expansion” (higher is better). Historically the process has run at around 70-percent expansion. That Guy is very excited to design an experiment:

Figure 2: Expansion variables

He talks to his team, and asks them to identify three factors that they can control on the process that might affect expansion:

Figure 3: Three factors that might affect expansion

Obviously, he has decided to ignore this article (heck, he probably doesn’t even read my articles, the Philistine), so he goes forth and designs a full factorial with these factors and runs the experiment with n = 2 per combination. He places the data on the group hard drive, so you can download it, too [3].

He jumps right in and analyzes the experiment (totally not checking any test assumptions, of course) with an α = 0.05 and finds this:

Figure 4: Analysis of variance (ANOVA) ignoring humidity

Now first off, there is a subtle signal here that something is wrong—do you see it?



Take a look at the F-ratios. Recall that the F-statistic is:

It’s just the ratio of the variability we see between effects (the Mean Square for the different sources) and the variability we see within each combination (the Mean Square of the error). The expected value of F-ratios is 1 if there is no effect (all between variability is statistically the same as the within variability).

On the other hand, if there is an effect of the factor, the between variability should be larger than the within variability, leading to a ratio greater than 1. (Don’t understand why? Try reading “ANOVA by Any Other Name [4].”) Yet we see one F-ratio (the temp*speed interaction) that is fairly far below 1. This is a possible indication of some sort of confounding—a lot of variability going into the error term (the within-cell variance) making it big (146.565), as compared to the much smaller variance between the six means of the interaction (7.092). Of course this could be due to random variability, and since ANOVA is a directional test, we cannot make a statistical conclusion about it being too small, but it would be something to be suspicious about.

Well, That Guy Over There certainly doesn’t know to do this, so he proceeds although he’s clueless. He looks at that R2 (incorrectly) and sees 0.662 and assumes that he has made progress on the problem of expansion. He concludes that we can ignore everything except the main effect of temperature—everything else has a p-value above 0.05, his chosen α, so just run those wherever the operators want. Here is what happened with the expansion as temperature varied in the experiment.



Figure 5: The expansion as temperature varied in the experiment

Based on that, he selects the following setting as optimum:

Figure 6: Prediction of optimum for nonblocked experiment

It looks like he is expecting to increase the expansion by about 7 percent. That rather large standard deviation means that there is a pretty big chunk of unexplained variability. In fact, that 7-percent increase in expansion is going to be hard to see over the random noise of the process.

Now, what he didn’t know, since he didn’t read this article, is that you have to identify every known potential factor and deal with it somehow. If he had simply asked, “What other factors could affect expansion?” his team would have immediately said, “Well, humidity might.” Or perhaps he knew about humidity, but since he can’t control it, he assumed that it would be a small effect.

You all know want happens when you make assumptions like that, right? It makes an ass out of your… umptions. Something like that.

Anyway, humidity did its own thing during this time (a kind of sinusoidal up and down pattern through the experiment), and he didn’t bother to record it on an engineering log, nor did he even randomize the order of the runs (which meant that humidity’s effects were



confounded with the process settings, especially temperature). I guess he figures that order is as random as anything else.

So now you decide to take a crack at it, and, having read this article, you do ask what else can affect expansion, and you find out that the humidity is a likely culprit.

You could then use the second technique to handle nuisance variables: limit or control the factor. I suppose you could perform the experiment in the lab at some level of controlled humidity, or maybe on the factory floor during a time of some constant humidity level, but what would you learn that you could apply to the actual process? This technique leaves you open to questions about the external validity [5] of anything you find.

Or, you could use the third technique, blocking. This underused technique is very powerful and will save you lots of money and time and get you better results.

In blocking, we have a factor that we think affects the dependent variable, but we don’t care to include it in the experiment. You know humidity is going to vary, and it is going to affect the expansion somewhat. You don’t care to prove that; you just need to control for it. (However, if you think the factor interacts with any of the other main effects, you would need to incorporate it into the experiment.) The amazing thing is that we don’t have to increase sample size to add a block. If that isn’t better than getting a free lunch, I don’t know what is.

So you design the same experiment, adding in humidity as a blocked factor. Logistically, you choose to run the first set of process settings (in random order) at a low-humidity time and the second set (in random order) at a high-humidity time.

Figure 7: Experiment variables adding humidity as a blocked factor

You run the experiment and also post the results on the group hard drive, which you can download here [6].

Go ahead and try analyzing that; see if you come up with something different than That Guy Over There. I’ll give you the analysis in Part 2.



About The Author

Steven Ouellette is the founder of six-sigma-online.com, president of The ROI Alliance LLC, an instructor at the University of Colorado Engineering Management Program, and chief consultant for the Center for Statistical Solutions. He has been in process design and improvement since 1992 as an engineer and later as a consultant working in many industries. He has a Master Black Belt certification, a master’s degree in engineering management, and a bachelor’s degree in metallurgical and material science. He also acts as a board member on Orion Registrar’s committee to safeguard impartiality

Quality Insider Article Twitter Ed

© 2011 Quality Digest Magazine. All Rights Reserved.

Homethe InsiderVideosMagazinePrivacy PolicyResourcesSearch

SubscribeAdvertiseAbout Us

Source URL (retrieved on 09/29/2011): http://www.qualitydigest.com/inside/quality-insider-article/blocking-out-nuisance-part-one.html

Links: [1] http://www.amazon.com/Design-Experiments-Quality-Engineering-Jeffrey/dp/0070388075 [2] http://www.qualitydigest.com/IQedit/Images/Articles_and_Columns/Aug_2011/ouellette-lg1.png# [3] http://www.roi-ally.com/images/stories/downloads/heretic053_unblocked.txt [4] http://www.qualitydigest.com/inside/six-sigma-column/anova-any-other-name# [5] http://en.wikipedia.org/wiki/External_validity [6] http://www.roi-ally.com/images/stories/downloads/heretic053_blocked.txt



That Guy Over There’s data from experiment when humidity as ignored. Temperature Speed Feed_Rate Expansion 1 1 1 70.49 1 1 2 69.8 1 1 3 75.32 1 2 1 71.51 1 2 2 63.73 1 2 3 64.99 2 1 1 54.49 2 1 2 54.82 2 1 3 59.09 2 2 1 69.34 2 2 2 56.33 2 2 3 49.77 3 1 1 79.51 3 1 2 75.39 3 1 3 81.53 3 2 1 96.32 3 2 2 74.21 3 2 3 53.29 1 1 1 86.83 1 1 2 87.88 1 1 3 95.07 1 2 1 91.28 1 2 2 85.61 1 2 3 85.64 2 1 1 73.38 2 1 2 67.96 2 1 3 66.82 2 2 1 69.68 2 2 2 51.12 2 2 3 40.31 3 1 1 64.84 3 1 2 59.81 3 1 3 59.68 3 2 1 75.34 3 2 2 51.05 3 2 3 33.84

Date with Humidity as a Blocking variable: Temperature Speed Feed_Rate Humidity Expansion 1 1 1 1 84.565 1 1 2 1 88.582 1 1 3 1 95.748 1 2 1 1 94.433 1 2 2 1 91.647 1 2 3 1 95.791 2 1 1 1 86.501 2 1 2 1 85.077 2 1 3 1 89.666 2 2 1 1 92.298 2 2 2 1 79.517 2 2 3 1 69.868 3 1 1 1 94.254 3 1 2 1 89.618 3 1 3 1 87.091 3 2 1 1 103.719 3 2 2 1 77.124 3 2 3 1 54.724 1 1 1 2 58.274 1 1 2 2 58.639 1 1 3 2 67.966 1 2 1 2 66.684 1 2 2 2 61.381 1 2 3 2 67.102 2 1 1 2 56.711 2 1 2 2 53.297 2 1 3 2 58.723 2 2 1 2 64.439 2 2 2 2 49.049 2 2 3 2 39.153 3 1 1 2 62.395 3 1 2 2 58.876 3 1 3 2 59.9 3 2 1 2 73.773 3 2 2 2 44.711 3 2 3 2 24.196

DOE - Blocking out the Nuisance - Part One.pdf

Documents

Transcript of DOE - Blocking out the Nuisance - Part One.pdf