Planning for DOE, Lesson 1

58
7/18/2019 Planning for DOE, Lesson 1 http://slidepdf.com/reader/full/planning-for-doe-lesson-1 1/58  Engineering Statistics Training Introduction to Design of Experiments Planning and Executing Dave Jackson, M.S. EW Engineering Statistics Introduction to Design of Experiments, Planning and Execution, July ‘05 1 

description

Statistics - Design of Experiments

Transcript of Planning for DOE, Lesson 1

Page 1: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 1/58

 

Engineering Statistics Training

Introduction to Design of Experiments

Planning and Executing

Dave Jackson, M.S.

EW Engineering Statistics

Introduction to Design of Experiments, Planning and Execution, July ‘05 1 

Page 2: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 2/58

Contents

Introduction to DOE -------------------------------------------------------------- 4

Prerequisites ------------------------------------------------------------------- 5Training objective ------------------------------------------------------------- 6What is DOE? ------------------------------------------------------------------ 7

 Applications of DOE ---------------------------------------------------------- 8DOE overview ----------------------------------------------------------------- 9Why make an effort to learn DOE?---------------------------------------10DOE vocabulary --------------------------------------------------------------13

2 and 3 level designs -------------------------------------------------------14The empirical model ---------------------------------------------------------16The steps for completing a DOE -----------------------------------------18The sequential nature of DOE --------------------------------------------19

Planning for DOE ----------------------------------------------------------------20

•  DOE planning prior to selecting a design --------------------------21

•  Define the process you want to study -------------------------------22•  Define the experiment objectives ------------------------------------23

•  Identify responses -------------------------------------------------------24

•  The sample mean --------------------------------------------------------25

•  The sample standard deviation ---------------------------------------36

•  The %CV -------------------------------------------------------------------38

•  Pass/fail responses ------------------------------------------------------39

•  Sample size ---------------------------------------------------------------30

•  Response calculation worksheet (example) -----------------------33

•  Measuring the response ------------------------------------------------34

Introduction to Design of Experiments, Planning and Execution, July ‘05 2 

•  Response worksheet ----------------------------------------------------40

Page 3: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 3/58

Contents

Planning for DOE ------------------------------------------------------ continued

•  Process variables worksheet (example) ----------------------------42

•  Process variables (noise) ----------------------------------------------45

•  The importance of wide factor level ranges -----------------------46

•  Signal-to-noise review --------------------------------------------------48

•  Factor that are a nuisance to vary -----------------------------------49

•  Additional notes on factors --------------------------------------------50

Executing a DOE ----------------------------------------------------------------51

•  Checklist for executing a DOE ----------------------------------------52

Final Thoughts --------------------------------------------------------------------53

 Appendix A ------------------------------------------------------------------------54

References ------------------------------------------------------------------------55

Introduction to Design of Experiments, Planning and Execution, July ‘05 3 

Page 4: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 4/58

 

Introduction to DOE

Introduction to Design of Experiments, Planning and Execution, July ‘05 4 

Page 5: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 5/58

Prerequisites

•  An interest in using Design of Experiments (DOE)

•  An expectation for using DOE in the near future

•  The ability to navigate in a Windows environment

•   A working knowledge of basic statistical concepts. The

training will reinforce and expand on these concepts usingpractical examples. 

Introduction to Design of Experiments, Planning and Execution, July ‘05 5 

Page 6: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 6/58

Training Objective

The objective of this course is to increase the likelihood of successful

experiment outcomes at Edwards.

The course will cover DOE planning, design selection, proper

execution, and analysis of DOE data using the Statgraphics software

package.

 A successful experiment,

•  Yields useful information that moves the project forward

•  Does not have to be repeated 

Introduction to Design of Experiments, Planning and Execution, July ‘05 6 

Page 7: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 7/58

What is DOE?

DOE is a procedure for learning how process variables actingsimultaneously affect, and interact to affect process performance.

Introduction to Design of Experiments, Planning and Execution, July ‘05 7 

Estimated Response SurfaceMixer Speed=25.0

TemperatureTime

   A   f   T   e   m   p

   e   r   a   t   u   r   e

556 558   560   562 564  240250260270280290300

-11

-10

-9-8

-7

-6

-5

Page 8: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 8/58

 Applications of DOE

•  Determine the process variable settings that simultaneouslyoptimize several performance characteristics 

•  Screen out the important process variables from a large set ofcandidate variables. 

•  Reduce development time for new processes 

•  Minimize cost by finding process variable settings that reduce thecost of running a process 

•  Reduce variation in a process 

Introduction to Design of Experiments, Planning and Execution, July ‘05 8 

Page 9: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 9/58

Why Make the Effort to Learn DOE?

DOE is a vast improvement over one-factor-at-a-time experimentation

and/or a series of trial-and-error efforts:

•  More powerful – DOE gets the right answer more often

•  More efficient – DOE gets the answer in fewer runs, with

fewer test articles and overall shortens

development time

•  More actionable – DOE results are easier to interpret

Introduction to Design of Experiments, Planning and Execution, July ‘05 9 

Page 10: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 10/58

Why Make the Effort to Learn DOE? cont.

Statistically Designed Experiments Can Detect and Describe

Interactions Between Input variables

Two variables interact if changing the level of one variable changes

the relationship between a second variable and the response. Time

and Temperature interact in the process of baking cookies.

Introduction to Design of Experiments, Planning and Execution, July ‘05 10 

highlow

Flavor

Oven Temperature

Oven Time = short

Oven Time = long

Page 11: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 11/58

Introduction to Design of Experiments, Planning and Execution, July ‘05 11 

Why Make the Effort to Learn DOE? cont.

Statistically Designed Experiments Can Deal With Curvature in the

Response.

Flavor

low

 Amount of Salt in Food

medium high

Page 12: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 12/58

ction to Design of Experiments, Planning and Execution, July ‘05 12 

DOE Overview

 A statistically designed experiment consist of a series of runs, or trials, in whicchanges are made simultaneously to the input variables of a process, and (2) pobserved.

Introdu

 

TissueThickness

FlowRate Power Pressure Duration

 Ave.LesionWidth

SDLesionWidth

9 1 8 225 6012 6 8 225 60

9 1 8 75 90

12 1 8 225 90

12 1 12 225 60

9 6 8 75 60

12 6 12 225 90

9 6 12 75 90

10.5 3.5 10 150 75

12 1 8 75 6012 6 8 75 90

12 1 12 75 90

9 6 8 225 90

Page 13: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 13/58

DOE Vocabulary

•  There are typically many process variables that affect aresponse. Those process variables whose levels you changeper your experiment design are called factors.

•  Responses are the process performance characteristics thatyou are interested in studying.

•  An experiment design consists of the entire set of runs in theDOE.

•  Runs are different combinations of factor levels. Identical back-to-back runs require a complete reset of the process betweenruns.

•  Center points are runs located in the center of the experimentspace.

•  The experiment space is the multidimensional envelopeextending to the low and high levels of the factors.

Introduction to Design of Experiments, Planning and Execution, July ‘05 13 

Factor B

Factor A

Factor C

Center points

Experiment space 

Page 14: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 14/58

2 and 3-level Designs

Two Level Designs:

•  Factors are run at two levels (low and high) in the basedesign.

•  Even when center points are added, the design is stillconsidered two-level.

•  Used to model responses with, at most, a small amount ofcurvature.

•  Screening experiments1 

•  Range finding

Three Level Designs:

•  Factors are run at three or more levels in the base design

•  Center points are often an integral part of the base design

•  Used to model significant curvature in the response

Introduction to Design of Experiments, Planning and Execution, July ‘05 14 

1 A two-level experiment is called a “screening experiment” if its primary purposeis to screening-out the most important factors from a large group of candidatefactors. In this scenario, we are interested in identifying significant main effects,rather than interaction effects or curvature effects; the latter assumed to be anorder of magnitude less important.

Page 15: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 15/58

Which Response Can You Model With a 2-Level Design?

Response Surface A

X

Y

   R  e  s  p  o  n  s  e

00.2

0.40.6

0.81

0

80

Y

 

Response Surface B

XY

   R  e  s  p  o  n  s  e

-4-2

-100

 

Response Surface C

X

Y

   R  e  s  p  o  n  s  e

-2

-30

-12

(X 1000)

 

Introduction to Design of Experiments, Planning and Execution, July ‘05 15 

 Answer: A only. B and C require a design with at least threelevels.

Page 16: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 16/58

The Empirical Model

The software builds a model which is a polynomial approximation forthe true and unknown relationship between each response and the

factors.

Here is the complete model (up to 3rd order) for a three-level

experiment involving three factors,  x1,  x2, and  x3.

0 1 1 2 2 3 3

12 1 2 13 1 3 23 2 3 123 1 2 3

2 2 211 1 22 2 33 3

2 2 2 2 2112 1 2 113 1 3 122 1 2 133 1 3 223 2 3 233 2 3

3 3 3111 1 222 2 333 3

ˆ y b b x b x b x

b x x b x x b x x b x x x

b x b x b x

b x x b x x b x x b x x b x x b x x

b x b x b x   ε +

= + + +

+ + + +

+ + +

+ + + + + +

+ + + +

2

 

Model Term Definition

ˆ y  The estimated average response overthe experiment space.

0b   Constant.

1 1 2 2 3 3, ,b x b x b x  Main effects representing the linearpart of the model.

Introduction to Design of Experiments, Planning and Execution, July ‘05 16 

Page 17: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 17/58

The Empirical Model cont.

Model Term Definition

12 1 2 13 1 3 23 2 3, ,b x x b x x b x x  

The two-factor interactionsrepresenting mild curvature in themodel.

1 2 3 1 2 3b x x x  

The three-factor interaction. Usuallyconsidered small or not important formany processes.

2 211 1 22 2 33 3

, ,b x b x b x2  Quadratic effects representingsignificant curvature in the model.

2112 1 2b x x , , ,2

113 1 3b x x   2122 1 2b x x

2133 1 3

b x x , ,2

223 2 3b x x   2

233 2 3b x x

Partial cubic effects. Usuallyconsidered small or not important formany processes.

3111 1b x , ,3222 2b x   3333 3b x

 Cubic effects. Usually consideredsmall or not important for manyprocesses.

ε   

The error term representing commoncause variation not accounted for bythe other terms. Assumed to have anormal distribution with mean zero and

variance, 2σ   .

Note that you can select designs to estimate three-factor interactions,partial cubic, and cubic effects, but they are seldom important andtherefore usually not worth the extra expense to estimate.

Introduction to Design of Experiments, Planning and Execution, July ‘05 17 

Page 18: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 18/58

Steps for Completing a DOE

The font size indicates the relative importance for having asuccessful DOE outcome.

1. Planning

2. Select Experiment Design

3. Execute Experiment & Collect Data

4. Analyze Data

5. Perform best-setup confirmation run, or validation

Introduction to Design of Experiments, Planning and Execution, July ‘05 18 

Page 19: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 19/58

The sequential Nature of DOE

DOE is iterative; rarely does one run a large, comprehensive design

in which a final conclusion is made.

Introduction to Design of Experiments, Planning and Execution, July ‘05 19 

Factor A

Direction of better performance

First DOE

Second DOE

Factor B

Page 20: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 20/58

 

Planning for DOE

Introduction to Design of Experiments, Planning and Execution, July ‘05 20 

Page 21: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 21/58

DOE Planning Prior to Selecting an Experiment Design

Introduction to Design of Experiments, Planning and Execution, July ‘05 21 

Define the process you want to study

Define experiment objectives

Identify responses and measurement methods (IQ/OQ)

Identify process variables and measurement methods (IQ/OQ)

Start

 Are interactions between factors likely?

Is curvature in the response likely?

Will any factor be a nuisance to vary?

End

Make sure process is stable (IQ/OQ)

Page 22: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 22/58

Define the Process You Want To Study

•  A process is a sequence of tasks required to accomplishsomething, or

•  A process is a means of transforming inputs into outputs

IPO process model

Introduction to Design of Experiments, Planning and Execution, July ‘05 22 

Product Design Influences

Product Performance

Variation in Product

PerformanceEquipment & Tooling

Procedures & Methods

Operator Influences

Material Properties

Environmental Influences

Process

Controllable process variables

(Inputs)

Responses

(Outputs)

Uncontrollable process variables(Noise)

Page 23: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 23/58

Define the Experiment Objectives

•  Clear statement of what you want to accomplish

•  Prioritize objectives

•  State constraints

•  Get team agreement

Poorly Stated Objective

Maximize seal strength.

Better Objective

Learn the process variable settings that maximize seal strengthsubject to the following constraints:

(1) Sterile barrier pouches must pass the SOP leak test;

(2) The visual appearance of the seals must pass SOPrequirements; and

(3) Throughput must be greater than or equal to 20 pouches perminute.

Introduction to Design of Experiments, Planning and Execution, July ‘05 23 

Page 24: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 24/58

Identify Responses

•  Select responses that are good predictors of field performance.

•  Use the mean of several observations from a run as the officialresponse value for a run.

•  Use the standard deviation of several observations from a runas an additional response representing the variation in theprocess.

•  Measure responses as continuous variables instead ofcategorical variables, whenever possible.

•  If unsure about two measurement methods, use both.

•  Omitting a key response variable can be a big mistake.

•  Choosing responses and their measurement methods aretypically the most time-consuming aspects of DOE planning.

Introduction to Design of Experiments, Planning and Execution, July ‘05 24 

Page 25: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 25/58

 The Sample Mean

If n observations are denoted by , their mean is:1 2 ,, ,   n x x xL

 

1 2

1( )n x x x

n x   + + +=   L .

Or, in more compact notation:

1 i x xn=   ∑

 

Suppose that these response data were taken during a particular run:

545935

4146

The mean response for the run is

1(54 59 35 41 46

5

47

1i x x

n

= + + + +

=

=   ∑

Introduction to Design of Experiments, Planning and Execution, July ‘05 25 

)  

Page 26: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 26/58

The Sample Standard Deviation

 An experiment objective might be to minimize the standard deviation,

or at least make sure that there is no obvious indication that the

standard deviation appears noticeably worse within the

manufacturing window you end up selecting.

If n observations are denoted by , their variance is:1 2

,, ,   n x x xL

 

2 2 21 2

1( ) ( ) ( )

1   ns x x x x xn

⎡ ⎤⎣ ⎦

= − + − + + −−

  L  2 x .

Or more compactly:

2 21( )

1   ii

s xn

= −−   ∑

  x  

The standard deviation, s, is the square root of the variance

2s s=  

Introduction to Design of Experiments, Planning and Execution, July ‘05 26 

Page 27: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 27/58

The Sample Standard Deviation cont.

Suppose these response data were taken during a particular run:

5459354146

The sample standard deviation response is

2

2 2

2 2 2 2

1( )

1

1(54 47) (59 47) (35 47) (41 47) (46 47)

1

93.51

9.67

i

s

s x xn

n

s

⎡ ⎣

= −−

= − + − + − + − + −−

=

=

=

∑   2

 

Introduction to Design of Experiments, Planning and Execution, July ‘05 27 

Page 28: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 28/58

The %CV

Like the standard deviation, the coefficient of variation (%CV) may be

used as a response representing variation in performance. An

experiment objective might be to minimize the %CV.

The %CV of n observations1 2

,, ,   n x x xL  is

%  sample standard deviation

100 100sample mean

CV   s

 x=   ⋅ ⋅=  

For many processes, the variation increases as the mean increases.In this situation, the %CV is a better response than the standarddeviation. See Appendix B for more information.

Introduction to Design of Experiments, Planning and Execution, July ‘05 28 

Page 29: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 29/58

Pass/Fail Responses

Consider as a last resort because the sample size requirements foreach run can reach into the hundreds

The percent defective for DOE run i , is

number defective in runˆ   100

total number tested in runi p  = ⋅  

Strategies:

•  Substitute a continuous response whenever possible

•  If a Pass/Fail test must be used, redefine it so more test articlesfail

o  Change a 10 psig test to a 30 psig test

•  If a Pass/Fail test must be used, rate each test article as to theextent of failure

o  0=no failure, 1=25% failure, 2=50%, 3=75%, 4=100%failure

Introduction to Design of Experiments, Planning and Execution, July ‘05 29 

Page 30: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 30/58

More Than One Sample Per Run Usually Required

The software assumes that the response value you provide for aparticular run is close to the true (long-run) response for that run.

 An approach for getting near the true response value for the run isto use the mean of multiple samples processed during the run.

Response (units)

   d  e  n  s   i   t  y

0 3 6 9 12 15

0

0.1

0.2

0.3

0.4

0.5

Run #1 Run #2

Long-run Performance

 

Suppose you make one observation per run. . .

  You might get lucky and get these values; then the software canaccurately estimate the difference between the runs.

  You might get these values; then the software will over estimatethe difference between the runs.

 You might get these values; then the software will underestimatethe difference between the runs.

Introduction to Design of Experiments, Planning and Execution, July ‘05 30 

Page 31: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 31/58

What Sample Size (# Observations per Run) Do I Need?

Use available feasibility data taken under a single set of process

conditions, and plot the data as shown below.

 After about 12 test articles (for this example only) the mean remains

roughly unchanged. 

n

  m  e  a  n  s  o   f   t

   h  e   f   i  r  s   t  n   t  e  s   t  a  r   t   i  c   l  e  s

   2 3 4 5 6 7 8 9   1   0

   1   1

   1   2

   1   3

   1   4

   1   5

   1   6

   1   7

   1   8

   1   9

-2

-1

0

1

2

3

4

Note: all 20 test articles were from the same run 

Introduction to Design of Experiments, Planning and Execution, July ‘05 31 

Page 32: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 32/58

More on Sample Size and Noise

•  The greater the variation (noise) in the response, the larger the

sample size needed to get good estimates for the response

values for the runs.

•  To reduce noise, consider reporting the official measurement of

a test article as the mean of several measurements made on

that test article

o  Use when there is noticeable within-test article variation

o  Use when there is noticeable measurement variation

•  The process must be stable during the execution of a DOE;

otherwise, process drift or jumps will likely obscure the real

effects of the factors

o  Stability means in a state of statistical control. See

references on statistical process control (SPC).

Introduction to Design of Experiments, Planning and Execution, July ‘05 32 

•  Rule-of-thumb: it usually takes about twice as many samples

(observations) to get as good an estimate of the standard

deviation as it does the mean; however, most experimenters

will go with the sample size required for the mean and glean

whatever information they can about the standard deviation

response. This approach works pretty well unless your primary

experiment objective is to study and reduce process variation. 

Page 33: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 33/58

Response Calculation Worksheet (an example)

In this example, there were 10 samples used in Run #1 and 5measurements per sample. The “Run Mean” and “Run SD” werethe response values for Run #1 that were entered intoStatGraphics

Introduction to Design of Experiments, Planning and Execution, July ‘05 33 

Page 34: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 34/58

Measuring the Response

•  Measurement processes include:

o  instruments (gages)

o  procedures

o  fixtures

o  software

o  people

o  environment

o  assumptions

•  Basic requirements for measurement processes:

o  good discrimination

o  adequate range

o  low bias

o  good repeatability

Introduction to Design of Experiments, Planning and Execution, July ‘05 34 

Page 35: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 35/58

Measurement Discrimination

•  Discrimination is the smallest change response that the

measurement process can faithfully detect and indicate

•  Discrimination should be no larger than about 1/10 of the

smallest difference in the response you require the experiment to

detect

Response (units)

   d  e  n  s   i   t  y

0 3 6 9 12 15

0

0.1

0.2

0.3

0.4

0.5

Run #1 Run #2

Difference in Response

Long-run Performance

 

Introduction to Design of Experiments, Planning and Execution, July ‘05 35 

Page 36: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 36/58

Measurement Discrimination cont.

•  Alternatively, the discrimination should divide the process

variation into 10 parts or more (rule of tens). The process

variation is the long run, ±3σ spread in the response observed

over a single set of process conditions. The ±3σ spread is also

called the natural tolerance of the process. 

Introduction to Design of Experiments, Planning and Execution, July ‘05 36 

response measured in standard deviations

        d      e      n      s        i        t      y

-4 -3 -2 -1 0 1 2 3 4

0

0.1

0.2

0.3

0.4

0.5

99.7% of observations

95% of observations

68% of observations

Normal Distribution Model

Page 37: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 37/58

Measurement Discrimination cont.

The process standard deviation, , may be estimated as follows:

4

ˆ  s

cσ   = ,

where s is the center line from a standard deviation control chart,

and is a constant found in SPC references.4c

 

 Alternatively,

ˆ   sσ   = ,

Introduction to Design of Experiments, Planning and Execution, July ‘05 37 

where is the sample standard deviation of n observations made over

a single set of process conditions.

s

Page 38: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 38/58

Measurement Range & Bias

•  Range refers to the smallest and largest values that a

measurement process can faithfully detect and indicate. Select

a gage will adequate range.

•  Bias2 is the difference between the mean of multiple

measurements on the same reference standard and the

accepted value of the reference standard. Bias is a smaller-

the-better characteristic. Bias is usually handled through

calibration.

Introduction to Design of Experiments, Planning and Execution, July ‘05 38 

2 Bias is sometimes referred to as “inaccuracy”

Page 39: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 39/58

Introduction to Design of Experiments, Planning and Execution, July ‘05 39 

Measurement Repeatability

Repeatability is that portion of the observed process variation

attributable to the measurement process alone (assumes the same

operator makes all of the measurements).

Repeatability may be thought of as the sample standard deviation of

n measurements made at the same location on the same test article,

by the same person3.

Guidelines:

•  Repeatability must be small compared to the smallestdifference in the response that you want the experiment to beable to detect

•  Alternatively, repeatability must be small compared to the ±3σ  process variation

•  Where appropriate, report the official measurement for a testarticle as the mean of two or more observations made on thattest article

3 This working definition may be used in the early stages of the DOE effort, if theresults of a formal gage capability study are not be available

Page 40: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 40/58

ction to Design of Experiments, Planning and Execution, July ‘05 40 

Response Worksheet

Response

Procedure &

Spec

Estimated

Range ofData

/

Ga

Di

Low:

High:

Low:

High:

Low:

High:

Low:

High:

= Smallest change in the response that is worth investigating= Standard deviation of the experimental error (standard deviation of the means

Introdu

 

Page 41: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 41/58

 

Completing the Response Worksheet

•  For each line item, think about how you can reduce the overall

noise in the experiment.

•  / expresses the smallest change in the response that is

worth investigating, as a multiple of the background noise.

•  You use / in the StatGraphics Power Curve procedure to

calculate the number of extra replicate runs you need to add to

the base design.

•  If a transformation is to be performed on a response variable,

then and need to be in transformed units.

Estimate (sigma) as follows:

1. Select one set of run conditions

2. Set up, stabilize and run the process in the same manner as youwill for the DOE

3. Collect n observations, and calculate their mean

4. Completely reset the process

5. Repeat steps 2 through 4 two or more times.

Introduction to Design of Experiments, Planning and Execution, July ‘05 41 

6. Calculate the standard deviation of the three or more means.

Page 42: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 42/58

 

Process Variables Worksheet (example)

Constant,

Noise, orFactor

Process Variable DOE Value

Operator Influences

C Process operator Single operator, Peng

C Af measurement operator Single operator, Peng

Material PropertiesC BioPhysio valve Model 3100

C BioPhysio 2-D flat pattern 193396-029 Rev E, Lot # 0989MG 3011/248-10

C Nitinol flat sheet MetalTex, p/n 193402, Lot #248-10

C Salt Nitrate Salt, p/n 123456001

CDoes salt go bad over time or over# parts processed? No

C Quench water Tap water. Room temperature (20°C ± 5°C),

Equipment and Tooling

C Shape-setting mandrel, 29 mm p/n 392664

C Salt bath Ajax Hultgren Salt Bath Furnace, model # 3100.

C Mandrel rack, 5 up p/n 392827

C Mixer Accumix variable speed controller model MM23401C. ¼Hp motor. Propeller location and configuration unknown.

C Salt bath temp controller

For gross adjustment only; record setting for each DOErun, but use the chart recorder to setup the DOE runs.

Met ID 116229.

CChart recorder & Temperatureprobe Met ID 117406

Procedures and Methods

C Shape setting SOP 2394 red-lined, attached

C Bath stabilization time >= 90 minutes

CTest article position on mandrelrack

Position the test article on the mandrel location marked.Do not load frames on the remaining 4 mandrels(assumption: the missing frames will not invalidate theDOE results with respect to how the process will behavein production).

C Number of mandrels on rack 5 mandrels

C Orientation of rack in bath Test article at 6:00 with respect to operator.

C

Depth of salt in bath

Start DOE: 21.5 inches

Introduction to Design of Experiments, Planning and Execution, July ‘05 42 

Minimum height is 21 – 22 inches from bottom of bath.

C Depth of valve frames in bathFixed by the geometry of the mandrel rack and by thedepth of the salt.

Page 43: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 43/58

Page 44: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 44/58

 

Completing the Process Variables Worksheet

•  For each line item, think about how you can reduce the overallnoise in the experiment.

•  Serves as good documentation.

•  Include all variables, even those believed only remotelyinfluential.

•  Keep in mind the six process variables categories.

•  Brainstorm as the team watches the process operation or theproduct build.

•  Brainstorm as the team reviews applicable SOPs, processsheets, equipment manuals, and the like.

•  Consider measurement discrimination, range, and bias as youdo for responses.

•  Consider completing an input variables worksheet for eachresponse measurement process.

Introduction to Design of Experiments, Planning and Execution, July ‘05 44 

•  Failure to include an important factor in your DOE will likelymake the results unusable, especially if the factor is involved inan interaction.

Page 45: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 45/58

 

Process Variables of the Noise Variety

•  Noise variables are not closely controlled because, (1) they

are thought to have at most a small effect on performance,

and (2) they are considered difficult to closely control in long

term production.

•  During the execution of the DOE, hold noise variables

constant if you can, and record their values.

Introduction to Design of Experiments, Planning and Execution, July ‘05 45 

Page 46: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 46/58

 

The Importance of Wide Factor Level Ranges

•  Wide ranges magnify effects, making it easier for the software

to detect which factors are important and the extent of their

importance.

•  Wide ranges minimize the need for extrapolation post-DOE.

Introduction to Design of Experiments, Planning and Execution, July ‘05 46 

   M  a  g  n

   i   f   i  e   d  e

   f   f  e  c

   t

Response

Smalleffect

Factor A

Possible DOE factor ranges

Page 47: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 47/58

 

Wide Factor Ranges, But Not Too Wide

•  Typically wider than production ranges.

•  Some runs may yield bad product; that’s OK, as long as the

product is recognizable and measurable in terms of the

responses.

•  Make sure that all of the runs are doable before executing the

DOE; one missing run can prevent you from drawing useful

conclusions.

•  The DOE procedure works only when the response is “smooth”

over the experiment space, so avoid factor ranges that yield

responses of zero or infinity.

Introduction to Design of Experiments, Planning and Execution, July ‘05 47 

Page 48: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 48/58

 

Signal-to-Noise Review

DOE operates on a signal-to-noise basis; signals are the effects of

the factors; noise includes the variation in the measurement process,

the inherent variation in the process under study, and the variation

within individual test articles.

Increase signal where you can

•  Use wide factor ranges

Decrease noise where you can

•  Report response values as the means of several test articlesper run.

•  Where appropriate, report the official measurement for a testarticle as the mean of two or more observations made on thattest article.

•  Make sure the process is stable prior to executing the DOE.

•  Thoroughly account for all process variables and control themduring the execution of the DOE.

•  During the execution of the DOE, treat each run and each testarticle identically except for the factor level changes indicatedby the experiment design.

•  During the execution of the DOE, wait for the process tostabilize after making factor level changes.

Introduction to Design of Experiments, Planning and Execution, July ‘05 48 

Page 49: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 49/58

 

What about Factors that are a Nuisance to Vary?

Sort the design by the nuisance factor, and perform the runs in the

sorted order; that way you minimize the number of level changes

for the nuisance factor.

•  With this approach, lurking variables may influence the

experiment, so make sure that the process conditions are

identical from run to run.

Introduction to Design of Experiments, Planning and Execution, July ‘05 49 

•  Of course, the best approach for guarding against lurking

variables is to randomize the run order.

Page 50: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 50/58

 

Factors, Additional Notes

If factor levels are indicated by a knob position that does not

correspond to a calibrated scale. . .

•  Use an independent gage to learn the true value of eachfactor level; it may be that knob position does not correspondto a linear scale. For example, knob position #4 may notcorrespond to twice the factor level of knob position #2. Thissituation can be a big barrier to analyzing your experiment.

Factors, continuous and categorical

•  The levels for continuous factors can be set to (almost) anyvalue along some region of the real number line.

•  The levels for categorical factors fall into a finite number ofcategories.

•  Use continuous factors whenever possible.

Introduction to Design of Experiments, Planning and Execution, July ‘05 50 

Page 51: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 51/58

 

Introduction to Design of Experiments, Planning and Execution, July ‘05 51 

Executing a DOE

Page 52: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 52/58

 

Checklist for Executing a DOE

 ____ Are there enough test articles and materials for the DOE plus extra tocover mishaps and allow for a confirmation experiment?

 ____ Is the principal investigator available to answer questions?

(It’s a good practice for the principle investigator to be present for thefirst couple of runs to help make sure that there are nomisunderstandings about how the DOE should be executed)

Has team practiced/rehearsed all procedures related to the DOE?

 ____ Will the process be stable for the duration of the DOE execution?

 ____ Perform runs in the order specified.

 ____ Note any unusual occurrences during the DOE; record the time, run#, and test article number.

 ____ Hold all conditions the same for every run and every test article

except for the specified factor level changes.

 ____ Allow the process to warm up and stabilize after factor level changes.

 ____ Save all test articles at least until the DOE report is approved.

 ____ Always record individual data observations, even if means or otherstatistics will be used as “official” response values.

 ____ If a run is accidentally skipped, do it at the end of the experiment; the

run order is not as important as completing all of the runs.

 ____

Introduction to Design of Experiments, Planning and Execution, July ‘05 52 

Know the minimum number of decimal places for recording the data.

Page 53: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 53/58

 

Final Thoughts

•  Include DOE tasks in your project schedule.

•  Begin collecting DOE planning information long before you

need to execute your DOE.

•  Process knowledge and engineering/scientific expertise

must be applied throughout the DOE process; the key is to

get good data; statistical aspects are of secondary

importance.

•  Team effort

Introduction to Design of Experiments, Planning and Execution, July ‘05 53 

Page 54: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 54/58

 

 Appendix A

If your response variables include both the mean and standard

deviation of a performance characteristic, plot the standard deviations

vs their corresponding means (choose Plot… Scatterplots… X-Y

Plot… from the menu bar). If the plot shows the standard deviations

increasing with the means, as shown below, it’s possible that one of

the basic DOE assumptions (homogeneity of variances) has not been

met. Take the natural log of the individual response measurements,generate the scatter plot again, and see if the transformation corrects

the problem; if the transformation solves the problem, analyze the log

response data just as you would without the transformation.

Introduction to Design of Experiments, Planning and Execution, July ‘05 54 

Run Means

   R  u  n

   S   t  a  n

   d  a  r   d

   D  e  v

   i  a   t   i  o  n  s

   5   1   0

   5

   1   0

 .   5

   5   1   1

   5

   1   1

 .   5

   5   1   2

   5

   1   2

 .   5

   5   1   3

   5

   1   3

 .   5

(X 0.001)

4.2

6.2

8.2

10.2

12.2

14.2

(X 0.0001)

 

Page 55: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 55/58

 

References

Box, G. E. P., Hunter, W. G., and Hunter, J.S. (1978). Statistics for

Experimenters. Wiley, New York

Schmidt, S. R. and Launsby R. G. (1998). Understanding Industrial

Designed Experiments 4th ed. Air Academy Press, Colorado Springs,

Colorado

Taylor, W. T. (1995). Screening Experiments. Baxter Healthcare

Corp. short course

Hahn, G. J. (1977). Some Things Experimenters Should Know About

Experimental Design. J. Quality Technology. Vol. 9, No. 1

Cusimano J. (1996). Understanding & Using Design of Experiments.

Quality, April 1996

Introduction to Design of Experiments, Planning and Execution, July ‘05 55 

Page 56: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 56/58

 

Introduction to Design of Experiments, Planning and Execution, July ‘05 56 

Notes

Page 57: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 57/58

 

Introduction to Design of Experiments, Planning and Execution, July ‘05 57 

Notes

Page 58: Planning for DOE, Lesson 1

7/18/2019 Planning for DOE, Lesson 1

http://slidepdf.com/reader/full/planning-for-doe-lesson-1 58/58

 

Notes