A Further Step in Experimental Design (III): The Response Surface

54 Journal of Validation technology [Autumn 2011] i v t home.com

For more Author

information,

go to

gxpandjvt.com/bios [

Statistical Viewpoint.John A. Wass[

ABOUT THE AUTHORJohn A. Wass is a consulting statistician with Quantum Cats Consulting in the Chicago area, as well as a contributing editor at Scientific Computing, and administrator of the Great Lakes JMP Users Group. He served for a number of years as a senior statistician at Abbott Laboratories (in both pharmaceuticals and diagnostics). John may be contacted by e-mail at [email protected].

A Further Step in Experimental Design (III): The Response Surface

“Statistical Viewpoint” addresses principles of statistics useful to practitioners in compliance and validation. We intend to present these concepts in a meaningful way so as to enable their application in daily work situations.

Reader comments, questions, and suggestions are needed to help us fulfill our objective for this column. Please send any comments to managing editor Susan Haigney at [email protected].

KEY POINTSThe following key points are discussed:

• Design of experiments (DOE) consists of three basic stages: screening (to identify important fac-tors), response surface methodology (to define the optimal space), and model validation (to confirm predictions).

• A critical preliminary step in the screening stage is for subject matter experts to identify the key list of factors that might influence the process.

• A DOE design consists of a table whose rows represent experimental trials and whose columns (vectors) give the corresponding factor levels. In a DOE analysis, the factor level columns are used to estimate the corresponding factor main effects.

• Interaction columns in a design are formed as the “dot” product of two other columns. In a DOE analysis, the interaction columns are used to estimate the corresponding interaction effects.

• When two design columns are identical, the cor-responding factors or interactions are aliased and their corresponding effects cannot be distinguished.

• The order of running and testing experimental trials is often randomized to protect against the presence of unknown lurking variables.

• Blocking variables (e.g., day or run or session) may be included in a design to raise the design efficiency.

• Factor effects may be missed because they were not included in the original screening experi-ment, because they were not given sufficiently wide factor ranges, because the design was under-powered for those factors, because trial order was not properly randomized or blocked, or because of an inadequate model.

• Unusual interactions and higher-order effects occassionaly may be needed to account for cur-vature and work around regions of singularity.

• Where there are inequality constraints (e.g., areas where standard settings will not work), special designs are needed.

• The designs may become rather challenging and a statistician becomes an invaluable part of the team when considering problems of non-normal responses, unbalanced data, specialized cova-riance structures, and unusual or unexpected physical or chemical effects.

John A. Wass

gxpand jv t .com Journal of Validation technology [Autumn 2011] 55

John A. Wass.

Silica

Ab

rasi

on

Ab

rasion

Silica

Silane

Silane

4060

80

100120

140

160

180

200

220

4060

80

100

120

140

160

180

200

220

1020

30

40

50

10

0

1

2

0

1

2

20

30

40

50

Figure 1: Surface point.

(+1)

(+1)

(+1)

(-1)(-1)

(-1)

Figure 2: Box-Behnken design.

-�

-�

-�

�1

�3

�2

�

�

�

Figure 3: Central composite design.

Figure 4: CCD data.

Pattern Reagent1 Reagent2 Enzyme TempMix

Speed Y

1 a0000 -2 0 0 0 0 1.2

2 -+-+- -1 1 -1 1 -1 0.9

3 0A000 0 2 0 0 0 1.35

4 0000a 0 0 0 0 -2 1.4

5 A0000 2 0 0 0 0 1.85

6 -++++ -1 1 1 1 1 2.6

7 +--+- 1 -1 -1 1 -1 1.5

8 ---++ -1 -1 -1 1 1 1.2

9 ++--- 1 1 -1 -1 -1 1.3

10 ++++- 1 1 1 1 -1 4

11 000A0 0 0 0 2 0 2.6

12 00a00 0 0 -2 0 0 0.4

13 +-+-- 1 -1 1 -1 -1 2

14 00A00 0 0 2 0 0 1.8

15 000a0 0 0 0 -2 0 0.6

16 -++-- -1 1 1 -1 -1 1.1

17 -+--+ -1 1 -1 -1 1 0.9

18 0000A 0 0 0 0 2 0.8

19 +---+ 1 -1 -1 -1 1 1.5

20 00000 0 0 0 0 0 2.2

21 ++-++ 1 1 -1 1 1 2.5

22 --++- -1 -1 1 1 -1 2.3

23 0a000 0 -2 0 0 0 1.5

24 +-+++ 1 -1 1 1 1 2.7

25 +++-+ 1 1 1 -1 1 3.3

26 00000 0 0 0 0 0 3

27 ----- -1 -1 -1 -1 -1 0.9

28 --+-+ -1 -1 1 -1 1 2


Statistical Viewpoint.

INTRODUCTIONResponse surface methodology (RSM) is the develop-ment of the specific types of special designs to most efficiently accept a small number of inputs (relative to screening designs) that are known to be relevant to the final product and optimize a process result to a desired target (1). Once the results are confirmed, the analyst’s load becomes lighter (excepting in the case of non-reproducibility or results drifting out of specification). In effect, the response surface maps the fine detail in the area of optimal response (i.e., deter-mines the most desirable values of the inputs to get the optimal output of whatever is being manufactured, controlled, or studied). The three most important targets usually sought are optimal concentrations, variance reduction, and robustness (2). The adequacy of the model is most often checked by residual analy-sis, influence diagnostics, and lack-of-fit testing (3). JMP 9 is utilized herein for the design and analysis of an industrial example (4).

THEORYMany response surface designs are collections of spe-cialized statistical and mathematical techniques that have been well implemented in software using effi-cient algorithms (4, 5). In many real world cases the output includes more than one response and these need not be continuous functions. Let’s examine the case of a chemical engineer who wishes to maximize an important property (y) based on given levels of two chemical inputs, (x1) and (x2). The desired property is now a function of the two chemical entities plus error (ε), as follows:

y = f(x1, x2) + ε

The surface is represented by the following:

y = f(x1, x2)

The response surface is usually displayed graphi-cally as a smoothly curving surface, a practice that may obscure the magnitude of local extremes (see Figure 1).

In many problems using RSM, the experimenter does not know the exact mathematical form of the relationship between the input and output variables and, therefore, must find a workable approximation. The first “guesstimate” is a low order polynomial (e.g., first order model), as follows:

Figure 5: Actual by predicted plot.

Summary of FitRSquare 0.626836

RSquare Adj 0.407328

Root Mean Square Error 0.670639

Mean of Response 1.764286

Observations (or Sum Wgts) 28

Analysis of Variance

Source DFSum of Squares

Mean Square F Ratio

Model 10 12.843426 1.28434 2.8556

Error 17 7.645859 0.44976 Prob > F

C. Total 27 20.489286 0.0275*

Lack of Fit

Source DFSum of Squares

Mean Square

F Ratio 0.3815

Lack of Fit 14 4.8958594 0.349704 Prob > F

Pure Error 3 2.7500000 0.916667 0.9083

Total Error 17 7.6458594 Max RSq

0.8658

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.50.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

Y Predicted P=0.0275RSq=0.63 RMSE=0.6706

Y A

ctu

al


John A. Wass.

y = β0 + β1x1 + β2x2 + … + βn xn + ε

Obviously, this is a linear model that will not accom-modate curvature. If curvature is suspected, a higher order polynomial may be tried. The following is a second order model:

y = β0 + Σβixi + Σβiixi2 + ΣΣβij xixj + ε

where we sum over i, and i<j. The above two models are found to work well in a variety of situations. However, although these may be reasonable approximations over the entire design space, they will not be an exact fit in all regions. The goal is to find a smaller region of interest where they fit well. A proper experimental design will result in the best estimate of the model parameters. The response surface is usually fitted by a least squares procedure, and optimal values are pursued through sequential algorithms such as the method of steepest ascent. The details of these algorithms are beyond the scope of this series.

Another caveat is the importance of using the software tools with an incomplete understanding of the problem. It is sometimes tempting to overlook expert advice from the chemists or engineers and try to force-fit a simple solu-tion. As in the real world, we often find extremely messy data. It might be necessary to use a cubic factor, a three-way interaction, or a highly-customized design. At this point the reader may wish to review the previous article in the series (Journal of Validation Technology, Winter 2011) to re-examine the use of data transformations, blocking, and factor levels.

TECHNIQUES–THE DESIGNSThere are a variety of specialized designs and analytic techniques available for RSM (6). In this review, we will concentrate on four of the more popular (i.e., com-monly used). One will be chosen to analyze standard but messy industrial data.

Factorial DesignsThese are one of the most common designs used in situations where there are several factors and the experimenter wishes to examine the interaction effects of these on the response variable(s). The class of 2k factorial designs is often used in RSM and finds wide application in the following three areas (6):

•As a screening design prior to the actual response surface experiments

•To fit a first-order response surface model•As a building block to create other response sur-

face designs.

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5

-1.0

-0.5

0.0

0.5

1.0

Y PredictedY

Res

idu

al

Figure 6: Residual by predicted plot.

Term Estimate Std Error t Ratio Prob> |t|Enzyme 0.5041667 0.136894 3.68 0.0018*Temp 0.3625 0.136894 2.65 0.0169*Reagent1 0.3416667 0.136894 2.50 0.0231*Enzyme*Enzyme -0.108594 0.132547 -0.82 0.4240Reagent2 0.0916667 0.136894 0.67 0.5121Enzyme*Temp 0.10625 0.16766 0.63 0.5347Reagent1*Enzyme 0.06875 0.16766 0.41 0.6869Reagent2*Reagent2 -0.027344 0.132547 -0.21 0.8390Reagent1*Temp 0.03125 0.16766 0.19 0.8543Reagent1*Reagent1 -0.002344 0.132547 -0.02 0.9861

Figure 7: Sorted parameter estimates.

-3 -2 -1 0 1 2 3-4

-3

-2

-1

0

1

2

3

4

+

+

+ + + + +

+ +

+

Reagent1Temp

Enzyme

Normal Quantile

Blue line is Lenth’s PSE, from the estimates population.Red line is RMSE, Root Mean Squared Error from the residual.

No

rmal

ized

Est

imat

es (

Ort

ho

g t

)

Figure 8: Normal plot.



For more details on these designs, the reader is again referred to the article on screening designs in this series.

D-Optimal DesignsThis is a class of designs based on the determinant of a matrix, which has an important relationship to the moment matrix (M):

M = X’X/N,

where X’ is the inverse on X, the design matrix of inputs and N is the number of rows in X.

It is noted that the inverse of the moment matrix, M-1 = N(X’X)-1 contains variances and covariances of the regression coefficients scaled by N divided by the variance. Therefore, control of the moment matrix implies control of the variances and covariances and here is the value of these D-optimal designs (5).

4

0.978Reagent1 Reagent2 Enzyme Temp Desirability

Des

irab

ility

0.98

832

Y3.

1788

47±

0.85

8115

0.022 1 1

321

1

-1 0 1 -1 0 1 -1 0 1 -1 0 00.

25 0.5

0.751 1

0.5

0

0

Figure 9: Prediction profiler.

Reagent1

Reagent1

-2

-2

2

-2

3

3

2.5

2.5

2

2

1.5

1.5

2

11.5

0.5

1

1

0

0

-1.5

-1.5

-1.5

1.5

1.52

-1.5

1.5

-1

-1

-11

1

-1

1

-0.5

-0.5-0.5

-0.50.5

0.5

0.5

0

00

0

Y

Y

Enzyme

Enzyme

Figure 10: Surface plot.

23

2.5

2

Y Y

1.5

1

0.5

3

2.5

2

1.5

1

0.5

1.5 1 0.5 0 -0.5 -1 -1.5 -2

2 1.5 1 0.5 0 -0.5 -1 -1.5 -2

Figure 11: Surface curvature.


John A. Wass.

Box-Behnken DesignsThese are three-level designs for fitting response surfaces. They are made by combining 2k factorials designs with incomplete block designs. They pos-sess the advantages of run efficiency (i.e., number of required runs and rotateability). This last property ensures equivalence of variance for all points in the design space equidistant from the center. The design is constructed by placing each factor, or independent variable, at one of three equally spaced values from the center. By design then, the estimated variances will depend upon the distance of the points from the center (Figure 2).

It is noticed that the Box-Behnken design has no points at the vertices or corners of the design space. This is a plus when it is desired to avoid those areas because of engineering constraints but unfortunately places a higher prediction uncertainty in those areas.

Central Composite DesignsCentral composite design (CCD) is a popular and effi-cient design for fitting a second-order model (Figure 3). The CCD most often consists of a 2k factorial n runs, 2k axial runs, and nc center runs. The experi-menter must specify the distance from the axial runs to the design center (this is done by many software platforms) and the number of center points. As it is desirable to have a reasonably stable variance at the points of interest, the design should have the property of rotateability.

CCD–An Example of the Response Surface TechniqueFor an example, the use of the central composite design is employed in yet another industrial chemi-cal process. The chemists are now concerned with a formulation that is only slightly different than that used in the screening design, and concentrations have changed due to knowledge gathered from the first series of experiments. We now wish to use this new knowledge to optimize the process. The experiment

Figure 12: Data table output.Reagent1 Reagent2 Enzyme Y

1 L2 L3 L1 2.1

2 L1 L2 L2 1

3 L1 L2 L1 0.8

4 L3 L3 L1 2

5 L1 L1 L3 1.7

6 L2 L1 L1 1.6

7 L2 L3 L3 2.4

8 L3 L2 L2 2.4

9 L3 L1 L2 2.2

10 L1 L2 L3 1.2

11 L3 L3 L2 2.5

12 L3 L2 L1 1.9

13 L2 L3 L2 2.2

14 L3 L1 L3 2.8

15 L3 L3 L3 3.1

Figure 13: Three reagents and a Reagent1* Enzyme interaction.

Summary of FitRSquare 0.967811

RSquare Adj 0.887338

Root Mean Square Error 0.21696

Mean of Response 1.993333

Observations (or Sum Wgts) 15

Analysis of VarianceSource DF Sum of

SquaresMean Square

F Ratio12.0265

Model 10 5.6610476 0.566105

Error 4 0.1882857 0.047071 Prob > F

C. Total 14 5.8493333 0.0143*

1 1.5 2 2.5 3

1

1.5

2

2.5

3

Y Predicted P=0.0143RSq=0.97 RMSE=0.217

Y A

ctu

al



is designed in JMP 9 software, then run, and the data appear as seen in Figure 4.

The design includes 28 runs with a center point as well as face centers on the design hypercube (there are more than three dimensions). When these data are analyzed, it is apparent that, while the model is significant (analysis of variance [ANOVA] p = 0.0275) and there is no significant lack of fit (p= 0.9803), the adjusted R2 is low (~0.41) and more terms might need to be added to the model (Figure 5).

There is some confidence in the integrity of the data as the plot of errors (by increasing Y value) shows randomness rather than any defining patterns that would indicate a problem (Figure 6).

It appears that from the sorted parameter estimates, what the chemists had long suspected was true (i.e., the importance of Reagent 1, the enzyme, and the reaction temperature to the output). See Figure 7. This is also seen on the normal plot as these three parameters are flagged (Figure 8).

The interaction profiles evidence of a (expected) putative interaction between the enzyme and reac-tion temperature, but this did not appear significant from the parameter estimates.

Use of such tools as prediction profilers will allow the optimization of the process, which in this case, seen by the position of the dotted red lines, is accomplished by maximizing the values of Reagent 1, enzyme, and temperature. Reagent 2 does not seem important in this respect (f lat slope) (Figure 9).

The design space is easily visualized on the surface plot, which corroborates the results of the profiler (Figure 10).

When the figure is rotated, the surface curvature is seen. It is apparent by the absence of lack of fit that the inclusion of interaction and quadratic terms in the model were justified (Figure 11).

Although there is good evidence that the model is adequate, the low R2 suggests that input values may need to change, or more likely, factors may need to be added. There are many ways to model this system. A custom design may simplify the system.

Custom DesigningThe chemists are comfortable with setting the tem-perature at a level that was determined as optimal in both theoretic calculations and previous batch experience. They now wish to zero in on concentra-tions for the two reagents and the enzyme. In most modern software with experimental design mod-ules, it is possible to “roll your own” custom design to non-standard specifications. Suppose the new data set consists of the two reagents and the enzyme, with the reaction temperature and mix speed held at the pre-determined optima. The software designs a 15-run experiment, and the analyst fills in the data table output (Figure 12).

The model is then analyzed with the three reagents and a Reagent1*Enzyme interaction as seen in Figure 13.

The adjusted R2 is now ~ 0.89 evidencing a good fit, and the ANOVA indicates a significant model

Figure 14: Effects test.Source Nparm DF Sum of Squares F Ratio Prob >F

Reagent1 2 2 2.5261587 26.8332 0.0048*

Reagent2 2 2 0.1583810 1.6823 0.2950

Enzyme 2 2 0.9055974 9.6194 0.0296*

Reagent1*Enzyme 4 4 0.1200593 0.6376 0.6632

0.2

0.1

0

1 1.5 2

Y Predicted

Y R

esid

ual

2.5 3

-0.1

-0.2

Figure 15: Residual by predicted plot.


John A. Wass.

(p=0.014). In linear regression with standard least squares fitting model, significance is a test of zero slope (i.e., a steep slope indicating good predictive ability). The effects test confirms that Reagent1 and the enzyme are most important to the reaction (Fig-ure 14). Chemistry may dictate that Reagent2 is also important, but any of the concentrations used in the experiment will work. The exact figure will be based on convenience and cost. Although there is no overall significance to the Reagent1*enzyme interaction, the enzyme is necessary. Opening up the range of concentrations used would demonstrate this.

Again, a quick check of data integrity is evidenced by the random pattern of the residual by predicted plot (Figure 15).

With this model, it is possible to produce an output that overlaps that of the larger, CCD model. And this was accomplished in just 15 runs (Figure 16).

The response surface here is a stair-step because of the linear nature of this model with non-contin-uous inputs. It again illustrates that maximal out-put is achieved through maximal concentrations of Reagent1 and the enzyme for the ranges used in this experiment (Figure 17).

Des

irab

ility

0.60

7334

Y3.

0785

71±

0.48

2975

Reagent1L3 L3 L3

3

2.5

2

1.5

1

0.75

1

L1 L2 L3 L1 L2 L3 L1 L2 L3 0

0.25 0.5

0.75 1

0.25

0

Reagent2 Enzyme Desirability

Figure 16: Prediction profiler.

3

2.5

2Y

Y1.5

1

0

3

L1

L2

L3

L1

L2

L3

L3

L3

L2L2

2.5

2

1.5

1

0

Reagent1

Reagen

t1

Enzyme

Enzyme

Figure 17: Maximal output.



SOFTWAREThere are numerous software products available to assist the practitioner in design and analysis of their experiments. The author has had experience with the following commercial packages:

• Design Expert (www.statease.com) • GenStat (www.vsni.co.uk) • JMP (www.jmp.com) • Minitab (www.minitab.com) • MODDE (www.umetrics.com) • STATISTICA (www.statsoft.com) • SYSTAT (www.systat.com) • Unscrambler (www.camo.no).

CONCLUSIONSModern experimental design is sometimes art as well as science. It is the objective of this column to acquaint the reader with the rudiments of the screen-ing and response surface designs, introduce them to the nomenclature, and supplement the learning experience with real-world examples.

REFERENCES1. S.R. Schmidt and R.G. Launsby, Understanding Indus-

trial Designed Experiments (4th ed.), Air Academy Press, 1997.

2. G.E.P. Box, J.S Hunter and, W.G. Hunter, Statistics for Experimenters (2nd Ed.), Wiley Interscience, 2005.

3. D. C. Montgomery, Design and Analysis of Experiments (5th ed.), John Wiley,2001.

4. JMP 9 Design of Experiments Guide, SAS Institute Inc., 2010.

5. ECHIP Reference Manual, Version6, ECHIP Inc., 1983-1993.

6. R.H. Myers and D.C. Montgomery, Response Surface Methodology, John Wiley, 1995. JVT

A Further Step in Experimental Design (III): The Response Surface

Documents

Transcript of A Further Step in Experimental Design (III): The Response Surface