Optimal Blocking of Orthogonal Arrays in Designed ExperimentsPeter Goos
Optimal Blocking of Orthogonal Arrays in Designed ExperimentsPeter GoosIn collaboration with Eric Schoen and Bagus Sartono
• Factorial experiments- Treatments are described by combinations of factor levels- Interest is in main effects and two-factor interaction effects
• Experimental tests or runs need to be partitioned in blocks (due to different days, batches of raw material, …)• Block effects are treated as fixed• Experimenting is expensive, so we have small data !
Starting point
1. Many processes have sources of variability that are uncontrollable.
2. Examples are day-to-day variation, batch-to-batch variation, etc.
3. When experimenting, this leads to groups of observations.
4. The groups are called blocks. 5. The grouping variable (day, batch) is called a
blocking factor. 6. Responses within each group are more
homogeneous or similar than responses from different groups.
Experiments in blocks
• PART 1: the number of observations exceeds the number of main effects and two-factor interactions
- Vitaming stability experiment- 32 observations (8 blocks of size 4)- 64 observations (16 blocks of size 4)
• PART 2: the number of observations is too small to estimate all two-factor interaction effects
Outline
PART 1
The number of runs is big enough to estimate all main effects and two-factor
interactions.
Focus on 2-level factors
n ≥ 1 + k + k(k‒1)/2
• I have always advocated optimal design of experiments- Flexible in terms of numbers of runs- Different types of factors- Constraints on the factor levels- …- Implicitly assuming that `traditional designs’ do a
good job when the number of observations is a power of 2 or a multiple of 4
• Today, I start talking about situations where n is a power of 2, as well as the number of blocks
Context
• Vitamins degrade when exposed to light• Can be stabilized when embedded in a
special molecule, called a fatty molecule• Five different fatty molecules• Binding with sugar might help as well to
stabilize the vitamins• Experiment involving 6 two-level factors
Vitamin stability experiment
1. Boundedness with sugar.
2. Oil Red O.
3. Oxybenzone.
4. Beta Carotene.
5. Sulisobenzone
6. Deoxybenzone
Six factors
Fatty molecules
• Vitamins degrade when exposed to light• Can be stabilized when embedded in a
special molecule, called a fatty molecule• Five different fatty molecules• Binding with sugar might help as well to
stabilize the vitamins• Experiment involving 6 two-level factors
Vitamin stability experiment
• Vitamins degrade when exposed to light• Can be stabilized when embedded in a
special molecule, called a fatty molecule• Five different fatty molecules• Binding with sugar might help as well to
stabilize the vitamins• Experiment involving 6 two-level factors• There is day-to-day variation in the process
Vitamin stability experiment
• Vitamins degrade when exposed to light• Can be stabilized when embedded in a
special molecule, called a fatty molecule• Five different fatty molecules• Binding with sugar might help as well to
stabilize the vitamins• Experiment involving 6 two-level factors• There is day-to-day variation in the process
‒ 4 runs per day are possible‒ 8 days are available
Vitamin stability experiment
Vitamin stability experiment
• 6 two-level factors• 8 days of 4 runs or observations• 32 runs in total• model
- 6 main effects- 15 two-factor interaction effects- 1 intercept - 7 contrasts for the 8-level blocking factor - 29 parameters in total
Traditional design approach
Table 4B.3 in Wu & Hamada (26-1 design)
Traditional approach
• Design generator 6=12345 to choose 32 treatments or factor level combinations
• Block generators to arrange 32 runs in 8 blocks of 4 runs- B1 = 135- B2 = 235- B3 = 145
32-run orthogonal
design
Traditional approach• Perfect design for main effects
- Can be estimated independently, with maximum precision
- Estimates not affected by day-to-day variation- No variance inflation- No multicollinearity
• Not so for interaction effects- 12 of the 15 interactions can be estimated
independently, with maximum precision- 3 interaction effects (12, 34, 56) cannot be
estimated- Perfect collinearity with the blocks
Traditional approach (bis)
• Double the number of runs !• 64 instead of 32 runs• Full factorial design instead of half
fraction• 16 blocks of size 4• Table 3A in Wu & Hamada (2000)
Traditional approach (bis)
• Block generators to arrange 64 runs in 16 blocks of 4 runs- B1 = 136
- B2 = 1234
- B3 = 3456
- B4 = 123456
• We can estimate all two-factor interaction effects except 12, 34 and 56
Conclusion
The 64-run design is a waste of resources.
The traditional approach doesn’t work.
Semi-traditional approach
• 64 observations in 16 blocks of size 4• Do not start from full factorial design !• Instead, cleverly combine two half
fractions of 32 observations arranged in 8 blocks of size 4
First half fraction• Design generator 6=12345 to choose
32 treatments or factor level combinations
• Block generators to arrange 32 observations in 8 blocks of 4 runs- B1 = 135- B2 = 235- B3 = 145
• We can estimate all two-factor interaction effects except 12, 34 and 56
• This was the original idea
Second half fraction• Design generator 6=‒12345 to choose
32 treatments or factor level combinations
• Block generators to arrange 32 observations in 8 blocks of 4 runs- B1 = 135 124- B2 = 235 134- B3 = 145 125
• We can estimate all two-factor interaction effects except 23, 45 and 16
Semi-traditional approach
• Result is a full factorial design• From the first half of the experiment, we
cannot estimate 12, 34 and 56• But we can estimate these effects from
the second half• From the second half of the experiment,
we cannot estimate 23, 45 and 16• But we can estimate them from the first
half
Effect VIF Variance Effect VIF Variance Effect VIF VarianceX1 1 0.01563 X1*X3 1 0.01563 X2*X6 1 0.01563X2 1 0.01563 X1*X4 1 0.01563 X3*X4 2 0.03125X3 1 0.01563 X1*X5 1 0.01563 X3*X5 1 0.01563X4 1 0.01563 X1*X6 2 0.03125 X3*X6 1 0.01563X5 1 0.01563 X2*X3 2 0.03125 X4*X5 2 0.03125X6 1 0.01563 X2*X4 1 0.01563 X4*X6 1 0.01563
X1*X2 2 0.03125 X2*X5 1 0.01563 X5*X6 2 0.03125
Semi-traditional approach
Some similar scenarios• 5 two-level factors, 32 runs, 8 blocks of
size 4: better to use two (cleverly selected) half fractions than it is to use a full factorial design
• 6 two-level factors, 64 runs, 16 blocks of size 4: - better to use two 32-run half fractions
than to use a full factorial- but you can also combine a 32-run half
fraction with a 16-run quarter fraction !
Advice to experimenters
Do not trust tables in DOE textbooks !
Do not trust options for screening designs in your favorite software !
Advice to DOE textbook writers
Make clear that certain designs in the tables should not be used !
And describe the better alternatives.
Advice to DOE software developers
Make clear that certain screening design options should not be used !
And provide the better alternatives.
Advice to experimenters
Do not trust tables in DOE textbooks !
Do not trust options for screening designs in your favorite software !
Advice to experimenters
Throw away the DOE textbooks !
Do not trust options for screening designs in your favorite software !
Advice to experimenters
Throw away the DOE textbooks !
Use optimal design of experiments !
D-optimal design I
• Calculate a 64-run D-optimal design with 16 blocks of size 4
• Main effects + two-factor interactions• Really easy with SAS, JMP, Design
Expert, …• D-optimal design is 3% better than the
design produced by the semi-traditional approach
• Design is not orthogonally blocked
Effect VIF Variance Effect VIF Variance Effect VIF VarianceX1 1.3 0.01979 X1*X3 1.2 0.01812 X2*X6 1.2 0.01822X2 1.3 0.01971 X1*X4 1.2 0.01912 X3*X4 1.2 0.01932X3 1.2 0.01821 X1*X5 1.3 0.01969 X3*X5 1.2 0.01920X4 1.2 0.01822 X1*X6 1.2 0.01829 X3*X6 1.3 0.02059X5 1.1 0.01753 X2*X3 1.2 0.01819 X4*X5 1.2 0.01832X6 1.2 0.01815 X2*X4 1.2 0.01888 X4*X6 1.2 0.01836
X1*X2 1.1 0.01760 X2*X5 1.3 0.01964 X5*X6 1.1 0.01751
D-optimal design I
• D-optimality criterion: seeks designs that maximize determinant of information matrix
• Algorithms by Atkinson & Donev (1989) and Cook and Nachtsheim (1989)
• I used JMP’s coordinate-exchange algorithm
Optimal design
This is interesting …
… but it does not solve the original problem …
… which was to find a 32-run two-level design in 8 blocks of size 4 for estimating main effects and two-factor interaction
effects
D-optimal design II
• Calculate a 32-run D-optimal design with 8 blocks of size 4
• Main effects + two-factor interactions• Really easy with SAS, JMP, Design
Expert, …• All 2fis are estimable• Design is not orthogonally blocked• VIFs range from 1 to 2.6 only
Effect VIF Variance Effect VIF Variance Effect VIF VarianceX1 1.3 0.04016 X1*X3 1.0 0.03125 X2*X6 1.3 0.04129X2 1.2 0.03636 X1*X4 1.2 0.03636 X3*X4 1.2 0.03636X3 1.9 0.05788 X1*X5 1.0 0.03125 X3*X5 1.0 0.03125X4 1.4 0.04419 X1*X6 1.0 0.03125 X3*X6 1.6 0.05028X5 1.3 0.03961 X2*X3 1.4 0.04458 X4*X5 2.2 0.06878X6 1.6 0.05013 X2*X4 1.3 0.03961 X4*X6 1.3 0.04127
X1*X2 1.3 0.03918 X2*X5 2.6 0.08029 X5*X6 1.0 0.03125
D-optimal design II
Effect VIF Variance Effect VIF Variance Effect VIF VarianceX1 1.0 0.03125 X1*X3 1.0 0.03125 X2*X6 1.0 0.03125X2 1.0 0.03125 X1*X4 1.0 0.03125 X3*X4 inf infX3 1.0 0.03125 X1*X5 1.0 0.03125 X3*X5 1.0 0.03125X4 1.0 0.03125 X1*X6 1.0 0.03125 X3*X6 1.0 0.03125X5 1.0 0.03125 X2*X3 1.0 0.03125 X4*X5 1.0 0.03125X6 1.0 0.03125 X2*X4 1.0 0.03125 X4*X6 1.0 0.03125
X1*X2 inf inf X2*X5 1.0 0.03125 X5*X6 inf inf
Traditional design
43
Conclusion Part 1• 64 runs
‒ 64-run textbook design was beaten by manually constructed design
‒ manually constructed design was beaten by optimal design
• 32-run textbook design was beaten by optimal design
• So, optimal designs do a better job than classical designs even in scenarios that are ideal for classical designs
PART 2
The number of runs is not big enough to estimate all main effects and two-factor
interactions.
Optimal design approach not feasible since information matrix is singular in that case.
Factors with 2, 3 and 4 levels
45
Orthogonal arrays
• There exist many orthogonal arrays (OAs) that can be used as an experimental design
• 2-level arrays‒ Regular full and fractional factorial designs‒ Plackett-Burman designs‒ Other nonregular arrays
• 3-level arrays: regular full and fractional factorial designs, nonregular arrays
• Mixed-level arrays: not all factors have the same number of levels (e.g. Taguchi’s L18)
46
Strength-2 (or resolution-III) arrays
• Main effects can be estimated independently from each other
• But they are aliased with two-factor interactions• Using complete catalogs of OAs, we sought
optimal blocking patterns based on the concept of “generalized word-length pattern”‒ Orthogonal blocking for main effects‒ As little aliasing and confounding for two-
factor interactions as possible• We listed optimally blocked designs with 12, 16,
20, 24 and 27 runs
47
20 runs, eight 2-level factors, five blocks
Blocks 00001111222233334444 ------- ------------------------------- X1
00110011001100110011X2 00110101010101011100X3 00110110101011000101X4 00111001110010100110X5 01010011011011001010X6 01010101100110101001X7 01011010010101100101X8 01011100001110010110
48
27 runs, nine 3-level factors, nine blocks
Blocks 000111222333444555666777888-------- ------------------------------------------X1 012012012012012012012012012X2 012012012120120120201201201X3 012012012201201201120120120X4 012120201012120201012120201X5 012120201120201012201012120X6 012120201201012120120201012X7 012201120012201120120012201X8 012201120120012201012201120X9 012201120201120012201120012
49
Strength-3 (or resolution IV) arrays
• Main effects can be estimated independently‒ From each other‒ From two-factor interaction effects
• Two-factor interactions are aliased with each other
• Enumerating all possible blocking patterns for all OAs in catalogs was infeasible
• We used mixed integer linear programming instead to find blocking arrangements of good orthogonal arrays:‒ Orthogonally blocked for main effects‒ As little confounding between two-factor
interactions and blocks as possible
50
Mixed integer linear programming
• Input:‒ A good OA which allows estimation of many
two-factor interactions‒ Number of blocks required
• Output:‒ Optimal blocking pattern (orthogonal for the
main effects)‒ Tells you when it is infeasible to find such a
pattern• Implementations
‒ SAS/OR ‒ Matlab + CPLEX
51
40 runs, one 5-level factor, six 2-level factors, four blocks
0 0 0 0 1 1 1 10 1 0 1 1 0 0 11 1 0 0 1 0 0 11 1 1 0 0 1 1 12 0 1 1 0 1 1 12 1 0 1 0 0 0 13 0 0 1 1 1 0 13 0 1 0 0 1 1 14 0 1 0 0 0 0 14 1 1 1 1 0 1 1
---------------------------------------------------------0 0 1 1 1 1 0 20 1 1 0 1 0 1 21 1 0 1 1 1 1 21 1 1 1 0 0 0 22 0 0 0 0 1 0 22 0 0 1 1 0 1 23 0 1 1 0 0 0 23 1 0 0 0 1 0 24 0 1 0 1 1 1 24 1 0 0 0 0 1 2
---------------------------------------------------------0 0 0 0 0 0 0 30 1 0 1 0 1 1 31 0 0 1 0 1 0 31 0 1 1 1 0 1 32 0 1 0 1 0 0 32 1 1 0 0 0 1 33 0 0 0 1 0 1 33 1 1 1 1 1 1 34 1 0 0 1 1 0 34 1 1 1 0 1 0 3
---------------------------------------------------------0 0 1 1 0 0 1 40 1 1 0 0 1 0 41 0 0 0 0 0 1 41 0 1 0 1 1 0 42 1 0 0 1 1 1 42 1 1 1 1 1 0 43 1 0 1 0 0 1 43 1 1 0 1 0 0 44 0 0 1 0 1 1 44 0 0 1 1 0 0 4
52
Conclusion Part 2• Catalogs of orthogonal arrays offer a
starting point for designing blocked experiments
• For small numbers of observations, it is possible to completely enumerate all possible designs and select the best
• For larger numbers of observations, a mixed integer linear programming approach can be used to arrange an appropriate orthogonal array in blocks
Based on …
• Sartono, B., Goos, P., Schoen, E.D. (2014) Blocking Orthogonal Designs with Mixed Integer Linear Programming, Technometrics 56, to appear.
• Schoen E.D., Sartono B., Goos, P. (2013) Optimum blocking for general resolution-3 designs, Journal of Quality Technology 45, 166-187.
• Goos, P., Jones, B. (2011) Optimal Design of Experiments: A Case-Study Approach, Wiley.
Optimal Blocking of Orthogonal Arrays in Designed ExperimentsPeter GoosIn collaboration with Eric Schoen, Bagus Sartono and Nha Vo-Thanh
Top Related