Post on 28-Sep-2020
Math Models of OR: The Klee-Minty Cube
John E. Mitchell
Department of Mathematical SciencesRPI, Troy, NY 12180 USA
September 2018
Mitchell The Klee-Minty Cube 1 / 23
Visiting all extreme points
Outline
1 Visiting all extreme points
2 The Klee-Minty cube in IR3
3 The iterations
4 Alternative pivot rules
5 Extending to IRn
6 Average performance
Mitchell The Klee-Minty Cube 2 / 23
Visiting all extreme points
Worst-case performance of simplex
The simplex algorithm proceeds from one extreme point to aneighboring extreme point that is at least as good.
How many extreme points can it visit?
The Klee-Minty cube is an example where it visits every extremepoint. It’s a problem that is expressed in terms of n inequalityconstraints on n variables, together with nonnegativity constraints. Thenumber of extreme points is exponential in the size of the problem, 2n.
Thus, simplex has what is called exponential runtime in the worst case.Polynomial runtime (for example, 4n3 iterations) is far preferable: itgrows far more slowly than exponential runtime.
For large scale problems, linear run time (for example, 3n iterations) isdesirable.
Mitchell The Klee-Minty Cube 3 / 23
The Klee-Minty cube in IR3
Outline
1 Visiting all extreme points
2 The Klee-Minty cube in IR3
3 The iterations
4 Alternative pivot rules
5 Extending to IRn
6 Average performance
Mitchell The Klee-Minty Cube 4 / 23
The Klee-Minty cube in IR3
The Klee-Minty cube in 3 dimensions
Consider the linear program
minx∈IR3 −100x1 − 10x2 − x3subject to x1 ≤ 1
20x1 + x2 ≤ 100200x1 + 20x2 + x3 ≤ 10000
x1, x2, x3 ≥ 0
Mitchell The Klee-Minty Cube 5 / 23
The Klee-Minty cube in IR3
The Klee-Minty cube
x1
x2
x3
0 64
(0,0,0)(0,100,0)
(0,0,10000)
(1,0,0)
(1,80,0)
(0,100,8000)
(1,80,8200)
(1,0,9800)
Mitchell The Klee-Minty Cube 6 / 23
The iterations
Outline
1 Visiting all extreme points
2 The Klee-Minty cube in IR3
3 The iterations
4 Alternative pivot rules
5 Extending to IRn
6 Average performance
Mitchell The Klee-Minty Cube 7 / 23
The iterations
Pivot rule
The variable that enters the basis is the nonbasic variable with themost negative reduced cost.
There are no ties in the minimum ratio for this example, so it is notnecessary to specify the tie-breaking mechanism.
Mitchell The Klee-Minty Cube 8 / 23
The iterations
Get initial canonical form
After introducing slack variables, we have a problem in canonical formso we can proceed directly with simplex. The initial basis consists ofthe slack variables, denoted x4, x5, x6.
↓ratio x1 x2 x3 x4 x5 x6
0 −100 −10 −1 0 0 01 1 1© 0 0 1 0 05 100 20 1 0 0 1 0
50 10000 200 20 1 0 0 1
Mitchell The Klee-Minty Cube 9 / 23
The iterations
After one iteration
R0+100R1,R2−20R1,R3−200R1−→
↓ratio x1 x2 x3 x4 x5 x6
100 0 −10 −1 100 0 0− 1 1 0 0 1 0 080 80 0 1© 0 −20 1 0
490 9800 0 20 1 −200 0 1
Mitchell The Klee-Minty Cube 10 / 23
The iterations
After two iterations
R0+10R2,R3−20R2−→
↓ratio x1 x2 x3 x4 x5 x6
900 0 0 −1 −100 10 01 1 1 0 0 1© 0 0− 80 0 1 0 −20 1 041 8200 0 0 1 200 −20 1
Mitchell The Klee-Minty Cube 11 / 23
The iterations
After three iterations
R0+100R1,R2+20R1,R3−200R1−→
↓ratio x1 x2 x3 x4 x5 x6
1000 100 0 −1 0 10 0− 1 1 0 0 1 0 0− 100 20 1 0 0 1 0
8000 8000 −200 0 1© 0 −20 1
Mitchell The Klee-Minty Cube 12 / 23
The iterations
After four iterations
R0+R3−→
↓ratio x1 x2 x3 x4 x5 x6
9000 −100 0 0 0 −10 11 1 1© 0 0 1 0 05 100 20 1 0 0 1 0− 8000 −200 0 1 0 −20 1
Mitchell The Klee-Minty Cube 13 / 23
The iterations
After five iterations
R0+100R1,R2−20R1,R3+200R1−→
↓ratio x1 x2 x3 x4 x5 x6
9100 0 0 0 100 −10 1− 1 1© 0 0 1 0 080 80 0 1 0 −20 1 0− 8200 0 0 1 200 −20 1
Mitchell The Klee-Minty Cube 14 / 23
The iterations
After six iterations
R0+10R2,R3+20R2−→
↓ratio x1 x2 x3 x4 x5 x6
9900 0 10 0 −100 0 11 1 1 0 0 1© 0 0− 80 0 1 0 −20 1 0− 9800 0 20 1 −200 0 1
Mitchell The Klee-Minty Cube 15 / 23
The iterations
After seven iterations
R0+100R1,R2+20R1,R3+200R2−→
x1 x2 x3 x4 x5 x610000 100 10 0 0 0 1
1 1 0 0 1 0 0100 20 1 0 0 1 0
10000 200 20 1 0 0 1
This is optimal, with value −10000 and x1 = 0, x2 = 0, x3 = 10000.
The number of pivots required was 7 = 23 − 1.
Every extreme point was visited.
Mitchell The Klee-Minty Cube 16 / 23
The iterations
All the iterations
x1
x2
x3
(0,0,0)(0,100,0)
(0,0,10000)
(1,0,0)
(1,80,0)
(0,100,8000)
(1,80,8200)
(1,0,9800)
Mitchell The Klee-Minty Cube 17 / 23
Alternative pivot rules
Outline
1 Visiting all extreme points
2 The Klee-Minty cube in IR3
3 The iterations
4 Alternative pivot rules
5 Extending to IRn
6 Average performance
Mitchell The Klee-Minty Cube 18 / 23
Alternative pivot rules
Alternative pivot rules
If the best improvement rule had been used to choose the incomingvariable, x3 would have entered the basis on the first iteration and theproblem would have been solved in one step.
A variant of the Klee-Minty cube has been designed that also requiresexponentially many iterations in the worst-case, even with the bestimprovement rule.
Other variants with exponential worst-case performance have beendesigned for all known pivot rules.
Mitchell The Klee-Minty Cube 19 / 23
Extending to IRn
Outline
1 Visiting all extreme points
2 The Klee-Minty cube in IR3
3 The iterations
4 Alternative pivot rules
5 Extending to IRn
6 Average performance
Mitchell The Klee-Minty Cube 20 / 23
Extending to IRn
Extending to n variables
minx∈IRn −∑n
j=1 10n−jxj
subject to 2∑i−1
j=1 10i−jxj + xi ≤ 100i−1 i = 1, . . . ,nxj ≥ 0 j = 1, . . . ,n
The simplex method choosing the most negative entry to enter thebasis requires 2n − 1 iterations to solve this problem.
Mitchell The Klee-Minty Cube 21 / 23
Average performance
Outline
1 Visiting all extreme points
2 The Klee-Minty cube in IR3
3 The iterations
4 Alternative pivot rules
5 Extending to IRn
6 Average performance
Mitchell The Klee-Minty Cube 22 / 23
Average performance
Average performance of simplex
There is theoretical analysis showing that the average number ofiterations is linear in the size of the linear optimization problem.
(Need to make assumptions about the distribution of instances.)
This is also the typical performance in practice.
Mitchell The Klee-Minty Cube 23 / 23