Math Models of OR: The Klee-Minty Cube

John E. Mitchell

Department of Mathematical SciencesRPI, Troy, NY 12180 USA

September 2018

Mitchell The Klee-Minty Cube 1 / 23

Visiting all extreme points

Outline

1 Visiting all extreme points

2 The Klee-Minty cube in IR3

3 The iterations

4 Alternative pivot rules

5 Extending to IRn

6 Average performance

Mitchell The Klee-Minty Cube 2 / 23

Visiting all extreme points

Worst-case performance of simplex

The simplex algorithm proceeds from one extreme point to aneighboring extreme point that is at least as good.

How many extreme points can it visit?

The Klee-Minty cube is an example where it visits every extremepoint. It’s a problem that is expressed in terms of n inequalityconstraints on n variables, together with nonnegativity constraints. Thenumber of extreme points is exponential in the size of the problem, 2n.

Thus, simplex has what is called exponential runtime in the worst case.Polynomial runtime (for example, 4n3 iterations) is far preferable: itgrows far more slowly than exponential runtime.

For large scale problems, linear run time (for example, 3n iterations) isdesirable.

Mitchell The Klee-Minty Cube 3 / 23

The Klee-Minty cube in IR3

Outline

1 Visiting all extreme points

2 The Klee-Minty cube in IR3

3 The iterations

4 Alternative pivot rules

5 Extending to IRn

6 Average performance

Mitchell The Klee-Minty Cube 4 / 23

The Klee-Minty cube in IR3

The Klee-Minty cube in 3 dimensions

Consider the linear program

minx∈IR3 −100x1 − 10x2 − x3subject to x1 ≤ 1

20x1 + x2 ≤ 100200x1 + 20x2 + x3 ≤ 10000

x1, x2, x3 ≥ 0

Mitchell The Klee-Minty Cube 5 / 23

The Klee-Minty cube in IR3

The Klee-Minty cube

x1

x2

x3

0 64

(0,0,0)(0,100,0)

(0,0,10000)

(1,0,0)

(1,80,0)

(0,100,8000)

(1,80,8200)

(1,0,9800)

Mitchell The Klee-Minty Cube 6 / 23

The iterations

Outline

1 Visiting all extreme points

2 The Klee-Minty cube in IR3

3 The iterations

4 Alternative pivot rules

5 Extending to IRn

6 Average performance

Mitchell The Klee-Minty Cube 7 / 23

The iterations

Pivot rule

The variable that enters the basis is the nonbasic variable with themost negative reduced cost.

There are no ties in the minimum ratio for this example, so it is notnecessary to specify the tie-breaking mechanism.

Mitchell The Klee-Minty Cube 8 / 23

The iterations

Get initial canonical form

After introducing slack variables, we have a problem in canonical formso we can proceed directly with simplex. The initial basis consists ofthe slack variables, denoted x4, x5, x6.

↓ratio x1 x2 x3 x4 x5 x6

0 −100 −10 −1 0 0 01 1 1© 0 0 1 0 05 100 20 1 0 0 1 0

50 10000 200 20 1 0 0 1

Mitchell The Klee-Minty Cube 9 / 23

The iterations

After one iteration

R0+100R1,R2−20R1,R3−200R1−→

↓ratio x1 x2 x3 x4 x5 x6

100 0 −10 −1 100 0 0− 1 1 0 0 1 0 080 80 0 1© 0 −20 1 0

490 9800 0 20 1 −200 0 1

Mitchell The Klee-Minty Cube 10 / 23

The iterations

After two iterations

R0+10R2,R3−20R2−→

↓ratio x1 x2 x3 x4 x5 x6

900 0 0 −1 −100 10 01 1 1 0 0 1© 0 0− 80 0 1 0 −20 1 041 8200 0 0 1 200 −20 1

Mitchell The Klee-Minty Cube 11 / 23

The iterations

After three iterations

R0+100R1,R2+20R1,R3−200R1−→

↓ratio x1 x2 x3 x4 x5 x6

1000 100 0 −1 0 10 0− 1 1 0 0 1 0 0− 100 20 1 0 0 1 0

8000 8000 −200 0 1© 0 −20 1

Mitchell The Klee-Minty Cube 12 / 23

The iterations

After four iterations

R0+R3−→

↓ratio x1 x2 x3 x4 x5 x6

9000 −100 0 0 0 −10 11 1 1© 0 0 1 0 05 100 20 1 0 0 1 0− 8000 −200 0 1 0 −20 1

Mitchell The Klee-Minty Cube 13 / 23

The iterations

After five iterations

R0+100R1,R2−20R1,R3+200R1−→

↓ratio x1 x2 x3 x4 x5 x6

9100 0 0 0 100 −10 1− 1 1© 0 0 1 0 080 80 0 1 0 −20 1 0− 8200 0 0 1 200 −20 1

Mitchell The Klee-Minty Cube 14 / 23

The iterations

After six iterations

R0+10R2,R3+20R2−→

↓ratio x1 x2 x3 x4 x5 x6

9900 0 10 0 −100 0 11 1 1 0 0 1© 0 0− 80 0 1 0 −20 1 0− 9800 0 20 1 −200 0 1

Mitchell The Klee-Minty Cube 15 / 23

The iterations

After seven iterations

R0+100R1,R2+20R1,R3+200R2−→

x1 x2 x3 x4 x5 x610000 100 10 0 0 0 1

1 1 0 0 1 0 0100 20 1 0 0 1 0

10000 200 20 1 0 0 1

This is optimal, with value −10000 and x1 = 0, x2 = 0, x3 = 10000.

The number of pivots required was 7 = 23 − 1.

Every extreme point was visited.

Mitchell The Klee-Minty Cube 16 / 23

The iterations

All the iterations

x1

x2

x3

(0,0,0)(0,100,0)

(0,0,10000)

(1,0,0)

(1,80,0)

(0,100,8000)

(1,80,8200)

(1,0,9800)

Mitchell The Klee-Minty Cube 17 / 23

Alternative pivot rules

Outline

1 Visiting all extreme points

2 The Klee-Minty cube in IR3

3 The iterations

4 Alternative pivot rules

5 Extending to IRn

6 Average performance

Mitchell The Klee-Minty Cube 18 / 23

Alternative pivot rules

Alternative pivot rules

If the best improvement rule had been used to choose the incomingvariable, x3 would have entered the basis on the first iteration and theproblem would have been solved in one step.

A variant of the Klee-Minty cube has been designed that also requiresexponentially many iterations in the worst-case, even with the bestimprovement rule.

Other variants with exponential worst-case performance have beendesigned for all known pivot rules.

Mitchell The Klee-Minty Cube 19 / 23

Extending to IRn

Outline

1 Visiting all extreme points

2 The Klee-Minty cube in IR3

3 The iterations

4 Alternative pivot rules

5 Extending to IRn

6 Average performance

Mitchell The Klee-Minty Cube 20 / 23

Extending to IRn

Extending to n variables

minx∈IRn −∑n

j=1 10n−jxj

subject to 2∑i−1

j=1 10i−jxj + xi ≤ 100i−1 i = 1, . . . ,nxj ≥ 0 j = 1, . . . ,n

The simplex method choosing the most negative entry to enter thebasis requires 2n − 1 iterations to solve this problem.

Mitchell The Klee-Minty Cube 21 / 23

Average performance

Outline

1 Visiting all extreme points

2 The Klee-Minty cube in IR3

3 The iterations

4 Alternative pivot rules

5 Extending to IRn

6 Average performance

Mitchell The Klee-Minty Cube 22 / 23

Average performance

Average performance of simplex

There is theoretical analysis showing that the average number ofiterations is linear in the size of the linear optimization problem.

(Need to make assumptions about the distribution of instances.)

This is also the typical performance in practice.

Mitchell The Klee-Minty Cube 23 / 23