2007 : Exploiting Problem Structure in the MOSEK simplex optimizers (seattle 2007)

http://www.mosek.com

Exploiting Problem Structure in theMOSEK Simplex Optimizers.

Bo Jensen ∗

MOSEK ApS,Fruebjergvej 3, Box 16, 2100 Copenhagen,

Denmark.Email: [email protected]

INFORMS Annual Meeting Seattle Nov. 5, 2007

∗Erling D. Andersen

Introduction

2 / 27

What is MOSEK

IntroductionTopics

The simplexoptimizers

Exploiting problemdependent structurein the simplexoptimizer, exampleone:

Exploiting problemdependent structurein the simplexoptimizer, exampletwo:

Conclusions

3 / 27

n A software package for solving large-scale optimizationproblems.

n Solves linear, conic, and nonlinear convex problems.n Has mixed-integer capabilities.n Stand-alone as well as embedded.n Used to solve problems with up to millions of constraints

and variables.n Version 1 released in 1999.n Version 5 released July 2007.

Topics

IntroductionTopics




Conclusions

4 / 27

n The problem:(P ) min cT x

st Ax = b,x ≥ 0.

n The linear optimizers.

u Interior-point optimizer (Not main focus in this talk).u Simplex optimizer.

n Talk aim : Demonstrate by example the importantness ofusing model structure in the simplex optimizer.

The simplex optimizers

5 / 27

What makes a good simplex optimizer ?

Introduction

The simplexoptimizersWhat makes a goodsimplex optimizer ?MOSEKsimplex-overview



Conclusions

6 / 27

n Exploit sparsity (i.e. LU and FTRAN and BTRANroutines).

n Exploit problem dependent structure.n Choose right path (i.e. good pricing strategy).n Long steps (i.e. avoid degeneracy).n Numerical stability (i.e. reliable and consistent results).n Fast hotstarts (i.e. MIP and other hotstart applications).n Other tricks.

MOSEK simplex-overview

Introduction

The simplexoptimizersWhat makes a goodsimplex optimizer ?MOSEKsimplex-overview



Conclusions

7 / 27

n Primal and dual simplex optimizer.

u Efficient cold start and warm start.u Crashes an initial basis.u Multiple pricing options:

n Full (Dantzig).n Partial.n Approximate/exact steepest edge.n Hybrid.

u Degeneration handling.

n Revised simplex algorithm + many enhancements.n Many enhancements still possible!.

Exploiting problem dependent structure inthe simplex optimizer, example one:

8 / 27

Problem specifications ex : neos2 [HM:07]

Introduction


Exploiting problemdependent structurein the simplexoptimizer, exampleone:ProblemspecificationsProblem structureSolution times fromdifferent solversObservationsFirst improvement-generalSecondimprovement-Primalsimplex


Conclusions

9 / 27

Constraints :

n 132569 constraints.n 552596 non-zeroes in constraint matrix.n Only one constraint has a non-zero lower bound.n All upper bounds are infinity.

Variables :

n 1560 variables.n Only 77 variables has non-zero a cost.n All lower bounds are zero.n All upper bounds are infinity.

Problem structure

Introduction




Conclusions

10 / 27

Solution times from different solvers [HM:07]

Introduction




Conclusions

11 / 27

Non Commercial :

Problem Rows Cols Clp Qsopt Soplex GLPKneos1 131582 1892 113 987 497 261neos2 132569 1560 814 1634 746 > 30000neos3 512209 6624 31471 ? ? f

Commercial :

Problem Cplex (D) Cplex (P) Mosek 5 (D) Mosek 5 (P)neos1 18 930 39 8neos2 31 532 75 13neos3 6350 4046 7008 59

Warning : The numerical results displayed is only use to illustrate a specific simplex issue and should not be readas a benchmark, please see http://plato.la.asu.edu/bench.html or http://www.mosek.com/index.php?id=103for a benchmark on a larger problem set.

Why are the primal simplex so fast on these problems ?

Observations

Introduction




Conclusions

12 / 27

n High tall problem i.e. has many more constraints thanvariables.

n Few non-zero costs in both primal and dualformulation, potentially degeneracy problems.

n Dual simplex does quite well, but it is hard to explainneos3 times i.e. a very similar problem as neos1-2.

First improvement-general

Introduction




Conclusions

13 / 27

Solve dual formulation instead :

n Maximization.n 1560 constraints and 132569 variables.n Smaller basis dimension.n Only one non-zero cost.

Second improvement-Primal simplex

Introduction




Conclusions

14 / 27

Exploit the objective is defined by the level of one variable :0 ≤ xo ≤ inf where co < 0.

min coxo

st Ax = b,x ≥ 0.

Idea :

n High objective level in primal start solution ( even if itshould be infeasible ).

n Temporary impose a fake lower bound 0 < lo on xo.n If the new problem is primal infeasible lower lo.

Second improvement-Primal simplex-(continued)

Introduction




Conclusions

15 / 27

Advantages :

n Higher objective when feasibility is achieved.n Normally fast reoptimization from one adjustment to an-

other.n Avoids getting xo into the basis before late in the process.n Less degenerated.

Disadvantages :

n Primal feasibility is achieved later.

u Skip the process if phase one takes a ”long time”.

n If xo does not have an finite upper bound, how should wechoose lo ?

u Use presolve techniques to find bounds on xo.u Use other information to bound xo i.e dual feasible so-

lutions.u Skip the process if bounds on xo are not ”tight”.

Second improvement-Primal simplex-(continued)

Introduction




Conclusions

16 / 27

Solution times primal simplex :

Problem Without trick With trickIter Time Iter Time

neos1 5339310 1377 12064 3neos2 8878560 2240 8792 3neos3 939365 1969 22010 23

Run on a 2.33GHz Xeon Quad Core with 8GB RAM.Huge speed up !

Exploiting problem dependent structure inthe simplex optimizer, example two:

17 / 27

Problem specifications ex : rail4284 [BEAS:87]

Introduction



Exploiting problemdependent structurein the simplexoptimizer, exampletwo:ProblemspecificationsObservations basedon primal simplexPrimal restrictedpricing

Conclusions

18 / 27

Set covering problem :

(P ) min cT xst Ax ≥ 1,

x ≥ 0.

n 4284 constraints.n 1092610 variables.n 12372358 non-zeroes in constraint matrix (all ones).

Observations based on primal simplex

Introduction




Conclusions

19 / 27

n Problem is long and slim i.e. has many more variablesthan constraints (scheduling problems).

n Duals tend to be unstable i.e. changing a lot within fewiterations.

n Partial pricing does not work well.n Updated row is expensive (excludes steepest-edge (SE)).

What can we do ?

Primal restricted pricing

Introduction




Conclusions

20 / 27

Idea :

1. Choose a subset XSP of dual infeasible columns from (P ).

2. If XSP = ∅ then stop, the solution is optimal.3. Form a subproblem (SP ) with columns in XSP and

columns in the basis B.4. Optimize (SP ) with primal simplex.5. Reduce (SP ) to contain only columns in B.6. Goto 1.

Yes, this is just a special case of partial pricing !

Restricted pricing (continued)

Introduction




Conclusions

21 / 27

Advantages :

n Updated row for the subproblems is less expensive.n SE,ASE and Devex pricing is now an option.

Disadvantages :

n Induces some overhead.n On some problems there can be a small slow down.

Restricted pricing (continued)

Introduction




Conclusions

22 / 27

Mosek 5 WOR Mosek 5Problem Rows Cols Iter Time Iter Timerail582 582 55515 71905 9.11 6560 2.24rail2586 2586 920683 1748004 1605 73675 154.27rail4284 4286 1092610 ? > 6000 111363 420.41

Run on a 2.33GHz Xeon Quad Core with 8GB RAM.Improvements :

n Fewer iterations.n Faster solve.

See http://www.mosek.com/index.php?id=103 forbenchmarks between Mosek 4 and 5.

Conclusions

23 / 27

Conclusions

Introduction




ConclusionsConclusionsDifficultiesReferencesWant to know moreabout Mosek ?

24 / 27

n Exploiting problem dependent structure can speed upsolution times dramatically.

n Tricky sparsity and degeneracy issues often comes intoplay.

Difficulties

Introduction





25 / 27

n The modeler often has valuable model information whichthe optimizer has not.

n The modeler often has limited knowledge aboutadvanced simplex features.

n Hard to auto-detect and generalize.

References

Introduction





26 / 27

[HM:07] H.Mittelmann http://plato.la.asu.edu/bench.html

[BEAS:87] J.E.Beasley ”An algorithm for set covering problems”European Journal of Operational Research 31 (1987) 85-93.

Want to know more about Mosek ?

Introduction





27 / 27

Session Information :Wednesday Nov 07, 10:00 - 11:30.

Title :Joint Session ICS/OS: Recent Development in Software forLinear and Convex Optimization Problems.

Chair :Erling Andersen,CEO, MOSEK ApS, Fruebjergvej 3 Box 16,Copenhagen O 2100, Denmark, [email protected].

2007 : Exploiting Problem Structure in the MOSEK simplex optimizers (seattle 2007)

Technology

Transcript of 2007 : Exploiting Problem Structure in the MOSEK simplex optimizers (seattle 2007)