Forming and Solving Optimization Problems Theory and Software Tools

CES 735/6

Imre Polik

Outline

Optimizationproblems

Modelling

Solvers

Modellingenvironments

Resources

Forming and Solving Optimization ProblemsTheory and Software Tools

Imre Polik, PhD

McMaster UniversitySchool of Computational Engineering and Science

February 14, 2008

CES 735/6

Imre Polik

Outline


Modelling

Solvers


Resources

Outline

1 Optimization problems

2 Modelling

3 Solvers

4 Modelling environments

CES 735/6

Imre Polik

Outline


Modelling

Solution

Problem input/output

Modelling

Solvers


Resources

Optimization problems

Linear (LP)

Convex quadratic (QP)

Piecewise linear, quadratic

Second order (SOCP)

Semidefinite (SDP)

Convex nonlinear (NLP)

Mixed integer, binary (MILP, MIQP)

Nonconvex quadratic

Quadratically constrained quadratic (QCQP)

Geometric (GP)

Nonlinear semidefinite (NLSDP)

General nonlinear (NLP)

Global optimization

CES 735/6

Imre Polik

Outline


Modelling

Solution


Modelling

Solvers


Resources

Choosing a solver

Identify the problem type

Decision Tree for Optimization Software

Requirements

accuracysolution timeproblem sizeplatform, programming environmentlicensing, cost

Choose the most specific solver

Reformulate the problem to use a better solver

Starting solution

CES 735/6

Imre Polik

Outline


Modelling

Solution


Modelling

Solvers


Resources

Entering the problem

Native solver input

usually text-basedsolver specific

Modelling environment

standard interface to many solverscustom GUI designautomatic derivativesplotting(Matlab), AMPL, GAMS, MPL, ILOG, XPRESS-IVE,AIMMS, LINDO, CVX, YALMIP

Spreadsheet application

Database integration

Callable library, embedded solvers

tighter integrationC, C++, Fortran, Java, .NET, C#, VB, Delphi,Python, (Matlab, R, etc.)

CES 735/6

Imre Polik

Outline


Modelling

Solution


Modelling

Solvers


Resources

Interpreting the solution

Solver exit flag

(nearly) optimalinfeasible problemnumerical problemslack of progress

Does it make sense?

better than previous solutiontoo good to be truemissing constraints, incorrect data

Optimality

the solution is rarely optimalthe role of convexity

Infeasible problem

certificateconflicting constraints

CES 735/6

Imre Polik

Outline


Modelling

Reformulating theproblem

Managing constraints

Solvers


Resources

Solvers


2 ModellingReformulating the problemManaging constraints

3 Solvers


CES 735/6

Imre Polik

Outline


Modelling



Solvers


Resources

Equalities and inequalities

Equalities into inequalities

h(x) = 0⇔ h(x) ≤ 0, h(x) ≥ 0

possibly empty interiorincreased problem size

h(x) = 0⇔ h(x) ≤ ε, h(x) ≥ −ε

nonempty interiorexplicit tolerance

Inequalities into equalities

g(x) ≤ 0⇔ g(x) + s = 0, s ≥ 0

slack variableincreased problem size

g(x) ≤ 0⇔ g(x) = −z2

nonlinearity

CES 735/6

Imre Polik

Outline


Modelling



Solvers


Resources

Managing constraints and variables

New variables

trigonometry

u, ±√

1− u2 ⇔ sinα, cos α

grouping common terms

Smart variables

extremely important in integer programming

Adding constraints gradually

only a few constraints are active at optimalitycolumn generationquick resolve

CES 735/6

Imre Polik

Outline


Modelling



Solvers


Resources

Relaxation

Remove some constraints

integralitysemidefiniteness

Solve the easier problem

LP instead of MILPSDP instead of NLSDP

Hope for the best

manage unsatisfied constraintsprovides a bound on the optimumbranch-and-bound

Powerful technique

CES 735/6

Imre Polik

Outline


Modelling

Solvers

Linear/quadraticoptimization

Conic optimization

Nonlinearoptimization

NEOS


Resources

Solvers


2 Modelling

3 SolversLinear/quadratic optimizationConic optimizationNonlinear optimizationNEOS


CES 735/6

Imre Polik

Outline


Modelling

Solvers


Conic optimization


NEOS


Resources

LP/MILP

CPLEX, XPRESS-MP

LP, MIP, QP, MIQPmodel development environmentcallable from C, C++, Java, Fortran, VB6 and .NETindustry leadersfree limited student version

LpSolve, Soplex, CLP

LP/MILPfree, open source

CES 735/6

Imre Polik

Outline


Modelling

Solvers


Conic optimization


NEOS


Resources

Conic optimization

LP, SOCP, SDP

CSDP, SDPA

written in Clarge scale problemsparallel, distributed operationno SOCP

SeDuMi, SDPT3

written in Matlabmixed LP/SOCP/SDPeasy to use

Others: DSDP, SDPLR, PENNON, SBMethod,

CES 735/6

Imre Polik

Outline


Modelling

Solvers


Conic optimization


NEOS


Resources

Nonlinear optimization

IPOPT

open source

SNOPT, KNITRO, MINOS, etc.

commercial

LGO, Baron, Globsol

global optimizationreasonable sizes

Issues

availability of gradient, Hessiancost of function evaluationstarting point

CES 735/6

Imre Polik

Outline


Modelling

Solvers


Conic optimization


NEOS


Resources

NEOS

Free online access to solvers

Various interfaces

webemailmodelling lenguages

Great for comparisons, testing

CES 735/6

Imre Polik

Outline


Modelling

Solvers


ILOG OPLDevelopment Studio

XPRESS-IVE

AIMMS

AMPL and GAMS

Matlab

CVX

YALMIP

Resources

Modelling environments


2 Modelling

3 Solvers

4 Modelling environmentsILOG OPL Development StudioXPRESS-IVEAIMMSAMPL and GAMSMatlabCVXYALMIP

CES 735/6

Imre Polik

Outline


Modelling

Solvers



XPRESS-IVE

AIMMS

AMPL and GAMS

Matlab

CVX

YALMIP

Resources

ILOG OPL Development Studio

Built around CPLEX

LP, QP, MILP, MIQP

Extensive GUI creation

Mapping tools

Integration with C++, Java

Spreadsheet and database integration

Embeddability

ILOG Optimization Decision Manager

planning scheduling

CES 735/6

Imre Polik

Outline


Modelling

Solvers



XPRESS-IVE

AIMMS

AMPL and GAMS

Matlab

CVX

YALMIP

Resources

XPRESS-IVE

Built around the XPRESS-MP solvers

LP, QP, MILP, MIQP

Integration with C++, Java

Spreadsheet and database integration

Embeddability

Abilities similar to ILOG OPL

CES 735/6

Imre Polik

Outline


Modelling

Solvers



XPRESS-IVE

AIMMS

AMPL and GAMS

Matlab

CVX

YALMIP

Resources

AIMMS

General nonlinear modelling environment

A handful of solvers (LP, MILP, QP, MIQP, NLP)

Graphical user interface

GUI building

Database, spreadsheet integration

Free student version

CES 735/6

Imre Polik

Outline


Modelling

Solvers



XPRESS-IVE

AIMMS

AMPL and GAMS

Matlab

CVX

YALMIP

Resources

AMPL and GAMS

General nonlinear modelling environments

Plenty of solvers

Free student version available

Text-based, no GUI (AMPL has a simple one)

No SDP

CES 735/6

Imre Polik

Outline


Modelling

Solvers



XPRESS-IVE

AIMMS

AMPL and GAMS

Matlab

CVX

YALMIP

Resources

Matlab

Powerful computational background

Optimization Toolbox (LP, QP, NLP)

Robust Control Toolbox (SDP)

Not embeddable

CES 735/6

Imre Polik

Outline


Modelling

Solvers



XPRESS-IVE

AIMMS

AMPL and GAMS

Matlab

CVX

YALMIP

Resources

CVX

Disciplined convex optimization

Maintain convexity by construction

Only two solvers (SeDuMi and SDPT3)

LP, QP, SOCP, SDP, GP

Written in Matlab

CES 735/6

Imre Polik

Outline


Modelling

Solvers



XPRESS-IVE

AIMMS

AMPL and GAMS

Matlab

CVX

YALMIP

Resources

YALMIP

Written in Matlab

Free, open source

Powerful modelling abilities

Covers all kinds of optimization problems

Connected to more than 30 solvers

CES 735/6

Imre Polik

Outline


Modelling

Solvers


Resources

Resources

NETLIB

Software repositorybenchmark problems

COIN-OR

Open source initiative for optimizationLP, MILP, QP, MIQP, SDP, NLPautomatic differentiation

Decision Tree for Optimization Software

Hans Mittelmann’s benchmarks

Forming and Solving Optimization Problems Theory and Software Tools

Documents

Transcript of Forming and Solving Optimization Problems Theory and Software Tools