Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical...
Transcript of Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical...
![Page 1: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/1.jpg)
Modeling with Mixed-Integer NonlinearOptimization
Summer School on Optimization of Dynamical Systems
Sven Leyffer and Jeff Linderoth
Argonne National Laboratory
September 3-7, 2018
![Page 2: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/2.jpg)
Course Outline: Mixed-Integer Nonlinear Optimization
Mixed-Integer Nonlinear Programming (MINLP)1 Monday, September 3: Two-Part Lecture
1 Modeling with Mixed-Integer Nonlinear Optimization2 Methods for Convex Mixed-Integer Nonlinear Optimization
2 Tuesday, September 4: Two-Part Lecture & Tutorial1 Advanced Methods for Convex MINLPs2 Methods for Nonconvex Mixed-Integer Nonlinear Optimization
3 Tutorial: 10:30am – Noon (starting with short intro)
Time permitting: Short transition to dynamical MINLPs ...
2 / 34
![Page 3: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/3.jpg)
Outline: Modeling with Mixed-Integer NonlinearOptimization
1 Problem Definition and Assumptions
2 MINLP Modeling Practices
3 A Short Introduction to AMPL
4 Model: Quadratic Uncapacitated Facility Location
3 / 34
![Page 4: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/4.jpg)
Mixed-Integer Nonlinear Optimization
Mixed-Integer Nonlinear Program (MINLP)
minimizex
f (x)
subject to c(x) ≤ 0x ∈ Xxi ∈ Z for all i ∈ I
X bounded polyhedral set, e.g. X = {x : l ≤ AT x ≤ u}f : Rn → R and c : Rn → Rm twice continuouslydifferentiable (sometimes convex)
I ⊂ {1, . . . , n} subset of integer variables
Relaxations satisfy a constraint qualification (technical)
4 / 34
![Page 5: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/5.jpg)
NP-Super Hard
Challenges of MINLP
Combines challenges of handling nonlinearitieswith combinatorial explosion of integer variables
The great watershed in op-timization isn’t between lin-earity and nonlinearity, butconvexity and nonconvexity.
- R. Tyrrell Rockafellar
If f and c are convexfunctions, then we havea convex MINLP
If f and c are notconvex, then we have anonconvex MINLP
5 / 34
![Page 6: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/6.jpg)
NP-Super Hard
Challenges of MINLP
Combines challenges of handling nonlinearitieswith combinatorial explosion of integer variables
The great watershed in op-timization isn’t between lin-earity and nonlinearity, butconvexity and nonconvexity.
- R. Tyrrell Rockafellar
If f and c are convexfunctions, then we havea convex MINLP
If f and c are notconvex, then we have anonconvex MINLP
5 / 34
![Page 7: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/7.jpg)
The Importance of Being Convex
2D Rastrigin Test Function
Not All Solvers Are Equal
Without convexity, many solversonly guarantee local optimality
Impact of Convexity
Nonconvex MINLPs are much harder to solve
May not be able to “prove” global optimality
Make sure you really need nonconvexity in your MINLP!
6 / 34
![Page 8: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/8.jpg)
The Importance of Being Convex
2D Rastrigin Test Function
Not All Solvers Are Equal
Without convexity, many solversonly guarantee local optimality
Impact of Convexity
Nonconvex MINLPs are much harder to solve
May not be able to “prove” global optimality
Make sure you really need nonconvexity in your MINLP!
6 / 34
![Page 9: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/9.jpg)
Mixed-Integer Nonlinear Optimization
Mixed-Integer Nonlinear Program (MINLP)
minimizex
f (x)
subject to c(x) ≤ 0x ∈ Xxi ∈ Z for all i ∈ I
MINLPs are NP-hard ... includes MILP, which are NP-hard,see [Kannan and Monma, 1978]
Worse: MINLP are undecidable, see [Jeroslow, 1973]:∃ quadratically constrained IP for which no computing devicecan compute the optimum for all problems in this class... but we’re OK if X is compact!
7 / 34
![Page 10: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/10.jpg)
MINLP Specializations
MISOCP: Mixed Integer Second Order Cone Program
Convex-MINLP with ci (x) = ‖Aix + bi‖2 − pTi x + qi
MIPP: Mixed Integer Polynomial Program: ci (x) polynomial
MIQP: Mixed Integer Quadratic Program
May be convex or nonconvex: ci (x) = xTQix + pTi + qiConvex MIQP is a special case of MISOCP, if Qi � 0If f is convex quadratic and c is an affine mapping, then thereare specialized algorithms for convex-MIQP
MILP: Mixed Integer Linear Program
Most efficient solvers: ci (x) = aTi x − bi
8 / 34
![Page 11: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/11.jpg)
MINLP Tree
MINLP
NonconvexMINLP
ConvexMINLP
NonconvexNLP
ConvexNLP
PolynomialOpt.
MI-Polynomial
Opt.MISOCP SOCP
NonconvexQP
NonconvexMIQP
ConvexMIQP
Convex QP
MILP LP
9 / 34
![Page 12: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/12.jpg)
Practical Complexity ' Is There Hope to Solve it?
The Leyffer-Linderoth-Luedtke (LLL) Measure of Complexity
Given a problem of class X with Y decision variables, what is thelargest value of Y for which Jim, Jeff, or Sven would be willing tobet $50 that a “state-of-the-art” solver could solve the problem?
Convex NonconvexProblem Class (X ) # Var (Y ) Problem Class (X ) # Var (Y )
MINLP 500 MINLP 100NLP 5× 104 NLP 100
MISOCP 1000 MIPP 150SOCP 105 PP 150MIQP 1000 MIQP 300
QP 5× 105 QP 300MILP 2× 104
LP 5× 107
10 / 34
![Page 13: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/13.jpg)
Practical Complexity ' Is There Hope to Solve it?
The Leyffer-Linderoth-Luedtke (LLL) Measure of Complexity
Given a problem of class X with Y decision variables, what is thelargest value of Y for which Jim, Jeff, or Sven would be willing tobet $50 that a “state-of-the-art” solver could solve the problem?
Convex NonconvexProblem Class (X ) # Var (Y ) Problem Class (X ) # Var (Y )
MINLP 500 MINLP 100NLP 5× 104 NLP 100
MISOCP 1000 MIPP 150SOCP 105 PP 150MIQP 1000 MIQP 300
QP 5× 105 QP 300MILP 2× 104
LP 5× 107
10 / 34
![Page 14: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/14.jpg)
Outline
1 Problem Definition and Assumptions
2 MINLP Modeling Practices
3 A Short Introduction to AMPL
4 Model: Quadratic Uncapacitated Facility Location
11 / 34
![Page 15: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/15.jpg)
MINLP Modeling Practices
Modeling plays a fundamental role in MILP see [Williams, 1999]... even more important in MINLP
MINLP combines integer and nonlinear formulations
Reformulations of nonlinear relationships can be convex
Interactions of nonlinear functions and binary variables
Sometimes we can linearize expressions
MINLP Modeling Preference
We prefer linear over convex over nonconvex formulations.
12 / 34
![Page 16: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/16.jpg)
Modeling with Integer Variables
Linderoth “Fundamental Theorem of Integer Variables”
97.238% of MINLPs have integer variables for two purposes only:
1 Binary variables to make a “multiple choice” selection
2 Binary indicator variables that turn on/off continuousvariables and/or constraints.
1 Multiple Choice Selection
y ∈ {D1,D2, . . . ,Dk}
where Di are discrete parameters (e.g. pipe diameters)
2 Indicator Variables
if yi = 1 then ci (x) ≤ 0, otherwise ci (x) ≤ ∞
13 / 34
![Page 17: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/17.jpg)
Modeling Multiple Choice Selection
Discrete Choices
We can model y ∈ {D1,D2, . . . ,Dk}, where Di discrete parametersas special ordered set (SOS) [Beale and Tomlin, 1970]
y =k∑
i=1
ziDi , 1 =k∑
i=1
zi , zi ∈ {0, 1}
Similarly linearize univariate functions f (z), z ∈ ZGeneralizes to higher dimensions, [Beale and Forrest, 1976]
Solvers detect SOS structure and use special branching rules
14 / 34
![Page 18: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/18.jpg)
Calculus of Logical Modeling: Indicator/Lookout Variables
1 Indicator variable: yi ∈ {0, 1} to force a constraint to hold
yi = 1 ⇒ ci (x) ≤ 0
can be modeled as
yi ∈ {0, 1} and ci (x) ≤ M(1− yi )
where M > 0 is known upper bound on c(x) for x ∈ X2 Lookout variable: yi ∈ {0, 1} is forced if a constraint holds
aTi x + bi ≤ 0 ⇒ yi = 1
can be modeled as
yi ∈ {0, 1} and aTi x + bi ≥ (m − ε)yi
where m > 0 lower bound on aTi x + bi for x ∈ X , and ε > 0... is tolerance tol (e.g. 10−4)
15 / 34
![Page 19: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/19.jpg)
Some Useful Nonlinear Variable Transformations
Design of multiproduct batch plant includes nonconvex terms∑j∈M
αjNjVjβj ; CiNj ≥ τij ;
∑i∈N
ψi
BiCi ≤ γ
where variables are upper case, parameters are Greek letters.
Introduce log-transform variables:
vj = ln(Vj), nj = ln(Nj), bi = ln(Bi ), ci = ln(Ci )
Transformed expressions are convex:∑j∈M
αjenj+βjvj , ci + nj ≥ ln(τij),
∑i∈N
ψieci−bi ≤ γ
16 / 34
![Page 20: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/20.jpg)
Linearization of Constraints
Assume x2 6= 0. A simple transformation (a constant parameter):
x1x2
= a ⇔ x1 = ax2
Linearization of bilinear terms x1x2 with:
Binary indicator variable x2 ∈ {0, 1}Variable upper bound: 0 ≤ x1 ≤ Ux2
... new variable x12 replaces x1x2 ... using constraints
0 ≤ x12 ≤ x2U and 0 ≤ x1 − x12 ≤ U(1− x2),
Proof: x12 ∈ {0, x1} follows from constraints.
17 / 34
![Page 21: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/21.jpg)
Never Multiply a Nonlinear Function by a Binary
Previous example generalizes to nonlinear functions: x2c(x1) ≤ 0
Warning
Never model on/off constraints by multiplying by a binary variable.
Three alternative approaches
Disjunctive programming, [Grossmann and Lee, 2003]
Perspective formulations, [Gunluk and Linderoth, 2012]
Big-M formulation (weak relaxations)
18 / 34
![Page 22: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/22.jpg)
Another Example of Bad Nonlinear Models
Warning
Never replace a binary variable by a nonlinear expression!
We can write a binary constraint, x ∈ {0, 1}, equivalently as
0 ≤ x ≤ 1 and x(1− x) = 0
or also as a complementarity constraint ...
0 ≤ x ⊥ x ≤ 1
... both are bad ... hide integrality from solvers!
19 / 34
![Page 23: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/23.jpg)
Outline
1 Problem Definition and Assumptions
2 MINLP Modeling Practices
3 A Short Introduction to AMPL
4 Model: Quadratic Uncapacitated Facility Location
20 / 34
![Page 24: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/24.jpg)
A Short Introduction to AMPL
AMPL: A Mathematical Programming Language
Algebraic modeling language for optimization
Three main model/instance components1 Model file (describes algebraic form of equations) *.mod2 Data file (describe data of the instance) *.dat3 Command file (optional: describe control sequence) *.ampl
Link to solvers (binaries compiled with AMPL-solver interface)
1 NLP Solvers: CONOPT, Knitro, LOQO, Minos, SNOPT2 MINLP Solvers: Baron, KNITRO3 MILP Solvers: GuRoBi, XPRESS
Other AMPL Solvers
Minotaur (Argonne) download and binaries (Linux/MacOS)https://wiki.mcs.anl.gov/minotaur/index.php/
Minotaur_Download
COIN-OR Project https://www.coin-or.org/projects/
21 / 34
![Page 25: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/25.jpg)
Collections of MINLP Test Problems
AMPL Collections of MINLP Test Problems
1 MacMINLP www.mcs.anl.gov/~leyffer/macminlp/
2 IBM/CMU collection egon.cheme.cmu.edu/ibm/page.htm
GAMS Collections of MINLP Test Problems
1 GAMS MINLP-world www.gamsworld.org/minlp/
2 MINLP CyberInfrastructure www.minlp.org/index.php
Solve MINLPs online on the NEOS server,www.neos-server.org/neos/
... and there are even a few CUTEr problems in SIF!
22 / 34
![Page 26: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/26.jpg)
Introduction to AMPL Modeling Language
We have a full (temporary) license of AMPL for the course
Optimization Problem
minx
exp(−x1) +3∑
i=2
x2i
s.t. x1 log(x2) + x32 ≥ 1
5 ≥ x1, x2, x3 ≥ 0
AMPL Formulation
var x{1..3} >=0, <=5; # ... variables
minimize # ... objective functn
f: exp(-x[1]) + sum{i in 2..3} x[i]^2;
subject to # ... constraints
con: x[1]*log(x[2]) + x[2]^3 >= 1;
Beware: x2 > 0 ... log(x2) undefined for x2 ≤ 0!
23 / 34
![Page 27: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/27.jpg)
Running & Trouble Shooting an AMPL Model
1 Create a *.mod model file (see file)2 Start ampl; load model (e.g. simple.mod); select solver:
ampl: reset; model simple.mod;
ampl: option solver ipopt;
ampl: solve;
3 Display the answer or trouble shoot
ampl: display _varname, _var.lb, _var, _var.ub;
ampl: display _conname, _con.lb, _con.body, _con.ub;
ampl: expand;
... list variable/constraint name, lower bnd, body, upper bnd
... shows all constraints and objective functions4 We forgot to ensure x2 > 0 so that log(x2) defined:
ampl: let x[2] := 1;
ampl: solve;
... assigns an initial value to x2 (different from default, 0).
24 / 34
![Page 28: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/28.jpg)
Running & Trouble Shooting an AMPL Model
1 Create a *.mod model file (see file)2 Start ampl; load model (e.g. simple.mod); select solver:
ampl: reset; model simple.mod;
ampl: option solver ipopt;
ampl: solve;
3 Display the answer or trouble shoot
ampl: display _varname, _var.lb, _var, _var.ub;
ampl: display _conname, _con.lb, _con.body, _con.ub;
ampl: expand;
... list variable/constraint name, lower bnd, body, upper bnd
... shows all constraints and objective functions
4 We forgot to ensure x2 > 0 so that log(x2) defined:
ampl: let x[2] := 1;
ampl: solve;
... assigns an initial value to x2 (different from default, 0).
24 / 34
![Page 29: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/29.jpg)
Running & Trouble Shooting an AMPL Model
1 Create a *.mod model file (see file)2 Start ampl; load model (e.g. simple.mod); select solver:
ampl: reset; model simple.mod;
ampl: option solver ipopt;
ampl: solve;
3 Display the answer or trouble shoot
ampl: display _varname, _var.lb, _var, _var.ub;
ampl: display _conname, _con.lb, _con.body, _con.ub;
ampl: expand;
... list variable/constraint name, lower bnd, body, upper bnd
... shows all constraints and objective functions4 We forgot to ensure x2 > 0 so that log(x2) defined:
ampl: let x[2] := 1;
ampl: solve;
... assigns an initial value to x2 (different from default, 0).
24 / 34
![Page 30: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/30.jpg)
Short Quiz (see geartrain.mod in MacMINLP)Consider the gear-train design problem for best matching gear ratio
minimizex
(1
6.931− x3x2
x1x4
)2
x ∈ Z4, 12 ≤ xi ≤ 60
Is the problem a convex or nonconvex MINLP?
Is there an equivalent but simpler formulation?
25 / 34
![Page 31: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/31.jpg)
Outline
1 Problem Definition and Assumptions
2 MINLP Modeling Practices
3 A Short Introduction to AMPL
4 Model: Quadratic Uncapacitated Facility Location
26 / 34
![Page 32: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/32.jpg)
Uncapacitated Facility Location
Problem introduced by Gunluk, Lee, and Weismantel (’07) andclasses of strong cutting planes derived
M: Facility
N: Customer
xij : percentage of customer j ∈ N demand met by facilityi ∈ M
zi = 1⇔ facility i ∈ M is built
Fixed cost for opening facility i ∈ M
Quadratic cost for meeting demand j ∈ N from facility i ∈ M
27 / 34
![Page 33: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/33.jpg)
Quadratic Uncapacitated Facility Location Problem
A very simple MIQP
z∗def= min
∑i∈M
cizi +∑i∈M
∑j∈N
qijx2ij
subject to
xij ≤ zi ∀i ∈ M,∀j ∈ N∑i∈M
xij = 1 ∀j ∈ N
xij ≥ 0 ∀i ∈ M, ∀j ∈ N
zi ∈ {0, 1} ∀i ∈ M
28 / 34
![Page 34: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/34.jpg)
Partial AMPL Model for Quadratic facility Location
Declare variables
var x{I,J} >= 0, <= 1; # ... % of customer i served by j
var z{I} binary; # ... z_i=1, iff facility i built
Declare objective function
minimize quadCost: sum{i in I} C[i]*z[i]
+ sum{i in I, j in J} Q[i,j]*x[i,j]^2;
Declare constraints
subject to
# ... only serve customers from open facilities
varUBD{i in I, j in J}: x[i,j] <= z[i];
# ... meet all customer’s demands
meetDemand{j in J}: sum{i in I} x[i,j] = 1;
Missing: Definition of sets I,j, and data Q, C!
29 / 34
![Page 35: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/35.jpg)
Base Case of Induction to “Linderoth Theorem”
Binary variables used as indicators: zi = 0⇒ xij = 0
If z = 1, then we need to model the epigraph of x2ij
Building on Years of MIP Expertise
Mixed Integer Linear Programmers carefully study simpleproblem structures for “good” formulations of problems
Goal: closely approximate convex hull of feasible points
... solve LP relaxation
Study structure of a special MINLP with indicator variables
30 / 34
![Page 36: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/36.jpg)
Base Case of Induction to “Linderoth Theorem”
Binary variables used as indicators: zi = 0⇒ xij = 0
If z = 1, then we need to model the epigraph of x2ij
Building on Years of MIP Expertise
Mixed Integer Linear Programmers carefully study simpleproblem structures for “good” formulations of problems
Goal: closely approximate convex hull of feasible points
... solve LP relaxation
Study structure of a special MINLP with indicator variables
30 / 34
![Page 37: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/37.jpg)
A Very Simple Structure
Rdef={
(x , y , z) ∈ R2 × B | y ≥ x2, 0 ≤ x ≤ uz}
z = 0⇒ x = 0, y ≥ 0
z = 1⇒ x ≤ u, y ≥ x2
x
y
z = 1
z
y ≥ x2
Deep Insights
conv(R) ≡ line connecting 0 to y = x2 in the z = 1 plane
31 / 34
![Page 38: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/38.jpg)
A Very Simple Structure
Rdef={
(x , y , z) ∈ R2 × B | y ≥ x2, 0 ≤ x ≤ uz}
z = 0⇒ x = 0, y ≥ 0
z = 1⇒ x ≤ u, y ≥ x2
x
y
z = 1
z
y ≥ x2
Deep Insights
conv(R) ≡ line connecting 0 to y = x2 in the z = 1 plane
31 / 34
![Page 39: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/39.jpg)
A Very Simple Structure
Rdef={
(x , y , z) ∈ R2 × B | y ≥ x2, 0 ≤ x ≤ uz}
z = 0⇒ x = 0, y ≥ 0
z = 1⇒ x ≤ u, y ≥ x2
x
y
z = 1
z
y ≥ x2
Deep Insights
conv(R) ≡ line connecting 0 to y = x2 in the z = 1 plane
31 / 34
![Page 40: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/40.jpg)
Characterization of Convex Hull
Deep Theorem #1
R ={
(x , y , z) ∈ R2 × B | y ≥ x2, 0 ≤ x ≤ uz}
conv(R) ={
(x , y , z) ∈ R3 | yz ≥ x2, 0 ≤ x ≤ uz , 0 ≤ z ≤ 1, y ≥ 0}
x2 ≤ yz , y , z ≥ 0 ≡
Second Order Cone Programming
∃ effective, robust algorithms for optimizing over conv(R)
32 / 34
![Page 41: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/41.jpg)
Characterization of Convex Hull
Deep Theorem #1
R ={
(x , y , z) ∈ R2 × B | y ≥ x2, 0 ≤ x ≤ uz}
conv(R) ={
(x , y , z) ∈ R3 | yz ≥ x2, 0 ≤ x ≤ uz , 0 ≤ z ≤ 1, y ≥ 0}
x2 ≤ yz , y , z ≥ 0 ≡
Second Order Cone Programming
∃ effective, robust algorithms for optimizing over conv(R)
32 / 34
![Page 42: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/42.jpg)
The Beauty of Cones
Remarkable Result for Convex MINLPs
All 333 convex problems in MINLPLIB2 can be represented withonly four types of cones (for x ∈ Rn) [Lubin et al., 2016]
1 Quadratic cones: ‖x‖22 ≤ x0, for x ∈ Rn
2 Rotated quadratic cones: 2x1x2 ≥(x23 + . . .+ x2n
)1/23 Power cones: |x |p ≤ x0 ... or `p-norms ‖x‖p ≤ x04 Exponential cones: e.g. ex ≤ x0
Advantages of Cones
Convex cones give strong relaxations (more later)
Build on “Disciplined Convex Modeling” [Grant et al., 2006]to assure convexity
Snag: Need interior-point solvers ... or use cutting planes!
33 / 34
![Page 43: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/43.jpg)
Teaching Points: Modeling MINLPs
Modeling MINLP
minimizex
f (x) subject to c(x) ≤ 0, x ∈ X , xi ∈ Z ∀ i ∈ I
Most binary variables used as indicator or lookout variables
Reformulation tricks are important ... linearization of x1x2 forx2 ∈ {0, 1}AMPL modeling language: convenient & intuitive
Tighter relaxations/formulations from Conic Programming
Use of perspective and conic formulations important
34 / 34
![Page 44: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/44.jpg)
Beale, E. and Tomlin, J. (1970).Special facilities in a general mathematical programming system for non- convexproblems using ordered sets of variables.In Lawrence, J., editor, Proceedings of the 5th International Conference onOperations Research, pages 447–454, Venice, Italy.
Beale, E. M. L. and Forrest, J. J. H. (1976).Global optimization using special ordered sets.Mathematical Programming, 10:52–69.
Grant, M., Boyd, S., and Ye, Y. (2006).Disciplined convex programming.In Global optimization, pages 155–210. Springer.
Grossmann, I. and Lee, S. (2003).Generalized convex disjunctive programming: Nonlinear convex hull relaxation.Computational Optimization and Applications, pages 83–100.
Gunluk, O. and Linderoth, J. T. (2012).Perspective reformulation and applications.In IMA Volumes, volume 154, pages 61–92.
Jeroslow, R. G. (1973).There cannot be any algorithm for integer programming with quadraticconstraints.Operations Research, 21(1):221–224.
Kannan, R. and Monma, C. (1978).On the computational complexity of integer programming problems.
34 / 34
![Page 45: Modeling with Mixed-Integer Nonlinear Optimization · 2018-09-03 · AMPL: A Mathematical Programming Language Algebraic modeling language for optimization Three main model/instance](https://reader033.fdocuments.us/reader033/viewer/2022042420/5f38490069dff7431151761e/html5/thumbnails/45.jpg)
In Henn, R., Korte, B., and Oettli, W., editors, Optimization and OperationsResearch,, volume 157 of Lecture Notes in Economics and MathematicalSystems, pages 161–172. Springer.
Lubin, M., Yamangil, E., Bent, R., and Vielma, J. P. (2016).Extended formulations in mixed-integer convex programming.In International Conference on Integer Programming and CombinatorialOptimization, pages 102–113. Springer.
Williams, H. P. (1999).Model Building in Mathematical Programming.John Wiley & Sons.
34 / 34