Logic-Based Methods for Global Optimization
45
1 Logic-Based Methods for Global Optimization J. N. Hooker Carnegie Mellon University, USA November 2003
description
Logic-Based Methods for Global Optimization. J. N. Hooker Carnegie Mellon University, USA November 2003. Basic Idea Assume the problem becomes convex when certain variables are fixed. - PowerPoint PPT Presentation
Transcript of Logic-Based Methods for Global Optimization
1
Logic-Based Methods for Global Optimization
J. N. Hooker
Carnegie Mellon University, USA
November 2003
2
Basic Idea
• Assume the problem becomes convex when certain variables are fixed.
• If these variables are discrete, we can reformulate the problem as disjunctions of convex constraints.
• If some of them are continuous, discretize them to obtain an approximate global solution.
• Motivation is to take advantage of advanced solution methods:
• Branch-and-bound method chooses the appropriate disjunct in each constraint.
• Nonlinear programming method solves convex subproblems that result when disjuncts are chosen.
3
Outline
• General form of problem
• Structural design example
• Disjunctive formulation
• Branch and bound with convex relaxations
• Big-M formulation
• Convex hull formulation
• Logic-based outer approximation
• Logic-based Benders decomposition
• Branch and bound with convex quasi-relaxations
• Realistic structural design problem
• Solution by MILP
• Solution by quasi-relaxation
• Other applications
4
jjn
jj
Yyx
yL
Jjyxg
x
,
)(
,0),(subject to
min 0
General Form of Problem
If is continuous, discretize it, to get approximate global solution.
jy
Logical conditions on y
y yAssume that when is fixed to , we get a convex problem
convex functions of x
Vector of functions
nj
j
x
Jjyxg
x
,0),(subject to
min 0
5
jjn
jj
Yyx
yL
Jjyxg
x
,
)(
,0),(subject to
min 0
Objective is defined in the constraints
We assume one per constraint.
Many problems have this form. If not, constraints can in principle be put into this form by change of variable.
jy
6
For example, consider
}2,1,0{ˆ
ˆˆ
ˆˆ
32
21
jy
byyx
byyx
Use the change of variable
}4,,0{
ˆˆ
ˆˆ
322
211
jy
yyy
yyy
and the constraints have the desired form:
}4,,0{
221
2
1
jy
yy
byx
byx One yj per constraint
Logical condition
7
Structural Design Example
jy
1j
2j
= compression of bar jjx
2xdisplacement =
21 xx displacement =
= thickness (cross-sectional area) of bar j
22
22121 )(300300 xxxyy cost =
cost of steel penalty for displacement
load = 10
load = 20
Choose bar thickness that minimizes cost.
This example is intended only to illustrate the algorithms. A more realistic model for structural design is presented at the
end of the talk.
8
}2,1{
0
20
10subject to
)(300300min
22
11
22
22121
j
j
y
x
yx
yx
xxxyy
Global optimization problem:
Hooke’s law
or many closely spaced values for continuous problem
}2,1{0
0
20
3000
0
10
3000)(s.t.
min
22
22
11
11
022
22121
0
jj yxyx
zyyx
zyxxxxzz
x
Can be written in desired form:
displace-ment
thickness
11 ),,( yzxg
22 ),,( yzxg
9
jjn
jj
Yyx
yL
Jjyxg
x
,
)(
,0),(subject to
min 0
How to Solve It?
Can in principle use branch and bound by branching on .
But continuous relaxations at nodes of the search tree are in general nonconvex.
So we write the problem in disjunctive form.
jy
10
n
jj
Yv
x
yL
Jjvxg
vyx
j
)(
,0),(
subject to
min 0
Disjunctive Formulation
Now each disjunct is convex. We will solve by:
• Branch and bound with convex relaxations (use disjunctive programming or MINLP).
• Logic-based outer approximation with linear relaxations.
Relaxations can be large when there are many disjunctions. In this case consider:
• Logic-based Benders decomposition with discrete relaxation.
• Branch and bound with convex quasi-relaxations (requires that constraint functions satisfy certain properties).
11
}2,1{0
0
20
3000
0
010
300s.t.
)(min
22
22
11
11
22
22121
jj yxyx
zyyx
zyxxxzz
Recall the example…
Disjunctive formulation is:
00220
0600
2
020
0300
10210
0600
2
010
0300
1
subject to
)(min
2
2
2
2
2
2
1
1
1
1
1
1
22
22121
jxx
z
y
x
z
yx
z
y
x
z
yxxxzz
Disjuncts are convex (linear)
12
Branch and Bound with Convex Relaxations
• Two convex relaxations of a disjunction
• Big-M : Write a big-M formulation with 0-1 variables and take its continuous relaxation (i.e., drop the integrality requirement on the 0-1 variables).
• Convex hull : Write a convex hull formulation with 0-1 variables and take its continuous relaxation.
• Two solution options
• Disjunctive programming : branch on disjunctions.
• Mixed integer nonlinear programming (MINLP): branch on 0-1 variables.
• Optimal value of relaxation provides a lower bound that is used in a branch-and-bound scheme.
13
Big-M formulation of disjunction
0),( vxgYv
The disjunction:
Big-M formulation:
Yvvv
vv YvMvxg
}1,0{,1
),1(),(
Where Mv is a vector of valid upper bounds on the component functions of g(x,v).
It is assumed that x is bounded above and below.
To obtain relaxation, replace
}1,0{v with ]1,0[v
14
Relaxation…
10
11,31
91)2(
)1(91
21
22
21
22
21
xx
xx
xx
Projection is 112)2( 22
21
21 xxx
Example of big-M relaxation
1x
2x
1)2(1 22
21
22
21 xxxx
Disjunction…
15
0),( vxgYv
The disjunction:
Convex hull formulation of disjunctionStubbs & Mehrotra; Grossmann & Lee
Assume each g(x,v) is bounded as well as convex. Also assume xL x xU
Write every point in the relaxation as a convex combination of points satisfying the disjuncts
Yvvv
LU
vYv
vv
xxx
Yvvxg
xx
]1,0[,1
ˆ
,0),ˆ(
ˆ
Use change of variablev
vv xx ˆ
Yvvv
Uv
vLv
v
vYv
vv
Yvxxx
Yvvx
g
xx
]1,0[,1
,
,0,
ˆ
Nonconvex
16
Restore convexity by multiplying by v
This is a convex hull relaxation (i.e., projects onto convex hull in x-space).
But disaggregation of x adds many new variables.
To get 0-1 formulation, replace
convex
}1,0{vwith]1,0[v
Yvvv
Uv
vLv
v
vYv
vv
Yvxxx
Yvvx
g
xx
]1,0[,1
,
,0,
ˆ
Yvvv
Uv
vLv
v
v
v
Yv
vv
Yvxxx
Yvvx
g
xx
]1,0[,1
,
,0,
ˆ
17
Example of convex hull relaxation
1)2(1 22
21
22
21 xxxx
Disjunction…
1x
2x
10
0)1(341 12
222
212
221
211
2221
1211
2
1
xxx
xx
xx
xx
x
x
Convex hull relaxation…
18
Solve structural design example with big-M formulation
}1,0{
0220)1(1020
3006000300
0210)1(510
3006000300subject to
)(min
222
222
111
111
22
22121
j
xx
zz
xx
zz
xxxzz
Big-M formulation:
Disjunctive formulation:
00220
0600
2
020
0300
10210
0600
2
010
0300
1
subject to
)(min
2
2
2
2
2
2
1
1
1
1
1
1
22
22121
jxx
z
y
x
z
yx
z
y
x
z
yxxxzz
Solve by disjunctive programming or MINLP. Get optimal solution at the root node.
Thus
)0,1(),( 21
)2,1(),( 21 yy
19
Solve structural design example with convex hull formulation
}1,0{
)1(2020
)1(600300
)1(1010
)1(600300
subject to
)(min
222221
222221
112111
112111
2221
1211
2
1
2221
1211
2
1
22
22121
j
xx
zz
xx
zzxx
xx
x
x
zz
zz
z
zxxxzz
Convex hull formulation:
Disjunctive formulation:
00220
0600
2
020
0300
10210
0600
2
010
0300
1
subject to
)(min
2
2
2
2
2
2
1
1
1
1
1
1
22
22121
jxx
z
y
x
z
yx
z
y
x
z
yxxxzz
Solve by disjunctive programming or MINLP.
20
Logic-Based Outer ApproximationTürkay and Grossmann
Allows one to use linear relaxations. But one must solve a mixed integer linear programming (MILP) master problem repeatedly.
• Solve a master problem containing 1st-order approximations of the disjuncts to obtain a value for y.
• Solve with MILP, which uses linear relaxations.
• Solve the subproblem that results when y is fixed to this value, to get value for x.
• Compute 1st-order approximations about previously obtained values of x, y.
• Continue until value of master problem best value obtained in a subproblem so far.
• Begin with warm start by precomputing 1st-order approximations about several values of (x,y).
21
n
jj
Yv
x
yL
Jjvxg
vyx
j
)(
,0),(
subject to
min 0
Disjunctive formulation again:
n
kkjkjj
Yv
xyL
KkJjxxvxgvxg
vyx
j
),(
,,1,
,0))(,(),(
subject to
min 0
The master problem in iteration K + 1 is
where (xk,yk) are solutions from previous iterations.
n
Kj
x
Jjyxg
x
,0),(subject to
min 0
The nonlinear subproblem in iteration K is
22
Solve structural design example with logic-based outer approximation
Master problem:
Disjunctive formulation again:
00220
0600
2
020
0300
10210
0600
2
010
0300
10)(subject to
min
2
2
2
2
2
2
1
1
1
1
1
1
022
22121
0
jxx
z
y
x
z
yx
z
y
x
z
yxxxxzz
x
00220
0600
2
020
0300
10210
0600
2
010
0300
1,,1,0))(2(2
))((2)()(subject to
min
2
2
2
2
2
2
1
1
1
1
1
1
02221
11212
22
2121
0
j
kkk
kkkkkk
xx
z
y
x
z
yx
z
y
x
z
yKkxxxxx
xxxxxxxzz
x
Disjuncts already linear
23
Solve master problem as MILP (Big-M formulation):`
}1,0{
0220)1(1020
06000300
0210)1(510
06000300
,,1,0))(2(2
))((2)()(subject to
min
222
22
111
11
02221
11212
22
2121
0
j
kkk
kkkkkk
xx
zz
xx
zz
Kkxxxxx
xxxxxxxzz
x
For warm start, solve subproblem for 2 y’s:
y1 = (1,1), which yields x1 = (20,20)y2
= (2,2), which yields x2 = (5,10).
This results in the master problem:
}1,0{
0220)1(1020
06000300
0210)1(510
06000300
0)10(50)5(30325
0)20(120)20(802000subject to
min
222
22
111
11
02121
02121
0
j
xx
zz
xx
zz
xxxzz
xxxzz
x
24
Solve master problem and get
1375)0,1( 0 x
which implies )2,1(3 y
Subproblem solution is
1400)10,10( 03 xx
Next master problem is
}1,0{
0220)1(1020
06000300
0210)1(510
06000300
0)10(60)10(40500
0)10(50)5(30325
0)20(120)20(802000subject to
min
222
22
111
11
02121
02121
02121
0
j
xx
zz
xx
zz
xxxzz
xxxzz
xxxzz
x
Solution is 1400)0,1( 0 x
and the algorithm terminates with y = (1,2).
same
new
25
Logic-Based Benders DecompositionHooker and Ottosson
Can be useful when variables have a large number of discrete values, resulting in a large number of disjuncts.
Convergence can be slow.
• Solve a master problem for y.
• The master problem incompletely describes the projection of the original problem onto the y-space.
• Solve the subproblem that results when y is fixed to this value.
• Obtain Benders cut from inference dual of the subproblem.
• Add the cut to the master problem to rule out some solutions that are no better than the previous one.
• Continue until the master and subproblem converge in value.
• Best to have a warm start with “don’t-be-stupid” constraints involving y.
26
n
jj
Yv
x
yL
Jjvxg
vyx
j
)(
,0),(
subject to
min 0
Disjunctive formulation again:
n
jK
Kj
x
Jjyxg
x
)(,0),(subject to
min 0
The nonlinear subproblem in iteration K is
Lagrange multiplier
)(
,,1,subject to
min
0
0
yL
Kkxzyy
z
kkjjj
jk
The master problem in iteration K + 1 is
Optimal value = Kx0
Logical Benders cuts
27
Solve structural design example with logic-based Benders decomposition
Initial master problem:
}2,1{
subject to
min
21
jy
yy
z
One solution is zy )1,1(1
0020
0300010
03000)(subject to
min
2
2
1
1
022
22121
0
jxx
zx
zxxxxzz
x
Solve subproblem:
60
111
100
121
Corresponds to y1 = 1
Corresponds to y2 = 1
Lagrange multipliers
Subproblem solution is 190010 z
Don’t-be-stupid constraint
28
Since the master problem is0, 21
11
}2,1{
1900)1,1(
subject to
min
21
jy
zy
yy
z
Solution is zy )2,1(2
Continue in this fashion. Master problem in iteration 4 is:
}2,1{
1525)2,2(
1400)2,1(
1900)1,1(
subject to
min
21
jy
zy
zy
zy
yy
z
Solution is 1400)2,1(4 zy
The algorithm terminates with y = (1,2).
same
29
Branch and Bound with Convex Quasi-Relaxations
Does not require disjunctive formulation and is therefore useful when there are many discrete values.
But the constraint functions must have a certain form.
• Solve the problem by branch and bound.
• Obtain bounds from quasi-relaxations at each node.
Given problem P: Sxxf |)(min
a problem Q: Sxxf |)(min
is a quasi-relaxation of P if for any feasible solution x of P, there is a feasible solution x of Q with f (x ) f (x).
Thus one can obtain a valid lower bound by solving a quasi-relaxation.
30
jjnj
j
Yyx
Jjyxg
x
,
,0),(subject to
min 0
Consider the problem,
Theorem. Suppose that each is either
(a) convex [for (i,j) J1] or(b) concave in yj and homogeneous in x:
]1,0[for ),(),( jjij
ji yxgyxg
Then the following is a convex quasi-relaxation :
Jjxx
Jjxxx
Jjxxx
Jjxxx
Jjiyxgyxg
Jjiyyxg
x
jnjj
jj
Uj
jLj
Uj
jLj
Uj
jji
Lj
jji
Uj
Lj
ji
],1,0[,,
,
,)1()1(
,
),(,0,,
),(,0)1(,s.t.
min
21
21
2
12
211
0
),( jji yxg
[for (i,j) J2].
Suppose also that ULUL yyyxxx ,
31
Why?
Take any feasible solution of
jjnj
j
Yyx
JJjyxg
x
,
,0),(subject to
min
21
0
),( yx
To obtain a feasible solution of
do the following:
xxxx
yyy
jj
jj
Ujj
Ljjj
)1(,set
)1( that so ]1,0[ choose21
Then ),(),(,)1(,
),()1(),()1(,),(21 U
jjj
iLj
jji
Ujj
ji
Ljj
ji
Uj
jij
Lj
jij
Ujj
Ljj
ijj
ji
yxgyxgyxgyxg
yxgyxgyyxgyxg
concavity
homogeneity
Jjxx
Jjxxx
Jjxxx
Jjxxx
Jjiyxgyxg
Jjiyyxg
x
jnjj
jj
Uj
jLj
Uj
jLj
Uj
jji
Lj
jji
Uj
Lj
ji
],1,0[,,
,
,)1()1(
,
),(,0,,
),(,0)1(,s.t.
min
21
21
2
12
211
0
32
So we have a feasible solution of the quasi-relaxation with value that is less than or equal to (in fact equal to) that of the original problem.
convex, because gj(x,y) is convex
convex, because gj(x,y) is convex in x
Jjxx
Jjxxx
Jjxxx
Jjxxx
Jjiyxgyxg
Jjiyyxg
x
jnjj
jj
Uj
jLj
Uj
jLj
Uj
jji
Lj
jji
Uj
Lj
ji
],1,0[,,
,
,)1()1(
,
),(,0,,
),(,0)1(,s.t.
min
21
21
2
12
211
0
satisfied, by above argument
satisfied, by construction
satisfied, by construction
33
Solve continuous version of structural design example with quasi-relaxations
Original formulation:
Put in proper form:
}0.3,,1.0,0{
020
0300
010
0300
0)(s.t.
min
22
22
11
11
22
22121
0
jj yx
yx
zy
yx
zy
xxxzz
x
}0.3,,1.0,0{
30
2020,1010
2010,10
0
0
0)(300300s.t.
min
21
2
222
111
022
22121
0
j
j
y
y
ss
xx
yxs
yxs
xxxxyy
xconvex
Concave in yj & homogeneous in 1st argument (sj, xj)
Discretize
34
The quasi-relaxation is:
]1,0[,)1(
)1(20)1(20,2020
)1(10)1(10,1010
)1(20),20
)1(10),10
0
0
0)(300300s.t.
min
2221
1211
22
11
2222222121
1121211111
022
22121
0
jUjj
Ljjj
UL
UL
yyy
ss
ss
xx
xx
yxsyxs
yxsyxs
xxxxyy
x
Can now re-aggregate sj:
]1,0[,)1(
)1(20),20
)1(10),10
020
010
0)(300300s.t.
min
22
11
222221
112111
022
22121
0
jU
jL
jj
UL
UL
yyy
xx
xx
yxyx
yxyx
xxxxyy
x
35
Beginning of branch-and-bound tree
Root node
x0 = 1177.8 = (0.667,0.667)
y = (1,1)
x0 = 1322 = (0,0.667)
y = (1,1)
x0 =1900 = (0,0) y = (1,1)
feasible solution
y1[0,1]y1[1.1,3]
x0 = 1283 = (0.816,0.667)
y = (1.45,1)
y2[0,1] y2[1.1,3]
x0 =1352 = (0,0.816)
y = (1,1.45)
Total 63 nodes out of 3131 possible solutions.
Get y = (1.1, 2.0)with z0 = 1394.5
Global optimum isy = (1.126, 1.972)with z0 = 1394.1
36
Degree of freedom i
iLoad
Bar j
ij
jhLength
jvElongation
jyCross-
sectional area
Displacement ix
jj
Ujj
Lj
Ujj
Lj
ijiji
jiijj
jjjh
Ej
jjj
Yy
jxxx
jvvv
jvx
if
jfvy
yhc
j
j
all,
all,
all,cos
all,cos
all,subject to
min
Realistic Structural Design Problem
Hooke’s law
Equilibrium
Compatibility
Elongation bounds
Displacement bounds
37
Solution as MI(N)LP Ghattas and Grossmann
The disjunctive formulation is
jj
Ujj
Lj
Ujj
Lj
ijiji
jiijj
jjjkh
Ejkjj
k
Yy
jxxx
jvvv
jvx
if
jfvA
Ahczz
j
j
all,
all,
all,cos
all,cos
all,subject to
min
0
0
Since everything is linear, the big-M and convex hull formulations are linear.
Can solve as a mixed integer linear programming (MILP) problem.
Discrete sizes for bar j
38
}1,0{
all,
, all,
all,cos
all,cos
all,subject to
min
jk
Ujj
Lj
jkUjjkjk
Lj
i kjkiji
jiijj
jjkjkh
Ej k
jkjkjj
jxxx
kjvvv
jvx
if
jfvA
Ahc
j
j
The convex hull MILP model is
39
Solution with convex quasi-relaxationsBollapragada, Ghattas and Hooker
Check that the problem has the right form:
convex
Concave (linear) in yj and homogeneous in vj, fj
jj
Ujj
Lj
Ujj
Lj
ijiji
jiijj
jjjh
Ej
jjj
Yy
jxxx
jvvv
jvx
if
jfvy
yhc
j
j
all,
all,
all,cos
all,cos
all,subject to
min
40
]1,0[
all,)1(
all,
all,
all,)1()1(
all,
all,cos
all,cos
all,subject to
min
21
2
1
21
j
Ujj
Ljjj
jjj
Ujj
Lj
Ujjj
Ljj
Ujjj
Ljj
ijiji
jiijj
jvyvyh
Ej
jjj
jyyy
jvvv
jxxx
jvvv
jvvv
jvx
if
jf
yhc
jUjj
Lj
j
j
The quasi-relaxation is
41
10-bar cantilever truss
25-bar electrical transmission tower
Some problem instances
42
72-bar building
Use symmetries to help solve problem
43
Computational Results (seconds)
Problem Instance MILP(CPLEX)
Quasi-relaxation
10-bar cantilever truss11 discrete areas
1 load 1.3 0.3
1 load, wider stress bounds 1.6 0.3
1 load, wider stress bounds 2.6 1.2
1 load, wider stress bounds 2.6 1.4
2 loads 23.6 5.8
1 load + displacement bounds 1089.4 67.5
2 loads + displacement bounds
13743.9 1654.0
25-bar transmission tower
2 loads, 11 discrete areas 271.7 225.8
Building11 discrete areas2 loads
72 bars 12692.7 207.9
90 bars * 168.9
108 bars * 329.4
*No solution after 20 hours (72,000 seconds)
44
Applications of Logic-Based Methods for Nonlinear Global
Optimization
• Logic-based outer approximation applied to chemical processing network design
• Quesada and Grossmann 1992; Türkay and Grossmann 1996
• Disjunctive programming applied to chemical processing network design
• Vecchietti and Grossmann 1999
• Convex quasi-relaxations applied to truss structure design
• Bollapragada, Ghattas and Hooker 2001
• Disjunctive programming applied to tray placement in distillation columns
• Barttfeld, Aguire and Grossmann 2003
• Logic-based Benders decomposition applied to planning and scheduling (linear case)
• Hooker 2000, 2003.; Jain and Grossmann 2001
45
Surveys
• I. E. Grossmann, Review of nonlinear mixed-integer and disjunctive programming techniques for process systems engineering, Carnegie Mellon University, June 2001
• J. N. Hooker, Logic-Based Methods for Optimization, John Wiley & Sons, 2000.
