1 OR II GSLM 52800. 2 Outline separable programming quadratic programming.
-
Upload
benedict-turner -
Category
Documents
-
view
226 -
download
2
Transcript of 1 OR II GSLM 52800. 2 Outline separable programming quadratic programming.
1
OR IIOR IIGSLM 52800GSLM 52800
2
OutlineOutline
separable programming
quadratic programming
3
Separable ProgramsSeparable Programs
a separable NLP if f and all gj are separable functions
0 xi i, a finite number1
( ) ( )n
i ii
f f x
x1
( ) ( )n
j ji ii
g g x
x
4
Idea of Separable ProgramIdea of Separable Program
min f(x), s.t. gj(x) 0 for j = 1, …, m. hard NLP but simple LP problems approximating a separable NL
program by a LP a non-linear function by a piecewise
linear one
5
A Fact About Convex FunctionsA Fact About Convex Functions
f: a convex function for any > 0, possible to find a
sequence of piecewise linear convex functions fn
such that |f fn|
6
Example 6.1Example 6.1
a separable program
21 1 2
1 2
1 2
1 2
min 2 ,
. . 2 5,
2x 9,
, 0.
x x x
s t x x
x
x x
7
Example 6.1Example 6.1
approximating by a piecewise linear function
two representations, -form and -form
21 1 2
1 2
1 2
1 2
min 2 ,
. . 2 5,
2x 9,
, 0.
x x x
s t x x
x
x x
21x
points O A B C
x1 0 2 4 4.5
y 0 4 16 20.25
1
C
B
A
O
20.25
16
9
4
4321
8
FormForm
a piecewise linear function with (segment) break points
any point = the convex combination of the two break points of the linear segment
i ( 0) = the weight of break point i
1
C
B
AO
20.2516
94
4321
1 0 2 4 4.5 ,
0 4 16 20.25 ,
1,
at most two adjacent take non-zero values.
O A B C
O A B C
O A B C
i
x
y
9
Example 6.1Example 6.1
the program becomes
1 2
1 2
1 2
1
1 2
min 2 ,
. . 2 5,
2 9,
0 2 4 4.5 ,
0 4 16 20.25 ,
1,
at most two adjacent take non-zero values,
, , ,
O A B C
O A B C
O A B C
i
O
y x x
s t x x
x x
x
y
y x x
, , , 0. A B C
the last but one type of constraints is non-linear
21 1 2
1 2
1 2
1 2
min 2 ,
. . 2 5,
2x 9,
, 0.
x x x
s t x x
x
x x
10
FactFact
nonlinear constraint: at most two adjacent i taking non-zero values possible to have only one i = 1
for convex f and gj: no need to have the non-linear constraint non-optimal to have more than two non-zero i, or
two i not adjacent
11
FactFact
non-optimal to have more than two non-zero i, or two i not adjacent
e.g., f being an objective function any convex combination between two non-adjacent break
points being above the piecewise non-linear function
similarly, the point for three or more non-zero I’s lying above the piecewise non-linear function
think about A = 0.3, B = 0.4, and C = 0.3
1
C
B
AO
20.25
16
94
4321
12
FactFact
non-optimal to have more than two non-zero i, or two i not adjacent
e.g., gj being a constraint
gj(0.3A+0.7B) gj(0.3A+0.7C) bj the feasible set of {0.3A+0.7B} is larger than that by {0.3A+0.7C} the solution from {0.3A+0.7C} cannot be minimum
similar argument for three or more non-zero i’s lying above the piecewise non-linear function
C
B
AO
13
Example 6.1Example 6.1
the program becomes a linear program
1 2
1 2
1 2
1
1 2
min 2 ,
. . 2 5,
2 9,
0 2 4 4.5 ,
0 4 16 20.25 ,
1,
, , , , , , 0.
O A B C
O A B C
O A B C
O A B C
y x x
s t x x
x x
x
y
y x x
14
Example 6.2: Non-Convex ProblemExample 6.2: Non-Convex Problem
min f(x),
s.t. 1 x 3.
approximating f(x) by a piecewise linear function y = 0 + 10A + 6B
x = 0 + 2A + 3B
min 10 6 ,
. . 2 3 3,
2 3 1,
1,
at most two adjacent 's can be positive at the same time,
, , 0.
O A B
O A B
O A B
O A B
O A B
s t
15
Example 6.2: Non-Convex ProblemExample 6.2: Non-Convex Problem
adding slack variable s, surplus variable u, and artificial variable a1 and a2:
1
2
min 10 6 ,
. . 2 3 3,
2 3 1,
+ 1,
at most two adjacent 's can be positive at the
O A B
O A B
O A B
O A B
s t s
u a
a
1 2
same time,
, , , , , 0. O A B s a a
16
Example 6.2: Non-Convex ProblemExample 6.2: Non-Convex Problem
1
2
min 10 6 ,
. . 2 3 3,
2 3 1,
+ 1,
at most two adjacent 's can be positive at the
O A B
O A B
O A B
O A B
s t s
u a
a
1 2
same time,
, , , , , 0. O A B s a a
17
Example 6.2: Non-Convex ProblemExample 6.2: Non-Convex Problem
18
Example 6.2: Non-Convex ProblemExample 6.2: Non-Convex Problem
most negative 0
B in basis only A qualified to enter, not O
19
Example 6.2: Non-Convex ProblemExample 6.2: Non-Convex Problem
20
Example 6.2: Non-Convex ProblemExample 6.2: Non-Convex Problem
21
Example 6.2: Non-Convex ProblemExample 6.2: Non-Convex Problem
22
FormForm
again, the last constraint is unnecessary for a convex program
1 2 2 0.5 ,
4 12 4.25 ,
0 , , 1,
there exists > 0 such that = 1 for and = 0 for .
OA AB BC
OA AB BC
OA AB BC
j i i
x
y
i j i j
1
C
B
AO
20.25
16
94
4321
23
Quadratic ProgrammingQuadratic Programming
24
Quadratic Objective Function Quadratic Objective Function & Linear Constraints & Linear Constraints
Langrangian function
T T12
min ( ) + ,
. . ,
f
s t
x c x x Qx
Ax b x 0.
T T12
( , ) + L x c x x Qx + (Ax b)
25
KKT KKT ConditionsConditions
positive definite Q a convex program
a unique global minimum
the KKT sufficient
otherwise, KKT necessary
26
KKT KKT ConditionsConditions
cT + xTQ + TA 0 Qx + A y = c
Ax b 0 Ax + v = b
xT(c + Qx + A) = 0 xTy = 0
T(Ax b) = 0 Tv = 0
x 0, 0, y 0, v 0
solving the set of equations phase-1 of a linear program
27
Example 7.1Example 7.1(Example 10.14 of JB)(Example 10.14 of JB)
2 21 2 1 2
1 2
1
1 2
min ( ) 8 16 4 ,
. . 5,
3,
, 0.
f x x x x
s t x x
x
x x
x
8
16
c2 0
0 8
Q
1 1
1 0
A
5
3
b
28
Example 7.1Example 7.1(Example 10.14 of JB)(Example 10.14 of JB)
KKT conditions
2x1 + 1 + 2 y1 = 8,
8x2+ 1 y2 = 16,
x1+ x2 + v1 = 5,
x1 + v2 = 3.
x1y1 = x2y2 = 1v1 = 2v2 = 0
x1, y1, x2, y2, 1, v1, 2, v2 0
29
Example 7.1Example 7.1(Example 10.14 of JB)(Example 10.14 of JB)
30
Example 7.1(Example 10.14 of JB)