Kuhn Tucker Conditions - WU
Transcript of Kuhn Tucker Conditions - WU
Chapter 16
Kuhn Tucker Conditions
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 1 / 22
Constraint Optimization
Find the maximum of function
f (x, y)
subject tog(x, y) ≤ c, x, y ≥ 0
Example:Find the maxima of
f (x, y) = −(x− 5)2 − (y− 5)2
subject tox2 + y ≤ 9, x, y ≥ 0
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 2 / 22
Graphical Solution
For the case of two variables we can find a solution graphically.
1. Draw the constraint g(x, y) ≤ c in the xy-plain (feasible region).
2. Draw appropriate contour lines of objective function f (x, y).
3. Investigate which contour lines of the objective function intersectwith the feasible region.Estimate the (approximate) location of the maxima.
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 3 / 22
Example – Graphical Solution
1 2 3 4 5 6 7 8 9
1
2
3
4
5
6
7
8
9
maximum in (2.2, 4.3)
Maximum of f (x, y) = −(x− 5)2 − (y− 5)2
subject to g(x, y) = x2 + y ≤ 9, x, y ≥ 0.
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 4 / 22
Example – Graphical Solution
1 2 3 4 5 6 7 8 9
1
2
3
4
5
6
7
8
9
maximum in (1, 1)
Maximum of f (x, y) = −(x− 1)2 − (y− 1)2
subject to g(x, y) = x2 + y ≤ 9, x, y ≥ 0.
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 5 / 22
Constraint Optimization
Compute the maximum of function
f (x1, . . . , xn)
subject to
g1(x1, . . . , xn) ≤ c1...
gk(x1, . . . , xn) ≤ ck
x1, . . . , xn ≥ 0 (non-negativity constraint)
Optimization problem:
max f (x) subject to g(x) ≤ c and x ≥ 0.
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 6 / 22
Non-Negativity Constraint
Univariate function f with non-negativity constraint.
We find for the maximum x∗ of f :I x∗ is an interior point of the feasible region:
x∗ > 0 and f ′(x∗) = 0; orI x∗ is a boundary point of the feasible region:
x∗ = 0 and f ′(x∗) ≤ 0.
Summary:
f ′(x∗) ≤ 0, x∗ ≥ 0 and x∗ f ′(x∗) = 0
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 7 / 22
Non-Negativity Constraint
For the case of a multivariate function f (x) with non-negativityconstraints xj ≥ 0, we obtain such a condition for each of the variables:
fxj(x∗) ≤ 0, x∗j ≥ 0 and x∗j fxj(x
∗) = 0
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 8 / 22
Slack Variables
Maximizef (x1, . . . , xn)
subject to
g1(x1, . . . , xn) + s1 = c1...
gk(x1, . . . , xn) + sk = ck
x1, . . . , xn ≥ 0s1, . . . , sk ≥ 0 (new non-negativity constraint)
Lagrange function:
L̃(x, s, λ) = f (x1, . . . , xn) +k
∑i=1
λi(ci − gi(x1, . . . , xn)− si)
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 9 / 22
Slack Variables
L̃(x, s, λ) = f (x1, . . . , xn) +k
∑i=1
λi(ci − gi(x1, . . . , xn)− si)
Apply non-negativity conditions:
∂L̃∂xj≤ 0, xj ≥ 0 and xj
∂L̃∂xj
= 0
∂L̃∂si≤ 0, si ≥ 0 and si
∂L̃∂si
= 0
∂L̃∂λi
= 0 (no non-negativity constraint)
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 10 / 22
Elimination of Slack Variables
Because of∂L̃∂si
= −λi the second line is equivalent to
λi ≥ 0, si ≥ 0 and λisi = 0
Equations∂L̃∂λi
= ci − gi(x)− si = 0 imply si = ci − gi(x)
and consequently the second line is equivalent to
λi ≥ 0, ci − gi(x) ≥ 0 and λi(ci − gi(x)) = 0 .
Therefore there is no need of slack variables any more.
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 11 / 22
Elimination of Slack Variables
So we replace L̃ by Lagrange function
L(x, λ) = f (x1, . . . , xn) +k
∑i=1
λi(ci − gi(x1, . . . , xn))
Observe that
∂L∂xj
=∂L̃∂xj
and∂L∂λi
= ci − gi(x)
So the second line of the condition for a maximum now reads
λi ≥ 0,∂L∂λi≥ 0 and λi
∂L∂λi
= 0
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 12 / 22
Kuhn-Tucker Conditions
L(x, λ) = f (x1, . . . , xn) +k
∑i=1
λi(ci − gi(x1, . . . , xn))
The Kuhn-Tucker conditions for a (global) maximum are:
∂L∂xj≤ 0, xj ≥ 0 and xj
∂L∂xj
= 0
∂L∂λi≥ 0, λi ≥ 0 and λi
∂L∂λi
= 0
Notice that these Kuhn-Tucker conditions are not sufficient.(Analogous to critical points.)
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 13 / 22
Example – Kuhn-Tucker Conditions
Find the maximum of
f (x, y) = −(x− 5)2 − (y− 5)2
subject tox2 + y ≤ 9, x, y ≥ 0
Lagrange function:
L(x, y; λ) = −(x− 5)2 − (y− 5)2 + λ(9− x2 − y)
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 14 / 22
Example – Kuhn-Tucker Conditions
Lagrange function:
L(x, y; λ) = −(x− 5)2 − (y− 5)2 + λ(9− x2 − y)
Kuhn-Tucker Conditions:
(A) Lx = −2(x− 5)− 2λx ≤ 0(B) Ly = −2(y− 5)− λ ≤ 0(C) Lλ = 9− x2 − y ≥ 0
(N) x, y, λ ≥ 0
(I) xLx = −x(2(x− 5) + 2λx) = 0(I I) yLy = −y(2(y− 5) + λ) = 0(I I I) λLλ = λ(9− x2 − y) = 0
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 15 / 22
Example – Kuhn-Tucker Conditions
Express equations (I)–(I I I) as
(I) x = 0 or 2(x− 5) + 2λx = 0(I I) y = 0 or 2(y− 5) + λ = 0(I I I) λ = 0 or 9− x2 − y = 0
We have to compute all 8 combinations and check whether the resultingsolutions satisfy inequalities (A), (B), (C), and (N).I If λ = 0 (I I I, left), then by (I) and (I I) there exist four solutions
for (x, y; λ):
(0, 0; 0), (0, 5; 0), (5, 0; 0), and (5, 5; 0).
However, none of these points satisfiesall inequalities (A), (B), (C).Hence λ 6= 0.
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 16 / 22
Example – Kuhn-Tucker Conditions
If λ 6= 0, then (I I I, right) implies y = 9− x2.I If λ 6= 0 and x = 0, then y = 9 and because of (I I, right),
λ = −8. A contradiction to (N).I If λ 6= 0 and y = 0, then x = 3 and because of (I, right),
λ = 23 . A contradiction to (B).
I Consequently all three variables must be non-zero.Thus y = 9− x2 and λ = −2(y− 5) = −2(4− x2).Substituted in (I) yields 2(x− 5)− 4(4− x2)x = 0 and
x =√
11+12 ≈ 2.158 y = 12−
√11
2 ≈ 4.342λ =√
11− 2 ≈ 1.317The Kuhn-Tucker conditions are thus satisfied only in point
(x, y; λ) =(√
11+12 , 12−
√11
2 ;√
11− 2)
.
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 17 / 22
Kuhn-Tucker Conditions
Unfortunately the Kuhn-Tucker conditions are not necessary!
That is, there exist optimization problems where the maximum does notsatisfy the Kuhn-Tucker conditions.
maximum
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 18 / 22
Kuhn-Tucker Theorem
We need a tool to determine whether a point is a (global) maximum.
The Kuhn-Tucker theorem provides a sufficient condition:
(1) Objective function f (x) is differentiable and concave.
(2) All functions gi(x) from the constraints are differentiable andconvex.
(3) Point x∗ satisfy the Kuhn-Tucker conditions.
Then x∗ is a global maximum of f subject to constraints gi ≤ ci.
The maximum is unique, if function f is strictly concave.
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 19 / 22
Example – Kuhn-Tucker Theorem
Find the maximum of
f (x, y) = −(x− 5)2 − (y− 5)2
subject tox2 + y ≤ 9, x, y ≥ 0
The respective Hessian matrices of f (x, y) and g(x, y) = x2 + y are
H f =
(−2 00 −2
)and Hg =
(2 00 0
)
(1) f is strictly concave.
(2) g is convex.
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 20 / 22
Example – Kuhn-Tucker Theorem
H f =
(−2 00 −2
)and Hg =
(2 00 0
)
(1) f is strictly concave.
(2) g is convex.
(3) Point (x, y; λ) =(√
11+12 , 12−
√11
2 ;√
11− 2)
satisfy theKuhn-Tucker conditions.
Thus by the Kuhn-Tucker theorem, x∗ = (√
11+12 , 12−
√11
2 ) is themaximum we sought for.
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 21 / 22
Summary
I constraint optimizationI graphical solutionI Lagrange functionI Kuhn-Tucker conditionsI Kuhn-Tucker theorem
Josef Leydold – Mathematical Methods – WS 2021/22 16 – Kuhn Tucker Conditions – 22 / 22