Optimization Methods Lecture 1 - University of California...
Transcript of Optimization Methods Lecture 1 - University of California...
![Page 1: Optimization Methods Lecture 1 - University of California ...solmaz.eng.uci.edu/Teaching/MAE206/Lecture1.pdf · Optimization Methods Lecture 1 Solmaz S. Kia MechanicalandAerospaceEngineeringDept.](https://reader030.fdocuments.us/reader030/viewer/2022040905/5e79e68ef888964dc34f0521/html5/thumbnails/1.jpg)
Optimization MethodsLecture 1
Solmaz S. KiaMechanical and Aerospace Engineering Dept.
University of California [email protected]
Material to review: pages 1-15 of Ref[1] and Chapter 1 of Ref [2].
1 / 15
![Page 2: Optimization Methods Lecture 1 - University of California ...solmaz.eng.uci.edu/Teaching/MAE206/Lecture1.pdf · Optimization Methods Lecture 1 Solmaz S. Kia MechanicalandAerospaceEngineeringDept.](https://reader030.fdocuments.us/reader030/viewer/2022040905/5e79e68ef888964dc34f0521/html5/thumbnails/2.jpg)
Optimization problems
Examples of optimization problems:
- profit or loss in business setting
- speed, distance or fuel consumption in physical problems
- expected return in risky environment
- social welfare problems in the context of government planning
Optimization theory providesa suitable framework for analyzing and providing solution
2 / 15
![Page 3: Optimization Methods Lecture 1 - University of California ...solmaz.eng.uci.edu/Teaching/MAE206/Lecture1.pdf · Optimization Methods Lecture 1 Solmaz S. Kia MechanicalandAerospaceEngineeringDept.](https://reader030.fdocuments.us/reader030/viewer/2022040905/5e79e68ef888964dc34f0521/html5/thumbnails/3.jpg)
Mathematical models of optimization
Elements of optimization problem: A constraint set X and a cost function f thatmaps elements of X into a real numbers
X: The set of constraints of the available decisions x
f(x): the cost is a scalar measure of undesirability of choosing decision x
Objective: find an optimal decision x? ∈ X such that f(x?) 6 f(x) for ∀x ∈ X
minimize f(x) s.t.
x ∈ Xor x? = argmin f(x) s.t.
x ∈ X
In our studies X ⊆ Rn (i.e., x ∈ Rn)
3 / 15
![Page 4: Optimization Methods Lecture 1 - University of California ...solmaz.eng.uci.edu/Teaching/MAE206/Lecture1.pdf · Optimization Methods Lecture 1 Solmaz S. Kia MechanicalandAerospaceEngineeringDept.](https://reader030.fdocuments.us/reader030/viewer/2022040905/5e79e68ef888964dc34f0521/html5/thumbnails/4.jpg)
Mathematical models of optimization
Different types of optimization: depends on f(x) and X
Continuous vs. DiscreteContinuous problems: Constraint set X is infinite and has “continuous”characterExamples: X = Rn
X = {x ∈ R2 | x2 > x21, x1 + x2 6 2}
Tools to analyze: Mathematics of calculus and convexity
Discrete problems: mostly because constraint set X is finite
Examples:
routingschedulingMatching
Important class of discrete problems: integer programing (decision value fromsome range of integer numbers such as {0, 1})
Tools to analyze: Combinational and discrete mathematics; Other specialmethodology that relate to continuous problems
4 / 15
![Page 5: Optimization Methods Lecture 1 - University of California ...solmaz.eng.uci.edu/Teaching/MAE206/Lecture1.pdf · Optimization Methods Lecture 1 Solmaz S. Kia MechanicalandAerospaceEngineeringDept.](https://reader030.fdocuments.us/reader030/viewer/2022040905/5e79e68ef888964dc34f0521/html5/thumbnails/5.jpg)
Mathematical models of optimization
Different types of optimization: depends on f(x) and X
Nonlinear programmingcost f is nonlinear and/or
X is specified by nonlinear equations and inequalities
Linear programmingcost f is linear
X is specified by linear inequality constraints
5 / 15
![Page 6: Optimization Methods Lecture 1 - University of California ...solmaz.eng.uci.edu/Teaching/MAE206/Lecture1.pdf · Optimization Methods Lecture 1 Solmaz S. Kia MechanicalandAerospaceEngineeringDept.](https://reader030.fdocuments.us/reader030/viewer/2022040905/5e79e68ef888964dc34f0521/html5/thumbnails/6.jpg)
Our focus in this class
Our focus: nonlinear programing for continuous optimization problems.
x? =argminx∈Rn
f(x) s.t.
hi(x) = 0, i ∈ {1, · · · ,q}gi(x) > 0, i ∈ {1, · · · ,p}
f,h,g: continuously differentiable function of xe.g., f ∈ C1 continuously differentiablee.g., f ∈ C2 both f and its first derivative are continuously differentiable
6 / 15
![Page 7: Optimization Methods Lecture 1 - University of California ...solmaz.eng.uci.edu/Teaching/MAE206/Lecture1.pdf · Optimization Methods Lecture 1 Solmaz S. Kia MechanicalandAerospaceEngineeringDept.](https://reader030.fdocuments.us/reader030/viewer/2022040905/5e79e68ef888964dc34f0521/html5/thumbnails/7.jpg)
Continuous Optimization: Examples
Economic Dispatch
minimize f(p) = f1(p1) + · · ·+ fN(pN)
subject to pmini6 pi 6 pmaxi
i ∈ {1, · · · ,N}
p1 + · · ·+ pN = Demand
Economic dispatch with storage and transmission loss7 / 15
![Page 8: Optimization Methods Lecture 1 - University of California ...solmaz.eng.uci.edu/Teaching/MAE206/Lecture1.pdf · Optimization Methods Lecture 1 Solmaz S. Kia MechanicalandAerospaceEngineeringDept.](https://reader030.fdocuments.us/reader030/viewer/2022040905/5e79e68ef888964dc34f0521/html5/thumbnails/8.jpg)
Continuous Optimization: Examples
Economic Dispatch
minimize f(p) = f1(p1) + · · ·+ fN(pN)
subject to pmini6 pi 6 pmaxi
i ∈ {1, · · · ,N}
p1 + · · ·+ pN = Demand
Central Operation Distributed Operation
8 / 15
![Page 9: Optimization Methods Lecture 1 - University of California ...solmaz.eng.uci.edu/Teaching/MAE206/Lecture1.pdf · Optimization Methods Lecture 1 Solmaz S. Kia MechanicalandAerospaceEngineeringDept.](https://reader030.fdocuments.us/reader030/viewer/2022040905/5e79e68ef888964dc34f0521/html5/thumbnails/9.jpg)
Continuous Optimization: Examples
Unconstraint optimization
x? =argminx∈Rn
f(x)
Example:
x? =argminx∈R2
(x1 − 2)2 + (x2 − 1)2︸ ︷︷ ︸f(x)
9 / 15
![Page 10: Optimization Methods Lecture 1 - University of California ...solmaz.eng.uci.edu/Teaching/MAE206/Lecture1.pdf · Optimization Methods Lecture 1 Solmaz S. Kia MechanicalandAerospaceEngineeringDept.](https://reader030.fdocuments.us/reader030/viewer/2022040905/5e79e68ef888964dc34f0521/html5/thumbnails/10.jpg)
Continuous Optimization: Examples
Example:
x? =argminx∈Rn
f(x) s.t.
hi(x) = 0, i ∈ {1, · · · ,q}gi(x) > 0, i ∈ {1, · · · ,p}
x? =argminx∈R2
(x1 − 2)2 + (x2 − 1)2︸ ︷︷ ︸f(x)
s.t.
gi :
{−x21 + x2 > 0
−x1 − x2 + 2 > 0
10 / 15
![Page 11: Optimization Methods Lecture 1 - University of California ...solmaz.eng.uci.edu/Teaching/MAE206/Lecture1.pdf · Optimization Methods Lecture 1 Solmaz S. Kia MechanicalandAerospaceEngineeringDept.](https://reader030.fdocuments.us/reader030/viewer/2022040905/5e79e68ef888964dc34f0521/html5/thumbnails/11.jpg)
Local and global minima of an unconstrained optimization problem
x? ∈ Rn is an unconstrained local minimum of f if
∃ ε > 0 s.t. f(x?) 6 f(x), ∀x with ‖x− x?‖ < ε,
x? ∈ Rn is an unconstrained global minimum of f if
f(x?) 6 f(x), ∀x ∈ Rn,
x? ∈ Rn is an unconstrained strict local minimum of f if
∃ ε > 0 s.t. f(x?) < f(x), ∀x with ‖x− x?‖ < ε,
x? ∈ Rn is an unconstrained strict global minimum of f if
f(x?) < f(x), ∀x ∈ Rn,
11 / 15
![Page 12: Optimization Methods Lecture 1 - University of California ...solmaz.eng.uci.edu/Teaching/MAE206/Lecture1.pdf · Optimization Methods Lecture 1 Solmaz S. Kia MechanicalandAerospaceEngineeringDept.](https://reader030.fdocuments.us/reader030/viewer/2022040905/5e79e68ef888964dc34f0521/html5/thumbnails/12.jpg)
Local and global minima of a constrained optimization problem
12 / 15
![Page 13: Optimization Methods Lecture 1 - University of California ...solmaz.eng.uci.edu/Teaching/MAE206/Lecture1.pdf · Optimization Methods Lecture 1 Solmaz S. Kia MechanicalandAerospaceEngineeringDept.](https://reader030.fdocuments.us/reader030/viewer/2022040905/5e79e68ef888964dc34f0521/html5/thumbnails/13.jpg)
Local and global minima of an constrained optimization problem
Let X be the constraint set.
x? ∈ X is a local minimum of f if
∃ ε > 0 s.t. f(x?) 6 f(x), ∀x ∈ X with ‖x− x?‖ < ε,
x? ∈ X is a global minimum of f if
f(x?) 6 f(x), ∀x ∈ X,
x? ∈ X is a constrained strict local minimum of f if
∃ ε > 0 s.t. f(x?) < f(x), ∀x ∈ X with ‖x− x?‖ < ε,
x? ∈ X is a constrained strict global minimum of f if
f(x?) < f(x), ∀x ∈ X,
13 / 15
![Page 14: Optimization Methods Lecture 1 - University of California ...solmaz.eng.uci.edu/Teaching/MAE206/Lecture1.pdf · Optimization Methods Lecture 1 Solmaz S. Kia MechanicalandAerospaceEngineeringDept.](https://reader030.fdocuments.us/reader030/viewer/2022040905/5e79e68ef888964dc34f0521/html5/thumbnails/14.jpg)
Some preliminaries
Gradient of a f(x) : Rn → R, f ∈ C1
∇f(x) =
∂f∂x1∂f∂x2
...∂f
∂xn
Example:f(x) = 1
2(x1 − 2)2 + x1 x2 x3 +
13(x3 + 4)3
∇f(x) =
(x1 − 2) + x2 x3x1 x3
x1 x2 + (x3 + 4)2
Hessian of a f(x) : Rn → R, f ∈ C2
∇2f(x) =
∂2f
∂x1∂x1
∂2f∂x1∂x2
· · · ∂2f∂x1∂xn
∂2f∂x2∂x1
∂2f∂x2∂x2
· · · ∂2f∂x2∂xn
...... · · ·
...∂2f
∂xn∂x1
∂2f∂xn∂x2
· · · ∂2f∂xn∂xn
= [∂2f
∂xi∂xj]ij, it is a symmetric matrix
Example: f(x) = 12(x1 − 2)2 + x1 x2 x3 +
13(x3 + 4)3
∇2f(x) =
1 x3 x2x3 0 x1x2 x1 2(x3 + 4)
14 / 15
![Page 15: Optimization Methods Lecture 1 - University of California ...solmaz.eng.uci.edu/Teaching/MAE206/Lecture1.pdf · Optimization Methods Lecture 1 Solmaz S. Kia MechanicalandAerospaceEngineeringDept.](https://reader030.fdocuments.us/reader030/viewer/2022040905/5e79e68ef888964dc34f0521/html5/thumbnails/15.jpg)
References
[1] Nonlinear Programming: 3rd Edition, by D. P. Bertsekas
[2] Linear and Nonlinear Programming, by D. G. Luenberger, Y. Ye
[3] Numerical Optimization, by J. Nocedal and S. J. Wright
15 / 15