4M020 Design tools - Algorithms for numerical optimizationpiet/edu/dos/pdf/optimalgo.pdf ·...

Formulation classes Optimization algorithms

4M020 Design toolsAlgorithms for numerical optimization

L.F.P. Etman

Department of Mechanical EngineeringEindhoven University of Technology

Wednesday September 3, 2008

1 / 32


Outline

1 Problem formulation: classes and properties

2 Optimization algorithms

2 / 32


Problem formulation: classes and properties

The mathematical problem formulation is to the heart ofthe success of design optimization

3 / 32


Two-bar truss example

F

SS

h

d

minx

f (x) = C0 x22

√S2 + x21

s.t. g1(x) = C1x1 − 1 ≤ 0

g2(x) = C2

√S2 + x21

x22x1− 1 ≤ 0

g3(x) = C3

(S2 + x21

)3/2x42x1

− 1 ≤ 0

χ : x1, x2 > 0,

How can we solve this optimization problem?

4 / 32


Mathematical problem formulation

Minimizex

f (x) x = (column) vector of design variables

subject to hj (x) = 0 j = 1, . . . ,mh

gk (x) ≤ 0 k = 1, . . . ,mg

x ∈ X ⊆ Rn

[Papalambros & Wilde 2000: Principles of optimal design]

5 / 32


Formulation examples

Design variables:

• sizing (dimensions)

• shape (geometry of boundary)

• topological (material distribution)

6 / 32


Formulation examples

Design variables x:

• sizing (dimensions)

• shape (geometry of boundary)

• topological (material distribution)

Objective function f (x):

• profit (cost, efficiency, weight, ...)

Constraint functions hj (x) and gk (x):

• geometrical (width, length, height, ...)

• structural (stresses, displacements, ...)

• dynamical (accelerations, eigenfrequency, ...)

• physical (temperatures, pressures, ...)

7 / 32


Formulation classes

Design variables:

• continuous or discrete

• single-variable or multi-variable

Objective function:

• minimization or maximization

• single-objective or multi-objective

Constraint functions:

• unconstrained or constrained

• equality or inequality

8 / 32


Formulation classes: linear programming

Linear programming (LP) problem:

• Linear objective function

• Linear constraint functions

• Continuous design variables

Example:

minx∈R2

f (x) = −2x1 − x2

s.t. g1(x) = x1 + 2x2 − 8 ≤ 0g2(x) = 2x1 − 2x2 − 3 ≤ 0g3(x) = −2x1 + 1 ≤ 0g4(x) = −2x2 + 1 ≤ 0

ii

“LP˙temp” — 2006/6/23 — 11:29 — page 1 — #1 ii

ii

ii

x1

x2

F

g3

g4

g1

g2

f

(Some) design variables discrete:

• (Mixed-)integer linear programming problem: (M)ILP

9 / 32


Formulation classes: quadratic programming

Quadratic programming (QP) problem:

• Quadratic objective function

• Linear constraint functions


Example:

minx∈R2

f (x) = 3x21 − 2x1 + 5x22 + 30x2

s.t. g1(x) = −2x1 − 3x2 + 8 ≤ 0g2(x) = 3x1 + 2x2 − 15 ≤ 0g3(x) = x2 − 5 ≤ 0

ii

“QP˙temp” — 2006/6/23 — 11:51 — page 1 — #1 ii

ii

ii

x1

x2

g3

g1

f

F g2


• (Mixed-)integer quadratic programming problem: (M)IQP

10 / 32


Formulation classes: nonlinear programming

Nonlinear programming (NLP) problem:

• Non-linear objective function

• Non-linear constraint functions


Example:

minx∈R2

f (x) = 3x1 +√3x2

s.t. g1(x) = 18x1

+ 6√3

x2− 3 ≤ 0

g2(x) = 5.73 − x1 ≤ 0g3(x) = 7.17 − x2 ≤ 0

ii

“NLP˙temp” — 2006/6/25 — 10:15 — page 1 — #1 ii

ii

ii

x1

x2 F

g3

g1

g2

f


• (Mixed-)integer nonlinear programming problem: (M)INLP

11 / 32


Formulation properties

Design variables:

• Domain: {0,1}, N , N+, Z, R, R+

Objective and constraint functions:

• Linearity: linear or nonlinear

• Continuity: none/once/twice-differentiable

Optimization problem:

• Modality: uni-modal or multi-modal

12 / 32



F

SS

h

d

minx

f (x) = C0 x22

√S2 + x21

s.t. g1(x) = C1x1 − 1 ≤ 0

g2(x) = C2

√S2 + x21

x22x1− 1 ≤ 0

g3(x) = C3

(S2 + x21

)3/2x42x1

− 1 ≤ 0

χ : x1, x2 > 0,

Exercise 1: introduction to the Matlab algorithm fmincon

13 / 32


Optimization algorithms

Select an optimization algorithm in accordance with theoptimization problem class and properties

14 / 32


Searching for a zero

ii

“1Dintersect˙temp” — 2008/9/1 — 14:23 — page 1 — #1 ii

ii

ii

( )

0

Solving a set ofm nonlinear equations withm unknowns:

Φ(q) = 0

Example: 1 + e−q2 − q22e

−q1 = 01 − q1e−q2 + 2q2e−q1 = 0

15 / 32


Searching for a minimum

ii

“1Dminimize˙temp” — 2008/9/1 — 14:23 — page 1 — #1 ii

ii

ii

( )

Finding the minimum of a function with n variables:

Minimizex

f (x)

Example: Minx

f (x) = x1 + x2 + x1e−x2 + x22e−x1

16 / 32


Zero-search is special case of minimum-search

Finding the solution to a set of nonlinear equations:

Φ(q) = 0

can be reformulated as a minimization problem:

Minimizeq

eTe

with the error (deviations to zero) defined as:

e = Φ(q)

17 / 32


Analytical versus numerical solution

Some optimization problems can be analytically solved:

Minx

f (x) = 2x1 + x−21 + 2x2 + x−22

Many optimization problems can only be numerically solved:

Minx

f (x) = x1 + x2 + x1e−x2 + x22e−x1

An optimization algorithm is an iterative procedure to solve anunconstrained or constrained minimization (maximization)problem.

[Papalambros & Wilde 2000: Principles of optimal design]

18 / 32


Classification of optimization algorithms

Four classifiers of an optimization algorithm:

• unconstrained / constrained search

• local / global search

• determistic / stochastic search

• 0th / 1th / 2nd -order search

19 / 32


Unconstrained / constrained search

Unconstrained search:

• aims to minimize a nonlinear, possibly multi-modal, objectivefunction in n-dimensional space

Constrained search:

• aims to minimize a (non)linear objective function inn-dimensional space while accounting for (non)linear equalityand/or inequality constraint functions

20 / 32


Local / global search

Local search:

• seeks improvement based on the Taylor series expansion;

in a sufficiently small region any nonlinear function can berepresented by a quadratic approximation:

f̃ (x) = f (x∗)+n∑

i=1

∂f (x∗)∂xi

(xi −x∗i )+12

n∑i=1

n∑j=1

∂2f (x∗)∂xi∂xj

(xi −x∗i )2

Global search:

• seeks improvement by design space exploration

21 / 32


Deterministic / stochastic search

Deterministic search:

• gives exactly the same search path when running thealgorithm twice for unchanged algorithmic settings

Stochastic search:

• generates random search paths for every run of the algorithm

22 / 32


0th / 1th / 2nd -order search

A 0th-order algorithm

• uses function evaluations only

A 1th-order algorithm

• uses function and gradient evaluations

A 2nd-order algorithm

• uses function, gradient, and Hessian evaluations

23 / 32


Newton type of algorithms

Characteristics

• unconstrained/constrained

• local

• deterministic

• 2nd-order

Matlab Optimization toolbox

• fminunc

• fmincon

ii

“line2˙temp” — 2006/6/26 — 14:30 — page 1 — #1 ii

ii

ii

x1

x2

x0

x1

x2

x3

s0

s1

s2

s3

Line search

ii

“trust2˙temp” — 2006/6/26 — 20:51 — page 1 — #1 ii

ii

ii

x1

x2

x0

x1 x2

x30

1

2

3

s0 s1 s2

Trust-region

24 / 32


Bio-inspired algorithms

Characteristics


• global

• stochastic

• 0th-order

Matlab GA toolbox

• ga

Algorithms

• Particle swarms

• Genetic algorithms (GAs)

• Simulated Annealing

ii

“2Dparticleswarm˙temp” — 2008/9/1 — 14:23 — page 1 — #1 ii

ii

ii

1

2

25 / 32


Direct search algorithms

Characteristics


• local

• deterministic

• 0th-order

Matlab Optimization toolbox

• fminsearch

Matlab Direct Search toolbox

• patternsearch

Algorithms

• Nelder-Mead simplex search

• Generalized Pattern Search

ii

“2Dsimplex˙temp” — 2008/9/1 — 14:23 — page 1 — #1 ii

ii

ii

1

2

26 / 32



F

SS

h

d

minx

f (x) = C0 x22

√S2 + x21

s.t. g1(x) = C1x1 − 1 ≤ 0

g2(x) = C2

√S2 + x21

x22x1− 1 ≤ 0

g3(x) = C3

(S2 + x21

)3/2x42x1

− 1 ≤ 0

χ : x1, x2 > 0,

Exercise 2: visualization of the optimization search path

27 / 32


Gradients

Calculating derivatives:

• Analytically (by hand)

• Symbolically (e.g. Mathematica, Maple)

• Finite differencing

• By the CAE analysis code (e.g. MARC)

• Automatic differentiation (e.g. ADIFOR, ADIC)

Matlab:

• fmincon and fminunc calculate gradients by finitedifferencing if the user does not provide them

• Matlab Symbolic Toolbox

• Matlab Automatic Differentiation Toolbox

28 / 32


Conditions for optimality

Karush-Kuhn-Tucker (KKT) conditions

ii

“unc˙infwell2D˙temp” — 2007/4/4 — 15:12 — page 1 — #1 ii

ii

ii

10

9

8

f

x1

x2

P1

P2

ii

“ineq˙opt˙temp” — 2006/6/20 — 10:00 — page 1 — #1 ii

ii

ii

x1

x2

x1

x2

g

g1

g2

g3

f

g22

3 = 0

g11

f

g

F

F

Unconstrained: gradients objective function zero

Constrained: linear combination of gradients objective andgradients active constraints zero (i.e. gradients Lagrange functionL zero)

29 / 32


Termination criteria

Condition on optimality

‖∇L(xk+1)‖ < ε (Matlab: TolFun)

gj (xk+1) < ε and |hj (xk+1)| < ε (Matlab: TolCon)

or a condition on change in x

‖xk+1 − xk‖ < ε (Matlab: TolX)

or a condition on the number of iterations

k ≤ kmax (Matlab: MaxIter)

or a combination of these, with ε > 0

30 / 32


Scaling

Scale design variables, objective function and constraintfunctions to avoid numerical difficulties and prematuretermination

31 / 32



F

SS

h

d

minx

f (x) = m

s.t. g1(x) = C1x1 − 1 ≤ 0

g2(x) =σ(x)σy

− 1 ≤ 0

g3(x) =P(x)Pc(x)

− 1 ≤ 0

χ : x1, x2 > 0,

Exercise 3: FEM-model in the optimization loop

32 / 32

4M020 Design tools - Algorithms for numerical optimizationpiet/edu/dos/pdf/optimalgo.pdf ·...

Documents

Transcript of 4M020 Design tools - Algorithms for numerical optimizationpiet/edu/dos/pdf/optimalgo.pdf ·...