8/14/2019 6 OneD Unconstrained Opt
1/29
CBB 4333Process Optimisation
ONE-DIMENSIONALUNCONSTRAINED OPTIMISATION
Dr Abd Halim Shah Maulud
Universiti Teknologi PETRONAS
Sept 2013
8/14/2019 6 OneD Unconstrained Opt
2/29
TOPIC OUTCOMES
By the end of the topic, students are able to:
discuss the concept of one-dimensionalunconstrained optimisation
solve one-dimensional unconstrained optimisation analytically
via solving f= 0 & determine x*
analytically or numerically function value-based methods
derivative-based methods
8/14/2019 6 OneD Unconstrained Opt
3/29
OPTIMALITY
necessary conditions
x* is stationary point
sufficient conditions
max
min
saddle point
8/14/2019 6 OneD Unconstrained Opt
4/29
ANALYTICAL METHOD
necessary conditionoptimal solution
problems?
non-linear equations difficult to solve
need more convenient methods
nix
f
i
,,1;0
8/14/2019 6 OneD Unconstrained Opt
5/29
METHODS FOR OPTIMAL SOLUTIONS SEARCH
Function value-based methods
search by compare function values at a sequence of
trial points
Derivative-based methods involve derivative at each iteration and determine
potential optimum by necessary condition
8/14/2019 6 OneD Unconstrained Opt
6/29
FUNCTION VALUE-BASED METHODS
general procedure for minimisation
1. Start with an initial value, x0
2. Calculatef(x0)
3. Change x0for next stage x1
4. Calculatef(x1)
5. Make suref(xk+1)
8/14/2019 6 OneD Unconstrained Opt
7/29
Example 1
Solve minf(x) = x2xusing function values
Solution:
Assume x0= 3;
f(x) = 32
3 = 6
-1
0
1
2
3
4
5
6
7
8
9
10
-1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5
x
f(x)=
x
2-
x
x*= 0.5
x f(x) = x 2- x
-1 2
-0.5 0.75
0 0
0.5 -0.25
1 0
1.5 0.75
2 2
2.5 3.75
3 6
3.5 8.75
8/14/2019 6 OneD Unconstrained Opt
8/29
Narrow down the solution region
issues in one-dimensional problems
exhaustive, wide range
bracketing method
to narrow down the solution region
to avoid excessive search within a wide range
how?
assume an initial bracket, b0(contain optimum)
determine reduced bracket, bkat stage k
8/14/2019 6 OneD Unconstrained Opt
9/29
Example
Use bracketing method to solve:
minf(x) = ( x100 )2
Solution:
Consider a sequence of xgiven by
xk+1= xk+ b 2
(k-1)
Assume b= 1 and x1= 0,f(x1)= 10000
x2= x1+ (1) 2(1-1)= 0 + 1 = 1,f(x2)=9801
x3= x2+ (1) 2(2-1)= 1 + 2 = 3,f(x3)=9409
8/14/2019 6 OneD Unconstrained Opt
10/29
x 0 1 3 7 15 31 63 127 255
f(x) 104 9801 9409 8649 7225 4761 1369 729 2325
New bracket is 63 < x< 255
Continue search using bracketing method
Bracket bound decreases, bneeds to be reduced
Assume b= 0.5 and repeat calculations
8/14/2019 6 OneD Unconstrained Opt
11/29
Derivative-based method
Newtons method
(Quasi-Newton) Secant method
8/14/2019 6 OneD Unconstrained Opt
12/29
DERIVATIVE-BASED METHODS
general procedure for minimisation
1. Start with an initial value, x0
2. Calculatef(x0) and derivatives off(x0)
3. Change x0for next stage x1using the derivatives4. Calculatef(x1) and derivatives off(x1)
5. Make suref(xk+1)
8/14/2019 6 OneD Unconstrained Opt
13/29
8/14/2019 6 OneD Unconstrained Opt
14/29
NEWTONS METHOD
necessary condition forf(x)to have an optimum is
that f(x)= 0
apply Newtons method to solve f(x)= 0
)(
)(
21xf
xf
xx k
kk
f(x)f(x)
f(x)f(x)
8/14/2019 6 OneD Unconstrained Opt
15/29
NEWTONS METHOD
approximatef(x)by quadratic function at xk
using
Taylors expansion
f(x) = f(xk) + f(x
k) (x-x
k) + 2f(x
k) (x-x
k)2
stationary point can found by
df(x)/dx = f(xk) + 2 2f(x
k) (x-x
k) = 0
x = xk( f(x
k) / 2f(x
k) )
Newtons method is equivalent to using quadratic
approximation for a function & applying necessary
condition
8/14/2019 6 OneD Unconstrained Opt
16/29
8/14/2019 6 OneD Unconstrained Opt
17/29
Example 2
Find minimum off(x) =x4x +1, tol =10-7
Solution:
f(x) = 4x31, 2f(x)= 12x2
Assumex0= 3;
x1= x0(4(3)3-1)/12(32)
= 3 107/108
= 2.00926
x2 = x1(4(2.00926)3-1)/12(2.009262)
= 2.00926 31.4465/48.4454
= 1.36015
8/14/2019 6 OneD Unconstrained Opt
18/29
Solution
| x
k+1
x
k
|
x* = 0.6299605
8/14/2019 6 OneD Unconstrained Opt
19/29
NEWTONS METHOD
this method approximates the function by a
quadratic function
for quadratic functions, the optimum is obtained
in one iteration both f(x) and 2f(x) have to be calculated
if f(x)< 0, the method converges slowly
8/14/2019 6 OneD Unconstrained Opt
20/29
Derivative-based method
(Quasi-Newton) Secant method
8/14/2019 6 OneD Unconstrained Opt
21/29
SECANT METHOD
starts out by using two points xp
and xq
spanning the interval
first derivatives should have opposite signs at xp
andx
q
f (xp) = f(xq)
f (x)is approximated by a straight line
interval bounds are updated and narrow down at
each iteration
the procedure terminates when f(xkp) < tol
8/14/2019 6 OneD Unconstrained Opt
22/29
QUASI-NEWTON (SECANT) METHOD
mxfxx
xx
xfxfm
qqp
pq
pq
)(~
)()(
in next iteration replaces xp
Remarks:
this method only uses 1storder derivatives
px~
f(x)
8/14/2019 6 OneD Unconstrained Opt
23/29
Example
Find minimum off(x) =x4x +1, tol =10-7
Solution:
f(x) = 4x3
1
Assume xp= 3, xq= 3;
f(xp) =
109f(xq) = 107opposite signsacceptable choices
8/14/2019 6 OneD Unconstrained Opt
24/29
Example
f(x) =x4x +1
The shape off(x)implies
that a large number of
iterations are needed.
f(x)
f(x) = 4x3-1
8/14/2019 6 OneD Unconstrained Opt
25/29
Solution (Cont.)
( )p
f x
8/14/2019 6 OneD Unconstrained Opt
26/29
RATE OF CONVERGENCE
three rates of convergence are used to compare the
effectiveness of different search methods
Linear
Slow in practice
Secant method has a linear convergence
Order P
Fastest in practiceNewton method has order p = 2 convergence
Superlinear
fast in practice
large,10*
*
1kcc
xx
xx
k
k
large1p0, kcc
xx
xx
p
k
k,
*
*1
kcc
xx
xx
kk
k
k
k asandor0 0lim*
*1
8/14/2019 6 OneD Unconstrained Opt
27/29
8/14/2019 6 OneD Unconstrained Opt
28/29
RECAP
discuss the concept of one-dimensionalunconstrained optimisation
solve one-dimensional unconstrained optimisation
analytically via solving f= 0 & determine x*
analytically or numerically
function value-based methods derivative-based methods
Assignment 3
8/14/2019 6 OneD Unconstrained Opt
29/29
Top Related