L22 Numerical Methods part 2
description
Transcript of L22 Numerical Methods part 2
L22 Numerical Methods part 2
• Homework• Review• Alternate Equal Interval• Golden Section• Summary• Test 4
1
2
Problem 10.4
3
2 2 21 2 3 1 2 2 3: ( ) 2 2 2 2
( 3,10, 12) (1,2,3)
Determine whether is a descent direction. . 0
given f x x x x x x xand at
i e
xd x
dc d
1 2
2 1 3
3 2 *
2 2 2(1) 2(2) 6( *) 4 2 2 4(2) 2(1) 2(3) 16
4(3) 2(2) 164 2x
x xf x x x
x x
c x
1 1 2 2 3 3
03
6 16 16 10
12
6( 3) 16(10) 16( 12)18 160 19250 0
c d c d c d
T
c d
c d c d
Yes, descent direction
Prob 10.10
4
2 21 2 2 3
:( ) ( ) ( )
(4,8,4) (1,1,1)
Determine whether is a descent direction. . 0
givenf x x x xand at
i e
xd x
dc d
1 2
1 2 2 3
2 3 *
2( ) 2(1 1) 4( *) 2( ) 2( ) 2(1 1) 2(1 1) 8
2(1 1) 42( )x
x xf x x x x
x x
c x
1 1 2 2 3 3
04
4 8 4 8
4
4(4) 8(8) 4(4)16 64 1696 0
c d c d c d
T
c d
c d c d
No, not a descent direction
Prob 10.19
5
1 2
2 1 3
3 2 *
2 2 2(2) 2(4) 12( *) 4 2 2 4(4) 2(2) 2(10) 40
4(10) 2(4) 484 2
1212 40 48 40
4812( 12) 40( 40) 48( 48)
(144 1600 2304)4048
x
x xf x x x
x x
c x
c d
2 2 21 2 3 1 2 2 3
:
( ) 2 2 2 2( 12, 40, 48) (2,4,10)
What is the slope of the function at the point?Find the optimum step size.
given
f x x x x x x xand at
xd x
Slope
Prob 10.19 cont’d
6
( 1) ( ) ( )
( ) ( )
1 1 1
2 2 2
3 3 3( ) ( )
1
2
3
1
2
2
2 124 4010 48
2 ( 12) 2 124 ( 40) 4 4010 ( 48) 10 48
k k k
new old
new old
x x dx x dx x d
xxxxxx
x x d
2 2 21 2 3 1 2 2 3
2 2 2
( ) 2 2 2 2
(2 12 ) 2(4 40 ) 2(10 48 ) 2(2 12 )(4 40 ) 2(4 40 )(10 48 )
( ) 4048 25504 0* 4048 / 25504 0.15873
f x x x x x x x
f
x
Prob 10.19 cont’d
7
alpha 0.15872
x1 0.095358x2 -2.34881x3 2.38143
f(x) 10.75031
( 1) ( ) ( )
( ) ( )
1
2
3
1
2
2
2 124 0.15873 4010 48
2 0.15873( 12) 0.0954 0.15873( 40) 2.3510 0.15873( 48) 2.38
k k k
new oldxxx
xxx
x x d
Prob 10.30
8
( 1) ( ) ( )
( ) ( )
1
2
3
4
1
2
3
4
2221
4 23 2
2 ( 2)1 ( 2)4 ( 2)3 ( 2)
k k k
new oldxxxx
xxxx
x x d
2 2 2 2
2
( ) ((2 2 ) 1) ((1 2 ) 2) ((4 2 ) 3) ((3 2 ) 4)
( ) 16 16 4
f
f
2 2 2 21 2 3 4( ) ( 1) ( 2) ( 3) ( 4)
(2,1,4,3) ( 2,2, 2,2)( )
f x x x x
find f
xx d
The Search Problem
• Sub Problem AWhich direction to head next?
• Sub Problem BHow far to go in that direction?
9
( )kx
Search Direction… Min f(x): Let’s go downhill!
10
( *)Tf f x d
1( ) ( *) ( *)( *) ( *) ( *)( *)
2T Tf f f R x x x x x x x H x x x
1( * ) ( *) ( *)
2T Tf f f R x d x x d d H d
( ) ( )let ( *) then new oldor d x x x x * d x x d
( *) 0Tf x dDescent condition
( *)Tlet fc x
0c d
Step Size?
How big should we make alpha?Can we step too “far?”
i.e. can our step size be chosen so big that we step over the “minimum?”
11
12
Figure 10.2 Conceptual diagram for iterative steps of an optimization method.
We are hereWhich direction should we head?
Some Step Size Methods
• “Analytical”Search direction = (-) gradient, (i.e. line search)Form line search function f(α)Find f’(α)=0
• Region Elimination (“interval reducing”)Equal intervalAlternate equal intervalGolden Section
13
14
Figure 10.5 Nonunimodal function f() for 0
Nonunimodal functions
Unimodal if stay in locale?
Monotonic Increasing Functions
15
Monotonic Decreasing Functions
16
continous
17
Figure 10.4 Unimodal function f().
Unimodal functions
monotonic increasing then monotonic decreasing
monotonic decreasing then monotonic increasing
Some Step Size Methods
• “Analytical”Search direction = (-) gradient, (i.e. line search)Form line search function f(α)Find f’(α)=0
• Region Elimination (“interval reducing”)Equal intervalAlternate equal intervalGolden Section
18
19
Figure 10.3 Graph of f() versus .
Analytical Step size
( 1) ( )( ) ( + ) ( )k kf f f x x d
( )
( ) ( ) ( )
( 1) ( ) ( )
given , and letthen
old
new old k
k k k
d xx x dx x d
( 1) ( )( ) ( + ) ( )'( )=0
k kf f ff
x x dSlope of line
search=c d
Slope of line at fmin0c d
Analytical Step Size Example
20
2 21 2: ( ) ( 2) ( 1)
44
find optimal step size *and ( *)!
given f x x
and let and at
f
x
d c x
2 21 2
2 2
2
( ) ( 2) ( 1)
( ) ((4 4 ) 2) ((4 6 ) 1)( ) 52 52 13( ) 2(52 ) 52 0
* 1/ 2
f x x
fff
1
2
2 21 2
2 2
4 1/ 2( 4) 24 1/ 2( 6) 12
*1
( *) ( 2) ( 1)
(2 2) (1 1) 0
xx
f x x
x
x
1
2
2( 2) 2(4 2)2( 1) 2(4 1)
46
xx
d c
( 1) ( ) ( )
( ) ( )
1 1 1
2 2 2
1
2
4 ( 4) 4 44 ( 6) 4 6
k k k
new oldx x dx x dxx
x x d
2 2( ) ((4 4 ) 2) ((4 6 ) 1)( ) 2((4 4 ) 2)( 4) 2((4 6 ) 1)( 6) 0( *) (2 4 )( 4) (3 6 )( 6) 0
8 16 18 36 26 52 0* 26 / 52 1 / 2
fff
Alternative Analytical Step Size
21
( 1) ( )
( 1) ( 1) ( 1)
( 1) ( ) ( )
( 1)( )
( 1) ( )
( 1) ( )
( ) ( + ) ( )'( )=0( ) ( ) ( )
0
since( )
( ) 0
0
k k
k T k k
k k k
kk
k k
k k
f f ffdf f d
d d
d
df
x x d
x x x
xx x d
xd
x d
c d
( 1) ( ) ( )
1
2
4 ( 4) 4 44 ( 6) 4 6
k k k
xx
x x d
( 1) ( ) 04
4 8 , 6 126
4(4 8 ) 6(6 12 ) 016 32 36 72 052 104 0
52 / 104 1/ 2
k k
T
c d
( 1) 1
2
2( 2)2( 1)
2(4 4 2)2(4 6 1)
4 86 12
k xx
c
New gradient must be orthogonal to d for ' ( )=0f
Some Step Size Methods
• “Analytical”Search direction = (-) gradient, (i.e. line search)Form line search function f(α)Find f’(α)=0
• Region Elimination (“interval reducing”)Equal intervalAlternate equal intervalGolden Section
22
23
Figure 10.6 Equal-interval search process. (a) Phase I: initial bracketing of minimum. (b) Phase II: reducing the interval of uncertainty.
“Interval Reducing”Region elimination
“bounding phase”
Interval reductionphase”
( 1)( 1)
2
l
u
u l
I
2 delta!
24
Successive-Equal Interval Algorithm
25
x f(x)-5.0000 22.0067-4.0000 18.0183-3.0000 14.0498-2.0000 10.1353-1.0000 6.36790.0000 3.00001.0000 0.71832.0000 1.38913.0000 10.08554.0000 40.59825.0000 130.4132
( ) 2 - 4 exp( )f x x x x f(x)
0.0000 3.00000.2000 2.42140.4000 1.89180.6000 1.42210.8000 1.02551.0000 0.71831.2000 0.52011.4000 0.45521.6000 0.55301.8000 0.84962.0000 1.3891
x f(x)1.2000 0.52011.2400 0.49561.2800 0.47661.3200 0.46341.3600 0.45621.4000 0.45521.4400 0.46071.4800 0.47291.5200 0.49221.5600 0.51881.6000 0.5530
x f(x)1.3600 0.4561931.3680 0.4554881.3760 0.4550341.3840 0.4548331.3920 0.4548881.4000 0.4552001.4080 0.4557721.4160 0.4566051.4240 0.4577021.4320 0.4590651.4400 0.460696
x lower -5x upper 5
delta 1
02
0.2
1.21.6
0.04
“Interval” of uncertainty
1.361.44
0.008
More on bounding phase I
Swan’s method
Fibonacci sequence
26
( 1) ( )
( 1) ( )
210
k k k
k k k
0
(1.618)k
jq
j
Successive Alternate Equal Interval
27
Assume bounding phase has found Min can be on either side of
But for sure its not in this region!
1
32 1
3 3
b l
b l u
I
I I
u land
b
Point values… not a line
Successive Alt Equal Int
28
ME 482 Optimal Design alt_eq_int.xls RJE 4/11/2012
Alternate Equal Interval Search for locating minimum of f(x)x lower 0.5x upper 2.618
Iteration xl 1/3(xu+2xl) 1/3(2xu+xl) xu Interval Optimal
1 x 0.5000000 1.2060000 1.9120000 2.6180000 2.118000 1.2060000f(x) 1.6487213 0.5160975 1.1186085 5.2362796 0.5160975
2 x 0.5000000 0.9706667 1.4413333 1.9120000 1.412000 1.4413333f(x) 1.6487213 0.7570370 0.4609938 1.1186085 0.4609938
3 x 0.9706667 1.2844444 1.5982222 1.9120000 0.941333 1.2844444 f(x) 0.7570370 0.4748826 0.5513460 1.1186085 0.47488264 x 0.9706667 1.1798519 1.3890370 1.5982222 0.627556 1.3890370 f(x) 0.7570370 0.5344847 0.4548376 0.5513460 0.45483765 x 1.1798519 1.3193086 1.4587654 1.5982222 0.418370 1.3193086
f(x) 0.5344847 0.4635997 0.4655851 0.5513460 0.4635997
( ) 2 - 4 exp( )f x x x
Requires two function evaluations per iteration
29
Figure 10.8 Initial bracketing of the minimum point in the golden section method.
Fibonacci Bounding
30
Figure 10.9 Graphic of a section partition.
Golden section
22
1,2
1 1 4(1)( 1)4
2 2(1)
1 5 1 2.236
2 20.618, 1.618
b b ac
a
2
leftside = rightside(1 )
[ ] (1 )[ ] (1 )
1 0
I II I
Golden Section Example
31
ME 482 Optimal Design alt_gold.xls RJE 4/11/2012
Golden Section Region Elimiation Search for locating minimum of f(x)x lower 0.5 1-τ= 0.381966x upper 2.618 τ = 0.618034
Iteration xl xL+(1-τ) I xL+ τ I xu Interval I Optimal
1 x 0.5 1.309004 1.808996 2.618 2.118 1.30900399f(x) 1.6487213 0.4664682 0.8683316 5.2362796 0.4664682
2 x 0.5 0.999992 1.309004 1.808996 1.308996 1.30900404f(x) 1.6487213 0.7182921 0.4664682 0.8683316 0.46646819
3 x 0.999992 1.309004 1.499984 1.808996 0.809004 1.30900401 f(x) 0.7182921 0.4664682 0.4816814 0.8683316 0.46646824 x 0.999992 1.1909719 1.309004 1.499984 0.499992 1.30900403 f(x) 0.7182921 0.5263899 0.4664682 0.4816814 0.466468195 x 1.1909719 1.309004 1.3819519 1.499984 0.3090121 1.38195187
f(x) 0.5263899 0.4664682 0.4548602 0.4816814 0.45486022
( ) 2 - 4 exp( )f x x x
Summary• General Opt Algorithms have two sub problems:
search direction, and step size
• Descent condition assures correct direction• For line searches…in local neighborhood…
we can assume unimodal!
• Step size methods: analytical, region elimin.• Region Elimination (“interval reducing”)
Equal interval Alternate equal interval Golden Section 32