E. T. S. I. Caminos, Canales y Puertos 1
EngineeringComputation
Lecture 4
E. T. S. I. Caminos, Canales y Puertos 2
Error Analysis for N-R :
Recall that
Taylor Series gives:
ii 1 i
i
f (x )x x
f '(x )
2r i i r i r i
f "( )f (x ) f (x ) f '(x )(x x ) (x x )
2!
where xr x xi and f(xr) = 0
Open Methods (Newton-Raphson Method)
E. T. S. I. Caminos, Canales y Puertos 3
Dividing through by f '(xi) yields
Ei+1 is proportional to Ei2 ==> quadratic rate of convergence.
2ir i r i
i i
f (x ) f "( )0 (x x ) (x x )
f '(x ) 2f '(x )
2i 1 i r i r i
i
f '' ( )(x x ) (x x ) (x x )
2f '(x )
2r i 1 r i
i
f '' ( )(x x ) (x x )
2f '(x )
OR
2i 1 i
i
f "( )E E
2!f '(x )
Open Methods (Newton-Raphson Method)
E. T. S. I. Caminos, Canales y Puertos 4
Summary of Newton-Raphson Method:
Advantages:1. Can be fast
Disadvantages:1. May not converge
2. Requires a derivative
Open Methods (Newton-Raphson Method)
E. T. S. I. Caminos, Canales y Puertos 5
Secant Method
Approx. f '(x) with backward FDD:
Substitute this into the N-R equation:
to obtain the iterative expression:
i 1 i
i 1 i
f (x ) f (x )f '(x)
x x
ii 1 i
i
f (x )x x
f '(x )
i i 1 ii 1 i
i 1 i
f (x )(x x )x x
f (x ) f (x )
Open Methods (Secant Method)
E. T. S. I. Caminos, Canales y Puertos 6
Secant Method
xi = xi+1
i i 1 ii 1 i
i 1 i
f (x )(x x )x x
f (x ) f (x )
x
f(x)
f(xi)
xi
f(xi-1)
i-1 ii
i-1 i
f(x ) - f(x )f '(x )
x - x
f(x)
xi-1xi+1x
f(xi)
xi
f(xi-1)
i-1 ii
i-1 i
f(x ) - f(x )f '(x )
x - x
xi-1xi+1
Open Methods (Secant Method)
E. T. S. I. Caminos, Canales y Puertos 7
1) Requires two initial estimates: xi-1 and xi
These do NOT have to bracket root !
2) Maintains a strict sequence:
Repeated until:
a. | f(xi+1) | < k with k = small number
b.
c. Max. number of iterations is reached.
3. If xi and xi+1 were to bracket the root, this would be the same as the False-Position Method. BUT WE DON'T!
i i 1 ii 1 i
i 1 i
f (x )(x x )x x
f (x ) f (x )
i 1 ia s
i 1
x x100%
x
Open Methods (Secant Method)
E. T. S. I. Caminos, Canales y Puertos 8
Fixed point Method
predict a value of xi+1 as a function of xi.
Convert f(x) = 0 to x = g(x)
iteration steps: xi+1 = g(xi )
x(new) = g(x(old) )
Open Methods (Fixed point method)
E. T. S. I. Caminos, Canales y Puertos 9
Example II:
x = sin(x) –> xi+1 = sin(xi) OR
x = arcsin(x) –> xi+1 = arcsin(xi)
sin(x)f (x) 1.0 0.0
x
201 1 i
7,500 1,000i
20
i 11 1 i
i 1.07.5
Example I:
Open Methods (Fixed point method)
E. T. S. I. Caminos, Canales y Puertos 10
Convergence:
Does x move closer to real root (?)
Depends on:1. nature of the function2. accuracy of the initial estimate
Interested in:1. Will it converge or will it diverge?2. How fast will it converge ?
(rate of convergence)
Open Methods (Fixed point method)
E. T. S. I. Caminos, Canales y Puertos 11
Convergence of the Fixed point Method:
Root satisfies: xr = g(xr)
The Taylor series for function g is:
xi+1 = g(xr) + g'(x)(xi - xr) xr < x < xi
Subtracting the second equation from the first yields
(xr – xi+1) = g'(x) (xr – xi) or
1. True error for next iteration is smaller than the true error in the previous iteration if |g'(x)| < 1.0 (it will converge).
2. Because g'(x) is almost constant, the new error is directly proportional to the old error (linear rate of convergence).
E i1 g' () E i
Open Methods (Fixed point method)
E. T. S. I. Caminos, Canales y Puertos 12
Further Considerations:
Convergence depends on how f(x) = 0 is
converted into x = g(x)
So . . .
Convergence may be improved
by recasting the problem.
Open Methods (Fixed point method)
E. T. S. I. Caminos, Canales y Puertos 13
can be small, even though xnew is not close to root.
Remedy: Do not completely rely on a to ensure that the problem is solved.
Check to make sure |f(xnew) | < .
new olda
new
x xx 100%
x
Convergence Problem:
For slowly converging functions
Open Methods (Fixed point method)
E. T. S. I. Caminos, Canales y Puertos 14
Open Methods (Fixed point method)
E. T. S. I. Caminos, Canales y Puertos 15
Open Methods
E. T. S. I. Caminos, Canales y Puertos 16
Why do open methods fail?Function may not look linear.
Remedy: recast into a linear form. For example,
201 (1 i)f (i) 7500 0
i
Is a poorly constrained problem in that there is a large, nearly flat zone for which the derivative is near zero. Recast as:
i f(i) = 0 = 7,500 i - 1000 [ 1 - (1+i)-20 ]
Open Methods
E. T. S. I. Caminos, Canales y Puertos 17
Recast as: i f(i) = 0 = 7,500 i - 1000 [ 1 - (1+i)-20 ]
– The recast function, "i f(i) will have the same roots as f(i) plus an additional root at i = 0.
– It will not have a large, flat zone, thus:
h(i) = i f(i) = 7,500 i – 1000 [ 1 – (1+ i)–20]
– To apply N-R we also need the first derivative:
h'(i) = 7,500 - 20,000 (1+ i)-21
201 (1 i)f (i) 7500 0
i
Open Methods
E. T. S. I. Caminos, Canales y Puertos 18
Cases of Multiple Roots
Multiple Roots: f(x) = (x – 2)2 (x – 4)
x = 2 represents two of the three roots.
-3
-2
-1
0
1
2
3
4.03.02.01.0
Open Methods
E. T. S. I. Caminos, Canales y Puertos 19
Problems and Approaches:Cases of Multiple Roots
1.Bracketing Methods fail locating x = 2. Note that f(x) f(xr) > 0.
2. At x = 2, f(x) = f '(x) = 0. • Newton-Raphson and Secant methods may experience
problems.
• Rate of convergence drops to linear.
• Luckily, f(x) 0 faster than f '(x) 0
3. Other remedies, recasting problem: Find x such that u(x) = 0 where:
Note that u(x) and f(x) have same roots.
f (x)u(x) 0
f '(x)
Open Methods
E. T. S. I. Caminos, Canales y Puertos 20
m = 1: linear convergence
m = 2: quadratic convergence
Method m
Bisection 1
False Position 1
Secant, mult. root 1
NR, mult. root 1
Secant, single root 1.618 "super linear"
NR, single root 2
Accel. NR, mult. root (f(x)/f'(x)=0) 2
i 1mi
i
Elim A 0
E
Summary -- Rates of Convergence
E. T. S. I. Caminos, Canales y Puertos 21
A real rootfinding problem can be viewed as having three phases:
1) Opening moves: One needs to find the region of the parameter space in which desired root can be found. Understanding of problem, physical insight, and
common sense are valuable.
2) Middle Game: Use robust algorithm to reduce initial region of uncertainty.
3) End game: Generate a highly accurate solution in a few iterations.
Three Phase Rootfinding Strategy
Top Related