Lec 13 Newton Raphson Method

8/9/2019 Lec 13 Newton Raphson Method

http://slidepdf.com/reader/full/lec-13-newton-raphson-method 1/30

Topic: Newton-Raphson Method

Dr. Nasir M Mirza

Optimization Techniques Optimization Techniques

Email: [email protected]



Mathematical Background

Objective: Maximize or Minimize f (x)

subject to

sConstraint*,,2,1)(

*,,2,1)(

⎭⎬⎫

===≤

pibe

miad

ii

ii

Κ Κ

x

x

x = { x1, x2, …, xn}

f (x): objective functiond i(x): inequality constraints

ei(x): equality constraints

ai and bi are constants)(

)(

x

x

f Minimize

f Maximize

−

χ



A function is said to be multimodal on a given interval ifthere are more than one minimum/maximum point inthe interval.

Global and Local Optima



Characteristics of Optima

To find the optima, we can find the zeroes of f' ( x).



Newton’s Method

Let f ( x) = g' ( x). Thus the zeroes of f ( x) is the optima of g( x).

Substituting f ( x) into the updating formula of Newton-

Rahpson method, we have

So, Minimize g(x) = Maximize - g(x);

If g'(x) exists, then to find the optima of g(x), we

can find the zero of g'(x);

Beware of inflection points of g(x).

)("

)('

)('

)(1

i

ii

i

iii

xg

xg x

x f

x f x x −=−=+



Newton-Raphson Method

• This method is one of the most widely used methods to solve

equations also known as the Newton-Raphson.

• The idea for the method comes from a Taylor series expansion,where you know the function and its’ first derivative.

f(xk+1) = f(xk ) + (xk+1 - xk )*f ‘(xk ) + ...

• The goal of the calculations is to find a f(x)=0, so set f(xk+1) =

0 and rearrange the equation. [ f ‘(xk ) is the first derivative of

f(x). ]0 = f(xk ) + (xk+1 - xk )*f ‘(xk )

xk+1 = xk - [ f(xk ) / f ‘(xk ) ]



Newton-Raphson Method

f(x)

f(xi)

f(xi-1)

xi+2 xi+1 xi x

θ

( )[ ]ii x f x ,

)(x f

) f(x -= x x

i

iii

′+1

• The method uses the slope

of the line to project to the

x axis and find the root.• The method converges on

the solution quickly.



Derivation

f(x)

f(xi)

xi+1 xi x

B

CAα

)('

)(1

i

iii

x f

x f x x −=+

1

)()('

+−=

ii

ii

x x

x f x f

AC

AB=)α tan(

)("

)('

)('

)(1

i

ii

i

iii

xg

xg x

x f

x f x x −=−=+



Newton-Raphson Method:

010x

1

1 x

- x x =

i

iia

+

+∈

STEP 2: Calculate the next estimate of the root

Find the absolute relative approximate error

STEP 1: Evaluate df/dx = f ′ (x) symbolically:

that is find an analytical expression for the first derivative of the

function with respect to x.

)(")('

)(')(

1

i

ii

i

iii

xg xg x

x f x f x x −=−=+



• STEP 3: Find if the absolute relative approximateerror is greater than the pre-specified relative error

tolerance.

• If so, go back to step 2, else stop the algorithm.

• Also check if the number of iterations has exceededthe maximum number of iterations.

Newton-Raphson Method:



Newton-Raphson Method: Example

• You are working for ‘DOWN THE TOILET COMPANY’ that makes

floats for ABC commodes. The ball has a specific gravity of 0.6

and has a radius of 5.5 cm. You are asked to find the optimum

distance to which the ball will get submerged when floating in

water.



Newton-Raphson Method: Example 1

Solution: The equation that gives the depth ‘x’ to which the ball issubmerged under water is given by

Use the Newton’s method of finding rootsof equations to find the optimum depth ‘x’

to which the ball is submerged underwater.

Conduct three iterations to estimate the

optimum for the above model equation.

( )

( ) x.- x x f

.+ x.- x xg x f

x.+ x.- x xg

-

-

3303

10x99331650)(

10x993305504

1)(

2

423

434

=′

=′=

=



Graph of function f(x)

-0.02 0.00 0.02 0.04 0.06 0.08 0.10 0.1

-0.0004

-0.0003

-0.0002

-0.0001

0.0000

0.0001

0.0002

0.0003

0.0004

0.0005

f ( x )

x

( )

4

23

10x9933

1650

-.+

x.- x x f =



Iteration #1

( )( )

%96.75

0.08320 10x4.5

10.413x302.0

'

02.0

3

4

1

0

001

0

=∈

=−

−=

−=

=

−

−

a

x

x f

x f x x

x



Iteration #2

( )

( )

%86.42

0.05824 10x689.6

10x670.108320.0

'

08320.0

3

4

2

1

1

12

1

=∈

= −

−−=

−=

=

−

−

a

x

x f

x f

x x

x



Iteration #3

( )

( )

%592.6

0.06235

10x043.9

10x717.305284.0

'

05824.0

a

3

5

2

2

23

2

=∈

=−

−=

−=

=

−

−

x f

x f

x x

x



Newton’s Method: computer algo

Do while |x2 - x1| >= tolerance value 1

or |f(x2)|>= tolerance value 2

or f’(x1) ~= 0

Set x2 = x1 - f(x1)/f’(x1)

Set x1 = x2;

END loop



Advantages

• Converges fast, if it converges

• Requires only one guess



Drawbacks

-20

-15

-10

-5

0

5

10

-2 -1 0 1 2

1

2 3

f(x)

x Inflection Point

( ) ( ) 013 =−= x x f



Drawbacks (continued)

-1.00E-05

-7.50E-06

-5.00E-06

-2.50E-06

0.00E+00

2.50E-06

5.00E-06

7.50E-06

1.00E-05

-0.03 -0.02 -0.01 0 0.01 0.02 0.03 0.04

x

f(x)

0.02

Division by zero

( ) 010x4.203.0 623 =+−= − x x x f




-1.5

-1

-0.5

0

0.5

1

1.5

-2 0 2 4 6 8 10

x

f(x)

-0.0630690.54990 4.462 7.53982

Root Jumping

( ) 0== xSin x f




-1

0

1

2

3

4

5

6

-2 -1 0 1 2 3

f(x)

x

3

4

2

1

-1.75 -0.3040 0.5 3.142

Oscillations near Local Maxima or Minima

( ) 022 =+= x x f



Example 2:

• Let us solve the following

problem:

• g(x) = x 4 /4 - 4x 3 /3 + 0.5x 2 – 10x = 0

• Then

f(x) =g’(x) = x 3 - 4x 2 + x – 10

df/d x = 3x 2 -8 x + 1

• Let us first draw a graph and seewhere are the roots;

• The roots are between (3,6).

• Then let us write a computer

program in MATLAB that willcompute the root using Newton-

Raphson method.

-6 -4 -2 0 2 4 6

-400

-350

-300

-250

-200

-150

-100

-50

0

50

100

f ( x )

x



Example 2: Computer Program

% A program that Uses Newton-Raphson method

% to find the roots of x = x^3 +x^2 -3x -3

% Input: x0 = initial guess;

% n = number of iterations; default: n = 10;

% Output: x = estimate of the root;

n = 10; x0 = 3.0

x = x0; % Initial Guess

fprintf(' k f(x) dfdx x(k+1)\n');

for k=1:n

f = x^3 - 4*x^2 + x – 10.0;

dfdx = 3*x^2 - 8*x + 1.0;

x = x - f/dfdx;fprintf('%3d %12.3e %12.3e %18.15f\n',k-1,f,dfdx,x);

end



Example 2: results

k f(x) dfdx x(k+1)

0 -1.600e+001 4.000e+000 7.000000

1 1.440e+002 9.200e+001 5.4347832 3.781e+001 4.613e+001 4.615102

3 7.716e+000 2.798e+001 4.339292

4 7.280e-001 2.277e+001 4.307327

5 9.181e-003 2.220e+001 4.3069136 1.526e-006 2.219e+001 4.306913

7 4.796e-014 2.219e+001 4.306913

8 3.553e-015 2.219e+001 4.306913

9 3.553e-015 2.219e+001 4.306913

>>

Result of the computer program: x0 = -2.0 and iterations n = 10.

The root value is x = 4.306913 and it was obtained after 5 iterations.



Example 2:

• In this example we were finding optimum for :

g(x) = x 4 /4 - 4x 3 /3 + 0.5x 2 – 10x = 0

• Then f(x) =g’(x) = x 3 - 4x 2 + x – 10

The root for f(x) is at x = x = 4.306913

It means g’(x) = 0 at this point.

Now

df/dx = d 2g/dx2 = 3x 2 - 8 x + 1

and At x = 4.306913 it is positive real value.

That is d 2g/dx2 > 0

which means x =4.306913

is a local minimum



Example 3:

• Let us solve the following problem

of finding optimum for:

g(x) = exp(x) + 0.5 x2 – 10x = 0

Then

f(x) = dg/dx = exp(x) + x – 10.

df/dx = exp(x) + 1.

• Let us first draw a graph and see

where are the roots;

• The roots are between (0, 4).

• Then let us write a computer

program in MATLAB that will

compute the root using Newton-

Raphson method.-4 -2 0 2 4

-20

-10

0

10

20

30

40

50

f ( x )

x



Example 3: Computer Program

% A program that Uses Newton-Raphson method

% to find the roots of x = x^3 +x^2 -3x -3

% Input: x0 = initial guess;

% n = number of iterations; default: n = 10;

% Output: x = estimate of the root;n = 10; x0 = 0.0

x = x0; % Initial Guess

fprintf(' k f(x) dfdx

x(k+1)\n');

for k=1:n

f = exp(x) + x – 10.0;

dfdx = exp(x) + 1.0;

x = x - f/dfdx;fprintf('%3d %12.3e %12.3e %18.15f\n',k-1,f,dfdx,x);

end



Example 3: results

k f(x) dfdx x(k+1)

0 -9.000e+000 2.000e+000 4.5000001 8.452e+001 9.102e+001 3.571415

2 2.914e+001 3.657e+001 2.774566

3 8.806e+000 1.703e+001 2.257515

4 1.817e+000 1.056e+001 2.085456

5 1.337e-001 9.048e+000 2.0706786 8.746e-004 8.930e+000 2.070580

7 3.803e-008 8.929e+000 2.070580

8 0.000e+000 8.929e+000 2.070580

9 0.000e+000 8.929e+000 2.070580

>>

Result of the computer program: x0 = -2.0 and iterations n = 10.

The root value is x = 2.070580



Example 3:

• In this example we were finding optimum for :

g(x) = exp(x) + 0.5 x2 – 10x = 0

Then f(x) = dg/dx = exp(x) + x – 10.

The root for f(x) is at x = 2.07058

It means g’(x) = 0 at this point.

Now

df/dx = d 2g/dx2 = exp(x) + 1. and

At x = 2.07058 it is positive real value.

That is d 2g/dx2 > 0

which means x = 2.07058 is a local minimum

Lec 13 Newton Raphson Method

Documents

Transcript of Lec 13 Newton Raphson Method