Wegstein Method Metodos Numericos

054374 NUMERICAL METHODS LECTURE FIVE

Daniel R. Lewin, Technion1

LECTURE FIVENUMERICAL METHODS - (c) Daniel R. Lewin1

Process Analysis using Numerical Methods

LECTURE FIVE

Solution of Sets of Non-linear Equations


Lecture Five: Nonlinear Equations Methods for the solution of a nonlinear equation are at

the heart of many numerical methods: from the solution of M & E balances, to the optimization of chemical processes. Furthermore, the need for numerical solution of nonlinear equations also arises from the formulation of other numerical methods.

Solution f(x)=0

Nonlinear

Regression

Linear

Regression

Solution

of ODE's

Solution

of IVPDE's

Solution

of BVP's

Part One: Basic Building Blocks

Part Two: Applications

Solution Ax=b

Interpolation

min/max f(x)

Solution f(x)=0 Line Integrals

Finite Difference Approximations




Examples:♦ The minimization or maximization of a multivariable

objective function can be formulated as the solution of a set of nonlinear equations generated by differentiating the objective relative to each of the independent variables. These applications are covered in Day 6.

♦ The numerical solution of a set of ordinary differential equations can either be carried out explicitly or implicitly. In implicit methods, the dependent variables are computed in each integration step in an iterative manner by solving a set of nonlinear equations. These applications are covered in Day 10.


Lecture Five: Objectives This is an extension of last week’s lecture to sets of

equations.

On completion of this material, the reader should be able to:– Formulate and implement the Newton-Raphson method

for a set of nonlinear equations.– Use a steepest descent method to provide robust

initialization of Newton-Raphson’s method.– Formulate and implement the multivariable extension of

the method of successive substitution, including acceleration using Wegstein’s method.




5.1 Newton-Raphson MethodSolve the set of nonlinear equations:

( ) ( ) ( ) 0325.034, 2221

21211 =−−+−−= xxxxxxf

( ) ( ) ( ) 0225, 32

21212 =−−−−= xxxxf

(5.1)(5.2)

( ) ( )( )( )

( )( )( )

( )( )022

2

1011

1

101211

00

, xxxfxx

xfxfxxf

xx−

∂∂

+−∂∂

+≈

( ) 211

1 25.032 xxxf

+−−=∂∂ ( ) 12

2

1 25.032 xxxf

+−−=∂∂

Consider initial guess of x = [2,4]T. Approximating first equation using a Taylor expansion:

where:

Thus: , a linear plane.

(((( )))) (((( )))) (((( ))))1 1 2 1 2, 4 3 2 1.5 4f x x x x+ − − −+ − − −+ − − −+ − − − (5.3)


5.1 Newton-Raphson Method (Cont’d)

f1(x1,x2)( ) ( ) ( )45.1234, 21211 −−−+≈ xxxxf

( ) ( ) 045.1234 21 =−−−+ xx

(5.4)

Linear plane approximating f1

Intersection of linear plane approximating f1 with zero.




5.1 Newton-Raphson Method (Cont’d) Similarly, the intersection of the linear approximation

for f2(x1,x2) with the zero plane gives the line:

( ) ( ) 0412203 21 =−−−+− xx (5.5)

−−

=

−

−−

=

−

2500.04583.1

34

1205.13 1

2

1dd

(5.6)

Eqs.(5.4)-(5.5) are a system in 2 unknowns, d1 = x1 – 2, and d2 = x2 – 4, which are the changes in x1 and x2 from the previous estimate:

The solution of the linear system of equations is a vector that defines a change in the estimate of the solution, found by the intersection of Eqs.(5.4) and (5.5). Thus, an updated estimate for the solution is: x = [0.5417, 3.7500]T


5.1 Newton-Raphson Method (Cont’d)f1(x1,x2)f2(x1,x2)

The intersection ofthe two linear planes generate a linear vector d(0), from initial guess,x(0). This intersects the zero plane at x(1), which is the next estimate of the solution.




This constitutes a single step of the NR method. This is continued to convergence to 4 sig. figs. in four iterations:


k x1 x2 f(x2) f(x1) ||d(k)||2 ||f(x(k))||2

0 2.0000 4.0000 4.000 –3.000 1.4796 5.00001 0.5417 3.7500 –2.098 –2.486 0.3611 3.25312 0.8606 3.5806 –0.144 –0.247 0.0348 0.28623 0.8836 3.5546 –1.36×10-3 –3.72×10-3 4.92×10-4 3.95×10-3

4 0.8838 3.5542 –2.64×10-7 –1.00×10-6 1.33×10-7 1.04×10-6

It is noted that the Newton-Raphson method exhibits quadratic convergence near the solution; norms of both the correction and residual vectors in the last iteration are the square of those in the previous one.


5.1 Newton-Raphson Method (Cont’d) The method can be generalized for an n-dimensional

problem. Given the set of nonlinear equations:( ) nixfi ,,2,1,0 …== (5.7)

( ) ( ) ( )kkk dxx +=+1 (5.8) then a single step of the Newton-Raphson method is:

( )

( )( )

( )( )

( ) ( )( )

( )

( )( )

( )( )

( ) ( )( )

( )

( )( )

( )( )

( ) ( )( ) 0

0

0

22

11

22

22

21

1

2

11

22

11

1

1

=+∂∂

++∂∂

+∂∂

=+∂∂

++∂∂

+∂∂

=+∂∂

++∂∂

+∂∂

kn

kn

xnnk

x

nk

x

n

kkn

xn

k

x

k

x

kkn

xn

k

x

k

x

xfdxfd

xfd

xf

xfdxfd

xfd

xf

xfdxfd

xfd

xf

kkk

kkk

kkk

…

…

…

(5.9)

where d(k) is the solution of the linear set of equations:




Newton-Raphson Algorithm:

Step 1: Initialize estimate, x(0) and k = 0.


Step 5: Test for convergence: use 2-norms for d(k) and f(x(k)).Consider scaling both. If convergence criteria are satisfied, END, else k = k + 1 and go to Step 2.

( ) ( ) ( )kkk dxx +=+1Step 4: Update x(k+1):

( ) ( )( )[ ] ( )( )kkk xfxJd 1−−=Step 3: Solve for d(k):

( )( ) ( )( )( )kxj

ikji

kxfxJxJ

∂∂

=≡ ,Step 2: Compute Jacobian matrix:


ISSUES 1: Initial estimate of the solution.• When solving a large set of nonlinear equations or a

set of highly nonlinear equations, the starting point used is important.

• It the initial guess is not a good estimate of the desired solution, the N-R method can seek an extraneous root or may not converge.

• A robust method for estimating a good starting point for the N-R method is the method of steepest descent, which is described next.





ISSUES 2: Computation of the Jacobian matrix.• The Jacobian matrix can either be determined analytically

or can be approximated numerically. • For numerical approximations, it is recommended that

central difference rather than forward or backward difference approximations be employed.

• If the system of nonlinear equations is well behaved (i.e. if they can be solved relatively easily), either method works well. However, for ill-behaved systems, analytical determination of the Jacobian is preferred.

• Apart from the computationally intensive Jacobian matrix calculation, the N-R method involves matrix inversion. Adopting so-called Quasi-Newton methods reduces these computational overheads.



• An accurate estimate of the solution of f(x) = 0 is necessary to fully exploit the quadratic convergence of the N-R method.

• The method of steepest descent achieves this robustly - it estimates a vector x that minimizes the function:

5.2 Method of Steepest Descent

Steepest Descent Algorithm: Step 1: Evaluate g(x) at x(0). Step 2: Determine steepest descent direction. Step 3: Move an appropriate amount in this direction and update x(1).

(((( )))) (((( ))))(((( ))))2

1

n

ii

g x f x====

==== ∑∑∑∑ - minimized when x is a solution of f(x).




• For the function g(x), the steepest descent direction is -∇g(x):

5.2 Method of Steepest Descent (Cont’d)

( ) ( )( ) ( )( ) ( )( )

( ) ( ) ( ) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( ) ( )

( ) ( )xfxJ

xxfxfx

xfxfx

xfxf

xxfxfx

xfxfx

xfxf

xxfxfx

xfxfx

xfxf

xfx

xfx

xfx

xg

T

T

nn

nnn

nn

nn

Tn

ii

n

n

ii

n

ii

2

2...22

,....2...22

,2...22

,,,

22

11

22

22

2

11

11

22

1

11

2

1

2

12

2

11

=

∂∂

++∂∂

+∂∂

∂∂

++∂∂

+∂∂

∂∂

++∂∂

+∂∂

=

∂∂

∂∂

∂∂

=∇ ∑∑∑===

…


Figure illustrates contours of g(x1, x2) = x12 + x2

2


Here, -∇g(x) = -(2x1, 2x2)T

at x = [2,2]T, -∇g(x) = -(4,4)T

at x = [-3, 0]T, -∇g(x) = (6, 0)T




Defining the steepest descent step:


Need to select the step length, α, that minimizes:

( ) ( ) ( )( ) 0, 001 >∇−= αα xgxx (5.15)

( ) ( ) ( )( )( )00 xgxgh ∇−= αα (5.16) Instead of differentiating h(α) with respect to α, we

construct an interpolating polynomial, P(α) and select α to minimize the value of P(α).

g1

g2

g3

α

h(α) P(α)



α̂

Interpolating polynomial, P(α):

where:(((( )))) (((( )))) (((( )))) (((( ))))2

1 1 1 1 1 2P g g gα α α α α α α∆ ∆= + − + − −= + − + − −= + − + − −= + − + − −

13

121

2

23

232

12

121 and ,

αααααα −∆−∆

=∆−−

=∆−−

=∆ggggggggg

(5.17)

∆∆

−=1

21

25.0ˆgg

αα

Differentiating (5.17) with respect to α:

α1 =0Set α3 so that g3 < g1.Set α2 = α3/2.

h(α)

α

g1

g2

g3

P(α)

α1 α2 α3

Hence, update for x(1) is:( ) ( ) ( )( ) ˆ 001 xgxx ∇−= α




Example: Estimating initial guess for N-R using the method of steepest descent (from x(0) = [0,0]T)


k α x1 x2 g(x(k)) 0 - 0.000 0.000 2271 1.234 0.299 1.198 48.522 0.881 0.912 1.831 16.313 1.189 0.296 2.848 11.974 0.419 0.696 2.973 6.2845 0.400 0.587 3.358 2.3776 0.286 0.862 3.435 0.562

Notes: (a) Values of α greater than 1 imply extrapolation.(b) Convergence is much slower that with N-R.


5.3 Wegstein’s Method Recall one-dimensional successive substitution:

( ) ( )( )kk xgx =+1 (5.19) Convergence rate of Eq.(5.19) depends on the gradient

of g(x). For gradients close to unity, very slow convergence is expected. As shown on the right,using two values of g(x),a third value can be predicted using linearextrapolation, using theestimated gradient:

( )

( ) ( )( ) ( )( )( ) ( ) s

xxxgxg

dxxdg

xx

≡−−

≈

=

01

01

1 : At




5.3 Wegstein’s Method (Con’t)

Linear interpolation gives an estimate for the function value at the next iteration, x(2):

( )( ) ( )( ) ( ) ( )( )1212 xxsxgxg −⋅+=

However, since at convergence, x(2) = g(x(2)), then:( ) ( )( ) ( ) ( )( )1212 xxsxgx −⋅+=

Defining q = s/(s -1), the Wegstein update is:( ) ( )( ) ( ) ( ) qxqxgx ⋅+−⋅= 112 1 (5.21)

Note: for q = 0, Eq.(5.21) is generic successive substitution


5.3 Wegstein’s Method (Con’t) For a nonlinear set of equations, the method of successive

substitutions is:( ) ( ) ( ) ( )( ) nixxxgx k

nkk

ik

i , ,2,1 ,, ,, 211 …… ==+ (5.22)

This method is commonly used in the solution of material and energy balances in the flowsheet simulators, where sets of equations such as Eq(5.22) are invoked while accounting for the unit operations models that are to be solved.

In cases involving significant material recycle, the convergence rate of the successive substitution method can be very slow, because the local gradient of the functions gi(x) are close to unity. In such cases, the Wegstein acceleration method is often used.




5.3 Wegstein’s Method (Con’t) For the multivariable case, the Wegstein method is

implemented as follows:1 Starting from an initial guess, x(0), Eq(5.22) is applied twice

to generate estimates x(1) = g(x(0)) and x(2) = g(x(1)), respectively.

( ) ( ) ( )( ) ( ) ( ) ( )( )( ) ( ) ni

xxxxxgxxxgs k

ik

i

kn

kki

kn

kki

i , ,2,1 ,, ,,, ,,

1

112

1121 …

……=

−

−= −

−−−

2 From k = 1, the local gradients si are computed:

3 The Wegstein update is used to estimate x(2) and subsequent estimates of the solution:

( ) ( ) ( ) ( )( ) ( ) ( ) ……… ,3,2 ,, ,2,1 ,1, ,, 211 ==⋅+−⋅=+ kniqxqxxxgx i

kii

kn

kki

ki

…… ,3,2 ,, ,2,1 ,1

wherei

ii ==

−= kni

ssq



When solving sets of nonlinear equations, it is often desirable to ensure that none of the equations converge at rates outside a pre-specified range.

Resetting values of qi that fall outside the desired limits (i.e., qmin < qi < qmax) ensures this.

Value of qi Expected convergence0 < qi < 1 Damped successive substitutions.

Slow, stable convergenceqi = 1 Regular successive substitutionsqi < 0 Accelerated successive substitutions.

Can speed convergence; may cause instabilities




Example Application: The figure shows a flowsheet for

the production of B from raw material A. The gaseous feed stream F , (99 wt.% of A and 1 wt.% of C, an inert material), is mixed with the recycle stream R (pure A) to form the reactor feed, S1. The conversion of A to B in the reactor is wt. fraction The reactor


P

S3

L

S2

R

S1F

Mixer

Separator

Reactor

Splitter

products, S2 are fed to the separator, that produces a liquid product, L , containing only B, and a vapor overhead product, S3, which is free of B. To prevent the accumulation of inertsin the synthesis loop, a portion of stream S3, P is purged.


P

S3

L

S2

R

S1F

Mixer

Separator

Reactor

Splitter• The material balances for the flowsheet consist of 15 equations involving 19 variables (4 degrees of freedom).

• Since it is known that ξ = 0.15, XA,F = 0.99 and F = 20 T/hr, the effect of P on the performance of the process is investigated by solving the 15 equations iteratively:1 Initial values are assumed for R and XA,R (usually zero).

2 The three mixer equations are solved.

3 The four reactor equations are solved.

4 The five separator equations are solved.

5 The three splitter equations are solved. 6 This provides updated estimates for R and XA,R. If the

estimates have changed by more than the convergence tolerance, return to Step 2.





P

S3

L

S2

R

S1F

Mixer

Separator

Reactor

Splitter• Steps 1 to 5 are equivalent to the two recursive formulae:


( ) ( ) ( )( )constants,,, ,11 PXRgR k

RAkk =+

( ) ( ) ( )( )constants,, ,21

,kRA

kkRA XRgX =+

• For example, Eq.(5.25) is generated using the process material and energy balances:

( )

( )

( )( ) ( )RXPXF

PRFRXFX

RF

PXSPSR

RAFA

RAFA

SA

,,

,,

1,

11

1

11 3

⋅−+−⋅−=

−

+

+−+=

−⋅−=

−=

ξξ

ξ

ξ


P

S3

L

S2

R

S1F

Mixer

Separator

Reactor

Splitter• Hence, the recursion formula for R(k+1) is:


• Similarly, the recursion formula for XA,R(k)is:

( ) ( )( ) ( )( ) ( ) ( )( )PRgRXPXFR kkkRAFA

k ,11 1,,1 =⋅−+−⋅−=+ ξξ

( ) ( ) ( ) ( )( )( ) ( )( )k

RAk

FA

kRA

kFAk

RA XRFXXRFX

X,,

,,1, 1

1 +−

+−=+

ξ

ξ

• Substituting for known XA,F and F in Eq.(5.27):( ) ( )( ) ( ) ( )( ) ( )kk

RAkk RXPPRgR ,1

1 15.0103.17 , −+−==+

• This means that local gradient of g1 is:

( )( )( )k

RAk XRg

,1 15.01 −=

∂

∂( ) 185.0 1 <

∂∂

< kRg( ) 10 , << k

RAX





This means that successive substitution will converge very slowly in such cases


Having completing this lesson, you should now be able to:– Formulate and implement the Newton-Raphson method for a set

of nonlinear equations. You should be aware that an accurate initial estimate of the solution may be required to guarantee convergence.

– Use a steepest descent method to provide robust initialization of the N-R method. Since this method only gives linear convergence, the N-R method should be applied once the residuals are sufficiently reduced.

– Formulate and implement the multivariable extension of the method of successive substitution, including acceleration using Wegstein’s method. This method is commonly used in the commercial flowsheet simulators to converge material and energy balances for flowsheets involving material recycle.

Summary

Wegstein Method Metodos Numericos

Documents

Transcript of Wegstein Method Metodos Numericos