engg-prob-calculus 3

7/29/2019 engg-prob-calculus 3

1/15

Engineering Computation ECL3-1

ENGINEERING COMPUTATION Lecture 3

Stephen RobertsMichaelmas Term

Iterative Solution of Simultaneous Equations

Topics covered in this section:

1. Iterative algorithms and the need to test their convergence.2. The Jacobi method of solution to solve Ax=b3. The Gauss-Seidel algorithm.4. Awareness of other numerical approached to solving Ax=b


Introduction

So far we have discussed the solution of the simultaneous linear equationset bAx = , and the conditions for ill-conditioning.

The underlying assumption was that we would go ahead and solve the setof equations by a direct method, such as Gaussian elimination.

How many calculations would this take?

Consider Nequations in Nvariables, where A is an NNmatrix. Assumeadditions are quick, and multiplications cost time.


2/15


Example: Consider using the echelon method to solve

=

4

3

1

643

432

321

z

y

x

Process the augmented matrix as follows:

1

1

1

100

210

321

1

1

1

320

210

321

4

3

1

643

432

321

2row2-row33row1-row3

2row1-row2

Then back-substitute: 21321121 ==++=== xzyxyz-y-z .. .

The reduction to echelon form took 42 + 31 = 11 multiplications.The back substitution took a further 1 + 2 + 3 multiplications or divisions,making a total of 17.


Introduction

General case:

Consider Nequations in Nvariables, where A is an NNmatrix. Assumeadditions are quick, and multiplications cost time.

Reduction to echelon form will take ~ N(N-1) + (N-1)(N-2) + ..... ~N

3 multiplications. For large systems (say 103 equations) this can bea very large number (~ 109), and solutions may take a long time, evenwith modern fast computers.

Can we use a better approach?


3/15


4/15


Revision: Iterative solutions of equations in one variable

We attempt to solve the equation ( ) 0=xf . e.g. ( ) 0cos == xxxf .

Rearrange the equation into the form ( )xgx = e.g. xx cos= .

Choose a starting value x0, and repeat the iterative calculation

( ) LL,2,1,0,1 ==+ nxgx nn until xnconverges on a solution.


In our example, using nn xx cos1 =+ , we get the following results, starting with x0

= 0.7000:

n xn Error en0 0.7000 -0.03911 0.7648 0.02582 0.7215 -0.01763 0.7508 0.0117... ..........10 0.7383 -0.007

This converges (somewhat slowly! Try it on your calculator) to the solution0.7391 accurate to 4 figures.

But there is no guarantee that the method will converge!

Consider an alternative formulationnn

xx 11 cos

+= .

n xn0 0.70001 0.79542 0.65113 0.8617 This is diverging!

=> need a method to show will converge


5/15


Test for Convergence

Let the true solution of ( )xgx = be sx = , so that ( )sgs = .

The error en+1 after n+ 1 iterations is given by

( ) ( )sgxgsxennn

==++ 11

Expanding the first two terms of a Taylor series about sxn

= ,

( ) ( ) ( ) ( ) ( ) ( ) LL ++=++= sgesgsgsxsgxgnnn

Thus ( )sgee nn +1 . (1)

The iteration scheme will converge if nen

as0 .


Thus ( )sgeenn

+1

.

The iteration scheme will converge if nen

as0 .

From ((1), previous slide) The condition for convergence is ( ) 1


6/15


Iterative Methods for solving Ax=b


Iterative Methods for solving Ax=b

Lets now consider how a related idea can be applied to develop aniterative technique for solving Ax=b.

As with the case of the one dimensional case, lets translate Ax=binto an equivalent system of the form x=Tx+c.

Then having selected an initial guess x0 we can apply a similar

method to that outlined above.


7/15


Jacobi Algorithm for solving bAx = .

Divide A into a diagonal matrix D and two triangular matrices L and U .

For a 33 matrix,

ULDA ++=

+

+

=

=

000

00

0

0

00

000

00

00

00

23

1312

3231

21

33

22

11

333231

232221

131211

A

AA

AA

A

A

A

A

AAA

AAA

AAA

Rewrite bAx = as ( ) bxULD =++ and re-arrange to give( )xULbDx += .

Now, the inverse D-1 of D is simply the diagonal matrix of the inverse ofthe diagonal elements of A, so write

( ) cBxbDxULDx +=++= 11

This is in the required form Tx+c and suggests the Jacobi iterative

scheme:

( ) cBxbDxULDx +=++= + nnn

11

1


Example: Jacobi solution of weighted chain.

Consider a hanging chain of m+ 1 light links with fixedends at height x0 = xm+1 = 0.

It could be the supporting chain for the Clifton

suspension bridge in Bristol, shown .

There is unit tension in the chain, the links are of unit

length.

A downward force bj is applied on the jth hinge.

If we make the approximation that the links make small angles to the horizontal,resolving forces vertically at each hinge gives:

mjbxxx jjjj =+ + 1,2 11


8/15


This is a matrix equation of the form bAx = . In the case m= 5, there are 5links

=

21000

12100

01210

00121

00012

A

Dividing the matrix A into D + L+ U , and taking the case where

[ ]T1,1,1,11,=b gives the Jacobi iteration algorithm in the form

( ) cBxbDxULDx +=++= + nnn11

1 :

=+

21

212

1

21

21

21

21

21

2

1

2

1

21

21

21

1

0000

000000

000

0000

nn xx


this is easy to code up in MATLAB.

% Jchain.m

% solves weighted chain with m+1 links with n iterations

function jchain(m,n)

b = ones(m+2,1); % array of 1's - downward forces

x = zeros(m+2,1); % array of 0's - initial guess of position

xnew = x;

fprintf('Iteration 0: ');

fprintf('%.4f ',x);

fprintf('\n');

for i = 1:n % n iterations

for j = 2:m+1 % miss out end points

xnew(j) = 0.5*(x(j-1) + x(j+1) - b(j));

end

x = xnew;

fprintf('Iteration %2d: ', i);

fprintf('%.4f ',x);

fprintf('\n');

end


9/15


Running this, with m = 5 gives, for the first 10 iterations:

jchain(5,10)

Iteration 0: 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000

Iteration 1: 0.0000 -0.5000 -0.5000 -0.5000 -0.5000 -0.5000 0.0000

Iteration 2: 0.0000 -0.7500 -1.0000 -1.0000 -1.0000 -0.7500 0.0000

Iteration 3: 0.0000 -1.0000 -1.3750 -1.5000 -1.3750 -1.0000 0.0000

Iteration 4: 0.0000 -1.1875 -1.7500 -1.8750 -1.7500 -1.1875 0.0000

Iteration 5: 0.0000 -1.3750 -2.0313 -2.2500 -2.0313 -1.3750 0.0000

Iteration 6: 0.0000 -1.5156 -2.3125 -2.5313 -2.3125 -1.5156 0.0000

Iteration 7: 0.0000 -1.6563 -2.5234 -2.8125 -2.5234 -1.6563 0.0000

Iteration 8: 0.0000 -1.7617 -2.7344 -3.0234 -2.7344 -1.7617 0.0000

Iteration 9: 0.0000 -1.8672 -2.8926 -3.2344 -2.8926 -1.8672 0.0000

Iteration 10: 0.0000 -1.9463 -3.0508 -3.3926 -3.0508 -1.9463 0.0000

It eventually converges to 4 significant figures in 80 iterations:

Iteration 76: 0.0000 -2.5000 -3.9999 -4.4999 -3.9999 -2.5000 0.0000

Iteration 77: 0.0000 -2.5000 -3.9999 -4.4999 -3.9999 -2.5000 0.0000

Iteration 78: 0.0000 -2.5000 -3.9999 -4.4999 -3.9999 -2.5000 0.0000

Iteration 79: 0.0000 -2.5000 -4.0000 -4.4999 -4.0000 -2.5000 0.0000

Iteration 80: 0.0000 -2.5000 -4.0000 -4.5000 -4.0000 -2.5000 0.0000Iteration 81: 0.0000 -2.5000 -4.0000 -4.5000 -4.0000 -2.5000 0.0000

You can substitute back into the equation Ax = b to check it is the solution!


To really go to town, we ran a 50 link chain in the figure below. As you can see ittook about 4000 iterations to converge. Note similarity of n= 2000 and n=4000 curves. Were getting close!


10/15


Convergence of Matrix Iterations

We need a test for convergence for matrix iteration analogous to the oneused for equations in one variable earlier in this lecture. Since we have astructure Tx+c we can use a similar argument:

For an iteration algorithm of the form cBxx +=+ nn 1

, let the exact solution

be the vector s. Then cBss += .

Define the error vector sxe =nn

.

Then ( ) ( ) ( )nnnnn

BecxBcBscBxsxe ==++==++ 11

Using the compatibility between matrix and vector norms,

nneBe

+1

Thus the iteration cBxx +=+ nn 1

converges if nn

ase 0 if there is

a matrix norm such that1


11/15


An alternative representation of theAn alternative representation of theAn alternative representation of theAn alternative representation of the JacobiJacobiJacobiJacobi AlgorithmAlgorithmAlgorithmAlgorithm

As outlined above we have written the Jacobi algorithm in matrix form. This was

convenient in order to understand the convergence properties of the method.

Alternatively, we could write the technique in its matrix element form as follows:

for j=1,2,.N, and n indicates the iteration index.

(If you have problems in seeing that this is so, compare with the Matlab example

above.)

This is a useful representation of the algorithm, both from the viewpoint of

implementation, but also because it suggests an improvement to the method that

we discuss next.

It also highlights the fact that we cannot have the conditionWhat could we do in this case?

,1

)(

)1(

ii

iN

jii

n

jijn

ia

b

a

xax +

=

=

+

0=ii

a


GaussGaussGaussGauss----Seidel AlgorithmSeidel AlgorithmSeidel AlgorithmSeidel Algorithm

the Jacobi algorithm ( ) ccccBxBxBxBxbbbbDDDDxxxxUUUULLLLDDDDxxxx +=++= + nnn11

1 updates the whole of the new

vectorxxxxn+1 from the old vectorxxxxn every iteration. But if we look at the MATLABalgorithm,

for j = 2:m+1 % miss out end points

xnew(j) = 0.5*(x(j-1) + x(j+1) - b(j));

end

we see thatxnew(j) depends onx(j-1).

But xnew(j-1) was calculated last time round the loop, so is now available in shiny newupdated form! so lets use it and modify the code to

. for j = 2:m+1 % miss out end pointsxnew(j) = 0.5*(xnew(j-1) + x(j+1) - b(j));

end


12/15


gschain(5,50)

Iteration 0: 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000

Iteration 1: 0.0000 -0.5000 -0.7500 -0.8750 -0.9375 -0.9688 0.0000

Iteration 2: 0.0000 -0.8750 -1.3750 -1.6563 -1.8125 -1.4063 0.0000

Iteration 3: 0.0000 -1.1875 -1.9219 -2.3672 -2.3867 -1.6934 0.0000

.................................................

Iteration 40: 0.0000 -2.5000 -3.9999 -4.4999 -4.0000 -2.5000 0.0000

Iteration 41: 0.0000 -2.5000 -4.0000 -4.5000 -4.0000 -2.5000 0.0000

This method, Known as the Gauss-Seidel method, converges more rapidly than the Jacobimethod.


The general formulation of the Gauss-Seidel method is

bDUxDLxDx11

1

1

1

+

+++=

nnn

It looks strange to see the term1

1

+

nLxD on the right hand side, but remember

that, since the matrix multiplications are carried out row by row, and L islower triangular, each line of

1

1

+

nLxD contains only references to elements of

1+nx which are already known


13/15


Elemental FormElemental FormElemental FormElemental Form

The elemental version of the Gauss-Seidel method also clearly shows why the

method is an improvement on the Jacobi method.This takes the form

where j=1,2,..N, and n indicates the iteration index as before.

ii

i

j

i

N

ij

n

jij

n

jij

n

ia

bxaxa

x

= +=

+

+

+

=

1

1 1

)()1(

)1(

)()(


GaussGaussGaussGauss----Seidel AlgorithmSeidel AlgorithmSeidel AlgorithmSeidel Algorithm convergence propertiesconvergence propertiesconvergence propertiesconvergence properties

Let us start from the elemental form of the Gauss-Seidel iteration equation where j=1,2,..N,

n indicates the iteration index as before.

Multiply by and collect the terms on the left hand side gives for i=1,2,N

Hence there are N equations

ii

i

ji

N

ij

n

jij

n

jijn

ia

bxaxa

x +

=

= +=

+

+

1

1 1

)()1(

)1(

)()(

iia1+nx

ibnNxNia

Ni

xiianixiia

nxianxia +++

=+++++ ,.........11,1...........1

221

11

Nbn

NxNNansxNa

nxNa

bnN

xN

anxanxanxa

bn

NxNan

xan

xan

xa

+=+

++

++

+=+++

+=+

1...........

12

111

223231

2221

121

113132121

111

M

M

KK

KK


14/15


Hence

or

This is in the standard form Tx+cTx+cTx+cTx+c,,,, hence the convergence condition is

Nbn

NxNNansxNa

nxNa

bn

N

x

N

an

xan

xan

xa

bn

NxNan

xan

xan

xa

+=+

++

++

+=+++

+=+

1...........

12

111

22323

1

222

1

121

113132121

111

M

M

KK

KK

bUxL)xDn1n

+=++

(

bL)(DUxL)(Dx1n11n +

+++=

1

engg-prob-calculus 3

Documents

Transcript of engg-prob-calculus 3