FLORIDA STATE UNIVERSI TY ITERATIVE ME HODS FOR THE...
Transcript of FLORIDA STATE UNIVERSI TY ITERATIVE ME HODS FOR THE...
,
),lay, 1958
..
FLORIDA STATE UNIVERSITY
ITERATIVE METHODS FOR THE SOLUTION
OF LINEAR EQUA1:IONS
By STANLEY R. pENDER,
A Paper Submitted to the -Graduate Council of Florida State University in partial fulfillment of the re~uirements ' for the degree of Master of Science.
Approved :_·JA~(l~'~IQ~-.J:""":'1..1..c~~~~!4-_ Professor D
none lfinor Professor
~~e~/ R 11
Dean of the Graduate School
FLORIDA STATE UNIVERSITY LlBRARIBB
MAR 14 2001
TALLAHASSEE, FLORIDA
FLORIDA STATE UNIVERSI TY
ITERATIVE ME HODS FOR THE SOLUTION
OF LINEAR EQUATIONS
BySTANLEY R. I ~END~,
A Pa erSubmitted to the Graduate Council ofFlorida State University in partialfulfillment of the requirements 'forthe degre of Master of Science.
Approved: .p~Professor D
Yay, 1958
noneMinor Professor
FLORIDA STATEUNIVERSITY L1BRARIBS
MAR 14 2001
TALLAHASSEE, FLORIDA
4CKNOlILEDGEMENTS
I wish to thank Dr. Paul J. llcCarthy for his patient assistance
in the preparation of this paper.
11
I wish to thank Dr. Paul J. McCarthy for his patient assistance
in the preparation of this paper.
11
TABLE OF CONTEtITS
ACKNOWLEDGEMENTS • • • • • • • • • • • • • • • • Page
11
Chapt.er I. INTRODUCTION • • • • • • • • • • • • • • 1
II. ITERATIVE METHODS
l~ Analytic basis • • • • • • •• •• 7 2~ Method or steepest descent • • • • • 10 3; Gauss-8eidel method •••••••• 12 4; Method of · relaxation • • •• ••• 13 5. Summary • • • • • • • • • • • • • • 14 6. Examples • • • • • • • • • • • • • • 16
BIBLIOGRAPHY • • • • • • • • • • • • • • • • • • 21
iii
TABLE OF CONTmITS
ACKNOWLEOOEMENTS • • · . • • • • • • . . . . . . Pageii
ChapterI. INTRODUCTION • • • • • • • • • • • • • • 1
II. ITEM.TIVE METHODS
l~ !nalytic basis • • • • • • • • • • • 12~ ethod of steepest descent • • • • • 103~ Gauss-Seidel method ••• • • • •• 12.4~ Method ot'relaxation • • • • • • • • l}5. Summary • • • • • • • • • • • • • • ]lJ.6. Examples • • • • • • • • • • • • • • 16
BIBLIOGRAPHY • • • • • • • • • • • • • • • • • • 21
iii
CHAPTER I
INTRODUCTION
The numerical solutions of ma~ types of problems are .generally
obtained by solving approxi'll8ting linear algebraic systems. Moreover,
in solving a nonlinear problem, one may replsce it by a sequence of
linear systems providing pr.ogressively improved approximations. l For
the study of these linear systems of equations a geometric termin-
ology with the compact symbolism of vectors and matrices is useful.
A resume ~f the basic principles of higher algebra necessary for the
development of the material to follow is therefore included.
A vector x of order n is defined to be a set of n real numbers
Xl' %2' ••• , xn arranged in a definite sequence and is written
x • (Xl' %2' ••• , xn). The vector 'f is wr:I. tten
Y - (Yl' Y2' ••• , Yn).
Equali ty of vectors x - y is defined to mean simultaneous equaU tr of
similarly numbered real number elements, xt • Yi for i • 1, 2, ••• , n.
The multiplication of 1by any real number c is defined as the
vector
!Householder, Alston S., P~incip1es of Numerical Analysis (New Yorb McGra_Hill Book Company, Inc., 1953) p. 44.
1
CHAP'TER I
INTRODUCTI ON
The numerical solutions of many types of problem aTe generally
obtained by solving approxl1!Jating linear algebraic systems. Moreover,
in solving a nonlinear problem, one may replace it by a sequence of
linear systems providing pr.ogressively improved approximations.l For
the study of these linear systeJllB of equations a geometric termin-
ology lIi th the compact symbolism of vectors and matrices is useful.
A resume of the basic principles of higher algebra necessary for the
development of the material to fol101'f' is therefore included.
A vector x of order n is defined to be a set of n real numbers
Xl' x2 ' • • • , xn arranged in a definite sequence and is WTitten
x • (Xl' x2' ••• , xn).
The vector y is written
Y• (Y1' Y2' • • • , Yn).
Equality of vectors x • -; is defined to mean simultaneous equality of
similarly numbered real number elements, ~ • Yi for i • 1, 2, ••• , n.
The multiplication of X by any real number c is defined as the
vector
IHouseholder, Alston S., Principles of Numerical Analysis (NewYork: McGraw-Hill Book Company, Inc., 1953) p. ,44.
1
2
Addition of two vectors x + y is defined by
(2) it + y • (xl + YIJ ~ + Y2J ••• , xn + Yn).
The totality of all vectors for a fixed integer n with the operations
(1) and (2) is called an n-dimensional vector space over the field of
real numbers R and is written Vn(R). A 8ubeet of the vectors in the
vector space Vn(R) which is closed with respect to the two basic opera
tions of vector algebra' (1) and (2) is called a vector subspace of Vn(R).
From (2) it follows that there is a unique null vector
~ • (OI, 02, ••• , On)
wi th the property that for any vector xJ
, for all it in Vn(R) and all c in R. For any vector ~ in Vn(R) there
is a unique vector x, such that x +~, • 0, namely the vector
:t' • (-xl' -~, • • , -Xu).
This vector x' is called the negative otiC and is usually denoted by
4. Since x' • (-l)~J the definition at -it may be written -1 • (-1)%.
Subtraction of vectors is defined by the equation x - '1 • x + (-1)1.
Vectors also have the following properties, valid tor all vectors x, Xl.
~ •••• , ~ in V (R) and all scalars c J cl' c2' ••• , cn in R:
(Xl + x2) + x, • Xl + (x2 + 'x,) ,
Xl + x2 • t2 + Xl,
c(xl + x2 + .••• + Xu) • cXl + c~ + ••• + c~,
{cl + c2 + ••• + cnlx • clx + ciX + ••• + cJ:. and
(clc2)'% • cl (c2'f).
2
Addition of two vectors i + Yis defined by
(2) x + Y• (xl + Y1' ~ + Y2' • • • , xn + Yn).
The totality of all vectors for a fixed integer n w-lth the operations
(1) and (2) is called an n-dimensional vector space .over the field of
real numbers R and is written Vn(R). A subset of the ~c.tors in the
vector space Vn(R) whieh is closed with respect to the two basic opera
tions of vector algebra (1) and (2) is called a vector subspace of Vn(R).
From (2) it f'ollows that there is a unique null vector
~ • (0}, 02, • • • • on)
'Wi th the property that for any vector ~,
, "0 • 'x • 0 and
f'or all x in Vn(R) and all c in R. For any vector x in Vn{R) there
is a unique vector~' such that x + x' '" 0, namely the vector
x' • (-Xl' -X2, • • , -xn).This vector xt is called the negative of x and is usually denoted by
-x. Since x' • (-l~, the definition of -x may be written -~ • (-1)%.
Subtraction of' vectors 1s defined by the equation x - 'i • x + (-l)"YO.
Vectors also have the following properties, valid for all vectors x, Xl.
x2, ••• , ~ in V (R) and all scalars c, el' c2' ••• , cn in R:
(Xl + x2) + x3 • Xl + (x2 + "X3)'
Xl + 'x2 • :t2 .. Xl'
e(xl + x2 + .•• e + Xn) - eXI + eX2 + • • • + exn,
(01 + c2 + e .• • + cnli • C1X + C2X + • e • + c~ and
(c1c2):X • cl(c2~)·
If any n vectors :tl , ~, •••• ~ lie in a vector space Vn(R).
they are called linearly dependent if there are scalars ch c2. • •
• • cn• not all zero, such that
c 'Ij. + c ~ + ••• + c X • 0. l-~ 22 nn
If no such scalars exist. the set~, x2 •••• , -in is called lin-
early independent. A vector space is .-dimensional if m is the max-
imIm number of linearly independent vectors in the space. Any 11\
linearly independent vectors in the space then constitute a coord i-
nate system for the space and any vector in the space can be written
as a linear combination of these linearly independent vectors.
We shall refer to the following set of n-dimensional vectors of
Vn(R) as the Ii coordinate systeJl1 for Vn(R)
el
• (1, 0, 0, • •• , 0)
"2 . (0, 1, 0, 0 • . . . , 0) • • • • • • en • (0. 0 •• • • , 0, 1) •
With regard to the vector X the scal:lrs let. for i • 1, 2, ••• , n
are called the coordinates of i in the. s;ylItem ar.d the vectors let1!i
are ita components.
When the coordinates of -f in the e fI;ylItem are arrsnged in column
form - the arrangement is known as a column matrix which we shall
deeignate by xe. The arrangeJl1ent (~l' ~ ••••• ~n) is called
a row matrix. Any rectangular array of numbers i4 called a rectan-
3
If any n vectors iI' ~2' ••• , ~ lie in a vector space Vn(R),
they are called linearly dependent if there are scalars clJ c2_ • •
• , cn, not all z ro, such that
c y. + e x +. • • + c'x • o.r1. 22 nn
If no such scalars exist, the eet~, )[2' ••• , .~ is called lin-
early independent. .It.. vector space is m-dimensional if m is the max-
!mum nuD!ber of linearly independent vec'tors in the space. Any m
linearly independent vectors in the space then constitute a cQordi-
nat.~~ system for the space and any vector in the space can be written
as a linear combination of these linearly independent vectors.
We shall refer to the following set of n-dimensional vectors of
Vn(R) as thee coordinate system for Vn(R)
e1 • (1, 0, 0, • • • , 0)
....(0, 1, 0, 0, 0)e2 • • • • ,
• •• •• •en .. (0, 0, • • • , 0, 1).
With regard to the vector x the scab.rs %t. lor i • 1, 2, ••• , n
are called the coordiDCltea of i in th a system ar.d the vectors ~lEi
are its coq:> nents.
When the coordinates of'x in the e system are arranged in column
form the arrangement is known as a column matr1Jc which we shall
designate by xe• The arrangement (XJ.1' XJ.2' ••• , "1n) is called
a row matrix. Any rectangular array of numbers is called a rectan-
4
gular matrix:-
'an 812 • • • aln a21 a22 • • • a2n
• • • • • • A • • • • • • • • [aijl~;
• • • • • •
aml 8M2 • • • a mn
throughout the discussion upper case Latin letters will be used to
designate matrices. The double subscripts indicate the tlro-<IIay
ordering of JIIItrices. The fint subSCript is the rOlf index 8nd the
second subscript is the column index.
Given any two matrices A and B • {bijJ~ where i • I, 2, ••• , p
and j ·1, 2, ••• , q; the" product AB is defined as the matrix n
C • r Ci!dm where cn • {aijbjkif n • p. The product III • D· dik R " q J-l
'" lI'here dik • j~' bija jk if III • q. In general JoB of BA. The identity
matrix under multiplication, I • fk:tjS~J is such that its elements
k:tj is equal to one it i • j and is equal to zero it i of j for all
values of i and j. The matrix I is therefore a diagonal matrix; that
is, it is square (has the same number of columns and rows) and all
elements off the main diagonal are zero.
The transpose of the matrix A, designated by attaching a super
script t to the matrix notation At • {aij'J: is such that for all aij
of A and all aji' of At, aij • a ji ' for all values of i and j. The
transpose of the column ma trix xe is the rOlf matrix xe t • (XIV x12'
• , • , xln)' The transpose of a rOlf matrix is a column matrix,
(Xet)t • xe ' The transpose of
4
gular matrix:-
"all 812 • • • aln
a2l a22 • • • 82n
• • • • • •A • • • • • • • • [aij}~;
• • • • • •aml Que • • • amn
throughout the discussion upper case Latin letters will be used to
designate matrices. The double subscripts indicate the two-way
ordering ot matrices. The fint subscript is the row index and the
second subscript is the column index.
Given any two matrices A and B • {bijJ~ where i • 1, 2, ••• , p
and j • 1, 2, ••• , q; the product AB is defined a8 the matrixn
G • fCitJ: wher cik· ~I 8 ijb jkif n p. The product BA. • D· dik KWI
where dik • j~r bij8 jk ltm • q. In general AB 'I BA. The identity
matrix under multiplication, I • f~j1R, is such that its elements
~j is equal to one if i • j and i9 equal to zero if i rf j for all
values of i and j e The matrix I is therefore a diagonal matrix; that
is, it is square (has the s me number of columns and rows) and all
elements off the main diagonal are zero.
The transpo.se of the matrix A, designated by attaching a super
8Cript t to the matrix notation At. { 1jtJ ~ is such that for all aij
of A and all 8ji t of At, 8ij • a ji' tor all values of' land j. The
transpose of the column JIB trix xe is the row matrix xet • (XII' %12'. .The transpose of a r~ trix is a column matrix,
5
m'" 1\1 AB • 2 ~ :::: a b • ~ i c • C is
j~I ~ . ,j" ij jk i'l k' ik 1(1 ' mn
(AB)t • Ct • ~ ~. cld. • t. ~,~, bkjaji • BtA t.
Let 11 , 12, ••• , 1n be another set ot linearly independent
vectors in this n-dimensional vector space; then each tj is express
ible as a linear colllbiDation of the 9 i 'S and vice versa. If we con
sider e am f to be row matrices,
e • (~, e2, ••• , en)
f • (fl , t 2 , ••• , f n>
in which the coordinates ot the ei ' s and t j , s, Wi th regard to the ~
am 1 coordinate systems respectively, are arranged column-Wise.
Then 1r9 can Write in abbreviated form,
e • f'E , r • eF • am
Then rtf • (eF}teF • Ftetl!F • Ftrr • rtF. H • Ht • frt. Since H • ~,
H ill called a symmetric matrix. Relati ng the e and r coordinate
system we have
I • ete • (f'E)tfE • Etrtf'E • EtHE.
If both e and t are unit (the only nonzero element of the vector is
equal to unity) orthogonal coordinate systems then H • I and
I • E~ • E~.
The geometriC vector -! can beeKpressed as the numerical vector
Xf, that is, as an n-tuple ot coordinates Xj' in the tj coordinate
systera. and 1r9 can wr1 te xf • Fxe-
We define the scalar or dot produet of two vectors x and Y in
the r coordinate systea by
5
m , n ~ ~
AB • ~ ~ ~ a b • ~ i c • C is'.:.) 110' j'l ij jk ~'-I "., ik
"I n "\ m n
(AB)t • Ct .~ ~,ck1 • ~t~i~/bkjaji • BtAt.
Let 11
, t2
, ••• , 1n
be another set or linearly independent
vectors in this n-dimensional vector space; then each f j is express
ible as linear combination of the e1 t s and vice versa. If we con
sider e and t to be row matrices,
e • (~, e2, •• • , en)
t • (fl' 12 , • •• , t n>in which the coordinates or the ei ' s and r j , s, with regard to the "i
and 1 coordinate sYBtems respectively, re rranged column-wise.
Then we can writ in abbreviated rorm,
e • f'E , t eF , am
H is called a symmetric matrix. Relating the e and t coordinate
systems we have
I • ete • (tE)tn • Etrttp; • EtHE.
If both e and f are unit (the only no ero element of the vector is
equal to unity) orthogonal coordinate systems then H • I and
I • Etrm • E~.The geometric vector -y can be spressed as the numerical vector
Xr, that ie, as an n-tuple of coordinates Xj' in the f j coordinate
systeru, and we can write xr • he-
We define the scalar or dot product of t 0 vectore X and :I in
the t coordinate ystelD by
6
and it is a pure number. If x and yare the same vector then the
scalar product is the square of the length,
x • x • IxrJ2 • XttHxr-
The matrix II is Baid to be positive definite since xttHxt • 0 and
equality can hold only when xf • O. Two vectors x and yare said
to be orthogonal to each other it their scalar product is zero.
The scalar product my be thought of geometrically as the product
of the length of one vector by the projection of the other upon it.
In the e coordinate sY!ltem
x • y • xetIYe • Xetye· l!:liYn· YU%tl· Yetxe • Y .1l'.
Similarly we can prove that the scalar product is ctllDllllltative in the
t coordinate 8Y!1tem.
The determinant of a square matrix A, m • n, is defined as
A • • • 8ni n
all permutatiOns ot 1, 2, • • • , n
where t is the number of transp08itions required to restore i l , ~,
••• , in to their natural ordering 1,2, ••• , .n. The matrix A
is nid to be nonsingular if the value of its determinant ill nODlllero.
If the row and column of a particular element aij ot a square
I118triX A are deleted. what remains is called the (n - l)x(n - 1) sub
DIIItrix ~j~ The scalar dij • (_l)i + jl"\jl is called the cotactor
of aij in A. The n x n matrix D • f djl.l g is called the adjoint ot A
and is denoted by 'A. The inverse of the matrix A (tor m • n) may
now be defined as A-l • A • tAl
6
and it is a pure number. If x and y 8r~ the same vector then the
scalar product is the square of the length,
x • i • /xrJ 2 • xftHxr "
The matrix Ii is said to be positive definite since %ttHxr • 0 and
equality can hold only when xf • O. Two vectors x and yare said
to be orthogonal to each other if their scalar product is zero.
The scalar product may be thought of geometrically as the product
of the length or one vector by the projection of the othe upon it.
In the e coordinate system
x • Y• xetIYe • :xetye· xliY:tl· Yli~l· yetxe • y • -X.
Similarly we can prove that the scalar product is connnutat.i in the
f coordi te system.
The determinant of a square matrix A, m • n, is defined a8
A • ~ (-l)tau 82i1 2
11 permutations
• •
of 1, 2,
•
• • • , n
where t is the number of transpositiona reqUired to restore 11' ~,
••• , in to tht~ir natural ordering I, 2, ••• I .n. The matrix A
is s3id to be nonsingular if t.hc value of its determinant is nonzero.
If the row nd column of 8 particular element 8ij of a square
matrix A are deleted, what remains is called the (n - 1)x(n - 1) sub
matrix ~j~ The 8calar dij • (_1)1 + j lMijl is c lIed the cofactor
of 8ij in A. The n x n mat.rix D • {dji) ~ is called the adjoint of A
and is denoted by1. The inverse of the matrix A (for m • n) may
now be defined as A-1. A •IAI
CHAPTER II
ITERATIVE METHODS
1. Anaqt:l.c bash. In solving any equation or systea of equat:l.oll8
there are two poss:l.ble approaches. We may be able to f:l.nd a solution
by means of a direct method which prescr:l.bes only a finite sequence of
operations whose c~letion yields an exact solution. When the nuaber
of unknowns ill great, the algebra:l.c ID9thodll become exceedingly labor:l.
OUII. The solution of a system Of . this type is then most easily found
by the use of an iterative method. Iterative methodll are ellpecially
valuable when machinell for handling the successive iterations effectively
are available, when a good approximation to the lIolution is known or when
we Wish to obtain solutions of adjusted equations.
In lIolving a set of m equations in m unknowns y - Ax • 0 an iter
ative method is defined as a rule for operating on an approximate solu-
tion ~ in order to obtain an improved solution ~ + 1; such that the
sequence [xJ so defined has the solution x as its limit.
Since the approximation Xo With which one may begin an i terati ve
method does not need to be ftcloseft to the solution x. We l118y start With
an arbitrary xo, however a good approxilllBtion to the solution will apprec-
iably diminish the number of iterations necessary to achieve a desired
accuracy. If no approximation With which to start the iterative IIIIIthod
is available take Xc • 0 unless the main diagonal of the coefficient
7
CHAPTER II
ITERATIVE :METHODS
1. Analytic basis. In solving any equation or system of equations
there are two possible approaches. We may be able to find a solution
by means of a direct method which prescribes only a finite sequence of
operations whose completion yields an exact solution. When the number
of unknowns is great, the algebraic methods become exceedingly labori
ous. '!"he solution of a system of this type is then most easily found
by the use of an iterative method. Iterative methods are especi81~y
valuable when chines for handling the successive iterations effeetively
are available, when a good approximation to the solation i known or when
we wish to obtain solutions of adjusted equations.
In solving a set of m equations in m unknowns y - Ax • 0 an iter
ative method 18 defined as a rule for operating on an approximate solu
tion ~ in order to obtain an improved solution Xp + 1; such that the
sequence f~ so defined has the solution x as i t8 limit.
Since the approximation X o with which one may begin an i tarative
method does not need to be Mclose" to the solution x. We may start wi th
an arb! trary xo, hO'll'ever a good approximation to the solution will apprec
iably diminish the number of 1tera tiona neeeesary to achieve a desired
accuracy. If no apprOXimation with which to start the iterative method
is available take Xc • 0 unless the main diagonal of the coefficient
1
g
matrix features relatively large terlllllJ then take xo· '"1/all.
Y2/a22 • • •
A large claaa of iterative methods 18 baaed upon the following
geometric idea: take any vector bo and a sequence of Tectora '\> and
define the sequence [bpJ by bo' ~ • ~ _ 1 - ¢pup where the scalar
¢p is chosen so that bp is orthogonal to~. It t.'le vecto!'8 '\> nfill
out" some m dimensional subspace of the vector space Vn(R) then the
vectors ~ approach as a lim t a vector b which is orthogonal to this
m-space. A possible choice for the Tectors ~ is any set of refer
ence vectors fi of the • space taken in order and then repeated
u.m + i • f i • where v • 1, 2, • • •• Let f • eF and H • FFt; then
we have
bp • ~ _ 1 - ;PUp. and iii nee ~ is orthogonal to '1>
o • '1> • ~ • Up • ~ - 1 - sfpllp • Up
• ~tHbp _ 1 - sfp~tH~
where I • UptRhp. _ l/UptHUp.
If we let y • Ax repreaent the set of m equations in m unknowns
to be solved and ~ represent any approxilllltion to the solution x,
then we have
rp • y - Ax" • Ax - Axp • A(x - xp) • Asp.
The coordinates of the Tector rp are called the residuals of the
equations. The s,mol rp and equivalently Asp are representations
g
matrix features Nlatively large terlDl!ll then take xo • Yl/all.
Y2/8 22•••
A large class of 1terative methods is based upon the following
geometric idea: take any vector bo and 8 sequence of vectors ~ and
define the sequence l bpJ by bo ' hp • bp _ 1 - ¢pup where the scalar
¢p is chosen so that bp is orthogonal to~. If the vectors ~ ufill
out" some m dimensional subspace of the vector space Vn(R) then the
vectors hp approach as a limit a vector b which is orthogonal to thi
space. A possible choice for the vectors ~ is any set of refer
ence vectors £i of the space taken in order and then repeated
~ + i • f i , where v • I, 2, • • •• Let f • eF and H • FFt; then
we have
bp • ~ _ I - ¢P'i>'and si nee ~ is orthogonal to '1>
o • Up • hp • Up • hp _ 1 - stpUp • Up
• ~tHbp _ I - ;p~tH~
where ~ • UptHbp _ l/uptHUp.
If we let y • Ax represent the set of m equations in m unknowns
to be solved and ~ represent any approximation to the solution x,
then we have
rp • y - Axp • Ax - Axp • A(x - Xp) • Asp.
The coordinates of the vector r p are called the residuals of the
equation. The symbol rp and equivalently Asp are representations
9
of the d.T1ation of the approximation trom the true solution.
Hence either rp or ep can be taken as bp'
Take the matrb: A to be positive definite. Any sY'lltem of
equations w • ax, B nonsingular, can be transformed by a premul
tiplication of Bt on both sides into a sY'lltem in which the coeffi
cient matrix is positive definit~. The solution of
y • Btw • Btax • Ax
as also the solution of w • Bx.
Let rp .~, then
., - Axp • bp • bp _ 1 - tlpUp • y - AXp _ 1 - ¢'pUp, and
~ • ~ _ 1 + "pUpf and
~ • ~ - 1 + -Il-1~.
Since Up is an arbitrary set of vectors lie may substitute '1> • Avp
to obtain
xp • xp _ 1 + t/pvp'
Now -Ip • Up ~ _ 1/ilp tHUp
• vp tA 'tHbp _ I/Vp tA tHAvp'
Take H • A-I, this gives us
'4 • vptrp _ l/VptAvp•
If on the other ham! lie take sp • bp then
y - Axp • A~ • A(bp _ 1 - "pUp) • y - Axp _ 1 - tJrf'1>f and
A~ • ~ _ 1 + -IpA'1>' and
~ • ~ - 1 + -Ip'1>" Now -Ip. ~Hbp _ 1/'1> tA'1>'
9
ot the d.nation ot the approximation from the true solution.
Hence either rp or Sp can be taken as bp•
Take the matru A to be positive de.finitee AnY' system of
equations w • Bx, B nonsingu1ar, can be transformed by a premu1
tip1ication of at on both sides into a system in which the coeffi
cient matrix is positive definit",. The solution ot
y • at" • BtBx • Ax
as also the solution of w • Bx.
Let r p • ~, then
y - Axp • bp • bp _ 1 - ¢pUp • y - AXp _ 1 - 4UpJ and
A.xp • A~ _ 1 + tpup, and
~ • ~ _ 1 + sip!-l~.
Since up is an arbitrary set of vectors we may substi tute IIp. vp
to obtain
XI> • Xp - 1 + ripvp •
Now Ip • UptHl>p _ l/llptHUp
• vp t.A. tHbp _ l/VptA tHAVp.
Take H • A-I, this gives us
If on the other ham we take sp • bp then
y - A:l]:> • A"P • A(bp _ 1 - 'pUp) • y - Axp _ 1 - s1pAUp, and
A:xp • A:J.P _ 1 + rlpAUp, and
~ • XI> - 1 + rip~.avr rip· ~Hbp _ 1/'\ltA~.
10
Take H • A then Ip • IIptrp _ J.lu~up.
When the vector fpllp is subtracted !'rom %p the new residual rp
may be thought of as a leg of a right triangle of which rp _ 1 is the
hypotenuse {provided up and rp _ 1 were not orthogonal}.
Any rule for selecting the vectors IIp (or equivalently the vp)
at each step defines a particular iterative process; of these the
following three aEe in common use.
2. Ilethod of steepest descent. This method prescribes that we
take up • rp _ 1., Since A in the equation y - Ax • 0 will be taken to
be positive definite, this implies that the matrix A can be expressed
as the product of a matrix C multiplied by its transpose Ct , A • ctc,
where the matrix C is , nonsingular.
If we let Cx • z; then y • Ax • ctex • Ctz. Defining the function
g{x) we have,
g{x) - (Cx - z)t{ex - z)
• xtctex - xtctz - ztex + ztz
• xtctex - xtctz - xtctz + xtotCx
- 2{xt.lx - xty).
!his function being a sum of squares is ne'Ver negative and has a min
iJmJm value of zero for J!: • A-ly • c-lz. The function g{x) is a func-. .
tion of m variables (I' fh, ••• ,'1m- By taking the partial deriva
tives of g(x) with respect to these variables we find the coordinates
of the gradient vector. Then
g{x) - ~ ;11 { ~ aij ~"~l - 2~"liY + ztz , 1</ ,., )./ )'/ t"
and since y and z are Imown constants
10
Take H • A then I p • Uptrp _ lIu~up.
When the vector fpup is subtracted from Xp the new residual rp
may be thought of as a leg of a right triangle of which. rp _ 1 is the
hypotenuse (provided up am r p _ 1 were not orthogonal).
Any rule for selecting the vectors Up (or equivalently the vp )
at each step defines a particular iterative process; of these the
following three a!e in cOJIDD.on use.
2. Method of steeEest desce~~. This method prescribes that we
take up • r p _ 1.. Since A in the equation y - Ax • 0 will be taken to
be positi" definite, this implies that the matrix A can be expressed
as the product of a matrix e multiplied by its transpose Ct , A • etc,
where the matrix C is. nonsingular.
If we let ex • z; then y • Ax • ctex • Ctz. Defining the function
g(x) we have,
g(x) • (ex'!" z) t(cx - z)
• xtctcx - xtctz - ztcx + ztz
!his function being a sum of squares is never negative and has a ndn
imum value of zero for " • A-ly • c-lz. The function g(x) is a func-
tion of m v: riables ¢lJ '12' ••• , '1m- By taking the part.ial deriva
tives of g(x) with respect to these variables we find the coordinates
of the gradient vector. Then
g(x) ·~,'li~.~I aij ~~jl - 2~ ;liY + ztz ,
and s~.nce y and z are known constants
11
Now 111 • IjlJ therefore .... '" ~(x) • 2~~aij;E 'Ijl -y • 2(Ax - y).
" "" J ~ ' )_1
The function g(xl evaluated at the point x • ~ _ 1 is undergoing its
most rapid variation in the direction .rp _ 1 since,
gx(Xp - 1) • 2(Axp _ 1 - y) • 2(-rp _ 1) • -2rp - 1-
If we think of our problem as one of mLni.mizing. g(x} by suec.essin
steps, it is natural to take each step in the direction of most
rapid decrease_ Hence we have justified the selection of rp _ 1
as the vector ~ for most rapid minimLsation of the residuals.2
The method of steepest descent then proceeds as follOWl!l:
;y - AX() • rO
Xl • X() + tlrO where 1/1 • rOtrO/rotAro
y-AJ!:l·~
• • •
7'1' • Xp _ 1 + ¢prp _ 1 where rip • rpt_ lrp _ l/rpt_ lArp _ 1
;y - Axp • rp
• • •
Since this is a BeU correcting method it is desireable to S8C-
rifice temporar;y accurac;y for ultimate speed; an error of calculation
made in any step continually corrects itself in subsequent iterations.
n
mll'l m '" ",hl
~(x) .~ F' 8ij ~~jl + ~hi~, ;/81j - 2y.
Now '!U • ~jH therefore,." ... t>'l
fL. (x) • 2~~ 8ij ~ ~j1 -y • 2(Ax - y)."'X .~,J'/ ;,..
The funct.ion g(x) evaluated at the point x • ~ _ 1 is undergoing ita. .
DlOl!t rapid variation in the direct10nrp _ 1 einee,
gx(Xp - 1) • 2(Axp - 1 - y) • 2(-rp - 1) • -2rp - 1-
If we think of our problem as one of minimizing. g(x) by suec.essive
steps, it is natural to take each step in the direction of most
rapid decrease_ Hence we have justified the selection of r p ... 1
l!l the vector Up for most rapid minimization of the resid als.2
The method of steepest descent then proceeds 8S follows:
y - AX() • rO
Xl • XC + IIrO where t1 • rOtrO/rOtAro
•••
Top • xp _ 1 + ¢'prp _ 1 where 'p • r pt_ lrp _ Jlrpt_ lArp _ 1
y - AXp • r p
•••
Since this is a selt oorrecting method it is deBireable to S8C-
r1.flce t mporary accuracy for ultimate speed; an elTor of calculation
made in any step continually COIT9cts itself' in subsequent iterations.
12
3. Gauss-Seidel method. This method is the classical iterative
method for solving linear equations that feature symetry' and rela-
tively large diagonal terms, however it may also be used for certain
other systema of equations.
Usi.ng this method we let ~ • 1l.nn + i - et where v - 1, 2, • • • •
Then Xp differs f'r0l!l Xp _ 1 only in the i th element. In selecting up
we do so in such a manner that the ith element of rp (the ith residual)
:Us zero. This means that the ith equation is satisfied exactly. This
process 'frill require more steps than either of the other two methods;
however the sinpl1city of choice and the ease of computation give it a
great advantage in automatic machinery work.
In using this method for hand computation it is usually best to
set up a -At table and a residual table.3 The residuals are then suc
cessibe1y reduced to zero in the following manner: divide the residual
ri to be eliminated by the corresponding main diagonal element of A,
aU, and use the quotient ipei (,fp - ei trp _ 1/0t tAot) as the change
in Xi. Change each of the residuals ~, r2, ••• , rm by ,fpBiaji
where j ·1, 2, ••• , m; that is, add ,fpei(-aji) to each of the
reSiduals. Each step of the calculation thereafter involves a s i ngle
application of an operation of the "basic kind". The residual table
merely records in tabular form the operation used, the _extent to which
it is used and the consequent effects on the residuals. One operation
of this type corresponds to each unknown in the problem and is called
3Al1en, Deryck N., Relaxation IIethods (New- York: IlcGraw-H111 Book Company, Inc., 1954), pp. 3-1.
12
3. Gauss-Seidel method. This method is the classical iterative
method ror ~olving linear equations that feature symetry and re18-
tively large diagonal terms, however it may also be used for certain
other systems of equations.
Usi,ng this method we let up • Uvm + i • e1 where v-I, 2, ••••
Then xp differs from Xp _ 1 only in the i tb element. In selecting up
we do so in such a JMnner that the ith element of rp (the ith residual)
is zero. This means that the i th equation is satie.fied exactly. Thie
process will require more steps than either of the other two methods;
however the sinplicity of choice and the ease of computation give ita
great advantage in automatic machinery work.
In using this method for hand computation it is usually best to
set up a -At table and a residual table.3 The residuals are then sue-
cessibely reduced to zero in the follOlrlng manner:: divide the residual
r i to be eliminated by the corresponding main diagonal element of A,
aii, and use the quotient ~pei (;p - ei trp _ l/~ tAei ) as the change
in Xi_ Change each of the residuals TI, r2, ••• , I'm by 'peiaji
where j • 1, 2, ••• , m; that is, add Ipei(-aji) to each of the
residuals. Each step of the calculation thereafter involves a single
application of an operation of the "basic kind". The residual table
merely records in tabular form the operation used, the extent to which
it is used and the consequent effects on the residuals. One operation
of this type corresponds to each unknown in the problem and is called
3Allen, Deryck N., Relaxation Methods (New Yorkr McGraw-Hill BookCompany, Inc., 1954), pp. 3-1.
13
a basic unit operation.
The residuals, one of which stan6s identically for the non-zero
side of each equation in the system are such that for any values
assigned to the original unknowns xl' ~, ••• , xm corresponding
values can be calculated for rl' r2' • • • , rm directly from the
defining relations. The method would then be self-cheeking. To
save time, however, use is made of the _At tabla and each of the resid-
uals ri is ~hanged by 'pei(-aji) as previously indicated. A cheek on
the residuals at periodic intervals and at the end of the computation
is therefore necessary.
Under certain circumstanees changing the coefficient of xt in
the ith equation to one is a useful preliminary step. The teehniques
for handling the computation in the Gauss-8eidel method also apply to
the handling of the eomputation in the following method.
4. Method of relaxation. This method is an adaptation of the
Gauss-5e1del method, and therefore always takes Up to be some 81. but
the selection is made as the computation progresses. Since the choiee
of up • Sf eliminates the ith element of rp one can ehoose to eliminate
the largest residual (or make this residual assume any desireable value
in view of future iterations). Selecting the largest residual is not
nece4sarily the best choice. The magnitude of the correction is
measured by the correction vector ¢pup and this magni tude is
Uptrp _ If Up tA Up}-l/2
Now when u • e this becomes Sf trp _ lfaii)-1/2. Hence one
should examine the. quotients of the residual components divided by
13
a basic unit operatt"on.
The residuals, one of which stan6s identically for the non-zero
sid of each equation in the system are such t~at for any values
assigned to the original unknowns xl' ~, ••• , ~ corresponding
values can be calculated for rl' r2' • • • , r m directly from the
defining relations. The method would then be self-ehecking. To
save time, however, use is made of the _At table and each of the resid-
uals ri is ~hanged by sipei (-aji) as previously indicated. A cheek on
the residuals at periodic intervals and at the end of the computation
is therefore necessary.
Under certain circumstances changing the coe.fflcient of Xi in
the i ttl equation to one is a useful prellmir'lary step. The techniques
for handling the computation in the Gau8s-8eidel method also apply to
the handling of the computati.on in the following method.
4. Method of relaxation. This method is an adapta t,c:.on of the
Gausa-Seidel method, and therefore alwa.Y.S takes Up ~o be some ei, but
the selection is made as the computation progresses. Since the choice
of up ~ eliminates the ith element of rp one can choose to eli~n8te
the largest residual (or ma~ this residual assume any desireable value
in view of future iterations). Selecting the largest residual is not
necessarily the best choice. The magnitude of the correction is
measured by the correction vector ~pup and this magnitude is
Upt rp _ l.fuptAUp)-1/2
Now when u • e this becomes ~ trp _ Ifaii)-1/2. Hence one
should examine the quotients of the residual components divided by
the corresponding -iiii and eliminate the largest quotient (in absolute
value).
Various refinements have been made on the method of relaxation
to make it more practical for hand computation. These refinements
include: overrelaxation, reduction of the residuals beyond the zero
value; underrelaxation; block relaxation, changing several variables
. by the same amount; and group relaxation, changing several variables
by ,varying amounts. Group relaxation is sometimes used to produce a
modification of the _At table that has a main diagonal of relatively
large elements. This table is const~Jcted with the idea that each
change in the unknowns li~ted will produce a relatively large change
in only one of the residuals and a different residual for each entry.
Relaxation with this table should show much quicker convergence to a
solution than with the -At table. • , .
5. Summary. USing the method of relaxation causes the xt's to
converge to a solution more rapidly than the method of Seidel. This
feature together with the advantages incurred with the use of under
and overrelaxation make this method the best of the three discussed
for systems with a "reasonable" number of unknowns. The ease of com
putation distinguishes the Gauss-8eidel method. The method of steep-
est descent involves the most difficult computations of the three,
but relatively rapid convergence to a solution for systems in which
the number of unknowns is "large" is its reward. The method to use
in solving any particular system is then chosen with the special char-
acteristics of the system, the requirements of the solution and the
capabilities of the computor in mind.
the corresponding"l'i'ii and eliminate the largest quotient (in absolute
value).
Various refinements have been made on the method of relaxation
to make it more practical for hand computation. These refinements
include: overrelaxation, reduction of the residuals beyond the zero
value; underrelaxation; block relaxation, changing several variables
. by the same amount; and group relaxation, cha~ng several variables
by .varying amounts. Group relaxation is somet.imes used to produce a
modification of the _At table that has a main diagonal of relatively
large elements. This table is constl"Jct.ed with the idea that each
change in the unknowns listed will produce a relatively large change
in only one of the residuals and a different residual for each entry.
Relaxation with this table should show much quicker convergence to a
solution than with the -At table.. -
5. Summary. 'O'sing the method or relaxation causes the ~fS to
converge to a solution more rapidly than the method of Seidel. This
feature together "Wi th the advantages incurred with the use of under
and overrelaxation make this method the best of the three discussed
for systems nth a "reasonable" number of unlmowns. The ease of com-
putation distinguishes the Gaus8-Seid~1 method. The method of steep-
est descent irrvolves the most difficult computations of the three,
but relatively rapid convergence to a solution for 8ystems in which
the number of unknowns is ltlarge" is its reward. The method to use
iJ'l solving any particular system is then chosen with the special char-
acteristics of the system, the requirements of the eolution and the
capab11ities of the computor in mind.
15
In using an iterative method a computor ceases to look upon the
problem as that of finding those values of xl' X2. • • • , ~ which
satis~ the equations, but regards it instead as the determination of
a set of values which make the residuals zero.
Since each iteration is a check on the progress of the method,
it is usually possible to see when things are getting "out of hand".
After one or two iterations the changes in the computed values should
be gradual. Erratic changes will mean that the method is either not
applicable to this type of problem or that an error of some kind has
been made in obtaining the erratic value and therefore the value
whould be checked before proceeding further.
To illustrate the essential characteristics of each of the tp.ree
methods discussed the same problem will be worked by each method. The
last two methods described were originally self checking. but the adap
tations made here were not, therefore a check was taken at periodic
intervals, that is, substitution of ~ was made in each of the original
equations and the values of the residuals so computed were taken as the
starting point of the next series of iterations.
Each method was begun with Xo • " for purposes of comparison. The
solution of this system (by direct methods) is ~1, 1, 2. The approximate
solutions achieved are such that they sre stable to three decimal places
(in the methods of Gauss-5eide1 and relaxation). This can be verified
by examination of the previous values for the unknowns and the observa
tion that the total of the residuals whether added or subtracted from
the values os the unknowns would not influence the third deCimal place
15
In using an iterative method a computor ceases to look upon the
problem as that of finding those valup.s of xl' x2' • • • , Xm hleh
satisfy the equations, but regards it instead as the determination of
a set of values which make the residuals zero.
Since each iteration is a check on the progress of the method,
it is usually po!!sible to see when things are getting "out of hand".
After one or two iterations the changes in the computed values should
be gradual. Erratic changes w:i.l1 mean that the method is either not
applicable to this type of problem or that an error of some kind has
been made in obtaining the erratic value and therefore the value
whou1d be checked before proceeding further.
To i1lustra te the essential characteristics of each of the three
methods discussed the same problem will be worked by each method. The
last two methods described were originally self checking, but the adap
tations made here were not, therefore a check was taken at periodic
intervals, that is, substitution of xp was made in each of the original
equations and the values of the residuals ao computed were taken as the
starting point of the next series of iterations.
Each method was begun With Xo • 0 for purposes of comparison. The
solution of this system (by direct methods) is ...1, 1, 2. The approximate
solutions achieved are such that they re stable to three decimal places
(in the m.ethods of Gau8s-8eidel and relaxa ti on). This can be verified
by examination of the previous values for the unknowns and the observa
tion that the total of the residuals whether added or subtracted from
the values os the unknowns would not influence the third decimal place
16
of the unknowns.
Only seventeen steps were needed for this result ld. th the method
of relaxation while twenty steps were needed for the Gauss-Seidel
method. By under and ~rrelaxation the solution ll'I4y be arrived at much
faster; see the last exaft\)le. The method of steepest descent which CO\1-
verged ,so slowly in this example is really the fastest converging when
when n is "large".
In certain iterations the reSidual had to be changed by amounts
too large or too small. An autoll'l4tic II!Ichine would probably be designed
to change the residuals by too large an amount. A better proceedure
would be to consistently round the change in the variable to the nearest
even or odd figure. This occurred in the fourteenth iteration of the
Gauss-5eidel method. At times the values of the unknowns or the residuals
may even worsen rather than iuprove; see ' the sixteenth iteration of the
Gauss-5eidel method.
6. Exauples. Let the equations be:
fl(~,~~X3)-4+5;+ ~ -0
f 2(xl' "2, X3) -1 - Xl - Sx2 + 4x3 - 0
f 3(xl' X:!, x3 ) -13 - 2xl + 3X:! - 7x3 - 0
1-5 -1 01
A- lS-4 -2 -3 7 ~~ X - 2
• 3 (-1, 1, 2)
Since a positive definite matrix is needed for most applications of the
method of steepest descent we will premultiply both sides of the pre
ceeding equation by At.
16
of the unknowns.
Only seventeen steps were needed for this result lri.th the method
of relaxation while twenty steps were needed tor the Gauss-Seidel
method. By under and O"¥errelaxation the solution !!'By be arrived 8t much
f'a8ter; see the last example. The method of s·~pest descent which con
vetrged so slowly in this example 1s really the fastest donverging hen
when n is "large".
In certain iterationa the reSidual had to be changed by amounts
too large or too small. An automatic machine would probably be designed
to change the residuals by too large n amount. A better proceedure
would be to consistently round the ohange in the variable to the nearest
even or odd figure. This occurred in the fourteenth iteration of the
Gauss-Seidel lMthod. At times the values of the unknmms or the residuals
may even worsen rather than improve;. see'the sixteenth iteration of the
Gau8s-5eidel method.. .
6. Examples. Let the equations be:
£1(~, ~~ x3) • 4 + 5~ + ~ - 0
t'2(xl' x2' x3) -1 - Xl - ~2 + 4x3 • 0
f'3{xl , ~, x3) -13 - 2x1 + 3~ - 7x3 • 0
y - Ax • 0 y ·l-~13
-5 -1 0\A-I g-4
-2 -3 7 ~~x.. X23 (-1, 1, 2)
Since a po itive definite matrix is needed for most applications of the
method of steepest descent we will premultiply both sides of the pr
ceeding equation by At.
17
Lettillg II • At,- and B • At! we have an equation in lIhich the coeffi-
cient matrix B is positive defin1te:
z - Bx - o.
The solution of this equation is also the solution of the equation
y - Ax • o.
l~o 19 - -1~
B· 1974--5~ 1~ -5~ 65
The transformed equations being:
gl (;,.. ~. x~) • -47 -~Ox1 -1~ *.. 19X~ • 0
g2(x1' ~. x3) ~ -51 -19x1 -7~ + 5~x~ • 0
g~(xl' ~. ~) • 95 +1~1 +5~~ - 65x3 • o.
11
Letting z • Aty and B • AtA we have an equation in 'Which the eoeffi-
cient matrix B is positive definite:
z - Bx • o.
The solution of this equation is also the solution of the equation
y - Ax • o.
The transfonned equations being:
g1(;, x2 , x3) • -47 -3Ox1 -1~ ~1.lgX3 • 0
g2(xl~ x2' x3) • -51 -19x1 -7~ +53X3• 0
g3(xp ~, x3)· 95 "·1~1 +53~ - 65x3 • o.
II!!
The method of steepest descent
~ xl ~ x3 rp r1 r2 r3
XC 0 o· 0 ro -47 -51 95 stiro - .40369 - .43r04 .81596
xl - ;40369 - ;43804 .~1596 ~ -11.87926 32.33095 11.L!.!!lOO6 --2~ - .30079 1.03636 .36799 ~ - .7gL!M • 59S32 1.1S395 r2 -13.52258 -17.62121 35.63357
__ 3r2 - .11690 - ~15233 .30004 x3 - .90138 ;/j4599 1.49199 r3 -1.57659 12.19S43 5J:.332S
--4r3 - .05135 ~39730 .17696 x4 - .95273 ~84329 1.66g95 r4 4.39951 -6.S4724 14.063M
'5r4 ~OM93 - .07615 .15640 x5 - .90380 .76714 1.~535 r5 -1.60536 6.1L!739 .'74227
--(jr5 - ;03138 .12018 .01L!51 X6 - .9351g .88732 1.g3~6 r6 -2.M620 -1. 380M 5.60382
tl7r6 - .02790 - .01434 .05820 x7 ":' .96308 .!'!72~ 1.!'!9OO6 r7 - .52914 3.2951g .55g6()
tl8r7 - ;01042 ~06489 .01100 Xg - .97350 .93787 1.90906 r8 - .13659 1.79003 .3~39
--~g - ' .00270 .03540 .00721 ~ - .~923 .96571 1;',14851 r9 - .59S41 - .39614 1!72334
tl10r9 - .00625 - .00413 .01799 xIO - .99548 .96158 1.96650 1'].0 - .00!!l62 .9S170 .22260
1111'].0 - .00002 .00195 .00044 x11 - .99550 .96353 1.96694 rn - .03715 .86332 .29699
H~
The method of steepest descent
Xp xl x2 x3 rp rl r2 r3
Xo 0 0- 0 ro -41 -51 95tl1rO - .40369 - .43804 .81596
xl - .40,69 - ~43804 .~1596 I1. -11-.81926 32:.33095 11.40006t21"J. - .3eo79 1.03636 .36799
x2 - .n448 ~59832 1.~395 r2 -13.52258 -11.62121 35.63351rJ3r2 - .11690 - ~15233 .30804
x3 - .90138 ~L4599 1.49199 r3 ~1.57659 12.19e'43 5.43328rl4r3 - .05135 ~39730 .17696
x4 - .95273 ~g4329 1.6M95 r4 4.39951 -6.e'472l+ 14.06348'5r4 ~01.J~93 - .07615 .15640
x5 - .90380 .76714 1.~2535 r5 -1.60536 6.14739 .74227lye, - ~03138 ~12018 .01451
X6 - .93518 .88732 1.~3~6 r6 -2.68620 -1.3806S 5.60382rJ7r6 - .02790 - .01434 .05820
x7 ~ ~96308 .87298 1.89806 r7 - .,291.4 3.29518 .55860tlgr7 - .01042 ~064g9 .01100
Xg - .97350 .93787 1.90906 rg - .13659 1.79003 .3l439;C)I'8 -' .00270 .03540 .00721
x9 - .98923 .96571 1~94g51 r9 - .59841 - .39614 1.723-34rlI0r9 - .00625 - .00413 ~01799
x10 - .99548 •9615g 1.96650 rto - .00062 .98170 .22260111liO - .00002 .00195 .00044
XII - .99550 .96353 1.96694 iiI - .03715 .86332 .29699
19
The Gauss-8eidel method
lrt X:2 x~ ¢'p6,t r ~ r2 r~
0 0 0 ~ • x2 , · . x~ ·0 4 -1 1~ - .8 281 • -.8 0 - .2 1104
- .025 ~82 • - ~025 - ~025 0 11.~25 1.61~ ;t~· 1~617S6 - .025 6.1!71h4 - .00002
- .795 ~e1· ~005 o· 6;M6114 .00998 .78~~ . ~82 ~ ;gOg~ ~g08~ .~ 2.43488
1.96570 ~~ ~ ~~4784 .OO8~ 1.~914 0 - .95666 ~1 • - .16166 0 1.55~06 - .~2"2
.m4~ ~e2· .1941~ ~1941~ .00002 .25907 2.00271 9~ • . ' .0~701 .1941~ .14006 0
- .99549 r,0s:L • - ~O~gg~ -~00002 .18689 - .07766 1.00079 ~le2· ~02~~6 ;02~34 .00001 - .00758
2.oo16~ ~e~ • - .00108 ~02~34 - .004~1 - .00002 -1.00016 ~~e1 a - ;00467 - ;00001 .000~6 - .009~6
1.000/34 . ~e2· .00005 .00004 - .00004 - .00921
-1.00017' 2.ooo~1 ~5e, • ~ .001~2 .00004 - .005~2 ;OOOO~
~6el • - .00001 - .00001 - .005~1 .00001 1.00018 ' ~~ • - .00066 - .00067 - .OOOO~ - .00197
2.oooo~ 18e~ . - .00028 - .00067 - .00115 - .00001
-1.00004 ~981 " .0001~ - .00002 - .0012S ;00025 1.00002 2082 ~ - .00016 - .0001S 0 - .0002~
19
The Gauss~e1de1 method
~ ~ x3 ;p~ r r1 r2 r;
0 0 0 ~ • x2 • x3 • 0 4 -1 13- .g ~e1 • _. ~8 0 - .2 11.4
- .025 ~~ • - ~025 - ~025 0 11.3251.61786 ;r.e;. 1~617g6 - .025 6.~7144 - .00002
- .795 ~e1· ~005 O· 6~66WJ .OO99~.7g;; ~~ ~ ~80e; ~g08; .00004 2.4348S
1.96570 ~e;. ~34784 .80~3 1.3914 0- .95666 1J~1 • - ~16166 0 1.55306 - ~;23;2
.97743 ~e2· .1<;41; ~1941; .00002 .259072.002n 9~ • ..•03701 .1941; .14806 0
- .99549 ~0t!J. • - .038S3 - ~00002 .18689 - .077661.00079 ii1e2 • .02336 ~02334 .00001 - .00758
2.00163 ~e3 - .0010S ~02334 - .00431 - .00002-1.00016 ~3el z - ~00467 - ~OOOOI .000;6 - .00936
1.00084 - ~e2· .00005 .00004 - .00004 - .00921
-1.00017'2.000;1 ~5e; c ~ .00132 .00004 - .005;2 ~00003
~6el • - .00001 - .00001 - .00531 .000011.00018 ~-re2 - - .00066 - .00067 - .0000; - .00197
2.00003 IS 3 - ~OO02g - .00067 - .00115 - .00001
-1.00004 ~~ ~ .00013 - .00002 - .00128 ~OO0251.00002 20~ .. - .00016 - .00018 0 - .00023
2D
The method of relaxation
Xl ~ x~ ;pei r1 r2 ~
0 0 0 x .~·x -0 4 -1 1~ 1.S5714 ~e~. 1~e5714 4 6.42e56 .00002
.00~57 ~2- ~00~57 4.oo~57 o· 2.410n - .96071 ~e1 • - .96071 ~00002 .96071 .48931
.92~66 ~e2- .12009 .12011 - .00001 .S4958 1.97S51 :.ra~. .121~7 ~12011 .4e547 - .00001
.91"434 ~e2- .0606S .10079 ~OOOO~ .1e20~ 2.00451 t . .02600 .10079 .1040~ .00003
- .996S7 ~ei • - .0~616 - .00001 .]4019 - .07229 1.00le6 ge2- .01752 .01751 .oooo~ - .019n
2.00169 ~oe~ • - .002e2 .01751 - .01125 .00001 -1.ooo~7 . ~le1 • - ~00~50 .00001 - .00775 - .00699
1.000g9 ~e2 • - .00097 - .00096 .00001 -.00990 2.ooo2g ?,~e3 • - .00141 - .00096 - .0056~ - .oooo~
1.00019 - ~e2 • - .00070 - .00166 - .OOOO~ - .0021~ 1.~ ?,5e3 • - .OOO~O - .00166 - .OO12~ - .oooo~
-1.00004 ?,6e1. .OOO~~ - ~OOOOl - .00156 .0006~ .99999 11'2 • - .00020 - .00021 .00004 .OOOO~
The method of re~tion with under and over-relaxation
Xl ~ x~ ;pei r1 r2 r~
0 0 0 (i.~.x~.o 4 -1 1~ 2 ~e~. 2 4 7 -1
1 ~. 1 5 -1 2 -1 ~~ ·-1 0 0 0
2D
The mthod of relaxation
xl ~ x:; ;pei r'1 r2 r"3
0 0 0 x .~·x -0 4 -1 131.85714 ~e:;. l~g~714 4" 6.42856 .00002
.S0357 ~2" ~go357 4.00357 0" 2.41073- .960n ·3el .. - .96071 ~OOOO2 .96071 ~931
.92366 ~e2" .12009 .12011 - .00001 .84.95g1.9~1 ~3" .12137 ~12011 .48547 - oo1סס.
.98434 ~e2· .06068 ~18079 ~OOOO3 .1820:;2.00451 of3· .02600 .18079 .1040:; .00003
- .~7 ~el III - ~03616 - .00001 .14019 - .072291.0011l6 ge2 - .01752 .01151 .00003 - .01973
2.00169 ~oe3 • ~ .00282 .01751 - .01125 .00001-1.00037 . ~lel .. - ~O0350 oo1סס. - .00775 - .00699
1.00089 ~e2 - - .00097 -.00096 .00001 - .009902.0002g ¢J,'3e3 • - .00l41 - .00096 - .00563 - ~OOOO3
1.00019· .. " ~e2 .. - .00070 - .00166 - ~OOOO; - .002131.99998 ~5e; .. - .00030 - ~00166 - .00123 - .00003
-1.00004 ~6e1" ~OOO33 - .00001 - .00156 .00063.99999 1,.,e2 • - .00020 - .00021 .00004 .00003
The method of relaxation with under and over-re1aocartions
Xl x2 %3 tip s i 1'1 r2 1':;
0 0 0 Ii .. ~ .. %3 .. 0 4 -1 132 ~e3 .. 2 4 7 -1
1 ;;~ .. 1 5 -1 2-1 3€I • -1 0 0 0
BIBLlOORAPHY
~llen, Deryck N~ Relaxation Methods. New York: McGraw Hill Book Company, Inc., 1954.
Buckingham, Richard A~ Numerical Methods. London: Sir Isaac l'i tman and Sons, Ltd., 1957.
Dwyer, .Paul S. Linear Computations. New York: John Wiley and Sons, Inc., 1951.
Hote1ling, Harold. Anal sis of 8 co lex of statistical variables into frinCiP81 components, Journa of Educationa Psychology, Vol.
OCtober, 1933) pp. 502-4.
Rotelling, Hsrold. Some new methods in matrix calculation, Annals of Mathematical Statistics, Vol. 14 (1943) pp. 1-34.
Householder, Alston S. Pr; ncipals of Numerical Analysis. New York: McGraw-Hill Book Company, Inc., 1953. .
PerUs, Sam. The0rr, of ' Matrices. Cambridge, Massachusetts: Addison-Wesley Press, Inc., 952.
Scarborough, James B. Numerical Mathematical Analysis. Baltimore, Ohio: Johns Hopkins Press, 1955.
Wittaker, Edmund T., and RObinson, George. The Calculus of Observations. London: Blackie and Son, Lim! ted, 1926.
21
\
\
BIBLlOORAPHY
Allen, Deryck N~ Relaxation Methods. New York: McGraw Hill Book Company,Inc., 1954.
Buckingham, Richard A.~ Numerical Methods. London: Sir Isaac Pitman andons, Ltd., 1957.
Dwyer, Paul S. Linear Computations, New York: John Wiley and SOM, Inc.,1951.
Rotelling, Harold. Analysis of a complex of statistical variables intotrinCipa1 components, Journal of Educational Psychology, Vol. 24
OCtober, 1933) pp. 502-4.
Rotelling, Harold. Some new methods in matrix calculation, Annals ofMathematical Stati~tics, Vol. 14 (l943) pp. 1-34.
Householder, Alston S. Principals of Numerical Analysis. New York: McGralf-. Hi11 Book Company, Inc., 1953. . "PerIis, Sam. TheorY of "Matrices. Cambridge, Massachusetts: Addison-Wesley
Press, Inc.,~952.
Scarborough, James B. Numerical Mathematical Analysis. Baltimore, Ohio:Johns Hopkins Press, 1955.
Wittaker, Edmund T., and Robinson, George. The calculus of Observations.Londom Blackia and Son, Limited, 1926.
21