CE 601 Numerical methods
Transcript of CE 601 Numerical methods
CE 601: Numerical MethodsLecture 5
Course Coordinator:
Dr. Suresh A. Kartha,
Associate Professor,
Department of Civil Engineering,
IIT Guwahati.
Elimination Methods
• For a system [A]{x} = {b} where [A] is an n X n matrix, the number of operations = (2/3)n3+(3/2)n2-(7/6)n.
• Note: The formula (2/3)n3+(3/2)n2-(7/6)n will always yield integer values.
• Like Gauss elimination, you might have studied Gauss-Jordan elimination method. The objective of Gauss-Jordan scheme was to convert [A]{x} = {b} => [I]{x} = {x}, [I] -> Identity matrix, through systematic elimination process.
• We are not going to discuss on this method in the class and request you to work on your our referring literature.
• The Gauss-Jordan scheme is computationally less efficient than the Gauss elimination scheme. The no. of operations involved in Gauss-Jordan scheme = n3+n2-n.
• Like Gauss elimination, Gauss-Jordan, etc. you might have studied matrix inverse methods to solve linear systems
i.e. [A]{x} = {b}
so, {x} = [A]-1{b}
There are two evaluations in this scheme
→ First evaluate the inverse of matrix [A]
→Second, to evaluate the product of [A]-1{b}
• In both the evaluations, there are arithmetic operations involved. For matrix inverse it takes 2n2-n operations i.e. total of 2n3 operations.
• Based on the significant digits assigned to the variables or components, round-off errors may creep in the solutions while using Gauss elimination scheme.
→ These errors can be minimized by performing partial pivoting or scaled partial pivoting.
• LU Decompositiono We have discussed that matrix can be factored i.e., it can
be given as product of two different matrix. [A] = [B][C]
o There can be many possibilities of obtaining factor matrices.→ However if we specify the diagonal elements of either
[L] or [U], then we will have a unique a factorization for [A].
→The LU decomposition methods like Doolittle and Croutwork on these principles.
o In a similar tone, one can also factorize [A] as product of [L] and [U] i.e., [A]= [L][U] where [L] is lower triangular and [U] is a upper triangular matrix.
11 12 1 11 11 12 1
21 22 2 21 22 22 2
1 2 1 2
0 0
0 0
0 0
n n
n n
n n nn n n nn nn
a a a l u u u
a a a l l u u
a a a l l l u
• Property of LU decomposition
o [A]{x} = {b}
[L][U] {x} = {b}
Multiply left and right side by [L]-1
[L]-1[L][U] {x} = [L]-1 {b}
[I][U]{x} = [L]-1 {b}
[U}{x} = [L]-1 {b}
o Let us define the system [U]{x} = {c}
So, we get {c} = [L]-1 {b}
or, [L]{c} = {b}
o The meaning here is:
Lower triangular matrix [L] will transform RHS vector {b} to {c} using the relation [L]-1{c} = {b}.
On obtaining {c} , using the relation [U]{x} = {c}, we obtain the solution vector.
So if we can suitable algorithm such that process involved in LU decomposition are”
Factorize [A] = [L][U]
Forward substitution to evaluate {c}
Backward substitution to evaluate {x]
If the no. of arithmetic operations in factorization can be reduced then LU decomposition becomes efficient.
• The Advantage of LU Decomposition
o If there are many linear systems with same coefficient matrix i.e.,
[A]{x} = {b}
[A]{y} = {c}
[A]{z} = {d}
o Then if we adopt Gauss elimination, for each system it may take (2/3)n3+(3/2)n2-(7/6)n operations. The purpose may become tedious.
o If we do LU decomposition, then you may see that only once we need to decompose [A]=[L][U]
o After that for each system, we need to utilize only the forward and backward substitutions.
• Doolittle LU Decomposition
• [A] = [L][U]
• If we factorize in such a way that
i.e. diagonal elements of [L] are 1, then the approach is Doolittle’s method.
• If
the approach is called Crout’s method.
11 12 1
21 22 2
1 2
0 0 1
0 0 1[ ]
0 0 1
n
n
n n nn
l u u
l l uA
l l l
11 12 1 11 12 1
21 22 2 21 22 2
1 2 1 2
1 0 0
1 0 0
1 0 0
n n
n n
n n nn n n nn
a a a u u u
a a a l u u
a a a l l u
Algorithm for Doolittle’s LU Decomposition
• Doolittle algorithm is developed using knowledge of Gauss elimination.
• Recall Gauss elimination at any step k
• You can now write
( ) ( 1) ( 1)
( 1) ( 1)1,2,3,..., 1; , 1,..., ; 1, 2,...,
k k k
ij ij ik kj
k k
ik ik kk
a a l ak n j k k n i k k n
l a a
( ) ( 1) ( 1)
( ) ( 2) ( 1) ( 2)
( 1) ( 1)
( ) ( 1)
1
( ) ( 1)
1
i.e.,
Extending,
or, ; 1, 2,..., ; , 1,...,
k k k
ij ij ik kj
k k k k
ij ij ik kj i k k j
kk m
ij ij im mj
m
kk m
ij ij im mj
m
a a l a
a a l a l a
a a l a
a a l a i k k n j k k n
• This is nothing but
• So you get
( ) ( 1)
1
[ ] [ ][ ]
where elements of lower triangular matrix
; for ; 1,2,3,..., 1
such that 1 ; for
0 ; for
kk m
ij ij im mj
m
ij
ik
ij
A L U
a a l a
l
l i k k n
l i j
i j
11 12 13 1
(1) (1) (1)
21 22 23 2
(2) (2)
31 32 33 3
( 1)
1 2 3
(1) (1)
21 21 11 32 32 22
1 0 0 0
1 0 0 0
[ ] 1 0 ,[ ] 0 0
1 0 0 0
where , , etc.
n
n
n
n
n n n nn
a a a a
l a a a
L l l U a a
l l l a
l a a l a a
• So the steps involved are:
o No. of steps as in Gauss elimination
k = 1,2,3,…,n-1
o At any k , i= k+1,k+2,…,n and j=k,k+1,…,n
( ) ( 1) ( 1)
( 1) ( 1)
1
0;
0
;
kk
ik
k k k
ij ij ij ik kj
ij
k k
ik ik kk
l
l i k
u a a l a
u
l a a i k
o Forward substitution for c
o Back substitution for x
1 1
1
1
; 2,3, ,k
i i im m
m
c b
c b l c i n
1 ( 1)
1
( 1)( 1)
1In genral, ; ( 1),( 2),...,2,1
n n nn
n n n n
n
n n
n
i ij j
j i
i
ii
x c u
c u xx
u
c u x
x i n nu
• Example1
2
3
11 12 13
21 22 23
31 32 33
21 31 32
11 12 13
21 22
80 20 20 20
20 40 20 20
20 20 130 20
1 0 0
1 0 , 0
1 0 0
1 / 4, 1 / 4, 5 / 7
80, 20, 20
0, 40 ( 1 / 4)
x
x
x
A L U
u u u
L l U u u
l l u
l l l
u u u
u u
23
31 32
2( 1)
33 3
1
( 20) 35
20 ( 1 / 4)( 20) 25
0
130
750 / 7
km
m mj
m
u
u u
u l a
Forward Substitution:
Now you can do back substitution easily.
1 1
2
3
1
2
3
20
20 ( 20 / 4) 25
20 ( 20 / 4) ( 5 25 / 7) 300 / 7
[ ]{ } { }
80 20 20 20
i.e., 0 35 25 25
0 0 750 / 7 300 / 7
c b
c
c
U x c
x
x
x
• Number of operations
• In Doolittle’s algorithm,
o to evaluate lik for any kth step for i> k, it takes (n-k) operations.
o To evaluate uij = aij(k) it takes 2(n-k)2
operations.
o In (n-k) elimination steps to form [L] and [u], it takes, 1
1
23
( )(2 2 1)
2( 1)(4 1)
6 3 2 6
n
p
n p n p
n n nn n n
o To perform forward substitution,
as c1 = b1 (No operation required)
There are 2i operations for each i.
So, no. of operations =
o To perform backward substitution, it requires n2 operations (as in Gauss)
o Total no. of operations =
1
1
; 2,3,4, ,i
i i im m
m
c b l c i n
2
2
2n
i
i n n
23 2
3 2
22
3 2 6
2 3 7
3 2 6
n nn n n
n n n