07. Matrix methods
-
Upload
swagata-saha -
Category
Documents
-
view
224 -
download
0
Transcript of 07. Matrix methods
-
8/7/2019 07. Matrix methods
1/54
Systems of Linear Equations
-
8/7/2019 07. Matrix methods
2/54
Systems of Linear Equations
Vectors
Matrices
Special matrices Vector vector scalar product
Matrix multiplied by vector
Linear system of equations
Matrix multiplication
-
8/7/2019 07. Matrix methods
3/54
Systems of Linear Equations
Is there a solution?
If there is a solution, is there only one?
How can we express the solution with
the inverse of the coefficient matrix?
Is it usually a good idea to get thesolution via the inverse?
-
8/7/2019 07. Matrix methods
4/54
The determinant is a number that can be
calculated knowing the elements of the
(cofficient) matrix. The matrix inverse doesnt exist if the
determinant is zero.
A condition for the inverse to exist is that the
determinant is not zero.
Determinant zero; coefficient matrix is singular;
solution does not exist; system is contradictory
OR undetermined; matrix rank is less than
number of equations (or just division by zero,
overflow etc.): all mean the same
-
8/7/2019 07. Matrix methods
5/54
Solution of Systems of
Linear Equations
Ax = b
solution
X = A-1
b (A-1
is the inverse matrix)or, Ix = A-1b (I is identity matrix)
-
8/7/2019 07. Matrix methods
6/54
Types of matrix problems
-
8/7/2019 07. Matrix methods
7/54
Example 1 - Unique solution.Consider the system
2x + 3y = 8
5x - 6y = 30
Geometrically this
corresponds to
two linescrossing in one
point.
-6
-4
-2
0
2
4
0 2 4 6 8 10
Equation 1
Equation 2
-
8/7/2019 07. Matrix methods
8/54
Example 2 - No unique solution.
Consider the system
2x + 3y = 8
4x + 6y = 16
Geometrically this
corresponds to two
coincident lines.Infinite number of
solutions.
-6
-4
-2
0
2
4
0 2 4 6 8 10
Equation 1
Equation 2
-
8/7/2019 07. Matrix methods
9/54
Example 3 - No solution.
Consider the system
2x + 3y = 8
4x + 6y = 30
Geometrically this
corresponds to
two parallel linesnever crossing. -6-4
-
0
2
4
6
0 2 4 6 8 10
Equat on 1
Equat on 2
-
8/7/2019 07. Matrix methods
10/54
Example 4 Ill-conditioned matrix
Consider the system
2x + 3y = 8
4x + 7y = 15Geometrically this
corresponds to two
almost parallel
lines barelycrossing. Sensitive
to round-off error.
-6
-4
-2
0
2
4
0 2 4 6 8 10
qua ion 1
qua ion 2
-
8/7/2019 07. Matrix methods
11/54
Existence and Uniqueness
Statement with Rank The rank of a matrix tells us how many
independent rows (or how many independent
columns) the coefficient matrix has -- it may be
less than the total number of equations.
If rank is less than the number of equations,two cases can happen:
Consistent (if rank of coeff matrix = rank of
augmented matrix) Infinite number ofsolutions
Non-Consistent (if rank of coeff matrix < rank
of augmented matrix) No solution
-
8/7/2019 07. Matrix methods
12/54
Definition: Augmented matrix
A
nnnb
b
a
a
.
.
...
....
....
... 111
-
8/7/2019 07. Matrix methods
13/54
Example 1Consider the system2x + 3y = 8
4x + 6y = 16
Thus the determinant is 2*6 - 3*4 = 0.
And indeed the second equation is 2 times the first.
The solution can be written as x = 4 - 3/2 y for any y.
Geometrically this corresponds to two coincident
lines.
In terms of rank: the coefficient matrix has rank 1,
the augmented matrix has rank 1
16
8
64
32
-
8/7/2019 07. Matrix methods
14/54
-
8/7/2019 07. Matrix methods
15/54
Direct Methods Gauss-Jordan elimination
Gaussian elimination
LU decomposition / factorization
Thomas algorithm
Iterative Methods Gauss-Seidel
-
8/7/2019 07. Matrix methods
16/54
Gaussian-JordanStep 1. Forward elimination:
Choose a pivot element on the main diagonal (start attop left corner)
Eliminate (set to zero) all elements above and belowthe main diagonal
Move to the next column
Repeat until matrix is an Identity matrix
-
8/7/2019 07. Matrix methods
17/54
Gauss-Jordan Elimination
Reduce an augmented matrix to the
identity to solve a system of linear
equations.
Example:
-
!
-
-
1
3
4
x
x
x
113
111
112
Solve
3
2
1
-
8/7/2019 07. Matrix methods
18/54
Gauss-Jordan Elimination
-
11133111
4112
:matrixAugmented
-
1113
3111
22/12/11
:2by1rowDivide
-
8/7/2019 07. Matrix methods
19/54
Gauss-Jordan Elimination
-
!
!
52/52/10
12/12/30
22/12/11
:1333
122
rowrowrowand
rowrowrowSet
-
8/7/2019 07. Matrix methods
20/54
Gauss-Jordan Elimination
-
52/52/10
3/23/110
22/12/11
:2/3by2roivide
-
8/7/2019 07. Matrix methods
21/54
Gauss-Jordan Elimination
-
3/163/800
3/23/110
3/73/201
:2ro2/13ro3roet
2ro2/11ro1roet
-
8/7/2019 07. Matrix methods
22/54
Gauss-Jordan Elimination
-
2100
3/23/110
3/73/201
:3/8by3roivide
-
8/7/2019 07. Matrix methods
23/54
Gauss-Jordan Elimination
-
!
!
2100
0010
1001
:33/211
33/122
rowrowrowSet
rowrowrowSet
-
2
0
1
: xSolution
-
8/7/2019 07. Matrix methods
24/54
Summary
Gauss-Jordan process
loop over rows (i)
scale row such that diagonal is 1.0 subtract a multiple of row i from
every other row in order to fill the
rest of the column with zeroes. move to next row
Final solution appears at the end of
the the augmented matrix.
-
8/7/2019 07. Matrix methods
25/54
Sub GaussJordan(A() As Double) 'Augmented matrix
Dim PivElt As Double, TarElt As DoubleDim n As Integer 'number of equations
Dim PivRow As Integer, TarRow As Integer 'pivot row; target row
Dim i As Integer, j As Integern = UBound(A, 1)
For PivRow = 1 To n 'process every rowPivElt = A(PivRow, PivRow) 'choose pivot element
If PivElt = 0 Then
MsgBox ("Zero pivot element encountered")
Stop
End IfFor j = 1 To n + 1
A(PivRow, j) = A(PivRow, j) / PivElt 'divide whole row
Next
For TarRow = 1 To n 'now replace all other rows
If Not (TarRow = PivRow) ThenTarElt = A(TarRow, PivRow)
For j = 1 To n + 1
A(TarRow, j) = A(TarRow, j) - A(PivRow, j) * TarElt
Next j
End IfNext TarRow
Next PivRow
End Sub
-
8/7/2019 07. Matrix methods
26/54
Gaussian EliminationPivoting:
If the main diagonal pivot element is small orzero, interchange rows (from below) to
maximize the pivot element
The determinant:
After forward elimination, the determinant is theproduct of all the main diagonal elements of the upper
triangular matrix. det(A) = u11u22u33 unn
-
8/7/2019 07. Matrix methods
27/54
Gaus. Elim. forward elimination
Eliminate elements in column 1
Example:
-
!
-
-
1
3
4
x
x
x
113
111
112
Solve
3
2
1
-
8/7/2019 07. Matrix methods
28/54
Gaus. Elim. forward elimination
Eliminate a21 by calculating a ratio:
R = a21/a11
Set row 2 = row 2 row 1
-
!
-
-
1
1
4
113
2/12/30
112
:Result
3
2
1
x
x
x
-
8/7/2019 07. Matrix methods
29/54
Gaus. Elim. forward elimination
Continue until all elements below
main diagonal = 0
Matrix is now upper triangular
-
!
-
-
3/16
1
4
3/800
2/12/30
112
:Result
3
2
1
x
x
x
-
8/7/2019 07. Matrix methods
30/54
-
8/7/2019 07. Matrix methods
31/54
Gaussian eliminationStep 1. Forward elimination:
Choose a pivot element on the main diagonal (start attop left corner)
Eliminate (set to zero) all elements below the maindiagonal
Move to the next column
Repeat until matrix is upper triangular
Step 2. Backward substitution:
Solve the last row explicitly for the last unknown Solve explicitly for the next row up
Repeat until all the rows are solved
-
8/7/2019 07. Matrix methods
32/54
LU Factorization
LU Factorization is a form of
Gaussian Elimination
The A matrix (in the problem Ax=b) is
factored into the product of two
matrices L and U (A=LU)
L is a lower triangular matrix and U isan upper triangular matrix
-
8/7/2019 07. Matrix methods
33/54
LU Factorization
How do we find the entries in L and
U?
there are nine unknowns (3 Ls and
6 Us)
there are nine equations (each
setting a row of L times a column
of U equal to one of the A values)
-
-
-
333231
232221
131211
33
2322
131211
3231
21
aaa
aaa
aaa
U00
UU0
UUU
1LL
01L
001
-
8/7/2019 07. Matrix methods
34/54
LU Factorization
.etc
a
aasoa
a
aasoa
asoa1
aaa
aaa
aaa
00
0
1
01
001
11
31
11
3131311131
11
21
11
21
21211121
11111111
333231
232221
131211
33
2322
131211
3231
21
!!!v
!!!v
!!v
-
!
-
-
-
8/7/2019 07. Matrix methods
35/54
Pivoting
What would happen if a11 were zero?
We would get a divide by zero error
when building L. Algorithms for LU factorization check
for this and swap the order of the
rows in A (pivoting) to avoid thisproblem.
-
8/7/2019 07. Matrix methods
36/54
LU Factorization
Once L and U have been determined
we have:
LUx = b How do we solve for x?
Set Ux=L-1b = y
Therefore Ly = b
Solve Ly=b for y
Solve Ux = y for x
-
8/7/2019 07. Matrix methods
37/54
Forward Substitution We solve Ly=b using a process
called forward substitution.
23213133
33232131
12122
22121
11
11
3
2
1
3
2
1
3231
21
yLyLby
by1yLyL
yLby
by1yLby
by1
b
b
b
y
y
y
1LL
01L
001
vv!
!vvv
v!
!vv
!
!v
-
!
-
-
-
8/7/2019 07. Matrix methods
38/54
Back Substitution
We solve Ux=y using a processcalled back substitution.
33
33
3333
3
2
1
3
2
1
33
2322
131211
yx
yx
yy
y
xx
x
000
!
!v
-
!
-
-
-
8/7/2019 07. Matrix methods
39/54
Back Substitution
11
31321211
1313212111
22
3232
2
2323222
U
xUxUyx
yxUxUxU
U
xUy
x
yxUxU
vv!
!vvv
v!
!vv
-
8/7/2019 07. Matrix methods
40/54
LU Decomposition and
Backsubstitution
Basic:
Call LUDecompose(A, Indx)
Call LUSubstitute(A, Indx, b, x)
-
8/7/2019 07. Matrix methods
41/54
Sub LUDecompose(A() As Double, Indx() As Integer)
' input a(n,n)
' output a(n,n) and indx(n)
Const Tiny As Double = 1E-16
Dim n As Integern = UBound(A, 1)
Dim i As Integer, j As Integer, k As Integer, m As
Integer
Dim dum As Double
For k = 1 To n - 1m = k
For i = k + 1 To n
If (Abs(A(i, k)) > Abs(A(m, k))) Then m = i
Next i
Indx(k) = mdum = A(m, k)
If m > k Then
A(m, k) = A(k, k)
A(k, k) = dumEnd IfIf Abs(dum) > Tiny Then
dum = 1 / dum
Else
MsgBox ("Singular coefficient matrix")
Stop
End IfNext k
-
8/7/2019 07. Matrix methods
42/54
Sub LUSubstitute(A() As Double, Indx() As Integer, _
b() As Double, x() As Double)' input A(n,n) containing the LU decomposition
' input Indx(n) index vector' input b(n) right hand side
' output x(n) solution
Dim n As Integern = UBound(A, 1)Dim i As Integer, j As Integer, k As Integer
Dim dum As DoubleFor i = 1 To n
End sub
-
8/7/2019 07. Matrix methods
43/54
Direct vs Iterative
Gauss elimination, Gauss-Jordan, LU-
decomposition are DIRECT methods
We can predict the number of
multiplications and additions if the
number of equations (n) is known
There are other direct methods for special
matrices. One of them is important for us:
It is the Thomas algorithm forTridiagonal
equations.
-
8/7/2019 07. Matrix methods
44/54
Occurs often in reservoir simulation
Thomas algorithm is efficient
Uses LU factorization
Tri-diagonal matrix
-
8/7/2019 07. Matrix methods
45/54
Tri-diagonal matrix
X1
.
.
.
.
.
.
.
.
.
.
.
.
.
xn
d1
.
.
.
.
.
.
.
.
.
.
.
.
.
dn
-
8/7/2019 07. Matrix methods
46/54
Thomas algorithm save a,b,c
vectors, not aii elementsX1
.
.
.
.
.
.
.
.
.
.
.
.
.
xn
d1
.
.
.
.
.
.
.
.
.
.
.
.
.
dn
ai
cibi
-
8/7/2019 07. Matrix methods
47/54
Thomas algorithmSub Thomas (a() As Double, b() As Double, c() As Double, d() As Double, x() As
Double)
'Tridiagonal system ofequations
Dim n As Integer, i As Integer
n = UBound(b)
ReDim w(n) As Double, g(n) As Double
w(1) = b(1)
g(1) = d(1) / w(1)
Fori = 2Ton
w(i) = b(i) a(i) * c(i 1) / w(i 1)
g(i) = (d(i) a(i) * g(i 1)) / w(i)
Next ix(n) = g(n)
Fori = n 1To1 Step 1
x(i) = g(i) c(i) * x(i + 1) / w(i)
Next i
nd Sub
-
8/7/2019 07. Matrix methods
48/54
Direct vs Iterative
Iterative methods are generalization ofthe direct iteration we learned for onevariable x=g(x) equations.
Iterative methods are used for verylarge but sparse systems (reservoirsimulation).
You can not predict the number ofoperations for an indirect method andyou need a starting guess and aconvergence criterion.
The method we are going to learn is theGauss-Seidel method.
-
8/7/2019 07. Matrix methods
49/54
Iterative methods
Gauss Seidel
-
8/7/2019 07. Matrix methods
50/54
Gauss-Seidel
Suppose we are trying to solve a
system of three equations:
1x2xx
11x2x4x
8xx2x3
321
321
321
!
!
!
-
8/7/2019 07. Matrix methods
51/54
Gauss-Seidel
These equations can be rewritten as:
2
xx1x
4
x2x11x
3xx28x
213
312
321
!
!
!
-
8/7/2019 07. Matrix methods
52/54
Gauss-Seidel Iteration
Gauss-Seidel starts from an initial
guess of the xs and updates that
guess in an attempt to converge to asolution:
2
xx1x
4x2x11x
3
xx28x
1k
2
1k
11k
3
k31k11k
2
k
3
k
21k
1
!
!
!
-
8/7/2019 07. Matrix methods
53/54
Diagonally Dominant Matrices
Diagonally dominant matrices:
The absolute value of the diagonal element is
greater than the sum of the absolute values
of all the other elements in a given row.
Identity matrix is diagonally dominant
Diagonally Dominant?
YES NO
-
1032
252
214
-
1032
232
214
-
8/7/2019 07. Matrix methods
54/54
Diagonal Dominance
Gauss-Seidel will only work if/only if
the matrix is diagonally dominant.