Solvers for Systems of Nonlinear Algebraic Equations; Where are we TODAY?
MA2213 Lecture 5 Linear Equations (Direct Solvers)
-
Upload
britney-stone -
Category
Documents
-
view
216 -
download
0
Transcript of MA2213 Lecture 5 Linear Equations (Direct Solvers)
MA2213 Lecture 5
Linear Equations
(Direct Solvers)
Systems of Linear Equations p. 243-248
Occur in a wide variety of disciplines
Mathematics
Statistics
Physics
Chemistry Biology
Economics
Sociology
Psychology
Archaeology
Geology
AstronomyAnthropology
EngineeringManagement
Business
Medicine
Finance
Matrix Form
for a system of linear equations bxA
nnRA nRbnRx
coefficient matrix
(solution) column vector
(right) column vector
Linear Equations in Mathematics
Numerical Analysis
Geometry
Interpolation
Least Squares
Quadrature
Algebra
find intersection of lines or planes
partial fractions 111
22
x
b
x
a
x
0ba2 ba 1a
1b
Coefficient Matrix
Vandermonde (for polyn. interp.)
or Gramm
Transpose of VandermondeLec 4 vufoil 13 (to compute weights)
BBT
2
0
11
11
b
a
Matrix Arithmetic p. 248-264
Matrix Inverse
Matrix MultiplicationnmRA
100
010
001
3IIdentity Matrix
pnRB pmRAB
10
012I
ac
bd
bcaddc
ba 11
Theorem 6.2.6 p. 255 A square matrix has an inverse iff (if and only if) its determinant is not equal to zero.
Solution of(this means )0det A
exists and is unique.
bAxAAbAx 11 )(
bAxAA 11 )(
bAxbAxI 11
multiplication is associative
for nonsingular
bxA
Proof
A
Remark In MATLAB use: x = A \ b;
Column Rank of a Matrix
Definition The column rank of a matrix
},...,1,0{cr , mMRM nm
dimension of the subspace of 1 mm RR
?021
242cr
spanned by the column vectors of
Mcr Remark
Mmaximal number of
linearly independent column vectors
M
Question
is the
of
Row Rank of a Matrix
Definition The row rank of a matrix},...,1,0{rr , nMRM nm
is the dimension of the subspace of 1nR
?021
242rr
spanned by the row vectors of
Mrr Remark
Mmaximal number of
linearly independent row vectors of M
Question
A Matrix Times a Vector
nnnnnn
n
n
b
b
b
x
x
x
aaa
aaa
aaa
bxA
2
1
2
1
21
22221
11211
has solution iff b is a linear combination of columns of A
nnn
n
n
n
nn b
b
b
a
a
a
x
a
a
a
x
a
a
a
x
2
1
2
1
2
22
12
2
1
21
11
1
The equation
Existence of Solution in General
The linear equation bxA has a solution if and only if
AbA cr][cr A IS SINGULAR!EVEN IF Example
2
1
2
1
21
42
b
b
x
xthis has a solution iff
21 2bb then it has an infinite number of solutions
called Augmentedmatrix p. 265
Computing the Column and Row RanksThe ranks of a matrix nmRM can be computed using a sequence of elementary row operations p. 253-254.
i. Interchange two rowsii. Multiply a row by a nonzero scalariii. Add a nonzero multiple of one row to another rowQuestion Show that each of the ERO i, ii, iii has an inverse ERO i, ii, iii.
Elementary Row Operations
00010
01000
00100
10000
00001
iE
can be performed
on the left by by multiplying
nmRM on a matrix
Mnonsingular matrices
mmRE
MEM i
10000
05000
00100
00010
00001
iiE
MEM ii MEM ii
10000
01000
00100
02010
00001
iiiE
Invariance of Row Rank Under EROTheorem 1. Ifis an ERO matrix, thenProof Clearly, interchanging two rowsand multiplying a row by a nonzero scalar does not change the row rank. Finish the proof by showing that adding a multiple of any row to another row does not change the row rank.
nmRM and mmRE .rr rr MME
Remark Clearly the row rank of a matrix is invariant under sequence of ERO’s.
Matrix Multiplication
nmn RvvvM 21
mnn Rvvvvv ,,...,,, 1321
mmRE nm
n REvEvEvEM 21
Invariance of Column Rank under EROnmRM Theorem 2 If
is nonsingular then .cr cr MME Proof It suffices to show that for a set
are linearly dependent iff the set ofMofof column vectors
rkkk vvv ,...,,21
rkkk EvEvEv ,...,,21
are linearly dependent. Show why it suffices and then show it. Hint: prove
mmRE and
column vectors of ME
0011 11
rr krkkrk vcvcEvcEvc
Row Echelon Matrices
Definition A matrix
an row echelon matrix if
i. the nonzero rows come first
nmRM
ii. the first nonzero element in each row =1 (called a pivot) has all zeros below it
is called
iii. each pivot lies to the right of the pivot in the row above
Row Echelon MatricesThese three properties produce a staircase pattern in the matrix below
000000000
610000000
075231000
4407.931610
13541506.831
Question Where are the pivots ?
Row Rank of an Row Echelon Matrixequals the number of nonzero rows.
000000000
610000000
075231000
4407.931610
13541506.831
Question What is the rank of this matrix ?
Prove this by showing that the rows must be linearly independent. Hint : use pivots.
Col. Rank of a Row Echelon Matrixequals the number of nonzero rows.
000000000
610000000
075231000
4407.931610
13541506.831
Question Show this by showing that the col. vectors that contain pivots form a basis for the space spanned by col. vectors. Hint: do elem. col. operations on the matrix above.
Reduction to Row Echelon FormTheorem 3 For every matrix
nmRM there exists a nonsingular matrix
is an echelon matrix. such that
kEEEEEE 4321
mmRE ME
Furthermore, the matrix E is a product where each
kjjE 1, is an ERO matrix.
Application of the sequence of ERO’s is called reduction to row echelon form.
Proof Based on Gaussian elimination.
Row Rank = Column RankTheorem 4 For every matrix
nmRM
Proof. Theorem 3 implies that there exists
kEEEEEE 4321
EMM rr rr
MM rr cr
MEa product
of ERO matrices such that is a rowechelon matrix. Theorems 1 implies that
and theorems 2 implies that
.cr cr EMM Since ME is a row echelon
matrix, EMME cr rr hence .cr rr MM
Applications of Row Echelon Reduction
The linear equation bxA iff the last nonzero row of the reduced ][ bAE
Example
22
2
2
2
2
1
1
11
00
21
21
21
21
42b
bb
bbb
b
has a solution
has its pivot NOT in the last column.
Hence the condition above is satisfied iff .0221 bb
Applications of Row Echelon Reduction
A basis of column vectors for a matrix
],...,,[ 21 nvvvME can be obtained by first computing the reduction
nmRM
then choosing the column vectors
that form a basis for the space spanned by the column
rkkk vvv ,...,,21
that contain the pivots. Then the vectors
rkkk vEvEvE 111 ,...,,21
are column vectors of
vectors of
.M
M
Generalities on Gaussian EliminationGaussian elimination is the process of reducing a matrix to row echelon form through a sequence of ERO’s.
It can also be used to solve a system of linear equations
The final step of solving a system of equations after the augmented matrix has been reduced is called back substitution, this process is related to elementary column operations and will be addressed in the homework.
It is ‘best’ taught through showing examples.
We will show how to solve a system of linear equationsusing Gaussian elimination, it will become obvious how to use Gaussian elimination for reduction.
Gaussian Elimination (p. 264-269)Case 1.
nna
a
a
A
00
00
00
22
11
nnnn bxabxabxa ,...,, 22221111
The equations for this matrix are
Question How do we use the nonsingular assumption?
therefore, if A is nonsingular then
nn
nn a
bx
a
bx
a
bx ,...,,
22
22
11
11
Question What type of matrix is this ?
Back SubstitutionCase 2.
nn
n
n
a
aa
aaa
A
00
0 222
11211
A nonsingular solution by back-substitution p. 265
Question How do we use the nonsingular assumption?
nnnn bax 1
Question What is this matrix called ?
Question What are the associated equations ?
][ ,111
1,11 nnnnnnn xabax
][ 131321211
111 nnxaxaxabax
Question Why is this method called back-substitution ?
Gaussian Elimination on EquationsCase 3. Apply elementary row operations on equations to to obtain equations with an upper triangular matrix
22 321 xxx 1r
122 3rrr
133 321 xxx 2r421 xx 3r
2/,5/ 3322 rrrr
22 321 xxx55 3 x
133 rrr 222 32 xx22 321 xxx132 xx13 x 32 rr
Question How can we solve these equations ?
Gaussian Elimination on Augmented Matrix
nnnnn
n
n
b
b
b
aaa
aaa
aaa
2
1
21
22221
11211
11rarr iii
1
12
1
112
12
122
11211
0
0
nnnn
n
n
b
b
b
aa
aa
aaa
jjiii rarr
nj ,...,2for
loop) ( end i
1111 / arr
loop) ( end j
0,1, 11 kk akrr0 If 11 a
ni ,...,2for
)loop ( end i
0 If jja 0,, jkkj ajkrr
jjjj arr /nji ,...,1for
Gaussian Elimination
122 3rrr
133 rrr
4011
1133
2211
Question What is the solution ?
2220
5500
2211
1100
1110
2211
251
2 Rr
32 rr 32
13 rr
122 3rrr
bAx
EbEAx
Partial Pivoting p. 270-273
the integer ||max1
kjnk
j aS
that gives
kj Rr
For the j-th column in Gaussian elimination compute
nkj
then perform the row interchange
Read p. 273-276 about how Gaussian elimination can be used to compute the inverse of a matrix.
LU Decomposition p. 283-285
ULATo solve
where
1
01
001
21
21
nn
L
nn
n
n
u
uu
uuu
U
00
0 222
11211
Then for each b use forward substitution to solve L y = b then use backward substitution to solve U x = y.
first compute the factorization
bAx for many values of b with same A
LU Decomposition AlgorithmAlgorithm
Step 1 njau jj ,...,1,11 niuaii ,...,2,/ 1111
Step 2 for r = 2,…,n do
njruau jk
r
kkrjrjr
,1
1
niruau rk
r
kkirirrri
1,1
1
1
Question How many operations does this require ?
Homework Due Tutorial 3Question 1. Prove that the row rank of an row echelon matrix equals the number of nonzero rows.Question 2. Prove that the column rank of an row echelon matrix equals the number of nonzero rows by showing that the set of its column vectors having pivots is a maximal set of linearly independent column vectors. Question 3. Use Gaussian elimination to solve
901565,54963,2842 32132132 uuuuuuuuQuestion 4. Derive expressions for the entries of the Land U in the LU decomposition of a 3 x 3 matrix A.Question 5. Show how elementary column operations can be applied to a row echelon matrix M to obtain a row echelon matrix with exactly one 1 in each nonzero row. Use this to determine a basis for the space { x : Mx = 0 }.