Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems
description
Transcript of Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems
Lecture 10Dimensions, Independence, Basis and Complete Solution of Linear Systems
Shang-Hua Teng
Linear Independence
Linear Combination
in
v
,...,v,vv
n
1i i
21
is vectorsofset a of
Linear Independence
vectorsofset A 21 n,...,v,vv
is linearly independent if only if none of them can be expressed as a linear combination of the others
Examples
1
1
0
,
4
2
2
,
1
0
1
1
1
2
2
1
1
0
1,,
Linear Independence and Null Space
Theorem/Definition
vectorsofset A 21 n,...,v,vvis linearly independent if and only1v1+2v2+…+nvn=0 only happens when all ’s are zero
The columns of a matrix A are linearly independent when only solution to Ax=0 is x = 0
2D and 3Dv
w
u
v
How do we determine a set of vectors are independent?
Make them the columns of a matrix
Elimination
Computing their null space
Permute Rows and Continuing Elimination (permute columns)
011121
131111
021111
110011
A
Theorem
If Ax = 0 has more have more unknown than equations (m > n: more columns than rows), then it has nonzero solutions.
There must be free variables.
Echelon Matrices
*
*
*
*
000000
**0000
*****0
******
A
Free variables
Reduced Row Echelon Matrix R
1
0
0
0
000000
*10000
*0**10
*0**01
A
Free variables
Computing the Reduced Row Echelon Matrix
• Elimination to Echelon Matrix
E1PA = U
• Divide the row of pivots by the pivots
• Upward Elimination
E2E1PA = R
Example: Gauss-Jordan Method for Matrix Inverse
• [A I]
• E1[A I] = [U, E1]
• In its reduced Echelon Matrix
• A-1 [A I] = [I A-1]
A Close Look at Reduced Echelon Matrix
• The last equation of R x = 0 is redundant
0 = 0
• Rank of A is the number of pivots rank(A).
00000
34100
12031
R
What is the Rank of Outer Product
n
n
vv
u
u
1
1
Rank and Reduced Row Echelon Matrix
0000000
1
0
0
0
000000
*10000
*0**10
*0**01
A
Free variables• Theorem/Definition
• Rank(A) = number of independent rows
• Rank(A) = number of independent columns
Dimension of the Column Space and Null Space
• The dimensions of the column space of A is equal to Rank(A).
• The dimension of the null space of A is equal to the number of free variables which is n – Rank(A)
• A is an m by n matrix
Rank and Reduced Row Echelon Matrix
0000000
1
0
0
0
000000
*10000
*0**10
*0**01
A
Free variablesFree Columns
Pivot columns
The Pivot columns are not combinations of earlier columns
Reduced Echelon and Null Space Matrix
• Nullspace Matrix• Special Solutions
0
0
0
00000
34100
12031
5
4
3
2
1
x
x
x
x
x
Rx
100
010
340
001
123
N
100
010
001
100
010
340
001
123
Null Space Matrix
• Ax=0 has n-Rank(A) free variables and special solutions
• The Nullspace matrix has n-Rank(A) columns• The columns of the nullspace matrix are
independent• The dimension of the Null space is n – rank(A)
Complete Solution of Ax = 0• After column permutation, we can write
rows zeros
rowspivot
00 m-r
rFIR
r pivot columns n-r free columns
• Nullspace matrix
m-r
r
I
FN
Pivot variables
Free variables
• Moreover: RN = [0]
Complete Solution to Ax = b• A is an m by n matrix, and b is an n-place vector
– Unique solution– Infinitely many solution– No solution
Suppose Ax = b has more then one solution, say x1, x2 then A x1 = bA x2 = b
So A (x1 - x2 ) = 0
(x1 - x2 ) is in nullspace(A)
Complete Solution to Ax = b
Suppose we found a particular solution xp to Ax = b i.e, A xp = b
Let F be the indexes of free variables of Ax = 0 Let xF be the column vector of free variablesLet N be the nullspace matrix of A
Then
defines the complete set of solutions to Ax = b
FNx xp
Example: Complete Solution to Ax = b
7
6
1
6131
4100
2031
4
3
2
1
x
x
x
x
7
6
1
6131
4100
2031
Augmented matrix [A b]
Elimination to obtain [R d]
0
6
1
0000
4100
2031
Set free variables to 0 to find a particular solution(1,0,6,0)T
Compute the nullspace matrix
10
40
01
23
Complete solution is
1
4
0
2
0
0
1
3
0
6
0
1
10
40
01
23
0
6
0
1
424
2 xxx
xx
Full Rank Matrix
• Suppose A is an m by n matrix. Then
• A is full column if rank(A) = n– columns of A are independent
• A is full row rank if rank(A) = m– Rows of A are independent
),min()(rank nmA
Full Column Rank Matrix
• Columns are independent
• All columns of A are pivot columns
• There are non free variables or special solutions
• The nullspace N(A) contains only the zero vector
• If Ax=b has a solution (it might not) then it has only one solution
0
IR
n by n
m-n rows of zeros
Full Row Rank Matrix
• Rows are independent
• All rows of A have pivots, R has no zero rows
• Ax=b has a solution for every right hand side b
• The column space is the whole space Rm
• There are n-m special solutions in the null space of A
FIR
The Whole Picture• Rank(A) = m = n Ax=b has unique solution IR
FIR
0
IR
00
FIR
• Rank(A) = m < n Ax=b has n-m dimensional solution
• Rank(A) = n < m Ax=b has 0 or 1 solution
• Rank(A) < n, Rank(A) < m Ax=b has 0 or n-rank(A) dimensions
Basis and Dimension of a Vector Space
• A basis for a vector space is a sequence of vectors that – The vectors are linearly independent– The vectors span the space: every vector in the
vector can be expressed as a linear combination of these vectors
Basis for 2D and n-D
• (1,0), (0,1)
• (1 1), (-1 –2)
• The vectors v1,v2,…vn are basis for Rn if and only if they are columns of an n by n invertible matrix
Column and Row Subspace
• C(A): the space spanned by columns of A– Subspace in m dimensions
– The pivot columns of A are a basis for its column space
• Row space: the space spanned by rows of A– Subspace in n dimensions
– The row space of A is the same as the column space of AT, C(AT)
– The pivot rows of A are a basis for its row space
– The pivot rows of its Echolon matrix R are a basis for its row space
Important Property I: Uniqueness of Combination
• The vectors v1,v2,…vn are basis for a vector space V, then for every vector v in V, there is a unique way to write v as a combination of v1,v2,…vn .
v = a1 v1+ a2 v2+…+ an vn
v = b1 v1+ b2 v2+…+ bn vn
• So: 0=(a1 - b1) v1 + (a2 -b2 )v2+…+ (an -bn )vn
Important Property II: Dimension and Size of Basis
• If a vector space V has two set of bases– v1,v2,…vm . V = [v1,v2,…vm ]– w1,w2,…wn . W= [w1,w2,…wn ].
• then m = n– Proof: assume n > m, write W = VA– A is m by n, so Ax = 0 has a non-zero solution– So VAx = 0 and Wx = 0
• The dimension of a vector space is the number of vectors in every basis– Dimension of a vector space is well defined
Dimensions of the Four SubspacesFundamental Theorem of Linear
Algebra, Part I• Row space: C(AT) – dimension = rank(A)
• Column space: C(A)– dimension = rank(A)
• Nullspace: N(A) – dimension = n-rank(A)
• Left Nullspace: N(AT) – dimension = m –rank(A)
•