MAT 2401 Linear Algebra 4.4 Spanning Sets and Linear Independence .
ENGG2013 Unit 5 Linear Combination & Linear Independence
description
Transcript of ENGG2013 Unit 5 Linear Combination & Linear Independence
ENGG2013 Unit 5 Linear Combination
& Linear IndependenceJan, 2011.
Last time• How to multiply a matrix and a vector• Different ways to write down a system of linear equations
– Vector equation
– Matrix-vector product
– Augmented matrix
kshum ENGG2013 2
Column vectors
Review: matrix notation
• In ENGG2013, we use capital bold letter for matrix.• The first subscript is the row index, the second subscript
is the column index.• The number in the i-th row and the j-th column is called
the (i,j)-entry.– cij is the (i,j)-entry in C.
kshum ENGG2013 3
m n
Matrix-vector multiplication
kshum ENGG2013 4
Today• When is A x = b solvable?
– Given A, under what condition does a solution exist for all b?
• For example, the nutrition problem: find a combination of food A, B, C and D in order to satisfy the nutrition requirement exactly.
kshum ENGG2013 5
Food A Food B Food C Food D Requirement
Protein 9 8 3 3 5
Carbohydrate 15 11 1 4 5
Vitamin A 0.02 0.003 0.01 0.006 0.01
Vitamin C 0.01 0.01 0.005 0.05 0.01
Different people havedifferent requirements
Can we solve A x = b for fixed A and various b?
Today
• Basic concepts in linear algebra– Linear combination– Linear independence– Span
kshum ENGG2013 6
Three cases: 0, 1,
kshum ENGG2013 7
A x = b
No solution Unique solution Infinitely manysolutions
How to determine?m equationsn variables
GEOMETRY FOR LINEAR SYSTEM TWO EQUATIONS
kshum ENGG2013 8
Scaling
kshum ENGG2013 9
y
x1
1
y
x
c
c
c is any real number
Representing a straight line by vector
kshum ENGG2013 10
y
x
c
c
y
x
y=x
Any point on the line y=x canbe written as
Adding one more vector
kshum ENGG2013 11
y
x
y=xy
x
y=x+1
We can add another vector and get the same result
kshum ENGG2013 12
y
x
y=x+1
y
x
y=x+1
=
The whole plane
kshum ENGG2013 13
y
x
Scanner
Question 1
kshum ENGG2013 14
Can you find c and d such that
1
2
3
4
5
6
7
2 43 5
?
Question 2
kshum ENGG2013 15
Can you find c and d such that
1
2
3
4
5
6
7
2 43 5
Question 3
kshum ENGG2013 16
Can you find c, d, and e such that
1
2
3
4
5
6
7
2 43 5
GEOMETRY FOR LINEAR SYSTEM THREE EQUATIONS
kshum ENGG2013 17
From line to plane to space
kshum ENGG2013 18
x
y
z
Scalar multiplesof
x
z
Any point in the 3-D spacecan be written as
x
yz
Any point in the x-y planecan be written as
Question 4
kshum ENGG2013 19
x
y
z
The three red arrowsall lie in the x-y plane
Can you find a, b, and c, such that
?
Question 5
kshum ENGG2013 20
x
y
z
The three red arrowsall lie in the shaded plane.
Can you scale up (or down) the three red arrowssuch that the resulting vector sum is equal to theblue vector?
Question 6
kshum ENGG2013 21
x
y
z
The three red arrowsall lie in a straight line.
Can you find x, y and z such that
?
Question 7
kshum ENGG2013 22
x
y
z
The three red arrowsand the blue arroware all on the same line.
Can you find x, y and z such that
?
ALGEBRA FOR LINEAR EQUATIONS
kshum ENGG2013 23
Review on notation
• A vector is a list of numbers.• The set of all vectors with two components is
called .• is a short-hand notation for saying
that – v is a vector with two components– The two components in v are real numbers.
kshum ENGG2013 24
• The set of all vectors with three components is called .
• is a short-hand notation for saying that – v is a vector with three components– The three components in v are real numbers.
kshum ENGG2013 25
• The set of all vectors with n components is called .
• We use a zero in boldface, 0, to represent the all-zero vector
kshum ENGG2013 26
Definition: Linear Combination
• Given vectors v1, v2, …, vi in , and i real number c1, c2, …, ci, the vector w obtained by w = c1 v1+ c2 v2+ …+ ci vi
is called a linear combination of v1, v2, …, vi .
• Examples of linear combination of v1 and v2:
kshum ENGG2013 27
Picture
• Linear combinations of two vectors u and v.
kshum ENGG2013 28
u
v
2u+2v
u–2v
–v
0
3u
2u+0.5v
Definition: Span
• Given r vectors v1, v2, …, vr, the set of all linear combinations of v1, v2, …, vr called the span of v1, v2, …, vr,
• We use the notation span(v1, v2, …, vr) for the span of span of v1, v2, …, vr.
• We also say that span(v1, v2, …, vr) is spanned by, or generated by v1, v2, …, vr .
• span(v1, v2, …, vr) is the collections of all vectors which can be written as c1v1 + c2v2 + … + c2vr for some scalars c1, c2, …, cr.
kshum ENGG2013 29
Span of u and v• Linear combinations of this two vectors u and v form the whole plane
kshum ENGG2013 30
u
v
2u+2v
u–2v
–v
0
3u
Span of a single vector u
kshum ENGG2013 31
x
y
z
consists of thepoints on a straight linewhich passes through the origin.
u
Span of two vectors in 3D
kshum ENGG2013 32
x
y
z
u
v
is a planethrough the origin.
Example
kshum ENGG2013 33
1
2
3
4
5
6
7
2 43 5
is a linear combination of
and , because
We therefore say that
Mathematical language
kshum ENGG2013 34
President Obama is not a Chinese.
Ordinary language Mathematical language
Let C be the set of all Chinese people.
President Obama
Example
kshum ENGG2013 35
x
y
z
A fundamental fact• Let
– A be an mn matrix– b be an m1 vector
• Let the columns of A be v1, v2,…, vn. • The followings are logically equivalent:
kshum ENGG2013 36
We can find a vector x such that1
2
3
“Logically equivalent” meansif one of them is true, then all of them is trueif one of them is false, then all of them is false.
Theorem 1
• With notation as in previous slide, if the span of be v1, v2,…, vn contains all vectors inthen the linear system Ax = b has at least one solution.
• In other words, if every vector in can be written as a linear combination of v1, v2,…, vn, then Ax = b is solvable for any choice of b.
kshum ENGG2013 37
“Solvable” means there is onesolution or more than one solutions.
Example
kshum ENGG2013 38
1
2
3
4
5
6
7
2 43 5
Since and span
the whole plane, the linearsystem
is solvable for any b1 and b2.
Example
kshum ENGG2013 39
x
y
z
The three red arrowsall lie in the x-y plane
x
y
z
(Infinitely many solutions)
Example
kshum ENGG2013 40
1
2
3
4
5
6
7
2 43 5
because is not a
linear combination of
Example
kshum ENGG2013 41
1
2
3
4
5
6
7
2 43 5
has infinitely many solutions.
Infinitely many solutions
kshum ENGG2013 42
x
y
z
x
y
Notice that is a scalar
multiple of is a linear combination of and
There is one common feature inthe examples with infinitely many solutions
The common feature is that one of the vectoris a linear combination of the others.
Definition: Linear dependence• Vectors v1, v2, …, vr are said to be linear dependent
if we can find r real number c1, c2, …, cr, not all of them equal to zero, such that 0 = c1 v1+ c2 v2+ …+ cr vr
• Otherwise, are v1, v2, …, vr are said to be linear independent.
• In other words, v1, v2, …, vr are be linear independent if, the only choice of c1, c2, …, cr, such that 0 = c1 v1+ c2 v2+ …+ cr vr
is c1 = c2 = …= cr=0. kshum ENGG2013 43
Example of linear independent vectors
kshum ENGG2013 44
Example of linear dependent vectors
kshum ENGG2013 45
Example of linear independent vectors
kshum ENGG2013 46
Example
• and are linear dependent, because
• , and are linear dependent because
kshum ENGG2013 47
Picture
kshum ENGG2013 48
x
y
z
The three vectors lie onthe same plane, namely, the x-y plane.
Theorem 2
• Let – A be an mn matrix– b be an m1 vector
• Let the columns of A be v1, v2,…, vn.
• Theorem: If v1, v2,…, vn, are linear independent, then Ax = b has at most one solution.
kshum ENGG2013 49
Proof (by contradiction)• Suppose that and are two different solutions to Ax=b,
i.e.,
• Therefore
• Move every term to the left
• But v1, v2,…, vn are linear independent by assumption. So, the onlychoice is
• This contradicts the fact that vector x and vector x’ are different.
kshum ENGG2013 50
Example
• and are linearly independent.
• has a unique solution for
any choice of b1 and b2.
kshum ENGG2013 51
In fact, x must equal b1,and y must equal b2/3in this example.
Example
• is solvable
by Theorem 1, because the bluevector lies on the planespanned by the two red vectors.
• The solution is unique
because and are linearly independent.
kshum ENGG2013 52
x
z
y
Summary
kshum ENGG2013 53
At most one solution At least one solution
Ax=bm equationsn variables
Every vector inis a linear combinationof the columns in A.
Columns of A are linearly independent
Uniqu
e so
lution
The columns of Acontain a lot of informationabout the nature of thesolutions.
A kind of mirror symmetry
kshum ENGG2013 54
If the columns of A are linear independent,
then I am pretty sure that there is one or no
solution to Ax=b,no matter what b is.
If any vector in can be written as a
linear combination of the column vectors in A,
then Ax=b must have one or more than one
solutions.
Basis
• A set of vector in which are simultaneously – linearly independent, and – spanning the whole space
is of particular importance, and is called a set of basis vectors.
(We will talk about basis in more detail later.)
kshum ENGG2013 55