ENGG2013 Unit 5 Linear Combination

& Linear IndependenceJan, 2011.

Last time• How to multiply a matrix and a vector• Different ways to write down a system of linear equations

– Vector equation

– Matrix-vector product

– Augmented matrix

kshum ENGG2013 2

Column vectors

Review: matrix notation

• In ENGG2013, we use capital bold letter for matrix.• The first subscript is the row index, the second subscript

is the column index.• The number in the i-th row and the j-th column is called

the (i,j)-entry.– cij is the (i,j)-entry in C.

kshum ENGG2013 3

m n

Matrix-vector multiplication

kshum ENGG2013 4

Today• When is A x = b solvable?

– Given A, under what condition does a solution exist for all b?

• For example, the nutrition problem: find a combination of food A, B, C and D in order to satisfy the nutrition requirement exactly.

kshum ENGG2013 5

Food A Food B Food C Food D Requirement

Protein 9 8 3 3 5

Carbohydrate 15 11 1 4 5

Vitamin A 0.02 0.003 0.01 0.006 0.01

Vitamin C 0.01 0.01 0.005 0.05 0.01

Different people havedifferent requirements

Can we solve A x = b for fixed A and various b?

Today

• Basic concepts in linear algebra– Linear combination– Linear independence– Span

kshum ENGG2013 6

Three cases: 0, 1,

kshum ENGG2013 7

A x = b

No solution Unique solution Infinitely manysolutions

How to determine?m equationsn variables

GEOMETRY FOR LINEAR SYSTEM TWO EQUATIONS

kshum ENGG2013 8

Scaling

kshum ENGG2013 9

y

x1

1

y

x

c

c

c is any real number

Representing a straight line by vector

kshum ENGG2013 10

y

x

c

c

y

x

y=x

Any point on the line y=x canbe written as

Adding one more vector

kshum ENGG2013 11

y

x

y=xy

x

y=x+1

We can add another vector and get the same result

kshum ENGG2013 12

y

x

y=x+1

y

x

y=x+1

=

The whole plane

kshum ENGG2013 13

y

x

Scanner

Question 1

kshum ENGG2013 14

Can you find c and d such that

1

2

3

4

5

6

7

2 43 5

?

Question 2

kshum ENGG2013 15

Can you find c and d such that

1

2

3

4

5

6

7

2 43 5

Question 3

kshum ENGG2013 16

Can you find c, d, and e such that

1

2

3

4

5

6

7

2 43 5

GEOMETRY FOR LINEAR SYSTEM THREE EQUATIONS

kshum ENGG2013 17

From line to plane to space

kshum ENGG2013 18

x

y

z

Scalar multiplesof

x

z

Any point in the 3-D spacecan be written as

x

yz

Any point in the x-y planecan be written as

Question 4

kshum ENGG2013 19

x

y

z

The three red arrowsall lie in the x-y plane

Can you find a, b, and c, such that

?

Question 5

kshum ENGG2013 20

x

y

z

The three red arrowsall lie in the shaded plane.

Can you scale up (or down) the three red arrowssuch that the resulting vector sum is equal to theblue vector?

Question 6

kshum ENGG2013 21

x

y

z

The three red arrowsall lie in a straight line.

Can you find x, y and z such that

?

Question 7

kshum ENGG2013 22

x

y

z

The three red arrowsand the blue arroware all on the same line.

Can you find x, y and z such that

?

ALGEBRA FOR LINEAR EQUATIONS

kshum ENGG2013 23

Review on notation

• A vector is a list of numbers.• The set of all vectors with two components is

called .• is a short-hand notation for saying

that – v is a vector with two components– The two components in v are real numbers.

kshum ENGG2013 24

• The set of all vectors with three components is called .

• is a short-hand notation for saying that – v is a vector with three components– The three components in v are real numbers.

kshum ENGG2013 25

• The set of all vectors with n components is called .

• We use a zero in boldface, 0, to represent the all-zero vector

kshum ENGG2013 26

Definition: Linear Combination

• Given vectors v1, v2, …, vi in , and i real number c1, c2, …, ci, the vector w obtained by w = c1 v1+ c2 v2+ …+ ci vi

is called a linear combination of v1, v2, …, vi .

• Examples of linear combination of v1 and v2:

kshum ENGG2013 27

Picture

• Linear combinations of two vectors u and v.

kshum ENGG2013 28

u

v

2u+2v

u–2v

–v

0

3u

2u+0.5v

Definition: Span

• Given r vectors v1, v2, …, vr, the set of all linear combinations of v1, v2, …, vr called the span of v1, v2, …, vr,

• We use the notation span(v1, v2, …, vr) for the span of span of v1, v2, …, vr.

• We also say that span(v1, v2, …, vr) is spanned by, or generated by v1, v2, …, vr .

• span(v1, v2, …, vr) is the collections of all vectors which can be written as c1v1 + c2v2 + … + c2vr for some scalars c1, c2, …, cr.

kshum ENGG2013 29

Span of u and v• Linear combinations of this two vectors u and v form the whole plane

kshum ENGG2013 30

u

v

2u+2v

u–2v

–v

0

3u

Span of a single vector u

kshum ENGG2013 31

x

y

z

consists of thepoints on a straight linewhich passes through the origin.

u

Span of two vectors in 3D

kshum ENGG2013 32

x

y

z

u

v

is a planethrough the origin.

Example

kshum ENGG2013 33

1

2

3

4

5

6

7

2 43 5

is a linear combination of

and , because

We therefore say that

Mathematical language

kshum ENGG2013 34

President Obama is not a Chinese.

Ordinary language Mathematical language

Let C be the set of all Chinese people.

President Obama

Example

kshum ENGG2013 35

x

y

z

A fundamental fact• Let

– A be an mn matrix– b be an m1 vector

• Let the columns of A be v1, v2,…, vn. • The followings are logically equivalent:

kshum ENGG2013 36

We can find a vector x such that1

2

3

“Logically equivalent” meansif one of them is true, then all of them is trueif one of them is false, then all of them is false.

Theorem 1

• With notation as in previous slide, if the span of be v1, v2,…, vn contains all vectors inthen the linear system Ax = b has at least one solution.

• In other words, if every vector in can be written as a linear combination of v1, v2,…, vn, then Ax = b is solvable for any choice of b.

kshum ENGG2013 37

“Solvable” means there is onesolution or more than one solutions.

Example

kshum ENGG2013 38

1

2

3

4

5

6

7

2 43 5

Since and span

the whole plane, the linearsystem

is solvable for any b1 and b2.

Example

kshum ENGG2013 39

x

y

z

The three red arrowsall lie in the x-y plane

x

y

z

(Infinitely many solutions)

Example

kshum ENGG2013 40

1

2

3

4

5

6

7

2 43 5

because is not a

linear combination of

Example

kshum ENGG2013 41

1

2

3

4

5

6

7

2 43 5

has infinitely many solutions.

Infinitely many solutions

kshum ENGG2013 42

x

y

z

x

y

Notice that is a scalar

multiple of is a linear combination of and

There is one common feature inthe examples with infinitely many solutions

The common feature is that one of the vectoris a linear combination of the others.

Definition: Linear dependence• Vectors v1, v2, …, vr are said to be linear dependent

if we can find r real number c1, c2, …, cr, not all of them equal to zero, such that 0 = c1 v1+ c2 v2+ …+ cr vr

• Otherwise, are v1, v2, …, vr are said to be linear independent.

• In other words, v1, v2, …, vr are be linear independent if, the only choice of c1, c2, …, cr, such that 0 = c1 v1+ c2 v2+ …+ cr vr

is c1 = c2 = …= cr=0. kshum ENGG2013 43

Example of linear independent vectors

kshum ENGG2013 44

Example of linear dependent vectors

kshum ENGG2013 45

Example of linear independent vectors

kshum ENGG2013 46

Example

• and are linear dependent, because

• , and are linear dependent because

kshum ENGG2013 47

Picture

kshum ENGG2013 48

x

y

z

The three vectors lie onthe same plane, namely, the x-y plane.

Theorem 2

• Let – A be an mn matrix– b be an m1 vector

• Let the columns of A be v1, v2,…, vn.

• Theorem: If v1, v2,…, vn, are linear independent, then Ax = b has at most one solution.

kshum ENGG2013 49

Proof (by contradiction)• Suppose that and are two different solutions to Ax=b,

i.e.,

• Therefore

• Move every term to the left

• But v1, v2,…, vn are linear independent by assumption. So, the onlychoice is

• This contradicts the fact that vector x and vector x’ are different.

kshum ENGG2013 50

Example

• and are linearly independent.

• has a unique solution for

any choice of b1 and b2.

kshum ENGG2013 51

In fact, x must equal b1,and y must equal b2/3in this example.

Example

• is solvable

by Theorem 1, because the bluevector lies on the planespanned by the two red vectors.

• The solution is unique

because and are linearly independent.

kshum ENGG2013 52

x

z

y

Summary

kshum ENGG2013 53

At most one solution At least one solution

Ax=bm equationsn variables

Every vector inis a linear combinationof the columns in A.

Columns of A are linearly independent

Uniqu

e so

lution

The columns of Acontain a lot of informationabout the nature of thesolutions.

A kind of mirror symmetry

kshum ENGG2013 54

If the columns of A are linear independent,

then I am pretty sure that there is one or no

solution to Ax=b,no matter what b is.

If any vector in can be written as a

linear combination of the column vectors in A,

then Ax=b must have one or more than one

solutions.

Basis

• A set of vector in which are simultaneously – linearly independent, and – spanning the whole space

is of particular importance, and is called a set of basis vectors.

(We will talk about basis in more detail later.)

kshum ENGG2013 55