Chapter 4 – Control Structures Part 1cmliu/Courses/LA/chap01.pdf · 2010. 9. 26. · 19 Theorem...

1

Linear Algebra

Chi-Min Liu

Sep. 11, 2010Department of Computer Science and Information [email protected]

2

Preface

Time: EC122,

Wednesday 10:10 – 12:30.

Friday 3:30 – 4:30 (hw & test)

Textbooks

Score:

3 Exam (70%)

HW + MATLAB + Test (30%)

Linear Algebra with Applications

8th Edition

by

Steven J. Leon

3

Contents

Chapter 1: Matrices and Systems of Equations

Chapter 2: Determinants

Chapter 3: Vector Spaces

Chapter 4: Linear Transformations

Chapter 5: Orthogonality

Chapter 6: Eigenvalues

Chapter 7: Numerical Linear Algebra

Chapter 8: Iterative Mathods

Chapter 9: Canonical Forms

http://wps.prenhall.com/am_leon_linearalg_7/0,9761,2869031-,00.htmlhttp://wps.prenhall.com/am_leon_linearalg_7/0,9761,2869122-,00.htmlhttp://wps.prenhall.com/am_leon_linearalg_7/0,9761,2869177-,00.htmlhttp://wps.prenhall.com/am_leon_linearalg_7/0,9761,2869286-,00.htmlhttp://wps.prenhall.com/am_leon_linearalg_7/0,9761,2869341-,00.htmlhttp://wps.prenhall.com/am_leon_linearalg_7/0,9761,2869341-,00.htmlhttp://wps.prenhall.com/am_leon_linearalg_7/0,9761,2869468-,00.htmlhttp://wps.prenhall.com/am_leon_linearalg_7/0,9761,2869468-,00.htmlhttp://wps.prenhall.com/am_leon_linearalg_7/0,9761,2869613-,00.htmlhttp://wps.prenhall.com/am_leon_linearalg_7/0,9761,2869740-,00.htmlhttp://wps.prenhall.com/am_leon_linearalg_7/0,9761,2869740-,00.htmlhttp://wps.prenhall.com/am_leon_linearalg_7/0,9761,2869744-,00.html

Chapter 1

Matrices and Systems of Equations

Systems of Linear Equations

Row Echelon Form

Matrix Arithmetic

Matrix Algebra

Elementary Matrices

Partitioned Matrices

4

5

1. Systems of Linear Equations

A linear system of m equations and n unknowns

6


Example

7


8


Equivalent Systems

Definition

Two systems of equations involving the same variables are

said to be equivalent if they have the same solution set.

9


Three Operations Types

Obtain an equivalent system easier to solve.

Definition-- Strict Triangular Form

A system is said to be in strict triangular form if in the kth

equation the coefficients of the first k-1 variables are all

zero and the coefficient of xk is nonzero (k=1, …, n)

10


11


12

2. Row Echelon Form

13

2. Row Echelon Form

14

2. Row Echelon Form

Definition–A linear system is said to be

overdetermined if there are more

equations than unknowns.

•A system of m linear equations in

n unknowns is said to be

underdetermined if there are fewer

equations than unknown.

http://www.math.drexel.edu/~pg/fb//java/la_applets/GaussElim/index.html

http://www.math.drexel.edu/~pg/fb/java/la_applets/GaussElim/index.html

15

2. Row Echelon Form

16

2. Row Echelon Form

17

2. Row Echelon Form

The process of using elementary row

operations to transform a matrix into reduced

row echelon form is called Gauss-Jordan

reduction.

2. Row Echelon Form

18

Using Gauss-Jordan reduction to solve the system

Sol: 022

03

03

4321

4321

4321

xxxx

xxxx

xxxx

19

Theorem 1.2.1

An m×n homogeneous system of linear equations has a nontrivial solution if n > m (underdetermined).

Pf:

A homogeneous system is always consistent. There are at most m lead variables. Since there are n variables altogether and n>m, there must be some free variables and they can be assigned arbitrary values. So, we can get a nontrivial solution if n>m.

20

3. Matrix Arithmetic

)( ijaA

21


22


23


24

4. Matrix Algebra

25

Proof of Rule 4

is a m×n matrix, and and

are both n×r matrices. Let and

and

But

so that and hence A(B + C) = AB + AC.

)( ijaA )( ijbB )( ijcC

)( CBAD ACABE

n

k

kjkjikij cbad1

)(

n

k

kjik

n

k

kjikij cabae11

n

k

kjik

n

k

kjik

n

k

kjkjik cabacba111

)(

ijij ed

26

Proof of Rule 3

A be a m×n matrix, and B an n×r matrix and C an r×s matrix. Let and

The (i, j) entry of DC is

and the (i, j) entry of AE is

ABD BCE

r

l

ljklkj

n

k

klikil cbebad11

and

r

l

lj

n

k

klik

r

l

ljil cbacd1 11

n

k

lj

r

l

klik

n

k

kjik cbaea1 11

27

Example 1

12

01 and ,

23

12 ,

43

21CBA

1116

56

21

14

43

21)(BCA

1116

56

12

01

116

54)( CAB

28

Example 1

Thus

ThereforeA(B + C) = AB + AC

155

71

411

25

116

54

155

71

31

13

43

21)(

)(1116

56)(

ACAB

CBA

CABBCA

29

Example 1

Since

it follows that

(AB)C = DC = AE = A(BC)

n

k

r

l

ljklik

r

l

n

k

ljkliklj

r

l

n

k

klik cbacbacba1 11 11 1

30

Notation

If A is an n×n matrix and k is a positive integer, then

timesk

AAAAk

31

Example 2

If

then

In general

,11

11

A

22

22

11

11

11

112A

44

44

22

22

11

1123 AAAAAA

11

11

22

22nn

nn

nA

32

4. Matrix Algebra

33

Example

is a 3×3 identity matrix

100

010

001

)(

810

362

143

810

362

143

100

010

001

AIA

)(

810

362

143

100

010

001

810

362

143

AAI

34

Matrix Inversion

A real number a is said to have a multiplicative inverse if there exists a number b such that ab = 1.

Any nonzero number a has a multiplicative inverse b=1/a.

35

Definition

An n×n matrix A is said to be nonsingular orinvertible if there exists a matrix B such that AB = BA = I. The matrix B is said to be a multiplicative inverse of A.

• If B and C are both multiplicative inverse of A (i.e.,

BA = AB = I and CA = AC = I), then B = BI = B(AC)

= (BA)C = IC = C

• A matrix can have at most one multiplicative inverse.

• Notation: The inverse of A is denoted by A-1.

36

Example 3

The matrices and are inverse of each other, since

and

13

42

51

103

52

101

10

01

13

42

51

103

52

101

10

01

13

42

51

103

52

101

37

Example 4

The 3×3 matrices and

are inverse, since

and

100

410

321

100

410

521

100

010

001

100

410

521

100

410

321

100

010

001

100

410

321

100

410

521

38

Example 5

The matrix has no inverse.

Sol: Let

then

00

01A

2221

12111

bb

bbBA

10

01

0

0

00

01

21

11

2221

1211

b

b

bb

bbBA

39

4. Matrix Algebra

40

Note

If A1, A2,…, Ak are all nonsingular n×n matrices, then the product A1A2…Ak is nonsingular and

(A1A2…Ak)-1 = Ak

-1… A2-1A1

-1

41

4. Matrix Algebra

42

Example 6

Let

145

112

201

,

142

533

121

BA

9815

142334

5610

145

112

201

142

533

121

AB

9145

8236

153410

151

432

231

112

410

521TT AB

43

Definition

An n×n matrix A is said to be symmetric if AT = A (i.e., aij = aji)

354

513

432

A

354

513

432TA

• Example

44

5. Elementary Matrices

45

Example 1

100

001

010

1E

333132

232122

131112

333231

232221

131211

1

333231

131211

232221

333231

232221

131211

1

100

001

010

100

001

010

aaa

aaa

aaa

aaa

aaa

aaa

AE

aaa

aaa

aaa

aaa

aaa

aaa

AE

46

Example 2

300

010

001

2E

333231

232221

131211

333231

232221

131211

2

333231

232221

131211

333231

232221

131211

2

3

3

3

300

010

001

33300

010

001

aaa

aaa

aaa

aaa

aaa

aaa

AE

aaa

aaa

aaa

aaa

aaa

aaa

AE

47

Example 3

100

010

301

3E

33313231

23212221

13111211

3

333231

232221

331332123111

3

3

3

3:operationColumn

333:operation Row

aaaa

aaaa

aaaa

AE

aaa

aaa

aaaaaa

AE

48

Note

Suppose that E is an n×n elementary matrix. If A is an n×r matrix,premultiplying A by E has the effect of performing that same row operation on A. If B is an m×n matrix, postmultiplyingB by E is equivalent to performing that same column operation on B.

49

Theorem 1.5.1

If E is an elementary matrix, then E is nonsingular and E-1 is an elementary matrix of the same type.

• Proof

– Type I: interchange of two rows

E1E1 = I E1-1 = E1

50

Note

B is row equivalent to A if B can be obtained from A by a finite number of row operations.

Two augmented matrices (A | b) and (B| c) are row equivalent iff Ax = b and Bx = c are equivalent systems.

If A is row equivalent to B, B is row equivalent to A.

If A is row equivalent to B, and B is row equivalent to C, then A is row equivalent to C.

51


Type II: multiplying the ith row of I by a nonzero scalar

rowth

1

1

1

1

2 i

0

0

E

52


rowth

1

1

1

1

1

1

2 i

0

0

E

53


Type III: adding m times the ith row to the jth row

rowth

rowth

1000

10

10

1

3

j

i

m

0

E

54


rowth

rowth

1000

10

10

1

1

3

j

i

m

0

E

55


56

Theorem 1.5.2

Equivalent conditions for Nonsingularity

Let A be an n×n matrix. The following are

equivalent:

(a) A is nonsingular.

(b) Ax = 0 has only the trivial solution 0.

(c) A is row equivalent to I.

• Pf: (a) (b)

If A is nonsingular (i.e., A-1 exists), then for Ax = 0

00xxxx 111 )ˆ(ˆ)(ˆˆ AAAAAI

57


(b) (c)

If Ax = 0 has only the trivial solution 0

Ax = 0 Ux = 0 (using elementary rowoperation) where U is in row echelonform and U = Ek . . . E1A

From Theorem 1.2.1.

In U, the number of nonzero rows mustbe the same as unknowns (m = n)

58


Thus,

U must be a strictly triangular matrix with diagonal elements all equal to 1.

∴ I will be the reduced row echelon form of A

∴ A is row equivalent to I

59


(c) (a)

If A is row equivalent to I, there exist elementary matrices E1, E2, …, Ek such that

A = EkEk-1…E1I = EkEk-1…E1∵ E1, E2, …, Ek are all invertible

∴ A-1 = (EkEk-1…E1)-1 = E1

-1E2-1… Ek-1

-1 Ek-1

∴ A is nonsingular (invertible)

60


(1) If A is nonsingular, and x is any solution to Ax=b, then we can multiply both

sides of the equation by A-1 and conclude that x will be the only solution.

(2) If Ax=b has a unique solution x, then we claim A cannot be singular. If A

were singular, then the equation AX=0 would have a solution z/=0, But this

would imply that y=x+z is a second solution to AX=b since Ay=A(x+z)=Ax+Az

= b+0=b

61

Corollary 1.5.3

The system of n linear equations in n unknown Ax = b has a unique solution iff A is nonsingular.

• Pf: () If A is nonsingular, and is any solution of

Ax = b, then

∴ is the unique solution.

x̂

bx ˆA

bx11 )ˆ( AAA

bx1

ˆ A

62


() Suppose that Ax = b has a unique solution

and A is singular

∴ Ax = 0 has a solution z≠0 (i.e., Az = 0)

∴ Let , clear

∴ y is also the solution to Ax = b

∴ contradiction

∴ A must be nonsingular

x̂),ˆ( bx A

zxy ˆ xy ˆ

00bzxzxy AAAA ˆ)ˆ(

63


If A is nonsingular, A is row equivalent to I, so there exist elementary matrices E1, E2, …, Ek such that

Ek Ek-1 … E1 A = I(Ek Ek-1 … E1 A)A

-1 = (I)A-1

Ek Ek-1 … E1 I = A-1

64


The same series of elementary row operations that transform a nonsingular matrix A into I will transform I into A-1. That is, the reduced row echelon form of the augmented matrix (A | I) will be (I | A-1).

65


66


67


If A is nonsingular, A is row equivalent to I, so there exist elementary matrices E1, E2, …, Ek such that

Ek Ek-1 … E1 A = I(Ek Ek-1 … E1 A)A

-1 = (I)A-1

Ek Ek-1 … E1 I = A-1

68


The same series of elementary row operations that transform a nonsingular matrix A into I will transform I into A-1. That is, the reduced row echelon form of the augmented matrix (A | I) will be (I | A-1).

69

Example 4

Compute A-1 if

Sol:

322

021

341

A

1 100

010

001

322

021

341-AI

61

21

61

41

41

41

21

21

21

1A

70

Example 5

Solve the system:

Sol: Ax = b x = A-1b

8322

122

1234

321

21

321

xxx

xx

xxx

3

84

4

8

12

12

61

21

61

41

41

41

21

21

21

1bx A

71

Diagonal and Triangular Matrices

An n×n matrix A is said to be upper triangular if aij = 0 for i > j, eg.,

An n×n matrix A is said to be lower triangular if aij = 0 for i < j. e.g.,

500

120

123

341

006

001

72

Note

A matrix is said to be triangular if it is either upper triangular or lower triangular

73


An n×n matrix A is said to be diagonal if aij = 0 whenever i ≠ j. e.g.,

Note: A diagonal matrix is both upper triangular and lower triangular.

100

030

001

74

Triangular Factorization

If an n×n matrix A can be reduced to strict upper triangular form using only row operation Ⅲ, then it is possible to represent the reduction process in terms of a matrix factorization.

75

Example 6

Ull

l

A

800

130

2423

590

130

2422

914

251

242

3231

21

21

132

01

001

1

01

001

21

3231

21

ll

lL

ALU

914

251

242

800

130

242

132

01

001

21

76

Note

L is unit lower triangular.

The factorization of the matrix A into a product of a unit lower triangular matrixL times a strictly upper triangular matrixU is often referred to as an LU factorization.

77


Why LU factorization works?

Since E3 E2 E1 A = U, where

A = E1-1 E2

-1 E3-1U

E1-1 E2

-1 E3-1 =

L

130

010

001

102

010

001

100

01

001

21

,

102

010

001

2

E

130

010

001

3E,

100

01

001

21

1

E

78

6. Partitioned Matrices

A matrix is composed of a number of submatrices.

Computation Merits

Analysis Merits

79


If A is an m×n matrix and B is n×r which has been partitioned into columns [b1, b2, …, br], then the block multiplication of Atimes B is given by

AB = A [b1, b2, …, br] = [Ab1, Ab2, . . ., Abr]

80


For example

212

131A

1

5 ,

1

15 ,

2

6321 bbb AAA

1

5

1

15

2

6) , ,( hence 321 bbbA

• In particular,

[a1, a2, …, an] = A = AI = [Ae1, Ae2, …, Aen]

81


For example,

111

323 and

71

43

52

BA

494

5105

191

3

2

1

B

B

B

a

a

a

4 9 4

5105

19 1

ˆ

ˆ

ˆ

3

2

1

B

B

B

AB

a

a

a

82


Block Multiplication

83


84


85


86

6. Partitioned MatricesMultiplying Partitioned Matrices

Partitioned matrices can be multiplied as though the submatrices were scalars :

is the partitioned matrix whose ijth “entry” is the

matrix

provided that all the products AikBkj are defined.

pmpjpp

mj

mj

npnn

ipii

p

p

BBBB

BBBB

BBBB

AAA

AAA

AAA

AAA

AB

21

222221

111211

21

21

22221

11211

p

k

kjikpjipjiji BABABABA1

2211

87

Example 1

Let

Sol: (1) If

then

2123

1113

1121

1111

and

2233

1122

1111

2221

1211

BB

BBBA

2233

1122

1111

2221

1211

AA

AAA

12101518

76910

5468

2123

1113

1121

1111

2233

1122

1111

AB

88

Example 1

(2) If

then

2233

1122

1111

2221

1211

AA

AAA

12101518

76910

5468

2123

1113

1121

1111

2233

1122

1111

AB

89

Example 2

Let A be an n×n matrix of the form

where A11 is a k×k matrix (k < n). Then A is

nonsingular iff A11 and A22 are nonsingular.

Sol: () If A11 and A22 are nonsingular, then

Thus A is nonsingular.

22

11

AO

OAA

IIO

OI

AO

OA

AO

OA

I IO

OI

AO

OA

AO

OA

kn

k

-

-

kn

k

-

-

1

22

1

11

22

11

22

11

1

22

1

11 and

90

Example 2

() If A is nonsingular, let B = A-1 and

Since BA = I = AB, i.e.,

Thus,

B11A11 = Ik = A11B11B22A22 = In-k = A22B22

And hence A11 and A22 are nonsingular, with

inverse B11 and B22.

2221

1211

BB

BBB

22222122

12111111

22221121

22121111

2221

1211

22

11

22

11

2221

1211

BABA

BABA

IO

OI

ABAB

ABAB

BB

BB

AO

OA

IO

OI

AO

OA

BB

BB

n-k

k

n-k

k

91

Outer Product Expansions

Given two vectors x and y in Rn, the scalar product or the inner product is defined as the matrix product xTy, which is the product of a row vector (a 1×nmatrix) times a column vector (an n×1 matrix) and results in a 1×1 matrix or simply a scalar

nn

n

n

T yxyxyx

y

y

y

xxx , , , 22112

1

21

yx

92


The outer product is defined as the

matrix product xyT, which is the product

of an n×1 matrix times an 1×n matrix and

results in an n×n matrix:

Each of the rows is a multiple of yT

Each of the column vectors is a multiple of x

nnnn

n

n

n

n

T

yxyxyx

yxyxyx

yxyxyx

yyy

x

x

x

21

22212

12111

21

2

1

, , , xy

93


For example

2

5

3

and

3

1

4

yx

6159

253

82012

253

3

1

4T

xy

94


Let X be an m×n matrix and Y a k×nmatrix, the outer product expansion of Xand Y is a matrix product XYT. XYT can be calculated by partitioning X into columns and YT into rows and then perform the block multiplication:

TnnTT

T

n

T

T

n

TXY yxyxyx

y

y

y

xxx , , , 22112

1

21

95

Example 3

Given

1

4

2

3

2

1

and

2

4

1

1

2

3

YX

142

2

4

1

321

1

2

3

142

321

2

4

1

1

2

3

TXY

284

4168

142

321

642

963

96

Conclusion

Systems of Linear Equations

Row Echelon Form

Matrix Arithmetic

Matrix Algebra

Elementary Matrices

Partitioned Matrices

Chapter 4 – Control Structures Part 1cmliu/Courses/LA/chap01.pdf · 2010. 9. 26. · 19 Theorem...

Documents

Transcript of Chapter 4 – Control Structures Part 1cmliu/Courses/LA/chap01.pdf · 2010. 9. 26. · 19 Theorem...