Lesson 16 (S&H, Section 14.6)The Spectral Theorem and Applications

Math 20

October 26, 2007

Announcements

I Welcome parents!

I Problem Set 6 on the website. Due October 31.

I OH: Mondays 1–2, Tuesdays 3–4, Wednesdays 1–3 (SC 323)

I Prob. Sess.: Sundays 6–7 (SC B-10), Tuesdays 1–2 (SC 116)

Outline

Hatsumon

Concept ReviewEigenbusinessDiagonalization

The Spectral TheoremThe split caseThe symmetric caseIterations

ApplicationsBack to FibonacciMarkov chains

A famous math problem

“A certain man had one pairof rabbits together in acertain enclosed place, andone wishes to know howmany are created from thepair in one year when it is thenature of them in a singlemonth to bear another pair,and in the second monththose born to bear also.Because the abovewritten pairin the first month bore, youwill double it; there will betwo pairs in one month.”

Leonardo of Pisa(1170s or 1180s–1250)

a/k/a Fibonacci

Diagram of rabbits

f (0) = 1

f (1) = 1

f (2) = 2

f (3) = 3

f (4) = 5

f (5) = 8

Diagram of rabbits

f (0) = 1

f (1) = 1

f (2) = 2

f (3) = 3

f (4) = 5

f (5) = 8

Diagram of rabbits

f (0) = 1

f (1) = 1

f (2) = 2

f (3) = 3

f (4) = 5

f (5) = 8

Diagram of rabbits

f (0) = 1

f (1) = 1

f (2) = 2

f (3) = 3

f (4) = 5

f (5) = 8

Diagram of rabbits

f (0) = 1

f (1) = 1

f (2) = 2

f (3) = 3

f (4) = 5

f (5) = 8

Diagram of rabbits

f (0) = 1

f (1) = 1

f (2) = 2

f (3) = 3

f (4) = 5

f (5) = 8

An equation for the rabbits

Let f (k) be the number of pairs of rabbits in month k . Each newmonth we have

I The same rabbits as last month

I Every pair of rabbits at least one month old producing a newpair of rabbits

Sof (k) = f (k − 1) + f (k − 2)

An equation for the rabbits

Let f (k) be the number of pairs of rabbits in month k . Each newmonth we have

I The same rabbits as last month

I Every pair of rabbits at least one month old producing a newpair of rabbits

Sof (k) = f (k − 1) + f (k − 2)

Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007

Page1of19

Some fibonacci numbers

k f (k)

0 11 12 23 34 55 86 137 218 349 55

10 8911 14412 233

QuestionCan we find an explicit formula for f (k)?

Outline

Hatsumon

Concept ReviewEigenbusinessDiagonalization

The Spectral TheoremThe split caseThe symmetric caseIterations

ApplicationsBack to FibonacciMarkov chains

Concept Review

DefinitionLet A be an n × n matrix. The number λ is called an eigenvalueof A if there exists a nonzero vector x ∈ Rn such that

Ax = λx. (1)

Every nonzero vector satisfying (??) is called an eigenvector of Aassociated with the eigenvalue λ.

Diagonalization Procedure

I Find the eigenvalues and eigenvectors.

I Arrange the eigenvectors in a matrix P and the correspondingeigenvalues in a diagonal matrix D.

I If you have “enough” eigenvectors (that is, one for eachcolumn of A), the original matrix is diagonalizable and equalto PDP−1.

I Pitfalls:I Repeated eigenvaluesI Nonreal eigenvalues

Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007

Page2of19

Outline

Hatsumon

Concept ReviewEigenbusinessDiagonalization

The Spectral TheoremThe split caseThe symmetric caseIterations

ApplicationsBack to FibonacciMarkov chains

QuestionUnder what conditions on A would you be able to guarantee thatA is diagonalizable?

Theorem (Baby Spectral Theorem)

Suppose An×n has n distinct real eigenvalues. Then A isdiagonalizable.

Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007

Page3of19

Theorem (Spectral Theorem for Symmetric Matrices)

Suppose An×n is symmetric, that is, A′ = A. Then A isdiagonalizable. In fact, the eigenvectors can be chosen to bepairwise orthogonal with length one, which means that P−1 = P′.Thus a symmetric matrix can be diagonalized as

A = PDP′,

Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007

Page4of19

Powers of diagonalizable matrices

Remember if A is diagonalizable then

Ak = (PDP−1)k = (PDP−1)(PDP−1) · · · (PDP−1)︸ ︷︷ ︸k

= PD(P−1P)D(P−1P) · · ·D(P−1P)DP−1 = PDkP−1

Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007

Page5of19

Another way to look at it

I If v is an eigenvector corresponding to eigenvalue λ, then

Akv =

λkv

I If v1, . . . vn are eigenvectors with eigenvalues λ1, . . . , λn, then

Ak(c1v1 + · · ·+ cnvn) = c1λk1v1 + · · ·+ cnλ

knvn

I If A is diagonalizable, there are n linearly independenteigenvectors, so any v can be written as a linear combinationof them.

Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007

Page6of19

Another way to look at it

I If v is an eigenvector corresponding to eigenvalue λ, then

Akv = λkv

I If v1, . . . vn are eigenvectors with eigenvalues λ1, . . . , λn, then

Ak(c1v1 + · · ·+ cnvn) = c1λk1v1 + · · ·+ cnλ

knvn

I If A is diagonalizable, there are n linearly independenteigenvectors, so any v can be written as a linear combinationof them.

Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007

Page7of19

Another way to look at it

I If v is an eigenvector corresponding to eigenvalue λ, then

Akv = λkv

I If v1, . . . vn are eigenvectors with eigenvalues λ1, . . . , λn, then

Ak(c1v1 + · · ·+ cnvn)

= c1λk1v1 + · · ·+ cnλ

knvn

I If A is diagonalizable, there are n linearly independenteigenvectors, so any v can be written as a linear combinationof them.

Another way to look at it

I If v is an eigenvector corresponding to eigenvalue λ, then

Akv = λkv

I If v1, . . . vn are eigenvectors with eigenvalues λ1, . . . , λn, then

Ak(c1v1 + · · ·+ cnvn) = c1λk1v1 + · · ·+ cnλ

knvn

I If A is diagonalizable, there are n linearly independenteigenvectors, so any v can be written as a linear combinationof them.

Another way to look at it

I If v is an eigenvector corresponding to eigenvalue λ, then

Akv = λkv

I If v1, . . . vn are eigenvectors with eigenvalues λ1, . . . , λn, then

Ak(c1v1 + · · ·+ cnvn) = c1λk1v1 + · · ·+ cnλ

knvn

I If A is diagonalizable, there are n linearly independenteigenvectors, so any v can be written as a linear combinationof them.

Outline

Hatsumon

Concept ReviewEigenbusinessDiagonalization

The Spectral TheoremThe split caseThe symmetric caseIterations

ApplicationsBack to FibonacciMarkov chains

Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007

Page9of19

Setting up the Fibonacci sequence

Recall the Fibonacci sequence defined by

f (k + 2) = f (k) + f (k + 1), f (0) = 1, f (1) = 1

Let’s let g(k) = f (k + 1). Then

g(k + 1) = f (k + 2) = f (k) + f (k + 1) = f (k) + g(k).

So if y(k) =

(f (k)g(k)

), we have

y(k + 1) =

(f (k + 1)g(k + 1)

)=

(g(k)

f (k) + g(k)

)=

(0 11 1

)y(k)

So if A is this matrix, then

y(k) = Aky(0).

Setting up the Fibonacci sequence

Recall the Fibonacci sequence defined by

f (k + 2) = f (k) + f (k + 1), f (0) = 1, f (1) = 1

Let’s let g(k) = f (k + 1). Then

g(k + 1) = f (k + 2) = f (k) + f (k + 1) = f (k) + g(k).

So if y(k) =

(f (k)g(k)

), we have

y(k + 1) =

(f (k + 1)g(k + 1)

)=

(g(k)

f (k) + g(k)

)=

(0 11 1

)y(k)

So if A is this matrix, then

y(k) = Aky(0).

Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007

Page11of19

Setting up the Fibonacci sequence

Recall the Fibonacci sequence defined by

f (k + 2) = f (k) + f (k + 1), f (0) = 1, f (1) = 1

Let’s let g(k) = f (k + 1). Then

g(k + 1) = f (k + 2) = f (k) + f (k + 1) = f (k) + g(k).

So if y(k) =

(f (k)g(k)

), we have

y(k + 1) =

(f (k + 1)g(k + 1)

)=

(g(k)

f (k) + g(k)

)=

(0 11 1

)y(k)

So if A is this matrix, then

y(k) = Aky(0).

Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007

Page12of19

Setting up the Fibonacci sequence

Recall the Fibonacci sequence defined by

f (k + 2) = f (k) + f (k + 1), f (0) = 1, f (1) = 1

Let’s let g(k) = f (k + 1). Then

g(k + 1) = f (k + 2) = f (k) + f (k + 1) = f (k) + g(k).

So if y(k) =

(f (k)g(k)

), we have

y(k + 1) =

(f (k + 1)g(k + 1)

)=

(g(k)

f (k) + g(k)

)=

(0 11 1

)y(k)

So if A is this matrix, then

y(k) =

Aky(0).

Setting up the Fibonacci sequence

Recall the Fibonacci sequence defined by

f (k + 2) = f (k) + f (k + 1), f (0) = 1, f (1) = 1

Let’s let g(k) = f (k + 1). Then

g(k + 1) = f (k + 2) = f (k) + f (k + 1) = f (k) + g(k).

So if y(k) =

(f (k)g(k)

), we have

y(k + 1) =

(f (k + 1)g(k + 1)

)=

(g(k)

f (k) + g(k)

)=

(0 11 1

)y(k)

So if A is this matrix, then

y(k) = Aky(0).

Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007

Page13of19

Diagonalize

The eigenvalues of A =

(0 11 1

)are found by solving

0 =

∣∣∣∣−λ 11 1− λ

∣∣∣∣ = (−λ)(1− λ)− 1

= λ2 − λ− 1

The roots are

ϕ =1 +√

5

2ϕ̄ =

1−√

5

2

Notice thatϕ+ ϕ̄ = 1, ϕϕ̄ = −1

(These facts make later calculations simpler.)

Diagonalize

The eigenvalues of A =

(0 11 1

)are found by solving

0 =

∣∣∣∣−λ 11 1− λ

∣∣∣∣ = (−λ)(1− λ)− 1

= λ2 − λ− 1

The roots are

ϕ =1 +√

5

2ϕ̄ =

1−√

5

2

Notice thatϕ+ ϕ̄ = 1, ϕϕ̄ = −1

(These facts make later calculations simpler.)

Diagonalize

The eigenvalues of A =

(0 11 1

)are found by solving

0 =

∣∣∣∣−λ 11 1− λ

∣∣∣∣ = (−λ)(1− λ)− 1

= λ2 − λ− 1

The roots are

ϕ =1 +√

5

2ϕ̄ =

1−√

5

2

Notice thatϕ+ ϕ̄ = 1, ϕϕ̄ = −1

(These facts make later calculations simpler.)

Eigenvectors

We row reduce to find the eigenvectors:

A− ϕI =

(−ϕ 11 1− ϕ

)=

(− ϕ 11 ϕ̄

)←−−ϕ̄

+

(−ϕ 10 0

)

So

(1ϕ

)is an eigenvector for A corresponding to the eigenvalue ϕ.

Similarly,

(1ϕ̄

)is an eigenvector for A corresponding to the

eigenvalue ϕ̄. So now we know that

y(k) = c1ϕk

(1ϕ

)+ c2ϕ̄

k

(1ϕ̄

)

Eigenvectors

We row reduce to find the eigenvectors:

A− ϕI =

(−ϕ 11 1− ϕ

)=

(− ϕ 11 ϕ̄

)←−−ϕ̄

+

(−ϕ 10 0

)

So

(1ϕ

)is an eigenvector for A corresponding to the eigenvalue ϕ.

Similarly,

(1ϕ̄

)is an eigenvector for A corresponding to the

eigenvalue ϕ̄.

So now we know that

y(k) = c1ϕk

(1ϕ

)+ c2ϕ̄

k

(1ϕ̄

)

Eigenvectors

We row reduce to find the eigenvectors:

A− ϕI =

(−ϕ 11 1− ϕ

)=

(− ϕ 11 ϕ̄

)←−−ϕ̄

+

(−ϕ 10 0

)

So

(1ϕ

)is an eigenvector for A corresponding to the eigenvalue ϕ.

Similarly,

(1ϕ̄

)is an eigenvector for A corresponding to the

eigenvalue ϕ̄. So now we know that

y(k) = c1ϕk

(1ϕ

)+ c2ϕ̄

k

(1ϕ̄

)

What are the constants?

To find c1 and c2, we solve(11

)= c1

(1ϕ

)+ c2

(1ϕ̄

)=

(1 1ϕ ϕ̄

)(c1

c2

)=⇒

(c1

c2

)=

(1 1ϕ ϕ̄

)−1(11

)=

1

ϕ̄− ϕ

(ϕ̄ −1−ϕ 1

)=

1

ϕ̄− ϕ

(ϕ̄− 1ϕ+ 1

)(11

)=

1√5

(ϕ−ϕ̄

)

Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007

Page18of19

Finally

Putting this all together we have

y(k) =ϕ√

5ϕk

(1ϕ

)− ϕ̄√

5ϕ̄k

(1ϕ̄

)(

f (k)g(k)

)=

1√5

(ϕk+1 − ϕ̄k+1

ϕk+2 − ϕ̄k+2

)

So

f (k) =1√5

(1 +√

5

2

)k+1

−

(1−√

5

2

)k+1

Finally

Putting this all together we have

y(k) =ϕ√

5ϕk

(1ϕ

)− ϕ̄√

5ϕ̄k

(1ϕ̄

)(

f (k)g(k)

)=

1√5

(ϕk+1 − ϕ̄k+1

ϕk+2 − ϕ̄k+2

)So

f (k) =1√5

(1 +√

5

2

)k+1

−

(1−√

5

2

)k+1

Markov Chains

I Recall the setup: T is a transition matrix giving theprobabilities of switching from any state to any of the otherstates.

I We seek a steady-state vector, i.e., a probability vector usuch that Tu = u.

I This is nothing more than an eigenvector of eigenvalue 1!

Markov Chains

I Recall the setup: T is a transition matrix giving theprobabilities of switching from any state to any of the otherstates.

I We seek a steady-state vector, i.e., a probability vector usuch that Tu = u.

I This is nothing more than an eigenvector of eigenvalue 1!

Markov Chains

I Recall the setup: T is a transition matrix giving theprobabilities of switching from any state to any of the otherstates.

I We seek a steady-state vector, i.e., a probability vector usuch that Tu = u.

I This is nothing more than an eigenvector of eigenvalue 1!

TheoremIf T is a regular doubly-stochastic matrix, then

I 1 is an eigenvalue for T

I all other eigenvalues of T have absolute value less than 1.

Let u be an eigenvector of eigenvalue 1, scaled so it’s a probabilityvector. Let v2, . . . , vn be eigenvectors corresponding to the othereigenvalues λ2, . . . , λn. Then for any initial state x(0), we have

x(k) = Akx(0) = Ak (c1u + c2λ2v2 + · · ·+ cnλnvn)

= c1u + c2λk2v2 + · · ·+ cnλ

knvn

Sox(k)→ c1u

Since each x(k) is a probability vector, c1 = 1. Hence

x(k)→ c1u