Lecture 10: Unsymmetric Eigen Value Problem · 2018. 9. 23. · Lecture 10: Unsymmetric Eigen Value...

Copyright ©2008 by K. Pattipati 1

Lecture 10: Unsymmetric Eigen Value

Problem

Prof. Krishna R. Pattipati

Dept. of Electrical and Computer Engineering

University of Connecticut Contact: [email protected] (860) 486-2890

Fall 2008

October 29, 2008

ECE 6435Adv Numerical Methods in Sci Comp

mailto:[email protected]


Outline of Lecture 10

What is the Eigen value ?

Major Applications

Properties of Eigen values and Eigen vectors

Eigen value conditioning problem

Power method for finding the maximum Eigen value and its Eigen

vector

Inverse power method

• Smallest Eigen value and its Eigen vector

• Iterative improvement

QR method

• Hessenberg decomposition

• Shifted QR

• Implicit Q-theorem

• Francis double QR-step


What is the Eigen value problem?

• Computing the Eigen values and Eigen vectors of a matrix A

• Eigen values are also called modes, spectrum, characteristic

values or latent roots

Major applications

• Converting a system into a canonical form (e.g., SCF,

SOF, Diagonal form, Jordan form)

• Solution of Riccati equation by Potter’s method

Properties of Eigen values and Eigen vectors:

• Consider:

λ: Eigen value of A; x: Eigen vector of A

•

Preliminaries

x Ax bu

Ax x

0 ( ) 0 or ( )Ax I x A I x x N A I

1

2 1| | 0 ch. equation of ... 0n n

nA I A a a a


•

•

•

•

•

•

Properties of Eigen Values & Eigen Vectors

1

1 1

( ) ; | |nn

i n i

i i

tr A a A a

;

is called the right Eigen vector of

is called the left Eigen vector of = Eigen vector of

T T

i ii ii i

i

T

i

Ax x y A y

x A

y A A

1

1 1

1

0 why?

... ; ( ... )

0;

T

i jT

n

T

n

T T T T

i i i ii i ij j i i

x y i j

y

T AT I T T x x

y

y Ax y x y Ax y x

min max

For symmetric matrices, we have

/ ( )

/ ( ) is called the Rayleigh quotient

T T

T T

x Ax x x

x Ax x x

minFor PD and symmetric 0 min

iia

| | || || a pessimistic boundi

A


Gershgorin Circle Theorem


• Valid for arbitrary n x n matrices

• Eigen values lie in the union of circles, i.e.,

where

Proof:

Let λ be an Eigen value with Eigen vector,

But don’t know k a priori. So, take all possible circles

( ) , 1,...,i i

e

A D i n 1

{ : | | | | }n

i ii ij i

jj i

D C a a r

1 2( ... )T

nx x x x

1

1

1 1

( ) 1,...,

pick | | || || , i.e., | | max | |

| |Then, | | | | | |

| |

n

ii i ij j

jj i

k k jj n

n n

j

kk kj kj k

j jk

j k j k

Ax x a x a x i n

i k x x x x

xa a a r

x


Example:

1 2 3

10 2 3

1 0 2 ; (A)=(10.226, 0.3870+2.2216i, 0.3870 - 2.2216i)

1 -2 1

{ : | -10| 5}; { : | | 3}; { : | -1| 3};

A

D D D


x

3

0 1

x

x

3

D1

D2

D3

5

10


Eigen Value Conditioning Problem - 1


•

• Suppose A is perturbed A+dA=B

•

Example:

10

10

10 10 1

0 1 . . . 00 1 . . . 0

0 0 1. . 00 0 1. . 0,

0 . . . 0 10 . . . 0 1

0 . . . 0 0 10 . . . 0

10 | | 0

SCF for | | 10 | | 10

A B

n I A

I B

Q: how close ( ) and ( ) are?i i

A B

1 2Suppose A has Eigen values ... .

n


Analysis

• Consider where x is the left Eigen vector

and y is the right Eigen vector

•

• So, sensitivity ≈ inverse of inner product of left and right Eigen

vectors corresponding to the Eigen value

• Now, to formalize this notion

–

–

– Noting that and neglecting second order

terms, we obtain:

– So, smaller the inner product of left and right Eigen vectors,

larger is the sensitivity of Eigen values

, T T

Ax x y A y

2 2Assume || || || || 1 (normalized)x y

/ ( ) sensitivity 1/ | |T

d d S y x

if inner product is small trouble

Consider ( )( ) ( )A A x x d x xd d d Ax A x A x A x x x x xd d d d d d d d

T T

y A x y xd d

/ ( ) | |||| |||| || / | | || || / | |T T T T

y Ax y x A x y y x A y xd d d d



– So, condition number of the Eigen value problem is related to the

inverse of the inner product of left and right Eigen vectors

How are Eigen vectors affected• To first order, we have for Eigen value and Eigen vector

• Sensitivity of xk

depends on Si(λ) and EV separation

• Eigen vectors associated with repeated Eigen values are wobbly!

nearly to problems

1Sensitivity, ( )

| |

r

T

x y

Sy x

1

1

.

( )

(since 0 )( )

( )

k k k kk k

n

k i i k ki i i k k

i i k

T

Tki

ki iT

ik i i

Tk

ki

k iT

i ik i i

A x A x x x

x x x Ax x

y Axi k y x i k

y x

y Axx x

y x

d d d d

d d d

d

dd

k

kx



•

• Take any arbitrary u0. Then, since {x

i} span R

n,

• Let us consider the following iteration

•

•

• So, a good idea is to normalize uk

after each iteration

maxPower Method for finding ( )A

1 2

1 2

Suppose { } are simple, real and | | | | ... | |

Eigen vectors ... are independent

i n

nx x x

0

1

n

ii

i

u c x

1

1

1 11 1

21 1

[ ]

k k

kn

k i i

k i

i

u Au

cu c x x

c

1

1 1 11 1As , proportional to k

kk u c x x

1

1

Note that if | | 1 || || 0 as not good

if | | 1 || || as

ki

ki

u k

u k

Power Method for max min( ) ( )A or A


Normalized scheme

• But, a better way is to treat it as an overdetermined set of equations

1 2

1 21 2

1 1

1

1 1

1

1 1

1 1

1

1

...|| ||

and continue|| ||

as ,

2 successive iterates are linearly dependent

( ) can get

( )

k kk

n

k nn

k

k

k k k

k

k k

k

k

u c x c x c xu

uu Au u

u

k u u

u

u

1 1 11

can get ; 1T T

k k k k k ku u u u recall u u

Normalized Power Method


Power Method & Rayleigh Quotient

What happens in the symmetric case ?

Example

2 1

2

1 1

1

1

Since 0 , we have

1 O

Reyleigh quotient for symmetric matrices

T

i j

k

T

k k

T T

k k k k

x x j i

u u

u u u Au

11

22

0 1 2 4

2 -1 0.7071 has 3 with

-1 2 0.7071

0.7071and 1 with

0.7071

The power method gives a sequence:

1 0.8994 0.7809 , , ,

0 0.4472 0.6247

A x

x

u u u u

10

0.7328 0.7071, . . .,

0.6805 0.7071u u


Power algorithm for Real Eigen values:

Power Algorithm for Real Eigen values - 1

max

2

Given an x matrix and a tolerance parameter , the following algorithm

computes ( ).

Initialize arbitrary

0

0

|| ||

For 0, 1, 2, . . . DO

|| ||

/

old

n n A

A

u

k

s

c A

k

u

w u

u

1max

If | | stop: ,

else

1

end DO

T

new

new old new

old new

Aw

s u w

s s c s x w

k k

s s


Don’t expect convergence for at least iterations

Note: can be thought of as QR decomposition of uk

Can I get p biggest λi? Yes !!

What if λiis complex:

• Fact: complex Eigen values always occur in pairs

• So, start with

•

n

ˆ || ||k k k

u u u

0

2

1

Start with an x orthogonal matrix

0 and || || 1

For 1, 2, ... DO

SIMULTANEOUS ITERATION

(QR factorization)

end DO

T

i j

i

k k

k k k

n p Q q q

i j q

k

Z A Q

Q R Z

*

0 1 21 1...u c x c x

2

1 2

1 2 1 2

Assume and are rooots of 0

and

p q

p q

Power Algorithm for Real Eigen values - 2


• Then*

1 21 1 2 1

1 1 *

1 1 21 1 2 1

1 2 * 2

2 1 22 1 2 1

2 11 2 1 2

...

...

...

as , it is easy to prove that

( + ) 0 . . . (

k k

k

k k

k k

k k

k k

k k k

u c x c x

u c x c x A u

u c x c x A u

k

u u u

2

1

1 1

1 2

2 1 1 2 22 2

1)

3 successive iterates are linearly dependent

solve for , as follows:

unit norm vector || ||

; =|| ||

|| || || ||

k

k

k

k

k k k

k

k k k k k

p q

uu

u

uu A u u

u

u A u u u u

Power Method for Complex Eigen values - 1


Least-squares normal equations

1 2 2 1 12 2 2

1 2

1 1 2 22 2

then, from (1),

|| || || || || || 0

|| |||| || || ||

k k k k k k

k

k k k k k

u u u p u u qu

p uu u u u u

q

11 1 22

1 22 2

1 2

|| ||1 || || || ||

1

T T

kk k k k

k kT T

k k k k

p uu u u uu u

qu u u u



1 2 1 2

2 2 2

1

2 2 1

2 22

1

2 1 1 2

1 22 2 2

1

2

1 22 2

|| ||

1

|| ||

1

|| || || ||

1

|| || || ||

T T T

k k k k k k

kT

k k

TT

k k k k k

k T

k k

T T T

k k k k k k

k kT

k k

T

k k k

k k

u u u u u u

p u

u u

u u u u u

u

u u

u u u u u u

q u u

u u

u u u

u u

1 2 1

2

11

T

k k

T

k k

u u

u u



Power Algorithm for Complex Eigen Values - 1

Algorithm for finding complex Eigen values

• Given an n x n matrix A and tolerance parameter ε, the following

algorithm finds the complex Eigen values

0

1 0

11 2

2 1

22

1 2

0 1

0 2

1 2

0

any unit norm

|| ||

|| ||

0

0

For 0,1, 2, ... DO

|| ||

old

old

T

T

T

k

u

u Au

r u

c A

p

q

k

u Au

r u

r r r

a u u

b u u

c u u


2

2 0 12

2

0 2 1

1

2

1

( ) / (1 )

( ) / (1 )

If | | and | | :

Re( ) / 2

Im( ) 4 / 2

else

T

new

T

new

new old new old

new

new new

ol

p r u bu u a

q ru u cu a

p p c q q c

p

p q

p

0 1

1 2

1 2

1

end if

end DO

d new

old new

p

q q

u u

u u

r r

k k

Power Algorithm for Complex Eigen values - 2


2 2

11 2

1 2

1 2

1 2

1 1

- if 1; 1

- docomponent wise:

( ) (( ) ( ) )( ) ( ) ( )

( ) ( ) 2( ) ( )

where , , weregenerated from theusualalgorithm.

( )( ) ( )

2

k

k

k i k i k ik i k i k i

k i k i k i k i

k k k

k kk

k i i k k

u x a k

u u uu u u

u u u u

u u u

a au x a

a a

2

2 2( 1) 2

2 2

2

(1 )tofirst order

(1 )

- so,can doAitken -iteration onceevery threeiterations

k

kk

i ik

a

ax a x

a

d

• To improve convergence, use Aitken - iteration 2d

Aitken’s Acceleration


Inverse Power iteration

• Useful for finding the smallest Eigen value as well as for iterative

improvement

do LU on A

store L and U factors

• can be used for iterative improvement as follows:

1

1 1k k k kv A v Av v

1 1

2

1 2

1

1 1

1

Suppose and areapproximations to and .

isnormalized,so that || || 1

| | | | ... | |

- then ( )hasEigen value 0with Eigen vector .

- ( ) hasEigen value( ) big with Eigen vector .

- usepower method

n

i

i

x x

x x

A I x

A I x

1on ( )

Do LUon ( )

B A I

A I LU


Inverse Power Iteration Algorithm - 1 • Example again:

- the inverse power method gives the sequence

Inverse Power Iteration Algorithm:

1

1k kv A v

0 1 2 3 9

0.6 0.6727 0.6958 0.7034 0.7071, , , ,...,

0.8 0.7399 0.7182 0.7108 0.7071v v v v v v

1 1

1 1

2

Given andapproximate and for and respectively,

thealgorithmcomputesbetter approximation to and

For =0,1,2,...DO

|| ||

solve

solve

old

A x x

x

u x

s

k

uw

u

Lv w

Uu y


1 1

1

1 1

end DO

if :Stop / ,

else

1

end if

end DO

- when haveconverged / ( )

-method worksextremely well:

since( )

,

-shifting of spectrumis the

T

new

T T

new old

old new

s T T

k k k k k

ni

k iki i

k

s u w

s s u Au u u x u

s s

k k

u u Au u u

cu x

if then u x

keyidea of QR algorithmas well

-shifting of spectrumimprovesconvergence rate

Inverse Power Iteration Algorithm - 2


QR Method

A form of simultaneous iteration and much more

Key ideas of QR

• Reduce A to upper Hessenberg

- this can be done via Householder

- Perform QR decomposition

For k=1,2,…

- Note:

1 0 0 0

0 1

a similarity transformation

x x x x x x x x

x x x x x x x x

x x x x 0 x x x

x x x x 0 0 x x

TA H Q AQ

A H A

0 0

TQ AQ

1

via Givens transformation

~ upper Hessenberg if is!!

k k k

T

k k k k k k k

A Q R

A R Q Q A Q A

0 1( ) ( )and ( ) ( )i i i k i kA H A A


Example

• Illustration of

• So, the off-diagonal elements go from s to s3

• Subsequent iterations go to s9, s27, s81 …. because of a

shifting operation we will employ in a practical

scheme

almost diagonal and we can obtain the

Eigen values very fast!!

• Theorem: If is cyclic (i.e., n independent

Eigen vectors), then the sequence

1

T

k k k k k k k k kA Q R and A R Q Q A Q

2

2 3

3 2

1

0 0

(1 )

c s c s csA

s s c s

c s sRQ

s cs

Q R

1kA

1kA A

1

1 0 0... ....

T

k k k k k k k k k

T T T

k k k

A Q R A R Q Q A Q

Q Q Q AQ Q

Converges to upper Schur form

or upper quasi-triangular matrix


• Eigen values from Rii subblocks

• If Rii is 1x1, we get a simple Eigen value

• If Rii is 2x2, we get a complex Eigen value

Why do we transform to upper Hessenberg matrix first?

• QR decomposition on a full n x n matrix requires O(2n3/3)

operations

• If A is upper Hessenberg, QR decomposition takes only

O(4n2) operations

• If Q is accumulated, a further 4n2 operations

• More importantly, QR iteration maintains upper

Hessenberg structure

11 12 1

22 2

..

0 ..

..

n

n

nn

R R R

R R

R

where each Rii is a 1x1 or a 2x2 matrix

Upper Schur Form


Hessenberg Decomposition - 1

• Want an upper Hessenberg matrix H0 from A such that

• Note that if first r components of u are zero, then

• Suppose r = 3, n=5 and

0 0 0 0 0

0

and

~ product of Householder matrices [ 2 /( )]

x x x x x y y y x y y y

x x x x y y y y y y * *

x x x x 0 y y y 0 * * *

x x x x 0 y y y 0 0 * *

T T

T T

H Q AQ Q Q I

Q I uu u u

W1 W2

Done! n-2 steps

in general.

11 12

21 22

11

0where

0

and

where x block of

r TI

W

A AWAW

A A

A r r A

x x x | x x

x x x | x x

- - - + -

0 0 x | x x

0 0 x | x x

A


• 3x3 subblock does not change with Housholder

• In general, at step k, pivot on element ak+1,k

• pivot column

4

4 5 6 53 63

2 1/2

3 4 43 3

4

pick 0 0 0 ..

pick , , to zeroout , , .

5 ( )

T

n

n

i i i

i

u u u

u u u a a etc

u a i u a sign a

2 1/2

1, 1,

1

2,

,

0

.

( )( )

.

n

k k k k ik

i k

k k

n k

a sign a au

a

a



• Given a matrix A, the following algorithm computes

an upper Hessenberg decomposition.

- For k=1, 2,…,n-2 Do

Determine Householder

2 1/2

1,

11,

,

1, 1,

2,

( )( )

0.

.

0

[ ]

( )

where.

n k

n

k k ik

i kk k

n k

n k

T

n k n kn k n k

k k k k

k k

n k

nk

sign a aa

a

u uI

a sign a s

au

a



• Finding QR decomposition of H:

- need H=QR

- but, actually need Hnew = RQ in the QR-algorithm

- use Givens since H is almost upper .

For k = 1, 2,…, n-1 DO

2 1/ 2

1

1

11 12

21 22

( ) | |

( , )

end DO

n

ik k

i k

k k n k

n k

k k

n k n k n k

s a and u s

W Diag I

A AA W AW

A A

x x x x x

x x x x x

x x x

x x x

QR Decomposition of H - 1


1, 2 2

1,

1,

1,

, 1

Determine cos and sin

; , , sgn( )0

sgn( ) affectsonly two rows; 0

For 1,...,

k k k k

kkk k k kkkkk kk k k

k kk k

kk kk k k

kj kjk k

k j k k

c s

hc s x hhc s r h h k

hs c r r

h h r h

j k n DO

h hc s

h s c

1,

end DO

end DO

k jh

, 1 , 1

For =1, 2,..., -1

For =1, 2,..., +1

end DO

k k

ik i k ik i k

k k

k n

i k

c sh h h h

s c

QR Decomposition of H - 2


• This algorithm requires O(4n2) operations

• Example:

form of upper Hessenberg is maintained throughout

subsequent iterations

1 2

2 1 1 2

3 1 2 .6 .8 0 1 0 0

4 2 3 ; .8 .6 0 ; 0 .9996 .0249

0 .01 1 0 0 1 0 .0249 .9996

4.76 2.5442 5.4653

.32 .1856 2.1786

0 .0263 1.054

T T

H J J

J J HJ J

QR Decomposition of H - Example


• Obtain via Householder Transformation

- For k=1,2,…

Obtain via Givens Transformation

If at some point Ak+1 is a subdiagonal element ap+1,p is

“small”, i.e., |ap+1,p| ≤ [|app|+|ap+1,p+1|] then

set |ap+1,p| ~0 and break Ak+1 into subblocks as

A11 ~ upper Hessenberg

A22 ~ upper Hessenberg

- If p=n-1, then

- If p=n-2, solve 2x2 subblock A22 for

- For other p, work on A22 and then on A11

• Convergence rate:

where

• Typically the above algorithm converges very slowly

Crude Version of QR

1 0 0A H Q AQ

k k kA Q R

1

T

k k k k k kA R Q Q A Q

11 12

1

220k

A AA

A

nn na

1n nand

1| / |kn n

1 2| | | | ..... | |n


• To speed up, use a shifted version of QR

• Consider a shifted version of Ak: Ak-skI

• recall that

Choice of sk

• Compute eigen values of lower 2x2 subblock

Acceleration Methods

1

1, 1

, 1 1

1

( ) ( )

~ similar to

convergenceof 0 with rate| | / | |

so,if then 0 with rate | | / | |

if , Nogood!should beexpected from the

i k k i k

k k k k

T

k k k k k k k k

k k

p p p k p k

k k

n k n n n k n k

n n

A s I A s

A s I Q R

A R Q s I Q A Q A

a s s

s a s s

conditioning

argumentsearlier

1, 1 1, *

, 1 ,

* *

1, 1 1, 1 , 1 1,

, (complex)

;

n n n n

k k

n n n n

k k n n nn k k n n nn n n n n

a as s

a a

s s a a s s a a a a


• Now, consider the following double shift QR

• What is happening here?

• What is

Double Shifts for Complex Eigen Values - 1

1

*

1 1 1 1 1

2 1 1 1

;

k k k k

k k k k

k k k k k k

k k k k

A s I Q R

A R Q s I

A s I Q R s s

A R Q s I

2 1 1( ) ( ) unitary

(real)

H H

k k k k k k k

T

k

A Q Q A Q Q Q

Q A Q

1 1 ?k k k kM Q Q R R QR

1 1

2

1 1 1

1

1

( )

( ) ( ) ( )

( ) ( )

( ) ( )

k k k k

k k k k k k k k k k k k k k

k k k k k k k k k k

k k k k k k

Q A s I R

Q A R s A s I Q R s Q R s A s I

A s I Q R s Q R s A s I

A A s I s A s I


M is a real matrix

Since M is real, is real

so, have :

Ak+2 can be obtained as a QR decomposition of real

matrix:

• But, computation of M requires O(n3) operations

• Q: can we obtain Q directly? Yes

• A: via FRANCIS QR-DOUBLE STEP

1

2 *

1

( )( )

( )

k k k k

k k k k k k

A s I A s I

A s s A s s I

1k kQ Q Q

2 *

1

2

( )k k k k k k

T

k k

A s s A s s I M

and A Q A Q

2 *

1

2

-form ( )

-form

-form

k k k k k k

T

k k

M A s s A s s I

M QR

A Q A Q

Double Shifts for Complex Eigen Values - 2


• Let us get greedy and ask: Q: can we go from Ak →Ak+2 in O(n2)

operations?

• A: Yes, via Francis QR-double step

Ideas of Francis double step

- Compute the first column of M=Me1

- Find a Householder matrix W0 W0Me1=e1

- Apply Householder matrices W1W2….Wn-2 to W0 AkW0 to reduce

it to an upper Hessenberg form.

• Essentially, we have used Q1TAkQ1, where

• A theorem called implicit Q-theorem (see below) tells us that Q=Q1

• Fact: Me1 has only 3 non-zero components

Me1 = [x y z 0….0]T

where

Francis Double Step

1 2 3 2 1 0....n nQ W W W WW

2 *

11 12 21 1 11

*

1

21 11 22 1

21 32

( ) ;

( : )

( ( ))

; where areelementsof

k k k k

k k

k k

ij k

x a a a s s a s s

note s s

y a a a s s

z a a a A


• When n=6, W0AkW0 will change rows and columns 1, 2 &3 only

• Implicit Q-Theorem:

- suppose Q = (q1 q2 … qn) and V=(v1 v2 … vn) are orthogonal

matrices QTAQ = H and VTAV = G are upper Hessenberg (e.g., H

and G are obtained via Householder or Givens transformation)

- if v1=q1, then vi= ±qi and |hi,i-1|=|gi,i-1| for i=1,2,…,n

- formally, AQ=QH, AV=VG and v1=q1 then

Q=VD, V=D-1GD=DGD; |di|=1 i

If AQ=QH and the first column of Q is given, then Q and H are

essentially determined.

0 0

x x x x x x

x x x x x x

x x x x x x

x x x x x x

0 0 0 0 0 0

0 0 0 0 0 0

kW A W

Implict Q- Theorem - 1


• Proof: Consider

- If we insist on positive hk+1,k, then qi are uniquely determined for i 2 .

- Both Givens and Householder provide the same Q, since q1=e1 for both.

11 12 1

21 22 2

32 31 2 1 2

, 1

1 11 1 21 2 11 1 1 21 2 1 11 1

21 1 11 1 2 21

2 12 1

...

...

0 ...[ ... ] [ ... ]

. . . .

0 ..

;

| | || || so that is determined to within +or -sign

n

n

nn n

n n nn

T

h h h

h h h

h hAq Aq Aq q q q

h h

Aq h q h q h q Aq h q Aq h q

h Aq h q h

Aq h q

22 2 32 3

12 1 2 22 2 2

32 2 12 1 22 2 2

; ;

| | || ||

T T

h q h q

h q Aq h q Aq

h Aq h q h q

Implict Q- Theorem - 2


• Coming back to QR Algorithm

- the matrix we have is

- Francis QR step avoids forming explicitly

- Direct formation of M requires O(n3) operations and not practical

- We lose advantage of the upper Hessenberg structure

• We use implicit Q-theorem as follows:

- Step 1: where Ak is an upper Hessenberg

matrix,

2 * *( )k k k k k kM A s s A s s I

2 * *( )k k k k k kM A s s A s s I

*

1 1( )( )k k k kMe A s I A s I e

11 12 1

21 12 2

32 3

, 1

* *

11 11 11 12 21

*

21 21 11 22 21

1 32 21

...

...

0 ...

0

( )( )

( ) ( )

00

. ..

0 00

k n

k n

n

n n nn

k k k

k k

a s a a

a a s a

a a

a a

xa s a s a s a a

ya a a s a s a

zMe a a

Use of Implicit Q-Theorem - 1


- Step 2: Find

• Key: Similarity transformation with W0 affects rows 1, 2 and 3 and

columns 1, 2 and 3 only.

2 2 2 1/ 2

0 0 0

* sgn( )( )

0

0, ,

0 . 0

. . .

0 0 0

T

x x x x y z

y y

z zuuW W W I u

0 0

x x x xx x x ... x

x x x x x x x x x x x x xx x x ... x

x x x 0 x x x 0 x x x x x x x 0=x x ... x

x x x x x x 00 x x x x x. . . . .

0 I 0 I .....0 . 0 I0 .. 0 x x

000 x

kW A W



x x x x x x

x x x x x x

x x x x x x

= x x x x x x

0 0 0 x x x

. . . .

0 0 0 0 x x

two more nonzero elements than needed

(3,1), (4,1), (4,2)

- Step 3: Compute Householder matrices W1, W2,…, Wn-1 to convert

W0AkW0 such that Wn-2Wn-3…W1W0W1..Wn-2 is upper Hessenberg

adds two more elements in each column

x x x x x x x x x xx x x x x x x x x x x x

x x x x x x x x x xx x x x x x x

0 x x x x 0 x x x xx x x x x x

. x x x x . 0 x ..x x x x x x

. x x x x . 0 x ..0 0 x x x x

0 0 .. . x ... . . . .

. . . . . .0 0 0 0 x x

0 0 0 x x 0 0 0 x x

x x x x x x

x x x x x x x x x x x

0 x x x x x 0 x x x x x

etc... .. x x x . 0 x x x x

. .. x x x . . . x x x

. . . . . . . 0 x x

0 0 0 0 0 x 0 0 0 0 0 x



• The mechanization of Francis QR-double step is as follows:

Francis QR-double step:

For k=1, 2, …, n-2

3

2 2 2

k k k n k

n n n

W Diag I W I

W Diag I W

2

100

01

x x x

x x x

x x x

I

W

Note: Wke1=e1W0 and Q have the same first

column Q1e1=Qe1 where Q1=W0W1…Wn-2

Q1=Q by the implicit Q-theorem

2

11 12 21 11

21 11 22

21 32

1;

( )

( )

mm nn

mm nn mn nm

m n s a a

t a a a a

x a a a a s t

y a a a s

z a a

Formalizing Francis QR-double Step - 1


3

2 2

2 2 2

*

if - 2 then Determine 0

0

; [ ]

* else Determine

0

; [

k k

k k k k k n k

n n

n n n

x

k n W W y

z

A W AW W Diag I W I

xW W

y

A W AW W Diag

2 2

2, 1 3, 1

4, 1

]

end if

;

if -3

end if

end DO

n n

k k k k

k k

I W

x a y a

k n

z a

Formalizing Francis QR-double Step - 2


The overall QR-algorithm employing Francis double-step is as follows:

Overall QR-algorithm

Form A0 = Q0TAQ0 upper Hessenberg via Householder transformation

Form the orthogonal matrix Q0 ← Wn-2 … W1

Repeat

set to zero all subdiagonal elements satisfying

Find the largest nonnegative q and smallest p

where A33 isupper quasi-triangular and A22 is unreduced upper

Hessenberg

If q = n, then

upper triangularize all 2 by 2 diagonal blocks in A that have real eigen

values. Accumulate orthogonal transformations if necessary and quit .

1, 1, 1| | [| | ||]i i ii i ia a a

11 12 13

22 23

33

- -

0 note: can have 0

0 0

p n q p q

A A A p

A A A n p q q p

A q

Overall QR- Algorithm - 1


Apply a Francis QR double – step to A22:

else

A22 ← Q1TA22Q1

if Q and upper schur form of A are desired then

Q ← Diag(Ip Q1 Iq)

A12 ← A12Q1

A23 ← Q1TA23

go to Repeat

• Total Computational effort O(15n3) operations.

• If only Eigen values are desired, the computational load is O(8n3)

• Balance A if A has widely differing Eigen values

Overall QR- Algorithm - 2


Summary

Properties of Eigen values and Eigen vectors


Power method for finding the maximum Eigen value and

its Eigen vector

Inverse power method

• Smallest Eigen value and its Eigen vector

• Iterative improvement

QR method

• Hessenberg decomposition

• Shifted QR

• Implicit Q-theorem

• Francis double QR-step

Lecture 10: Unsymmetric Eigen Value Problem · 2018. 9. 23. · Lecture 10: Unsymmetric Eigen Value...

Documents

Transcript of Lecture 10: Unsymmetric Eigen Value Problem · 2018. 9. 23. · Lecture 10: Unsymmetric Eigen Value...