QR Algorithm - ETH Z
Transcript of QR Algorithm - ETH Z
![Page 1: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/1.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
QR Algorithm
Katja Grasic
May 15, 2008
Katja Grasic QR Algorithm
![Page 2: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/2.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Introduction
Definitions
The Power Method
QR algorithm without shifts
Hessenberg form
QR algorithm with shifts
Implicit shifted QR algorithm
Katja Grasic QR Algorithm
![Page 3: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/3.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Introduction
I QR algorithm most widely used method for computingeigenvalues.
I The name QR is derived from the use of the letter Q todenote orthogonal matrices and the letter R to denote righttriangular matrix.
Katja Grasic QR Algorithm
![Page 4: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/4.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Introduction
I QR algorithm most widely used method for computingeigenvalues.
I The name QR is derived from the use of the letter Q todenote orthogonal matrices and the letter R to denote righttriangular matrix.
Katja Grasic QR Algorithm
![Page 5: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/5.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Triangular matrix
Lower triangular matrix:l1,1 0l2,1 l2,2
l3,1 l3,2. . .
......
. . .. . .
ln,1 ln,2 . . . ln,n−1 ln,n
Katja Grasic QR Algorithm
![Page 6: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/6.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Triangular matrix
Upper triangular matrix:u1,1 u1,2 u1,3 . . . u1,n
u2,2 u2,3 . . . u2,n
. . .. . .
.... . . un−1,n
0 un,n
.
Katja Grasic QR Algorithm
![Page 7: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/7.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Unitary matrix
I Satisfying condition:
TTT = TTT = In
I T−1 = TT
Katja Grasic QR Algorithm
![Page 8: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/8.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Unitary matrix
I Satisfying condition:
TTT = TTT = In
I T−1 = TT
Katja Grasic QR Algorithm
![Page 9: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/9.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Shur decomposition
The Schur decomposition reads as follows: if A is an n × n squarematrix with complex entries, then A can be expressed as
A = QTQ∗
where Q is a unitary matrix, Q∗ is the conjugate transpose of Q,and T is an upper triangular matrix called the Schur form. Thediagonal entries of T are exactly the eigenvalues of A.
Katja Grasic QR Algorithm
![Page 10: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/10.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Similar matrix
Two n × n matrices A and B are called similar if there existsinvertible n × n matrix P, so that:
P−1AP = B
Similar matrices share many properties:
I rank
I determinant
I trace
I eigenvalues (though the eigenvectors will in general bedifferent)
I characteristic polynomial
Katja Grasic QR Algorithm
![Page 11: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/11.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Similar matrix
Two n × n matrices A and B are called similar if there existsinvertible n × n matrix P, so that:
P−1AP = B
Similar matrices share many properties:
I rank
I determinant
I trace
I eigenvalues (though the eigenvectors will in general bedifferent)
I characteristic polynomial
Katja Grasic QR Algorithm
![Page 12: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/12.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Similar matrix
Two n × n matrices A and B are called similar if there existsinvertible n × n matrix P, so that:
P−1AP = B
Similar matrices share many properties:
I rank
I determinant
I trace
I eigenvalues (though the eigenvectors will in general bedifferent)
I characteristic polynomial
Katja Grasic QR Algorithm
![Page 13: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/13.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Similar matrix
Two n × n matrices A and B are called similar if there existsinvertible n × n matrix P, so that:
P−1AP = B
Similar matrices share many properties:
I rank
I determinant
I trace
I eigenvalues (though the eigenvectors will in general bedifferent)
I characteristic polynomial
Katja Grasic QR Algorithm
![Page 14: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/14.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Similar matrix
Two n × n matrices A and B are called similar if there existsinvertible n × n matrix P, so that:
P−1AP = B
Similar matrices share many properties:
I rank
I determinant
I trace
I eigenvalues (though the eigenvectors will in general bedifferent)
I characteristic polynomial
Katja Grasic QR Algorithm
![Page 15: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/15.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
The Power Method
I Let A be a matrix with a complete set of eigenvalues andeigenvectors pairs (λi , qi )
I Assume the eigenvalues can be ordered:|λ1| > |λ2| > . . . > |λn|
I Then for any vector u, Aku = γ1λk1q1 +
∑i γiλ
ki qi
I As k →∞, the term containing λ1 will dominate and Akuapproaches a multiple of the dominant eigenvector q1.
Katja Grasic QR Algorithm
![Page 16: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/16.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
The Power Method
I Let A be a matrix with a complete set of eigenvalues andeigenvectors pairs (λi , qi )
I Assume the eigenvalues can be ordered:|λ1| > |λ2| > . . . > |λn|
I Then for any vector u, Aku = γ1λk1q1 +
∑i γiλ
ki qi
I As k →∞, the term containing λ1 will dominate and Akuapproaches a multiple of the dominant eigenvector q1.
Katja Grasic QR Algorithm
![Page 17: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/17.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
The Power Method
I Let A be a matrix with a complete set of eigenvalues andeigenvectors pairs (λi , qi )
I Assume the eigenvalues can be ordered:|λ1| > |λ2| > . . . > |λn|
I Then for any vector u, Aku = γ1λk1q1 +
∑i γiλ
ki qi
I As k →∞, the term containing λ1 will dominate and Akuapproaches a multiple of the dominant eigenvector q1.
Katja Grasic QR Algorithm
![Page 18: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/18.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
The Power Method
I Let A be a matrix with a complete set of eigenvalues andeigenvectors pairs (λi , qi )
I Assume the eigenvalues can be ordered:|λ1| > |λ2| > . . . > |λn|
I Then for any vector u, Aku = γ1λk1q1 +
∑i γiλ
ki qi
I As k →∞, the term containing λ1 will dominate and Akuapproaches a multiple of the dominant eigenvector q1.
Katja Grasic QR Algorithm
![Page 19: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/19.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
The Power Iteration
Choose v (0) such that∥∥v (0)
∥∥ = 1, then for k = 1, 2, . . . we do thefollowing steps:
I w=Avk−1
I v(k) = w‖w‖
I λk = (v (k))TAv (k).
Katja Grasic QR Algorithm
![Page 20: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/20.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
The Power Iteration
Choose v (0) such that∥∥v (0)
∥∥ = 1, then for k = 1, 2, . . . we do thefollowing steps:
I w=Avk−1
I v(k) = w‖w‖
I λk = (v (k))TAv (k).
Katja Grasic QR Algorithm
![Page 21: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/21.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
The Power Iteration
Choose v (0) such that∥∥v (0)
∥∥ = 1, then for k = 1, 2, . . . we do thefollowing steps:
I w=Avk−1
I v(k) = w‖w‖
I λk = (v (k))TAv (k).
Katja Grasic QR Algorithm
![Page 22: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/22.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
QR decomposition
A QR decomposition of a real square matrix A is a decompositionof A as
A = QR
I Q is an orthogonal matrix (meaning that QTQ = I )
I R is an upper triangular matrix
I If A nonsingular, then this factorization is unique if we requirethat the diagonal elements of R are positive.
Katja Grasic QR Algorithm
![Page 23: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/23.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
QR decomposition
A QR decomposition of a real square matrix A is a decompositionof A as
A = QR
I Q is an orthogonal matrix (meaning that QTQ = I )
I R is an upper triangular matrix
I If A nonsingular, then this factorization is unique if we requirethat the diagonal elements of R are positive.
Katja Grasic QR Algorithm
![Page 24: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/24.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
QR algorithm
I A(0) = A
I for k = 1, 2, 3, . . .:
I Q(k)R(k) = A(k−1)
I A(k) = R(k)Q(k)
I With some assumptions, A(k) converge to a Shur form of A(diagonal if A is symmetric)
Katja Grasic QR Algorithm
![Page 25: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/25.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
QR algorithm
I A(0) = A
I for k = 1, 2, 3, . . .:
I Q(k)R(k) = A(k−1)
I A(k) = R(k)Q(k)
I With some assumptions, A(k) converge to a Shur form of A(diagonal if A is symmetric)
Katja Grasic QR Algorithm
![Page 26: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/26.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
QR algorithm
I A(0) = A
I for k = 1, 2, 3, . . .:
I Q(k)R(k) = A(k−1)
I A(k) = R(k)Q(k)
I With some assumptions, A(k) converge to a Shur form of A(diagonal if A is symmetric)
Katja Grasic QR Algorithm
![Page 27: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/27.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
QR algorithm
I A(0) = A
I for k = 1, 2, 3, . . .:
I Q(k)R(k) = A(k−1)
I A(k) = R(k)Q(k)
I With some assumptions, A(k) converge to a Shur form of A(diagonal if A is symmetric)
Katja Grasic QR Algorithm
![Page 28: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/28.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
QR algorithm
I A(0) = A
I for k = 1, 2, 3, . . .:
I Q(k)R(k) = A(k−1)
I A(k) = R(k)Q(k)
I With some assumptions, A(k) converge to a Shur form of A(diagonal if A is symmetric)
Katja Grasic QR Algorithm
![Page 29: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/29.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
QR algorithm
I To understand the QR algorithm, first consider a simpleralgorithm
I Simultaneous Iteration is power iteration applied to severalvectors
I Start with linearly independent v(0,)1 , v
(0,)2 , . . . , v
(0,)n
I We know from power iteration that Akv(0)1 converges to q1
I With some assumptions, the space⟨Akv
(0)1 ,Akv
(0)2 , . . . ,Akv
(0)n
⟩should converge to q1, . . . , qn
I Notation: Define initial matrix V (0) and V (k) at step k :
V (0) =[v
(0)1
∣∣∣v (0)2
∣∣∣. . . ∣∣∣v (0)n
],V (k) = A(k)V (0) =
[v
(k)1
∣∣∣v (k)2
∣∣∣. . . ∣∣∣v (k)n
]
Katja Grasic QR Algorithm
![Page 30: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/30.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
QR algorithm
I To understand the QR algorithm, first consider a simpleralgorithm
I Simultaneous Iteration is power iteration applied to severalvectors
I Start with linearly independent v(0,)1 , v
(0,)2 , . . . , v
(0,)n
I We know from power iteration that Akv(0)1 converges to q1
I With some assumptions, the space⟨Akv
(0)1 ,Akv
(0)2 , . . . ,Akv
(0)n
⟩should converge to q1, . . . , qn
I Notation: Define initial matrix V (0) and V (k) at step k :
V (0) =[v
(0)1
∣∣∣v (0)2
∣∣∣. . . ∣∣∣v (0)n
],V (k) = A(k)V (0) =
[v
(k)1
∣∣∣v (k)2
∣∣∣. . . ∣∣∣v (k)n
]
Katja Grasic QR Algorithm
![Page 31: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/31.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
QR algorithm
I To understand the QR algorithm, first consider a simpleralgorithm
I Simultaneous Iteration is power iteration applied to severalvectors
I Start with linearly independent v(0,)1 , v
(0,)2 , . . . , v
(0,)n
I We know from power iteration that Akv(0)1 converges to q1
I With some assumptions, the space⟨Akv
(0)1 ,Akv
(0)2 , . . . ,Akv
(0)n
⟩should converge to q1, . . . , qn
I Notation: Define initial matrix V (0) and V (k) at step k :
V (0) =[v
(0)1
∣∣∣v (0)2
∣∣∣. . . ∣∣∣v (0)n
],V (k) = A(k)V (0) =
[v
(k)1
∣∣∣v (k)2
∣∣∣. . . ∣∣∣v (k)n
]
Katja Grasic QR Algorithm
![Page 32: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/32.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
QR algorithm
I To understand the QR algorithm, first consider a simpleralgorithm
I Simultaneous Iteration is power iteration applied to severalvectors
I Start with linearly independent v(0,)1 , v
(0,)2 , . . . , v
(0,)n
I We know from power iteration that Akv(0)1 converges to q1
I With some assumptions, the space⟨Akv
(0)1 ,Akv
(0)2 , . . . ,Akv
(0)n
⟩should converge to q1, . . . , qn
I Notation: Define initial matrix V (0) and V (k) at step k :
V (0) =[v
(0)1
∣∣∣v (0)2
∣∣∣. . . ∣∣∣v (0)n
],V (k) = A(k)V (0) =
[v
(k)1
∣∣∣v (k)2
∣∣∣. . . ∣∣∣v (k)n
]
Katja Grasic QR Algorithm
![Page 33: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/33.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
QR algorithm
I To understand the QR algorithm, first consider a simpleralgorithm
I Simultaneous Iteration is power iteration applied to severalvectors
I Start with linearly independent v(0,)1 , v
(0,)2 , . . . , v
(0,)n
I We know from power iteration that Akv(0)1 converges to q1
I With some assumptions, the space⟨Akv
(0)1 ,Akv
(0)2 , . . . ,Akv
(0)n
⟩should converge to q1, . . . , qn
I Notation: Define initial matrix V (0) and V (k) at step k :
V (0) =[v
(0)1
∣∣∣v (0)2
∣∣∣. . . ∣∣∣v (0)n
],V (k) = A(k)V (0) =
[v
(k)1
∣∣∣v (k)2
∣∣∣. . . ∣∣∣v (k)n
]
Katja Grasic QR Algorithm
![Page 34: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/34.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
QR algorithm
I To understand the QR algorithm, first consider a simpleralgorithm
I Simultaneous Iteration is power iteration applied to severalvectors
I Start with linearly independent v(0,)1 , v
(0,)2 , . . . , v
(0,)n
I We know from power iteration that Akv(0)1 converges to q1
I With some assumptions, the space⟨Akv
(0)1 ,Akv
(0)2 , . . . ,Akv
(0)n
⟩should converge to q1, . . . , qn
I Notation: Define initial matrix V (0) and V (k) at step k :
V (0) =[v
(0)1
∣∣∣v (0)2
∣∣∣. . . ∣∣∣v (0)n
],V (k) = A(k)V (0) =
[v
(k)1
∣∣∣v (k)2
∣∣∣. . . ∣∣∣v (k)n
]Katja Grasic QR Algorithm
![Page 35: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/35.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Unnormalized Simultaneous Iteration
I Define well-behaved basis for column space of V (k) byˆQ(k) ˆR(k) = V (k)
Make the assumptions:
I The leading n + 1 eigenvalues are distinct
I All principal leading principal submatrices of QTV (0)arenonsingular, where columns of Q are q1, . . . , qn
We then have that the columns of ˆQ(k) converge to eigenvectorsof A
Katja Grasic QR Algorithm
![Page 36: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/36.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Unnormalized Simultaneous Iteration
I Define well-behaved basis for column space of V (k) byˆQ(k) ˆR(k) = V (k)
Make the assumptions:
I The leading n + 1 eigenvalues are distinct
I All principal leading principal submatrices of QTV (0)arenonsingular, where columns of Q are q1, . . . , qn
We then have that the columns of ˆQ(k) converge to eigenvectorsof A
Katja Grasic QR Algorithm
![Page 37: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/37.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Simultaneous Iteration
The matrices V (k) = A(k)V (0) are highly ill-conditioned.Orthonormalize at each step rather than at the end:
I Pick Q(0)
I for k = 1, 2, . . .:
I Z = AQ(k−1)
I Q(k)R(k) = Z
I The QR algorithm is equivalent to simultaneous iteration withˆQ(0) = I
Katja Grasic QR Algorithm
![Page 38: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/38.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Simultaneous Iteration
The matrices V (k) = A(k)V (0) are highly ill-conditioned.Orthonormalize at each step rather than at the end:
I Pick Q(0)
I for k = 1, 2, . . .:
I Z = AQ(k−1)
I Q(k)R(k) = Z
I The QR algorithm is equivalent to simultaneous iteration withˆQ(0) = I
Katja Grasic QR Algorithm
![Page 39: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/39.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Simultaneous Iteration
The matrices V (k) = A(k)V (0) are highly ill-conditioned.Orthonormalize at each step rather than at the end:
I Pick Q(0)
I for k = 1, 2, . . .:
I Z = AQ(k−1)
I Q(k)R(k) = Z
I The QR algorithm is equivalent to simultaneous iteration withˆQ(0) = I
Katja Grasic QR Algorithm
![Page 40: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/40.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Simultaneous Iteration
The matrices V (k) = A(k)V (0) are highly ill-conditioned.Orthonormalize at each step rather than at the end:
I Pick Q(0)
I for k = 1, 2, . . .:
I Z = AQ(k−1)
I Q(k)R(k) = Z
I The QR algorithm is equivalent to simultaneous iteration withˆQ(0) = I
Katja Grasic QR Algorithm
![Page 41: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/41.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Simultaneous Iteration
The matrices V (k) = A(k)V (0) are highly ill-conditioned.Orthonormalize at each step rather than at the end:
I Pick Q(0)
I for k = 1, 2, . . .:
I Z = AQ(k−1)
I Q(k)R(k) = Z
I The QR algorithm is equivalent to simultaneous iteration withˆQ(0) = I
Katja Grasic QR Algorithm
![Page 42: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/42.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Hessenberg form
I It is now to focus on the cost of the QR algorithm.
I This is prohibitely expensive as usually a large number ofiterations is required to attain convergence.
I The cure to this problem is the transformation of the matrixto upper-Hessenberg form H. When this is done thecost/iteration reduces to O(n2).
Katja Grasic QR Algorithm
![Page 43: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/43.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Hessenberg form
I It is now to focus on the cost of the QR algorithm.
I This is prohibitely expensive as usually a large number ofiterations is required to attain convergence.
I The cure to this problem is the transformation of the matrixto upper-Hessenberg form H. When this is done thecost/iteration reduces to O(n2).
Katja Grasic QR Algorithm
![Page 44: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/44.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Hessenberg form
I It is now to focus on the cost of the QR algorithm.
I This is prohibitely expensive as usually a large number ofiterations is required to attain convergence.
I The cure to this problem is the transformation of the matrixto upper-Hessenberg form H. When this is done thecost/iteration reduces to O(n2).
Katja Grasic QR Algorithm
![Page 45: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/45.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Householder reflections
We define a Householder reflection as:
H = I − uu∗,
where ‖u‖ =√
2.
Katja Grasic QR Algorithm
![Page 46: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/46.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Hessenberg form via Householder reflection
A =
(α11 a12
a21 A22
),
choose the householder reflection H1 such that H1a21 = ν1e1 then
H1AH1 =
(α11 a12H1
νe1 H1A22H1
),
thus we have annihilated the elements below the first column via asimilarity transformation. We can repeat this process for all stepsto obtain an upper Hessenberg form.
Katja Grasic QR Algorithm
![Page 47: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/47.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
QR algorithm with shifts
The explicit QR algorithm with shifts can be written as follows forany matrix A. Let A0 = A, and let k = 1, 2, 3... then given shiftsκi
I Ak − κk I = QkRk
I Ak+1 = RkQk + κk I
Notice that the iterates satisfy
Ak+1 = RkQk + κk I = Q∗k (Ak − κk I )Qk + κk I = Q∗kAQk ,
and therefore are related through a similarity transformation.
Katja Grasic QR Algorithm
![Page 48: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/48.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
QR algorithm with shifts
The explicit QR algorithm with shifts can be written as follows forany matrix A. Let A0 = A, and let k = 1, 2, 3... then given shiftsκi
I Ak − κk I = QkRk
I Ak+1 = RkQk + κk I
Notice that the iterates satisfy
Ak+1 = RkQk + κk I = Q∗k (Ak − κk I )Qk + κk I = Q∗kAQk ,
and therefore are related through a similarity transformation.
Katja Grasic QR Algorithm
![Page 49: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/49.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Outline of the implicit shifted QR algorithm
I We determine the first column c ofC = H2 − 2Re(κ)H + |κ|2 I .
I Now let Q0 be a Householder transformation such thatQ∗0c = σe1.
I Next let H1 = Q∗0AQ0.
I Use Householder transformations to reduce H1 into upperHessenberg form and call it H. Call Q1 the accumulatedtransformations.
I Set Q = Q0Q1
Katja Grasic QR Algorithm
![Page 50: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/50.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Outline of the implicit shifted QR algorithm
I We determine the first column c ofC = H2 − 2Re(κ)H + |κ|2 I .
I Now let Q0 be a Householder transformation such thatQ∗0c = σe1.
I Next let H1 = Q∗0AQ0.
I Use Householder transformations to reduce H1 into upperHessenberg form and call it H. Call Q1 the accumulatedtransformations.
I Set Q = Q0Q1
Katja Grasic QR Algorithm
![Page 51: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/51.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Outline of the implicit shifted QR algorithm
I We determine the first column c ofC = H2 − 2Re(κ)H + |κ|2 I .
I Now let Q0 be a Householder transformation such thatQ∗0c = σe1.
I Next let H1 = Q∗0AQ0.
I Use Householder transformations to reduce H1 into upperHessenberg form and call it H. Call Q1 the accumulatedtransformations.
I Set Q = Q0Q1
Katja Grasic QR Algorithm
![Page 52: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/52.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Outline of the implicit shifted QR algorithm
I We determine the first column c ofC = H2 − 2Re(κ)H + |κ|2 I .
I Now let Q0 be a Householder transformation such thatQ∗0c = σe1.
I Next let H1 = Q∗0AQ0.
I Use Householder transformations to reduce H1 into upperHessenberg form and call it H. Call Q1 the accumulatedtransformations.
I Set Q = Q0Q1
Katja Grasic QR Algorithm
![Page 53: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/53.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Outline of the implicit shifted QR algorithm
I To show that this algorithm works we need to show that Qand R are the matrices that one would have obtained usingthe explicit QR algorithm.
I We also need to show that c and the reduction of H1 to upperHessenberg can be computed rapidly
Katja Grasic QR Algorithm
![Page 54: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/54.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Outline of the implicit shifted QR algorithm
I To show that this algorithm works we need to show that Qand R are the matrices that one would have obtained usingthe explicit QR algorithm.
I We also need to show that c and the reduction of H1 to upperHessenberg can be computed rapidly
Katja Grasic QR Algorithm
![Page 55: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/55.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Implicit Q theorem
The implicit Q theorem states that for a matrix A of order n, letH = Q∗AQ be a reduction of A to Hessenberg form. If theelements in the lower diagonal od H are non-zero then Q and Hare uniquely determined by the first or last column of Q.
Katja Grasic QR Algorithm
![Page 56: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/56.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Comparison of explicit QR and implicit QR algorithm
I The computation of c takes only O(n) perations because onlythe first three components of c are nonzero
I The reduction of H1 to upper Hessenberg can be done inO(n2)
I Total cost for implicit QR for k iterations is 2kn3 additionsplus 2kn3 multiplications
I Total cost for explicit QR for k iterations is 8kn3 additionsplus 12kn3 multiplications
I Further notice that one step of implicit = two steps of explicit
Katja Grasic QR Algorithm
![Page 57: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/57.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Comparison of explicit QR and implicit QR algorithm
I The computation of c takes only O(n) perations because onlythe first three components of c are nonzero
I The reduction of H1 to upper Hessenberg can be done inO(n2)
I Total cost for implicit QR for k iterations is 2kn3 additionsplus 2kn3 multiplications
I Total cost for explicit QR for k iterations is 8kn3 additionsplus 12kn3 multiplications
I Further notice that one step of implicit = two steps of explicit
Katja Grasic QR Algorithm
![Page 58: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/58.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Comparison of explicit QR and implicit QR algorithm
I The computation of c takes only O(n) perations because onlythe first three components of c are nonzero
I The reduction of H1 to upper Hessenberg can be done inO(n2)
I Total cost for implicit QR for k iterations is 2kn3 additionsplus 2kn3 multiplications
I Total cost for explicit QR for k iterations is 8kn3 additionsplus 12kn3 multiplications
I Further notice that one step of implicit = two steps of explicit
Katja Grasic QR Algorithm
![Page 59: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/59.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Comparison of explicit QR and implicit QR algorithm
I The computation of c takes only O(n) perations because onlythe first three components of c are nonzero
I The reduction of H1 to upper Hessenberg can be done inO(n2)
I Total cost for implicit QR for k iterations is 2kn3 additionsplus 2kn3 multiplications
I Total cost for explicit QR for k iterations is 8kn3 additionsplus 12kn3 multiplications
I Further notice that one step of implicit = two steps of explicit
Katja Grasic QR Algorithm
![Page 60: QR Algorithm - ETH Z](https://reader031.fdocuments.us/reader031/viewer/2022020703/61fb310e2e268c58cd5b3de6/html5/thumbnails/60.jpg)
OutlineIntroduction
DefinitionsThe Power Method
QR algorithm without shiftsHessenberg form
QR algorithm with shiftsImplicit shifted QR algorithm
Comparison of explicit QR and implicit QR algorithm
I The computation of c takes only O(n) perations because onlythe first three components of c are nonzero
I The reduction of H1 to upper Hessenberg can be done inO(n2)
I Total cost for implicit QR for k iterations is 2kn3 additionsplus 2kn3 multiplications
I Total cost for explicit QR for k iterations is 8kn3 additionsplus 12kn3 multiplications
I Further notice that one step of implicit = two steps of explicit
Katja Grasic QR Algorithm