Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical...
Transcript of Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical...
![Page 1: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/1.jpg)
Pseudospectraand the
Behavior of Dynamical Systems
Mark Embree
Computational and Applied Mathematics
Rice University
Houston, Texas
Trogir, Croatia
October 2011
![Page 2: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/2.jpg)
Overview of Lectures
These lectures address modern toolsfor the spectral analysis of dynamical systems.
We shall cover a mix of theory, computation, and applications.
Goal: By the end, you should be equipped to understand phenomena thatmany people find quite mysterious when encountered in the wild.
Lecture 1:
I Normality and Nonnormality
I Pseudospectra
I Bounding Functions of Matrices
Lecture 2:
I Balanced Truncation and Lyapunov Equations
I Moment Matching Model Reduction
I Differential Algebraic Equations
![Page 3: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/3.jpg)
Overview of Lectures
These lectures address modern toolsfor the spectral analysis of dynamical systems.
We shall cover a mix of theory, computation, and applications.
Goal: By the end, you should be equipped to understand phenomena thatmany people find quite mysterious when encountered in the wild.
Lecture 1:
I Normality and Nonnormality
I Pseudospectra
I Bounding Functions of Matrices
Lecture 2:
I Balanced Truncation and Lyapunov Equations
I Moment Matching Model Reduction
I Differential Algebraic Equations
![Page 4: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/4.jpg)
Overview of Lectures
These lectures address modern toolsfor the spectral analysis of dynamical systems.
We shall cover a mix of theory, computation, and applications.
Goal: By the end, you should be equipped to understand phenomena thatmany people find quite mysterious when encountered in the wild.
Lecture 1:
I Normality and Nonnormality
I Pseudospectra
I Bounding Functions of Matrices
Lecture 2:
I Balanced Truncation and Lyapunov Equations
I Moment Matching Model Reduction
I Differential Algebraic Equations
![Page 5: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/5.jpg)
Quiz
The plots below show the solution to two dynamical systems, x(t) = Ax(t).
Which system is stable?
That is, for which system, does x(t)→ 0 as t →∞?
(a) neither system is stable(b) only the one on the blue system is stable(c) only the one on the red system is stable(d) both are stable
![Page 6: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/6.jpg)
Quiz
The plots below show the solution to two dynamical systems, x(t) = Ax(t).
Which system is stable?
That is, for which system, does x(t)→ 0 as t →∞?
(a) neither system is stable(b) only the one on the blue system is stable(c) only the one on the red system is stable(d) both are stable
![Page 7: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/7.jpg)
Quiz
The plots below show the solution to two dynamical systems, x(t) = Ax(t).
Which system is stable?
That is, for which system, does x(t)→ 0 as t →∞?
original unstable system stabilized system
Eigenvalues of 55× 55 Boeing 767 flutter models [Burke, Lewis, Overton].These eigenvalues do not reveal the exotic transient behavior.
![Page 8: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/8.jpg)
Why is Transient Growth a Big Deal?
Many linear systems arise from the linearization of nonlinear equations, e.g.,Navier–Stokes. We compute eigenvalues as part of linear stability analysis.
Transient growth in a stable linearized system has implicationsfor the behavior of the associated nonlinear system.
Trefethen, Trefethen, Reddy, Driscoll“Hydrodynamic stability without eigenvalues,” Science, 1993.
![Page 9: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/9.jpg)
Why is Transient Growth a Big Deal?
Many linear systems arise from the linearization of nonlinear equations, e.g.,Navier–Stokes. We compute eigenvalues as part of linear stability analysis.
Transient growth in a stable linearized system has implicationsfor the behavior of the associated nonlinear system.
Consider the autonomous nonlinear system u′(t) = f(u).
I Find a steady state u∗, i.e., f(u∗) = 0.I Linearize f about this steady state and analyze small perturbations,
u = u∗ + v:
v′(t) = u′(t) = f(u∗ + v)
= f(u∗) + Av + O(‖v‖2)
= Av + O(‖v‖2).
I Ignore higher-order effects, and analyze the linear system v′(t) = Av(t).The steady state u∗ is stable provided A is stable.
But what if the small perturbation v(t) growsby orders of magnitude before eventually decaying?
![Page 10: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/10.jpg)
Spectra and Pseudospectra
Source for much of the content of these lectures:
Princeton University Press2005
![Page 11: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/11.jpg)
1. Normality and Nonnormality
![Page 12: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/12.jpg)
Motivating Applications
Many applications lead to models involving linear, non-self-adjoint operators.
I convective fluid flows
I damped mechanical systems
I atmospheric science
I magnetohydrodynamics
I neutron transport
I population dynamics
I food webs
I directed social networks
I Markov chains
I lasers
ut(x, t) = ν∆u(x, t)
− (a · ∇)u(x, t)
Mx′′(t) = −Kx(t)
−Dx′(t)
![Page 13: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/13.jpg)
Motivating Applications
Many applications lead to models involving linear, non-self-adjoint operators.
I convective fluid flows
I damped mechanical systems
I atmospheric science
I magnetohydrodynamics
I neutron transport
I population dynamics
I food webs
I directed social networks
I Markov chains
I lasers
ut(x, t) = ν∆u(x, t)− (a · ∇)u(x, t)
Mx′′(t) = −Kx(t)−Dx′(t)
![Page 14: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/14.jpg)
Normality and Nonnormality
Unless otherwise noted, all matrices are of size n × n, with complex entries.
The adjoint is denoted by A∗ = AT .
Definition (normal)
The matrix A is normal if it commutes with its adjoint, A∗A = AA∗.
A =
»1 1−1 1
–: A∗A = AA∗ =
»2 00 2
–=⇒ normal
A =
»−1 10 1
–: A∗A =
»1 −1−1 2
–6=»
2 11 1
–= AA∗ =⇒ nonnormal
Important note:The adjoint is defined via the inner product: 〈Ax, y〉 = 〈x,A∗y〉.hence the definition of normality depends on the inner product.Here we always use the standard Euclidean inner product, unless noted.In applications, one must use the physically relevant inner product.
![Page 15: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/15.jpg)
Normality and Nonnormality
Unless otherwise noted, all matrices are of size n × n, with complex entries.
The adjoint is denoted by A∗ = AT .
Definition (normal)
The matrix A is normal if it commutes with its adjoint, A∗A = AA∗.
A =
»1 1−1 1
–: A∗A = AA∗ =
»2 00 2
–=⇒ normal
A =
»−1 10 1
–: A∗A =
»1 −1−1 2
–6=»
2 11 1
–= AA∗ =⇒ nonnormal
Important note:The adjoint is defined via the inner product: 〈Ax, y〉 = 〈x,A∗y〉.hence the definition of normality depends on the inner product.Here we always use the standard Euclidean inner product, unless noted.In applications, one must use the physically relevant inner product.
![Page 16: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/16.jpg)
Conditions for Normality
Many (∼ 100) equivalent definitions of normality are known;see [Grone et al. 1987] . . . .
By far, the most important of these concerns the eigenvectors of A.
Theorem
The matrix A is normal if and only if it is unitarily diagonalizable,
A = UΛU∗,
for U unitary (U∗U = I) and Λ diagonal.
Equivalently, A is normal if and only if is possesses an orthonormal basis ofeigenvectors (i.e., the columns of U).
Hence, any nondiagonalizable (defective) matrix is nonnormal.But there are many interesting diagonalizable nonnormal matrices.Our fixation with diagonalizability has caused us to overlook these matrices.
![Page 17: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/17.jpg)
Conditions for Normality
Many (∼ 100) equivalent definitions of normality are known;see [Grone et al. 1987] . . . .
By far, the most important of these concerns the eigenvectors of A.
Theorem
The matrix A is normal if and only if it is unitarily diagonalizable,
A = UΛU∗,
for U unitary (U∗U = I) and Λ diagonal.
Equivalently, A is normal if and only if is possesses an orthonormal basis ofeigenvectors (i.e., the columns of U).
Hence, any nondiagonalizable (defective) matrix is nonnormal.But there are many interesting diagonalizable nonnormal matrices.Our fixation with diagonalizability has caused us to overlook these matrices.
![Page 18: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/18.jpg)
Orthogonality of Eigenvectors
Theorem
The matrix A is normal if and only if it is unitarily diagonalizable,
A = UΛU∗,
for U unitary (U∗U = I) and Λ diagonal.
Equivalently, A is normal if and only if is possesses an orthonormal basis ofeigenvectors (i.e., the columns of U).
An orthogonal basis of eigenvectors gives a perfect coordinate system forstudying dynamical systems:
x′(t) = Ax(t) =⇒ U∗x′(t) = U∗AUU∗x(t)
=⇒ z′(t) = Λz(t)
=⇒ z ′j (t) = λj zj (t),
with ‖x(t)‖ = ‖z(t)‖ for all t.
![Page 19: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/19.jpg)
The Perils of Oblique Eigenvectors
Now suppose we only have a diagonalization, A = VΛV−1:
x′(t) = Ax(t) =⇒ V−1x′(t) = V−1AVV−1x(t)
=⇒ z′(t) = Λz(t)
=⇒ z ′j (t) = λj zj (t),
with ‖x(t)‖ 6= ‖z(t)‖ in general.
The exact solution is easy:
x(t) = Vz(t) =nX
k=1
etλk zk (0)vk .
Suppose ‖x(0)‖ = 1. The coefficients zk (0) might still be quite large:
x(0) = Vz(0) =nX
k=0
zk (0)vk .
The “cancellation” that gives x(0) is washed out for t > 0 by the etλk terms.
![Page 20: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/20.jpg)
The Perils of Oblique Eigenvectors
Now suppose we only have a diagonalization, A = VΛV−1:
x′(t) = Ax(t) =⇒ V−1x′(t) = V−1AVV−1x(t)
=⇒ z′(t) = Λz(t)
=⇒ z ′j (t) = λj zj (t),
with ‖x(t)‖ 6= ‖z(t)‖ in general.
The exact solution is easy:
x(t) = Vz(t) =nX
k=1
etλk zk (0)vk .
Suppose ‖x(0)‖ = 1. The coefficients zk (0) might still be quite large:
x(0) = Vz(0) =nX
k=0
zk (0)vk .
The “cancellation” that gives x(0) is washed out for t > 0 by the etλk terms.
![Page 21: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/21.jpg)
Oblique Eigenvectors: Example
Example
»x ′1(t)x ′2(t)
–=
»−1/2 500
0 −5
– »x1(t)x2(t)
–,
»x1(0)x2(0)
–=
»11
–.
Eigenvalues and eigenvectors:
λ1 = −1/2, v1 =
»10
–, λ2 = −5, v2 =
»1
−.009
–Initial condition:
x(0) =
»11
–=
1000
9v1 −
1009
9v2.
Exact solution:
x(t) =1000
9e−λ1tv1 −
1009
9e−λ2tv2.
![Page 22: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/22.jpg)
Oblique Eigenvectors: Example
‖x(0)‖ = 1
‖x(.4)‖ = 76.75
Note the different scales of the horizontal and vertical axes.
![Page 23: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/23.jpg)
Oblique Eigenvectors Can Lead to Transient Growth
Transient growth in the solution: a consequence of nonnormality.
See Figure 1 in “Nineteen Dubious Ways to Compute the Exponential of aMatrix” by Moler and Van Loan, SIAM Review, 1978.
![Page 24: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/24.jpg)
Transient Behavior of the Power Method for Computing Eigenvalues
Large coefficients in the expansion of x0 in the eigenvector basis can lead tocancellation effects in xk = Ak x0.
Example: here different choices of α and β affect eigenvalue conditioning,
A =
241 α 00 3/4 β0 0 −3/4
35 , v1 =
24100
35 , v2 =
24−4α10
35 , v3 =
248αβ/21−2β/3
1
35 .
:v1
6v3
}v2
x0x2
x4x6
x8
x1
x3x5
x7 :v2:v1
6v3
x0
x1, ..., x8 :v1
iv3
kv2
x0x2
x4x6
x8x1
x3 x5 x7
normal nonnormal nonnormalv1 ⊥ v2 ⊥ v3 v1 6⊥ v2 ⊥ v3 v1 ⊥ v2 6⊥ v3
α = β = 0 α = 10, β = 0 α = 0, β = 10
Cf. [Beattie, E., Rossi 2004; Beattie, E., Sorensen 2005]
![Page 25: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/25.jpg)
Tools for Measuring Nonnormality
Given a matrix, we would like some effective way to measure whether we shouldbe concerned about the effects of nonnormality.
First, we might seek a scalar measure of normality.Any definition of normality leads to one such gauge.
I ‖A∗A− AA∗‖I min
Z normal‖A− Z‖
(See work on computing the nearest normal matrix by Gabriel, Ruhe 1987.)
I Henrici’s departure from normality:
dep2(A) = minA=U(D+N)U∗
Schur factorization
‖N‖.
No minimization is needed in the Frobenius norm:
depF (A) = minA=U(D+N)U∗
Schur factorization
‖N‖F =
vuut‖A‖2F −
nXj=1
|λj |2.
These are related by equivalence constants [Elsner, Paardekooper, 1987].None of these measures is of much use in practice.
![Page 26: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/26.jpg)
Tools for Measuring Nonnormality
Given a matrix, we would like some effective way to measure whether we shouldbe concerned about the effects of nonnormality.
First, we might seek a scalar measure of normality.Any definition of normality leads to one such gauge.
I ‖A∗A− AA∗‖I min
Z normal‖A− Z‖
(See work on computing the nearest normal matrix by Gabriel, Ruhe 1987.)
I Henrici’s departure from normality:
dep2(A) = minA=U(D+N)U∗
Schur factorization
‖N‖.
No minimization is needed in the Frobenius norm:
depF (A) = minA=U(D+N)U∗
Schur factorization
‖N‖F =
vuut‖A‖2F −
nXj=1
|λj |2.
These are related by equivalence constants [Elsner, Paardekooper, 1987].None of these measures is of much use in practice.
![Page 27: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/27.jpg)
Tools for Measuring Nonnormality
Given a matrix, we would like some effective way to measure whether we shouldbe concerned about the effects of nonnormality.
First, we might seek a scalar measure of normality.Any definition of normality leads to one such gauge.
I ‖A∗A− AA∗‖
I minZ normal
‖A− Z‖(See work on computing the nearest normal matrix by Gabriel, Ruhe 1987.)
I Henrici’s departure from normality:
dep2(A) = minA=U(D+N)U∗
Schur factorization
‖N‖.
No minimization is needed in the Frobenius norm:
depF (A) = minA=U(D+N)U∗
Schur factorization
‖N‖F =
vuut‖A‖2F −
nXj=1
|λj |2.
These are related by equivalence constants [Elsner, Paardekooper, 1987].None of these measures is of much use in practice.
![Page 28: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/28.jpg)
Tools for Measuring Nonnormality
Given a matrix, we would like some effective way to measure whether we shouldbe concerned about the effects of nonnormality.
First, we might seek a scalar measure of normality.Any definition of normality leads to one such gauge.
I ‖A∗A− AA∗‖I min
Z normal‖A− Z‖
(See work on computing the nearest normal matrix by Gabriel, Ruhe 1987.)
I Henrici’s departure from normality:
dep2(A) = minA=U(D+N)U∗
Schur factorization
‖N‖.
No minimization is needed in the Frobenius norm:
depF (A) = minA=U(D+N)U∗
Schur factorization
‖N‖F =
vuut‖A‖2F −
nXj=1
|λj |2.
These are related by equivalence constants [Elsner, Paardekooper, 1987].None of these measures is of much use in practice.
![Page 29: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/29.jpg)
Tools for Measuring Nonnormality
Given a matrix, we would like some effective way to measure whether we shouldbe concerned about the effects of nonnormality.
First, we might seek a scalar measure of normality.Any definition of normality leads to one such gauge.
I ‖A∗A− AA∗‖I min
Z normal‖A− Z‖
(See work on computing the nearest normal matrix by Gabriel, Ruhe 1987.)
I Henrici’s departure from normality:
dep2(A) = minA=U(D+N)U∗
Schur factorization
‖N‖.
No minimization is needed in the Frobenius norm:
depF (A) = minA=U(D+N)U∗
Schur factorization
‖N‖F =
vuut‖A‖2F −
nXj=1
|λj |2.
These are related by equivalence constants [Elsner, Paardekooper, 1987].None of these measures is of much use in practice.
![Page 30: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/30.jpg)
Tools for Measuring Nonnormality
Given a matrix, we would like some effective way to measure whether we shouldbe concerned about the effects of nonnormality.
First, we might seek a scalar measure of normality.Any definition of normality leads to one such gauge.
I ‖A∗A− AA∗‖I min
Z normal‖A− Z‖
(See work on computing the nearest normal matrix by Gabriel, Ruhe 1987.)
I Henrici’s departure from normality:
dep2(A) = minA=U(D+N)U∗
Schur factorization
‖N‖.
No minimization is needed in the Frobenius norm:
depF (A) = minA=U(D+N)U∗
Schur factorization
‖N‖F =
vuut‖A‖2F −
nXj=1
|λj |2.
These are related by equivalence constants [Elsner, Paardekooper, 1987].
None of these measures is of much use in practice.
![Page 31: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/31.jpg)
Tools for Measuring Nonnormality
Given a matrix, we would like some effective way to measure whether we shouldbe concerned about the effects of nonnormality.
First, we might seek a scalar measure of normality.Any definition of normality leads to one such gauge.
I ‖A∗A− AA∗‖I min
Z normal‖A− Z‖
(See work on computing the nearest normal matrix by Gabriel, Ruhe 1987.)
I Henrici’s departure from normality:
dep2(A) = minA=U(D+N)U∗
Schur factorization
‖N‖.
No minimization is needed in the Frobenius norm:
depF (A) = minA=U(D+N)U∗
Schur factorization
‖N‖F =
vuut‖A‖2F −
nXj=1
|λj |2.
These are related by equivalence constants [Elsner, Paardekooper, 1987].None of these measures is of much use in practice.
![Page 32: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/32.jpg)
Tools for Measuring Nonnormality
If A is diagonalizable,A = VΛV−1,
it is often helpful to characterize nonnormality by κ(V) := ‖V‖‖V−1‖ ≥ 1.
I This quantity depends on the choice of eigenvectors;scaling each column of V to be a unit vector gets within
√n
of the optimal value, if the eigenvalues are distinct [van der Sluis 1969].
I For normal matrices, one can take V unitary, so κ(V) = 1.
I If κ(V) is not much more than 1 for some diagonalization, then the effectsof nonnormality will be minimal.
Example (Bound for Continuous Systems)
‖x(t)‖ = ‖etAx(0)‖ ≤ ‖etA‖‖x(0)‖
≤ ‖VetΛV−1‖‖x(0)‖
≤ κ(V) maxλ∈σ(A)
|etλ|‖x(0)‖.
![Page 33: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/33.jpg)
Tools for Measuring Nonnormality
If A is diagonalizable,A = VΛV−1,
it is often helpful to characterize nonnormality by κ(V) := ‖V‖‖V−1‖ ≥ 1.
I This quantity depends on the choice of eigenvectors;scaling each column of V to be a unit vector gets within
√n
of the optimal value, if the eigenvalues are distinct [van der Sluis 1969].
I For normal matrices, one can take V unitary, so κ(V) = 1.
I If κ(V) is not much more than 1 for some diagonalization, then the effectsof nonnormality will be minimal.
Example (Bound for Continuous Systems)
‖x(t)‖ = ‖etAx(0)‖ ≤ ‖etA‖‖x(0)‖
≤ ‖VetΛV−1‖‖x(0)‖
≤ κ(V) maxλ∈σ(A)
|etλ|‖x(0)‖.
![Page 34: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/34.jpg)
Numerical Range (Field of Values)
Another approach: identify a set in the complex plane to replace the spectrum.This dates to the early 20th century literature in functional analysis, e.g., thenumerical range, Von Neumann’s spectral sets, and sectorial operators.
Definition (Numerical Range, a.k.a. Field of Values)
The numerical range of a matrix is the set
W (A) = {x∗Ax : ‖x‖ = 1} .
I W (A) is the set of all Rayleigh quotient eigenvalue estimates.
I σ(A) ⊂W (A) [Proof: take x to be an eigenvector.]
I W (A) is a closed, convex subset of C.
I If A is normal, then W (A) is the convex hull of σ(A).
I Unlike σ(A), the numerical range is robust to perturbations:
W (A + E) ⊆W (A) + {z ∈ C : |z | ≤ ‖E‖}.
![Page 35: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/35.jpg)
Numerical Range (Field of Values)
Another approach: identify a set in the complex plane to replace the spectrum.This dates to the early 20th century literature in functional analysis, e.g., thenumerical range, Von Neumann’s spectral sets, and sectorial operators.
Definition (Numerical Range, a.k.a. Field of Values)
The numerical range of a matrix is the set
W (A) = {x∗Ax : ‖x‖ = 1} .
I W (A) is the set of all Rayleigh quotient eigenvalue estimates.
I σ(A) ⊂W (A) [Proof: take x to be an eigenvector.]
I W (A) is a closed, convex subset of C.
I If A is normal, then W (A) is the convex hull of σ(A).
I Unlike σ(A), the numerical range is robust to perturbations:
W (A + E) ⊆W (A) + {z ∈ C : |z | ≤ ‖E‖}.
![Page 36: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/36.jpg)
A Gallery of Numerical Ranges
normal random Grcar Jordan
If z ∈W (A), then
Re z =z + z
2=
1
2(x∗Ax + x∗A∗x) = x∗
“A + A∗
2
”x.
Using properties of Hermitian matrices, we conclude that
Re(W (A)) =hλmin
“A + A∗
2
”, λmax
“A + A∗
2
”i.
Similarly, one can determine the intersection of W (A) with any line in C.
![Page 37: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/37.jpg)
A Gallery of Numerical Ranges
normal random Grcar Jordan
If z ∈W (A), then
Re z =z + z
2=
1
2(x∗Ax + x∗A∗x) = x∗
“A + A∗
2
”x.
Using properties of Hermitian matrices, we conclude that
Re(W (A)) =hλmin
“A + A∗
2
”, λmax
“A + A∗
2
”i.
Similarly, one can determine the intersection of W (A) with any line in C.
![Page 38: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/38.jpg)
Computation of the Numerical Range
This calculation yields points on the boundary of the numerical range.Use convexity to obtain polygonal outer and inner approximations[Johnson 1980]; Higham’s fv.m.
Neat problem: Given z ∈W (A), find unit vector x such that z = x∗Ax[Uhlig 2008; Carden 2009].
![Page 39: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/39.jpg)
Computation of the Numerical Range
This calculation yields points on the boundary of the numerical range.Use convexity to obtain polygonal outer and inner approximations[Johnson 1980]; Higham’s fv.m.
Neat problem: Given z ∈W (A), find unit vector x such that z = x∗Ax[Uhlig 2008; Carden 2009].
![Page 40: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/40.jpg)
Numerical Range and the Matrix Exponential
What does the numerical range reveal about matrix behavior?
d
dt‖etAx0‖
˛t=0
=d
dt
“x∗0 etA∗etAx0
”1/2
=d
dt
“x∗0 (I + tA∗)(I + tA)x0
”1/2
=1
‖x0‖x∗0“A + A∗
2
”x0
So, the rightmost point in W (A) reveals the maximal slope of ‖etA‖ at t = 0.
Definition (numerical abscissa)
The numerical abscissa is the rightmost in W (A):
ω(A) := maxz∈W (A)
Re z .
![Page 41: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/41.jpg)
Numerical Range and the Matrix Exponential
What does the numerical range reveal about matrix behavior?
d
dt‖etAx0‖
˛t=0
=d
dt
“x∗0 etA∗etAx0
”1/2
=d
dt
“x∗0 (I + tA∗)(I + tA)x0
”1/2
=1
‖x0‖x∗0“A + A∗
2
”x0
So, the rightmost point in W (A) reveals the maximal slope of ‖etA‖ at t = 0.
Definition (numerical abscissa)
The numerical abscissa is the rightmost in W (A):
ω(A) := maxz∈W (A)
Re z .
![Page 42: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/42.jpg)
Intitial Transient Growth via Numerical Abscissa
A =
»−1.1 10
0 −1
–
eω(A)t
κ(V)e−t
![Page 43: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/43.jpg)
Crouzeix’s Conjecture
The numerical range can be used to help gauge the size of matrix functions.
Theorem (Crouzeix, 2007)
Let f be a function analytic on W (A). Then
‖f (A)‖ ≤ 11.08 maxz∈W (A)
|f (z)|.
Crouzeix’s Conjecture:‖f (A)‖ ≤ 2 max
z∈W (A)|f (z)|.
Obstacle: in applications, W (A) is often too large, e.g., it contains the origin,or points in the right-half plane, or points of magnitude larger than one.
![Page 44: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/44.jpg)
Crouzeix’s Conjecture
The numerical range can be used to help gauge the size of matrix functions.
Theorem (Crouzeix, 2007)
Let f be a function analytic on W (A). Then
‖f (A)‖ ≤ 11.08 maxz∈W (A)
|f (z)|.
Crouzeix’s Conjecture:‖f (A)‖ ≤ 2 max
z∈W (A)|f (z)|.
Obstacle: in applications, W (A) is often too large, e.g., it contains the origin,or points in the right-half plane, or points of magnitude larger than one.
![Page 45: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/45.jpg)
Boeing 737 Example, Revisited
Numerical range of 55× 55 stable matrix.
σ(A) W (A)
![Page 46: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/46.jpg)
2. Pseudospectra
![Page 47: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/47.jpg)
Pseudospectra
The numerical range and the eigenvalues both have limitations.For nonnormal matrices, W (A) can be “too big,” while σ(A) is “too small.”
Here we shall explore another option that, loosely speaking,interpolates between σ(A) and W (A).
Example
Compute eigenvalues of three similar 100× 100 matrices using MATLAB’s eig.
266640 1
1 0. . .
. . .. . . 11 0
3777526664
0 1/2
2 0. . .
. . .. . . 1/22 0
3777526664
0 1/3
3 0. . .
. . .. . . 1/33 0
37775
![Page 48: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/48.jpg)
Pseudospectra
The numerical range and the eigenvalues both have limitations.For nonnormal matrices, W (A) can be “too big,” while σ(A) is “too small.”
Here we shall explore another option that, loosely speaking,interpolates between σ(A) and W (A).
Example
Compute eigenvalues of three similar 100× 100 matrices using MATLAB’s eig.
266640 1
1 0. . .
. . .. . . 11 0
3777526664
0 1/2
2 0. . .
. . .. . . 1/22 0
3777526664
0 1/3
3 0. . .
. . .. . . 1/33 0
37775
![Page 49: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/49.jpg)
Pseudospectra
The numerical range and the eigenvalues both have limitations.For nonnormal matrices, W (A) can be “too big,” while σ(A) is “too small.”
Here we shall explore another option that, loosely speaking,interpolates between σ(A) and W (A).
Example
Compute eigenvalues of three similar 100× 100 matrices using MATLAB’s eig.
266640 1
1 0. . .
. . .. . . 11 0
3777526664
0 1/2
2 0. . .
. . .. . . 1/22 0
3777526664
0 1/3
3 0. . .
. . .. . . 1/33 0
37775
![Page 50: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/50.jpg)
Pseudospectra
Definition (ε-pseudospectrum)
For any ε > 0, the ε-pseudospectrum of A, denoted σε(A), is the set
σε(A) = {z ∈ C : z ∈ σ(A + E) for some E ∈ Cn×n with ‖E‖ < ε}.
We can estimate σε(A) by conducting experiments with random perturbations.
For the 20× 20 version of A = tridiag(2, 0, 1/2), 50 trials each:
ε = 10−2 ε = 10−4 ε = 10−6
![Page 51: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/51.jpg)
Pseudospectra
Definition (ε-pseudospectrum)
For any ε > 0, the ε-pseudospectrum of A, denoted σε(A), is the set
σε(A) = {z ∈ C : z ∈ σ(A + E) for some E ∈ Cn×n with ‖E‖ < ε}.
We can estimate σε(A) by conducting experiments with random perturbations.
For the 20× 20 version of A = tridiag(2, 0, 1/2), 50 trials each:
ε = 10−2 ε = 10−4 ε = 10−6
![Page 52: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/52.jpg)
Equivalent Definitions of the Pseudospectrum
Theorem
The following three definitions of the ε-pseudospectrum are equivalent:
1. σε(A) = {z ∈ C : z ∈ σ(A + E) for some E ∈ C n×n with ‖E‖ < ε};2. σε(A) = {z ∈ C : ‖(z − A)−1‖ > 1/ε};3. σε(A) = {z ∈ C : ‖Av − zv‖ < ε for some unit vector v ∈ C n}.
![Page 53: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/53.jpg)
Equivalent Definitions of the Pseudospectrum
Theorem
The following three definitions of the ε-pseudospectrum are equivalent:
1. σε(A) = {z ∈ C : z ∈ σ(A + E) for some E ∈ C n×n with ‖E‖ < ε};2. σε(A) = {z ∈ C : ‖(z − A)−1‖ > 1/ε};3. σε(A) = {z ∈ C : ‖Av − zv‖ < ε for some unit vector v ∈ C n}.
Proof. (1) =⇒ (2)
If z ∈ σ(A + E) for some E with ‖E‖ < ε, there exists a unit vector v such that(A + E)v = zv. Rearrange to obtain
v = (z − A)−1Ev.
Take norms:
‖v‖ = ‖(z − A)−1Ev‖ ≤ ‖(z − A)−1‖‖E‖‖v‖ < ε‖(z − A)−1‖‖v‖.
Hence ‖(z − A)−1‖ > 1/ε.
![Page 54: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/54.jpg)
Equivalent Definitions of the Pseudospectrum
Theorem
The following three definitions of the ε-pseudospectrum are equivalent:
1. σε(A) = {z ∈ C : z ∈ σ(A + E) for some E ∈ C n×n with ‖E‖ < ε};2. σε(A) = {z ∈ C : ‖(z − A)−1‖ > 1/ε};3. σε(A) = {z ∈ C : ‖Av − zv‖ < ε for some unit vector v ∈ C n}.
Proof. (2) =⇒ (3)
If ‖(z − A)−1‖ > 1/ε, there exists a unit vector w with ‖(z − A)−1w‖ > 1/ε.Define bv := (z − A)−1w, so that 1/‖bv‖ < ε, and
‖(z − A)bv‖‖bv‖ =
‖w‖‖bv‖ < ε.
Hence we have found a unit vector v := bv/‖bv‖ for which ‖Av − zv‖ < ε.
![Page 55: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/55.jpg)
Equivalent Definitions of the Pseudospectrum
Theorem
The following three definitions of the ε-pseudospectrum are equivalent:
1. σε(A) = {z ∈ C : z ∈ σ(A + E) for some E ∈ C n×n with ‖E‖ < ε};2. σε(A) = {z ∈ C : ‖(z − A)−1‖ > 1/ε};3. σε(A) = {z ∈ C : ‖Av − zv‖ < ε for some unit vector v ∈ C n}.
Proof. (3) =⇒ (1)
Given a unit vector v such that ‖Av − zv‖ < ε, define r := Av − zv.Now set E := −rv∗, so that
(A + E)v = (A− rv∗)v = Av − r = zv.
Hence z ∈ σ(A + E).
![Page 56: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/56.jpg)
Equivalent Definitions of the Pseudospectrum
Theorem
The following three definitions of the ε-pseudospectrum are equivalent:
1. σε(A) = {z ∈ C : z ∈ σ(A + E) for some E ∈ C n×n with ‖E‖ < ε};2. σε(A) = {z ∈ C : ‖(z − A)−1‖ > 1/ε};3. σε(A) = {z ∈ C : ‖Av − zv‖ < ε for some unit vector v ∈ C n}.
These different definitions are useful in different contexts:
1. interpreting numerically computed eigenvalues;
2. analyzing matrix behavior/functions of matrices;computing pseudospectra on a grid in C;
3. proving bounds on a particular σε(A).
![Page 57: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/57.jpg)
History of Pseudospectra
Invented at least four times, independently:
I Jim Varah in his 1967 Stanford PhD thesis.
I Henry Landau in a 1975 paper, motivated by laser theory.
I S. K. Godunov and colleagues in Novosibirsk in the 1980s.
I L. N. Trefethen, motivated by stability of spectral methods (1990).
Early adopters include Wilkinson (1986), Demmel (1987), Chatelin (1990s).
![Page 58: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/58.jpg)
Properties of Pseudospectra
A is normal ⇐⇒ σε(A) is the union of open ε-balls about each eigenvalue:
A normal =⇒ σε(A) =[
j
λj + ∆ε
A nonnormal =⇒ σε(A) ⊃[
j
λj + ∆ε
A circulant (hence normal) matrix:
A =
2666640 1 0 0 00 0 1 0 00 0 0 1 00 0 0 0 11 0 0 0 0
377775
![Page 59: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/59.jpg)
Properties of Pseudospectra
A is normal ⇐⇒ σε(A) is the union of open ε-balls about each eigenvalue:
A normal =⇒ σε(A) =[
j
λj + ∆ε
A nonnormal =⇒ σε(A) ⊃[
j
λj + ∆ε
Re z
Im z
‖(z − A)−1‖
A circulant (hence normal) matrix:
A =
2666640 1 0 0 00 0 1 0 00 0 0 1 00 0 0 0 11 0 0 0 0
377775
![Page 60: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/60.jpg)
Normal Matrices: matching distance
An easy misinterpretation: “A size ε perturbation to a normal matrix can movethe eigenvalues by no more than ε.”
This holds for Hermitian matrices, but not all normal matrices.
An example constructed by Gerd Krause (see Bhatia, Matrix Analysis, 1997):
A = diag(1, (4 + 5√−3)/13, (−1 + 2
√−3)/13).
Now construct the (unitary) Householder reflector
U = I− 2vv∗
for v = [p
5/8, 1/2,p
1/8]∗, and define E via
A + E = −U∗AU.
By construction A + E is normal and σ(A + E) = −σ(A).
![Page 61: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/61.jpg)
Normal Matrices: matching distance
A = diag(1, (4 + 5√−3)/13, (−1 + 2
√−3)/13).
A + E = −U∗AU.
−1.5 −1 −0.5 0 0.5 1 1.5 2 2.5
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
![Page 62: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/62.jpg)
Normal Matrices: matching distance
A = diag(1, (4 + 5√−3)/13, (−1 + 2
√−3)/13).
A + E = −U∗AU.
−1.5 −1 −0.5 0−1
−0.5
0
0.5
![Page 63: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/63.jpg)
Properties of Pseudospectra
Theorem (Facts about Pseudospectra)
For all ε > 0,
I σε(A) is an open, finite set that contains the spectrum.
I σε(A) is stable to perturbations: σε(A + E) ⊆ σε+‖E‖(A).
I If U is unitary, σε(UAU∗) = σε(A).
I For V invertible, σε/κ(V)(VAV−1) ⊆ σε(A) ⊆ σεκ(V)(VAV−1).
I σε(A + α) = α + σε(A).
I σ|γ|ε(γA) = γσε(A).
I σε“»A 0
0 B
–”= σε(A) ∪ σε(B).
![Page 64: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/64.jpg)
Relationship to κ(V) and W(A)
Let ∆r := {z ∈ C : |z | < r} denote the open disk of radius r > 0.
Theorem (Bauer–Fike, 1963)
Let A be diagonalizable, A = VΛV−1. Then for all ε > 0,
σε(A) ⊆ σ(A) + ∆εκ(V).
If κ(V) is small, then σε(A) cannot contain points far from σ(A).
Theorem (Stone, 1932)
For any A,σε(A) ⊆W (A) + ∆ε.
The pseudospectrum σε(A) cannot be bigger than W (A) in an interesting way.
![Page 65: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/65.jpg)
Relationship to κ(V) and W(A)
Let ∆r := {z ∈ C : |z | < r} denote the open disk of radius r > 0.
Theorem (Bauer–Fike, 1963)
Let A be diagonalizable, A = VΛV−1. Then for all ε > 0,
σε(A) ⊆ σ(A) + ∆εκ(V).
If κ(V) is small, then σε(A) cannot contain points far from σ(A).
Theorem (Stone, 1932)
For any A,σε(A) ⊆W (A) + ∆ε.
The pseudospectrum σε(A) cannot be bigger than W (A) in an interesting way.
![Page 66: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/66.jpg)
Toeplitz Matrices
Pseudospectra of the 100× 100 matrix A = tridiag(2, 0, 1/2) that began ourinvestigation of eigenvalue perturbations.
A is diagonalizable (it has distinct eigenvalues), butBauer–Fike is useless here: κ(V) = 299 ≈ 6× 1029.
![Page 67: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/67.jpg)
Jordan Blocks
Sn =
266640 1
0. . .
. . . 10
37775
Near the eigenvalue, the resolvent norm grows with dimension n;outside the unit disk, the resolvent norm does not seem to get big.We would like to prove this.
![Page 68: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/68.jpg)
Jordan Blocks
Sn =
266640 1
0. . .
. . . 10
37775
Near the eigenvalue, the resolvent norm grows with dimension n;outside the unit disk, the resolvent norm does not seem to get big.We would like to prove this.
![Page 69: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/69.jpg)
Jordan Blocks
Sn =
266640 1
0. . .
. . . 10
37775
Near the eigenvalue, the resolvent norm grows with dimension n;outside the unit disk, the resolvent norm does not seem to get big.We would like to prove this.
![Page 70: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/70.jpg)
Jordan Blocks/Shift Operator
Consider the generalization of the Jordan block to the domain
`2(N) = {(x1, x2, . . .) :∞X
j=1
|xj |2 <∞}.
The shift operator S on `2(N) is defined as
S(x1, x2, . . .) = (x2, x3, . . .).
In particular,
S(1, z , z2, . . .) = (z , z2, z3, . . .)
= z(1, z , z2, . . .).
So if (1, z , z2, . . .) ∈ `2(N), then z ∈ σ(S).
If |z | < 1, then∞X
j=1
|z j−1|2 =1
1− |z |2 <∞.
So,{z ∈ C : |z | < 1} ⊆ σ(S).
![Page 71: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/71.jpg)
Jordan Blocks/Shift Operator
Consider the generalization of the Jordan block to the domain
`2(N) = {(x1, x2, . . .) :∞X
j=1
|xj |2 <∞}.
The shift operator S on `2(N) is defined as
S(x1, x2, . . .) = (x2, x3, . . .).
In particular,
S(1, z , z2, . . .) = (z , z2, z3, . . .)
= z(1, z , z2, . . .).
So if (1, z , z2, . . .) ∈ `2(N), then z ∈ σ(S).
If |z | < 1, then∞X
j=1
|z j−1|2 =1
1− |z |2 <∞.
So,{z ∈ C : |z | < 1} ⊆ σ(S).
![Page 72: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/72.jpg)
Jordan Blocks/Shift Operator
Consider the generalization of the Jordan block to the domain
`2(N) = {(x1, x2, . . .) :∞X
j=1
|xj |2 <∞}.
The shift operator S on `2(N) is defined as
S(x1, x2, . . .) = (x2, x3, . . .).
In particular,
S(1, z , z2, . . .) = (z , z2, z3, . . .)
= z(1, z , z2, . . .).
So if (1, z , z2, . . .) ∈ `2(N), then z ∈ σ(S).
If |z | < 1, then∞X
j=1
|z j−1|2 =1
1− |z |2 <∞.
So,{z ∈ C : |z | < 1} ⊆ σ(S).
![Page 73: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/73.jpg)
Jordan Blocks/Shift Operator
S(x1, x2, . . .) = (x2, x3, . . .).
We have seen that{z ∈ C : |z | < 1} ⊆ σ(S).
Observe that‖S‖ = sup
‖x‖=1
‖Sx‖ = 1,
and soσ(S) ⊆ {z ∈ C : |z | ≤ 1}.
The spectrum is closed, so
σ(A) = {z ∈ C : |z | ≤ 1}.
For any finite dimensional n × n Jordan block Sn,
σ(Sn) = {0}.
So the Sn → S strongly, but there is a discontinuity in the spectrum:
σ(Sn) 6→ σ(S).
![Page 74: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/74.jpg)
Pseudospectra of Jordan Blocks
Pseudospectra resolve this unpleasant discontinuity.
Recall the eigenvectors (1, z , z2, . . .) for S.
Truncate this vector to length n, and apply it to Sn:
26666664
0 1
0. . .
. . . 10 1
0
37777775
26666641z...
zn−2
zn−1
3777775 =
2666664zz2
...zn−1
0
3777775
= z
26666641z...
zn−2
zn−1
3777775−2666664
00...0zn
3777775 .
Hence, ‖Snx− zx‖ = |z |n, so for all ε >|z |n
‖x‖ = |z |np
1− |z |2np1− |z |2
,
z ∈ σε(Sn).
We conclude that for fixed |z | < 1, the resolvent norm ‖(z − Sn)−1‖grows exponentially with n.
![Page 75: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/75.jpg)
Pseudospectra of Jordan Blocks
Pseudospectra resolve this unpleasant discontinuity.
Recall the eigenvectors (1, z , z2, . . .) for S.
Truncate this vector to length n, and apply it to Sn:
26666664
0 1
0. . .
. . . 10 1
0
37777775
26666641z...
zn−2
zn−1
3777775 =
2666664zz2
...zn−1
0
3777775 = z
26666641z...
zn−2
zn−1
3777775−2666664
00...0zn
3777775 .
Hence, ‖Snx− zx‖ = |z |n, so for all ε >|z |n
‖x‖ = |z |np
1− |z |2np1− |z |2
,
z ∈ σε(Sn).
We conclude that for fixed |z | < 1, the resolvent norm ‖(z − Sn)−1‖grows exponentially with n.
![Page 76: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/76.jpg)
Pseudospectra of Jordan Blocks
Pseudospectra resolve this unpleasant discontinuity.
Recall the eigenvectors (1, z , z2, . . .) for S.
Truncate this vector to length n, and apply it to Sn:
26666664
0 1
0. . .
. . . 10 1
0
37777775
26666641z...
zn−2
zn−1
3777775 =
2666664zz2
...zn−1
0
3777775 = z
26666641z...
zn−2
zn−1
3777775−2666664
00...0zn
3777775 .
Hence, ‖Snx− zx‖ = |z |n, so for all ε >|z |n
‖x‖ = |z |np
1− |z |2np1− |z |2
,
z ∈ σε(Sn).
We conclude that for fixed |z | < 1, the resolvent norm ‖(z − Sn)−1‖grows exponentially with n.
![Page 77: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/77.jpg)
Pseudospectra of Toeplitz Matrices
Toeplitz matrices are described by their symbol a with Laurent expansion
a(z) =∞X
k=−∞
ak zk ,
giving the matrix with constant diagonals containing the Laurent coefficients:
An =
2666666664
a0 a−1 a−2 · · ·
a1 a0
. . .. . .
...
a2
. . .. . . a−1 a−2
.... . . a1 a0 a−1
· · · a2 a1 a0
3777777775∈ Cn×n.
Call the image of the unit circle T under a the symbol curve, a(T).
![Page 78: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/78.jpg)
Pseudospectra of Toeplitz Matrices
Theorem (Landau; Reichel and Trefethen; Bottcher)
Let a(z) =PM
k=−M ak zk be the symbol of a banded Toepiltz operator.
I The pseudospectra of An converge to the pseudospectra of the Toeplitzoperator on `2(N) as n→∞.
Let z ∈ C have nonzero winding number w.r.t. the symbol curve a(T ).
I ‖(z − An)−1‖ grows exponentially in n.
I For all ε > 0, z ∈ σε(An) for all n sufficiently large.
n = 16 n = 32 n = 64 n = 128
![Page 79: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/79.jpg)
Pseudospectra of Toeplitz Matrices
symbol curve σε(A100)
![Page 80: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/80.jpg)
Pseudospectra of Toeplitz Matrices
symbol curve σε(A500)
![Page 81: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/81.jpg)
Computation of Pseudospectra
Naive algorithm: O(n3) per grid point
I Compute ‖(z − A)−1‖ using the SVD on a grid of points in C.
I Send data to a contour plotting routine.
Modern algorithm: O(n3) + O(n2) per grid point [Lui 1997; Trefethen 1999]
I Compute a Schur factorization A = UTU∗ and compute σε(T).
I Compute ‖(z −T)−1‖ on grid via inverse Lanczos on (z −T)−∗(z −T)−1.
Large-scale problems: [Toh and Trefethen 1996; Wright and Trefethen 2001]
I Pick an invariant subspace Ran(V) for V∗V = I corresponding toeigenvalues of physical interest (e.g., using ARPACK).
I Compute σε(V∗AV) ⊆ σε(A).
Alternative: [Bruhl 1996; Bekas and Gallopoulos, . . . ]
I Curve tracing: follow level sets of ‖(z − A)−1‖.
![Page 82: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/82.jpg)
EigTool
EigTool: Thomas Wright, 2002
http://www.cs.ox.ac.uk/pseudospectra/eigtool
![Page 83: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/83.jpg)
Pole Placement Example
Problem (Pole Placement Example of Mehrmann and Xu, 1996)
Given A = diag(1, 2, . . . ,N) and b = [1, 1, . . . , 1]T , find f ∈ C n such that
σ(A− bf∗) = {−1,−2, . . . ,−N}.
One can show that f = G−∗e, where e = [1, 1, . . . , 1]T , where
G:,k = (A− λk )−1b = (A + k)−1b.
This gives
A− bf∗ = G
264−1. . .
−n
375G−1,
with
Gj,k =1
j + k.
![Page 84: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/84.jpg)
Pole Placement Example: Numerics
In exact arithmetic, σ(A− bf∗) = {−1,−2, . . . ,−N}.
All entries in A− bf∗ are integers.(To ensure this, we compute f symbolically.)
For example, when N = 8,
A−bf∗ =
2666666664
73 −2520 27720 −138600 360360 −504504 360360 −10296072 −2518 27720 −138600 360360 −504504 360360 −10296072 −2520 27723 −138600 360360 −504504 360360 −10296072 −2520 27720 −138596 360360 −504504 360360 −10296072 −2520 27720 −138600 360365 −504504 360360 −10296072 −2520 27720 −138600 360360 −504498 360360 −10296072 −2520 27720 −138600 360360 −504504 360367 −10296072 −2520 27720 −138600 360360 −504504 360360 −102952
3777777775.
![Page 85: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/85.jpg)
Pole Placement Example: Numerics
In exact arithmetic, σ(A− bf∗) = {−1,−2, . . . ,−N}.
Computed eigenvalues of A− bf∗ (matrix is exact: all integer entries).
◦ exact eigenvalues• computed eigenvalues
![Page 86: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/86.jpg)
Pole Placement Example: Numerics
In exact arithmetic, σ(A− bf∗) = {−1,−2, . . . ,−N}.
Computed eigenvalues of A− bf∗ (matrix is exact: all integer entries).
◦ exact eigenvalues• computed eigenvalues
![Page 87: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/87.jpg)
Pole Placement Example: Numerics
In exact arithmetic, σ(A− bf∗) = {−1,−2, . . . ,−N}.
Computed eigenvalues of A− bf∗ (matrix is exact: all integer entries).
◦ exact eigenvalues• computed eigenvalues
![Page 88: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/88.jpg)
Pole Placement Example: Numerics
In exact arithmetic, σ(A− bf∗) = {−1,−2, . . . ,−N}.
Computed eigenvalues of A− bf∗ (matrix is exact: all integer entries).
◦ exact eigenvalues• computed eigenvalues
![Page 89: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/89.jpg)
Pole Placement Example: Numerics
In exact arithmetic, σ(A− bf∗) = {−1,−2, . . . ,−N}.
Computed eigenvalues of A− bf∗ (matrix is exact: all integer entries).
◦ exact eigenvalues• computed eigenvalues
![Page 90: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/90.jpg)
Pole Placement Example: Numerics
In exact arithmetic, σ(A− bf∗) = {−1,−2, . . . ,−N}.
Computed eigenvalues of A− bf∗ (matrix is exact: all integer entries).
◦ exact eigenvalues• computed eigenvalues
![Page 91: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/91.jpg)
Pole Placement Example: Numerics
In exact arithmetic, σ(A− bf∗) = {−1,−2, . . . ,−N}.
Computed eigenvalues of A− bf∗ (matrix is exact: all integer entries).
◦ exact eigenvalues• computed eigenvalues
![Page 92: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/92.jpg)
Pole Placement Example: Numerics
In exact arithmetic, σ(A− bf∗) = {−1,−2, . . . ,−N}.
Computed eigenvalues of A− bf∗ (matrix is exact: all integer entries).
◦ exact eigenvalues• computed eigenvalues
![Page 93: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/93.jpg)
Pole Placement Example: Numerics
In exact arithmetic, σ(A− bf∗) = {−1,−2, . . . ,−N}.
Computed eigenvalues of A− bf∗ (matrix is exact: all integer entries).
◦ exact eigenvalues• computed eigenvalues
![Page 94: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/94.jpg)
Pole Placement Example: Numerics
In exact arithmetic, σ(A− bf∗) = {−1,−2, . . . ,−N}.
Computed eigenvalues of A− bf∗ (matrix is exact: all integer entries).
◦ exact eigenvalues• computed eigenvalues
![Page 95: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/95.jpg)
Pole Placement Example: Numerics
In exact arithmetic, σ(A− bf∗) = {−1,−2, . . . ,−N}.
Computed eigenvalues of A− bf∗ (matrix is exact: all integer entries).
◦ exact eigenvalues• computed eigenvalues
![Page 96: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/96.jpg)
Pole Placement Example: Pseudospectra
Computed pseudospectra of A− bf∗.
Computed eigenvalues should be accurate to roughly εmach‖A− bf∗‖.For example, when N = 11, ‖A− bf∗‖ = 5.26× 108.
![Page 97: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/97.jpg)
Pole Placement Example: Pseudospectra
Computed pseudospectra of A− bf∗.
Computed eigenvalues should be accurate to roughly εmach‖A− bf∗‖.For example, when N = 11, ‖A− bf∗‖ = 5.26× 108.
![Page 98: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/98.jpg)
Pole Placement Example: Pseudospectra
Computed pseudospectra of A− bf∗.
Computed eigenvalues should be accurate to roughly εmach‖A− bf∗‖.For example, when N = 11, ‖A− bf∗‖ = 5.26× 108.
![Page 99: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/99.jpg)
Pole Placement Example: Pseudospectra
Computed pseudospectra of A− bf∗.
Computed eigenvalues should be accurate to roughly εmach‖A− bf∗‖.For example, when N = 11, ‖A− bf∗‖ = 5.26× 108.
![Page 100: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/100.jpg)
Pole Placement Example: Pseudospectra
Computed pseudospectra of A− bf∗.
Computed eigenvalues should be accurate to roughly εmach‖A− bf∗‖.For example, when N = 11, ‖A− bf∗‖ = 5.26× 108.
![Page 101: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/101.jpg)
Pole Placement Example: Pseudospectra
Computed pseudospectra of A− bf∗.
Computed eigenvalues should be accurate to roughly εmach‖A− bf∗‖.For example, when N = 11, ‖A− bf∗‖ = 5.26× 108.
![Page 102: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/102.jpg)
Pole Placement Example: Pseudospectra
Computed pseudospectra of A− bf∗.
Computed eigenvalues should be accurate to roughly εmach‖A− bf∗‖.For example, when N = 11, ‖A− bf∗‖ = 5.26× 108.
![Page 103: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/103.jpg)
Pole Placement Example: Pseudospectra
Computed pseudospectra of A− bf∗.
Computed eigenvalues should be accurate to roughly εmach‖A− bf∗‖.For example, when N = 11, ‖A− bf∗‖ = 5.26× 108.
![Page 104: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/104.jpg)
Pole Placement Example: Pseudospectra
Computed pseudospectra of A− bf∗.
Computed eigenvalues should be accurate to roughly εmach‖A− bf∗‖.For example, when N = 11, ‖A− bf∗‖ = 5.26× 108.
![Page 105: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/105.jpg)
Polynomial Zeros and Companion Matrices
MATLAB’s roots command computes polynomial zeros by computing theeigenvalues of a companion matrix.
For example, given p(z) = c0 + c1z + c2z2 + c3z3 + c4z4, MATLAB builds
A =
2664−c4/c0 −c3/c0 −c2/c0 −c1/c0
1 0 0 00 1 0 00 0 1 0
3775 .whose characteristic polynomial is p.
Problem (Wilkinson’s “Perfidious Polynomial”)
Find the zeros of the polynomial
p(z) = (z − 1)(z − 2) · · · (z − N)
from coefficients in the monomial basis.
MATLAB gives this as an example: roots(poly(1:20)).
![Page 106: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/106.jpg)
Polynomial Zeros and Companion Matrices
MATLAB’s roots command computes polynomial zeros by computing theeigenvalues of a companion matrix.
For example, given p(z) = c0 + c1z + c2z2 + c3z3 + c4z4, MATLAB builds
A =
2664−c4/c0 −c3/c0 −c2/c0 −c1/c0
1 0 0 00 1 0 00 0 1 0
3775 .whose characteristic polynomial is p.
Problem (Wilkinson’s “Perfidious Polynomial”)
Find the zeros of the polynomial
p(z) = (z − 1)(z − 2) · · · (z − N)
from coefficients in the monomial basis.
MATLAB gives this as an example: roots(poly(1:20)).
![Page 107: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/107.jpg)
Roots of Wilkinson’s Perfidious Polynomial
Computed eigenvalues of the companion matrix.
◦ exact eigenvalues• computed eigenvalues
![Page 108: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/108.jpg)
Roots of Wilkinson’s Perfidious Polynomial
Computed eigenvalues of the companion matrix.
◦ exact eigenvalues• computed eigenvalues
![Page 109: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/109.jpg)
Roots of Wilkinson’s Perfidious Polynomial
Computed eigenvalues of the companion matrix.
◦ exact eigenvalues• computed eigenvalues
![Page 110: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/110.jpg)
Roots of Wilkinson’s Perfidious Polynomial
Computed eigenvalues of the companion matrix.
◦ exact eigenvalues• computed eigenvalues
![Page 111: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/111.jpg)
Roots of Wilkinson’s Perfidious Polynomial
Computed eigenvalues of the companion matrix.
◦ exact eigenvalues• computed eigenvalues
![Page 112: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/112.jpg)
Roots of Wilkinson’s Perfidious Polynomial
Pseudospectra for N = 25.
◦ exact eigenvalues• computed eigenvalues
![Page 113: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/113.jpg)
Roots of Wilkinson’s Perfidious Polynomial
3d plot of the resolvent norm reveals the a local minimum.
Re z
Im z
‖(z − A)−1‖
◦ exact eigenvalues• computed eigenvalues
![Page 114: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/114.jpg)
3. Bounding Norms of Functions of Matrices
![Page 115: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/115.jpg)
Behavior of Matrices
We are primarily interested in using pseudospectra (and other tools)to study the behavior of a nonnormal matrix.
In particular, we seek to quantify (or bound) how nonnormality affectsthe value of functions of matrices.
Much research has been devoted to functions of matrices over the past decade;see the book by Nick Higham [Hig08]. We focus on functions that are analyticon the spectrum of A.
Theorem (Spectral Mapping Theorem)
Suppose f is analytic on σ(A). Then
σ(f (A)) = f (σ(A)).
Thus ‖f (A)‖ ≥ maxµ∈σ(f (A))
|µ| = maxλ∈σ(A)
|f (λ)|.
See [Huhtanen, 1999] for work on lower bounds.
![Page 116: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/116.jpg)
Spectral Representation of a Matrix
Theorem (See Kato, 1980)
Any matrix with m distinct eigenvalues can be written in the form
A =mX
j=1
λj Pj + Dj
where, for Jordan curves Γj surrounding λj and no other eigenvalues,
I Pj =1
2πi
ZΓj
(z − A)−1 dz is a spectral projector;
I Dj =1
2πi
ZΓj
(z − λj )(z − A)−1 dz is nilpotent;
I Pj A = APj = λj Pj ;
I Pj Pk = 0 if j 6= k;
I Dj = 0 if λj is not defective.
The resolvent plays a fundamental role in the structure of the matrix A.
![Page 117: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/117.jpg)
Functions of a Matrix
If A is diagonalizable (i.e., no defective eigenvalues), then
A =mX
j=1
λj Pj .
If f is any function that is analytic on σ(A) and on/inside all contours Γj , then
f (A) =mX
j=1
f (λj )Pj .
For all matrices, we have a more general formula.
Theorem (Cauchy Integral Formula for Matrices)
Let Γ be a finite union of Jordan curves containing σ(A) in its interior, andsuppose f is a function analytic on Γ and its interior. Then
f (A) =1
2πi
ZΓ
f (z)(z − A)−1 dz .
![Page 118: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/118.jpg)
Resolvent Bounds
Suppose the eigenvalues of A ∈ Cn×n are distinct.
Apply the previous formula to f (ζ) = (z − ζ)−1:
f (A) = (z − A)−1 =nX
j=1
1
z − λjPj .
If λj has right eigenvector vj and left eigenvector uj , then
Pj =vj u∗j
u∗j vj
and the norm of Pj is
κ(λj ) := ‖Pj‖ =‖vj‖‖uj‖|u∗j vj |
the condition number of the eigenvalue λj .
Hence for z near λj ,
‖(z − A)−1‖ ≈ κ(λj )
|z − λj |.
![Page 119: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/119.jpg)
Containment Regions for Pseudospectra
Hence for small ε > 0 and diagonalizable matrices, we can approximate
σε(A) ≈ ∪nj=1λj + ∆εκ(λj ),
where ∆r := {z ∈ C : |z | < r}.
Theorem (Bauer–Fike, 1963)
If A ∈ C n×n is diagonalizable, then for all ε > 0,
σε(A) ⊆n[
j=1
λj + ∆nεκ(λj ).
Unlike the earlier version of Bauer–Fike, the radii of the disks vary with j .
Near a defective eigenvalue of index d , Rellich’s perturbation theory requiresthat the pseudospectrum behave like a disk whose radius scales like ε1/d asε→ 0.
![Page 120: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/120.jpg)
Bounds on ‖f (A)‖
Suppose A is diagonalizable, A = VΛV−1. Then
f (A) =nX
j=1
f (λj )Pj = Vf (Λ)V−1.
This immediately suggests several upper bounds on ‖f (A)‖:
‖f (A)‖ = ‖Vf (Λ)V−1‖ ≤ ‖V‖‖V−1‖ maxλ∈σ(A)
|f (λ)|
= κ(V) maxλ∈σ(A)
|f (λ)|;
‖f (A)‖ ≤nX
j=1
|f (λj )|‖Pj‖ =nX
j=1
κ(λj )|f (λj )|.
We seek bounds that provide a more flexible way of handling nonnormality.
![Page 121: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/121.jpg)
Pseudospectral Bounds on ‖f (A)‖
Theorem (Cauchy Integral Formula for Matrices)
f (A) =1
2πi
ZΓ
f (z)(z − A)−1 dz .
Take norms of the expression for f (A):
‖f (A)‖ =‚‚‚ 1
2πi
ZΓ
f (z)(z − A)−1 dz‚‚‚ ≤ 1
2π
ZΓ
|f (z)‖(z − A)−1‖|dz |.
Now pick Γ to be the boundary of σε(A)
‖f (A)‖ ≤ 1
2π
Z∂σε
|f (z)‖(z − A)−1‖|dz |
=1
2πε
Z∂σε
|f (z)||dz |
≤ 1
2πεsup
z∈σε(A)
|f (z)|Z∂σε
|dz | ≤ Lε2πε
supz∈σε(A)
|f (z)|
where Lε denotes the arc-length of ∂σε(A) [Trefethen 1990].
![Page 122: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/122.jpg)
Pseudospectral Bounds on ‖f (A)‖
Theorem (Cauchy Integral Formula for Matrices)
f (A) =1
2πi
ZΓ
f (z)(z − A)−1 dz .
Take norms of the expression for f (A):
‖f (A)‖ =‚‚‚ 1
2πi
ZΓ
f (z)(z − A)−1 dz‚‚‚ ≤ 1
2π
ZΓ
|f (z)‖(z − A)−1‖|dz |.
Now pick Γ to be the boundary of σε(A)
‖f (A)‖ ≤ 1
2π
Z∂σε
|f (z)‖(z − A)−1‖|dz |
=1
2πε
Z∂σε
|f (z)||dz |
≤ 1
2πεsup
z∈σε(A)
|f (z)|Z∂σε
|dz | ≤ Lε2πε
supz∈σε(A)
|f (z)|
where Lε denotes the arc-length of ∂σε(A) [Trefethen 1990].
![Page 123: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/123.jpg)
Pseudospectral Bounds on ‖f (A)‖
Theorem
Let f be analytic on σε(A) for some ε > 0. Then
‖f (A)‖ ≤ Lε2πε
supz∈σε(A)
|f (z)|,
where Lε denotes the countour length of the boundary of σε(A).
Some key observations:
I This should be regarded as a family of bounds that vary with ε;
I The best choice for ε will depend on the problem;
I Sometimes it is excellent; usually it is decent; on occasion it is poor;
I The choice of ε has nothing to do with rounding errors; do not expect thebound to be most descriptive when ε = εmach or ε = ‖A‖εmach.
![Page 124: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/124.jpg)
GMRES Convergence
Example
We wish to use the pseudospectral analysis just presented to bound the normof the residual in the GMRES algorithm for iteratively solving Ax = b.
GMRES produces optimal iterates xk whose residuals rk = b− Axk satisfy
‖rk‖ ≤ minp∈Pkp(0)=1
‖p(A)b‖,
where Pk denotes the set of polynomials of degree k or less.
Applying the previous analysis yields
‖rk‖‖b‖ ≤
Lε2πε
minp∈Pkp(0)=1
supz∈σε(A)
|p(z)|.
Different ε values give the best bounds at different stages of convergence.
Illustrations and applications: [Trefethen 1990; E. 2000; Sifuentes, E., Morgan 2011].
![Page 125: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/125.jpg)
GMRES Convergence: Example
Pseudospectral bound for convection–diffusion problem, n = 2304.
ε = 10−3.5, 10−3, 10−3.5, . . . , 10−13
�ε = 10−3.5
-ε =10−13
Bound σε(A) with a circle centered at c, use pk (z) = (1− z/c)k .
![Page 126: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/126.jpg)
Bounds on the Matrix Exponential
To understand behavior of x′(t) = Ax(t) [and LTI control systems], we wish touse pseudospectra to bound ‖etA‖.
Definition
The spectral abscissa is the rightmost point in the spectrum:
α(A) := maxλ∈σ(A)
Re z .
The ε-pseudospectral abscissa is the supremum of the real parts of z ∈ σε(A):
αε(A) := supz∈σε(A)
Re z .
Applying the Cauchy integral bound to f (z) = etz gives an upper bound.
Theorem (Upper Bound)
For any A ∈ C n×n and ε > 0,
‖etA‖ ≤ Lε2πε
etαε(A),
where Lε denotes the contour length of the boundary of σε(A).
![Page 127: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/127.jpg)
Bounds on the Matrix Exponential
To understand behavior of x′(t) = Ax(t) [and LTI control systems], we wish touse pseudospectra to bound ‖etA‖.
Definition
The spectral abscissa is the rightmost point in the spectrum:
α(A) := maxλ∈σ(A)
Re z .
The ε-pseudospectral abscissa is the supremum of the real parts of z ∈ σε(A):
αε(A) := supz∈σε(A)
Re z .
Applying the Cauchy integral bound to f (z) = etz gives an upper bound.
Theorem (Upper Bound)
For any A ∈ C n×n and ε > 0,
‖etA‖ ≤ Lε2πε
etαε(A),
where Lε denotes the contour length of the boundary of σε(A).
![Page 128: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/128.jpg)
Upper Bound on Matrix Powers
A =
26666664
−1 2
−1. . .
. . . 2−1 2
−1
37777775 ∈ C20×20.
Question: can you estimate σε(A) for this matrix?
![Page 129: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/129.jpg)
Upper Bound on Matrix Powers
A =
26666664
−1 2
−1. . .
. . . 2−1 2
−1
37777775 ∈ C20×20.
Question: can you estimate σε(A) for this matrix?
ε = 10−20 ε = 10−1
![Page 130: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/130.jpg)
Lower Bounds on the Matrix Exponential
We would like to guarantee the potential for transient growth.
Theorem (Lower bound on etA)
Suppose α(A) < 0. Then for all ε > 0,
supt≥0‖etA‖ ≥ αε(A)
ε.
Proof.
Z ∞0
etAe−ztdt = (z − A)−1.
Suppose ‖etA‖ ≤ M for all t ≥ 0. Then for any z ∈ σε(A), Re z > 0:
1
ε< ‖(z − A)−1‖ =
‚‚‚Z ∞0
etAe−zt dt‚‚‚
≤Z ∞
0
‖etA‖|e−zt | dt ≤ M
Z ∞0
e−(Re z)t dt =M
Re z.
Hence M ≥ (Re z)/ε. Take the sup over all z ∈ σε(A) to get the bound.
![Page 131: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/131.jpg)
Lower Bounds on the Matrix Exponential
We would like to guarantee the potential for transient growth.
Theorem (Lower bound on etA)
Suppose α(A) < 0. Then for all ε > 0,
supt≥0‖etA‖ ≥ αε(A)
ε.
Proof. Recall the formula for the Laplace transform of etα: For s > α,Z ∞0
etαe−stdt =1
s − α.
This formula generalizes for matrices, e.g., when α(A) < 0 and Re z > 0:Z ∞0
etAe−ztdt = (z − A)−1.
Suppose ‖etA‖ ≤ M for all t ≥ 0. Then for any z ∈ σε(A), Re z > 0:
1
ε< ‖(z − A)−1‖ =
‚‚‚Z ∞0
etAe−zt dt‚‚‚
≤Z ∞
0
‖etA‖|e−zt | dt ≤ M
Z ∞0
e−(Re z)t dt =M
Re z.
Hence M ≥ (Re z)/ε. Take the sup over all z ∈ σε(A) to get the bound.
![Page 132: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/132.jpg)
Lower Bounds on the Matrix Exponential
We would like to guarantee the potential for transient growth.
Theorem (Lower bound on etA)
Suppose α(A) < 0. Then for all ε > 0,
supt≥0‖etA‖ ≥ αε(A)
ε.
Proof. Z ∞0
etAe−ztdt = (z − A)−1.
Suppose ‖etA‖ ≤ M for all t ≥ 0. Then for any z ∈ σε(A), Re z > 0:
1
ε< ‖(z − A)−1‖ =
‚‚‚Z ∞0
etAe−zt dt‚‚‚
≤Z ∞
0
‖etA‖|e−zt | dt ≤ M
Z ∞0
e−(Re z)t dt =M
Re z.
Hence M ≥ (Re z)/ε. Take the sup over all z ∈ σε(A) to get the bound.
![Page 133: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/133.jpg)
Lower Bound on the Matrix Exponential
A =
26666664
−1 2
−1. . .
. . . 2−1 2
−1
37777775 ∈ C20×20.
![Page 134: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/134.jpg)
Lower Bound on the Matrix Exponential
A =
26666664
−1 2
−1. . .
. . . 2−1 2
−1
37777775 ∈ C20×20.
ε = 10−5
ε = 10−1
![Page 135: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/135.jpg)
Matrix Powers
Definition
The spectral radius is the largest point in the spectrum:
ρ(A) := maxλ∈σ(A)
|z |.
The ε-pseudospectral radius is the supremum of magnitudes of points in σε(A):
ρε(A) := supz∈σε(A)
|z |.
Theorem (Upper Bound)
For any A ∈ C n×n and ε > 0,
‖Ak‖ ≤ ρε(A)k+1
ε
Proof: Apply the Cauchy integral bound, taking Γ to be the circle of radiusρε(A) centered at the origin.
![Page 136: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/136.jpg)
Matrix Powers
Definition
The spectral radius is the largest point in the spectrum:
ρ(A) := maxλ∈σ(A)
|z |.
The ε-pseudospectral radius is the supremum of magnitudes of points in σε(A):
ρε(A) := supz∈σε(A)
|z |.
Theorem (Upper Bound)
For any A ∈ C n×n and ε > 0,
‖Ak‖ ≤ ρε(A)k+1
ε
Proof: Apply the Cauchy integral bound, taking Γ to be the circle of radiusρε(A) centered at the origin.
![Page 137: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/137.jpg)
Upper Bound on Matrix Powers
A =
26666664
0 2
0. . .
. . . 20 2
0
37777775 ∈ C20×20.
A20 = 0, but lower powers show transient growth.
![Page 138: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/138.jpg)
Upper Bound on Matrix Powers
A =
26666664
0 2
0. . .
. . . 20 2
0
37777775 ∈ C20×20.
A20 = 0, but lower powers show transient growth.
ε = 100
ε = 10−20
![Page 139: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/139.jpg)
Lower Bounds on Matrix Powers
Theorem (Lower bound on power growth)
Suppose ρ(A) < 1. Then for all ε > 0,
supk≥0‖Ak‖ ≥ ρε(A)− 1
ε.
Proof. Since ρ(A) < 1, Ak → 0 as k →∞. Let M denote the maximum valueof ‖Ak‖, k ≥ 0, and suppose z ∈ σε(A) for |z | > 1. Then
1
ε< ‖(z − A)−1‖ =
‚‚‚ 1
|z |
“1 +
1
|z |A +1
|z |2 A2 + · · ·”‚‚‚
≤ 1
|z |
“M +
M
|z | +M
|z |2 + · · ·”
=M
|z |1
1− 1/|z | =M
|z | − 1.
Rearrange to obtain M ≥ (|z | − 1)/ε for all z ∈ σε(A) with |z | > 1.Take the supremum of |z | over all z ∈ σε(A) to get the result.
![Page 140: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/140.jpg)
Lower Bounds on Matrix Powers
Theorem (Lower bound on power growth)
Suppose ρ(A) < 1. Then for all ε > 0,
supk≥0‖Ak‖ ≥ ρε(A)− 1
ε.
Proof. Since ρ(A) < 1, Ak → 0 as k →∞. Let M denote the maximum valueof ‖Ak‖, k ≥ 0, and suppose z ∈ σε(A) for |z | > 1. Then
1
ε< ‖(z − A)−1‖ =
‚‚‚ 1
|z |
“1 +
1
|z |A +1
|z |2 A2 + · · ·”‚‚‚
≤ 1
|z |
“M +
M
|z | +M
|z |2 + · · ·”
=M
|z |1
1− 1/|z | =M
|z | − 1.
Rearrange to obtain M ≥ (|z | − 1)/ε for all z ∈ σε(A) with |z | > 1.Take the supremum of |z | over all z ∈ σε(A) to get the result.
![Page 141: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/141.jpg)
Lower Bounds on Matrix Powers
Theorem (Lower bound on power growth)
Suppose ρ(A) < 1. Then for all ε > 0,
supk≥0‖Ak‖ ≥ ρε(A)− 1
ε.
Proof. Since ρ(A) < 1, Ak → 0 as k →∞. Let M denote the maximum valueof ‖Ak‖, k ≥ 0, and suppose z ∈ σε(A) for |z | > 1. Then
1
ε< ‖(z − A)−1‖ =
‚‚‚ 1
|z |
“1 +
1
|z |A +1
|z |2 A2 + · · ·”‚‚‚
≤ 1
|z |
“M +
M
|z | +M
|z |2 + · · ·”
=M
|z |1
1− 1/|z | =M
|z | − 1.
Rearrange to obtain M ≥ (|z | − 1)/ε for all z ∈ σε(A) with |z | > 1.Take the supremum of |z | over all z ∈ σε(A) to get the result.
![Page 142: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/142.jpg)
Lower Bounds on Matrix Powers
Theorem (Lower bound on power growth)
Suppose ρ(A) < 1. Then for all ε > 0,
supk≥0‖Ak‖ ≥ ρε(A)− 1
ε.
Proof. Since ρ(A) < 1, Ak → 0 as k →∞. Let M denote the maximum valueof ‖Ak‖, k ≥ 0, and suppose z ∈ σε(A) for |z | > 1. Then
1
ε< ‖(z − A)−1‖ =
‚‚‚ 1
|z |
“1 +
1
|z |A +1
|z |2 A2 + · · ·”‚‚‚
≤ 1
|z |
“M +
M
|z | +M
|z |2 + · · ·”
=M
|z |1
1− 1/|z | =M
|z | − 1.
Rearrange to obtain M ≥ (|z | − 1)/ε for all z ∈ σε(A) with |z | > 1.Take the supremum of |z | over all z ∈ σε(A) to get the result.
![Page 143: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/143.jpg)
Lower Bound on Matrix Powers
A =
26666664
0 2
0. . .
. . . 20 2
0
37777775 ∈ C20×20.
![Page 144: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/144.jpg)
Lower Bound on Matrix Powers
A =
26666664
0 2
0. . .
. . . 20 2
0
37777775 ∈ C20×20.
ε = 10−5.5
ε = 10−5.75
ε = 10−4
ε = 10−2
ε = 100
![Page 145: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/145.jpg)
Pseudospectra are not a Panacea
(in the 2-norm)
Key question: “Do pseudospectra determine behavior of a matrix?”[Greenbaum & Trefethen, 1993].
Greenbaum and Trefethen define “behavior” to mean “norms of polynomials”.
They prove that pseudospectra do not determine behavior.
A1 =
2666640 1
0 10
0 00
377775 , A2 =
2666640 1
0 10
0 α0
377775 .
If α ∈ (1,√
2], then σε(A1) = σε(A2) for all ε > 0,
1 = ‖A1‖ 6= ‖A2‖ =√
2.
Ransford et al. [2007, 2009] have gone on to prove a host of similarlypessimistic results along these lines, e.g., relating ‖Ak
1‖ to ‖Ak2‖,
even for matrices with superidentical pseudospectra.
However: Greenbaum and Trefethen [1993] do show that pseudospectradetermine behavior in the Frobenius norm (σε(A) defined via resolvent norms).
![Page 146: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/146.jpg)
Pseudospectra are not a Panacea (in the 2-norm)
Key question: “Do pseudospectra determine behavior of a matrix?”[Greenbaum & Trefethen, 1993].
Greenbaum and Trefethen define “behavior” to mean “norms of polynomials”.
They prove that pseudospectra do not determine behavior.
A1 =
2666640 1
0 10
0 00
377775 , A2 =
2666640 1
0 10
0 α0
377775 .
If α ∈ (1,√
2], then σε(A1) = σε(A2) for all ε > 0,
1 = ‖A1‖ 6= ‖A2‖ =√
2.
Ransford et al. [2007, 2009] have gone on to prove a host of similarlypessimistic results along these lines, e.g., relating ‖Ak
1‖ to ‖Ak2‖,
even for matrices with superidentical pseudospectra.
However: Greenbaum and Trefethen [1993] do show that pseudospectradetermine behavior in the Frobenius norm (σε(A) defined via resolvent norms).
![Page 147: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/147.jpg)
Complex versus Real Perturbations
In many applications A contains only real entries; uncertainties in physicalparameters will only induce real perturbations.
Perhaps instead of the usual definition
σε(A) = {z ∈ C : z ∈ σ(A + E) for some E ∈ Cn×n with ‖E‖ < ε}
we should have
σε(A) = {z ∈ C : z ∈ σ(A + E) for some E ∈ Rn×n with ‖E‖ < ε}
cf. [Hinrichsen & Pritchard, 1990, 1992, 2005].
Similarly, one can study structured perturbations that preserve other properties:stochasticity, nonnegativity, symplecticity, Toeplitz or companion structure. . .
For analysis involving more sophisticated structured perturbations,see the work of Michael Karow et al. [2003 – 2011. . . ]
![Page 148: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/148.jpg)
Complex versus Real Perturbations
The size of a complex perturbation required to make z an eigenvalue of A is:
dC(A, z) =“σ1
`(z − A)−1´”−1
.
The size of a real perturbation required to make z an eigenvalue of A is:
dR(A, z) =
inf
γ∈(0,1]σ2
"Re(z − A)−1 −γ Im(z − A)−1
γ−1Im(z − A)−1 Re(z − A)−1
#!!−1
[Qiu, Berhardsson, Rantzer, Davison, Young, Doyle, 1995].
The real structured pseudospectrum can be computed via the definition
σRε (A) = {z ∈ C : dR(A, z) < ε}
[Karow, 2003].
![Page 149: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/149.jpg)
Complex versus Real Perturbations: Example 1
Consider the following Toeplitz matrix studied by Demmel [1987]:
A =
266664−1 −M −M2 −M3 −M4
0 −1 −M −M2 −M3
0 0 −1 −M −M2
0 0 0 −1 −M0 0 0 0 −1
377775with M = 10.
The matrix is stable but small perturbations can move eigenvalues significantly.
How do the real structured pseudospectra compareto the (unstructured) pseudospectra?
![Page 150: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/150.jpg)
Complex versus Real Perturbations: Example 1
Complex and real pseudospectra for 5× 5 matrix, ε = 10−2, 10−3, . . . , 10−8
−3 −2 −1 0 1
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
−8
−7
−6
−5
−4
−3
−2
complex
![Page 151: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/151.jpg)
Complex versus Real Perturbations: Example 1
Complex and real pseudospectra for 5× 5 matrix, ε = 10−2, 10−3, . . . , 10−8
−3 −2 −1 0 1
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
−8
−7
−6
−5
−4
−3
−2
−3 −2 −1 0 1
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
−8
−7
−6
−5
−4
−3
−2
complex real
![Page 152: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/152.jpg)
Complex versus Real Perturbations: Example 1
Complex and real pseudospectra for 5× 5 matrix, ε = 10−4
−3 −2 −1 0 1
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
−4
−3 −2 −1 0 1
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
−4
complex real
![Page 153: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/153.jpg)
Complex versus Real Perturbations: Example 1
Complex and real pseudospectra for 5× 5 matrix, ε = 10−4
−3 −2 −1 0 1
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
−4
−3 −2 −1 0 1
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
−4
complex real
eigenvalues of 1000 random perturbations of size 10−4
![Page 154: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/154.jpg)
Complex versus Real Perturbations: Example 2
Real perturbations need not describe transient behavior of dynamical systems.
Consider the matrix
A =
»−1 M2
−1 −1
–,M ∈ R
with spectrum σ(A) = {−1± iM}. For M = 100:
![Page 155: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/155.jpg)
Complex versus Real Perturbations: Example 2
Real perturbations need not describe transient behavior of dynamical systems.
Consider the matrix
A =
»−1 M2
−1 −1
–,M ∈ R
with spectrum σ(A) = {−1± iM}. For M = 100:
−3 −2 −1 0 1
98
99
100
101
102
−2
−1.699
−1.398
−3 −2 −1 0 1
98
99
100
101
102
−2
−1.699
−1.398
complex real
![Page 156: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/156.jpg)
Complex versus Real Perturbations: Example 2
Real perturbations need not describe transient behavior of dynamical systems.
Consider the matrix
A =
»−1 M2
−1 −1
–,M ∈ R
with spectrum σ(A) = {−1± iM}. For M = 100:
−3 −2 −1 0 1
98
99
100
101
102
−1.398
−3 −2 −1 0 1
98
99
100
101
102
−1.398
complex real
eigenvalues of 1000 random perturbations of size 400/10001
![Page 157: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/157.jpg)
Complex versus Real Perturbations
Consider the dynamical system
x′(t) =
»−1 M2
−1 −1
–x(t)
with M = 100.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
20
40
60
80
100
t
‖etA‖
Real perturbations suggest this system is far from unstable,yet transient growth on the order of M is observed.
![Page 158: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/158.jpg)
4. Balanced Truncation, Lyapunov Equations
![Page 159: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/159.jpg)
Balanced Truncation and Lyapunov Equations
Consider the SISO linear dynamical system:
x′(t) = Ax(t) + bu(t)
y(t) = cx(t) + d u(t),
A ∈ Cn×n, b, cT ∈ Cn, d ∈ C. We assume that A is stable: α(A) < 0.
We wish to reduce the dimension of the dynamical system by projecting ontowell-chosen subspaces.
Balanced truncation: Change basis to match states that areeasy to reach and easy to observe,then project onto that prominent subspace.
Characterize how difficult it is to reach or observe a state via the infinitecontrollability and observability gramians P and Q:
P :=
Z ∞0
etAbb∗etA∗ dt, Q :=
Z ∞0
etA∗c∗cetA dt.
See, e.g., [Antoulas, 2005], Tuesday’s lectures. . . .
![Page 160: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/160.jpg)
Balanced Truncation and Lyapunov Equations
The gramians
P :=
Z ∞0
etAbb∗etA∗ dt, Q :=
Z ∞0
etA∗c∗cetA dt
(Hermitian positive definite, for a controllable and observable stable system)can be determined by solving the Lyapunov equations
AP + PA∗ = −bb∗, A∗Q + QA = −c∗c.
If x0 = 0, the minimum energy of u required to drive x to state bx is
bx∗P−1bx.Starting from x0 = bx with u(t) ≡ 0, the energy of output y is
bx∗Qbx.bx∗P−1bx: bx is hard to reach if it is rich in the lowest modes of P.bx∗Qbx: bx is hard to observe if it is rich in the lowest modes of Q.
Balanced truncation transforms the state space coordinate system to makethese two gramians the same, then it truncates the lowest modes.
![Page 161: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/161.jpg)
Balanced Truncation and Lyapunov Equations
Consider a generic coordinate transformation, for S invertible:
(Sx)′(t) = (SAS−1)(Sx(t)) + (Sb)u(t)
y(t) = (cS−1)(Sx(t)) + du(t), (Sx)(0) = Sx0.
With this transformation, the controllability and observability gramians arebP = SPS∗, bQ = S−∗QS−1.
For balancing, we seek S so that bP = bQ are diagonal.
Observation (How does nonnormality affect balancing?)
I σε/κ(S)(SAS−1) ⊆ σε(A) ⊆ σεκ(S)(SAS−1).
I The choice of internal coordinates will affect P, Q, . . .
I but not the Hankel singular values: bPbQ = SPQS−1,
I and not the transfer function:
d + (cS−1)(z − SAS−1)−1(Sb) = d + c(z − A)−1b,
I and not the system moments:
d = d , (cS−1)(Sb) = cb, (cS−1)(SAS−1)(Sb) = cAb, . . . .
![Page 162: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/162.jpg)
Balanced Truncation and Lyapunov Equations
Consider a generic coordinate transformation, for S invertible:
(Sx)′(t) = (SAS−1)(Sx(t)) + (Sb)u(t)
y(t) = (cS−1)(Sx(t)) + du(t), (Sx)(0) = Sx0.
With this transformation, the controllability and observability gramians arebP = SPS∗, bQ = S−∗QS−1.
For balancing, we seek S so that bP = bQ are diagonal.
Observation (How does nonnormality affect balancing?)
I σε/κ(S)(SAS−1) ⊆ σε(A) ⊆ σεκ(S)(SAS−1).
I The choice of internal coordinates will affect P, Q, . . .
I but not the Hankel singular values: bPbQ = SPQS−1,
I and not the transfer function:
d + (cS−1)(z − SAS−1)−1(Sb) = d + c(z − A)−1b,
I and not the system moments:
d = d , (cS−1)(Sb) = cb, (cS−1)(SAS−1)(Sb) = cAb, . . . .
![Page 163: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/163.jpg)
Balanced Truncation and Lyapunov Equations
Consider a generic coordinate transformation, for S invertible:
(Sx)′(t) = (SAS−1)(Sx(t)) + (Sb)u(t)
y(t) = (cS−1)(Sx(t)) + du(t), (Sx)(0) = Sx0.
With this transformation, the controllability and observability gramians arebP = SPS∗, bQ = S−∗QS−1.
For balancing, we seek S so that bP = bQ are diagonal.
Observation (How does nonnormality affect balancing?)
I σε/κ(S)(SAS−1) ⊆ σε(A) ⊆ σεκ(S)(SAS−1).
I The choice of internal coordinates will affect P, Q, . . .
I but not the Hankel singular values: bPbQ = SPQS−1,
I and not the transfer function:
d + (cS−1)(z − SAS−1)−1(Sb) = d + c(z − A)−1b,
I and not the system moments:
d = d , (cS−1)(Sb) = cb, (cS−1)(SAS−1)(Sb) = cAb, . . . .
![Page 164: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/164.jpg)
Decay of Singular Values of Lyapunov Solutions
Consider the Lyapunov equation
AX + XA∗ = −bb∗.
for stable A and controllable (A, b).
In many cases (since the right-hand side bb∗ is low-rank), the singular values ofX decay rapidly [Gudmundsson and Laub 1994; Penzl 2000].
kth
sin
gu
lar
valu
eo
fX
eigenvalues of A
α=
5/
2
α=
1/
2
α = 5/2
α = 1/2
k
Normal example: A = tridiag(−1, α, 1) with spectrum
σ(A) ⊆ {α + iy : y ∈ [−2, 2]}.
Convergence slows as α ↑ 0, i.e., eigenvalues move toward imaginary axis.
![Page 165: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/165.jpg)
Decay of Singular Values of Lyapunov Solutions
Consider the Lyapunov equation
AX + XA∗ = −bb∗.
for stable A and controllable (A, b).
In many cases (since the right-hand side bb∗ is low-rank), the singular values ofX decay rapidly [Gudmundsson and Laub 1994; Penzl 2000].
kth
sin
gu
lar
valu
eo
fX
eigenvalues of A
α=
5/
2
α=
1/
2
α = 5/2
α = 1/2
k
Normal example: A = tridiag(−1, α, 1) with spectrum
σ(A) ⊆ {α + iy : y ∈ [−2, 2]}.
Convergence slows as α ↑ 0, i.e., eigenvalues move toward imaginary axis.
![Page 166: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/166.jpg)
Decay of Singular Values of Lyapunov Solutions
Let sk denote the kth singular value of X, sk ≥ sk+1.
I For Hermitian A, bounds on sk/s1 have been derived byPenzl [2000], Sabino [2006] (see Ellner & Wachspress [1991]).
I For diagonalizable A = VΛV−1, Antoulas, Sorensen, and Zhou [2002] prove
sk+1 ≤ κ(V)2δk+1‖b‖2(N − k)2, δk =−1
2Reλk
k−1Yj=1
|λk − λj |2
|λk + λj |2.
The κ(V)2 term imposes a significant penalty for nonnormality;cf. [Truhar, Tomljanovic, Li 2009].
We seek a bound that gives a more flexible approach, by enlargingthe set over which we study rational functions like δk .
![Page 167: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/167.jpg)
Decay of Singular Values of Lyapunov Solutions
I sk+1 ≤ κ(V)2δk+1‖b‖2(N − k)2, δk =−1
2Reλk
k−1Yj=1
|λk − λj |2
|λk + λj |2.
I The ADI iteration [Wachspress 1988; Penzl 2000] constructs a rank-kapproximation Xk to X that satisfies
X− Xk = φ(A)Xφ(A)∗,
with X0 = 0 and
φ(z) =kY
j=1
µj − z
µj + z.
Using the fact that rank(Xk ) ≤ k, we obtain the bound
sk+1
s1≤ min
µ1,...,µk
‖φ(A)‖22 ≤
„Lε
2πε
«2
minµ1,...,µk
supz∈σε(A)
kYj=1
|µj − z ||µj + z | ,
where Lε is the boundary length of a contour enclosing σε(A)[Beattie, E., Sabino; cf. Beckermann (2004)].
![Page 168: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/168.jpg)
Examples of the Decay Bound Based on Pseudospectra
A =
26666664
−1 1/2
−1. . .
. . . 1/2−1 1/2
−1
37777775 ∈ C16×16
ε = 10−5
ε = 10−0.5
k
sk
![Page 169: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/169.jpg)
Examples of the Decay Bound Based on Pseudospectra
A =
26666664
−1 2
−1. . .
. . . 2−1 2
−1
37777775 ∈ C32×32
−3 −2 −1 0 1
−2
−1
0
1
2
−28
−26
−24
−22
−20
−18
−16
−14
−12
−10
−8
Spectrum, pseudospectra, and numerical range (boundary = dashed line)
![Page 170: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/170.jpg)
Examples of the Decay Bound Based on Pseudospectra
A =
26666664
−1 2
−1. . .
. . . 2−1 2
−1
37777775 ∈ C32×32
ε = 10−28
ε = 10−10
k
sk
![Page 171: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/171.jpg)
Any Decay Possible for Any Spectrum
Theorem (Penzl, 2000)
Let A be stable and b some vector such that (A, b) is controllable. Given anyHermitian positive definite X, there exists some invertible matrix S such that
(SAS−1)X + X(SAS−1) = −(Sb)(Sb)∗.
Any prescribed singular value decay is possible for a matrix with anyeigenvalues.
Proof. The proof is a construction.
I Solve AY + YA∗ = −bb∗ for Y.(Y is Hermitian positive definite, since (A, b) controllable.)
I Set S := X1/2Y−1/2.I Notice that SYS∗ = X1/2Y−1/2 YY−1/2X1/2 = X.I Define bA := SAS−1, bb := Sb.
Now it is easy to verify that X solves the desired Lyapunov equation:bAX + XbA∗ = (SAS−1)(SYS∗) + (SYS∗)(S−∗AS∗)
= S(AY + YA)S∗ = −(Sb)(Sb)∗ = −bbbb∗.
![Page 172: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/172.jpg)
A Nonnormal Anomaly
The pseudospectral bound and the bound of Antoulas, Sorensen, and Zhouboth predict that the decay rate slows as nonnormality increases.
However, for the solutions to Lyapunov equations this intuition can be wrong[Sabino 2006].
Consider
A =
»−1 α0 −1
–, b =
»t1
–, t ∈ R.
As α grows, A’s departure from normality grows.All bounds suggest that the ‘decay’ rate should worsen as α increases.
The Lyapunov equation AX + XA∗ = −bb∗ has the solution
X =1
4
»2t2 + 2αt + α2 α + 2t
α + 2t 2
–.
For each α, we wish to pick t to maximize the ‘decay’, i.e., s2/s1.This is accomplished for t = −α/2, giving
s2
s1=
α2/4, 0 < α ≤ 2;4/α2, 2 ≤ α.
![Page 173: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/173.jpg)
A Nonnormal Anomaly
The pseudospectral bound and the bound of Antoulas, Sorensen, and Zhouboth predict that the decay rate slows as nonnormality increases.
However, for the solutions to Lyapunov equations this intuition can be wrong[Sabino 2006].
Consider
A =
»−1 α0 −1
–, b =
»t1
–, t ∈ R.
As α grows, A’s departure from normality grows.All bounds suggest that the ‘decay’ rate should worsen as α increases.
The Lyapunov equation AX + XA∗ = −bb∗ has the solution
X =1
4
»2t2 + 2αt + α2 α + 2t
α + 2t 2
–.
For each α, we wish to pick t to maximize the ‘decay’, i.e., s2/s1.This is accomplished for t = −α/2, giving
s2
s1=
α2/4, 0 < α ≤ 2;4/α2, 2 ≤ α.
![Page 174: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/174.jpg)
A Nonnormal Anomaly
The pseudospectral bound and the bound of Antoulas, Sorensen, and Zhouboth predict that the decay rate slows as nonnormality increases.
However, for the solutions to Lyapunov equations this intuition can be wrong[Sabino 2006].
Consider
A =
»−1 α0 −1
–, b =
»t1
–, t ∈ R.
As α grows, A’s departure from normality grows.All bounds suggest that the ‘decay’ rate should worsen as α increases.
The Lyapunov equation AX + XA∗ = −bb∗ has the solution
X =1
4
»2t2 + 2αt + α2 α + 2t
α + 2t 2
–.
For each α, we wish to pick t to maximize the ‘decay’, i.e., s2/s1.
This is accomplished for t = −α/2, giving
s2
s1=
α2/4, 0 < α ≤ 2;4/α2, 2 ≤ α.
![Page 175: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/175.jpg)
A Nonnormal Anomaly
The pseudospectral bound and the bound of Antoulas, Sorensen, and Zhouboth predict that the decay rate slows as nonnormality increases.
However, for the solutions to Lyapunov equations this intuition can be wrong[Sabino 2006].
Consider
A =
»−1 α0 −1
–, b =
»t1
–, t ∈ R.
As α grows, A’s departure from normality grows.All bounds suggest that the ‘decay’ rate should worsen as α increases.
The Lyapunov equation AX + XA∗ = −bb∗ has the solution
X =1
4
»2t2 + 2αt + α2 α + 2t
α + 2t 2
–.
For each α, we wish to pick t to maximize the ‘decay’, i.e., s2/s1.This is accomplished for t = −α/2, giving
s2
s1=
α2/4, 0 < α ≤ 2;4/α2, 2 ≤ α.
![Page 176: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/176.jpg)
Jordan block: 2 × 2 case, numerical illustration
0 1 2 3 4 50
0.25
0.5
0.75
1
wor
st c
ase
ratio
α
Red line:s2
s1=
α2/4, 0 < α ≤ 2;4/α2, 2 ≤ α.
Blue dots: s2/s1 for random b and 2000 α values.
The effect of nonnormality on Lyapunov solutionsremains only partially understood.
![Page 177: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/177.jpg)
Jordan block: 2 × 2 case, numerical illustration
0 1 2 3 4 50
0.25
0.5
0.75
1
wor
st c
ase
ratio
α
Red line:s2
s1=
α2/4, 0 < α ≤ 2;4/α2, 2 ≤ α.
Blue dots: s2/s1 for random b and 2000 α values.
The effect of nonnormality on Lyapunov solutionsremains only partially understood.
![Page 178: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/178.jpg)
5. Moment Matching Model Reduction
![Page 179: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/179.jpg)
Krylov methods for moment matching model reduction
We now turn to a model reduction approach where nonnormality plays a crucialrole. Once again, begin with the SISO system
x′(t) = Ax(t) + bu(t)
y(t) = cx(t) + du(t),
with A ∈ Cn×n and b, cT ∈ Cn and initial condition x(0) = x0.
Compress the state space to dimension k � n via a projection method:
bx′(t) = bAbx(t) + bbu(t)by(t) = bcbx(t) + du(t),
where bA = W∗AV ∈ Ck×k , bb = W∗b ∈ Ck×1, bc = cV ∈ C1×k
for some V,W ∈ Cn×k with W∗V = I.
The matrices V and W are constructed by a Krylov subspace method.
![Page 180: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/180.jpg)
Krylov methods for moment matching model reduction
bA = W∗AV ∈ Ck×k , bb = W∗b ∈ Ck×1, bc = cV ∈ C1×k
Arnoldi Reduction
If V = W and the columns of V span the kth Krylov subspace,
Ran(V) = Kk (A, b) = span{b,Ab, . . . ,Ak−1b},
then the reduced model matches k moments of the system:
bcbAjbb = cAj b, j = 0, . . . , k − 1.
Bi-Lanczos Reduction
If the columns and V and W span the Krylov subspaces
Ran(V) = Kk (A, b) = span{b,Ab, . . . ,Ak−1b}Ran(W) = Kk (A∗, c∗) = span{c∗,A∗c∗, . . . , (A∗)k−1c∗},
then the reduced model matches 2k moments of the system:
bcbAjbb = cAj b, j = 0, . . . , 2k − 1.
![Page 181: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/181.jpg)
Stability of Reduced Order Models
Question
Does the reduced model inherit properties of the original system?
Properties include stability, passivity, second-order structure, etc.
In this lecture we are concerned with stability – and, more generally, thebehavior of eigenvalues of the reduced matrix bA.
Observation
For Arnoldi reduction, the eigenvalues of bA are contained in the numerical range
W (A) = {x∗Ax : ‖x‖ = 1} .
Proof: If bAz = θz for ‖z‖ = 1, then θ = z∗V∗AV∗z = (Vz)∗A(Vz), where‖Vz‖ = ‖z‖ = 1
No such bound for Bi-Lanczos: the decomposition may not even exist.For the famous ‘CD Player’ model, cb = 0: method breaks down at first step.
![Page 182: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/182.jpg)
Stability of Reduced Order Models
Question
Does the reduced model inherit properties of the original system?
Properties include stability, passivity, second-order structure, etc.
In this lecture we are concerned with stability – and, more generally, thebehavior of eigenvalues of the reduced matrix bA.
Observation
For Arnoldi reduction, the eigenvalues of bA are contained in the numerical range
W (A) = {x∗Ax : ‖x‖ = 1} .
Proof: If bAz = θz for ‖z‖ = 1, then θ = z∗V∗AV∗z = (Vz)∗A(Vz), where‖Vz‖ = ‖z‖ = 1
No such bound for Bi-Lanczos: the decomposition may not even exist.For the famous ‘CD Player’ model, cb = 0: method breaks down at first step.
![Page 183: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/183.jpg)
Stability of Reduced Order Models
Question
Does the reduced model inherit properties of the original system?
Properties include stability, passivity, second-order structure, etc.
In this lecture we are concerned with stability – and, more generally, thebehavior of eigenvalues of the reduced matrix bA.
Observation
For Arnoldi reduction, the eigenvalues of bA are contained in the numerical range
W (A) = {x∗Ax : ‖x‖ = 1} .
Proof: If bAz = θz for ‖z‖ = 1, then θ = z∗V∗AV∗z = (Vz)∗A(Vz), where‖Vz‖ = ‖z‖ = 1
No such bound for Bi-Lanczos: the decomposition may not even exist.For the famous ‘CD Player’ model, cb = 0: method breaks down at first step.
![Page 184: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/184.jpg)
Stability and Bi-Lanczos
For Bi-Lanczos, the stability question is rather more subtle.
For example, if b and c are nearly orthogonal,the (1,1) entry in bA = W∗AV will generally be very large.
Given a fixed b, one has much freedom to rig poor results via c:
[A]ny three-term recurrence (run for no more than (n +2)/2 steps,where n is the size of the matrix) is the two-sided Lanczos algo-rithm for some left starting vector. [Greenbaum, 1998]
![Page 185: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/185.jpg)
Example: Arnoldi Reduction for Normal versus Nonnormal
The eigenvalues of bA are known as Ritz values.
normal matrix nonnormal matrix
Eigenvalues (•), Ritz values (◦), and numerical range for isospectral matrices.
![Page 186: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/186.jpg)
A Remedy for Unstable Arnoldi Models?
One can counteract instability by restarting the Arnoldi algorithmto shift out unstable eigenvalues [Grimme, Sorensen, Van Dooren, 1994];cf. [Jaimoukha and Kasenally, 1997].
I bA := V∗AV has eigenvalues θ1, . . . , θk (Ritz values for A)
I Suppose θ1, . . . , θp are in the right half plane.
I Replace the starting vector b by the filtered vector
b+ = ψ(A)b, ψ(z) =
pYj=1
(z − θj ),
where the filter polynomial ψ “discourages” future Ritz values near theshifts θ1, . . . , θp.
I Build new matrices V, bA with starting vector b+ (implicit restart).
I Now modified moments, b∗ψ(A)∗Ajψ(A)b, will be matched.
Repeat this process until bA has no unstable modes.
![Page 187: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/187.jpg)
Matching the Moments of a Nonnormal Matrix
Model of flutter in a Boeing 767 from SLICOT (n = 55),stabilized by Burke, Lewis, Overton [2003].
−1000 −800 −600 −400 −200 0−600
−400
−200
0
200
400
600
σ(A)
−1 −0.5 0 0.5 1
x 107
−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1x 10
7
W (A)
![Page 188: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/188.jpg)
Matching the Moments of a Nonnormal Matrix
Model of flutter in a Boeing 767 from SLICOT (n = 55),stabilized by Burke, Lewis, Overton [2003].
−100 −50 0 50 100
−100
−50
0
50
100
−6.5
−6
−5.5
−5
−4.5
−4
−3.5
−3
−2.5
−2
σε(A) = {z ∈ C : ‖(z − A)−1‖ > 1/ε}
see [Trefethen & E. 2005]
![Page 189: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/189.jpg)
Reduction via Restarted Arnoldi
10−7
10−5
10−3
10−1
101
103
10−2
100
102
104
106
t
‖x(t
)‖
−100 −50 0 50 100
−100
−50
0
50
100
Transient behavior and spectrum of the original system
![Page 190: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/190.jpg)
Reduction via Restarted Arnoldi
10−7
10−5
10−3
10−1
101
103
10−2
100
102
104
106
t
‖x(t
)‖
−100 −50 0 50 100
−100
−50
0
50
100
Transient behavior and spectrum for reduced system, m = 40
![Page 191: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/191.jpg)
Reduction via Restarted Arnoldi
10−7
10−5
10−3
10−1
101
103
10−2
100
102
104
106
t
‖x(t
)‖
−100 −50 0 50 100
−100
−50
0
50
100
Transient behavior and spectrum for reduced system, m = 40,after one implicit restart
![Page 192: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/192.jpg)
Reduction via Restarted Arnoldi
10−7
10−5
10−3
10−1
101
103
10−2
100
102
104
106
t
‖x(t
)‖
−100 −50 0 50 100
−100
−50
0
50
100
Transient behavior and spectrum for reduced system, m = 40,after two implicit restarts
![Page 193: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/193.jpg)
Reduction via Restarted Arnoldi
10−7
10−5
10−3
10−1
101
103
10−2
100
102
104
106
t
‖x(t
)‖
−100 −50 0 50 100
−100
−50
0
50
100
Transient behavior and spectrum for reduced system, m = 40,after three implicit restarts
![Page 194: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/194.jpg)
Reduction via Restarted Arnoldi
10−7
10−5
10−3
10−1
101
103
10−2
100
102
104
106
t
‖x(t
)‖
−100 −50 0 50 100
−100
−50
0
50
100
Transient behavior and spectrum for reduced system, m = 40,after four implicit restarts
Reduced system is stable, but underestimatestransient growth by several orders of magnitude.
![Page 195: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/195.jpg)
Aggregate Filter Polynomials
Color indicates relative size of log10 |ψ(z)|.
first restart
![Page 196: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/196.jpg)
Aggregate Filter Polynomials
Color indicates relative size of log10 |ψ(z)|.
second restart
![Page 197: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/197.jpg)
Aggregate Filter Polynomials
Color indicates relative size of log10 |ψ(z)|.
third restart
![Page 198: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/198.jpg)
Aggregate Filter Polynomials
Color indicates relative size of log10 |ψ(z)|.
fourth restart
![Page 199: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/199.jpg)
Another Example: a Nonlinear Heat Equation
Linear models often arise as linearizations of nonlinear equations.
Consider the nonlinear heat equation on x ∈ [−1, 1] with u(−1, t) = u(1, t) = 0
ut(x , t) = νuxx (x , t)
+√νux (x , t) + 1
8u(x , t) + u(x , t)p
with ν > 0
and p > 1 [Demanet, Holmer, Zworski].
The linearization L, an advection–diffusion operator,
Lu = νuxx +√νux + 1
8u
has eigenvalues and eigenfunctions
λn = −1
8− n2π2ν
4, un(x) = e−x/(2
√ν) sin(nπx/2);
see, e.g., [Reddy & Trefethen 1994].
The linearized operator is stable for all ν > 0, but has interesting transients . . . .
![Page 200: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/200.jpg)
Another Example: a Nonlinear Heat Equation
Linear models often arise as linearizations of nonlinear equations.
Consider the nonlinear heat equation on x ∈ [−1, 1] with u(−1, t) = u(1, t) = 0
ut(x , t) = νuxx (x , t) +√νux (x , t)
+ 18u(x , t) + u(x , t)p
with ν > 0
and p > 1 [Demanet, Holmer, Zworski].
The linearization L, an advection–diffusion operator,
Lu = νuxx +√νux + 1
8u
has eigenvalues and eigenfunctions
λn = −1
8− n2π2ν
4, un(x) = e−x/(2
√ν) sin(nπx/2);
see, e.g., [Reddy & Trefethen 1994].
The linearized operator is stable for all ν > 0, but has interesting transients . . . .
![Page 201: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/201.jpg)
Another Example: a Nonlinear Heat Equation
Linear models often arise as linearizations of nonlinear equations.
Consider the nonlinear heat equation on x ∈ [−1, 1] with u(−1, t) = u(1, t) = 0
ut(x , t) = νuxx (x , t) +√νux (x , t) + 1
8u(x , t)
+ u(x , t)p
with ν > 0
and p > 1 [Demanet, Holmer, Zworski].
The linearization L, an advection–diffusion operator,
Lu = νuxx +√νux + 1
8u
has eigenvalues and eigenfunctions
λn = −1
8− n2π2ν
4, un(x) = e−x/(2
√ν) sin(nπx/2);
see, e.g., [Reddy & Trefethen 1994].
The linearized operator is stable for all ν > 0, but has interesting transients . . . .
![Page 202: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/202.jpg)
Another Example: a Nonlinear Heat Equation
Linear models often arise as linearizations of nonlinear equations.
Consider the nonlinear heat equation on x ∈ [−1, 1] with u(−1, t) = u(1, t) = 0
ut(x , t) = νuxx (x , t) +√νux (x , t) + 1
8u(x , t) + u(x , t)p
with ν > 0 and p > 1 [Demanet, Holmer, Zworski].
The linearization L, an advection–diffusion operator,
Lu = νuxx +√νux + 1
8u
has eigenvalues and eigenfunctions
λn = −1
8− n2π2ν
4, un(x) = e−x/(2
√ν) sin(nπx/2);
see, e.g., [Reddy & Trefethen 1994].
The linearized operator is stable for all ν > 0, but has interesting transients . . . .
![Page 203: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/203.jpg)
Another Example: a Nonlinear Heat Equation
Linear models often arise as linearizations of nonlinear equations.
Consider the nonlinear heat equation on x ∈ [−1, 1] with u(−1, t) = u(1, t) = 0
ut(x , t) = νuxx (x , t) +√νux (x , t) + 1
8u(x , t) + u(x , t)p
with ν > 0 and p > 1 [Demanet, Holmer, Zworski].
The linearization L, an advection–diffusion operator,
Lu = νuxx +√νux + 1
8u
has eigenvalues and eigenfunctions
λn = −1
8− n2π2ν
4, un(x) = e−x/(2
√ν) sin(nπx/2);
see, e.g., [Reddy & Trefethen 1994].
The linearized operator is stable for all ν > 0, but has interesting transients . . . .
![Page 204: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/204.jpg)
Nonnormality in the Linearization
−5 −4 −3 −2 −1 0 1−2.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
−6
−5
−4
−3
−2
−1
0
Spectrum, pseudospectra, and numerical range (L2 norm, ν = 0.02)
Transient growth can feed the nonlinearitycf. [Trefethen, Trefethen, Reddy, Driscoll 1993], . . . .
![Page 205: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/205.jpg)
Evolution of a Small Initial Condition
−1 −0.5 0 0.5 1−0.1
−0.05
0
0.05
0.1
0.15
0.2
0.25t = 0.0
x
u(x
,t)
Nonlinear model (blue) and linearization (black)
![Page 206: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/206.jpg)
Transient Behavior
0 20 40 60 80 1000
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
t
‖u(t
)‖L
2
Linearized system (black) and nonlinear system (dashed blue)
Nonnormal growth feeds the nonlinear instability.
![Page 207: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/207.jpg)
Transient Behavior: Reduction of the Linearized Model
−3 −2.5 −2 −1.5 −1 −0.5 0
x 104
−5
−4
−3
−2
−1
0
1
2
3
4
5
−5 −4 −3 −2 −1 0 1−3
−2
−1
0
1
2
3
entire spectrum zoom
Spectral discretization, n = 128 (black) and Arnoldi reduction, m = 10 (red).
[Many Ritz values capture spurious eigenvalues of the discretization of the left.]
![Page 208: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/208.jpg)
Transient Behavior: Reduction of the Linearized Model
0 20 40 60 80 1000
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
t
‖u(t
)‖L
2
Spectral discretization, n = 128 (black) and Arnoldi reduction, m = 10 (red).
![Page 209: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/209.jpg)
Transient Behavior: Reduction of the Linearized Model
−3 −2.5 −2 −1.5 −1 −0.5 0
x 104
−5
−4
−3
−2
−1
0
1
2
3
4
5
−5 −4 −3 −2 −1 0 1−3
−2
−1
0
1
2
3
entire spectrum zoom
Spectral discretization, n = 128 (black) and Arnoldi reduction, m = 10 (red)after a restart to remove the spurious eigenvalue.
[This effectively pushes Ritz values to the left.]
![Page 210: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/210.jpg)
Transient Behavior: Reduction of the Linearized Model
0 20 40 60 80 1000
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
t
‖u(t
)‖L
2
Spectral discretization, n = 128 (black) and Arnoldi reduction, m = 10 (red)after one restart to remove the spurious eigenvalue.
![Page 211: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/211.jpg)
Transient Behavior: Reduction of the Linearized Model
0 20 40 60 80 10010
−10
10−8
10−6
10−4
10−2
100
t
‖u(t
)‖L
2
Spectral discretization, n = 128 (black) and Arnoldi reduction, m = 10 (red)after one restart to remove the spurious eigenvalue.
![Page 212: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/212.jpg)
6. GEPs and DAEs
![Page 213: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/213.jpg)
Generalized Eigenvalue Problems
Problem
How should one adapt the definition of the ε-pseudospectrum to thegeneralized eigenvalue problem
Ax = λBx ?
Equivalent definitions of σε(A) lead to different meanings for σε(A,B).
I Approach 1: eigenvalues of perturbationsσε(A) = {z ∈ C : z ∈ σ(A + E) for some E ∈ Cn×n with ‖E‖ < ε}
I Approach 2: matrix behaviorσε(A) = {z ∈ C : ‖(z − A)−1‖ > 1/ε}
![Page 214: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/214.jpg)
Generalized Eigenvalue Problems
Problem
How should one adapt the definition of the ε-pseudospectrum to thegeneralized eigenvalue problem
Ax = λBx ?
Equivalent definitions of σε(A) lead to different meanings for σε(A,B).
I Approach 1: eigenvalues of perturbationsσε(A) = {z ∈ C : z ∈ σ(A + E) for some E ∈ Cn×n with ‖E‖ < ε}
I Approach 2: matrix behaviorσε(A) = {z ∈ C : ‖(z − A)−1‖ > 1/ε}
![Page 215: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/215.jpg)
GEPs: Eigenvalue Perturbation Approach
Approach 1: eigenvalues of perturbations
I σε(A) = {z ∈ C : z ∈ σ(A + E) for some E ∈ Cn×n with ‖E‖ < ε}
Fraysse, Gueury, Nicoud, Toumazou [1996] proposed:
σε(A,B) = {z ∈ C : (A + E0)x = z(B + E1)x for some
x 6= 0 and E0, E1 with ‖E0‖ < εα0, ‖E1‖ < εα1},
where, e.g., either α0 = α1 = 1, or α0 = ‖A‖ and α1 = ‖B‖.
I This has an equivalent resolvent-like formulation:
σε(A,B) = {z ∈ C : ‖(Bz − A)−1‖(α0 + α1|z |) > 1/ε}.
I Generalized to matrix polynomials by Tisseur & N. Higham [2001, 2002];see also Lancaster & Psarrakos [2005].
I Cf. [Boutry, Elad, Golub, Milanfar, 2005] for rectangular pencils.
![Page 216: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/216.jpg)
GEPs: Eigenvalue Perturbation Approach
Approach 1: eigenvalues of perturbations
I σε(A) = {z ∈ C : z ∈ σ(A + E) for some E ∈ Cn×n with ‖E‖ < ε}
Fraysse, Gueury, Nicoud, Toumazou [1996] proposed:
σε(A,B) = {z ∈ C : (A + E0)x = z(B + E1)x for some
x 6= 0 and E0, E1 with ‖E0‖ < εα0, ‖E1‖ < εα1},
where, e.g., either α0 = α1 = 1, or α0 = ‖A‖ and α1 = ‖B‖.
I This has an equivalent resolvent-like formulation:
σε(A,B) = {z ∈ C : ‖(Bz − A)−1‖(α0 + α1|z |) > 1/ε}.
I Generalized to matrix polynomials by Tisseur & N. Higham [2001, 2002];see also Lancaster & Psarrakos [2005].
I Cf. [Boutry, Elad, Golub, Milanfar, 2005] for rectangular pencils.
![Page 217: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/217.jpg)
GEPs: Eigenvalue Perturbation Approach
Approach 1: eigenvalues of perturbations
I σε(A) = {z ∈ C : z ∈ σ(A + E) for some E ∈ Cn×n with ‖E‖ < ε}
Fraysse, Gueury, Nicoud, Toumazou [1996] proposed:
σε(A,B) = {z ∈ C : (A + E0)x = z(B + E1)x for some
x 6= 0 and E0, E1 with ‖E0‖ < εα0, ‖E1‖ < εα1},
where, e.g., either α0 = α1 = 1, or α0 = ‖A‖ and α1 = ‖B‖.
I This has an equivalent resolvent-like formulation:
σε(A,B) = {z ∈ C : ‖(Bz − A)−1‖(α0 + α1|z |) > 1/ε}.
I Generalized to matrix polynomials by Tisseur & N. Higham [2001, 2002];see also Lancaster & Psarrakos [2005].
I Cf. [Boutry, Elad, Golub, Milanfar, 2005] for rectangular pencils.
![Page 218: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/218.jpg)
GEPs: Eigenvalue Perturbation Approach
A =
»−1 −5M0 −5
–, B =
»1 M0 1
–
−15 −10 −5 0
−8
−6
−4
−2
0
2
4
6
8
−2
−1.75
−1.5
−1.25
−1
−0.75
−0.5
−0.25
M = 0
σ(A,B) = {−1,−5}
![Page 219: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/219.jpg)
GEPs: Eigenvalue Perturbation Approach
A =
»−1 −5M0 −5
–, B =
»1 M0 1
–
−15 −10 −5 0
−8
−6
−4
−2
0
2
4
6
8
−2
−1.75
−1.5
−1.25
−1
−0.75
−0.5
−0.25
M = 0.25
σ(A,B) = {−1,−5}
![Page 220: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/220.jpg)
GEPs: Eigenvalue Perturbation Approach
A =
»−1 −5M0 −5
–, B =
»1 M0 1
–
−15 −10 −5 0
−8
−6
−4
−2
0
2
4
6
8
−2
−1.75
−1.5
−1.25
−1
−0.75
−0.5
−0.25
M = 0.50
σ(A,B) = {−1,−5}
![Page 221: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/221.jpg)
GEPs: Eigenvalue Perturbation Approach
A =
»−1 −5M0 −5
–, B =
»1 M0 1
–
−15 −10 −5 0
−8
−6
−4
−2
0
2
4
6
8
−2
−1.75
−1.5
−1.25
−1
−0.75
−0.5
−0.25
M = 0.75
σ(A,B) = {−1,−5}
![Page 222: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/222.jpg)
GEPs: Eigenvalue Perturbation Approach
A =
»−1 −5M0 −5
–, B =
»1 M0 1
–
−15 −10 −5 0
−8
−6
−4
−2
0
2
4
6
8
−2
−1.75
−1.5
−1.25
−1
−0.75
−0.5
−0.25
M = 1.00
σ(A,B) = {−1,−5}
![Page 223: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/223.jpg)
GEPs: Eigenvalue Perturbation Approach
A =
»−1 −5M0 −5
–, B =
»1 M0 1
–
−15 −10 −5 0
−8
−6
−4
−2
0
2
4
6
8
−2
−1.75
−1.5
−1.25
−1
−0.75
−0.5
−0.25
M = 1.25
σ(A,B) = {−1,−5}
![Page 224: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/224.jpg)
GEPs: Eigenvalue Perturbation Approach
A =
»−1 −5M0 −5
–, B =
»1 M0 1
–
−15 −10 −5 0
−8
−6
−4
−2
0
2
4
6
8
−2
−1.75
−1.5
−1.25
−1
−0.75
−0.5
−0.25
M = 1.50
σ(A,B) = {−1,−5}
![Page 225: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/225.jpg)
GEPs: Eigenvalue Perturbation Approach
A =
»−1 −5M0 −5
–, B =
»1 M0 1
–
−15 −10 −5 0
−8
−6
−4
−2
0
2
4
6
8
−2
−1.75
−1.5
−1.25
−1
−0.75
−0.5
−0.25
M = 1.75
σ(A,B) = {−1,−5}
![Page 226: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/226.jpg)
GEPs: Eigenvalue Perturbation Approach
A =
»−1 −5M0 −5
–, B =
»1 M0 1
–
−15 −10 −5 0
−8
−6
−4
−2
0
2
4
6
8
−2
−1.75
−1.5
−1.25
−1
−0.75
−0.5
−0.25
M = 2.00
σ(A,B) = {−1,−5}
![Page 227: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/227.jpg)
GEPs: Eigenvalue Perturbation Approach
Consider solutions to Bx′(t) = Ax(t) for the previous example:
A =
»−1 −5M0 −5
–, B =
»1 M0 1
–Note that
B−1A =
»−1 00 −5
–.
I Since B−1A is normal and stable, solutions to Bx′(t) = Ax(t) cannotexhibit growth.
I The parameter M affects the stability of eigenvalues of the pencil,but has no influence on the solution of Bx′(t) = Ax(t).
![Page 228: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/228.jpg)
GEPs: Eigenvalue Perturbation Approach
Consider solutions to Bx′(t) = Ax(t) for the previous example:
A =
»−1 −5M0 −5
–, B =
»1 M0 1
–Note that
B−1A =
»−1 00 −5
–.
I Since B−1A is normal and stable, solutions to Bx′(t) = Ax(t) cannotexhibit growth.
I The parameter M affects the stability of eigenvalues of the pencil,but has no influence on the solution of Bx′(t) = Ax(t).
![Page 229: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/229.jpg)
GEPs: Matrix Behavior Approach
I More generally, premultiplying
Bx′(t) = Ax(t)
by some invertible matrix S
SBx′(t) = SAx(t)
affects the perturbation theory of the pencil (SA,SB),but not the system driven by (SB)−1(SA) = B−1A.
This fact suggests an alternative definition.
Approach 2: matrix behavior
I [Ruhe, 1995], [Riedel, 1994] proposed:For A ∈ Cn×n and invertible B ∈ Cn×n,
σε(A,B) = σε(B−1A).
![Page 230: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/230.jpg)
GEPs: Matrix Behavior Approach
I More generally, premultiplying
Bx′(t) = Ax(t)
by some invertible matrix S
SBx′(t) = SAx(t)
affects the perturbation theory of the pencil (SA,SB),but not the system driven by (SB)−1(SA) = B−1A.
This fact suggests an alternative definition.
Approach 2: matrix behavior
I [Ruhe, 1995], [Riedel, 1994] proposed:For A ∈ Cn×n and invertible B ∈ Cn×n,
σε(A,B) = σε(B−1A).
![Page 231: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/231.jpg)
Comparison of GEP Pseudospectra
A =
»−1 −5M0 −5
–, B =
»1 M0 1
–
B−1A =
»−1 00 −5
–.
−15 −10 −5 0
−8
−6
−4
−2
0
2
4
6
8
−2
−1.75
−1.5
−1.25
−1
−0.75
−0.5
−0.25
−15 −10 −5 0
−8
−6
−4
−2
0
2
4
6
8
−2
−1.75
−1.5
−1.25
−1
−0.75
−0.5
−0.25
FGNT definition, M = 1 Ruhe definition
![Page 232: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/232.jpg)
GEP: Behavior in Different Norms
Ruhe’s definition is closely related to Riedel’s [1994]:
I If B is Hermitian positive definite with Cholesky factorization B = LL∗,then the ε-pseudospectrum of the matrix pencil (A,B) is the set
σε(A,B) = σε(L−1AL−∗).
This definition is the same as Ruhe’s definition, but in a different norm. Let
〈x, y〉B = y∗Bx, ‖x‖2B = x∗Bx.
Then‖(z − L−1AL−∗)−1‖2 = ‖(z − B−1A)−1‖B.
![Page 233: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/233.jpg)
Pseudospectra for DAEs
Suppose B is singular, but (A,B) is regular (det(zB−A) 6= 0 for some z ∈ C).
Bx′(t) = Ax(t) is a differential-algebraic equation (DAE).
Simple example:
x ′1(t) = −x1(t)
x1(t) = x2(t)
»1 00 0
– »x ′1(t)x ′2(t)
–=
»−1 0−1 1
– »x1(t)x2(t)
–
I Campbell & Meyer [1979], Campbell [1980]
I Kunkel & Mehrmann [2006]
I Descriptor systems: Benner, Byers, Mehrmann, Stykel, . . .
![Page 234: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/234.jpg)
DAEs, Simplest Case: A Invertible
Suppose that A is invertible, so that we can write
Bx′(t) = Ax(t)
in the formA−1Bx′(t) = x(t).
First take a (generalized) Schur decomposition,
A−1B = QTQ∗ =ˆ
Q1 Q2
˜ » G D0 N
– »Q∗1Q∗2
–,
where Q is unitary, G is invertible, and N is nilpotent.(The degree of nilpotency corresponds to the index of the DAE.)
This decomposition reveals the algebraic structure of the problem:
Bx′(t) = Ax(t), x(0) = x0 has a solution if and only if x0 ∈ Range Q1.
![Page 235: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/235.jpg)
DAEs, Simplest Case: A Invertible
A−1B =ˆ
Q1 Q2
˜ » G D0 N
– »Q∗1Q∗2
–We wish to write the solution to the DAE as
x(t) = Q1y(t) + Q2z(t).
One can show that z(t) = 0 for all t, so we seek:
Gy′(t) = y(t), z(t) = 0.
Hence write x(t) = Q1etG−1
y(0), i.e.,
x(t) = Q1etG−1
Q∗1 x0.
Special case: B invertible =⇒ Q1 = I and G = A−1B, so
x(t) = et G−1
x0 = et B−1Ax0.
![Page 236: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/236.jpg)
Pseudospectra of (A, B) for Transient Analysis of DAEs
Suppose x0 ∈ Range Q1, with the columns of Q1 forming an orthonormal basisfor the invariant subspace of the pencil associated with finite eigenvalues. Then
x(t) = Q1etG−1
Q∗1 x0,
where G = Q∗1 A−1BQ1 ∈ Cm×m (m = # of finite eigenvalues).
We can bound the norm of the solution by
‖x(t)‖ ≤ ‖etG−1
‖‖x0‖.
Definition (Pseudospectra of a Regular Pencil, A invertible)
Consider the matrix pencil A− λB with A invertible, and Schur factorization
A−1B =ˆ
Q1 Q2
˜ » G D0 N
– »Q∗1Q∗2
–for N nilpotent and 0 6∈ σ(G).The ε-pseudospectrum of the matrix pencil (A,B) is
σε(A,B) := σε((Q∗1 A−1BQ1)−1).
![Page 237: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/237.jpg)
Pseudospectra of (A, B) for Transient Analysis of DAEs
Suppose x0 ∈ Range Q1, with the columns of Q1 forming an orthonormal basisfor the invariant subspace of the pencil associated with finite eigenvalues. Then
x(t) = Q1etG−1
Q∗1 x0,
where G = Q∗1 A−1BQ1 ∈ Cm×m (m = # of finite eigenvalues).
We can bound the norm of the solution by
‖x(t)‖ ≤ ‖etG−1
‖‖x0‖.
Definition (Pseudospectra of a Regular Pencil, A invertible)
Consider the matrix pencil A− λB with A invertible, and Schur factorization
A−1B =ˆ
Q1 Q2
˜ » G D0 N
– »Q∗1Q∗2
–for N nilpotent and 0 6∈ σ(G).The ε-pseudospectrum of the matrix pencil (A,B) is
σε(A,B) := σε((Q∗1 A−1BQ1)−1).
![Page 238: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/238.jpg)
Examples: Stability Analysis for Imcompressible Flow
Pseudospectra for a matrix pencil derived from incompressible flowover a backward facing step, discretized via IFISS.
−6 −4 −2 0 2 4−5
−4
−3
−2
−1
0
1
2
3
4
5
dim = 667−4
−3.5
−3
−2.5
−2
−1.5
−1
−0.5
0
1/ν = 50
![Page 239: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/239.jpg)
Examples: Stability Analysis for Imcompressible Flow
Pseudospectra for a matrix pencil derived from incompressible flowover a backward facing step, discretized via IFISS.
−6 −4 −2 0 2 4−5
−4
−3
−2
−1
0
1
2
3
4
5
dim = 667−4
−3.5
−3
−2.5
−2
−1.5
−1
−0.5
0
1/ν = 100
![Page 240: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/240.jpg)
Examples: Stability Analysis for Imcompressible Flow
Pseudospectra for a matrix pencil derived from incompressible flowover a backward facing step, discretized via IFISS.
−6 −4 −2 0 2 4−5
−4
−3
−2
−1
0
1
2
3
4
5
dim = 667−4
−3.5
−3
−2.5
−2
−1.5
−1
−0.5
0
1/ν = 200
![Page 241: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/241.jpg)
Summary
![Page 242: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/242.jpg)
Overview of Lectures
These lectures address modern toolsfor the spectral analysis of dynamical systems.
We shall cover a mix of theory, computation, and applications.
Goal: By the end, you should be equipped to understand phenomena thatmany people find quite mysterious when encountered in the wild.
Lecture 1:
I Normality and Nonnormality
I Pseudospectra
I Bounding Functions of Matrices
Lecture 2:
I Balanced Truncation and Lyapunov Equations
I Moment Matching Model Reduction
I Differential Algebraic Equations
![Page 243: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/243.jpg)
Exercises
![Page 244: Pseudospectra and the Behavior of Dynamical Systems · Pseudospectra and the Behavior of Dynamical Systems Mark Embree Computational and Applied Mathematics ... Spectra and Pseudospectra](https://reader033.fdocuments.us/reader033/viewer/2022042923/5f6ffd211cbaaa7b2c4a8ddf/html5/thumbnails/244.jpg)
Exercises for the Afternoon
I Download EigTool (http://www.cs.ox.ac.uk/pseudospectra/eigtool)Run eigtool from a MATLAB prompt.Experiment with menu option: Demos/Dense/...
For dense matrices, experiment with menu option: Transients
For dense matrices, experiment with menu option: Numbers
For dense matrices, experiment with Pmode+epsilon buttonExperiment with menu option: Demos/Sparse/...
I Computation of pseudospectra of a vibrating string in energy norm.
I Population dynamics: design a transiently growing population.
I Analysis of a nonlinear dynamical system.
I Estimate pseudospectral bounds on ‖Ak‖ and ‖etA‖.