Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix...
Transcript of Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix...
![Page 1: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/1.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Five Theorems in Matrix Analysis,
with Applications
Nick Higham
School of Mathematics
The University of Manchester
http://www.ma.man.ac.uk/~higham/
Dundee (EMS)—March 17, 2006
Nick Higham Matrix Analysis 1
![Page 2: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/2.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
Outline
f (AB) and f (BA)WMFME
Λ(AB) and Λ(BA)f (αI + AB)
Symmetrization
Jordan Structure of f (A)
Matrix Sign Identities
Nick Higham Matrix Analysis 2
![Page 3: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/3.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
f (AB) and f (BA)For A, B ∈ C
n×n, AB 6= BA.
How are AB and BA related?
How are f (AB) and f (BA) related?
Same question if A ∈ Cm×n, B ∈ C
n×m.
Generalize to f (αIm + AB) and f (αIn + BA).
Nick Higham Matrix Analysis 3
![Page 4: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/4.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
Sherman–Morrison–Woodbury Formula
If U, V ∈ Cn×p and Ip + V ∗A−1U is nonsingular then
(A + UV ∗)−1 = A−1 − A−1U(Ip + V ∗A−1U)−1V ∗A−1.
Obtained, using A + UV ∗ = A(I + A−1U · V ∗), from its
simpler version
(Im + AB)−1 = I − A(In + BA)−1B
{A ∈ C
m×n
B ∈ Cn×m
Nick Higham Matrix Analysis 4
![Page 5: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/5.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
World’s Most Fundamental Matrix Equation
Nick Higham Matrix Analysis 5
![Page 6: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/6.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
World’s Most Fundamental Matrix Equation
(I + AB)A = A(I + BA), or
(AB)A = A(BA).
Nick Higham Matrix Analysis 5
![Page 7: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/7.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
Application of WMFME
(AB)A = A(BA)
⇒ (AB)2A = ABA(BA) = A(BA)2.
In general, for any poly p,
p(AB)A = Ap(BA).
◮ Does the same hold for arbitrary f?
Nick Higham Matrix Analysis 6
![Page 8: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/8.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
AB and BA
If A, B square and A nonsingular, WMFME implies
AB = A(BA)A−1, so Λ(AB) = Λ(BA).
Nick Higham Matrix Analysis 7
![Page 9: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/9.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
AB and BA
If A, B square and A nonsingular, WMFME implies
AB = A(BA)A−1, so Λ(AB) = Λ(BA).
Theorem (Flanders, 1951)
Let A ∈ Cm×n and B ∈ C
n×m.
The nonzero eigenvalues of AB have the same
Jordan structure as the nonzero eigenvalues of BA.
Any zero eigenvalues appear in Jordan blocks of AB
and BA differing in size by at most 1.
Nick Higham Matrix Analysis 7
![Page 10: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/10.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
Putnam Problem 1990-A5
If A, B ∈ Cn×n does ABAB = 0 imply BABA = 0?
Nick Higham Matrix Analysis 8
![Page 11: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/11.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
Putnam Problem 1990-A5
If A, B ∈ Cn×n does ABAB = 0 imply BABA = 0?
Yes for n ≤ 2; no for n > 2.
A =
0 0 1
0 0 0
0 1 0
, B =
0 0 1
1 0 0
0 0 0
.
(AB)2 = 0, (BA)2 =
0 0 1
0 0 0
0 0 0
.
Nick Higham Matrix Analysis 8
![Page 12: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/12.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
Tridiagonal Toeplitz Matrices
Tn(c, d , e) =
d e
c d. . .
. . .. . . e
c d
.
Eigenvalues known explicitly:
d + 2(ce)1/2 cos(kπ/(n + 1)), k = 1 : n.
What about simple modifications of Tn, e.g. to the
(1,1) and (n, n) elements?
Nick Higham Matrix Analysis 9
![Page 13: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/13.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
Second Difference Matrix
Tn =
2 −1
−1 2. . .
. . .. . . −1
−1 2
,
T̃n =
1 −1
−1 2. . .
. . .. . . −1
−1 2 −1
−1 1
.
Nick Higham Matrix Analysis 10
![Page 14: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/14.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
Second Difference Matrix (cont.)
Define
L =
1
−1 1
−1. . .. . . 1
−1
∈ R
(n+1)×n.
Then Tn = LT L, T̃n+1 = LLT .
So Λ(T̃n+1) = Λ(Tn) ∪ {0} (Strang, 2005).
Example:
n = 6; L = gallery(’triw’,n,-1,1)’;
L = L(:,1:n-1), A = L*L’, B = L’*L
Nick Higham Matrix Analysis 11
![Page 15: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/15.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
Definition of Matrix Function
Let A have distinct eigenvalues λ1, . . . , λs, and let ni be
order of the largest Jordan block in which λi appears.
Definition (Sylvester, 1883)
f (A) := r(A), where r is the unique Hermite interpolating
polynomial of degree less than∑s
i=1 ni that satisfies
r (j)(λi) = f (j)(λi), j = 0 : ni − 1, i = 1 : s.
Nick Higham Matrix Analysis 12
![Page 16: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/16.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
f (AB) and f (BA)Recall that for any polynomial p,
Ap(BA) = p(AB)A.
Nick Higham Matrix Analysis 13
![Page 17: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/17.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
f (AB) and f (BA)Recall that for any polynomial p,
Ap(BA) = p(AB)A.
Lemma
Let A ∈ Cm×n and B ∈ C
n×m and let f (AB) and f (BA) be
defined. Then
Af (BA) = f (AB)A.
Nick Higham Matrix Analysis 13
![Page 18: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/18.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
f (AB) and f (BA)Recall that for any polynomial p,
Ap(BA) = p(AB)A.
Lemma
Let A ∈ Cm×n and B ∈ C
n×m and let f (AB) and f (BA) be
defined. Then
Af (BA) = f (AB)A.
Proof. There is a single polynomial p such that
f (AB) = p(AB) and f (BA) = p(BA). Hence
Af (BA) = Ap(BA) = p(AB)A = f (AB)A.
Nick Higham Matrix Analysis 13
![Page 19: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/19.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
Special Case
Take f (t) = t1/2. When AB (and hence also BA) has no
eigenvalues on R−,
A(BA)1/2 = (AB)1/2A.
Nick Higham Matrix Analysis 14
![Page 20: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/20.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
Special Case
Take f (t) = t1/2. When AB (and hence also BA) has no
eigenvalues on R−,
A(BA)1/2 = (AB)1/2A.
———
Useful, but
Af (BA) = f (AB)A
cannot be solved for f (BA) in terms of f (AB).
Nick Higham Matrix Analysis 14
![Page 21: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/21.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
Theorem (Harris 1993; H 2005)
Let A ∈ Cm×n and B ∈ C
n×m, with m ≥ n, and assume BA
is nonsingular. Then
f (αIm + AB) = f (α)Im + A(BA)−1(f (αIn + BA) − f (α)In
)B.
Nick Higham Matrix Analysis 15
![Page 22: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/22.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
Theorem (Harris 1993; H 2005)
Let A ∈ Cm×n and B ∈ C
n×m, with m ≥ n, and assume BA
is nonsingular. Then
f (αIm + AB) = f (α)Im + A(BA)−1(f (αIn + BA) − f (α)In
)B.
Proof. Define g(X ) = X−1(f (αI + X ) − f (αI)
).
Then f (αI + X ) = f (α)I + Xg(X ).Hence, using the lemma,
f (αIm + AB) = f (α)Im + ABg(AB)
= f (α)Im + Ag(BA)B
= f (α)Im + A(BA)−1(f (αIn + BA) − f (α)In
)B.
Nick Higham Matrix Analysis 15
![Page 23: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/23.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
Example: Rank 2 Perturbation of I
Consider f (αIn + uv∗ + xy∗), where u, v , x , y ∈ Cn. Write
uv∗ + xy∗ = [ u x ]
[v∗
y∗
]≡ AB.
Then
C := BA =
[v∗u v∗x
y∗u y∗x
]∈ C
2×2.
f (αIn + uv∗ + xy∗) = f (α)In +
[ u x ] C−1(f (αI2 + C) − f (α)I2
) [v∗
y∗
]
Nick Higham Matrix Analysis 16
![Page 24: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/24.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function WMFME Λ(AB) and Λ(BA) f (αI + AB)
Example: Rank 2 Perturbation of I
Consider f (αIn + uv∗ + xy∗), where u, v , x , y ∈ Cn. Write
uv∗ + xy∗ = [ u x ]
[v∗
y∗
]≡ AB.
Then
C := BA =
[v∗u v∗x
y∗u y∗x
]∈ C
2×2.
f (αIn + uv∗ + xy∗) = f (α)In +
[ u x ] C−1(f (αI2 + C) − f (α)I2
) [v∗
y∗
]
For A ∈ C2×2, f (A) = f (λ1)I + f [λ1, λ2](A − λ1I).
Nick Higham Matrix Analysis 16
![Page 25: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/25.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Outline
f (AB) and f (BA)WMFME
Λ(AB) and Λ(BA)f (αI + AB)
Symmetrization
Jordan Structure of f (A)
Matrix Sign Identities
Nick Higham Matrix Analysis 17
![Page 26: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/26.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Symmetrization
Theorem (Frobenius, 1910)
For any A ∈ Fn×n (F = R or C) there exist symmetric
S1, S2 ∈ Fn×n, either one of which can be taken
nonsingular, such that A = S1S2.
Nick Higham Matrix Analysis 18
![Page 27: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/27.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Symmetrization
Theorem (Frobenius, 1910)
For any A ∈ Fn×n (F = R or C) there exist symmetric
S1, S2 ∈ Fn×n, either one of which can be taken
nonsingular, such that A = S1S2.
Implication
The generalized eigenproblem Ax = λBx with symmetric
A and B has no special eigenproperties: equivalent to
Cx := B−1Ax = λx , with C arbitrary.
Nick Higham Matrix Analysis 18
![Page 28: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/28.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Proof
Rational canonical form says A is similar to a direct sum
of companion matrices: A = X−1CX . But S−11 C = S2:
0 0 1
0 1 −β2
1 −β2 −β1
β2 β1 β0
1 0 0
0 1 0
=
0 1 0
1 −β2 0
0 0 β0
.
Then A = X−1S1S2X = X−1S1X−T · X T S2X ≡ S̃1S̃2.
Nick Higham Matrix Analysis 19
![Page 29: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/29.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Proof
Rational canonical form says A is similar to a direct sum
of companion matrices: A = X−1CX . But S−11 C = S2:
0 0 1
0 1 −β2
1 −β2 −β1
β2 β1 β0
1 0 0
0 1 0
=
0 1 0
1 −β2 0
0 0 β0
.
Then A = X−1S1S2X = X−1S1X−T · X T S2X ≡ S̃1S̃2.
TheoremFor any A ∈ F
n×n (F = R or C) there exists a nonsingular
symmetric S such that A = S−1AT S.
Nick Higham Matrix Analysis 19
![Page 30: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/30.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Application to Polynomial Zero-Finding
Lancaster (1961) takes companion linearization λI − C for
scalar poly p(t) = ak tk + · · · + a1t + a0:
C =
−ak−1/ak −ak−2/ak . . . −a0/ak
1 0 . . . 0. . .
. . ....
1 0
.
We can write C = S−11 S2 with S1, S2 symm. So
◮ S1(λI − C) = λS1 − S1C is a symm. pencil.
◮ Ditto S1Cℓ−1(λI − C) = λS1Cℓ−1 − S1Cℓ for ℓ ≥ 1.
Lancaster takes
S1 =
ak
. ..
ak−1
. ..
. .. ...
ak ak−1 . . . a1
.
Nick Higham Matrix Analysis 20
![Page 31: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/31.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Matrix Polynomial Case
This construction generalizes immediately to matrix
polynomials and provides block symmetric pencils
λX + Y [Xij = Xji , i 6= j ].
◮ What space of pencils is generated?
◮ What happens as ℓ increases?
◮ Is there anything special about this particular S1?
◮ How are ei’vecs of the pencils related to those of P?
Nick Higham Matrix Analysis 21
![Page 32: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/32.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Matrix Polynomial Case
This construction generalizes immediately to matrix
polynomials and provides block symmetric pencils
λX + Y [Xij = Xji , i 6= j ].
◮ What space of pencils is generated?
◮ What happens as ℓ increases?
◮ Is there anything special about this particular S1?
◮ How are ei’vecs of the pencils related to those of P?
Answered via a new theory of
vector spaces of linearizations:
H, D. S. Mackey, N. Mackey, Mehl,
Mehrmann, Tisseur (2005)
Nick Higham Matrix Analysis 21
![Page 33: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/33.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Outline
f (AB) and f (BA)WMFME
Λ(AB) and Λ(BA)f (αI + AB)
Symmetrization
Jordan Structure of f (A)
Matrix Sign Identities
Nick Higham Matrix Analysis 22
![Page 34: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/34.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Function of Jordan block
A = Zdiag(J1, . . . , Jp)Z−1 ⇒ f (A) = Zdiag(f (J1), . . . , f (Jp))Z
−1.
Jk =
λk 1
λk. . .. . . 1
λk
∈ C
mk×mk ,
f (Jk) =
f (λk) f ′(λk) . . .f (mk−1))(λk)
(mk − 1)!
f (λk). . .
.... . . f ′(λk)
f (λk)
.
Nick Higham Matrix Analysis 23
![Page 35: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/35.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Theorem
Let A ∈ Cn×n with eigenvalues λk .
1 If f ′(λk ) 6= 0 then for every J(λk ) in A there is a Jordan
block of the same size in f (A) for f (λk ).
2 Let f ′(λk ) = f ′′(λk ) = · · · = f (ℓ−1)(λk ) = 0 but f (ℓ)(λk ) 6= 0,
where ℓ ≥ 2, and consider J(λk ) of size r in A.
(i) If ℓ ≥ r , J(λk ) splits into r 1 × 1 Jordan blocks for
f (λk ) in f (A).
(ii) If ℓ ≤ r − 1, J(λk ) splits into Jordan blocks for f (λk )in f (A) as follows:
• ℓ − q Jordan blocks of size p,
• q Jordan blocks of size p + 1,
where r = ℓp + q with 0 ≤ q ≤ ℓ − 1, p > 0.
Nick Higham Matrix Analysis 24
![Page 36: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/36.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Theorem
Let A ∈ Cn×n with eigenvalues λk .
1 If f ′(λk ) 6= 0 then for every J(λk ) in A there is a Jordan
block of the same size in f (A) for f (λk ).
2 Let f ′(λk ) = f ′′(λk ) = · · · = f (ℓ−1)(λk ) = 0 but f (ℓ)(λk ) 6= 0,
where ℓ ≥ 2, and consider J(λk ) of size r in A.
(i) If ℓ ≥ r , J(λk ) splits into r 1 × 1 Jordan blocks for
f (λk ) in f (A).
(ii) If ℓ ≤ r − 1, J(λk ) splits into Jordan blocks for f (λk )in f (A) as follows:
• ℓ − q Jordan blocks of size p,
• q Jordan blocks of size p + 1,
where r = ℓp + q with 0 ≤ q ≤ ℓ − 1, p > 0.
Nick Higham Matrix Analysis 24
![Page 37: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/37.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Theorem
Let A ∈ Cn×n with eigenvalues λk .
1 If f ′(λk ) 6= 0 then for every J(λk ) in A there is a Jordan
block of the same size in f (A) for f (λk ).
2 Let f ′(λk ) = f ′′(λk ) = · · · = f (ℓ−1)(λk ) = 0 but f (ℓ)(λk ) 6= 0,
where ℓ ≥ 2, and consider J(λk ) of size r in A.
(i) If ℓ ≥ r , J(λk ) splits into r 1 × 1 Jordan blocks for
f (λk ) in f (A).
(ii) If ℓ ≤ r − 1, J(λk ) splits into Jordan blocks for f (λk )in f (A) as follows:
• ℓ − q Jordan blocks of size p,
• q Jordan blocks of size p + 1,
where r = ℓp + q with 0 ≤ q ≤ ℓ − 1, p > 0.
Nick Higham Matrix Analysis 24
![Page 38: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/38.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Application: Matrix Logarithm
Find all solutions to eX = A.
Let A have JCF A = Zdiag(Jk(λk))Z−1 = ZJZ−1.
Nick Higham Matrix Analysis 25
![Page 39: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/39.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Application: Matrix Logarithm
Find all solutions to eX = A.
Let A have JCF A = Zdiag(Jk(λk))Z−1 = ZJZ−1.
Since ddx
ex 6= 0 , X has Jordan form
JX = diag(Jk(µk)), where exp(µk) = λk and hence
µk = log λk + 2jkπi .
Nick Higham Matrix Analysis 25
![Page 40: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/40.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Application: Matrix Logarithm
Find all solutions to eX = A.
Let A have JCF A = Zdiag(Jk(λk))Z−1 = ZJZ−1.
Since ddx
ex 6= 0 , X has Jordan form
JX = diag(Jk(µk)), where exp(µk) = λk and hence
µk = log λk + 2jkπi .
Now consider L = diag(Lk), where
Lk = log(Jk(λk)) + 2jkπiI. Then eL = J , so by same
argument as above, L has Jordan form JX , i.e.,
X = WLW−1, some W .
Nick Higham Matrix Analysis 25
![Page 41: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/41.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Application: Matrix Logarithm
Find all solutions to eX = A.
Let A have JCF A = Zdiag(Jk(λk))Z−1 = ZJZ−1.
Since ddx
ex 6= 0 , X has Jordan form
JX = diag(Jk(µk)), where exp(µk) = λk and hence
µk = log λk + 2jkπi .
Now consider L = diag(Lk), where
Lk = log(Jk(λk)) + 2jkπiI. Then eL = J , so by same
argument as above, L has Jordan form JX , i.e.,
X = WLW−1, some W .
But eX = A implies WJW−1 = WeLW−1 = ZJZ−1, or
(Z−1W )J = J(Z−1W ).
Nick Higham Matrix Analysis 25
![Page 42: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/42.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Application: Matrix Logarithm
Theorem (Gantmacher, 1959)
Let A ∈ Cn×n be nonsing. with JCF A = Zdiag(Jk(λk))Z
−1.
All solutions to eX = A are given by
X = ZUdiag(L(j1)1 , L
(j2)2 , . . . , L
(jp)p )U−1Z−1,
where
L(jk )k = log(Jk(λk)) + 2jkπiI,
log(Jk(λk)) is the principal logarithm, jk is an arbitrary
integer, and U is an arbitrary nonsingular matrix
commuting with J.
Nick Higham Matrix Analysis 26
![Page 43: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/43.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Outline
f (AB) and f (BA)WMFME
Λ(AB) and Λ(BA)f (αI + AB)
Symmetrization
Jordan Structure of f (A)
Matrix Sign Identities
Nick Higham Matrix Analysis 27
![Page 44: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/44.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Matrix Sign Function
For A ∈ Cn×n with Jordan canonical form
A = Z
[J1 0
0 J2
]Z−1,
where λ(J1) ∈ open LHP, λ(J2) ∈ open RHP,
sign(A) = Z
[−I 0
0 I
]Z−1.
Introduced by Roberts (1971), who proposed Newton iter.
Xk+1 =1
2(Xk + X−1
k ), X0 = A.
Xk converges quadratically to sign(A).
Nick Higham Matrix Analysis 28
![Page 45: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/45.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Matrix Sign Relations
For nonsingular A ∈ Cn×n (Byers, 1984):
sign
([0 A
A∗ 0
])=
[0 U
U∗ 0
],
where A = UH is the polar decomposition.
Nick Higham Matrix Analysis 29
![Page 46: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/46.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Matrix Sign Relations
For nonsingular A ∈ Cn×n (Byers, 1984):
sign
([0 A
A∗ 0
])=
[0 U
U∗ 0
],
where A = UH is the polar decomposition.
For A ∈ Cn×n with no eigenvalues on R
− (H, 1997):
sign
([0 A
I 0
])=
[0 A1/2
A−1/2 0
].
Nick Higham Matrix Analysis 29
![Page 47: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/47.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
More General Matrix Sign Relation
Theorem (H, Mackey, Mackey, Tisseur, 2005)
Let A, B ∈ Cn×n and suppose AB has no eigenvalues on
R−. Then
sign
([0 A
B 0
])=
[0 C
C−1 0
],
where C = A(BA)−1/2.
Nick Higham Matrix Analysis 30
![Page 48: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/48.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Proof. P =[
0B
A0
]has no pure imaginary ei’vals. Hence
sign(P) = P(P2)−1/2 =
[0 A
B 0
] [AB 0
0 BA
]−1/2
=
[0 A
B 0
] [(AB)−1/2 0
0 (BA)−1/2
]
=
[0 A(BA)−1/2
B(AB)−1/2 0
]=:
[0 C
D 0
].
Now
I = (sign(P))2 =
[0 C
D 0
]2
=
[CD 0
0 DC
],
so D = C−1.
Nick Higham Matrix Analysis 31
![Page 49: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/49.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Puzzle
Proof of previous theorem shows that
A(BA)−1/2 =[B(AB)−1/2
]−1
= (AB)1/2B−1.
Why do we have equality?
Nick Higham Matrix Analysis 32
![Page 50: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/50.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Puzzle
Proof of previous theorem shows that
A(BA)−1/2 =[B(AB)−1/2
]−1
= (AB)1/2B−1.
Why do we have equality?
Recall
Af (BA) = f (AB)A .
Now
A(BA)−1/2 · B(AB)−1/2 = (AB)−1/2A · B(AB)−1/2
= (AB)−1/2AB(AB)−1/2
= I.
Nick Higham Matrix Analysis 32
![Page 51: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/51.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Application: Matrix Iterations
Apply any iteration for the matrix sign function to
[0 A
A∗ 0
]or
[0 A
I 0
]
and read off from the (1,2) block an iteration for polar
factor U or A1/2.
Applying the lemma to
[0 A
A⋆ 0
]
can derive new iterations for the generalized polar
decomposition this way (HMMT, 2005).
Nick Higham Matrix Analysis 33
![Page 52: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/52.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Summary
λ(AB) vs. λ(BA) : Flanders (1951).
f (αIm + AB) : Harris (1993), H (2005).
A = S1S2 : Frobenius (1910).
Jordan structure of f (J) .
sign([
0B
A0
]) : H, Mackey, Mackey, Tisseur (2005).
Nick Higham Matrix Analysis 34
![Page 53: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/53.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Bibliography I
A. J. Bosch.
Note on the factorization of a square matrix into two
Hermitian or symmetric matrices.
SIAM Rev., 29(3):463–468, 1987.
Harley Flanders.
Elementary divisors of AB and BA.
Proc. Amer. Math. Soc., 2(6):871–874, 1951.
F. R. Gantmacher.
The Theory of Matrices, volume two.
Chelsea, New York, 1959.
Nick Higham Matrix Analysis 35
![Page 54: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/54.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Bibliography II
P. R. Halmos.
Bad products of good matrices.
Linear and Multlinear Algebra, 29:1–20, 1991.
Lawrence A. Harris.
Computation of functions of certain operator matrices.
Linear Algebra Appl., 194:31–34, 1993.
Nicholas J. Higham.
Functions of a Matrix: Theory and Computation.
Book in preparation.
Nick Higham Matrix Analysis 36
![Page 55: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/55.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Bibliography III
Nicholas J. Higham, D. Steven Mackey, Niloufer
Mackey, and Françoise Tisseur.
Functions preserving matrix groups and iterations for
the matrix square root.
SIAM J. Matrix Anal. Appl., 26(3):849–877, 2005.
Nicholas J. Higham, D. Steven Mackey, Niloufer
Mackey, and Françoise Tisseur.
Symmetric linearizations for matrix polynomials.
MIMS EPrint 2005.25, Manchester Institute for
Mathematical Sciences, The University of
Manchester, UK, November 2005.
Submitted to SIAM J. Matrix Anal. Appl.
Nick Higham Matrix Analysis 37
![Page 56: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/56.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Bibliography IV
Roger A. Horn and Dennis I. Merino.
Contragredient equivalence: A canonical form and
some applications.
Linear Algebra Appl., 214:43–92, 1995.
Charles R. Johnson and Eric Schreiner.
The relationship between AB and BA.
Amer. Math. Monthly, 103(7):578–582, 1996.
Leonard F. Klosinski, Gerald L. Alexanderson, and
Loren C. Larson.
The fifty-first William Lowell Putnam mathematical
competition.
Amer. Math. Monthly, 98(8):719–727, 1991.
Nick Higham Matrix Analysis 38
![Page 57: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/57.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Bibliography V
Peter Lancaster.
Symmetric transformations of the companion matrix.
NABLA: Bulletin of the Malayan Math. Soc.,
8:146–148, 1961.
Heydar Radjavi.
Products of Hermitian matrices and symmetries.
Proc. Amer. Math. Soc., 21(2):369–372, 1969.
O. Taussky.
The role of symmetric matrices in the study of general
matrices.
Linear Algebra Appl., 5:147–154, 1972.
Nick Higham Matrix Analysis 39
![Page 58: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/58.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
Bibliography VI
Olga Taussky and Hans Zassenhaus.
On the similarity transformation between a matrix and
its transpose.
Pacific J. Math., 9:893–896, 1959.
R. C. Thompson.
On the matrices AB and BA.
Linear Algebra Appl., 1:43–58, 1968.
Nick Higham Matrix Analysis 40
![Page 59: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/59.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
World’s Most Fundamental Matrix Equation
(I + AB)−1 = I − A(I + BA)−1B.
I = I + AB − (I + AB)A(I + BA)−1B
= I + AB − A(I + BA)(I + BA)−1B
= I + AB − AB
= I√
Nick Higham Matrix Analysis 41
![Page 60: Five Theorems in Matrix Analysis, with Applicationshigham/talks/... · Five Theorems in Matrix Analysis, with Applications ... matrix? ...](https://reader035.fdocuments.us/reader035/viewer/2022081522/5f03ab547e708231d40a307c/html5/thumbnails/60.jpg)
f (AB), f (BA) Symmetr’n f(Jordan block) Sign function
World’s Most Fundamental Matrix Equation
(I + AB)−1 = I − A(I + BA)−1B.
I = I + AB − (I + AB)A(I + BA)−1B
= I + AB − A(I + BA)(I + BA)−1B
= I + AB − AB
= I√
Key equation: (I + AB)A = A(I + BA), or
(AB)A = A(BA).
Nick Higham Matrix Analysis 41