Eigenvalue Bounds for Double Saddle-Point

Systems

Susanne Bradley∗ Chen Greif†

October 25, 2021

Abstract

We derive bounds on the eigenvalues of a generic form of double saddle-point matrices. The bounds are expressed in terms of extremal eigenvaluesand singular values of the associated block matrices. Inertia and algebraicmultiplicity of eigenvalues are considered as well. The analysis includesbounds for preconditioned matrices based on block diagonal precondi-tioners using Schur complements, and it is shown that in this case theeigenvalues are clustered within a few intervals bounded away from zero.Analysis for approximations of Schur complements is included. Some nu-merical experiments validate our analytical findings.

1 Introduction

Given positive integer dimensions n ≥ m ≥ p, consider the (n+m+ p)× (n+m+ p) double saddle-point system

Ku = b, (1)

where

K =

A BT 0B −D CT

0 C E

; u =

xyz

; b =

pqr

. (2)

In (2), A ∈ Rn×n is assumed symmetric positive definite, D ∈ Rm×m andE ∈ Rp×p are positive semidefinite, and B ∈ Rm×n, C ∈ Rp×m.

Under the assumptions stated above, K is symmetric and indefinite, andsolving (1) presents several numerical challenges. When the linear system is toolarge for direct solvers to work effectively, iterative methods [35] are preferred;this is the scenario that is of interest to us in this work. A minimum residualKrylov subspace solver such as MINRES [23] is a popular choice due to its

∗Department of Computer Science, The University of British Columbia, Vancouver,Canada V6T 1Z4 (smbrad@cs.ubc.ca, greif@cs.ubc.ca).

1

arX

iv:2

110.

1332

8v2

[m

ath.

NA

] 1

4 Fe

b 20

22

optimality and short recurrence properties. Other solvers may be highly effectiveas well.

Linear systems of the form (1)–(2) appear frequently in multiphysics prob-lems, and their numerical solution is of increasing importance and interest.There is a large number of relevant applications [6, 18, 37]; liquid crystal prob-lems [2, 28], Darcy-Stokes equations [7, 15], coupled poromechanical equations[9], magma-mantle dynamics [31], and PDE-constrained optimization problems[26, 30] are just a small subset of linear systems that fit into this framework.

The matrices D and E are often associated with regularization (or stabiliza-tion). We refer to the special case of D = E = 0 as an unregularized form of K,and denote it by K0:

K0 =

A BT 0B 0 CT

0 C 0

. (3)

This simpler form is of much potential interest, as it may be considered a directgeneralization of the standard saddle-point form for 2× 2 block matrices:

K0 =

[A BT

B 0

]. (4)

Matrices of the form (4) have been extensively studied, and their analyticaland numerical properties are well understood; see [5, 32] for excellent surveys.Some properties of K follow from appropriately reordering and partitioning theblock matrix and then using known results for block-2×2 saddle-point matrices(as given in, for example, [11, 33, 34, 36]). Specifically, K can be reordered andpartitioned into a 2× 2 block matrix

K =

A 0 BT

0 E CB CT −D

. (5)

While this approach is often effective, we may benefit from considering theblock-3 × 3 formulation K directly, without resorting to (5). When E is rank-deficient or zero (as occurs, for example, in [10, 19]), the leading 2 × 2 blockof K is singular, even if A is full rank. It is then more challenging to developpreconditioners for K or to obtain bounds on its eigenvalues, as we are restrictedto methods that can handle singular leading blocks. Additionally, by consid-ering the block-3 × 3 formulation in deriving eigenvalue bounds, we will see inthis paper that we can derive effective bounds using the singular values of Band C (along with the eigenvalues of the diagonal blocks). Analyzing K usingestablished results for block-2× 2 matrices (such as those given by Rusten andWinther [34]) instead requires singular values of the larger off-diagonal block[B CT

], which may be more difficult to obtain than singular values of B and

C. (We can estimate the singular values of[B CT

]in terms of those of B and

C, but if these estimates are loose it will result in loose eigenvalue bounds.)There has been a recent surge of interest in the iterative solution of multiple

saddle-point problems, and our work adds to the increasing body of literature

2

that considers these problems. Recent papers that provide interesting analysisare, for example, [2, 3, 6, 25, 37].

The distribution of eigenvalues of K plays a central role in determining theefficiency of iterative solvers. It is therefore useful to gain an understanding ofthe spectral structure as part of the selection and employment of solvers. Effec-tive preconditioners are instrumental in accelerating convergence for sparse andlarge linear systems; see [4, 24, 35, 38] for general overviews, and [5, 27, 32] fora useful overview of solvers and preconditioners for saddle-point problems. Formultiphysics problems it has been demonstrated that exploiting the propertiesof the underlying discrete differential operators and other characteristics of theproblem at hand is beneficial in the development of robust and fast solvers; see,e.g., [9].

There are several potential approaches to the development of precondition-ers for the specific class of problem that we are considering. Monolithic pre-conditioners, which work on the entire matrix, have been recently shown to beextremely effective. Recent work such as [1] has pushed the envelope towardsscalable solvers based on this methodology. Another increasingly importantapproach is operator preconditioning based on continuous spaces; see [14, 20].This approach relies on the properties of the underlying continuous differentialoperators, and uses tools such as Riesz representation and natural norm consid-erations to derive block diagonal preconditioners. Block diagonal precondition-ers can also be derived directly by linear algebra considerations accompanied byproperties of discretized PDEs; see, for example, [8, 27].

The preconditioner we consider for our analysis is:

M :=

A 0 00 S1 00 0 S2

, (6)

whereS1 = D +BA−1BT ; S2 = E + CS−1

1 CT . (7)

We assume that S1 and S2 are both positive definite.The preconditioner M is based on Schur complements. It has been consid-

ered in, for example, [6, 37], and is a natural extension of [17, 22] for block-2×2matrices of the form (4). In practice the Schur complements S1 and S2 definedin (7) are too expensive to form and invert exactly. It is therefore useful toconsider approximations to those matrices when a practical preconditioner is tobe developed, and we include an analysis of that scenario.

The recent papers of Sogn and Zulehner [37] and Cai et al. [6] both analyzethe performance of a block-n × n block diagonal preconditioner analogous toM defined in (6), though the former focus on the spectral properties of thecontinuous (rather than discretized) preconditioned operator, and Cai et al.focus their analyses of this preconditioner on the case where all diagonal blocksexcept A are zero. Existing analyses of the eigenvalues of unpreconditionedblock-3 × 3 matrices have often been restricted to specific problems, such asinterior-point methods in constrained optimization; see [13, 21].

3

Our goal in this paper is to provide a general framework for analysis ofeigenvalue bounds for the double saddle-point case, with a rather minimal setof assumptions on the matrices involved. To our knowledge, ours is the firstwork to provide general spectral bounds for the unpreconditioned matrix K andfor cases where inexact Schur complements are used.

In Section 2 we derive bounds on the eigenvalues of K. In Section 3 we turnour attention to preconditioned matrices of the form M−1K. In Section 4 weallow the preconditioners to contain approximations of Schur complements andan approximate inversion of the leading block, and show how the bounds areaffected as a result. In Section 5 we validate some of our analytical observationswith numerical experiments. Finally, in Section 6 we draw some conclusions.

Notation. The n+m+ p eigenvalues of the unpreconditioned and precondi-tioned double saddle-point matrices will be denoted by λ. We will use µ withappropriate matrix superscripts and subscripts to denote eigenvalues of the ma-trix blocks, and σ will accordingly signify singular values. The eigenvalues ofA, for example, will be denoted by

µAi , i = 1, . . . , n,

and in terms of ordering we will assume that

µ1 ≥ µ2 ≥ · · · ≥ µn > 0.

To increase clarity, we will use µAmax to denote µA1 and µAmin to denote µAn , andso on.

Based on the above conventions, we use the following notation for eigenvaluesand singular values of the matrices comprising K:

matrix size type number notation max min

A n× n eigenvalues n µAi , i = 1, . . . , n µAmax µAmin

B m× n singular values m σBi , i = 1, . . . ,m σBmax σBmin

C p×m singular values p σCi , i = 1, . . . , p σCmax σCmin

D m×m eigenvalues m µDi , i = 1, . . . ,m µDmax µDmin

E p× p eigenvalues p µDi , i = 1, . . . , p µEmax µEmin

2 Eigenvalue bounds for K2.1 Inertia and solvability conditions

We first discuss the inertia and conditions for nonsingularity of K defined in(2). Recall that the inertia of a matrix is the triplet denoting the number of itspositive, negative, and zero eigenvalues [16, Definition 4.5.6].

Proposition 1. The following conditions are necessary for K to be invertible:

4

(i) ker(A) ∩ ker(B) = ∅;

(ii) ker(BT ) ∩ ker(D) ∩ ker(C) = ∅;

(iii) ker(CT ) ∩ ker(E) = ∅.

A sufficient condition for K to be invertible is that A, S1, and S2 are invertible.

Proof. We begin with the proof of statement (i) by assuming to the contrarythat the intersection of the kernels is not empty – namely, there exists a nonzerovector x such that Ax = Bx = 0. This would mean that the block vector[xT 0 0

]Twas a null vector of K, which would imply that K was singular.

Similar reasoning proves (ii) and (iii).For the sufficient condition, we observe that when A, S1, and S2 are invertible

we can write a block-LDLT factorization of K:A BT 0B −D CT

0 C E

=

I 0 0BA−1 I 0

0 −CS−11 I

A 0 00 −S1 00 0 S2

︸ ︷︷ ︸

=:D

I A−1BT 00 I −S−1

1 CT

0 0 I

.(8)

When A and S1 are invertible, S2 is well-defined and K is invertible if and onlyif D is invertible. The stated result follows.

Throughout the rest of this paper, we assume that the sufficient conditionholds: namely, that A, S1, S2 are invertible. Given our assumptions that D andE are semidefinite, this is equivalent to A, S1, and S2 being positive definite.This allows us to obtain the following result on the inertia of K, which will beuseful in deriving our bounds.

Lemma 1 (Inertia of a double saddle-point matrix). The matrix K has n + ppositive eigenvalues and m negative eigenvalues.

Proof. The matrices A, S1, and S2 are symmetric positive definite, by our as-sumptions. Sylvester’s Law of Inertia tells us the inertia of K is the same asthat of D defined in (8); the stated result follows.

Corollary 1. Let a, b, c, d, and e be scalars with a > 0, d, e ≥ 0, s1 := d+ b2

a > 0

and s2 := e+ c2

s1> 0. Any cubic polynomial of the form

p(λ) = λ3 + (d− a− e)λ2 + (ae− ad− de− b2 − c2)λ+ (ade+ ac2 + b2e)

has two positive real roots and one negative real root.

Proof. Consider the 3× 3 matrix

P =

a b 0b −d c0 c e

.5

p(

) y=0

x =

0

Figure 1: Plot of a cubic polynomial p(λ) of the form described in Corollary 1,with two positive roots and one negative root.

Using the characteristic polynomial of P , it is straightforward to confirm that

det(λI − P ) = λ3 − Tr(P )λ2 − 1

2

(Tr(P 2)− Tr2(P )

)λ− det(P )

= p(λ).

Because P is symmetric its eigenvalues are real, and because P is a doublesaddle-point matrix with n = m = p = 1, the two positive and one negativeroot follow by Lemma 1. See Figure 1 for a graphical illustration.

2.2 Derivation of bounds

Let us define three cubic polynomials, as follows:

p(λ) = λ3 + (µDmax − µAmin)λ2 −(µAminµ

Dmax + (σBmax)2 + (σCmin)2

)λ+ µAmin(σCmin)2;

(9a)

q(λ) =λ3 + (µDmin − µAmax − µEmax)λ2

+(µAmaxµ

Emax − µAmaxµ

Dmin − µDminµ

Emax − (σBmax)2 − (σCmax)2

)λ

+(µAmaxµ

Dminµ

Emax + µAmax(σCmax)2 + (σBmax)2µEmax

);

(9b)

r(λ) =λ3 + (µDmax − µAmin − µEmin)λ2

+(µAminµ

Emin − µAminµ

Dmax − µDmaxµ

Emin − (σBmax)2 − (σCmax)2

)λ

+ (µAminµDmaxµ

Emin + µAmin(σCmax)2 + (σBmax)2µEmin).

(9c)

All three of these polynomials are of the form described in Corollary 1. Thus,all roots are real and each polynomial has two positive roots and one negative

6

root. For notational convenience, we will denote the negative root of p(λ), forexample, by p−, and use subscripts max and min to distinguish between the twopositive roots. For example, p+

max will denote the largest positive root and p+min

will denote the smallest positive root. The same notational rules apply to q(λ)and r(λ).

Theorem 1 (Eigenvalue bounds, matrix K). Using the notation established in(9), the eigenvalues of K are bounded within the intervals[

r−,µAmax −

√(µAmax)2 + 4(σBmin)2

2

] ⋃ [p+

min, q+max

]. (10)

Proof. Upper bound on positive eigenvalues. We let v =[xT yT zT

]Tbe

a vector with x ∈ Rn, y ∈ Rm, and z ∈ Rp. Because K is symmetric, we can

derive an upper bound on the eigenvalues of K by bounding the value of vTKvvT v

.We can write

vTKv = xTAx− yTDy + zTEz + 2xTBT y + 2yTCT z. (11)

We use Cauchy-Schwarz to bound the mixed bilinear forms – such as xTBT y –by, for example,

−||x|| · ||BT y|| ≤ xTBT y ≤ ||x|| · ||BT y||.

Using this and the eigenvalues/singular values of the block of K, we can bound(11) from above by

vTKv ≤ µAmax||x||2 − µDmin||y||2 + µEmax||z||2 + 2σBmax||x|| · ||y||+ 2σCmax||y|| · ||z||

=[||x|| ||y|| ||z||

] µAmax σBmax 0σBmax −µDmin σCmax

0 σCmax µEmax

︸ ︷︷ ︸

=:R

||x||||y||||z||

.

An upper bound on vTKvvT v

is therefore given by the maximal eigenvalue of R. Thelargest positive eigenvalue of K is therefore less than or equal to the largest rootof the characteristic polynomial det (λI −R), which yields the desired result.

Lower bound on negative eigenvalues. The proof is similar to that for the

upper bound on the positive eigenvalues. Using Cauchy-Schwarz and the eigen-values/singular values of the blocks, we bound (11) from below by writing

vTKv ≥ µAmin||x||2 − 2σBmax||x|| · ||y|| − µDmax||y||2 − 2σCmax||y|| · ||z||+ µEmin||z||2

=[||x|| ||y|| ||z||

] µAmin −σBmax 0−σBmax −µDmax −σCmax

0 −σCmax µEmin

︸ ︷︷ ︸

=:R

||x||||y||||z||

.

7

A lower bound on vTKvvT v

is therefore given by the smallest eigenvalue of R.Taking the characteristic polynomial of R yields the stated result.

Upper bound on negative eigenvalues. We derive an upper bound on the

negative eigenvalues of K by finding a lower bound on the negative eigenvaluesof K−1. We begin by partitioning K as

K =

A BT 0B −D CT

0 C E

=

[K2 CT

C E

], (12)

where K2 =

[A BT

B −D

]and C =

[0 C

]. By [5, Equation (3.4)],

K−1 =

[K−1

2 +K−12 CTS−1

2 CK−12 −K−1

2 CS−12

−S−12 CK−1

2 S−12

],

where S2 = E + CK−12 CT = E + CS−1

1 CT . Notice that

K−1 =

[K−1

2 00 0

]+

[K−1

2 C−I

]S−1

2

[CTK−1

2 −I].

Because the second term is positive semidefinite, we conclude that the eigen-

values of K−1 are greater than or equal to the eigenvalues of

[K−1

2 00 0

]. Thus,

a lower bound on the negative eigenvalues of K−12 is also a lower bound on the

negative eigenvalues of K−1. This means that an upper bound on the negativeeigenvalues of K2 is also an upper bound on the negative eigenvalues of K. Thedesired result now follows from Silvester and Wathen [36, Lemma 2.2].

Lower bound on positive eigenvalues. We begin by noting that, because E

is positive semidefinite, the eigenvalues of K are greater than or equal to thoseof

KE=0 =

A BT 0B −D CT

0 C 0

.Moreover, K and KE=0 have the same inertia, by Theorem 2. Thus, the smallestpositive eigenvalue of K is greater than or equal to the smallest positive eigen-value of KE=0. We therefore obtain a lower bound on the positive eigenvaluesof K by using energy estimates with KE=0. The eigenvalue problem associatedwith KE=0 is A BT 0

B −D CT

0 C 0

xyz

= λ

xyz

. (13)

We pre-multiply the first block row by xT , the second by yT , and the third by

8

zT to obtain

xTAx+ xTBT y = λxTx; (14a)

yTBx− yTDy + yTCT z = λyT y; (14b)

zTCy = λzT z. (14c)

The third block row of (13) gives z = 1λCy. The first block row gives

BT y = (λI −A)x.

We now consider two cases based on the value of λ.

Case I: λ < µAmin: If λ < µAmin, then (λI − A) is negative definite, so we canwrite

x = (λI −A)−1BT y.

Substituting these into (14b) and rearranging gives

λyT y = yTB(λI −A)−1BT y − yTDy +1

λyTCTCy (15a)

≥ (σBmax)2

λ− µAmin

yT y − µDmaxyT y +

(σCmin)2

λyT y. (15b)

Dividing by yT y, using the fact that λ > 0 and λ − µAmin < 0, and rearrangingyields

λ3 + (µDmax − µAmin)λ2 −(µAminµ

Dmax + (σBmax)2 + (σCmin)2

)λ+ µAmin(σCmin)2︸ ︷︷ ︸

=p(λ)

≤ 0.

By applying Corollary 1 with a = µAmin, b = σBmax, c = σCmin, and d = µDmax,we know that this polynomial has two positive roots. Moreover, p(λ) is neg-ative between these two roots: this follows from the fact that p(0) > 0 andlimλ→∞ p(λ) = ∞. Therefore, we conclude that in this case λ is greater thanor equal to the smaller positive root of p(λ), namely p+

min.

Case II: λ ≥ µAmin: If λ ≥ µAmin, then (λI−A) may be indefinite (and possiblysingular). However, we can still obtain a lower bound for λ by observing thatµAmin is greater than p+

min. As stated earlier, p(λ) has two positive roots andthe value of p(λ) is negative between those roots. Moreover, these are the onlypositive values of λ for which p(λ) < 0. We observe, after simplification, that

p(µAmin) = −(σBmax)2µAmin < 0.

Therefore, the bound λ ≥ p+min also holds in this case, which completes the

proof.

9

Remark 1. From Theorem 1 we see that when B and C are rank deficient,the internal bounds are zero. Under mild conditions on the ranks and kernels ofD and E, the statement of the theorem and its proof may be revised to obtainnonzero internal bounds in this case. However, doing so is rather technical, andsince the case of full rank B and C is common, further details on this end caseare omitted.

The matrix K0 is a special case of K, and bounds on its eigenvalues can beobtained as a direct consequence of Theorem 1.

Corollary 2 (Eigenvalue bounds, matrix K0). Define the following three cubicpolynomials as special cases of p, q and r defined in (9) with D = E = 0:

p(λ) = λ3 − µAminλ2 −

((σBmax)2 + (σCmin)2

)λ+ µAmin(σCmin)2; (16a)

q(λ) = λ3 − µAmaxλ2 −

((σBmax)2 + (σCmax)2

)λ+ µmax

A (σCmax)2; (16b)

r(λ) = λ3 − µAminλ2 −

((σBmax)2 + (σCmax)2

)λ+ µAmin(σCmax)2. (16c)

Using (16), the eigenvalues of K0 are bounded within the intervals[r−,

µAmax −√

(µAmax)2 + 4(σBmin)2

2

] ⋃ [p+

min, q+max

]. (17)

The proof of Corollary 2 is omitted; it is similar to the proof of Theorem 1,with simplifications arising from setting D = E = 0.

2.3 Tightness of the bounds

While the tightness of the bounds we have obtained depends on the problemat hand, all bounds presented in this section are attainable, as we will demon-strate with small examples. For the extremal bounds (upper positive and lowernegative), we note that both bounds hold for all double saddle-point matriceswith n = m = p = 1, as the characteristic polynomial of the 3× 3 matrix is thesame as the polynomial given in the upper positive and lower negative boundsof Theorem 1.

For the upper bound on negative eigenvalues, we consider the example (withn = m = 2, and p = 1):

K =

µAmax 0 σBmin 0 0

0 µAmax 0 σBmin 0σBmin 0 0 0 0

0 σBmin 0 −µD σC

0 0 0 σC µE

.

10

We can permute the rows and columns of K to obtain a block diagonal matrix:

KD =

µAmax σBmin 0 0 0σBmin 0 0 0 0

0 0 µAmax σBmin 00 0 σBmin −µD σC

0 0 0 σC µE

.

The upper left block of KD has as an eigenvalueµAmax−√

(µAmax)2+4(σB

min)2

2 , whichis the bound given by Theorem 1.

Finally, for the lower positive bound, we consider the matrix (with n = m =p = 2)

K =

µAmin 0 σBmax 0 0 0

0 µAmin 0 σBmax 0 0σBmax 0 −µDmax 0 σCmin 0

0 σBmax 0 −µDmax 0 σCmin

0 0 σCmin 0 0 00 0 0 σCmin 0 µE

.

As in the previous example, we can permute the rows and columns to obtain ablock diagonal matrix:

KD =

µAmin σBmax 0 0 0 0σBmax −µDmax σCmin 0 0 0

0 σCmin 0 0 0 00 0 0 µAmin σBmax 00 0 0 σBmax −µDmax σCmin

0 0 0 0 σCmin µE

.

The characteristic polynomial of the upper left block is precisely p(λ) definedin (9a); thus, the bound of Theorem 1 is obtained.

3 Eigenvalue bounds for M−1KWe now derive bounds for the preconditioned matrix M−1K, with M definedin (6).

3.1 Inertia and eigenvalue multiplicity

We begin with some observations on inertia and eigenvalue multiplicity. Tosimplify the presentation and proof of this result, we restrict ourselves to thecase that B and C have full row rank. In some later results, we will lift thisrestriction to allow for rank-deficient B and C.

Theorem 2 (Inertia and algebraic multiplicity, matrix M−1K). Let K be de-fined as in (2) and M as in (6), and suppose that B and C have full row rank.The preconditioned matrix M−1K has:

11

(i) m negative eigenvalues;

(ii) p eigenvalues in (0, 1);

(iii) n−m eigenvalues equal to 1; and

(iv) m eigenvalues greater than 1.

Proof. BecauseM is symmetric positive definite,M1/2 exists and is invertible,and the inertias of M−1/2KM−1/2 and M−1K are equal to the inertia of K.Thus, Lemma 1 establishes that M−1K has n + p positive and m negativeeigenvalues, which proves (i).

For (ii)-(iv) we split the n+p positive eigenvalues into eigenvalues less than,equal to, or greater than 1. For this we compute the inertia of the shifted,split-preconditioned matrix

M−1/2KM−1/2 − I =

0 BT 0

B −D − I CT

0 C E − I

,where B = S

−1/21 BA−1/2, C = S

−1/22 CS

−1/21 , D = S

−1/21 DS

−1/21 , E = S

−1/22 ES

−1/22 .

The number of positive, negative, and zero eigenvalues of M−1/2KM−1/2 − Iwill be equal to the number of eigenvalues of M−1K greater than, less than, orequal to 1, respectively.

Noting that E + CCT = I, we write

M−1/2KM−1/2 − I =

[0 BBT T

],

where

T :=

[−D − I CT

C −CCT

]and B =

[BT 0

].

The matrix B is in Rn×(m+p) and has rank m. We define a matrix

N :=

[0m×pIp

],

where 0m×p is the m × p zero matrix and Ip is the p × p identity matrix. Thecolumns of N form a basis for ker(B). Denote the inertia of M by In(M) =(n+, n−, n0) . It is well known (see, e.g., [12, Lemma 3.4]) that

In(M−1/2KM−1/2 − I) = In(NTT N) + (m,m, n−m).

Because NTT N = −CCT is negative definite, this gives In(M−1/2KM−1/2 −I) = (m,m+ p, n−m), which yields (ii)-(iv).

12

3.2 Derivation of bounds

It is possible to obtain eigenvalue bounds on the preconditioned systemM−1Kby using the results of Theorem 1 on the (symmetric) preconditioned systemM−1/2KM−1/2; however, some of the resulting bounds will be loose. Thereason for this is that Theorem 1 assumes no relationships between the blocksof K and considers each block individually. In this section, we will derive tighteigenvalue bounds using energy estimates, which allow us to fully exploit therelationships between the blocks of the preconditioned system.

We begin by recalling the following result from Horn and Johnson [16, The-orem 7.7.3], which will be useful throughout the analysis that follows.

Lemma 2. Let M and N be symmetric positive semidefinite matrices such thatM+N is positive definite. Then (M+N)−1M is symmetric positive semidefinitewith all eigenvalues in [0, 1].

The case D = E = 0

When D and E are both zero, M−1K has six distinct eigenvalues given by: 1,1±√

52 , and the roots of the cubic polynomial λ3 − λ2 − 2λ+ 1. We refer to Cai

et al. [6, Theorem 2.2] for proof.

The case D = 0 and E 6= 0

When D = 0, it is necessary that B have full row rank in order for S1 to beinvertible; however, C may be rank-deficient. Suppose that CT has nullity k.The following result holds.

Theorem 3 (Eigenvalue bounds, matrixM−1K, D = 0, E 6= 0). When D = 0and E 6= 0, the eigenvalues of M−1K are given by: λ = 1 with multiplicity

n−m+ k; λ = 1±√

52 , each with multiplicity m− p+ k; and p− k eigenvalues

located in each of the three intervals:[ζ−, 1−

√5

2

),[ζ+min, 1

), and

(1+√

52 , ζ+

max

],

where ζ−, ζ+min and ζ+

max are the roots of the cubic polynomial λ3− λ2− 2λ+ 1,ordered from smallest to largest. (These are approximately −1.2470, 0.4450, and1.8019, respectively.)

Proof. We write out the (left-)preconditioned operator

M−1K =

I A−1BT 0S−1

1 B 0 S−11 CT

0 S−12 C S−1

2 E

,with corresponding eigenvalue equations

x+A−1BT y = λx; (18a)

S−11 Bx+ S−1

1 CT z = λy; (18b)

S−12 Cy + S−1

2 Ez = λz. (18c)

13

We obtain n−m eigenvectors for λ = 1 by choosing x ∈ ker(B) and y, z = 0.

By considering y ∈ ker(C) and z = 0, we obtain eigenvalues λ = 1±√

52 , each

with geometric multiplicity m− p+ k.

For the remaining eigenvalues, we assume that z 6= 0 and λ /∈ 1, 1±√

52 .

From (18a) we obtain

x =1

λ− 1A−1BT y,

which we substitute into (18b) and rearrange to get

y =λ− 1

λ2 − λ− 1S−1

1 CT z.

Substituting this into (18c) gives

λ− 1

λ2 − λ− 1S−1

2 CS−11 CT z + S−1

2 Ez = λz. (19)

Because S2 = E + CS−11 CT , we can write S−1

2 E = I − S−12 CS−1

1 CT . Wesubstitute this into (19) and rearrange to obtain(

− λ2 + 2λ

)S−1

2 CS−11 CT z =

(λ3 − 2λ2 + 1

)z.

Let δj , vj, for 1 ≤ j ≤ p, denote an eigenpair of S−12 CS−1

1 CT . For k of theseeigenpairs corresponding to vj ∈ ker(CT ), we note that

S−12 Evj = (I − S−1

2 CS−11 CT )vj = vj ,

and therefore[0 0 vTj

]Tis an eigenvector ofM−1K with λ = 1. For the p−k

remaining eigenpairs, we have 0 < δj ≤ 1 (by Lemma 2). We can then write(− λ2

j + 2λj

)δjzj =

(λ3j − 2λ2

j + 1

)zj ,

where λj corresponds to an eigenpair δj , zj of S−12 CS−1

1 CT . Because zj 6= 0this implies that

λ3j − (2− δj)λ2

j − 2δjλj + 1 = 0. (20)

Thus, each of the p − k positive eigenvalues δj of S−12 CS−1

1 CT yields three

distinct corresponding eigenvalues λ(1)j , λ

(2)j , and λ

(3)j of M−1K, corresponding

to the roots of the cubic polynomial (20). These 3(p−k) eigenvalues, combinedwith the eigenvalues described earlier, account for all eigenvalues of M−1K.Substituting δj = 0 and δj = 1 into (20) gives us the three intervals for theseeigenvalues stated in the theorem.

The case D 6= 0 and E = 0

When D 6= 0 and E = 0, the bounds are the same as when D and E are bothnonzero, which are given next.

14

The case D,E 6= 0

Theorem 4 (Eigenvalue bounds, matrix M−1K, D,E 6= 0). The eigenvaluesof M−1K are bounded within the intervals[

−1 +√

5

2,

1−√

5

2

]∪[ζ+min, ζ

+max

],

where ζ+min and ζ+

max are respectively the smallest and largest positive roots of thecubic polynomial λ3−λ2−2λ+1 (approximately 0.4450 and 1.8019, respectively).

Proof. Upper bound on positive eigenvalues. We know from Theorem 2 that

the upper bound on positive eigenvalues is greater than 1, so we assume herethat λ > 1. We begin by writing the eigenvalue equations as

Ax+BT y = λAx; (21a)

Bx−Dy + CT z = λS1y; (21b)

Cy + Ez = λS2z. (21c)

From (21a) we get

x =1

λ− 1A−1BT y, (22)

and from (21c) we get z = (λS2 − E)−1Cy (because λ > 1, we are guaranteedthat λS2 − E = (λ − 1)E + λCS−1

1 CT is positive definite). Substituting thesevalues back into (21b) and pre-multiplying by yT gives(

1

λ− 1

)yTBA−1BT y + yT (D + λS1)y − yTCT (λS2 − E)−1Cy = 0.

Recalling that S1 = D +BA−1BT we can rewrite this as(1

1− λ+ λ

)yTS1y +

(1− 1

1− λ

)yTDy − yTCT (λS2 − E)−1Cy = 0. (23)

Because λ > 1, we have

yTCT (λS2 − E)−1Cy ≤ yTCT(λ(S2 − E)

)−1

Cy

=1

λyTCT

(CS−1

1 CT)−1

Cy.

Therefore, (23) gives(1

1− λ+ λ

)yTS1y+

(1− 1

1− λ

)yTDy− 1

λyTCT (CS−1

1 CT )−1Cy ≤ 0. (24)

Next, we let y = S1/21 y and rewrite the first and third terms in (24) in terms of

y:(1

1− λ+ λ

)yT y+

(1− 1

1− λ

)yTDy− 1

λyT S

−1/21 CT (CS−1

1 CT )−1CS−1/21︸ ︷︷ ︸

:=P

y ≤ 0.

(25)

15

Because P defined in (25) is an orthogonal projector, we have yTP y ≤ yT y.And because λ > 1, we have 1 − 1

1−λ > 0, which means that the second term(1− 1

1−λ

)yTDy is non-negative and can be dropped from the inequality. The

inequality (25) therefore becomes(1

1− λ+ λ

)yT y − 1

λyT y ≤ 0,

which we can divide by yT y rearrange to give

λ3 − λ2 − 2λ+ 1 ≤ 0.

This, combined with the assumption that λ > 1, gives us the stated result thatλ is less than or equal to the largest root of λ3 − λ2 − 2λ+ 1. The polynomialλ3−λ2−2λ+1 is of the form given in Corollary 1 with a = b = c = 1, d = e = 0,so its value is negative between the two positive roots.

Lower bound on negative eigenvalues. We begin from (23) and note that

when λ < 0, by similar reasoning as was shown for the upper bound on positiveeigenvalues,(

1

1− λ+ λ

)yTS1y +

(1− 1

1− λ

)yTDy − 1

λyTCT (CS−1

1 CT )−1Cy ≥ 0.

Rewriting the inequality in terms of y = S1/21 y gives(

1

1− λ+ λ

)yT y +

(1− 1

1− λ

)yTS

−1/21 DS

−1/21 y − 1

λyTP y ≥ 0,

where P is the orthogonal projector defined in (25). For the second term, note

that S−1/21 DS

−1/21 is similar to S−1

1 D which has all eigenvalues between 0 and

1 (by Lemma 2). Thus, both yTS−1/21 DS

−1/21 y and yTP y are less than or equal

to yT y and we can therefore write(1

1− λ+ λ

)yT y +

(1− 1

1− λ

)yT y − 1

λyT y ≥ 0,

which, after dividing by yT y and simplifying, yields

λ2 + λ− 1 ≤ 0.

This along with the assumption that λ < 0 gives the desired bound of λ ≥− 1+

√5

2 .

Lower bound on positive eigenvalues. Assume that 0 < λ < 1 (we know

from Theorem 2 that the lower bound is in this interval). Substituting (22) into(21b) and solving for y gives

y =

(1

1− λBA−1BT +D + λS1

)−1

︸ ︷︷ ︸=:Q

CT z, (26)

16

When 0 < λ < 1, the value 11−λ is positive, so we are guaranteed that Q in

(26) is positive definite. If z ∈ ker(CT ), (26) gives y = 0 and (22) gives x = 0,and we can see from the eigenvalue equations (21a)-(21c) that this eigenvectorcorresponds to λ = 1 (because Ez = S2z for z ∈ ker(CT )). This contradicts ourassumption that λ < 1; thus, we assume z /∈ ker(CT ).

We can then write (21c) as

C

(1

1− λBA−1BT +D + λS1

)−1

CT z + (1− λ)Ez − λCS−11 CT z = 0. (27)

When 0 < λ < 1, we have 11−λ > 1. Therefore, if we take the inner product of

zT with (27), replace D by 11−λD, and drop the non-negative term (1−λ)zTEz,

we obtain the inequality

zTC

((1

1− λ+ λ

)S1

)−1

CT z − λzTCS−11 CT z ≤ 0.

After simplifying and dividing by zTCS−11 CT z, we obtain

λ3 − λ2 − 2λ+ 1 ≤ 0. (28)

This, combined with the assumption that 0 < λ < 1, gives us that λ must begreater than or equal to the smaller positive root of (28), as required.

Upper bound on negative eigenvalues. Assume that λ < 0. We begin from

Equation (23) and note that λS2 − E is negative definite. Therefore,(1

1− λ+ λ

)yTS1y +

(1− 1

1− λ

)yTDy ≤ 0.

Since 1 − 11−λ > 0, the yTDy term is non-negative and can be dropped while

keeping the inequality, giving us(1

1− λ+ λ

)yTS1y ≤ 0.

We thus require1

1− λ+ λ ≤ 0 with λ < 0,

which leads to the desired bound λ ≤ 1−√

52 .

4 Bounds for preconditioners with approxima-tions of Schur complements

In practice, it is too expensive to invert A, S1, and S2 exactly. In this sectionwe examine eigenvalue bounds on matrices of the form M−1K, where M uses

17

symmetric positive definite and ideally spectrally equivalent approximations forA, S1, and S2. Specifically, we consider the approximate block diagonal precon-ditioner

M =

A 0 0

0 S1 0

0 0 S2

, (29)

with A, S1 and S2 satisfying the following:

Assumption 5. Let Λ(·) denote the spectrum of a matrix. The diagonal blocksof the approximate preconditioner M, given by A, S1, and S2, satisfy:

i. Λ(A−1A) ∈ [α0, β0];

ii. Λ(S−11 S1) ∈ [α1, β1];

iii. Λ(S−12 S2) ∈ [α2, β2],

where 0 < αi ≤ 1 ≤ βi.

We note that to obtain spectral equivalence we seek approximations thatyield values of αi independent of the mesh size and bounded uniformly awayfrom zero. It is also worth noting that the values of αi and βi are typically notexplicitly available. We briefly address this at the end of this section, followingour derivation of the bounds.

To simplify our analyses, we define:

A−1A =: Q0; (30a)

S−11 S1 =: Q1; (30b)

S−12 S2 =: Q2. (30c)

To derive the eigenvalue bounds, we consider the split preconditioned matrix

M−1/2KM−1/2 =

Q0 BT 0

B −D CT

0 C E

, (31)

where Q0 = A−1/2AA−1/2, B = S−1/21 BA−1/2, C = S

−1/22 CS

−1/21 , D =

S−1/21 DS

−1/21 , and E = S

−1/22 ES

−1/22 . We proceed by bounding the eigen-

values and singular values of the blocks of M−1/2KM−1/2 and then applyingthe results for general double saddle-point matrices presented in Section 2.2. Inorder to avoid providing internal eigenvalue bounds equal to zero (see Remark1) we assume here that B and C have full row rank.

Lemma 3. When B and C have full row rank, bounds on the eigenvalues/singularvalues of the blocks of the matrix M−1/2KM−1/2 are as follows:

• Q0: eigenvalues are in [α0, β0].

18

• B: singular values are in[√

α0α1

1+ηD,√β0β1

], where ηD is the maximal

eigenvalue of (BA−1BT )−1D.

• C: singular values are in[√

α1α2

1+ηE,√β1β2

], where ηE is the maximal

eigenvalue of (CS−11 CT )−1E.

• D: eigenvalues are in [0, β1].

• E: eigenvalues are in [0, β2].

Proof. The eigenvalue bounds on Q0 follow from the fact that Q0 is similarto Q0. For D and E, the lower bounds follow from the fact that D and Eare semidefinite (if D and/or E are definite, this bound will be loose, but weuse the zero bound to simplify some results that we present in this section).

For the upper bound on D, we note that D = S−1/21 DS

−1/21 , which is similar

to S−11 D = Q1S

−11 D. The eigenvalues of S−1

1 D are less than or equal to 1by Lemma 2, meaning that those of Q1S

−11 D are less than or equal to β1.

Analogous reasoning gives the upper bound for E.We now present the results for B. For the upper bound, note that the matrix

BBT = S−1/21 BA−1BT S

−1/21 is similar to S−1

1 BA−1BT = Q1S−11 BA−1BT .

Because the eigenvalues of Q1 are in [α1, β1], we need only bound the eigenvaluesof S−1

1 BA−1BT . These are the same as the nonzero eigenvalues of

A−1BTS−11 B = Q0A

−1BTS−11 B.

The nonzero eigenvalues of A−1BTS−11 B are the same as those of S−1

1 BA−1BT ,which are all less than or equal to 1 by Lemma 2. Thus, the eigenvalues ofQ0A

−1BTS−11 B are less than or equal to β0, from which we conclude that the

eigenvalues of BBT are less than or equal to β0β1, giving an upper singularvalue bound of

√β0β1. Similarly, a lower bound on the eigenvalues of BBT is

given by α0α1 times a lower bound on the eigenvalues of S−11 BA−1BT . Because

we have assumed that B is full rank, BA−1BT is invertible, implying that

S−11 BA−1BT =

((BA−1BT )−1S1

)−1=(I + (BA−1BT )−1D

)−1.

The stated result then follows because the eigenvalues of S−11 BA−1BT are

greater than or equal to 11+ηD

, where ηD is the maximal eigenvalue of (BA−1BT )−1D.

Thus, a lower bound on the singular values of B is given by the square root ofthis value.

The bounds for C are obtained in the same way as those for B.

We can now present bounds on the eigenvalues of M−1/2KM−1/2. We definethree cubic polynomials:

u(λ) = λ3 + (β1 − α0)λ2 −(α0β1 +

α1α2

1 + ηE+ β0β1

)λ+

α0α1α2

1 + ηE; (32a)

v(λ) = λ3 − (β0 + β2)λ2 + (β0β2 − β0β1 − β1β2)λ+ 2β0β1β2; (32b)

w(λ) = λ3 + (β1 − α0)λ2 − (α0β1 + β0β1 + β1β2)λ+ α0β1β2. (32c)

19

These polynomials all have two positive roots and one negative root, by Corol-lary 1. As in Section Section 2, we let u− denote the (single) negative rootof a polynomial u, and let u+

min and u+max respectively denote the smallest and

largest positive roots.

Theorem 6 (Eigenvalue bounds, matrix M−1/2KM−1/2). When B and Chave full row rank, the eigenvalues of M−1/2KM−1/2 are bounded within theintervals w−, β0 −

√β2

0 + 4α0α1

1+ηD

2

⋃ [u+

min, v+max

]. (33)

Proof. The stated bounds follow from Lemma 3 and Theorem 1.

Remark 2. Obtaining ηD and ηE requires computation of eigenvalues oftwo matrices related to the Schur complements S1 and S2: (S1 − D)−1D =(BA−1BT )−1D and (S2 − E)−1E = (CS−1

1 CT )−1E, respectively. As we havepreviously mentioned, in practice when solving (1) the matrices S1 and S2 wouldtypically be approximated by sparse and easier-to-invert matrices, rather thanformed and computed explicitly. Therefore, we cannot expect to compute ηDand ηE exactly. Spectral equivalence relations may be helpful in providingreasonable approximations here. For example, in common formulations of theStokes-Darcy problem [7] the matrix A is a discrete Laplacian and B and C arediscrete divergence operators (or scaled variations thereof). In such a case, forcertain finite element discretizations BA−1BT = S1 − D is spectrally equiva-lent to the mass matrix [8, Section 5.5]. Denote it by Q. Then, estimating themaximal eigenvalue of ηD amounts to computing an approximation to the max-imal eigenvalue of Q−1D, which is computationally straightforward given thefavorable spectral properties of Q. If the above-mentioned Schur complement isapproximated by the mass matrix, then it can be shown that CS−1

1 CT = S2−Eis strongly related to the scalar Laplacian, and therefore maximal eigenvalue ofηE would be relatively easy to approximate as well.

When D = 0, ηD and the maximal eigenvalue of D are zero. Similarly, whenE = 0, ηE and the maximal eigenvalue of E are zero. In these cases, we cansimplify some of the bounds of Theorem 6. Some of the cubic polynomials thatdefine the bounds will change in these cases. We will use a bar (e.g., u) to denotecubic polynomials where D = 0, a hat (e.g., u) to denote the polynomials whereE = 0, and both (e.g., ˆu) to denote both D and E being zero. We define the

20

following cubic polynomials:

u(λ) = λ3 − α0λ2 −

(α1α2

1 + ηE+ β0β1

)λ+

α0α1α2

1 + ηE; (34a)

u(λ) = λ3 + (β1 − α0)λ2 − (α0β1 + α1α2 + β0β1)λ+ α0α1α2; (34b)

ˆu(λ) = λ3 − α0λ2 − (α1α2 + β0β1)λ+ α0α1α2; (34c)

v(λ) = λ3 − β0λ2 − (β0β1 + β1β2)λ+ β0β1β2; (34d)

w(λ) = λ3 − α0λ2 − (β0β1 + β1β2)λ+ α0β1β2. (34e)

The following results then follow directly from Theorem 6; the proof is omitted.

Corollary 3. In the case D = 0, E 6= 0: the eigenvalues of M−1/2KM−1/2

are bounded within the intervals[w−,

β0 −√β2

0 + 4α0α1

2

] ⋃ [u+

min, v+max

]. (35)

In the case D 6= 0, E = 0: the eigenvalues are bounded inw−, β0 −√β2

0 + 4α0α1

1+ηD

2

⋃ [u+

min, v+max

]. (36)

In the case D = 0, E = 0: the eigenvalues are bounded in[w−,

β0 −√β2

0 + 4α0α1

2

] ⋃ [ˆu+

min, v+max

]. (37)

Remark 3. There is some looseness in the bounds of Theorem 6 when appliedto the exact preconditioning case (i.e., αi = βi = 1). This is a consequence ofthe fact that Theorem 6 is based on the bounds for unpreconditioned matri-ces, which consider each matrix block individually, as opposed to the energyestimates approach in Section 3, which fully exploits the relationships betweenthe blocks of the preconditioned matrix. For the extremal (lower negative andupper positive) bounds, the looseness is minor: Theorem 6 gives a bound of 2for the positive eigenvalues and approximately −1.9 on the negative eigenvalues,while we know from Theorem 4 that tight bounds are approximately 1.8 and−1.6, respectively. For the interior bounds, however, we note that the boundsmay be quite loose if ηD or ηE is very large. Fortunately, this is often not aconcern in practical settings as having a large ηD or ηE generally implies thatthe spectral norm of D or E is large relative to that of BA−1BT or CS−1

1 CT ,respectively. Often D and/or E are regularization terms, which tend to havea fairly small norm. Nonetheless, it should be acknowledged that the interiorbounds may be pessimistic for some problems.

21

Remark 4. The values of αi and βi are rarely available in practical appli-cations; they are typically coercivity constants or related quantities, which areproven to be independent of the mesh size but are not known explicitly. Thevalues of βi should not typically generate a difficulty, and approximating themby 1 or a value close to 1 should provide a reasonable approximation for thebounds.

For the αi, let us provide some (partial) observations. For the solutions ofthe quadratic equations in Corollary 3, we can use the Taylor approximation√

1 + x / 1 + x2 for 0 < x 1 to conclude that

β0 −√β2

0 + 4α0α1

2' −α0α1

β0.

This means that if α0 and α1 are small, then the above displayed expressionwould be a valid (albeit slightly less tight) upper negative bound, and if the αiare uniformly bounded away from zero, then so is the bound.

5 Numerical experiments

In this section we consider two slightly different variants of a Poisson controlproblem as in [30]. First, we consider a distributed control problem with Dirich-let boundary conditions:

minu,f

1

2||u− u||2L2(Ω) +

β

2||f ||2L2(Ω) (38a)

s.t. −∇2u = f in Ω, (38b)

u = g on ∂Ω, (38c)

where u is the state, u is the desired state, 0 < β 1 is a regularizationparameter, f is the control, and Ω is the domain with boundary ∂Ω. Upondiscretization, we obtain the systemM K 0

K 0 −M0 −M βM

︸ ︷︷ ︸

=:K

uhλfh

=

bd0

, (39)

where M is a symmetric positive definite mass matrix and K is a symmetricpositive definite discrete Laplacian. All blocks of K are square (i.e., n = m = p).

As a second experiment we consider a boundary control problem:

minu,f

1

2||u− u||2L2(Ω) +

β

2||g||2L2(Ω) (40a)

s.t. −∇2u = 0 in Ω, (40b)

∂u

∂n= g on ∂Ω, (40c)

22

which after discretization yields the linear systemM K 0K 0 −E0 −ET βMb

︸ ︷︷ ︸

=:K∂

uhλgh

=

b∂d∂0

, (41)

where Mb ∈ Rnb×nb (with nb < n) is a boundary mass matrix. Thus, in thedistributed control problem the mass matrix in the (2,3)/(3,2)-block is squareand in the boundary control problem we consider a rectangular version of it.

In all experiments that follow we set Ω to be the unit square. We use uniformQ1 finite elements and set β = 10−3. The MATLAB code of Rees [29] was usedto generate the linear systems.

5.1 Eigenvalues of unpreconditioned matrices

Here we compare the eigenvalues of K (39) and K∂ (41) to the eigenvalue boundspredicted by Theorem 1. We use MATLAB’s eigs/svds functions to computethe minimum and maximum eigenvalues/singular values of the matrix blocksM,K,Mb, and E.

We note that for the distributed control matrix K, all blocks are squareand the (3,3)-block is positive definite; therefore, we can also use our results toobtain bounds on the re-ordered matrix

Kflip =

βM −M 0−M 0 K

0 K M

. (42)

Both orderings generate the same extremal bounds but different interior bounds.Comparisons of the predicted eigenvalue bounds to the actual eigenvalues

are shown in Section 2. In the distributed control case, we show the boundsobtained for both the original matrix K and those for the reordered matrixKflip. In all cases, the bounds for the extremal eigenvalues are quite tight: thisis because the K block has the largest eigenvalues (O(1) compared to O(h2) forthe others – see [8, Proposition 1.29 and Theorem 1.32]). Therefore, the largestpositive and smallest negative eigenvalue of both K and K∂ tend towards µKmax

and −µKmax, respectively. Referring to the proofs of the extremal bounds inTheorem 1, the 3 × 3 matrices R will contain a µKmax (or −µKmax) term, withother lower-order terms. This means that the extremal eigenvalues of R (andtherefore the predicted eigenvalue bounds) will also be close to ±µKmax.

It is more difficult to capture the interior bounds. In the distributed controlcase, each ordering (either the original ordering with M in the leading blockor the flipped ordering with βM in the leading block) gives one bound that isquite tight and one that is loose. In the boundary control case, both interiorbounds are loose.

23

2-2 2-3 2-4 2-5 2-6

h

10-10

10-5

100

Original ordering

Flipped ordering

(a) Positive eigenvalues of K

2-2 2-3 2-4 2-5 2-6

h

-10-5

-10-4

-10-3

-10-2

-10-1

-100

-101

Original ordering

Flipped ordering

(b) Negative eigenvalues of K

2-2 2-3 2-4 2-5 2-6

h

10-10

10-5

100

(c) Positive eigenvalues of K∂

2-2 2-3 2-4 2-5 2-6

h

-10-5

-10-4

-10-3

-10-2

-10-1

-100

-101

(d) Negative eigenvalues of K∂

Figure 2: Largest and smallest positive (left) and negative (right) eigenvaluesof the distributed control matrix K (top) and the boundary control matrix K∂(bottom). Blue circles indicate the eigenvalues, and black lines the bounds givenby Theorem 1. For K, the dashed lines indicate the bounds obtained by applyingTheorem 1 to the reordered matrix Kflip.

5.2 Eigenvalues of preconditioned matrices

We now consider preconditioning strategies for PDE-constrained optimization.We examine eigenvalue bounds with both exact and approximate Schur com-plements.

For the distributed control problem, we work with the reordered matrix Kflip

(42). The Schur complement preconditioner for Kflip is

M =

βM 0 00 1

βM 0

0 0 M + βKM−1K

. (43)

The first and second blocks are mass matrices, which are cheap to invert, sowe leave these terms as they are. For the second Schur complement S2 =M+βKM−1K, we use the approximation S2 =

(M +

√βK)M−1

(M +

√βK)

24

proposed by Pearson and Wathen [26] to obtain the preconditioner:

M =

βM 0 00 1

βM 0

0 0(M +

√βK)M−1

(M +

√βK) . (44)

Per [26, Theorem 4], the eigenvalues of S−12 S2 satisfy Λ(S−1

2 S2) ∈[

12 , 1]. Thus

M satisfies Theorem 5 with α0 = β0 = α1 = β1 = 1, α2 = 12 , and β2 = 1.

0 100 200 300 400 500 600 700

i

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

i

(a) M−1Kflip

0 100 200 300 400 500 600 700

i

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

i

(b) M−1Kflip

Figure 3: Eigenvalue plots for reordered distributed Poisson control matrix Kflip,

with exact preconditioner M (left) and approximate preconditioner M (right).Eigenvalues are shown by the blue circles; eigenvalue bounds (from Theorem 3on left and Corollary 3 on right) are shown by lines.

Plots of the preconditioned eigenvalues for h = 2−4 are shown in Section 3.The value ηE , defined as the maximal eigenvalue of

(CS−11 CT )−1E = β(KM−1K)−1M,

is approximately 2.6 × 10−7. We note that for 2D problems with uniform Q1finite element discretizations the value ηE is O(βh4), and will thus be small ingeneral.

Comparing Figures 3a and 3b, we notice that the bounds on the negativeeigenvalue bounds do not change when we use the approximate Schur comple-ment, but the lower positive bound becomes smaller (from 0.4450 with the exactSchur complement to 0.2929 for the approximate Schur complement) and theupper positive bound becomes larger (1.8019 in the exact case and 2 in theapproximate case). We note that the eigenvalues appear to be very close tothe predicted bounds except for the upper positive eigenvalues of M−1K. Asdiscussed in Remark 3, this kind of minor looseness in the upper bound canhappen when we have highly accurate Schur complement approximations (as inthis case, where we are exactly inverting the two M blocks).

For the matrix K∂ arising from the boundary control problem, the Schurcomplement preconditioner is

M∂ =

M 0 00 KM−1K 0

0 0 βMb + ET(KM−1K

)−1E

. (45)

25

In practice, the first Schur complement KM−1K can be inverted approximatelywith, for example, a multigrid method. For the second Schur complement,we note that for the 2D Poisson control problem on a uniform Q1 grid, theeigenvalues of K are between O(h2) and O(1), while those of the mass matrices

are all O(h2). Therefore, the term βMb will dominate ET(KM−1K

)−1E for all

but very small values of β (which are not commonly used in practice). Therefore,an approximate preconditioner for K∂ is given by

M∂ =

M 0 00 KM−1K 00 0 βMb

. (46)

We note that this preconditioner is presented in [30], though it is derived therefrom the block-2× 2 formulation of K.

0 200 400 600 800

i

-1.5

-1

-0.5

0

0.5

1

1.5

2

i

(a) M−1∂ K∂

0 200 400 600 800

i

-10

-5

0

5

10

i

(b) M−1∂ K∂ (blue); M−1

∂ K∂ (red)

Figure 4: Eigenvalue plots for the boundary control problem K∂ , with exact pre-conditionerM and approximate preconditioner M. On left, eigenvalue bounds(from Theorem 3) are shown by horizontal lines. Right: all but the single small-est and largest eigenvalues for both preconditioners; values for M are in blue,and those for M are in red.

Plots of the preconditioned eigenvalues for h = 2−4 are shown in Figure 4.For M−1

∂ K∂, we note that there is a single large-magnitude positive eigenvalue

and a large-magnitude negative eigenvalue, whose absolute values are nearly 200,and they make it difficult to see how the other eigenvalues compare to to thoseof M−1

∂ K∂. Thus in Figure 4b, we omit the largest and smallest eigenvalues of

M−1∂ K∂

and overlay the others on the corresponding eigenvalues ofM−1∂ K∂

. We

notice that most other eigenvalues of M−1∂ K∂

remain close to those ofM−1∂ K∂

.We also note that the performance of this preconditioner depends on β: inparticular, the performance of the Schur complement approximation deterioratesfor small β. We refer to [26, 30] for further discussion of this.

It is evident from Figure 4a that our bounds are tight and effective. Unlikein the boundary control example, the Schur complement approximation S2 usedhere does not have β- and h-independent constants α2, β2 such that Λ(S−1

2 S2) ∈[α2, β2], so our analyses on Schur complement approximations in Section Section

26

4 are difficult to apply. Nonetheless, we observe from Figure 4b that most ofthe eigenvalues of M−1

∂ K∂ remain very close to those of M−1∂ K∂ . Thus, we

see that the eigenvalue bounds for the “ideal” Schur complement preconditionermay still be of use, provided that we have an effective approximation of theSchur complement.

6 Conclusions

The increasing importance of double saddle-point systems requires attentionto spectral properties of the matrices involved. We have shown that energyestimates are an effective tool for obtaining eigenvalue bounds in this case. Theassumptions we make are rather general, and the analysis covers a large classof problems.

There are several directions for potential future work. Following Remark ??,specific assumptions on the magnitudes of the norms of the matrices D and Emay yield additional results and insights on the bounds. The rank structure ofthe blocks may also have a significant effect, and it may be useful to eliminatethe positive definiteness requirement of A and/or consider rank-deficient B andC. It may also be useful to consider nonsymmetric double saddle-point systems,and block triangular rather than block diagonal preconditioners, although whensymmetry is lost eigenvalue bounds may not be a sufficient tool for predictingconvergence rates of Krylov subspace solvers.

The tightness of our bounds indicates that spectral analysis is beneficialin capturing the properties of the matrices involved. This, in turn, makes itpossible to effectively predict the convergence rate of Krylov subspace solversfor this important class of problems.

References

[1] J. H. Adler, T. R. Benson, E. C. Cyr, P. E. Farrell, S. P. MacLachlan, andR. S. Tuminaro. Monolithic multigrid methods for magnetohydrodynamics.SIAM Journal on Scientific Computing, 43(5):S70–S91, 2021.

[2] F. Ali Beik and M. Benzi. Iterative methods for double saddle point sys-tems. SIAM Journal on Matrix Analysis and Applications, 39(2):902–921,2018.

[3] A. Beigl, J. Sogn, and W. Zulehner. Robust preconditioners for multiplesaddle point problems and applications to optimal control problems. SIAMJournal on Matrix Analysis and Applications, 41(4):1590–1615, 2020.

[4] M. Benzi. Preconditioning techniques for large linear systems: a survey. J.Comput. Phys., 182(2):418–477, 2002.

[5] M. Benzi, G. H. Golub, and J. Liesen. Numerical solution of saddle pointproblems. Acta Numerica, 14:1–137, 2005.

27

[6] M. Cai, G. Ju, and J. Li. Schur complement based preconditioners fortwofold and block tridiagonal saddle point problems, 2021.

[7] M. Cai, M. Mu, and J. Xu. Preconditioning techniques for a mixedStokes/Darcy model in porous media applications. Journal of Computa-tional and Applied Mathematics, 233(2):346 – 355, 2009.

[8] H. C. Elman, D. J. Silvester, and A. J. Wathen. Finite elements and fastiterative solvers: with applications in incompressible fluid dynamics. Nu-merical Mathematics and Scientific Computation. Oxford University Press,New York, 2005.

[9] M. Ferronato, A. Franceschini, C. Janna, N. Castelletto, and H. A.Tchelepi. A general preconditioning framework for coupled multiphysicsproblems with application to contact- and poro-mechanics. Journal ofComputational Physics, 398:108887, 2019.

[10] G. Gatica and N. Heuer. A dual-dual formulation for the coupling of mixed-FEM and BEM in hyperelasticity. SIAM Journal on Numerical Analysis,38:380–400, July 2000.

[11] N. Gould and V. Simoncini. Spectral analysis of saddle point matrices withindefinite leading blocks. SIAM J. Matrix Analysis Applications, 31:1152–1171, 01 2009.

[12] N. I. Gould. On practical conditions for the existence and uniqueness ofsolutions to the general equality quadratic programming problem. Mathe-matical Programming, 32:90–99, 1985.

[13] C. Greif, E. Moulding, and D. Orban. Bounds on eigenvalues of matri-ces arising from interior-point methods. SIAM Journal on Optimization,24(1):49–83, 2014.

[14] R. Hiptmair. Operator preconditioning. Comput. Math. Appl., 52(5):699–706, 2006.

[15] K. E. Holter, M. Kuchta, and K.-A. Mardal. Robust preconditioning forcoupled Stokes-Darcy problems with the Darcy problem in primal form.Comput. Math. Appl., 91:53–66, 2021.

[16] R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge UniversityPress, 1 edition, 1985.

[17] I. C. F. Ipsen. A note on preconditioning nonsymmetric matrices. SIAMJ. Sci. Comput., 23(3):1050–1051, 2001.

[18] D. E. Keyes, L. C. McInnes, C. S. Woodward, W. Gropp, E. Myra, M. Per-nice, J. Bell, J. Brown, A. Clo, J. M. Connors, E. M. Constantinescu, D. J.Estep, K. Evans, C. Farhat, A. Hakim, G. Hammond, G. Hansen, J. Hill,T. Isaac, X. Jiao, K. E. Jordan, D. K. Kaushik, E. Kaxiras, A. E. Koniges,

28

K. Lee, A. Lott, Q. Lu, J. Magerlein, R. Maxwell, M. McCourt, M. Mehl,R. P. Pawlowski, A. P. Randles, D. R. Reynolds, B. Riviere, U. Rude, T. D.Scheibe, J. N. Shadid, B. Sheehan, M. S. Shephard, A. R. Siegel, B. Smith,X. Tang, C. Wilson, and B. I. Wohlmuth. Multiphysics simulations: Chal-lenges and opportunities. Int. J. High Perform. Comput. Appl., 27(1):4–83,2013.

[19] U. Langer, G. Of, O. Steinbach, and W. Zulehner. Inexact data-sparseboundary element tearing and interconnecting methods. SIAM Journal onScientific Computing, 29(1):290–314, 2007.

[20] K.-A. Mardal and R. Winther. Preconditioning discretizations of systemsof partial differential equations. Numer. Linear Algebra Appl., 18(1):1–40,2011.

[21] B. Morini, V. Simoncini, and M. Tani. Spectral estimates for unreducedsymmetric KKT systems arising from interior point methods. Numer. Lin-ear Algebra Appl., 23:776–800, 2016.

[22] M. F. Murphy, G. H. Golub, and A. J. Wathen. A note on preconditioningfor indefinite linear systems. SIAM J. Sci. Comput., 21(6):1969–1972, 2000.

[23] C. C. Paige and M. A. Saunders. Solutions of sparse indefinite systems oflinear equations. SIAM J. Numer. Anal., 12(4):617–629, 1975.

[24] J. W. Pearson and J. Pestana. Preconditioners for Krylov subspace meth-ods: an overview. GAMM-Mitt., 43(4):e202000015, 35, 2020.

[25] J. W. Pearson and A. Potschka. A note on symmetric positive definitepreconditioners for multiple saddle-point systems, 2021.

[26] J. W. Pearson and A. J. Wathen. A new approximation of the Schur com-plement in preconditioners for PDE-constrained optimization. NumericalLinear Algebra with Applications, 19(5):816–829, 2012.

[27] J. Pestana and A. J. Wathen. Natural preconditioning and iterative meth-ods for saddle point systems. SIAM Rev., 57(1):71–91, 2015.

[28] A. Ramage and E. C. Gartland, Jr. A preconditioned nullspace method forliquid crystal director modeling. SIAM J. Sci. Comput., 35(1):B226–B247,2013.

[29] T. Rees. Github - tyronerees/poisson-control. https://github.com/

tyronerees/poisson-control, 2010. Accessed: 2021-10-22.

[30] T. Rees, H. Dollar, and A. Wathen. Optimal solvers for PDE-constrainedoptimization. SIAM Journal on Scientific Computing, 32(1):271–298, 2010.

[31] S. Rhebergen, G. Wells, A. Wathen, and R. Katz. Three-field block precon-ditioners for models of coupled magma/mantle dynamics. SIAM Journalon Scientific Computing, 37(5):A2270–A2294, 2015.

29

[32] M. Rozloznık. Saddle-point problems and their iterative solution. NecasCenter Series. Birkhauser/Springer, Cham, 2018.

[33] D. Ruiz, A. Sartenaer, and C. Tannier. Refining the lower bound on the pos-itive eigenvalues of saddle point matrices with insights on the interactionsbetween the blocks. SIAM Journal on Matrix Analysis and Applications,39(2):712–736, 2018.

[34] T. Rusten and R. Winther. A preconditioned iterative method for saddle-point problems. SIAM J. Matrix Anal. Appl., 13:887–904, 1992.

[35] Y. Saad. Iterative methods for sparse linear systems. Society for Industrialand Applied Mathematics, Philadelphia, PA, second edition, 2003.

[36] D. Silvester and A. Wathen. Fast iterative solution of stabilised Stokessystems part II: Using general block preconditioners. SIAM Journal onNumerical Analysis, 31(5):1352–1367, 1994.

[37] J. Sogn and W. Zulehner. Schur complement preconditioners for multiplesaddle point problems of block tridiagonal form with application to opti-mization problems. IMA Journal of Numerical Analysis, 39(3):1328–1359,05 2018.

[38] A. J. Wathen. Preconditioning. Acta Numer., 24:329–376, 2015.

30