Lecture Notes Methods of Mathematical Physics MATH 535

Lecture NotesMethods of Mathematical Physics

MATH 535Instructor: Ivan Avramidi

Textbook: S. Hassani, Mathematical Physics (Springer, 1999)

New Mexico Institute of Mining and Technology

Socorro, NM 87801

August 22, 2011

Author: Ivan Avramidi; File: mathphyshass.tex; Date: August 26, 2013; Time: 16:05

Contents

1 Preliminaries 31.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Finite-Dimensional Vector Spaces 52.1 Vectors and Linear Transformations . . . . . . . . . . . . . . . . 5

2.1.1 Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . 52.1.2 Inner Product and Norm . . . . . . . . . . . . . . . . . . 82.1.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . 112.1.4 Linear Transformations. . . . . . . . . . . . . . . . . . . 122.1.5 Algebras . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.2 Operator Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . 192.2.1 Algebra of Operators on a Vector Space . . . . . . . . . . 192.2.2 Derivatives of Functions of Operators . . . . . . . . . . . 232.2.3 Self-Adjoint and Unitary Operators . . . . . . . . . . . . 272.2.4 Trace and Determinant . . . . . . . . . . . . . . . . . . . 282.2.5 Finite Difference Operators . . . . . . . . . . . . . . . . . 302.2.6 Projection Operators . . . . . . . . . . . . . . . . . . . . 312.2.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . 33

2.3 Matrix Representation of Operators . . . . . . . . . . . . . . . . 362.3.1 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . 362.3.2 Operation on Matrices . . . . . . . . . . . . . . . . . . . 382.3.3 Inverse Matrix . . . . . . . . . . . . . . . . . . . . . . . 432.3.4 Trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442.3.5 Determinant . . . . . . . . . . . . . . . . . . . . . . . . . 442.3.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . 47

2.4 Spectral Decomposition . . . . . . . . . . . . . . . . . . . . . . . 492.4.1 Direct Sums . . . . . . . . . . . . . . . . . . . . . . . . . 492.4.2 Invariant Subspaces . . . . . . . . . . . . . . . . . . . . . 50

I

II CONTENTS

2.4.3 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . 512.4.4 Spectral Decomposition . . . . . . . . . . . . . . . . . . 522.4.5 Functions of Operators . . . . . . . . . . . . . . . . . . . 542.4.6 Polar Decomposition . . . . . . . . . . . . . . . . . . . . 562.4.7 Real Vector Spaces . . . . . . . . . . . . . . . . . . . . . 562.4.8 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . 59

3 Infinite-Dimensional Vector Spaces 613.1 Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

3.1.1 Homework . . . . . . . . . . . . . . . . . . . . . . . . . 623.2 Convergence in Normed Spaces . . . . . . . . . . . . . . . . . . 63

3.2.1 Finite Dimensional Normed Spaces . . . . . . . . . . . . 703.2.2 Homework . . . . . . . . . . . . . . . . . . . . . . . . . 72

3.3 Banach Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 733.3.1 Homework . . . . . . . . . . . . . . . . . . . . . . . . . 78

3.4 Linear Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . 793.4.1 Homework . . . . . . . . . . . . . . . . . . . . . . . . . 87

3.5 Completion of Normed Spaces . . . . . . . . . . . . . . . . . . . 883.5.1 Homework . . . . . . . . . . . . . . . . . . . . . . . . . 90

4 Hilbert Spaces and Orthonormal Systems 914.1 Inner Product Spaces . . . . . . . . . . . . . . . . . . . . . . . . 91

4.1.1 Finite-dimensional spaces . . . . . . . . . . . . . . . . . 924.1.2 Spaces of sequences . . . . . . . . . . . . . . . . . . . . 924.1.3 Spaces of continuous functions . . . . . . . . . . . . . . . 924.1.4 Spaces of square integrable functions . . . . . . . . . . . 934.1.5 Real Inner Product Spaces . . . . . . . . . . . . . . . . . 944.1.6 Direct Sum of Inner Product Spaces . . . . . . . . . . . . 944.1.7 Tensor Products of Inner Product Spaces . . . . . . . . . 944.1.8 Homework . . . . . . . . . . . . . . . . . . . . . . . . . 94

4.2 Norm in an Inner Product Space . . . . . . . . . . . . . . . . . . 954.2.1 Homework . . . . . . . . . . . . . . . . . . . . . . . . . 96

4.3 Hilbert Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 974.3.1 Finite dimensional Hilbert spaces . . . . . . . . . . . . . 974.3.2 Spaces of sequences . . . . . . . . . . . . . . . . . . . . 974.3.3 Spaces of continuous functions . . . . . . . . . . . . . . . 974.3.4 Spaces of square integrable functions. . . . . . . . . . . . 974.3.5 Sobolev Spaces . . . . . . . . . . . . . . . . . . . . . . . 98

mathphyshass.tex; August 26, 2013; 16:05; p. 1

CONTENTS 1

4.3.6 Homework . . . . . . . . . . . . . . . . . . . . . . . . . 994.4 Strong and Weak Convergence . . . . . . . . . . . . . . . . . . . 100

4.4.1 Homework . . . . . . . . . . . . . . . . . . . . . . . . . 1014.5 Orthogonal and Orthonormal Systems . . . . . . . . . . . . . . . 102

4.5.1 Homework . . . . . . . . . . . . . . . . . . . . . . . . . 1024.6 Properties of Orthonormal Systems . . . . . . . . . . . . . . . . . 103

4.6.1 Homework . . . . . . . . . . . . . . . . . . . . . . . . . 1064.7 Trigonometric Fourier Series . . . . . . . . . . . . . . . . . . . . 107

4.7.1 Homework . . . . . . . . . . . . . . . . . . . . . . . . . 1094.8 Orthonormal Complements and Projection Theorem . . . . . . . . 110

4.8.1 Homework . . . . . . . . . . . . . . . . . . . . . . . . . 1154.9 Linear Functionals and the Riesz Representation Theorem . . . . 116

4.9.1 Homework . . . . . . . . . . . . . . . . . . . . . . . . . 1184.10 Separable Hilbert Spaces . . . . . . . . . . . . . . . . . . . . . . 119

4.10.1 Homework . . . . . . . . . . . . . . . . . . . . . . . . . 1224.11 Hilbert Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1224.12 Generalized Functions . . . . . . . . . . . . . . . . . . . . . . . 1254.13 Orthogonal Polynomials . . . . . . . . . . . . . . . . . . . . . . 1254.14 Fourier Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

5 Complex Analysis 1275.1 Complex calculus . . . . . . . . . . . . . . . . . . . . . . . . . . 1275.2 Calculus of Residues . . . . . . . . . . . . . . . . . . . . . . . . 1275.3 Advanced Topics . . . . . . . . . . . . . . . . . . . . . . . . . . 127

6 Operators on Hilbert Spaces 1296.1 Operator Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 1296.2 Integral Equations . . . . . . . . . . . . . . . . . . . . . . . . . . 1296.3 Sturm-Liouville Systems . . . . . . . . . . . . . . . . . . . . . . 129

Bibliography 129

Answers To Exercises 133

Notation 135


2 CONTENTS


Chapter 1

Preliminaries

1.1 Preliminaries• Sets, subsets, empty set, proper subset, universal set, union, intersection,

power set, complement, Cartesian product

• Equivalence relations, equivalence classes, partitions, quotient set

• Maps, domain, codomain, image, preimage, graph, range, injections, sur-jections, bijections, binary operations

• Example.

• Metric spaces, sequences, convergence, Cauchy sequence, completeness

• Cardinality, countably infinite sets, uncountable sets

• Mathematical induction

3

4 CHAPTER 1. PRELIMINARIES


Chapter 2

Finite-Dimensional Vector Spaces

2.1 Vectors and Linear Transformations

2.1.1 Vector Spaces

• A vector space consists of a set E, whose elements are called vectors, anda field F (such as R or C), whose elements are called scalars. There are twooperations on a vector space:

1. Vector addition, + : E × E → E, that assigns to two vectors u, v ∈ Eanother vector u + v, and

2. Multiplication by scalars, · : R × E → E, that assigns to a vectorv ∈ E and a scalar a ∈ R a new vector av ∈ E.

The vector addition is an associative commutative operation with an addi-tive identity. It satisfies the following conditions:

1. u + v = v + u, ∀u, v, ∈ E

2. (u + v) + w = u + (v + w), ∀u, v,w ∈ E

3. There is a vector 0 ∈ E, called the zero vector, such that for any v ∈ Ethere holds v + 0 = v.

4. For any vector v ∈ E, there is a vector (−v) ∈ E, called the oppositeof v, such that v + (−v) = 0.

The multiplication by scalars satisfies the following conditions:

5

6 CHAPTER 2. FINITE-DIMENSIONAL VECTOR SPACES

1. a(bv) = (ab)v, ∀v ∈ E, ∀a, bR,

2. (a + b)v = av + bv, ∀v ∈ E, ∀a, bR,

3. a(u + v) = au + av, ∀u, v ∈ E, ∀aR,

4. 1 v = v ∀v ∈ E.

• The zero vector is unique.

• For any u, v ∈ E there is a unique vector denoted by w = v − u, called thedifference of v and u, such that u + w = v.

• For any v ∈ E,0v = 0 , and (−1)v = −v .

• Let E be a real vector space and A = e1, . . . , ek be a finite collection ofvectors from E. A linear combination of these vectors is a vector

a1e1 + · · · + akek ,

where a1, . . . , an are scalars.

• A finite collection of vectorsA = e1, . . . , ek is linearly independent if

a1e1 + · · · + akek = 0

implies a1 = · · · = ak = 0.

• A collection A of vectors is linearly dependent if it is not linearly inde-pendent.

• Two non-zero vectors u and v which are linearly dependent are also calledparallel, denoted by u||v.

• A collectionA of vectors is linearly independent if no vector ofA is a linearcombination of a finite number of vectors fromA.

• LetA be a subset of a vector space E. The span ofA, denoted by spanA,is the subset of E consisting of all finite linear combinations of vectors fromA, i.e.

spanA = v ∈ E | v = a1e1 + · · · + akek , ei ∈ A, ai ∈ R .

We say that the subset spanA is spanned byA.


2.1. VECTORS AND LINEAR TRANSFORMATIONS 7

• Theorem 2.1.1 The span of any subset of a vector space is a vector space.

• A vector subspace of a vector space E is a subset S ⊆ E of E which isitself a vector space.

• Theorem 2.1.2 A subset S of E is a vector subspace of E if and only ifspan S = S .

• Span ofA is the smallest subspace of E containingA.

• A collection B of vectors of a vector space E is a basis of E if B is linearlyindependent and spanB = E.

• A vector space E is finite-dimensional if it has a finite basis.

• Theorem 2.1.3 If the vector space E is finite-dimensional, then the numberof vectors in any basis is the same.

• The dimension of a finite-dimensional real vector space E, denoted bydim E, is the number of vectors in a basis.

• Theorem 2.1.4 If e1, . . . , en is a basis in E, then for every vector v ∈ Ethere is a unique set of real numbers (vi) = (v1, . . . , vn) such that

v =

n∑i=1

viei = v1e1 + · · · + vnen .

• The real numbers vi, i = 1, . . . , n, are called the components of the vectorv with respect to the basis ei.

• It is customary to denote the components of vectors by superscripts, whichshould not be confused with powers of real numbers

v2 , (v)2 = vv, . . . , vn , (v)n .

Examples of Vector Subspaces

• Zero subspace 0.

• Line with a tangent vector u:

S 1 = span u = v ∈ E | v = tu, t ∈ R .



• Plane spanned by two nonparallel vectors u1 and u2

S 2 = span u1,u2 = v ∈ E | v = tu1 + su2, t, s ∈ R .

• More generally, a k-plane spanned by a linearly independent collection ofk vectors u1, . . . ,uk

S k = span u1, . . . ,uk = v ∈ E | v = t1u1 + · · · + tkuk, t1, . . . , tk ∈ R .

• An (n − 1)-plane in an n-dimensional vector space is called a hyperplane.

• Examples of vector spaces: P[t], Pn[t], Mm×n, Ck([a, b]), C∞([a, b])

2.1.2 Inner Product and Norm• A complex vector space E is called an inner product space if there is a

function (·, ·) : E × E → R, called the inner product, that assigns to everytwo vectors u and v a complex number (u, v) and satisfies the conditions:∀u, v,w ∈ E, ∀a ∈ C:

1. (v, v) ≥ 0

2. (v, v) = 0 if and only if v = 0

3. (u, v) = (v,u)

4. (u + v,w) = (u,w) + (v,w)

5. (u, av) = a(u, v)

A finite-dimensional real inner product space is called a Euclidean space.

• Examples: On C([a, b])

( f , g) =

∫ b

af (t)g(t)w(t)dt

where w is a positive continuous real-valued function called the weightfunction.

• The Euclidean norm is a function || · || : E → R that assigns to every vectorv ∈ E a real number ||v|| defined by

||v|| =√

(v, v).



• The norm of a vector is also called the length.

• A vector with unit norm is called a unit vector.

• The natural distance function (a metric) is defined by

d(u, v) = ||u − v||

• Example.

• Theorem 2.1.5 For any u, v ∈ E there holds

||u + v||2 = ||u||2 + 2Re(u, v) + ||v||2 .

• If the norm satisfies the parallelogram law

||u + v||2 + ||u − v||2 = 2||u||2 + 2||v||2

then the inner product can be defined by

(u, v) =14

||u + v||2 − ||u − v||2 − i||u + iv||2 + i||u − iv||2

• Theorem 2.1.6 A normed linear space is an inner product space if and only

if the norm satisfies the parallelogram law.

• Theorem 2.1.7 Every finite-dimensional vector space can be turned intoan inner product space.

• Theorem 2.1.8 Cauchy-Schwarz’s Inequality. For any u, v ∈ E thereholds

|(u, v)| ≤ ||u|| ||v|| .

The equality

|(u, v)| = ||u|| ||v||

holds if and only if u and v are parallel.

• Corollary 2.1.1 Triangle Inequality. For any u, v ∈ E there holds

||u + v|| ≤ ||u|| + ||v|| .



• In real vector space the angle between two non-zero vectors u and v isdefined by

cos θ =(u, v)||u|| ||v||

, 0 ≤ θ ≤ π .

Then the inner product can be written in the form

(u, v) = ||u|| ||v|| cos θ .

• Two non-zero vectors u, v ∈ E are orthogonal, denoted by u ⊥ v, if

(u, v) = 0.

• A basis e1, . . . , en is called orthonormal if each vector of the basis is aunit vector and any two distinct vectors are orthogonal to each other, that is,

(ei, e j) =

1, if i = j0, if i , j .

• Theorem 2.1.9 Every Euclidean space has an orthonormal basis.

• Let S ⊂ E be a nonempty subset of E. We say that x ∈ E is orthogonal toS , denoted by x ⊥ S , if x is orthogonal to every vector of S .

• The setS ⊥ = x ∈ E | x ⊥ S

of all vectors orthogonal to S is called the orthogonal complement of S .

• Theorem 2.1.10 The orthogonal complement of any subset of a Euclideanspace is a vector subspace.

• Two subsets A and B of E are orthogonal, denoted by A ⊥ B, if everyvector of A is orthogonal to every vector of B.

• Let S be a subspace of E and S ⊥ be its orthogonal complement. If everyelement of E can be uniquely represented as the sum of an element of S andan element of S ⊥, then E is the direct sum of S and S ⊥, which is denotedby

E = S ⊕ S ⊥ .

• The union of a basis of S and a basis of S ⊥ gives a basis of E.



2.1.3 Exercises1. Show that if λv = 0, then either v = 0 or λ = 0.

2. Prove that the span of a collection of vectors is a vector subspace.

3. Show that the Euclidean norm has the following properties

(a) ||v|| ≥ 0, ∀v ∈ E;

(b) ||v|| = 0 if and only if v = 0;

(c) ||av|| = |a| ||v||, ∀v ∈ E,∀a ∈ R.

4. Parallelogram Law. Show that for any u, v ∈ E

||u + v||2 + ||u − v||2 = 2(||u||2 + ||v||2

)5. Show that any orthogonal system in E is linearly independent.

6. Gram-Schmidt orthonormalization process. Let G = u1, · · · ,uk be a linearlyindependent collection of vectors. Let O = v1, · · · , vk be a new collection ofvectors defined recursively by

v1 = u1,

v j = u j −

j−1∑i=1

vi(vi,u j)||vi||

2 , 2 ≤ j ≤ k,

and the collection B = e1, . . . , ek be defined by

ei =vi

||vi||.

Show that: a) O is an orthogonal system and b) B is an orthonormal system.

7. Pythagorean Theorem. Show that if u ⊥ v, then

||u + v||2 = ||u||2 + ||v||2 .

8. Let B = e1, · · · en be an orthonormal basis in E. Show that for any vector v ∈ E

v =

n∑i=1

ei(ei, v)

and

||v||2 =

n∑i=1

(ei, v)2 .



9. Prove that the orthogonal complement of a subset S of E is a vector subspace of E.

10. Let S be a subspace in E. Prove that

a) E⊥ = 0, b) 0⊥ = E, c) (S ⊥)⊥ = S .

11. Show that the intersection of orthogonal subsets of a Euclidean space is eitherempty or consists of only the zero vector. That is, for two subsets A and B, ifA ⊥ B, then A ∩ B = 0 or ∅.

2.1.4 Linear Transformations.• A linear transformation from a vector space V to a vector space W is a

mapT : V → W

satisfying the condition:

T (αu + βv) = αTu + βTv

for any u, v ∈ V and α, β ∈ C.

• Zero transformation maps all vectors to the zero vector.

• The linear transformation is called an endomorphism (or a linear operator)if V = W.

• The linear transformation is called a linear functional if W = C.

• A linear transformation is uniquely determined by its action on a basis.

• The set of linear transformations from V to W is a vector space denoted byL(V,W).

• The set of endomorphisms (operators) on V is denoted by End (V) or L(V).

• The set of linear functionals on V is called the dual space and is denotedby V∗.

• Example.

• The kernel (null space) (denoted by Ker T ) of a linear transformation T :V → W is the set of vectors in V that are mapped to zero.



• Theorem 2.1.11 The kernel of a linear transformation is a vector space.

• The dimension of a finite-dimensional kernel is called the nullity of thelinear transformation.

null T = dim Ker T

• Theorem 2.1.12 The range of a linear transformation is a vector space.

• The dimension of a finite-dimensional range is called the rank of the lineartransformation.

rank T = dim Im T

• Theorem 2.1.13 Dimension Theorem. Let T : V → W be a linear trans-formation between finite-dimensional vector spaces. Then

dim Ker T + dim Im T = dim V .

• Theorem 2.1.14 A linear transformation is injective if and only if its kernelis zero.

• An endomorphism of a finite-dimensional space is bijective if it is eitherinjective or surjective.

• Two vector spaces are isomorphic if they can be related by a bijective lineartransformation (which is called an isomorphism).

• An isomorphism is called an automorphism if V = W.

• The set of all automorphisms of V is denoted by Aut (V) or GL(V).

• A linear surjection is an isomorphism if and only if its nullity is zero.

• Theorem 2.1.15 An isomorphism maps linearly independent sets onto lin-early independent sets.

• Theorem 2.1.16 Two finite-dimensional vector spaces are isomorphic ifand only if they have the same dimension.

• All n-dimensional complex vector spaces are isomorphic to Cn.

• All n-dimensional real vector spaces are isomorphic to Rn.



• The dual basis fi in the dual space V∗ is defined by

fi(e j) = δi j,

where e j is the basis in V .

• Theorem 2.1.17 The dual space V∗ is isomorphic to V.

• The dual (or the pullback) of a linear transformation T : V → W is thelinear transformation T ∗ : W∗ → V∗ defined for any g ∈ W∗ by

(T ∗g)v = g(Tv), v ∈ V .

• Graph.

• If T is surjective then T ∗ is injective.

• If T is injective then T ∗ is surjective.

• If T is an isomorphism then T ∗ is an isomorphism.

2.1.5 Algebras• An algebra A is a vector space together with a binary operation called mul-

tiplication satisfying the conditions:

u(αv + βw) = αuv + βuw

(αv + βw)u = αvu + βwu

for any u, v,w, α, β ∈ C.

• Examples. Matrices, functions, operators.

• The dimension of the algebra is the dimension of the vector space.

• The algebra isassociative if

u(vw) = (uv)w

and commutative ifuv = vu



• An algebra with identity is an algebra with an identity element 1 satisfying

u1 = 1u = u

for any u ∈ A.

• An element v is a left inverse of u if

vu = 1

and the right inverse ifuv = 1.

• Example. Lie algebras.

• An operator D : A→ A on an algebra A is called a derivation if it satisfies

D(uv) = (Du)v + uDv

• Example. Let A = Mat(n) be the algebra of square matrices of dimensionn with the binary operation being the commutator of matrices.

• It is easy to show that for any matrices A, B,C the following identity (Jacobiidentity) holds

[A, [B,C]] + [B, [C, A]] + [C, [A, B]] = 0

• Let C be a fixed matrix. We define an operator AdC on the algebra by

AdC B = [C, B]

Then this operator is a derivation since for any matrices A, B

AdC[A, B] = [AdCA, B] + [A, AdC B]

• A linear transformation T : A → B from an algebra A to an algebra B iscalled an algebra homomorphism if

T (uv) = T (u)T (v)

for any u, v ∈ A.



• An algebra homomorphism is called an algebra isomorphism if it is bijec-tive.

• Example. The isomorphism of the Lie algebra so(3) and R3 with the crossproduct.

Let Xi, i = 1, 2, 3 be the antisymmetric matrices defined by

(Xi) jk = ε j

ik .

They form an algebra with respect to the commutator

[Xi, X j] = εki jXk .

We define a map T : R3 → so(3) as follows. Let v = viei be a vector in R3.Then

T (v) = viXi .

let R3 be equipped with the cross product. Then

T (v × u) = (Tv)(Tu)

Thus T is an isomorphism (linear bijective algebra homomorphism).

• Any finite dimensional vector space can be converted into an algebra bydefining the multiplication of the basis vectors by

eie j =

n∑k=1

Cki jek

where Cki j are some scalars called the structure constants of the algebra.

• Example. Lie algebra su(2).

Pauli matrices are defined by

σ1 =

(0 11 0

), σ2 =

(0 −ii 0

), σ3 =

(1 00 −1

). (2.1)

They are Hermitian traceless matrices satisfying

σiσ j = δi jI + iεi jkσk . (2.2)



They satisfy the following commutation relations

[σi, σ j] = 2iεi jkσk (2.3)

and the anti-commutation relations

σiσ j + σ jσi = 2δi jI (2.4)

Therefore, Pauli matrices form a representation of Clifford algebra in 2 di-mensions.

The matricesJi = −

i2σi (2.5)

are the generators of the Lie algebra su(2) with the commutation relations

[Ji, J j] = εki jJk (2.6)

Algebra homomorphism Λ : su(2) → so(3) is defined as follows. Letv = viJi ∈ su(2). Then Λ(v) is the matrix defined by

Λ(v) = viXi .

• Example. Quaternions. The algebra of quaternions H is defined by (herei, j, k = 1, 2, 3)

e20 = e0 , e2

i = −e0 , e0ei = eie0 = ei ,

eie j = εki jek i , j

There is an algebra homomorphism ρ : H→ su(2)

ρ(e0) = I, ρ(e j) = −iσ j

• A subspace of an algebra is called a subalgebra if it is closed under algebramultiplication.

• A subset B of an algebra A is called a left ideal if AB ⊂ B, that is, for anyu ∈ A and any v ∈ B, uv ∈ B.

• A subset B of an algebra A is called a right ideal if BA ⊂ B, that is, for anyu ∈ A and any v ∈ B, vu ∈ B.



• A subset B of an algebra A is called a two-sided ideal if it is both left andright ideal, that is, if ABA ⊂ B, or for any u,w ∈ A and any v ∈ B, uvw ∈ B.

• Every ideal is a subalgebra.

• A proper ideal of an algebra with identity cannot contain the identity ele-ment.

• A proper left ideal cannot contain an element that has a left inverse.

• If an ideal does not contain any proper subideals then it is the minimal ideal.

• Examples. Let x be and element of an algebra A. Let Ax be the set definedby

Ax = ux|u ∈ A

Then Ax is a left ideal.

• Similarly xA is a right ideal and AxA is a two-sided ideal.


2.2. OPERATOR ALGEBRA 19

2.2 Operator Algebra

2.2.1 Algebra of Operators on a Vector Space• A linear operator on a vector space E is a mapping L : E → E satisfying

the condition ∀u, v ∈ E, ∀a ∈ R,

L(u + v) = L(u) + L(v) and L(av) = aL(v).

• Identity operator I on E is defined by

I v = v, ∀v ∈ E

• Null operator 0 : E → E is defined by

0v = 0, ∀v ∈ E

• The vector u = L(v) is the image of the vector v.

• If S is a subset of E, then the set

L(S ) = u ∈ E | u = L(v) for some v ∈ S

is the image of the set S and the set

L−1(S ) = v ∈ E | L(v) ∈ S

is the inverse image of the set A.

• The image of the whole space E of a linear operator L is the range (or theimage) of L, denoted by

Im(L) = L(E) = u ∈ E | u = L(v) for some v ∈ E .

• The kernel Ker(L) (or the null space) of an operator L is the set of allvectors in E which are mapped to zero, that is

Ker (L) = L−1(0) = v ∈ E | L(v) = 0 .

• Theorem 2.2.1 For any operator L the sets Im(L) and Ker (L) are vectorsubspaces.



• The dimension of the kernel Ker (L) of an operator L

null (L) = dim Ker (L)

is called the nullity of the operator L.

• The dimension of the range Im(L) of an operator L

rank (L) = dim Ker (L)

is called the rank of the operator L.

• Theorem 2.2.2 For any operator L on an n-dimensional Euclidean spaceE

rank (L) + null (L) = n

• The set L(E) of all linear operators on a vector space E is a vector spacewith the addition of operators and multiplication by scalars defined by

(L1 + L2)(x) = L1(x) + L2(x), and (aL)(x) = aL(x) .

• The product of the operators A and B is the composition of A and B.

• Since the product of operators is defined as a composition of linear map-pings, it is automatically associative, which means that for any operators A,B and C, there holds

(AB)C = A(BC) .

• The integer powers of an operator are defined as the multiple compositionof the operator with itself, i.e.

A0 = I A1 = A, A2 = AA, . . .

• The operator A on E is invertible if there exists an operator A−1 on E, calledthe inverse of A, such that

A−1A = AA−1 = I .

• Theorem 2.2.3 Let A and B be invertible operators. Then:

(A−1)−1 = A , (AB)−1 = B−1A−1 .



• The operators A and B are commuting if

AB = BA

and anti-commuting ifAB = −BA .

• The operators A and B are said to be orthogonal to each other if

AB = BA = 0 .

• An operator A is involutive if

A2 = I

idempotent ifA2 = A ,

and nilpotent if for some integer k

Ak = 0 .

• Two operators A and B are equal if for any u ∈ V

Au = Bu

• If Aei = Bei for all basis vectors in V then A = B.

• Operators are uniquely determined by their action on a basis.

• Theorem 2.2.4 An operator A is equal to zero if and only if for any u, v ∈ V

(u, Av) = 0

• Theorem 2.2.5 An operator A is equal to zero if and only if for any u

(u, Au) = 0

Proof: Use (w, Aw) = 0 for w = au + bv with a = 1, b = i and a = i, b = 1.

• Theorem 2.2.6 1. The inverse of an automorphism is unique.



2. The product of two automorphisms is an automorphism.

3. A linear transformation is an automorphism if and only if it maps abasis to another basis.

• Polynomials of Operators.

Pn(T ) = anT n + · · · a1T + a0I,

where I is the identity operator.

• Commutator of two operators A and B is an operator [A, B] defined by

[A, B] = AB − BA

• Theorem 2.2.7 Properties of commutators.

Anti-symmetry[A, B] = −[B, A]

linearity[aA, bB] = ab[A, B]

[A, B + C] = [A, B] + [A,C]

[A + C, B] = [A, B] + [C, B]

right derivation[AB,C] = A[B,C] + [A,C]B

left derivation[A, BC] = [A, B]C + B[A,C]

Jacobi identity

[A, [B,C]] + [B, [C, A]] + [C, [A, B]] = 0

• Consequences[A, Am] = 0

[A, A−1] = 0

[A, f (A)] = 0



• Functions of Operators.

Negative powersT m = T · · · T︸︷︷︸

m

T−m = (T−1)m

T mT n = T m+n

(T m)n = T mn

Let f be an analytic function given by

f (x) =

∞∑k=0

f (k)(x0)k!

(x − x0)k

Then for an operator T

f (T ) =

∞∑k=0

f (k)(x0)k!

(T − x0I)k

• Exponential

exp(T ) =

∞∑k=0

1k!

T k

• Example.

2.2.2 Derivatives of Functions of Operators• A time-dependent operator is a map

H : R→ End (V)

Note that[H(t),H(t′)] , 0

• Example.A : R2 → R2

A(x, y) = (−y, x)



Note thatA2 = −I

soA2n = (−1)nI, A2n+1 = (−1)nA

Thereforeexp(tA) = cos tI + sin tA

which is a rotation by the angle t

exp(tA)(x, y) = (cos t x − sin t y, cos t y + sin t x)

So A is a generator of rotation.

• Derivative of a time-dependent operator is an operator defined by

dHdt

= limh→0

H(t + h) − H(t)h

• Rules of the differentiation

ddt

(AB) =dAdt

B + AdBdt

• Example.ddt

exp(tA) = A exp(tA)

• Exponential of the adjoint. Let

X(t) = etABe−tA

It satisfies the equationddt

X = [A, B]

with initial conditionX(0) = I

Let AdA be defined byAdAB = [A, B]

Then

X(t) = exp(tAdA)B =

∞∑k=0

tk

k![A, · · · [A, B]︸︷︷︸

k



• Duhamel’s Formula.

ddt

exp[H(t)] = exp[H(t)]∫ 1

0exp[−sH(t)]

dH(t)dt

exp[sH(t)]ds

Proof. Let

Y(s) = e−sH ddt

esH

ThenY(0) = 0

andddt

exp[H] = exp[H]Y(1)

We computedYds

= −[H,Y] + H′

So,(∂s + AdH)Y = H′

Therefore

Y(1) = exp(−AdH)∫ 1

0exp(sAdH)H′ds

which can be written in the form

Y(1) =

∫ 1

0e−(1−s)HH′e(1−s)Hds

By changing the variable s→ (1 − s) we get the desired result.

• Particular case. If H commutes with H′ then

∂teH(t) = eH∂tH

• Campbell-Hausdorff Formula.

exp A exp B = exp[C(A, B)]

ConsiderU(s) = eAesB = eC(s)



Of courseC(0) = A

We havedds

U = UB

Also,

∂sU = U∫ 1

0exp[−τC]∂sC exp[τC] dτ

Let

F(z) =

∫ 1

0e−τzdτ =

1 − e−z

zThen

∂sU = UF(AdC)∂sC

ThereforeF(AdC)∂sC = B

Now, let

Ψ(z) =1

F(log z)=

z log zz − 1

Note thateAdC = AdeC = AdeA · AdesB = eAdAesAdB

ThenΨ(eAdAesAdB)F(AdC) = I

Therefore, we get a differential equation

∂sC = Ψ(eAdAesAdB)B

with initial conditionC(0) = A

Therefore,

C(1) = A +

∫ 1

0Ψ(eAdAesAdB)Bds

This gives a power series in AdA, AdB.

• Particular case. If [A, B] commutes with both A and B then

eAeB = eA+B exp(12

[A, B])



2.2.3 Self-Adjoint and Unitary Operators• The adjoint A∗ of an operator A is defined by

(Au, v) = (u,A∗v), ∀u, v ∈ E.

• Theorem 2.2.8 For any two operators A and B

(A∗)∗ = A , (AB)∗ = B∗A∗ .

(A + B)∗ = A∗ + B∗

(aA)∗ = aA∗

• An operator A is self-adjoint (or Hermitian) if

A∗ = A

and anti-selfadjoint if

A∗ = −A

• Every operator A can be decomposed as the sum

A = AS + AA

of its selfadjoint part AS and its anti-selfadjoint part AA

AS =12

(A + A∗) , AA =12

(A − A∗) .

• Theorem 2.2.9 An operator H is Hermitian if and only if (u,Hu) is realfor any u.

• An operator A on E is called positive, denoted by A ≥ 0, if it is selfdadjointand ∀v ∈ E

(Av, v) ≥ 0.

• An operator H is positive definite (H > 0) if it is positive and

(u,Hu) = 0

only for u = 0.



• Example.H = A∗A ≥ 0

• An operator A is called unitary if

AA∗ = A∗A = I .

• An operator U is isometric if for any v ∈ E

||Uv|| = ||v||

• Example.U = exp(A), A∗ = −A

• Unitary operators preserve the inner product.

• Theorem 2.2.10 Let U be a unitary operator on a real vector space E.Then there exists an anti-selfadjoint operator A such that

U = exp A .

• Recall that the operators U and A satisfy the equations

U∗ = U−1 and A∗ = −A.

2.2.4 Trace and Determinant

• The trace of an operator A is defined by

tr A =

n∑i=1

(ei, Aei)

• The determinant of a positive operator on a finite-dimensional space isdefined by

det A = exp(tr log A)



• Propertiestr AB = tr BA

det AB = det A det B

tr (RAR−1) = tr A

det(RAR−1) = det A

• Theorem.ddt

det(I + tA)∣∣∣∣t=0

= tr A

det(I + tA) = I + ttr A + O(t2)

ddt

det A = det A tr(A−1 dA

dt

)• Note that

tr I = n , det I = 1 .

• Theorem 2.2.11 Let A be a self-adjoint operator. Then

det exp A = etr A .

• Let A be a positive definite operator, A > 0. The zeta-function of theoperator A is defined by

ζ(s) = tr A−s =1

Γ(s)

∫ ∞

0dt ts−1tr e−tA .

• Theorem 2.2.12 The zeta-functions has the properties

ζ(0) = n ,

andζ′(0) = − log det A .



2.2.5 Finite Difference Operators

• Let ei be an orthonormal basis. The shift operator E is defined by

Ee1 = 0, Eei = ei−1, i = 1, . . . , n,

that is,

E f =

n−1∑i=1

fi+1ei

or

(E f )i = fi+1

Let

∆ = E − I

∇ = I − E−1

Next, define an operator D by

E = exp(hD)

that is,

D =1h

log E =1h

log(I + ∆) = −1h

log(I − ∇)

Also, define an operator J by

J = ∆D−1

Then

∆ fi = fi+1 − fi

∇ fi = fi − fi−1

• Problem. Compute U(t) = exp[tD2].



2.2.6 Projection Operators• A Hermitian operator P is a projection if

P2 = P

• Two projections P1, P2 are orthogonal if

P1P2 = P2P1 = 0 .

• Let S be a subspace of E and E = S ⊕ S ⊥. Then for any u ∈ E there existunique v ∈ S and w ∈ S ⊥ such that

u = v + w .

The vector v is called the projection of u onto S .

• The operator P on E defined by

Pu = v

is called the projection operator onto S .

• The operator P⊥ defined byP⊥u = w

is the projection operator onto S ⊥.

• The operators P and P⊥ are called complementary projections. They havethe properties:

P∗ = P, (P⊥)∗ = P⊥ ,

P + P⊥ = I ,

P2 = P , (P⊥)2 = P⊥ ,

PP⊥ = P⊥P = 0 .

• More generally, a collection of projections P1, . . . ,Pk is a complete or-thogonal system of complimentary projections if

PiPk = 0 if i , k

andk∑

i=1

Pi = P1 + · · · + Pk = I .



• The trace of a projection P onto a vector subspace S is equal to its rank, orthe dimension of the vector subspace S ,

tr P = rank P = dim S .

• Theorem 2.2.13 An operator P is a projection if and only if P is idempotentand self-adjoint.

• Theorem 2.2.14 The sum of projections is a projection if and only if theyare orthogonal.

• The projection onto a unit vector |e〉 has the form

P = |e〉〈e|

• Let |ei〉mi=1 be an orthonormal set and S = span 1≤i≤m|ei〉. Then the opera-

tor

P =

m∑i=1

|ei〉〈ei|

is the projection onto S .

• If |ei〉 is an orthonormal basis then the projections

Pi = |ei〉〈ei|

for a complete orthogonal set.

Examples

• Let u be a unit vector and Pu be the projection onto the one-dimensionalsubspace (line) S u spanned by u defined by

Puv = u(u, v) .

The orthogonal complement S ⊥u is the hyperplane with the normal u. Theoperator Ju defined by

Ju = I − 2Pu

is called the reflection operator with respect to the hyperplane S ⊥u . Thereflection operator is a self-adjoint involution, that is, it has the followingproperties

J∗u = Ju , J2u = I .



The reflection operator has the eigenvalue −1 with multiplicity 1 and theeigenspace S u, and the eigenvalue +1 with multiplicity (n − 1) and witheigenspace S ⊥u .

• Let u1 and u2 be an orthonormal system of two vectors and Pu1,u2 be theprojection operator onto the two-dimensional space (plane) S u1,u2 spannedby u1 and u2

Pu1,u2v = u1(u1, v) + u2(u2, v) .

Let Nu1,u2 be an operator defined by

Nu1,u2v = u1(u2, v) − u2(u1, v) .

ThenNu1,u2Pu1,u2 = Pu1,u2Nu1,u2 = Nu1,u2

andN2

u1,u2= −Pu1,u2 .

A rotation operator Ru1,u2(θ) with the angle θ in the plane S u1,u2 is definedby

Ru1,u2(θ) = I − Pu1,u2 + cos θ Pu1,u2 + sin θ Nu1,u2 .

The rotation operator is unitary, that is, it satisfies the equation

R∗u1,u2Ru1,u2 = I .

2.2.7 Exercises1. Prove that the range and the kernel of any operator are vector spaces.

2. Show that

(aA + bB)∗ = aA∗ + bB∗ ∀a, b ∈ R ,

(A∗)∗ = A

(AB)∗ = B∗A∗

3. Show that for any operator A the operators AA∗ and A + A∗ are selfadjoint.

4. Show that the product of two selfadjoint operators is selfadjoint if and only if theycommute.

5. Show that a polynomial p(A) of a selfadjoint operator A is a selfadjoint operator.



6. Prove that the inverse of an invertible operator is unique.

7. Prove that an operator A is invertible if and only if Ker A = 0, that is, Av = 0implies v = 0.

8. Prove that for an invertible operator A, Im(A) = E, that is, for any vector v ∈ Ethere is a vector u ∈ E such that v = Au.

9. Show that if an operator A is invertible, then

(A−1)−1 = A .

10. Show that the product AB of two invertible operators A and B is invertible and

(AB)−1 = B−1A−1

11. Prove that the adjoint A∗ of any invertible operator A is invertible and

(A∗)−1 = (A−1)∗ .

12. Prove that the inverse A−1 of a selfadjoint invertible operator is selfadjoint.

13. An operator A on E is called isometric if ∀v ∈ E,

||Av|| = ||v|| .

Prove that an operator is unitary if and only if it is isometric.

14. Prove that unitary operators preserves inner product. That is, show that if A is aunitary operator, then ∀u, v ∈ E

(Au,Av) = (u, v) .

15. Show that for every unitary operator A both A−1 and A∗ are unitary.

16. Show that for any operator A the operators AA∗ and A∗A are positive.

17. What subspaces do the null operator 0 and the identity operator I project onto?

18. Show that for any two projection operators P and Q, PQ = 0 if and only if QP = 0.

19. Prove the following properties of orthogonal projections

P∗ = P, (P⊥)∗ = P⊥, P⊥ + P = I, PP⊥ = P⊥P = 0 .



20. Prove that an operator is projection if and only if it is idempotent and selfadjoint.

21. Give an example of an idempotent operator in R2 which is not a projection.

22. Show that any projection operator P is positive. Moreover, show that ∀v ∈ E

(Pv, v) = ||Pv||2 .

23. Prove that the sum P = P1+P2 of two projections P1 and P2 is a projection operatorif and only if P1 and P2 are orthogonal.

24. Prove that the product P = P1P2 of two projections P1 and P2 is a projectionoperator if and only if P1 and P2 commute.



2.3 Matrix Representation of Operators

2.3.1 Matrices• Cn is the set of all ordered n-tuples of complex numbers, which can be

assembled as columns or as rows.

• Let v be a vector in a n-dimensional vector space V with a basis ei. Then

v =

n∑i=1

viei

where v1, . . . , vn are complex numbers called the components of the vectorv. The column-vector is an ordered n-tuple of the form

v1

v2...

vn

.We say that the column vector represents the vector v in the basis ei.

• Let < v| = (|v >)∗ be a linear functional dual to the vector |v >. Let < ei| bethe dual basis. Then

< v| =n∑

i=1

vi < ei|

The row-vector (also called a covector) is an ordered n-tuple of the form

(v1, v2, . . . , vn) .

It represents the dual vector < v| in the same basis.

• A set of nm complex numbers Ai j, i = 1, . . . , n; j = 1, . . . ,m, arranged in anarray that has m columns and n rows

A =

A11 A12 · · · A1m

A21 A22 · · · A2m...

.... . .

...An1 An2 · · · Anm

is called a rectangular n × m complex matrix.


2.3. MATRIX REPRESENTATION OF OPERATORS 37

• The set of all complex n × m matrices is denoted by Mat(n,m;C).

• The number Ai j (also called an entry of the matrix) appears in the i-th rowand the j-th column of the matrix A

A =

A11 A12 · · · A1 j · · · A1m

A21 A22 · · · A2 j · · · A2m...

.... . .

......

...

Ai1 Ai2 · · · Ai j · · · Aim

......

......

. . ....

An1 An2 · · · An j · · · Anm

• Remark. Notice that the first index indicates the row and the second index

indicates the column of the matrix.

• The matrix whose all entries are equal to zero is called the zero matrix.

• Finally, we define the multiplication of column-vectors by matrices fromthe left and the multiplication of row-vectors by matrices from the right asfollows.

• Each matrix defines a natural left action on a column-vector and a rightaction on a row-vector.

• For each column-vector v and a matrix A = (Ai j) the column-vector u = Avis given by

u1

u2...ui...

un

=

A11 A12 · · · A1n

A21 A22 · · · A2n...

.... . .

...Ai1 Ai2 · · · Ain...

......

...An1 An2 · · · Ann

v1

v2...vi...

vn

=

A11v1 + A12v2 + · · · + A1nvn

A21v1 + A22v2 + · · · + A2nvn...

Ai1v1 + Ai2v2 + · · · + Ainvn...

An1v1 + An2v2 + · · · + Annvn

• The components of the vector u are

ui =

n∑j=1

Ai jv j = Ai1v1 + Ai2v2 + · · · + Ainvn .



• Similarly, for a row vector vT the components of the row-vector uT = vT Aare defined by

ui =

n∑j=1

v jA ji = v1A1i + v2A2i + · · · + vnAni .

• Let W be an m-dimensional vector space with aa basis fi and A : V → W bea linear transformation. Such an operator defines a n × m matrix (Ai j) by

Aei =

m∑j=1

A jif j

orA ji = (f j, Aei)

Thus the linear transformation A is represented by the matrix Ai j.

The components of a vector v are obtained by acting on the colum vector(vi) from the left by the matrix (A ji), that is,

(Av)i =

n∑j=1

Ai jv j

• Proposition. The vector space L(V,W) of linear transformations A : V →W is isomorphic to the space M(m × n,C) of m × n matrices.

• Proposition. The rank of a linear transformation is equal to the rank of itsmatrix.

2.3.2 Operation on Matrices

• The addition of matrices is defined by

A + B =

A11 + B11 A12 + B12 · · · A1m + B1m

A21 + B21 A22 + B22 · · · A2m + B2m...

.... . .

...An1 + Bn1 An2 + Bn2 · · · Anm + Bnm



and the multiplication by scalars by

cA =

cA11 cA12 · · · cA1m

cA21 cA22 · · · cA2m...

.... . .

...cAn1 cAn2 · · · cAnm

• A n × m matrix is called a square matrix if n = m.

• The numbers Aii are called the diagonal entries. Of course, there are ndiagonal entries. The set of diagonal entries is called the diagonal of thematrix A.

• The numbers Ai j with i , j are called off-diagonal entries; there are n(n−1)off-diagonal entries.

• The numbers Ai j with i < j are called the upper triangular entries. Theset of upper triangular entries is called the upper triangular part of thematrix A.

• The numbers Ai j with i > j are called the lower triangular entries. The setof lower triangular entries is called the lower triangular part of the matrixA.

• The number of upper-triangular entries and the lower-triangular entries isthe same and is equal to n(n − 1)/2.

• A matrix whose only non-zero entries are on the diagonal is called a diago-nal matrix. For a diagonal matrix

Ai j = 0 if i , j .

• The diagonal matrix

A =

λ1 0 · · · 00 λ2 · · · 0...

.... . .

...0 0 · · · λn

is also denoted by

A = diag (λ1, λ2, . . . , λn)



• A diagonal matrix whose all diagonal entries are equal to 1

I =

1 0 · · · 00 1 · · · 0...

.... . .

...0 0 · · · 1

is called the identity matrix. The elements of the identity matrix are

δi j =

1, if i = j

0, if i , j .

• A matrix A of the form

A =

∗ ∗ · · · ∗

0 ∗ · · · ∗...

.... . .

...0 0 · · · ∗

where ∗ represents nonzero entries is called an upper triangular matrix.Its lower triangular part is zero, that is,

Ai j = 0 if i < j .

• A matrix A of the form

A =

∗ 0 · · · 0∗ ∗ · · · 0...

.... . .

...∗ ∗ · · · ∗

whose upper triangular part is zero, that is,

Ai j = 0 if i > j ,

is called a lower triangular matrix.



• The transpose of a matrix A whose i j-th entry is Ai j is the matrix AT whosei j-th entry is A ji. That is, AT obtained from A by switching the roles of rowsand columns of A:

AT =

A11 A21 · · · A j1 · · · An1

A12 A22 · · · A j2 · · · An2...

.... . .

......

...

A1i A2i · · · A ji · · · Ani

......

......

. . ....

A1m A2m · · · A jm · · · Anm

or

(AT )i j = A ji .

• The Hermitian conjugate of a matrix A = (Ai j) is a matrix A∗ = (A∗i j)defined by

(A∗)i j = A ji

• A matrix A is called symmetric if

AT = A

and anti-symmetric ifAT = −A .

• A matrix A is called Hermitian if

A∗ = A

and anti-Hermitian ifA∗ = −A .

• An anti-Hermitian matrix has the form

A = iH

where H is Hermitian.

• A Hermitian matrix has the form

H = A + iB

where A is real symmetric and B is real anti-symmetric matrix.



• The number of independent entries of an anti-symmetric matrix is n(n−1)/2.

• The number of independent entries of a symmetric matrix is n(n + 1)/2.

• The diagonal entries of a Hermitian matrix are real.

• The number of independent real parameters of a Hermitian matrix is n2.

• Every square matrix A can be uniquely decomposed as the sum of its di-agonal part AD, the lower triangular part AL and the upper triangular partAU

A = AD + AL + AU .

• For an anti-symmetric matrix

ATU = −AL and AD = 0 .

• For a symmetric matrixAT

U = AL .

• Every square matrix A can be uniquely decomposed as the sum of its sym-metric part AS and its anti-symmetric part AA

A = AS + AA ,

whereAS =

12

(A + AT ) , AA =12

(A − AT ) .

• The product of square matrices is defined as follows. The i j-th entry ofthe product C = AB of two matrices A and B is

Ci j =

n∑k=1

AikBk j = Ai1B1 j + Ai2B2 j + · · · + AinBn j .

This is again a multiplication of the “i-th row of the matrix A by the j-thcolumn of the matrix B”.

• Theorem 2.3.1 The product of matrices is associative, that is, for any ma-trices A, B, C

(AB)C = A(BC) .

• Theorem 2.3.2 For any two matrices A and B

(AB)T = BT AT , (AB)∗ = B∗A∗ .



2.3.3 Inverse Matrix• A matrix A is called invertible if there is another matrix A−1 such that

AA−1 = A−1A = I .

The matrix A−1 is called the inverse of A.

• Theorem 2.3.3 For any two invertible matrices A and B

(AB)−1 = B−1A−1 ,

and(A−1)T = (AT )−1 .

• A matrix A is called orthogonal if

AT A = AAT = I ,

which means AT = A−1.

•

• A matrix A is called unitary if

A∗A = AA∗ = I ,

which means A∗ = A−1.

• Every unitary matrix has the form

U = exp(iH)

where H is Hermitian.

• A similarity transformation of a matric A is a map

A 7→ UAU−1

where U is a given invertible matrix.

• The similarity transformation of a function of a matrix is equal to the func-tion of the similar matrix

U f (A)U−1 = f (UAU−1) .



2.3.4 Trace

• The trace is a map tr : Mat(n,C) that assigns to each matrix A = (Ai j) acomplex number tr A equal to the sum of the diagonal elements of a matrix

tr A =

n∑k=1

Akk .

• Theorem 2.3.4 The trace has the properties

tr (AB) = tr (BA) ,

andtr AT = tr A , tr A∗ = tr A

• Obviously, the trace of an anti-symmetric matrix is equal to zero.

• The trace is invariant under a similarity transformation.

• A natural inner product on the space of matrices is defined by

(A, B) = tr (A∗B)

2.3.5 Determinant

• Consider the set Zn = 1, 2, . . . , n of the first n integers. A permutation ϕof the set 1, 2, . . . , n is an ordered n-tuple (ϕ(1), . . . , ϕ(n)) of these num-bers.

• That is, a permutation is a bijective (one-to-one and onto) function

ϕ : Zn → Zn

that assigns to each number i from the set Zn = 1, . . . , n another numberϕ(i) from this set.

• An elementary permutation is a permutation that exchanges the order ofonly two numbers.



• Every permutation can be realized as a product (or a composition) of ele-mentary permutations. A permutation that can be realized by an even num-ber of elementary permutations is called an even permutation. A permu-tation that can be realized by an odd number of elementary permutations iscalled an odd permutation.

• Proposition 2.3.1 The parity of a permutation does not depend on therepresentation of a permutation by a product of the elementary ones.

• That is, each representation of an even permutation has even number ofelementary permutations, and similarly for odd permutations.

• The sign of a permutation ϕ, denoted by sign(ϕ) (or simply (−1)ϕ), isdefined by

sign(ϕ) = (−1)ϕ =

+1, if ϕ is even,−1, if ϕ is odd

• The set of all permutations of n numbers is denoted by S n.

• Theorem 2.3.5 The cardinality of this set, that is, the number of differentpermutations, is

|S n| = n! .

• The determinant is a map det : Mat(n,C) → C that assigns to each matrixA = (Ai j) a complex number det A defined by

det A =∑ϕ∈S n

sign (ϕ)A1ϕ(1) · · · Anϕ(n) ,

where the summation goes over all n! permutations.

• The most important properties of the determinant are listed below:

Theorem 2.3.6 1. The determinant of the product of matrices is equal tothe product of the determinants:

det(AB) = det A det B .

2. The determinants of a matrix A and of its transpose AT are equal:

det AT = det A .



3. The determinant of the conjugate matrix is

det A∗ = det A .

4. The determinant of the inverse A−1 of an invertible matrix A is equalto the inverse of the determinant of A:

det A−1 = (det A)−1

5. A matrix is invertible if and only if its determinant is non-zero.

• The determinant is invariant under the similarity transformation.

• The set of complex invertible matrices (with non-zero determinant) is de-noted by GL(n,C).

• A matrix with unit determinant is called unimodular.

• The set of complex matrices with unit determinant is denoted by S L(n,C).

• The set of complex unitary matrices is denoted by U(n).

• The set of complex unitary matrices with unit determinant is denoted byS U(n).

• The set of real orthogonal matrices is denoted by O(n).

• An orthogonal matrix with unit determinant (a unimodular orthogonal ma-trix) is called a proper orthogonal matrix or just a rotation.

• The set of real orthogonal matrices with unit determinant is denoted byS O(n).

• Theorem 2.3.7 The determinant of an orthogonal matrix is equal to either1 or −1.

• Theorem. The determinant of a unitary matrix is a complex number ofmodulus 1.

• A set G of invertible matrices forms a group if it is closed under takinginverse and matrix multiplication, that is, if the inverse A−1 of any matrix Ain G belongs to the set G and the product AB of any two matrices A and Bin G belongs to G.



2.3.6 Exercises1. Show that

tr [A, B] = 0

2. Show that the product of invertible matrices is an invertible matrix.

3. Show that the product of matrices with positive determinant is a matrix with posi-tive determinant.

4. Show that the inverse of a matrix with positive determinant is a matrix with positivedeterminant.

5. Show that GL(n,R) forms a group (called the general linear group).

6. Show that GL+(n,R) is a group (called the proper general linear group).

7. Show that the inverse of a matrix with negative determinant is a matrix with nega-tive determinant.

8. Show that: a) the product of an even number of matrices with negative determinantis a matrix with positive determinant, b) the product of odd matrices with negativedeterminant is a matrix with negative determinant.

9. Show that the product of matrices with unit determinant is a matrix with unit deter-minant.

10. Show that the inverse of a matrix with unit determinant is a matrix with unit deter-minant.

11. Show that S L(n,R) forms a group (called the special linear group or the unimod-ular group).

12. Show that the product of orthogonal matrices is an orthogonal matrix.

13. Show that the inverse of an orthogonal matrix is an orthogonal matrix.

14. Show that O(n) forms a group (called the orthogonal group).

15. Show that orthogonal matrices have determinant equal to either +1 or −1.

16. Show that the product of orthogonal matrices with unit determinant is an orthogonalmatrix with unit determinant.

17. Show that the inverse of an orthogonal matrix with unit determinant is an orthogo-nal matrix with unit determinant.



18. Show that S O(n) forms a group (called the proper orthogonal group or the rota-tion group).


2.4. SPECTRAL DECOMPOSITION 49

2.4 Spectral Decomposition

2.4.1 Direct Sums• Let U and W be vector subspaces of a vector space V . Then the sum of the

vector spaces U and W is the space of all sums of the vectors from U andW, that is,

U + W = v ∈ V | v = u + w, u ∈ U,w ∈ W

• If the only vector common to both U and W is the zero vector then the sumU + W is called the direct sum and denoted by U ⊕W.

• Proposition. Let U and W be subspaces of a vector space V . Then V =

U ⊕W if and only if every vector v ∈ V in V can be written uniquely as thesum

v = u + w

with u ∈ U,w ∈ W.

• Proposition. Let V = U ⊕W. Then

dim V = dim U + dim W

• The direct sum can be naturally generalized for several subspaces so that

V =

r⊕i=1

Ui

• To such a decomposition one naturally associates orthogonal complemen-tary projections Pi on each subspace Ui.

• A complete orthogonal system of projections defines the orthogonal decom-position of the vector space

E = E1 ⊕ · · · ⊕ Ek ,

where Ei is the subspace the projection Pi projects onto.

• Theorem 2.4.1 1. The dimension of the subspaces Ei are equal to theranks of the projections Pi

dim Ei = rank Pi .



2. The sum of dimensions of the vector subspaces Ei equals the dimensionof the vector space E

n∑i=1

dim Ei = dim E1 + · · · + dim Ek = dim E .

• Let M be a subspace of a vector space V . Then the orthogonal complementof M is the vector space M⊥ of all vector in V orthogonal to all vectors inM

M⊥ = v ∈ V | (v,u) = 0 ∀u ∈ M

• Proposition. Every vector subspace M of V defines the orthogonal decom-position

V = M ⊕ M⊥.

2.4.2 Invariant Subspaces• A subspace M of a vector space V is an invariant subspace of an operator A

if it is closed under the action of this operator, that is, A(M) ⊂ M.

• A vector subspace M reduces the operator A if both M and its orthogonalcomplement M⊥ are invariant subspaces of A.

• Example. Let v be a vector in V and

M = span v,Av, . . . ,Anv

where n = dim V . Then M is an invariant space of A.

• If a subspace M reduces an operator A then we write

A = A1 ⊕ A2

where A1 acts on M and A2 acts on M⊥.

• In a natural basis, the matrix representation of the operator A has a block-diagonal form

A =

(A1 00 A2

)



• An operator whose matrix can be brought to this form by choosing a basisis called reducible; otherwise, it is irreducible.

• Theorem. Let M be a subspace of a vector space V . Then M reduces anoperator A if and only if M is invariant under both A and A∗.

2.4.3 Eigenvalues and Eigenvectors• Let A be an operator on a vector space V . A scalar λ is an eigenvalue of A

if there is a nonzero vector v in V such that

Av = λv, or (A − λI)v = 0

Such a vector is called an eigenvector corresponding to the eigenvalue λ.

• The eigenspace of A corresponding to the eigenvalue λ is the vector space

Mλ = Ker (A − λI)

• The eigenspace Mλ is the span of all eigenvectors corresponding to theeigenvalue λ.

• The dimension of the eigenspace of the eigenvalue λ is called the multiplic-ity (also called the geometric multiplicity) of λ.

• An eigenvalue of multiplicity 1 is called simple (or non-degenerate).

• An eigenvalue of multiplicity greater than 1 is called multiple (or degener-ate).

• The set Spec (A) of all eigenvalues of an operator is called the spectrum ofthe operator.

• The characteristic polynomial of A is defined by

χ(λ) = det(A − λI)

• The eigenvalues of A are the roots of its characteristic polynomial

χ(λ) = 0



• If there are p distinct roots then

χ(λ) = (λ1 − λ)m1 · · · (λp − λ)mp

• Here m j are called the algebraic multiplicity of λ j.

• Example. Let A : R2 → R2 be defined by

A(x, y) = (x + y, y)

Then it has one eigenvalue λ = 1 with geometric multiplicity 1 and algebraicmultiplicity 2. The eigenvectors are of the form (a, 0).

2.4.4 Spectral Decomposition

• An operator A on a vector space V is diagonalizable if there is a basis in Vconsisting of eigenvectors of A.

• In such basis the operator A is represented by a diagonal matrix.

• An operator is normal if it commutes with its adjoint.

• Both Hermitian and unitary operators are normal.

• An operator A is normal if and only if for any v ∈ V

||Av|| = ||A∗v||

• The eigenvalues of a Hermitian operator are real.

• The eigenvalues of a unitary operator are complex numbers of unit modulus.

• Eigenspaces of a normal operator are mutually orthogonal.

• The projections to the eigenspaces of a normal operator are Hermitian.

• Theorem. Let A be a normal operator on a vector space V . Let λ j, j =

1, . . . , p, be the distinct eigenvalues of A, M j be the corresponding eigenspacesand P j be the projections on M j. Then:



1.

V =

p⊕j=1

M j,

2.p∑

j=1

dim M j = dim V ,

3. the projections are Hermitian, orthogonal and completep∑

j=1

P j = I

PiP j = 0 if i , j,P∗i = Pi

4. there is the spectral decomposition of the operator

A =

p⊕j=1

A j =

p∑j=1

λ jP j

• In other words, for any

v =

n∑i=1

ei(ei, v) ,

we have

Av =

n∑i=1

λiei(ei, v) .

• Proposition. Every normal operator is diagonalizable.

• A Hermitian operator is positive if and only if all of its eigenvalues arepositive.

• A Hermitian operator is diagonalizable by a unitary operator.

• Let A be a self-adjoint operator with distinct eigenvalues λ1, . . . , λp withmultiplicities m j. Then the trace of the operator and the determinant ofthe operator A are defined by

tr A =

p∑i=1

m jλi ,



det A =

p∏j=1

λm j

j .

• The trace of a projection P onto a vector subspace S is equal to its rank, orthe dimension of the vector subspace S ,

tr P = rank P = dim S .

• Two operators are said to be simultaneously diagonalizable if they can berepresented in terms of the same projection operators.

• Theorem. Two operators are simultaneously diagonalizable if and only ifthey commute.

2.4.5 Functions of Operators• Let A be a normal operator on a vector space V given by its spectral decom-

position

A =

p∑i=1

λiPi ,

where Pi are the one-dimensional projections.

• Let f : C→ C be a complex function analytic at 0.

• Then one can define the function of the operator A by

f (A) =

p∑i=1

f (λi)Pi .

• The exponential of A is defined by

exp A =

∞∑k=1

1k!

Ak =

p∑i=1

eλiPi

• The trace of a function of a self-adjoint operator A is then

tr f (A) =

p∑i=1

m j f (λi) .

where m j is the multiplicity of the eigenvalue.



• Let A be a positive definite operator, A > 0. The zeta-function of theoperator A is defined by

ζ(s) = tr A−s =

p∑i=1

m j1λs

i.

• Every unitary operator U is an exponential of an anti-Hermitian operator

U = eiH =

p∑j=1

eiλ j P j

• The positive square root of a positive operator A is defined by

√A =

p∑j=1

√λ jP j

• For a given operator A let p j(z) be polynomials of the form

p j(z) =

p∏k, j;k=1

z − λk

λ j − λk

• The polynomial p j(z) has (p − 1) roots λk, k , j, that is,

p j(λk) = 0

• Moreover, the polynomial p j(z) satisfies the equation

p j(λ j) = 1.

• We have

p j(A) =

p∑k=1

p j(λk)Pk = P j

• Proposition. The projections P j to eigenspaces of a normal operator A havethe form

P j = p j(A) =

p∏k, j;k=1

A − λkIλ j − λk



• Dimension-independent definition of the determinant of a positive oper-ator:

1.det A = exp tr log A

2.det A = exp[−ζ′(0)]

3.

det A−1/2 =

∫Rn

∏j

dx j

π1/2 exp[−(x, Ax)]

2.4.6 Polar Decomposition• Theorem. Every invertible operator A can be written in a unique way as a

productA = UR

of a unitary operator U and a positive operator R.

• Proof. LetR =

√A∗A

andU = AR−1

2.4.7 Real Vector Spaces• Theorem. Let A be a symmetric operator on a real vector space V . Letλ j, j = 1, . . . , p, be the distinct eigenvalues of A, M j be the correspondingeigenspaces and P j be the projections on M j. Then:

1.

V =

p⊕j=1

M j,

2.p∑

j=1

dim M j = dim V ,



3. the projections are symmetric, orthogonal and complete

p∑j=1

P j = I

PiP j = 0 if i , j,PT

i = Pi

4. there is the spectral decomposition of the operator

A =

p⊕j=1

A j =

p∑j=1

λ jP j

• The only eigenvalues of an orthogonal operator (on a real vector space) are+1 and −1.

• Every orthogonal operator is an exponential of an anti-symmetric operator.

• Every anti-symmetric operator can have either zero or purely imaginaryeigenvalues.

• Purely imaginary eigenvalues appear in complex conjugated pairs, iθ j,−iθ j.

• The (complex) diagonal form of an anti-symmetric operator A is

diag (0, . . . , 0, iθ1,−iθ1, . . . , iθk,−iθk)

• The real block diagonal form of an anti-symmetric operator A is

diag (0, . . . , 0, θ1ε, . . . , θkε)

where

ε =

(0 1−1 0

)• Note that

ε2 = −I

andε2n = (−1)nI, ε2n+1 = (−1)nε



• Thereforeeθε = I cos θ + ε sin θ

• Theorem 2.4.2 Spectral Decomposition of Unitary Operators on RealVector Spaces. Let U be a unitary operator on a real vector space E. Thenthe only eigenvalues of U are +1 and −1 (possibly multiple) and there existsan orthogonal decomposition

E = E+ ⊕ E− ⊕ V1 ⊕ · · · ⊕ Vk ,

where E+ and E− are the eigenspaces corresponding to the eigenvalues 1and −1, and V1, . . . ,Vk are two-dimensional subspaces such that

dim E = dim E+ + dim E− + 2k .

Let P+, P−, P1, . . . ,Pk be the corresponding orthogonal complimentary sys-tem of projections, that is,

P+ + P− +

k∑i=1

Pi = I .

Then there exists a corresponding system of operators N1, . . . ,Nk satisfyingthe equations

N2i = −Pi , NiPi = PiNi = Ni ,

NiP j = P jNi = 0 , if i , j

and the angles θ1, . . . θk such that

U = P+ − P− +

k∑i=1

Ri(θi)

whereRi(θi) = cos θi Pi + sin θi Ni .

are the two-dimensional rotation operators in the planes corresponding toPi.

• Theorem. Every invertible operator A on a real vector space can be writtenin a unique way as a product

A = OR

of an orthogonal operator O and a symmetric positive operator R.



2.4.8 Exercises.1. Find the eigenvalues of a projection operator.

2. Prove that the span of all eigenvectors corresponding to the eigenvalue λ ofan operator A is a vector space.

3. LetE(λ) = Ker (A − λI) .

Show that: a) if λ is not an eigenvalue of A, then E(λ) = ∅, and b) if λ is aneigenvalue of A, then E(λ) is the eigenspace corresponding to the eigenvalueλ.

4. Show that the operator A−λI is invertible if and only if λ is not an eigenvalueof the operator A.

5. Let T be a unitary operator. Then the operators A and

A = TAT−1

are called similar. Show that the eigenvalues of similar operators are thesame.

6. Show that an operator similar to a selfadjoint operator is selfadjoint and anoperator similar to an anti-selfadjoint operator is anti-selfadjoint.

7. Show that all eigenvalues of a positive operator A are non-negative.

8. Show that the eigenvectors corresponding to distinct eigenvalues of a uni-tary operator are orthogonal to each other.

9. Show that the eigenvectors corresponding to distinct eigenvalues of a self-adjoint operator are orthogonal to each other.

10. Show that all eigenvalues of a unitary operator A have absolute value equalto 1.

11. Show that if A is a projection, then it can only have two eigenvalues: 1 and0.


Chapter 3

Infinite-Dimensional Vector Spaces

3.1 Vector Spaces• Example (Function Spaces). The vector space F(X, E) of all functions

f : X → E from a set X into a vector space E.

• Example. Let Ω ⊂ Rn be an open subset in Rn. The space of all functionsfrom Ω into C is a vector space. The following are subspaces of this vectorspace:

1. C(Ω) (continuous functions)

2. Ck(Ω) (functions with continuous partial derivatives of order k)

3. C∞(Ω) (smooth functions)

4. P(Ω) (polynomials)

• Example (Sequence Spaces). Let N be the set of positive integers. Thespace F(N,F) of all functions fromN into F is the vector space of sequencesof scalars. The following are subspaces of this vector space:

1. bounded sequences,

2. convergent sequences,

• Example (lp-Spaces). Let p ≥ 1. The space of infinite sequences of com-plex numbers, (zn), such that

∞∑n=1

|zn|p < ∞

61

62 CHAPTER 3. INFINITE-DIMENSIONAL VECTOR SPACES

Proof: Use Minkowski inequality.

• Notation: Let

‖ x ‖p=

∞∑n=1

|xn|p

1p

and let xy denote the sequence (xnyn).

•

Theorem 3.1.1 Minkowski’s Inequality Let p ≥ 1. Let (xn) and (yn)be any two sequences of complex numbers. Then

‖ x + y ‖p≤‖ x ‖p + ‖ y ‖p

Proof:

1. If p = 1 it is true by triangle inequality.2. If p > 1, then

(‖ x + y ‖p)p =

∞∑n=1

|xn + yn|p ≤

∞∑n=1

|xn ‖ xn + yn|p−1 +

∞∑n=1

|yn ‖ xn + yn|p−1

3. Further, by Holder inequality

(‖ x + y ‖p)p ≤ (‖ x ‖p + ‖ y ‖p)

∞∑n=1

|xn + yn|q(p−1)

1q

4. We have q(p − 1) = p. Thus

(‖ x + y ‖p)p ≤ (‖ x ‖p + ‖ y ‖p) (‖ x + y ‖p)pq

This gives the inequality.

• Examples (Cartesian Product of Vector Spaces). Let E jnj=1 be a collec-

tion of vector spaces over a field F. The Cartesian product (or product) ofvector spaces E j is the space

E = E1 × · · · × En

= (x1, . . . , xn) | x j ∈ E j, 1 ≤ j ≤ n .

3.1.1 Homework• Exercises: [1,2,4(c),6].


3.2. CONVERGENCE IN NORMED SPACES 63

3.2 Convergence in Normed Spaces• Examples.

1. Norm of Uniform Convergence. Let Ω ⊂ Rn be a closed boundedsubset of Rn and C(Ω) be the space of continuous functions on Ω.Norm in C(Ω)

‖ f ‖∞= maxx∈Ω| f (x)|

2. Norm in lp

‖ x ‖p=

∞∑n=1

|xn|p

1p

3. Find the limit p→ ∞.

•Definition 3.2.1 Normed Space. A vector space with a norm is calleda normed space.

• One can define different norms on the same vector space.

• A normed space is a pair (E, ‖ · ‖), where E is a vector space and ‖ · ‖ is anorm on E.

• Some vector spaces have standard norms.

• A vector subspace of a normed space is a normed space with the same norm.

• The norm can be used to define convergence.

•

Definition 3.2.2 Convergence in a Normed Space. Let (E, ‖ · ‖) bea normed space and (xn) be a sequence of vectors in E. The sequence(xn) converges to x ∈ E if for every ε > 0 there exists a positive integerM ∈ N such that for every n ≥ M we have

‖ xn − x ‖< ε.

Then we write x = limn→∞ xn or xn → x.

• xn → x simply means that ‖ xn − x ‖→ 0 in R.

• Properties of convergence in normed space.



• A convergent sequence has a unique limit.

• If xn → x and λn → λ, then λnxn → λx.

• If xn → x and yn → y, then xn + yn → x + y.

• Not every convergence in a vector space can be defined by a norm.

• Example (Uniform Convergence). Let C(Ω) be the space of all continuousfunctions on a closed bounded set Ω ⊂ Rn and let ( fn) ∈ C(Ω) be a sequenceof functions in C(Ω). The sequence ( fn) converges uniformly to f if forevery ε > 0 there exists a positive integer M = M(ε) ∈ N such that for allx ∈ Ω and for all n ≥ M we have

| f (x) − fn(x)| < ε.

The norm of uniform convergence defines the uniform convergence, i.e. thesequence ( fn) converges uniformly to f if and only if

‖ fn − f ‖∞= maxx∈Ω| fn(x) − f (x)| → 0.

• Example (Pointwise Convergence). Let C([0, 1]) be the space of continu-ous functions on the interval [0, 1] and let ( fn) be a sequence of functions inC([0, 1]). The sequence ( fn) converges pointwise to f if for all x ∈ [0, 1]and for every ε > 0 there exists a positive integer M = M(ε, x) ∈ N suchthat for all n ≥ M we have

| f (x) − fn(x)| < ε.

The pointwise convergence simply means that for every x ∈ [0, 1] the se-quence ( fn(x)) converges to f (x), i.e.

fn(x)→ f (x) or | fn(x) − f (x)| → 0.

• There is no norm on C([0, 1]) which defines the pointwise convergence.

Proof: (by contradiction). Construct a sequence ( fn) of functions such that

1. ‖ fn ‖= 1 for all n ∈ N and

2. fn(x)→ 0 as n→ ∞ ∀x ∈ [0, 1].



•

Definition 3.2.3 Equivalence of Norms. Two norms on the samevector space E are equivalent if they define the same convergence.

That is, the norms ‖ · ‖1 and ‖ · ‖2 are equivalent if for any sequence(xn) in E and x ∈ E,

‖ xn − x ‖1→ 0 if and only if ‖ xn − x ‖2→ 0 .

• Example. R2.

•

Theorem 3.2.1 Two norms ‖ · ‖1 and ‖ · ‖2 in a vector space E areequivalent if and only if there exist positive real numbers α, β ∈ R+ suchthat

α ‖ x ‖1≤‖ x ‖2≤ β ‖ x ‖1 for all x ∈ E.

Proof:

1. This condition implies the equivalence of norms (obvious).

2. Let the norms be equivalent.

3. Assume that there is no α such that

α ‖ x ‖1≤‖ x ‖2 for all x ∈ E.

4. Then there exists a sequence (xn) such that

1n‖ xn ‖1>‖ xn ‖2 .

5. Letyn =

1√

nxn

‖ xn ‖2.

6. Then‖ yn ‖2=

1√

nand ‖ yn ‖1≥

√n.

7. Contradiction.

• Every normed space (E, ‖ · ‖) is a metric space (E, d) with the metric

d(x, y) =‖ x − y ‖ .



•

Definition 3.2.4 A metric space (E, d) is a set E with a metric d. Ametric d on a set E is a function d : E × E → R satisfying the followingaxioms: ∀x, y, z ∈ E

1. d(x, y) ≥ 0,

2. d(x, y) = 0 if and only if x = y,

3. d(x, z) ≤ d(x, y) + d(y, z) .

•

Definition 3.2.5 A topological space (E,T ) is a set E with a topol-ogy T . A topology T on a set E is a collection T of subsets of E (calledopen sets) that contains E and ∅ and is closed under union and finiteintersection.Topology satisfies the following axioms:

1. E, ∅ ∈ T ,

2. ∪α∈AOα ∈ T for any subcollection of open sets Oαα∈A,

3. ∩nk=1Ok ∈ T for a finite subcollection of open sets Ok

nk=1 .

• The convergence defined by the norm ‖ · ‖ is the same as the convergencedefined by the metric d(x, y) =‖ x − y ‖.

• The metric defines a topology in E (open and closed sets).

• The basic topological notions can be defined without a metric.

•

Definition 3.2.6 Open Balls, Closed Balls, Spheres. Let E be anormed space, x ∈ E and r ∈ R+ a positive real number. We define thefollowing sets:

Open ballB(x, r) = y ∈ E | ‖ x − y ‖< r

Closed ballB(x, r) = y ∈ E | ‖ x − y ‖≤ r

SphereS (x, r) = y ∈ E | ‖ x − y ‖= r

Here x is the center and r is the radius.



• Examples. R2, C([0, 1]), ‖ · ‖∞.

•

Definition 3.2.7 Open and Closed Sets. A subset S ⊆ E of a normedspace E is open if for every x ∈ S there exist ε > 0 such that B(x, ε) ⊆ S .

A subset S ⊆ E of a normed space E is closed if its complement E \ Sis open.

• Example. Let Ω be a closed bounded set in Rn and C(Ω) be the space ofcontinuous functions on Ω with the norm of uniform convergence ‖ · ‖∞.Let f ∈ C(Ω) such that f (x) > 0 for all x ∈ Ω. The set

g ∈ C(Ω | g(x) < f (x), ∀x ∈ Ω

is open C(Ω), and the sets

g ∈ C(Ω | g(x) ≤ f (x), ∀x ∈ Ω

andg ∈ C(Ω | g(x0) = λ (x0 ∈ Ω, λ ∈ C)

are closed in C(Ω).

•

Theorem 3.2.2 1. The union of any number of open sets is open.

2. The intersection of a finite number of open sets is open.

3. The union of a finite number of closed sets is closed.

4. The intersection of any number of closed sets is closed.

5. The empty set and the whole space are both open and closed.

Proof: Exercise.

•

Theorem 3.2.3 A subset S of a normed space E is closed if and only ifevery sequence of elements of S convergent in E has its limit in S .That is, if xn ∈ S and xn → x, then x ∈ S .

Proof:

1. (I) Suppose S is closed, xn ∈ S , xn → x and x < S .

2. ∃ε > 0 such that B(x, ε) ⊆ E \ S .



3. But ‖ xn − x ‖→ 0 (contradiction).

4. (II) Suppose that for any xn ∈ S if xn → x, then x ∈ S but S is notclosed.

5. Then E \ S is not open. So ∃x ∈ E \ S such that every ball B(x, ε),∀ε > 0, contains elements of S .

6. ∃xn ∈ S such that xn ∈ B(x, 1n ).

7. Then xn → x ∈ S , which contradicts x ∈ E \ S .

•

Definition 3.2.8 Closure. Let S be a subset of a normed space E.The closure of S (denoted by S or cl S ) is the intersection of all closedsets containing S .

• The closure of a set is a closed set.

• The closure of a set is the smallest closed set which contains S .

•

Theorem 3.2.4 Let S be a subset of a normed space E. The closure ofS is the set of limits of all convergent sequences of elements of S .That is

cl S = x ∈ E | ∃xn ∈ S such that xn → x

Proof: Exercise.

• Examples (Weierstrass Approximation Theorem). The closure of the setof all polynomials on [a, b] is the whole space C([a, b]).

•Definition 3.2.9 Dense Subsets. A subset S of a normed space E isdense in E if cl S = E.

• Examples.

1. The set of all polynomials on [a, b] is dense in C([a, b]).

2. The set of all sequences of complex numbers which have only a finitenumber of nonzero terms is dense in lp for any p ≥ 1.



•

Theorem 3.2.5 Let S be a subset of a normed space E. The followingconditions are equivalent:

1. S is dense in E.

2. For every x ∈ E there exist xn ∈ S such that xn → x.

3. Every nonempty open subset of E contains an element of S .

Proof: Exercise.

•

Definition 3.2.10 Compact Sets. A subset S of a normed space E iscompact in E if every sequence in S contains a convergent subsequencewhose limit belongs to S .

• Examples. Rn, Cn

• Theorem 3.2.6 Compact sets are closed and bounded.

Proof:

1. (I) Let S be compact, xn ∈ S and xn → x.

2. ∃ a subsequence xnk which converges to some y ∈ S .

3. Since xnk → x, then x = y ∈ S .

4. So, S is closed.

5. (II) Suppose S is not bounded.

6. ∃ a sequence xn ∈ S such that ‖ xn ‖> n for all n ∈ N.

7. (xn) does not contain a convergent subsequence.

8. So, S is not compact.

• Example (Noncompact Closed and Bounded Set.) Let C([0, 1]) be thespace of continuous functions on [0, 1]. The closed unit ball B(0, 1) is aclosed and bounded set. Let xn(t) = tn ∈ B(0, 1) be the sequence of functionsof unit norm. Then (xn) does not have a convergent subsequence. So, theclosed unit ball B(0, 1) is not compact.



3.2.1 Finite Dimensional Normed Spaces

•

Lemma 3.2.1 Let X ⊂ E be a finite dimensional vector subspace of anormed space (E, ‖ · ‖). Let n = dim X and ei

ni=1 be a basis in X.

Define a norm ‖ · ‖1 on X as follows. For any x =∑n

i=1 αiei ∈ X, let

‖ x ‖1=n∑

i=1

|αi|.

Then there is a real number c ∈ R such that for any x ∈ X we have

‖ x ‖≥ c ‖ x ‖1 .

Proof:

1. This is equivalent to the following: for any ‖ x ‖1= 1 there is a c ∈ Rsuch that

‖ x ‖≥ c .

2. Suppose this is false. Then there exists a sequence (ym) ∈ X such that

‖ ym ‖1= 1, and (ym)→ 0.

3. Let

ym =

n∑i=1

αi,(m)ei.

4. Since ‖ ym ‖1= 1, then ∀i,m, |αi(m)| ≤ 1.

5. For each i, the sequence (αi(m))m∈N is bounded.

6. Then, the sequence (α1,(m)) has a convergent subsequence.

7. By relabeling the sequences, there exists a sequence (ym) ∈ X such that

‖ ym ‖1= 1, (ym)→ 0, and (α1,(m))→ α1.

8. Repeating this procedure n times we get a sequence (ym) ∈ X such that

‖ ym ‖1= 1, (ym)→ 0, and (αi,(m))→ αi.

9. Therefore,

‖ ym ‖1= 1, (ym)→ 0, ym → y =

n∑i=1

αiei.



10. Since‖ y ‖1= 1,

then y , 0.

11. We also have‖ ym ‖→‖ y ‖, 0,

which contradicts ym → 0.

•Theorem 3.2.7 Any two norms on a finite-dimensional vector space areequivalent.

Proof:

1. Let eini=1 be a basis in a finite-dimensional normed space E.

2. Let x =∑n

i=1 αiei ∈ E.

3. Then for a norm ‖ · ‖1, ∃c ∈ R such that

‖ x ‖1≥ cn∑

i=1

|αi|

4. For another norm ‖ · ‖2 by triangle inequality

‖ x ‖2≤n∑

i=1

|αi| ‖ ei ‖2≤ kn∑

i=1

|αi|,

where k = max1≤i≤n ‖ ei ‖2.

5. Thusα ‖ x ‖2≤‖ x ‖1,

where α = c/k.

6. Similarly, the second inequality.

•

Theorem 3.2.8 (Completeness.) Every finite dimensional subspace ofa normed space is complete. Every finite dimensional normed space iscomplete.

•Theorem 3.2.9 (Closedness.) Every finite dimensional subspace of anormed space is closed.



3.2.2 Homework• Exercises: [18,21,22,23,24,25,26]


3.3. BANACH SPACES 73

3.3 Banach Spaces

•

Definition 3.3.1 Cauchy Sequence. A sequence of vectors (xn) ina normed space is a Cauchy sequence if for every ε > 0 there existsM ∈ N such that for all n,m ≥ M,

‖ xm − xn ‖< ε.

•

Theorem 3.3.1 The following statements are equivalent:

1. (xn) is a Cauchy sequence.

2. Let (pn) and (qn) be increasing sequences of positive integers.Then

‖ xpn − xqn ‖→ 0 as n→ ∞.

3. Let (pn) be an increasing sequence of positive integers. Then

‖ xpn+1 − xpn ‖→ 0 as n→ ∞.

Proof:

1. We have (a) implies (b) and (b) implies (c). So, we have to prove (c)implies (a).

2. By contradiction. Suppose that (c) holds but (xn) is not Cauchy.

3. Get a contradiction to (c).

• Every convergent sequence is Cauchy. Proof: Exercise.

• Not every Cauchy sequence in a normed space E converges to a vector inE.

• Example. Incompleteness.

•Lemma 3.3.1 Let (xn) be a Cauchy sequence of vectors in a normedspace. Then the sequence (‖ xn ‖) or real numbers converges.

Proof:



1. We have | ‖ xm ‖ − ‖ xn ‖ | ≤‖ xm − xn ‖. Thus (‖ xn ‖) is Cauchy.

• Every Cauchy sequence is bounded.

Proof: Exercise.

•

Definition 3.3.2 Banach Space. A normed space E is complete (orBanach space) if every Cauchy sequence in E converges to an elementin E.

• Examples.

1. Rn and Cn (with any norm) are complete.

2. The space C(Ω) of continuous functions on a closed bounded subsetΩ ⊂ Rn with the norm of uniform convergence ‖ · ‖∞ is complete.

3. The space lp of complex sequences with the norm ‖ · ‖p is complete.

•Theorem 3.3.2 Completeness of l2. The space of complex sequenceswith the norm ‖ · ‖2 is complete.

Proof:

1. Let (an) ∈ l2 be Cauchy. Let an = (α(n)1 , α(n)

2 , . . . ).

2. Let ε > 0. Then ∃M such that for n,m ≥ M,

‖ an − am ‖2=

∞∑k=1

|α(n)k − α

(m)k |

2 < ε2 .

3. So, for any k ∈ N, for n,m ≥ M,

|α(n)k − α

(m)k | < ε

4. So, (α(n)k ) is Cauchy sequence of complex numbers for each k ∈ N.

5. Let αk = limn→∞ α(n)k and a = (α1, α2, . . . ).

6. Claim: a ∈ l2 and an → a.

7. We have, for any n ≥ M

‖ an − a ‖2< ε2 .



8. By Minkowski inequality

‖ a ‖=‖ a − aM + aM ‖≤‖ a − aM ‖ + ‖ aM ‖< ∞

So, a ∈ l2.

9. Alsolimn→∞‖ an − a ‖= 0

So, an → a.

•

Theorem 3.3.3 Completeness of C([a, b]). The space of complex-valued continuous functions on an interval [a, b] with the norm

‖ f ‖∞= max[a,b]| f (x)|

is complete.

Proof:

1. Let ( fn) ∈ C([a, b]) be Cauchy.

2. Let ε > 0. Then ∃M such that for n,m ≥ M,

‖ fn − fm ‖= max[a,b]| fn(x) − fm(x)| < ε .

3. So, for any x ∈ [a, b], for n,m ≥ M,

| fn(x) − fm(x)| < ε

4. So, ( fn(x)) is Cauchy sequence of complex numbers for each x ∈ [a, b].

5. Letf (x) = lim

n→∞fn(x) for x ∈ [a, b].

6. Claim: f ∈ C([a, b]) and fn → f .

7. We have, for any n ≥ M and ∀x ∈ [a, b]

| fn(x) − f (x)| < ε .



8. Let x, x0 ∈ [a, b]. Then, ∃δ > 0 such that if |x − x0| < δ, then

| fM(x) − fM(x0)| < ε

9. Then if |x − x0| < δ, then

| f (x) − f (x0)| = | f (x) − fM(x) + fM(x) − fM(x0) + fM(x0) − f (x0)|

≤ | f (x) − fM(x)| + | fM(x) − fM(x0)| + | fM(x0) − f (x0)| < 3ε .

So, f is continuous, f ∈ C([a, b]).

10. We have, for any n ≥ M

‖ fn − f ‖< ε

Thus fn → f .

•

Definition 3.3.3 Convergent and Absolutely Convergent Series. Aseries

∑∞n=1 xn converges in a normed space E if the sequence of partial

sums sn =∑n

k=1 xk converges in E.

That is, there is x ∈ E such that ‖ sn − x ‖→ 0 as n→ ∞.

If sn → x, then∑∞

n=1 xn = x.

If∑∞

n=1 ‖ xn ‖< ∞, then the series converges absolutely.

• An absolutely convergent series does not need to converge.

•Theorem 3.3.4 A normed space is complete if and only if every abso-lutely convergent series converges.

Proof:

1. (I) Let E be a Banach space (complete normed space).

2. Let (xn) be an absolutely convergent sequence in E such that

∞∑n=1

‖ xn ‖< ∞

and sn =∑n

k=1 xk.



3. Claim: (sn) is Cauchy.

4. Let ε > 0. Then ∃M such that∞∑

n=M+1

‖ xn ‖< ε

5. Then ∀m, n ≥ M,

‖ sn − sm ‖=‖

m∑j=n+1

x j ‖≤

∞∑j=M+1

‖ xn ‖< ε

So, (sn) is Cauchy in E.

6. Thus, ∃x ∈ E such that sn → x, or∑∞

n=1 xn = x.

7. (II) Conversely, assume that every absolutely convergent series con-verges in E.

8. Claim: E is complete.

9. Let (xn) be Cauchy in E.

10. Then there exists a strictly increasing sequence of positive integers(pk) ∈ N such that for all m, n ≥ pk

‖ xm − xn ‖< 2−k

11. Consider the telescopic series∞∑

k=1

(xpk+1 − xpk)

12. Since∞∑

k=1

‖ xpk+1 − xpk ‖≤

∞∑k=1

2−k < ∞ .

it converges absolutely.

13. Therefore, by assumption it converges.

14. Thus, the partial sums

sn =

n∑k=1

(xpk+1 − xpk) = −xp1 + xpn+1 → s ∈ E

converge to an s ∈ E.



15. Therefore, the sequence

xpn = xp1 + sn−1 → x = xp1 + s

converges to an x ∈ E.

16. Finally,‖ xn − x ‖≤‖ xn − xpn ‖ + ‖ xpn − x ‖→ 0

•Theorem 3.3.5 A closed vector subspace of a Banach space is a Ba-nach space.

Proof:

1. Let E be a Banach space.

2. Let F be a closed vector subspace of E.

3. Let (xn) be a Cauchy sequence in F.

4. Then (xn) is Cauchy in E and xn → x ∈ E.

5. Since F is closed, x ∈ F.

3.3.1 Homework• Exercises: [31,33,34,36]


3.4. LINEAR MAPPINGS 79

3.4 Linear Mappings

• Let L : E1 → E2 be a mapping from a vector space E1 into a vector spaceE2.

• If x ∈ E1, then L(x) is the image of the vector x.

• If A ⊂ E1 is a subset of E1, then the set

L(A) = y ∈ E2 | y = L(x) for some x ∈ A

is the image of the set A.

• If B ⊂ E2 is a subset of E2, then the set

L−1(B) = x ∈ E1 | L(x) ∈ B

is the inverse image of the set B.

• A mapping L : D(L) → E2 may be defined on a proper subset (called thedomain) D(L) ⊂ E1 of the vector space E1.

• The image of the domain, L(D(L)), of a mapping L is the range of L. Thatis the range of L is

R(L) = y ∈ E2 | y = L(x) for some x ∈ D(L) .

• The null space N(L) (or the kernel Ker(L)) of a mapping L is the set of allvectors in the domain D(L) which are mapped to zero, that is

N(L) = x ∈ D(L) | L(x) = 0 .

• The graph Γ(L) of a mapping L is the set of ordered pairs (x, L(x)), that is

Γ(L) = (x, y) ⊂ E1 × E2 | x ∈ D(L) and y = L(x) .



•

Definition 3.4.1 Continuous Mappings. A mapping f : E1 → E2

from anormed space E1 into a normed space E2 is continuous at x0 ∈

E1 if any sequence (xn) in E1 converging to x0 is mapped to a sequencef (xn) in E2 that converges to f (x0).

That is f is continuous at x0 if

‖ xn − x0 ‖→ 0 implies ‖ f (xn) − f (x0) ‖→ 0.

A mapping f : E1 → E2 is continuous if it is continuous at everyx ∈ E1.

• Proposition. The norm ‖ · ‖: E → R in a normed space E is a continuousmapping from E into R.

Proof: If ‖ xn − x ‖→ 0, then

| ‖ xn ‖ − ‖ x ‖ | ≤‖ xn − x ‖→ 0

•

Theorem 3.4.1 Let f : E1 → E2 be a mapping from a normed spaceE1 into a normed space E2. The following conditions are equivalent:

1. f is continuous.

2. The inverse image of any open set of E2 is open in E1.

3. The inverse image of any closed set of E2 is closed in E1.

Proof: Exercise.

•

Definition 3.4.2 Linear Mappings. A mapping L : E1 → E2 islinear if ∀x, y ∈ E1, ∀α, β ∈ F,

L(αx + βy) = αL(x) + βL(y).

• Let S ⊂ E1 be a subset of a vector space E1. A mapping L : S → E2 islinear if ∀x, y ∈ S and ∀α, β ∈ F such that αx + βy ∈ S ,

L(αx + βy) = αL(x) + βL(y).



• Proposition. If S is not a vector subspace of E1, then there is a uniqueextension of L : S → E2 to a linear mapping L : span S → E2 from thevector subspace span S to E2.

Proof: The extension L is defined by linearity. For any x ∈ span S such thatx = α1x1 + · · ·αnxn with xi ∈ S and αi ∈ F, we define

L(x) = α1L(x1) + · · · + αnL(xn) .

• Thus, one can assume that the domain of a linear mapping is a vector space.

• Proposition. The range, the null space and the graph of a linear mappingare vector spaces.

Proof: Exercise.

• For any linear mapping L, L(0) = 0. Thus, 0 ∈ N(L) and the null spaceN(L) is always nonempty.

•

Theorem 3.4.2 A linear mapping L : E1 → E2 from a normed spaceE1 into a normed space E2 is continuous if and only if it is continuousat a point.

Proof:

1. Assume L is continuous at x0 ∈ E1.

2. Let x ∈ E1 and (xn)→ x.

3. Then (xn − x + x0)→ x0.

4. Thus‖ L(xn) − L(x) ‖=‖ L(xn − x + x0) − L(x0) ‖→ 0

•

Definition 3.4.3 Bounded Linear Mappings. A linear mapping L :E1 → E2 from anormed space E1 into a normed space E2 is boundedif there is a real number K ∈ R such that for all x ∈ E1,

‖ L(x) ‖≤ K ‖ x ‖ .

•Theorem 3.4.3 A linear mapping L : E1 → E2 from a normed spaceE1 into a normed space E2 is continuous if and only if it is bounded.

Proof:



1. (I). Assume that L is bounded.

2. Claim: L is continuous at 0.

3. Indeed, xn → 0 implies

‖ L(xn) ‖≤ K ‖ xn ‖→ 0

4. Hence, L is continuous.

5. (II). Assume that L is continuous.

6. By contradiction, assume that L is unbounded.

7. Then, there is a sequence (xn) in E1 such that

‖ L(xn) ‖> n ‖ xn ‖

8. Letyn =

xn

n ‖ xn ‖, n ∈ N

9. Then

‖ yn ‖=1n, and ‖ L(yn) ‖> 1

10. Then yn → 0 but L(yn) 6→ 0.

11. Thus, L is not continuous at zero.

• Remark. For linear mappings, continuity and uniform continuity are equiv-alent.

• The set L(E1, E2) of all linear mappings from a vector space E1 into a vectorspace E2 is a vector space with the addition and multiplication by scalarsdefined by

(L1 + L2)(x) = L1(x) + L2(x), and (αL)(x) = αL(x) .

• The set B(E1, E2) of all bounded linear mappings from a normed space E1

into a normed space E2 is a vector subspace of the space L(E1, E2).



•

Theorem 3.4.4 The space B(E1, E2) of all bounded linear mappingsL : E1 → E2 from a normed space E1 into a normed space E2 is anormed space with norm defined by

‖ L ‖= supx∈E1,x,0

‖ L(x) ‖‖ x ‖

= supx∈E1,‖x‖=1

‖ L(x) ‖ .

Proof:

1. Obviously, ‖ L ‖≥ 0.

2. ‖ L ‖=0 if and only if L = 0.

3. Claim: ‖ L ‖ satisfies triangle inequality.

4. Let L1, L2 ∈ B(E1, E2).

5. Then

‖ L1 + L2 ‖ = sup‖x‖=1‖ L1(x) + L2(x) ‖

≤ sup‖x‖=1‖ L1(x) ‖ + sup

‖x‖=1‖ L2(x) ‖

= ‖ L1 ‖ + ‖ L2 ‖

• For any bounded linear mapping L : E1 → E2

‖ L(x) ‖≤‖ L ‖ ‖ x ‖, ∀x ∈ E1 .

• ‖ L ‖ is the least real number K such that

‖ L(x) ‖≤ K ‖ x ‖ for all x ∈ E1.

• The norm defined by ‖ L ‖= supx∈E1,‖x‖=1 ‖ L(x) ‖ is called the operatornorm.

• Convergence with respect to the operator norm is called the uniform con-vergence of operators.



• The strong convergence in B(E1, E2) is defined as follows:

Definition 3.4.4 A sequence of bounded linear mappings Ln ∈

B(E1, E2) converges strongly to L ∈ B(E1, E2) if for every x ∈ E1 wehave

‖ Ln(x) − L(x) ‖→ 0 as n→ ∞.

• Proposition. Uniform convergence implies strong convergence.

Proof: Follows from

‖ Ln(x) − L(x) ‖≤‖ Ln − L ‖ ‖ x ‖

• Converse is not true.

•Theorem 3.4.5 Let E1 be a normed space and E2 be a Banach space.Then B(E1, E2) is a Banach space.

Proof:

1. Claim: B(E1, E2) is complete.

2. Let (Ln) be a Cauchy sequence in B(E1, E2).

3. Then ∀x ∈ E1, as m, n→ ∞

‖ Lm(x) − Ln(x) ‖≤‖ Lm − L − n ‖‖ x ‖→ 0

4. So, ∀x ∈ E1, (Ln(x)) is Cauchy sequence in E2.

5. Therefore, Ln(x)→ y(x) ∈ E2.

6. Define L : E1 → E2 by

L(x) = limn→∞

Ln(x) .

7. Claim: L ∈ B(E1, E2).

8. Since (Ln) is Cauchy, ∃M such that ∀n ∈ N,

‖ Ln ‖≤ M

9. Hence

‖ L(x) ‖=‖ limn→∞

Ln(x) ‖= limn→∞‖ Ln(x) ‖≤ M ‖ x ‖ .



10. So, L is bounded, that is L ∈ B(E1, E2).

11. Claim: ‖ Ln − L ‖→ 0.

12. Let ε > 0.

13. ∃k ∈ N such that ∀n,m ≥ k,

‖ Lm − Ln ‖< ε

14. Thus,‖ Lm(x) − Ln(x) ‖≤‖ Lm − Ln ‖< ε ‖ x ‖ .

15. So, as n→ ∞ for any m ≥ k

‖ Lm(x) − L(x) ‖< ε ‖ x ‖ .

16. Thus, ∀ε > 0, ∃k ∈ N such that ∀m ≥ k,

‖ Lm − L ‖≤ ε

•

Theorem 3.4.6 Let E1 be a normed space and E2 be a Banach space.Let S ⊂ E1 be a subspace of E1 and L : S → E2 be a continuous linearmapping from S into E2. Then L has a unique extension to a continuouslinear mapping L : S → E2 defined on the closure of the domain of themapping L.

If S is dense in E1, then L has a unique extension to a continuous linearmapping L : E1 → E2.

Proof:

1. Let x ∈ S .

2. ∃(xn) ∈ S , xn → x.

3. Then

‖ L(xm) − L(xn) ‖≤‖ L ‖‖ xm − xn ‖→ 0 as n,m→ ∞.

4. Thus, (L(xn)) is a Cauchy sequence in E2.

5. So, L(xn)→ z ∈ E2.



6. Define the extension L : S → E2 of L by L(x) = z, that is

L(x) = limn→∞

(xn).

7. Claim: L(x) does not depend on the sequence xn but only on its limit.

8. Let yn ∈ S , yn → x.

9. Thenyn − xn → 0, and L(yn − xn)→ 0.

10. HenceL(yn) = L(yn − xn) + L(xn)→ z

11. Since L is continuous, for any x ∈ S , L(x) = L(x).

12. Also, L is a linear mapping.

13. Claim: L is continuous.

14. Let x ∈ S and xn ∈ S such that xn → x.

15. Then

‖ L(x) ‖ = ‖ limn→∞

L(xn) ‖= limn→∞‖ L(xn) ‖

≤‖ L ‖ limn→∞‖ xn ‖

=‖ L ‖‖ x ‖ .

16. Thus L is bounded and ‖ L ‖=‖ L ‖.

•

Theorem 3.4.7 Let E1 and E2 be normed spaces, S ⊂ E1 be a subspaceof E1 and L : S → E2 be a continuous linear mapping from S into E2.Then the null space N(L) is a closed subspace of E1.

If the domain S is a closed subspace of E1, then the graph Γ(L) of L isa closed subspace of E1 × E2.

Proof: Exercise.

• A bounded linear mapping L : E → F from a normed space E into the scalarfield F is called a functional.



• The space B(E,F) of functionals is called the dual space and denoted by E′

or E∗.

• The dual space is always a Banach space.

Proof: since F is complete.

•

Theorem 3.4.8 Diagonal Theorem. Let E be a normed space. LetX : N×N→ E be a mapping defined by an infinite matrix (xi j), i, j ∈ N,of elements of E such that

1. ∀ j ∈ N, limi→∞ xi j = 0, and

2. every increasing sequence (p j) of positive integers has a subse-quence (q j) such that

limi→∞

∞∑j=1

xqiq j = 0

Thenlimi→∞

xii = 0 .

Proof: Read in the textbook.

•

Theorem 3.4.9 Banach-Steinhaus Theorem (Uniform BoundednessPrinciple). Let T be a family of bounded linear mappings from a Ba-nach space X into a normed space Y. If for every x ∈ X there exists aconstant Cx such that ‖ T (x) ‖≤ Cx for all T ∈ T , then there exists aconstant M > 0 such that

‖ T ‖≤ M for all T ∈ T

Proof: Read in the textbook.




3.5 Completion of Normed Spaces

•

Definition 3.5.1 Let (E, ‖ · ‖) be a normed space. A normed space(E, ‖ · ‖1) is a completion of (E, ‖ · ‖) if

1. There exists a linear injection Φ : E → E,

2. For every x ∈ E

‖ x ‖=‖ Φ(x) ‖1,

3. Φ(E) is dense in E,

4. E is complete.

• The space E is defined as follows.

• Let (xn) and (yn) be Cauchy sequences in E.

• The sequences (xn) and (yn) are equivalent,

(xn) ∼ (yn), if lim ‖ xn − yn ‖= 0.

• The set of equivalent Cauchy sequences equivalent to a Cauchy sequence(xn) is the equivalence class of (xn)

[(xn)] = (yn) ∈ E | (yn) ∼ (xn)

• The set of all equivalent classes is

E = E/ ∼= [(xn)] | (xn) is Cauchy sequence in E

• The addition and multiplication by scalars in E are defined by

[(xn)] + [(yn)] = [(xn + yn)], λ[(xn)] = [(λxn)]

• The norm in E is defined by the limit

‖ [(xn)] ‖1= limn→∞‖ xn ‖,

which exists for every Cauchy sequence.


3.5. COMPLETION OF NORMED SPACES 89

• This definition is consistent since for any two equivalent Cauchy sequences(xn) and (yn)

‖ [(xn)] ‖1=‖ [(yn)] ‖1

Proof: Exercise.

• The linear bijection Φ : E → E is defined by a constant sequence

Φ(x) = [(xn)] such that xn = x, ∀n ∈ N.

• Then Φ is one-to-one.

Proof: Exercise.

• Obviously, ∀x ∈ E, ‖ x ‖=‖ Φ(x) ‖1.

• Claim: Φ(E) is dense in E.

Proof: Since every element [(xn)] of E is the limit of a sequence (Φ(xn)).

• Claim: E is complete.

Proof:

• Let (yn) be a Cauchy sequence in E.

• Then ∃(xn) such that

‖ Φ(xn) − yn ‖1<1n.

• Claim: (xn) is Cauchy sequence in E.

Proof:

‖ xn − xm ‖ = ‖ Φ(xn) − Φ(xm) ‖1

≤ ‖ Φ(xn) − yn ‖1 + ‖ yn − ym ‖1 + ‖ ym − Φ(xm) ‖1

≤ ‖ yn − ym ‖1 +1n

+1m.

• Let y = [(xn)].



• Claim:‖ yn − y ‖1→ 0.

Proof:

‖ yn − y ‖1 ≤ ‖ yn − Φ(xn) ‖1 + ‖ Φ(xn) − y ‖1

< ‖ Φ(xn) − y ‖1 +1n→ 0. (3.1)

•

Definition 3.5.2 Homeomorphism. Two topological spaces E1 andE2 are homeomorphic if there exists a bijection Ψ : E1 → E2 from E1

onto E2 such that both Ψ and Ψ−1 are continuous.

•

Definition 3.5.3 Isomorphism of Normed Spaces. Two normedspaces (E1, ‖ · ‖1) and (E2, ‖ · ‖2) are isomorphic if there exists a linearhomeomorphism Ψ : E1 → E2 from E1 onto E2.

• Any two completions of a normed space are isomorphic.

Proof: Read elsewhere.

3.5.1 Homework• Exercises: [ ]


Chapter 4

Hilbert Spaces and OrthonormalSystems

4.1 Inner Product Spaces

•

Definition 4.1.1 Inner Product Space. A complex vector space E iscalled an inner product space (or a pre-Hilbert space, or a unitaryspace) if there is a mapping (·, ·) : E×E → C, called an inner product,that satisfies the conditions: ∀x, y, z ∈ E, ∀α ∈ C:

1. (x, x) ≥ 0

2. (x, x) = 0 if and only if x = 0

3. (x + y, z) = (x, z) + (y, z)

4. (αx, y) = (x, y)

5. (x, y) = (y, x)∗

• Examples.

91

92 CHAPTER 4. HILBERT SPACES AND ORTHONORMAL SYSTEMS

4.1.1 Finite-dimensional spaces

• Cn is the space of n-tuples x = (x1, . . . , xN) of complex numbers. It is aHilbert space with the inner product

(x, y) =

n∑j=1

x jy∗j

4.1.2 Spaces of sequences

• l2 is the space of sequences of complex numbers x = (xn)∞n=1 such that

∞∑n=1

|xn|2 < ∞ .

It is an inner product space with the inner product

(x, y) =

∞∑j=1

x jy∗j

• l0 is the space of sequences of complex numbers with zero tails with theinner product

(x, y) =

∞∑j=1

x jy∗j

4.1.3 Spaces of continuous functions

• C([a, b]) with the inner product

( f , g) =

∫ b

af g∗

• C0(R) (space of continuous functions with compact support) with the innerproduct

( f , g) =

∫R

f g∗


4.1. INNER PRODUCT SPACES 93

4.1.4 Spaces of square integrable functions• L2([a, b]) is the space of complex valued functions such that∫ b

a| f |2 < ∞ .


( f , g) =

∫R

f g∗

• L2([a, b], µ) (space of square integrable functions with the measure µ) withthe inner product

( f , g) =

∫ b

aµ f g∗ ,

where µ > 0 almost everywhere.

• Let Ω be an open set in Rn (in particular, Ω can be the whole Rn). The spaceL2(Ω) is the set of complex valued functions such that∫

Ω

| f |2 < ∞ ,

where x = (x1, . . . , xn) ∈ Ω and dx = dx1 · · · dxn. It is an inner productspace with the inner product

( f , g) =

∫Ω

f g∗

• L2(Rn) with the inner product

( f , g) =

∫Rn

f g∗

• Let Ω be an open set in Rn (in particular, Ω can be the whole Rn) and V bea finite-dimensional vector space. The space L2(Ω,V) is the set of vectorvalued functions f = ( f1, . . . , fN) on Ω such that

N∑i=1

∫Ω

| fi|2 < ∞ .


( f , g) =

N∑i=1

∫Ω

fig∗i



4.1.5 Real Inner Product Spaces• Remark. The inner product in a real inner product space is symmetric.

• A finite dimensional real inner product space is called a Euclidean space.

• Rn is a Euclidean space.

4.1.6 Direct Sum of Inner Product Spaces• Let E1 and E2 be inner product spaces. The direct sum E = E1 ⊕ E2 of E1

and E2 is an inner product space of ordered pairs z = (x, y) with x ∈ E1 andy ∈ E2 with the inner product defined by

(z1, z2)E = (x1, x2)E1 + (y1, y2)E2 .

4.1.7 Tensor Products of Inner Product Spaces• Let E1 and E2 be inner product spaces. For each ϕ1 ∈ E1 and ϕ2 ∈ E2 letϕ1 ⊗ ϕ2 denote the conjugate bilinear form on E1 × E2 defined by

(ϕ1 ⊗ ϕ2)(ψ1, ψ2) = (ψ1, ϕ1)E1(ψ2, ϕ2)E2

where ψ1 ∈ E1 and ψ2 ∈ E2. Let E be the set of finite linear combinationsof such bilinear forms. An inner product on E can be defined by

(ϕ ⊗ ψ, η ⊗ µ)E = (ϕ, η)E1(ψ, µ)E2

(with ϕ, η ∈ E1 and ψ, µ ∈ E2) and extending by linearity on E.

• Let E be an inner product space. The space

F(E) = C ⊕∞n=1 E ⊗ · · · ⊗ E︸︷︷︸n

is called the Fock space over E.

4.1.8 Homework• Exercises: [4,5]


4.2. NORM IN AN INNER PRODUCT SPACE 95

4.2 Norm in an Inner Product Space

•

Definition 4.2.1 Norm in an Inner Product Space. Let E be aninner product space. The norm in E is a functional ‖ · ‖: E → R definedby

‖ x ‖=√

(x, x).

•

Theorem 4.2.1 Every inner product space is a normed space with thenorm ‖ x ‖=

√(x, x) and a metric space with the metric d(x, y) =√

(x − y, x − y).

•

Theorem 4.2.2 Schwarz’s Inequality. Let E be an inner productspace. Then for any x, y ∈ E we have

|(x, y)| ≤‖ x ‖ ‖ y ‖ .

The equality |(x, y)| =‖ x ‖ ‖ y ‖ holds if and only if x and y are linearlydependent.

Proof:

1.

•

Corollary 4.2.1 Triangle Inequality. Let E be an inner product space.Then for any x, y ∈ E we have

‖ x + y ‖≤‖ x ‖ + ‖ y ‖ .

Proof:

1.

•

Theorem 4.2.3 Parallelogram Law. Let E be an inner product space.Then for any x, y ∈ E we have

‖ x + y ‖2 + ‖ x − y ‖2= 2(‖ x ‖2 + ‖ y ‖2

)Proof:



1.

•Definition 4.2.2 Orthogonal Vectors. Let E be an inner productspace. Two vectors x, y ∈ E are orthogonal if (x, y) = 0.

• Notation. The orthogonality of the vectors x and y is denoted by

x ⊥ y

• The relation ⊥ is symmetric, that is, if x ⊥ y then y ⊥ x.

•

Theorem 4.2.4 Pythagorean Theorem. Let E be an inner productspace. If two vectors x, y ∈ E are orthogonal then

‖ x + y ‖2=‖ x ‖2 + ‖ y ‖2 .

Proof:

1.

• Examples.

4.2.1 Homework• Exercises: [9,10,11,12,13]


4.3. HILBERT SPACES 97

4.3 Hilbert Spaces

•Definition 4.3.1 Hilbert Space. An inner product space is called aHilbert space if it is complete as a normed space.

• Examples.

4.3.1 Finite dimensional Hilbert spaces• Cn is complete.

4.3.2 Spaces of sequences• The space of square summable sequences (l2) is complete. Proved before.

• The space of sequences with vanishing tails (l0) is not complete. Counterex-ample.

4.3.3 Spaces of continuous functions• C([a, b]) is not complete. Counterexample.

• C0(R) (space of continuous functions with compact support) is not com-plete. Counterexample.

4.3.4 Spaces of square integrable functions.

• Theorem 4.3.1 L2([a, b]) is complete.

Proof:

1.

• Theorem 4.3.2 L2(R) is complete.

Proof:

1.



• Theorem 4.3.3 L2([a, b], µ) is complete.

Proof:

1.

4.3.5 Sobolev Spaces• Let Ω be an open set in Rn (in particular, Ω can be the whole Rn) and V

a finite-dimensional complex vector space. Let Cm(Ω,V) be the space ofcomplex vector valued functions that have continuous partial derivatives ofall orders less or equal to m. Let

α = (α1, . . . , αn),

α j ∈ N, be a multiindex of nonnegative integers, αi ≥ 0, and let

|α| = α1 + · · · + αn.

Define

Dα f =∂|α|

∂xα11 · · · ∂xαn

nf .

Then f ∈ Cm(V,Ω) iff ∀α, |α| ≤ m, ∀i = 1, . . . ,N, ∀x ∈ Ω we have

|Dα fi(x)| < ∞ .

The space Hm(Ω,V) is the space of complex vector valued functions suchthat ∀α, |α| ≤ m,

Dα f ∈ L2(Ω,V)

i.e. such that ∀α, |α| ≤ m,N∑

i=1

∫Ω

|Dα fi(x)|2dx < ∞ .


( f , g) =∑

α, |α|≤m

N∑i=1

∫Ω

Dα fi(Dαgi)∗

The Sobolev space Hm(Ω,V) is the completion of the space Hm(Ω,V) de-fined above.



4.4 Strong and Weak Convergence

•

Definition 4.4.1 Strong Convergence. A sequence (xn) of vectors inan inner product space E is strongly convergent to a vector x ∈ E if

limn→∞‖ xn − x ‖= 0

• Notation. xn → x.

•

Definition 4.4.2 Weak Convergence. A sequence (xn) of vectors inan inner product space E is weakly convergent to a vector x ∈ E if forany y ∈ E

limn→∞

(xn − x, y) = 0

• Notation. xnw→ x

•Theorem 4.4.1 A strongly convergent sequence is weakly convergentto the same limit.

Proof:

1.

• Converse is not true. Counterexample.

•

Corollary 4.4.1 Let E be an inner product space. Then for every y ∈ Ethe linear functional ϕy : E → C defined by

ϕy(x) = (x, y) ∀x ∈ E

is continuous.

•

Theorem 4.4.2 Let (xn) be a sequence in an inner product space E. If:

1. xnw→ x and

2. ‖ xn ‖→‖ x ‖,

then xn → x.

Proof:


4.4. STRONG AND WEAK CONVERGENCE 101

1.

•

Theorem 4.4.3 Let S be a subset of an inner product space E and (xn)be a sequence in E. If:

1. span S is dense in E,

2. (xn) is bounded, and

3. for any y ∈ S , limn→∞(xn − x, y) = 0,

then xnw→ x.

Proof:

1.

•Theorem 4.4.4 Weakly convergent sequences in a Hilbert space arebounded.

Proof:

1.




4.5 Orthogonal and Orthonormal Systems

•

Definition 4.5.1 Orthogonal and Orthonormal Systems. Let E bean inner product space. A set S of vectors in E is called an orthogonalsystem if any pair of distinct vectors in S is orthogonal to each other.

An orthogonal system of unit vectors is an orthonormal system.

• Every orthogonal system can be made orthonormal.

• Let S be a set of vectors in E. If x ⊥ y for any y ∈ S , then x ⊥ span S .

• Theorem 4.5.1 Orthogonal systems are linearly independent.

Proof:

1.

•Definition 4.5.2 Orthonormal Sequence. A sequence of vectorswhich is an orthonormal system is an orthonormal sequence.

• Examples.

• Any orthogonal sequence can be always made orthonormal.

• Gram-Schmidt orthonormalization process. Any sequence of linearlyindependent vectors can be made orthonormal.

1. Let (yn) be a linearly independent sequence.

2. Then the sequence (zn) defined by

z1 = y1, zk = yk −

k−1∑n=1

(yk, zn)zn

‖ zn ‖2

is orthogonal.

3. Then the sequence (en) defined by en = zn/ ‖ zn ‖ is orthonormal.

4.5.1 Homework• Exercises: [32,33,37]


4.6. PROPERTIES OF ORTHONORMAL SYSTEMS 103

4.6 Properties of Orthonormal Systems

•

Theorem 4.6.1 Pythagorean Formula. Let E be an inner productspace and xn

Nn=1 be an orthonormal set in E. Then∣∣∣∣∣∣∣

∣∣∣∣∣∣∣N∑

n=1

en

∣∣∣∣∣∣∣∣∣∣∣∣∣∣2

=

N∑n=1

‖ en ‖2

Proof:

1.

•

Theorem 4.6.2 Bessel’s Equality and Inequality. Let E be an innerproduct space and en

Nn=1 be an orthonormal set in E. Then ∀x ∈ E

‖ x ‖2=N∑

n=1

|xn|2 +

∣∣∣∣∣∣∣∣∣∣∣∣∣∣x −

N∑n=1

xnen

∣∣∣∣∣∣∣∣∣∣∣∣∣∣2

andN∑

n=1

|xn|2 ≤‖ x ‖2

where xn = (x, en).

Proof:

1.

• Remarks. Let (en) be an orthonormal sequence in an inner product spaceE.

• The complex numbers xn = (x, en) are generalized Fourier coefficients ofx with respect to the orthonormal sequence (en).

• An orthonormal sequence (en) in E induces a mapping ϕe : E → l2 definedby

ϕ(x) = x = (xn)



• The sequence of Fourier coefficients x = (xn) is square summable, that isx ∈ l2 since for any x ∈ E

∞∑n=1

|xn|2 ≤‖ x ‖2

and, therefore, this series converges.

• The expansion

x ∼∞∑

n=1

xnen

is a generalized Fourier series of x.

• The question is whether the mapping ϕ is bijective and whether the Fourierseries converges.

•

Theorem 4.6.3 Let (en) be an orthonormal sequence in a Hilbert spaceH and (xn) be a sequence of complex numbers. Then the series

∑∞n=1 xnen

converges if and only if (xn) ∈ l2, that is the series∑∞

n=1 |xn|2 converges.

In this case ∣∣∣∣∣∣∣∣∣∣∣∣∣∣∞∑

n=1

xnen

∣∣∣∣∣∣∣∣∣∣∣∣∣∣2

=

∞∑n=1

|xn|2

Proof:

1.

• Fourier series of any x ∈ H in a Hilbert space H converges.

• Fourier series of x may converge to a vector different from x!

• Example.

• Let (en) be an orthonormal sequence in an inner product space E. The se-quence of Fourier coefficients xn = (x, en) is square summable, and, there-fore,

limn→∞

(x, en) = 0 ∀x ∈ E


4.6. PROPERTIES OF ORTHONORMAL SYSTEMS 105

• Thus, every orthonormal sequence weakly converges to zero.

• Orthonormal sequences are not strongly convergent since ‖ en ‖= 1 ∀n ∈ N.

•

Definition 4.6.1 Complete Orthonormal Sequence. Let E be aninner product space. An orthonormal sequence (en) in E is complete if∀x ∈ E the Fourier series of x converges to x.

That is

x =

∞∑n=1

xnen .

More explicitly,

limn→∞

∣∣∣∣∣∣∣∣∣∣∣∣∣∣x −

∞∑n=1

xnen

∣∣∣∣∣∣∣∣∣∣∣∣∣∣ = 0

• Example.

•

Definition 4.6.2 Orthonormal Basis. Let E be an inner productspace. An orthonormal system B in E is an orthonormal basis if forany x ∈ E there exists a unique orthonormal sequence (en) in B and aunique sequence (xn) of nonzero complex numbers such that

x =

∞∑n=1

xnen .

• Remarks.

• A complete orthonormal sequence in an inner product space is an orthonor-mal basis.

• Let E be an inner product space and (en) be a complete orthonormal se-quence. Then the set

S = span en | n ∈ N

is dense in E.

•

Theorem 4.6.4 Let H be a Hilbert space. An orthonormal sequence inH is complete if and only if the only vector orthogonal to this sequenceis the zero vector.

Proof:



1.

•

Theorem 4.6.5 Parseval’s Formula. Let H be a Hilbert space. Anorthonormal sequence (en) in H is complete if and only if ∀x ∈ H

‖ x ‖2=∞∑

n=1

|xn|2

where xn = (x, en).

Proof:

1.

•

Theorem 4.6.6 Let H1 and H2 be Hilbert spaces. If ϕk and ψl areorthonormal bases for H1 and H2 respectively, then ϕk ⊗ ψl is an or-thonormal basis for the tensor product H1 ⊗ H2.

Proof:

1.

• Examples.



4.7. TRIGONOMETRIC FOURIER SERIES 107

4.7 Trigonometric Fourier Series• Consider the Hilbert space L2([−π, π]).

• The sequence

ϕn(x) =1√

2πeinx, n ∈ Z

is an orthonormal sequence in L2([−π, π]).

• Consider the space L1([−π, π]).

• Identify the elements of L1([−π, π]) with 2π periodic functions on R.

• Then for any f ∈ L1([−π, π]),∫ π

π

dt f (t) =

∫ π

π

dt f (t − x)

• Define the sequence

hn =

n∑k=−n

( f , ϕk)ϕk, n ∈ N .

More explicitly,

hn(x) =

∫ π

−π

dt Gn(x − t) f (t)

where

Gn(x) =1

2π

n∑k=−n

eikx

• Define the sequence

Fn =1

n + 1

n∑k=0

hk =1

n + 1

n∑k=0

k∑j=−k

( f , ϕ j)ϕ j

More explicitly,

Fn(x) =

∫ π

−π

dt Kn(x − t) f (t),

where

Kn(x) =1

2π(n + 1)

n∑k=−n

(n + 1 − |k|) eikx

is the Fejer’s kernel.



•

Lemma 4.7.1 We have

Kn(x) =1

2π(n + 1)

sin2[(n + 1) x

2

]sin2

(x2

)Proof:

1.

•

Definition 4.7.1 A sequence κn of 2π-periodic continuous functionsis a summability kernel if it satisfies the conditions:

1. ∀n ∈ N ∫ π

−π

dt κn(t) = 1,

2. There is M ∈ R such that ∀n ∈ N∫ π

−π

dt |κn(t)| ≤ M,

3. For any δ ∈ (0, π)

limn→∞

∫ 2π−δ

δ

dt |κn(t)| = 0

• Lemma 4.7.2 The Fejer’s kernel is a summability kernel.

Proof:

1.

•

Theorem 4.7.1 Let (κn) be a summability kernel and f ∈ L1([−π, π]).Let (Fn) be a sequence defined by

Fn =

∫ π

−π

dt κn(t) f (x − t).

Then Fn strongly converges to f in L1 norm.


4.7. TRIGONOMETRIC FOURIER SERIES 109

Proof:

1.

•Theorem 4.7.2 Let f ∈ L1([−π, π]). If all Fourier coefficients fn =

( f , ϕn) vanish, then f = 0 almost everywhere.

Proof:

1.

•

Theorem 4.7.3 The sequence

ϕn(x) =1√

2πeinx, n ∈ Z

is a complete orthonormal sequence, (an orthonormal basis), inL2([−π, π]).

Proof:

1.

• Let f ∈ L2([−π, π]). The series

f (x) =

∞∑n=−∞

fnϕn(x),

wherefn =

∫ π

−π

dt f (t)ϕn(t),

is the Fourier series. The scalars fn are the Fourier coefficients.

• Fourier series does not converge pointwise!

• Fourier series of a function f ∈ L2([−π, π]) converges almost everywhere.

4.7.1 Homework• Exercises: [45]



4.8 Orthonormal Complements and Projection The-orem

• A subspace of a Hilbert space is an inner product space.

• A closed subspace of a Hilbert space is a Hilbert space.

•

Definition 4.8.1 Orthogonal Complement. Let H be a Hilbert spaceand S ⊂ H be a nonempty subset of H. We say that x ∈ H is orthogonalto S , denoted by x ⊥ S , if ∀y ∈ S , (x, y) = 0.

The setS ⊥ = x ∈ H | x ⊥ S

of all vectors orthogonal to S is called the orthogonal complement ofS .

Two subsets A and B of H are orthogonal, denoted by A ⊥ B, if everyvector of A is orthogonal to every vector of B.

• If x ⊥ H, then x = 0, that is

H⊥ = 0, 0⊥ = H .

• If A ⊥ B, then A ∩ B = 0 or ∅.

•Theorem 4.8.1 The orthogonal complement of any subset of a Hilbertspace is a Hilbert subspace.

Proof:

1. Let H be a Hilbert space and S ⊂ H.

2. Check directly that S ⊥ is a vector subspace.

3. Claim: S ⊥ is closed.

4. Let (xn) be a sequence in S ⊥ such that xn → x ∈ H.

5. Then ∀y ∈ S(x, y) = lim

n→∞(xn, y) = 0.

6. Thus x ∈ S ⊥.


4.8. ORTHONORMAL COMPLEMENTS AND PROJECTION THEOREM111

• Remarks.

• S does not have to be a vector subspace.

•

Definition 4.8.2 Convex Sets. A set U in a vector space E is calledconvex if ∀x, y ∈ U and ∀α ∈ (0, 1),

αx + (1 − α)y ∈ U .

• A vector subspace is a convex set.

•

Theorem 4.8.2 The Closest Point Property. Let H be a Hilbert spaceand S be a closed convex subset of H. Then ∀x ∈ H there exists aunique y ∈ S such that

‖ x − y ‖= infz∈S‖ x − z ‖ .

Proof:

1. (I). Existence. Let d = infz∈S ‖ x − z ‖.

2. Let (yn) be a sequence in S such that

limn→∞‖ x − yn ‖= d.

3. By convexity we have ∀n,m ∈ N

‖ x −12

(yn + ym) ‖≥ d

4. By the parallelogram law, we have

‖ yn − ym ‖2= 2 ‖ x − ym ‖

2 +2 ‖ x − yn ‖2 −4 ‖ x −

12

(ym + yn) ‖2

5. As n,m→ ∞, we have

‖ yn − ym ‖2→ 0 .

6. So, (yn) is Cauchy.



7. Since H is complete ∃ lim yn = y ∈ H.

8. Since S is closed y ∈ S .

9. By continuity, we get

‖ x − y ‖= lim ‖ x − yn ‖= d

10. (II). Uniqueness. Let y1 be another such point.

11. Then, by convexity

‖ y − y1 ‖2= 4d2 − 4 ‖ x −

12

(y + y1) ‖2≤ 0

12. Thus y = y1.

•

Theorem 4.8.3 Let H be a real Hilbert space. Let S be a closed convexsubset of H, y ∈ S and x ∈ H. Then

‖ x − y ‖= infz∈S‖ x − z ‖

if and only if

∀z ∈ S , (x − y, z − y) ≤ 0.

Proof:

1. (I). Let x ∈ H, y, z ∈ S and λ ∈ (0, 1).

2. Suppose ‖ x − y ‖= infz∈S ‖ x − z ‖ .

3. By convexity we have

‖ x − y ‖2 ≤ ‖ (x − y) − λ(z − y) ‖2

≤ ‖ x − y ‖2 −2λ(x − y, z − y) + λ2 ‖ z − y ‖2

4. Therefore, as λ→ 0, we get

(x − y, z − y) ≤ 0 .

5. (II). Let x ∈ H and y ∈ S .

6. Suppose ∀z ∈ S , (x − y, z − y) ≤ 0.



7. Then ∀z ∈ S

‖ x − y ‖2 − ‖ x − z ‖2= 2(x − y, z − y)− ‖ z − y ‖2≤ 0.

8. Thus‖ x − y ‖2≤‖ x − z ‖2

•

Theorem 4.8.4 Orthogonal Projection. Let H be a Hilbert space andS be a closed subspace of H. Then ∀x ∈ H there exist unique y ∈ S andz ∈ S ⊥ such that

x = y + z .

Proof:

1. (I). Existence. If x ∈ S , then let y = x ∈ S and z = 0 ∈ S ⊥ so thatx = y + z.

2. Let x < S .

3. Let y ∈ S be the unique point closest to x.

4. Let z = x − y.

5. Then ‖ z ‖= infw∈S ‖ x − w ‖ .

6. Claim: z ∈ S ⊥.

7. Let w ∈ S and λ ∈ C.

8. Then

‖ z ‖2≤‖ z − λw ‖2=‖ z ‖2 −2Re λ(w, z) + |λ|2 ‖ w ‖2

9. Thus−2Re λ(w, z) + |λ|2 ‖ w ‖2≥ 0 .

10. We obtainRe (w, z) ≤ 0, Im (w, z) ≤ 0 .

11. Since this is true for any w ∈ S , we obtain

(w, z) = 0 .

12. Thus, z ∈ S ⊥.



13. (II). Uniqueness. Suppose x = y + z = y1 + z1.

14. Then y − y1 = z1 − z.

15. Since y − y1 ∈ S and z − z1 ∈ S ⊥, then y − y1 = z − z1 = 0.

• Orthogonal decomposition of H. If every element of H can be uniquelyrepresented as the sum of an element of S and an element of S ⊥, then H isthe direct sum of S and S ⊥, which is denoted by

H = S ⊕ S ⊥

• The union of a basis of S and a basis of S ⊥ gives a basis of H.

• Orthogonal projection. An orthogonal decomposition H = S ⊕ S ⊥ of Hinduces a map P : H → S defined by P(y + z) = y, where y ∈ S and z ∈ S ⊥.

• Examples.

•

Theorem 4.8.5 Let H be a Hilbert space and S be a closed subspaceof H. Then

(S ⊥)⊥ = S .

Proof:

1. Let x ∈ S .

2. Then x ⊥ S ⊥, or x ∈ S ⊥⊥.

3. So, S ⊆ S ⊥⊥.

4. Let x ∈ S ⊥.

5. Since S is closed, there exist y ∈ S and z ∈ S ⊥ such that x = y + z.

6. Then y ∈ S ⊥⊥.

7. Since S ⊥⊥ is a vector space, z = x − y ∈ S ⊥⊥.

8. Since z ∈ S ⊥⊥ and z ∈ S ⊥.

9. Thus, z = 0, and x = y ∈ S .

10. Therefore, S ⊥⊥ ⊆ S .



4.8.1 Homework• Exercises: [51,52,53,55,56]



4.9 Linear Functionals and the Riesz Representa-tion Theorem

• Examples.

• L2((a, b))

• ϕ( f ) = ( f , g) is a linear bounded functional.

• Let x0 ∈ (a, b). Then ϕ( f ) = f (x0) is a linear but unbounded functional.

• Cn

• Let k ∈ 1, . . . , n. Then ϕ(x) = xk = (x, ek) is a linear bounded functional.

•

Lemma 4.9.1 Let E be an inner product space and f : E → C be abounded linear functional on E. Then

dim(N( f ))⊥ ≤ 1.

Proof:

1. If f = 0, then N( f ) = E.

2. Therefore, (N( f ))⊥ = 0 and dim(N( f ))⊥ = 0.

3. Suppose that f , 0.

4. Since f is bounded and linear it is continuous.

5. Therefore, N( f ) is a closed subspace of E.

6. Thus, (N( f ))⊥ is not empty.

7. Let x, y ∈ (N( f ))⊥ be two nonzero vectors.

8. Then f (x) , 0 and f (y) , 0.

9. Therefore, there exists α , 0 ∈ C such that f (x + αy) = 0.

10. Hence, x + αy ∈ N( f ).

11. Since x, y ∈ (N( f ))⊥, we also have x + αy ∈ (N( f ))⊥.

12. Thus x + αy = 0.

13. Therefore, x and y are linearly dependent, and, therefore,

dim(N( f ))⊥ = 1.


4.9. LINEAR FUNCTIONALS AND THE RIESZ REPRESENTATION THEOREM117

•

Theorem 4.9.1 Riesz Representation Theorem. let H be a Hilbertspace and f : H → C be a bounded linear functional on H. Thereexists a unique x0 ∈ H such that

f (x) = (x, x0)

for all x ∈ H. Moreover,

‖ f ‖=‖ x0 ‖ .

Proof:

1. (I). Existence.If f = 0, then x0 = 0.

2. Suppose f , 0.

3. Then dim(N( f ))⊥ = 1.

4. Let u ∈ (N( f ))⊥.

5. Then ∀x ∈ H,x = y + z

where y = x − (x, u)u ∈ N( f ) and z = (x, u)u ∈ (N( f ))⊥.

6. Therefore, f (y) = 0.

7. Further,f (x) = f (z) = (x, u) f (u) = (x, x0),

wherex0 = ( f (u))∗u.

8. (II). Uniqueness. Suppose there exists x0 and x1 such that ∀x ∈ H

f (x) = (x, x0) = (x, x1).

9. Then ∀x ∈ H(x, x0 − x1) = 0.

10. Thus, (x0 − x1) ∈ H⊥ = 0.

11. So, x0 = x1.



12. Finally, we have

‖ f ‖= supx,0

| f (x)|‖ x ‖

= supx,0

|(x, x0)|‖ x ‖

≤‖ x0 ‖ .

13. On another hand

| f (x0)|‖ x0 ‖

=|(x0, x0)|‖ x0 ‖

=‖ x0 ‖ .

14. Thus, ‖ f ‖=‖ x0 ‖.

• Remarks. The set H′ of all bounded linear functionals on a Hilbert spaceis a Banach space, called the dual space.

• The dual space H′ of a Hilbert space H is isomorphic to H.

4.9.1 Homework• Exercises:


4.10. SEPARABLE HILBERT SPACES 119

4.10 Separable Hilbert Spaces

•Definition 4.10.1 Separable Spaces. A Hilbert space is separable ifit is finite-dimensional or contains a complete orthonormal sequence.

• Examples.

• L2([−π, π])

• l2

• Example (Non-separable Hilbert Space). Let H be the space of all com-plex valued functions f : R → C on R such that they vanish everywhereexcept a countable number of points in R and∑

f (x),0

| f (x)|2 < ∞.

Define the inner product by

( f , g) =∑

f (x)g(x),0

f (x)g(x)∗.

Then for any sequence fn in H, there are non-zero functions f such that( f , fn) = 0 for all n ∈ N. Therefore, H is not separable.

•Theorem 4.10.1 Let H be a separable Hilbert space. Then H containsa countable dense subset.

Proof:

1. Let (en) be a complete orthonormal sequence in H.

2. Define the set

S =

n∑k=1

(αk + iβk)ek | αk, βk ∈ Q, n ∈ N

3. Then S is countable.

4. Also, ∀x ∈ H,

limn→∞‖

n∑k=1

(x, ek)ek − x ‖= 0.



5. Therefore, S is dense in H.

•Theorem 4.10.2 Let H be a separable Hilbert space. Then every or-thogonal set S in H is countable.

Proof:

1. Let S be an orthogonal set in H.

2. Let

S 1 =

x‖ x ‖

|x ∈ S.

3. Then ∀x, y ∈ S 1, x , y,

‖ x − y ‖2= 2.

4. Consider the collection of balls B2−1/2(x) for every x ∈ S 1.

5. Then, for any x, y ∈ S 1, x , y,

B2−1/2(x) ∩ B2−1/2(y) = ∅.

6. Since H is separable, it has a countable dense subset A.

7. Since A is dense in H it must have at least one point in every ballB2−1/2(x).

8. Therefore, S 1 must be countable.

9. Thus S is countable.

•

Definition 4.10.2 Unitary Linear Transformations. Let H1 and H2

be Hilbert spaces. A linear map T : H1 → H2 is unitary if ∀x, y ∈ H1

(T (x),T (y))H2 = (x, y)H1 .

•

Definition 4.10.3 Hilbert Space Isomorphism. let H1 and H2 beHilbert spaces. Then H1 is isomorphic to H2 if there exists a linearunitary bijection T : H1 → H2 (called a Hilbert space isomorphism).


4.10. SEPARABLE HILBERT SPACES 121

• Remark. Every Hilbert space isomorphism has unit norm

‖ T ‖= 1.

•

Theorem 4.10.3 1. Every infinite-dimensional separable Hilbertspace is isomorphic to l2.

2. Every finite-dimensional separable Hilbert space H is isomorphicto Cn, where n = dim H.

Proof:

1. (1). Let H be infinite-dimensional.

2. Let (en) be a complete orthonormal sequence in H.

3. Let x ∈ H.

4. Let xn = (x, en).

5. This defines a linear bijection T : H → l2 by

T (x) = (xn) .

6. Let x, y ∈ H.

7. Then

(T (x),T (y))l2 =

∞∑n=1

xny∗n

8. On another hand

(x, y)H = (∞∑

n=1

xnen, y) =

∞∑n=1

xn(en, y) =

∞∑n=1

xny∗n

9. Therefore T is unitary, and is, therefore, an isomorphism from H ontol2.

• Remarks.

• Isomorphism of Hilbert spaces is an equivalence relation.

• All separable infinite-dimensional Hilbert spaces are isomorphic.



4.10.1 Homework• Exercises: 3.12[58,60,61,62]

4.11 Hilbert Spaces• A metric space (M, d) is a set M with a map d : M × M → R called a

metric, which satisfies the conditions: ∀, x, y, z ∈ M

d(x, y) ≥ 0d(x, y) = d(y, x)d(x, y) = 0 ⇐⇒ x = yd(x, z) ≤ d(x, y) + d(y, z)

• A sequence xn in a metric space M converges to x ∈ M if

lim d(x, xn) = 0

Then we say xn → x or lim xn = x.

• A sequence xn in a metric space (M, d) is a Cauchy sequence if for anyε > 0 there is N ∈ Z+ such that for any n,m ≥ N

d(xn, xm) < ε

• Proposition. Every convergent sequence is Cauchy.

• A metric space is complete if all cahcy sequences in it converge to an ele-ment in it.

• A subset B of a metric space M is dense if for every element in M is a limitof a sequence in B, that is, for any x in M there is a sequence xn in B suchthat lim xn = x.

• A function f : M1 → M2 from a metric space M1 to a metric space M2

is continuous at x ∈ M1 if for any sequence xn in M1 converging to x thesequence f (xn) converges to f (x), that is,

lim f ()xn) = f (lim xn)



• An isometry is a bijection f : M1 → M2 from a metric space M1 to a metricspace M2 which preserves the metric, that is, for any x, y ∈ M1

d1(x, y) = d2( f (x), f (y))

Then M1 and M2 are called isometric.

• Every isometry is continuous.

• Theorem. Let M be an incomplete metric space. Then there is a completemetric space M isometric to M.

• Proof. Define M as equivalence classes of the equivalence relation of Cauchysequences in M, lim d(xn, yn) = 0. Define the natural metric on M by thelimit. Define the isometry by the limit of the Cauchy sequences.

• Topology of metric spaces. The notions of open balls, open sets, closedsets, neighborhoods, limit points, interior points, are naturally defined inany metric space.

• Theorem.

1. A set is closed if and only if it contains all its limit points.

2. A sequence xn converges to x if and only if every neighborhood of xcontails the tail of the sequence xn, n ≥ N for some N.

3. The set of interior points of a subset of a metric space is open.

4. The union of a set and all its limit points is closed.

5. A set is open if and only if it is a neighborhood of each of its points.

• A normed vector space is a vector space with a norm.

• Every normed vector space is a metric space with the induced metric

d(x, y) = ||x − y||

• A normed vector space is complete if it is complete as a metric space in theinduced metric.



• Theorem. The space C([0, 1]) of continuous functions on [0, 1] with themetric

d1( f , g) =∑

x∈[0,1]

| f (x) − g(x)|

is complete.

• A Banach space is a complete normed vector space.

• Example. The space C([0, 1]) with the norm

|| f || = supx∈[0,1]

| f (x)|

is Banach.

• An inner product space (pre-Hilbert space) is a vector space with an innerproduct.

• A Hilbert space is a complete inner product space.

• A finite set xiNi=1 is orthonormal if

(xi, x j) = δi j

• Let x, y be vectors in a vector space and xiNi=1 be an orthonormal set. Then:

• Pythagorean Theorem.

||x||2 =

N∑i=1

|(x, xi)|2 + ||x −N∑

i=1

(xi, x)xi||2

• Bessel Inequality.

||x||2 ≥N∑

i=1

|(x, xi)|2

• Schwarz Inequality.|(x, y)| ≤ ||x|| ||y||

• Parallelogram Law.

||x + y||2 + ||x − y||2 = 2||x||2 + 2||y||2


4.12. GENERALIZED FUNCTIONS 125

• Theorem. Every inner product space is a normed vector space with thenatural norm and a metric space with the induced metric.

• A linear transformation U : H1 → H2 from a Hilbert space H1 onto Hilbertspace H2 is unitary if for any x, y ∈ H1

(x, y) = (Ux,Uy)

• Two Hilbert spaces H1 and H2 are isomorphic if there is a unitary transfor-mation U : H1 → H2.

4.12 Generalized Functions•

4.13 Orthogonal Polynomials•

4.14 Fourier Analysis•


Chapter 5

Complex Analysis

5.1 Complex calculus•

5.2 Calculus of Residues•

5.3 Advanced Topics•

127

128 CHAPTER 5. COMPLEX ANALYSIS


Chapter 6

Operators on Hilbert Spaces

6.1 Operator Theory•

6.2 Integral Equations•

6.3 Sturm-Liouville Systems•

129

130 Bibliography


Bibliography

[1]

131

132 BIBLIOGRAPHY


Answers to Exercises

133

134 Answers To Exercises


Notation

135

136 Notation


Lecture Notes Methods of Mathematical Physics MATH 535

Documents

Transcript of Lecture Notes Methods of Mathematical Physics MATH 535