I .

IEEE PRESS Series on Electromagnetic Waves, I

The IEEE PRESS Series on Electromagnetic Waves consists of new titl~sas well as reprints and revisions of recognized classics that maintain

long-term archival significance in electromagnetic waves and applicatior/s.i

Series EditorDonald O. Dudley

University of ArizonaAdvisory BoardRobert E. Collin

Case Western UniversityAkira Ishimaru

....

University of WashingtonAssociate Editors

Electromagnetic Theory, Scattering, and DiffractionEhud Heyman

Tel-Aviv UniversityDifferential Equation Methods

Andreas C. CangellarisUniversIty of Arizona

Integral Equation MethodsDonald R. Wilton

Universi,ty of Houston

Antennas, Propagation, and MicrowavesDavid R. Jackson

University of Houston

Series Books Published

Collin, R. E., Field Theory ofGuided Waves, 2nd ed., 1991 ,Dudley, D.O., Mathematical Foundations for Electromagnetic Theory, 1994

Elliott, R. S., Electromagnetic.~: History, Theory, and Applications, 1993Felsen, L. B., and Marcuvitz, N., Radiation and Scattering ofWaves, 1994

Harrington, R. E, Field Computation by Moment Methods, 1993Tai, C. T., Dyadic Green Functions in Electromagnetic Theory, 2nd ed., 1993

Tai, C. T., Generalized Vector and Dyadic Analysis:Applied Mathematics in Field Theory, 1991

Mathematical Foundationsfor Electromagnetic Theory

Donald G. DudleyUniversity of Arizona, Tucson

IEEE PRESS Series onElectromagnetic WavesDonald G. Dudley, Series Editor

IEEEPRESS

I

lEEE Antenrias ancl Propagation Society, Sponsor

,IEEE PRESS445 Hoes Lane, PO Box 1331Piscataway, NJ 08855-1331

1994 Editorial BoardWilliam Perkins, Editor ill Chief

R. S. BlicqM.EdenD. M. EtterG. F. HoffnagleR. F. Hoyt

J. D: IrwinS. V, KartalopoulosP. LaplanteA. LLaubM. Lightner

J. M. F. MouraI. PcdenE. Sanchez-SincncioL. ShawD. J. Wells Dedicated to Professor Robert S. Elliott

Dudley R. Kay, Director ofRook PuhlishingCarrie Briggs, Administrative Assistant

Lisa Smey-Mizrahi. Review CoordinatorValerie Zaborski, Production Editor

IEEE Antennas and Propagation Society, SpollsorAPS Liaison to IEEE PRESS

Robert J. MaillouxRome Laboratory/ERI

".

I, I

This book may be purchased at a discount from the publisher when ordered in bullequantities. For more information contact:

IEEE PRESS MarketingAttn: Special SalesP.O. Box 1331445 Hoes LanePiscataway, NJ 08855-1331

1994 by the Institute of Electrical and EI,ectronics Engineers, Inc.345 East 47th Street, New York, NY 10017-2394

All rights reserved. No part of this book may be reproduced in any form,nor may it be stored in a retrieval system o~ transmitted in any fonn,without written permission from the publisher.

Printed in the United Stales of America10 9 8 7 6 5 4 3 2 I

ISBN 0-7803-1022-5IEEE Order Number: PC03715,Library of Congress Cataloging.in.Publication DataDudley, Donal

..

Contents

Preface ix

1 Linear Analysis 11.1 Introduction I1.2 Linear Space I1.3 Inner Product Space 71.4 Normed Linear Space 101.5 Hilbert Space 151.6 Best Approximation 191.7 Operators in Hilbert Space 241.8 Method of Moments 33A.I Appendix-Proof of Projection Theorem 36

Problems 38References 43

2 The Green's Function Method 452.1 Introduction 452.2 Delta Function 452.3 Sturm-Liouville Operator Theory 502.4 Sturm-Liouville Problem of the First Kind 532.5 Sturm-Liouville Problem of the Second Kind 68'2.6 Stun:n-LiouviIle Problem of the Third Kind 77

Problems 94References 97

I:1i

II

"

iI11 'Ii;

:1,

viii Contents

3 The Spectral Representation Method 993.1 Introduction 993.2 Eigenfunctions and Eigenvalues 993.3 Spectral Representations for SLP1 and SLP2 1063.4 Spectral Representations for SLP3 1113.5 Green's Functions and Spectral Representations 134

Problems 135 IReferences 138

4 Electromagnetic Sources 1394.1 "Introduction 1394.2 Delta Function Transformations 1394.3 Time-Harmonic Representations 1434.4 The Electromagnetic Model 1444.5 The Sheet Current Source 1474.6 The Line Source 153.4.7 The Cylindrical Shell Source 1664.8 The Ring Source 1684.9 Th'e Point Source 172

Problems 178References 179

5 Electromagnetic Boundary Value Problems 1815.1 Introduction 1815.2 SLP1 Extension to Three Dimensions 1825.3 SLPI in Two Dimensions 1915.4 SLP2 and SLP3 Extension to Three Dimensions 1945.5 The Parallel Plate Waveguide 1985.6 Iris in Parallel Plate Waveguide 2065.7 Aperture Diffraction '2165.8 Scattering by a Perfectly Conducting Cylinder 2265.9 Perfectly Conducting Circular Cylinder 2335.10 Dyadic Green's Functions 242

Problems 242References 244

Index 246

Preface

This book is written for the serious student ofelectromagnetic theory.It is a principal product of my experience over the past 25 years interactingwith graduate students in electromagnetics and applied mathematics at theUniversity of Arizona.

A large volume of literature has appeared since the latter days ofWorld War II, written by researchers expanding the basic principles ofelectromagnetic theory and applying the electromagnetic model to manyimportant practical problems. In spite of widespread and continuing in-terest in electromagnetics, the underlying mathematical principles usedfreely throughout graduate electromagnetic texts have not been systemati-cally presented in the texts as preambles. This is in contrast to the situationregarding undergraduate electromagnetic texts, most of which contain pre-liminary treatments of fundamental applied mathematical principles, suchas vector analysis, complex arithmetic, and phasors. It is my belief thatthere should be a graduate electromagnetic theory text with linear spac'es,Green's functions, and spectral expansions as mathematical cornerstones.Such a text should allow the reader access to the mathematics and the elec-tromagnetic applications without the necessity for consulting a wide rangeof mathematical books written at a variety of levels. This book is an effortto bring the power of the mathematics to bear on electromagnetic problemsin a single teXt. '

Since the mastery of the foundations for electromagnetics providedin this book can involve a considerable investment of time, I should liketo indicate some of the potential rewards. When the student first begins a

[6], Ishimaru [7], or Balanis [8]. I have therefore felt no necessity to includea chapter on Maxwell's equations or a chapter on analytic function theory,presupposing reader familiarity.

Chapter I is an introduction to modem linear analysis. It begins withthe notion of a linear space. Structure is added by the introduction of theinner product and the norm. With the addition of suitable convergencecriteria, the space becomes a Hilbert space. Included in the discussionof Hilbert space are the concepts of best approximation and projection.The chapter concludes with a discussion of operators in Hilbert space.Emphasis is placed on the matrix representation of operations, a conceptthat leads naturally to the Method of Moments, one of the most populartechniques for the numerical solution to integral equations occurring inelectromagnetic boundary value problems.

Chapter 2 covers Green's functions for linear, ordinary, differentialoperators of second order. The chapter begins with a discussion of thedelta functi9n. The Sturm-Liouville operator is introduced and discussedfor three cases, which we title SLPl, SLP2, and SLP3. A clear distinctionis made between self-adjoint and nonself-adjoint operators. In addition, theconcepts of:limit point and limit circle cases are introduced and exploredthrough exahlples applicable to electromagnetic problems. .

Chapter 3 introduces the spectral representation of the delta function.The theory is applied by example to various operators and boundary Con-ditions. Included are important representations associated with the limitpoint and limit circle cases introduced in the previous chapter. A wide va-riety of spectral representations are presented in a form suitable for use insolving electromagnetic boundary value problems in multiple dimensions.These representations are augmented by further examples in the Problems.

Chapter 4 contains a discussion of fundamental electromagneticsources represented by delta functions. The sources are analyzed us-ing spectral representations and Green'8 functions in Cartesian, cylindri-cal, and spherical conditions. A variety of useful alternative representa-tions emerge. Included are sheet sources, line sources, ring sources, shellsources, and point sources.

In Chapter 5, the ideas developed in the previous chapters are appliedto a sample of electromagnetic boundary value problems. No attemptis made to produce an exhaustive collection. Rather, the purpose of thechapter is to demonstrate the power of the structure developed in the firstthree chapters. Static problems included involve the rectangular box andrectangular cylinder. Dynamic problems include propagation in a parallelplate waveguide, scattering by an iris obstacle in a parallel plate waveguide,

, .

x lfu~study of electromagnetic theory at the graduate level, he/she is 'con'frontedwith a large array of series expansions and transforms with which to re-duce the differential equations, and boundary conditions in a wide 'varietyof canonical problems in Cartesian, cylindrical, and spherical coordin,ates.Often, it seems to the student that experience is the only way to determinespecifically which expansions or transforms to use in a given problem. Inaddition, convergence properties seem quite mysterious. These isstles canbe approached on a firm mathematical base through the foundations pro-vided in this book. Indeed, the reader will find that different diff~rentialoperators with their associated boundary conditions lead to specificexpan-sions aRd transforms that are "rtatural" in a concrete mathematical sense forthe problem being considered., My experience with graduate students hasbeen that mastery of the foundations allows them to appreciate why certainexpansions and transforms ar~ used in the study of canonical problems.Then, what is potentially more important, the foundations allow them tobegin the more difficult task of formulating and solving problems on theirown.

I first became interested ih Green's functions and spectral representa-tions during my graduate studies at UCLA in the I960s. I was particularlyinfluenced by the treatment of the spectral representations of the deltafunction by Bernard Friedman [l], whose book at that time formed thecornerstone of the Applied Mathematics Program at UCLA in the Coilegeof Engineering. Subsequently, examples of spectral representations beganappearing in texts on electromagnetic theory, such as [2]-[4], and, morerecently, [5]. However, no te~t specifically devoted to Green's functionsand spectral expansions and their application to electromagnetic problemshas been forthcoming. , .

The material in this book forms a two-semester sequence for graduatestudents at the University of Arizona. The first three chapters contain the;: I

mathematical foundations, and are covered in a course offered every yearto electrical engineering and applied mathematics graduate students witha wide range of interests. Indeed, the first three chapters in th~s bookcould be studied by applied mathematicians, physicists, and engineerswithno particular interest in the electromagnetic applications. The fourth andfifth chapters are concerned with the electromagnetics, and are ~overedin a course on advanced electromagnetic theory, offered biennially. Inthis book, I have presumed that the reader has a working knowledge ofcomplex variables. In addition, in the last two chapters, I have assumedthat the reader has studied an introductory treatment of electromagrietics atthe graduate level, as can he found, for example. in the texts hy Harhngton

!

Preface xi

xii Preface Chap. 1 References xiii

aperture diffraction, and scattering by a conducting cylinder. Emphasis hasbeen placed on the power of alternative representations by including usefulalternatives in the examples on :the parallel plate waveguide and scatteringfrom a conducting circular cylinder. .

My graduate students over the past 25 years have had a major influ-ence on this book. All have contributed through classroom and indivjdualdiscussions. Many too numerous to mention have made suggestions andcorrections in early drafts. Specifically, I should like to acknowledge somespecial help. In the early 1980s; K. A. Nabulsi and Amal Nabulsi painstak-ingly typed a portion of my handwritten class notes. These typed noteswere produced before the adv~nt of modem computational word proces-sors, and formed the basis for my subsequent writing of Chapters 1-3 ofthis book. Dr. Nabulsi, now a Professor in Saudi Arabia, sent me ai giftedstudent, Muntasir Sheikh, for doctoral training. Mr. Sheikh has cr~tiGallyread the entire book manuscript and offered suggestions and corrd:tibns.In addition, Charles Trantanella, Michael Pasik, and Jacob Adopley havecarefully read portions of the manuscript.

In the mid-1970s, I had the good fortune to be a part of the creationof the now greatly successful Program in Applied Mathematics at the Uni-versity of Arizona. W. A. Johnson was my first student to graduate dlrough

, .

the program. Because of him, I became acquainted with three professorsin the Department of Mathematics, C. L. DeVito, W. M. Greenlee, ~nd W.G. Faris. These four mathematicians have had a lasting influence pn theway I have come to consider many of the mathematical issues involved inelectromagnetic theory. !

Among my colleagues, there are several who have had a ~1arkedinfluence on this book. R. E. Kleinman, University of Delaware, h~s con-sistently encouraged me to pursue my mathematical interests appiied toelectromagnetic theory. L. B.Felsen, Polytechnic University, haS, influ-enced me in many ways, scientifically and personally. In addition, hiscomments concerning modem research applications led me to some im-portant additions in Chapter 5: K. J. Langenberg, University of Kassel,has read in detail the first three chapters and offered important advice andcriticism. R. W. Ziolkowski, University of Arizona, has taught a tourseusing the material contained in Chapters 1-3 and offered many suggestionsand corrections. I. Stakgold, University of Delaware, made me aware ofthe recent mathematicalliteratJre on limit point and limit circle problems.

Many reviewers, anonymbus and known, have made comments thathave led me to make changes ~nd additions. I would particularly like tomention Ehud Heyman, Tel Aviv University. whose comments concerning

I

alternative representations led me to strengthen this material in Chapter 5.I would also like to thank Dudley Kay and the staff at IEEE Press whosecompetence and diligence have been instrumental in the production phaseof this book project.

With Chalmers M. Butler, Clemson University, a distinguished ed-ucator and cherished friend, I have had the good fortune, to have a 20-year running' discussion concerning methods of teaching electromagneticsto graduate students. Part of the fun has been that we have not alwaysagreed. How:ever, one issue upon which there has been no disagreement isthe importance of presenting electromagnetics to students in a structurkllyorganized manner, stressing the common links between wide ranges ofproblems. I have drawn strength, satisfaction, and pleasure from our asso-ciation.

My family has always seemed to understand my many interests, andthis book has been a major one for more years than I should like to recall. Itis with love and affection that I acknowledge my wife, Marjorie A. Dudley;my children, Donald L. Dudley and Susan D. Benson; and the memory ofmy former wife, Marjorie M. Dudley. Love truly does "make the world go'round."

Finally, it is with gratitude that I dedicate this book to my teacher,mentor, and friend, Robert S. Elliott, University of California at Los Ange-les, a consummate scholar without whom none of this would have occurred.

REFERENCES[I] Friedman, B. (1956), Principles and Techniques of Applied

Mathematics. New York: Wiley.[2] Jackson, J. D. (1962), Classical Electrodynamics.' New York: Wi-

ley.[3] Collin, R. E. (1960, 2nd edition 1991), Field Theory of Guided

Waves. New York: IEEE Press.[4] Felsen, L. B., and N. Marcuvitz (1973), Radiation and Scattering of

Waves. Englewood Cliffs, NJ: Prentice-Hall.[5] Ishimaru, A. (1991), Electromagnetic Wave Propagation, Radiation,

and Scattering. Englewood Cliffs, NJ: Prentice-Hall.[6] Harrington, R. F. (1961, reissued 1987), Time-Harmonic Electromag-

netic Fields. New York: McGraw-Hill.[7] Op. cit. Ishimaru.[8] Balanis, C. A. (1989), Advanced EnRineerinR ElectromaRlletics.

New York: Wiley.

1Linear Analysis

1.1 INTRODUCTIONFundamental to the study of many of the differential equations describingphysical processes in applied physics and engineering is linear analysis.Linear analysis can be elegantly and logically placed in a mathematicalstructure called a linear space.

We begin this chapter with the definition of a linear space. We thenbegin to add structure to the linear space by introducing ,the concepts ofinner product and norm. Our study leads us to Hilbert space and, finally,to linear operators within Hilbert space. The characteristics of these oper-ators are basic to the ensuing development of the differential operators anddifferential equations found in electromagnetic theory.

Throughout this chapter, we shall be developing notions concerningvectors in a linear space. These ideas make use of both the real and complexnumber systems. A knowledge of the axioms and theorems governing realand complex numbers will be assumed in what follows. We shall use thisinformation freely in the proofs involving vectors.

1.2 LINEAR SPACE

Let a, b, c, be elements of a set S. These elements are called vectors.Let a, {J, he elements of the field of numhers F. In particular, let Rand

2 Linear Analysis . Chap. 1 Sec. 1.2 Linear Space 3

II. Rules for multiplication of vectors in S by elements of F:..

( 1.6)

(1.7)

(1.8)

(1.9)

(f + g)(~) = f(~) + g(~)= g(~) + f(~)= (g + f)(~)

(f + g)(~) = f(~) + g(~)(Cif)(~) = af(~)

is called a !inear comhination of the vectors Xk .

If the only way to satisfy (1.9) is ak = 0, k = 1,2, ... , n, then the elementsXk are linearly independent. The sum

We leave the completion of the proof for Problem 1.3.

In ordinary vector analysis over two or three spatial coordinates, weare often concerned with vectors that are parallel (collinear). This conceptcan be generalized in an abstract linear space. Let XI, X2, ... ,Xn be ele-ments of a set of vectors in S. The vectors are linearly dependent if thereexist ak E F, k = I, 2, ... , n, not all zero, such that

EXAMPLE 1.3 Consider C(O,I), the space of real-valued functions continuouson the interval (0, 1). For f and g in C(O, I) and a E R, we define addition andmultiplication as follows:

for all ~ E (0, 1). If we assume prior establishment of the rules for addition of tworeal-valued functions and multiplication of a real-valued function by a real scalar,it is easy to establish that C(O, 1) is a linear space by showing that the rules in I andII are satisfied. For example, for addition rule d,

EXAMPLE 1.2 Consider unitary space CIt. Vectors in the space are givenby (1.1) and (1.2), where Cik and f3ko k = I, 2, ... ,11 are in C. Addition andmultiplication are defined by (1.3) and (1.4) where a E C. Proof that CIt is a linearspace follows the same lines as in Example 1.1. Note that C is a linear space,where we make the identification C = C I .

(1.3)

(IA)

(1.5)

Cia ;::: Ci(CiI' , Cin):= (CiCiI' , CiCi,,)

a + b = (Cil + f31, ... , Ci" + f311)= ,({31 + CiJ, . , f311 + Cin )=b+a

a + b = (Cil,, an) + (f3I,., f3n)= (al + f3J' ... ,Cin + f3n)

I

C be the field of real and complex numbers, respectively. The set S is alinear space if the following rules for addition and multiplication apply:

,

a. a({3a) = (a{3)ab. la = ac. a(a+b)=aa+ab:d. (a + {3)a = aa + {3a

I. Rules for addition among vectors in S:

a. (a + b) + c = a + (lJ + c) Ib. There exists a zero vector 0 such that a + 0 = 0 + a = a.ic. For every a E S, there exists -a E S such that a + (-a)

(-a) +a = O.d. a +b = b+a

We leave the satisfaction of the remainder of the rules in this example for Problem1.2. Note that R is also a linear space, where we make the identification R = R I.

a = (CiI,CiZ, ,Cin) ;(1.1)b == (f3I, f3z, , f3n) , (1.2)

Iwhere Cik and fh, the components of vectors a and h, are in R, k =, 1,2, ... , n.Define addition and multiplication as follows: '

EXAMPLE 1.1 Consider Euclidean space Rn Define vectors a and bin Rn asfollows:

where a E R. If we assume prior establishment of rules for addition and I)1ultipli-cation in the field of real numbers, it is easy to show that Rn is a linear space. Wemust show that the rules in I and II are satisfied. For example, for addition rule d,

, .

4 Linear Analysis : Chap. I Sec. 1.2 Linear Space 5

EXAMPLE 1.4 In Rz, let Xl (l, 3), Xz = (2,6). We test Xl and Xz f?r lineardependencd. We fonn

from which we conclude that

:at+ 2az=0

3at + 6a2 = 0

In an abstract linear space S, it would be helpful to have a measureof how many and what sort of vectors describe the space. A linear spaceS has dimension n if it possesses a set of n independent vectors and ifevery set of n + 1 vectors is dependent. If for every positive integer k wecan find k independent vectors in S, then S has infinite dimension. Theset Xl, X2, ... ,Xn is a basis for S provided that the vectors in the set are "linearly independent, and provided that every XES can be written as alinear combination of the Xb viz.

where ak E R. The fb defined above, fonn an orthonormal set on ~ E, (0, I).That is,

EXAMPLE 1.5 InC(O,I), let a set of vectors be defined by fk(~) = .j2 sin kTr~,k = 1,2, ... ,n. We test the vectdrs !k for linear dependence. We fonn

I

These two equations are consistent and yield al = -2a2. Certainly, at ='az = 0satisfies,this equation, but there is 'also an infinite number of nonzero possibilities.The vectors are therefore linearlY,dependent. Indeed, the reader can easily makea sketch to show that XI and Xz are collinear.

(1.13)

(1.12)

(1.14)n

0= L(ak - 13k)xkk=l

The representation with respect to a given basis is unique. If it were not,then, in addition to the representation in (1.12), there would exist 13k E F,k = 1, 2, ... , n such that

n

X = L13kxkk=l

Subtraction of (1.13) from (1.12) yields(1.10)n

L~kJ2sinkTr~ = 0k=l

Since the Xk are linearly independent, we must have, (1.11)

ak - 13k = 0, k = 1,2, ... , n (1.15)

which proves uniqueness with respect to a given basis. 'Finally, if S isn-dimensional, any set of n linearly independent vectors Xl, X2, .. I , Xnforms a basis. Indeed, let XES. By the definition of dimension, the setX, Xl, X2, ... , Xn is linearly dependent, and therefore, I

Multiplication of both sides of (1.1 0) by .j2 sin mn ~, m = I, 2, ... , n and inte-gration over (0, I) give, with the help of (l.11), am = 0, m = I, 2, ... ,n. Thevectors fk are therefore linearly independent.

In Example 1.5, we note that the elements !k are finite in number.

We recognize them as a finite :subset of the countably infinite nu~ber ofelements in the Fourier sine series fk, k = 1, 2, .... A question arisesconcerning the linear independence of sets containing a countably infinitenumber of: vectors. Let Xl, X2: . .. be an infinite set of vectors in S. Thevectors are linearly independent if every finite subset of the vedtors islinearly independent. In Example 1.5, this requirement is realized, so thatthe infinite set of elements present in the Fourier sine series is linearlyindependent.' I

n

ax + Lakxk = 0k=l

where we must have a i 0. Dividing by a gives

n (-ak )x=L - Xkk=l a

Therefore, the set Xl, X2, ... , X'I is a basis.

(1.16)

(1. 17)

6 Linear AnalysisIChap. I Sec. 1.3 Inner Product Space 7

EXAMPLE 1.6 Consider Euclidean space Rn We shall show that the vectorse\ = (1,0, ... , 0),e2 = (0,1,.;.,0), , en = (0,0, ... ,1) satisfy the tworequirements for a basis. First, the set el, , en is independent (Proble~ 1.8).Second, if Q'E Rn ,

a = (at, a2, , an)= a\ (I, 0, , 0) + a2 (0, 1, ... , 0) + ... + an (0, 0, ... , I)

Expression (1.22) is a homogeneous set of n linear equations in n + 1 unknowns.The set is underdetermined, and as a result, always has a nontrivial solution [IJ.There is therefore at least one nonzero coefficient among Ym, m = 1, 2, ... , n + 1.The result is that the arbitrary set ai, ... , an, all + I is linearly dependent and thedimension of R II is n.

(1.18) 1.3 INNER PRODUCT SPACE

where we must show that there exist Ym E R, m = 1, 2, ... , 11 + 1, not aJI zero,such that (1.19) is satisfied. We express each of the members of the arbitrary setas a linear combination of the basis vectors, viz.

Therefore, any vector in the space can be expressed as a linear combination ofthe ek. A special case of this result is obtained by considering Euclidean spaceR3 . The vectors el = (1, 0, 0), e2: = (0, 1, 0), e3 = (0, 0, 1) are a basis. Thesevectors are perhaps best known as the unit vectors associated with the Cartesiancoordinate system.

(1.23)(0, y) = Indeed, the result follows immediately if we substitute a = 0 in rule c above.

A linear space S is a complex inner product space if for every ordered pair(x, y) of vectors in S, there exists a unique scalar in C, symbolized (x, y),such that:

a. (x, y) = (y, x)b. (x +y, z) = (x, z) + (y, z)

! -

c. (ax, 'y) = a(x, y), a E Cd. (x, x) ::: 0, with equality if and only if x = 0

In a, the overbar indicates complex conjugate. Similar to the above is thereal inner product space, which we produce by eliminating the overbar ina and requiring in c that a be in R. For the remainder of this section, weshall assume the complex case. We leave the reader to make the necessaryspecialization to the real inner product.

EXAMPLE 1.8 We show from the definition of complex inner product spacethat

:(1.19)n+1

LYmam = 0m=1

EXAMPLE 1.7 It would be consistent with notation if the dimension of Rn were,in fact, n. We now show that both r~quirements for dimension n are satisfied. First,since we have established an n-term basis for Rn in Example 1.6, the space has aset of n independent vectors. Second, we must show that any set of n + I vedorsis dependent. Let aj, a2, ... , an, ~n+ I be an arbitrary set of 11 + 1 vectors in, Rn We form the expression

Substitution of (1.20) into (1.19) and interchanging the order of the summationsgives

EXAMPLE 1.9 Given the rules for the complex inner product in a-d, the fol-lowing result holds:

(1.24)(x, ay) = a(x, y)

:(1.20)I '

m = 1, 2, ... , n + 1

t (~Yma~ml) ek = 0Since the ek are linearly independent,

, .

n+l'" (ml 0L YmO'k ~ ,m=1

k = 1.2, ... , n

(1.21 )

i~(1.22)

Indeed,(x,ay) = (O'y,x)

= O'(y, x)= a (y, x)= a(x, y)

EXAMPLE 1.10 Given the rules: for the inner product space, we may shqw that

EXAMPLE 1.11 In the space C~, with a and b defined in Example 1.2, de'finean inner product by

Then, Cn is a complex inner product space. To prove this, we must show that rulesa-d for the complex inner product space are satisfied. For rule d, there are threeparts to prove. First, we show that the inner product (a, a) is nonnegative. Indeed,

Herein, we refer to (1.28) as the CSB inequality. To prov~ the CSB in-equality, we first note that for I(x, y) I = 0, there is nothing to prove. Wemay therefor~ assume y i= 0, with the result (y, y) i= 0, and ,define

(x, y)a=--(y, y) II,

.!iiilI'"i11~i

-IIiI

9

(1.29)

(x, y}(y, x)(y, y)

= a(y,x}= a(x, y}= laI 2 (y, y}

I(x, y}1 2(y, y)

Inner Product SpaceSec. 1.3

from which we have the result

With the help of rule d, we form

(1.25)

,0 .26)

Chap. 1Linear Analysis

n

(a, b) = 'L ak~k, k=l

n n('L akx~, y) = 'L adxk, y)k=] k=1

The proof is left for Problem 1.9.

8

rj

.j

)

n

(a, a) = 'L lakl2 :::: 0k=l

Second, we ~how that (a, a) = 0 implies that a = O. We have

O:s (x - ay, x - ay) = (x, x) + laI 2 (y, y} - a(x, y} - a(y, x}= (x,x) _ l(x,y}1 2

(y, y)".:

t

The set Zk, k = 1, 2, ... is an orthogonal set if, for all members of the set,

from which the result in 0.28) follows.Two concepts used throughout this book involve the notions of or-

thogonality and orthonormality. The concepts are generalizations of theideas introduced in Example 1.5. Two vectors x and yare orthogonal if

, n

0= (a, a) = 'L ladk=l

I

Since all the terms in the sum are nonnegative, ak = 0, k = 1,2, ... , 'n, andtherefore a = O. Third, we must show that a = 0 implies (a, a) = O. wb leavethis for the reader. We also leave the reader to demonstrate that rules a-c ror theinner product space are satisfied. !

(x, y) = 0 0.30)

: fl(j, g) = i f(~)g(~)d~

EXAMPLE 1.12 Let f and g be iwo vectors in C(a, (3). Define an inner productby

(Zi, Zj) = 0,

The set is an orthonormal set if

ii=j (1.31 )

(1.32)Then, C(a, (3) is a real inner product space. We leave the proof for Problem 1.10.

One of the most importantinequalities in linear analysis follows from

the basic rules for the complex inner product space. The Cauchy-Schwarz-Bunjakowsky inequality is given by

(1.33)

where

(I i = j

Oij = '0, i i= j

An orthogonal set is called proper if it does not contain the zero vector. Wecan show that a proper orthogonal set of vectors is linearly independent.Indeed, we form(1.:28)I(x, Y}I:s~~

10 Linear Analysis Chap. 1 Sec. 1.4 Normed Linear Space 11

Indeed,

IIx +YII 2 + IIx - YII 2 = (x + y, x + y) + (x - y, x' - y)= 2(x, x} + 2(y, y}

Taking the square root of both sides yields the result in c.

EXAMPLE 1.13 From the basic rules for the norm and the definition in (1.34),we can show that

(1.36)

Using the CSB inequality, we obtain

adZi, Zi} = from which'we conclude that ai= 0, i = 1,2, ... , n and the set is li!learlyindependent. Further, if the index n is arbitrary, the countably infinite setZb k = 1,2, ... , is linearly independent.

Using (1.25) and (1.31), we obtain

n

(LakZb Zi) = (0, Zi)k=)

II

LQHk =0k=)

Taking the inner product of both sides with Zi gives

1.4 NORMED LINEAR SPACEA linear space S is a normed linear space if, for every vector XES; thereis assigned a unique number IIx1l E R such that the following rules apply:

a. IIx II ::: 0, with equality if and only if x = 0b. lIaxli = lalllxll, a E Fc. IIx) + x211 ::::: IIx)1I + 1IX:211 (triangle inequality)

Although there are many possible definitions of norms, we use exclusivelythe norm induced by the inner product, defined by

from which the result in (1.36) folIows.

(1.37)n

Iiall = L lakl2k=1

Unitary space is therefore a normed linear space.

EXAMPLE 1.14 For unitary space en, with inner product defined by (1.26),the norm of a vector a in the space is easily found to be

(1.34)IIxll=~Using (1.34), we find that the c'SB inequality in (1.28) can be writt~n :

I(x, Y}! ::::: IIxlillyli (1.35)It is easy to show that the norm defined by (1.34) meets the re~uire

ments in rules a, b, and c above. We leave the reader to show that a: and bare satisfied. For c, for x and y in S, we have

IIx +Yll2 = (x +,y,x + y)= (x, x) + (y, y) + (x, y) + (y, x)= IIxf + IIYll2 + 2Re(x, y}

Since the real part ofa complex ~umber is less than orequal to its magnitude,IIx + )'11 2 ::::: IIxll 2 + 1I.v11 2 + 21(x, y}1

EXAMPLE 1.15 For the real linear space C(a, fJ), with inner product definedby (1.27), the norm of a vector f in the space is

(1.38)

The space C(a, fJ) is therefore a normed linear space.

One of the useful consequences of the normed linear space is that itprovides a measure of the "closeness" of one vector to another. We notefrom rule a that IIx - yll = 0 if and only if x = y. Therefore, closenesscan be indicated by the relation IIx - y II < E. This observation brings us tothe notion of convergence. Among the many forms of convergence, there

li '

I.

12 Linear Analysis Chap. I Sec. 1.4 Nonned Linear Space 13

where h is any vector in 5. To prove (1.40), it is sufficient to show tha,t

iilil'IIj1.,

(1.44)n

(a, b) = Ladhk=!

_ (ml (m) (ml)am - u j .CJ2 ... ,a"i

Let am, m = 1.2.... be a Cauchy sequence in Rn , wheret

We can show that convergence implies Cauchy convergence. Indeed, letx E 5 be defined as the limit of a sequence, as in (1.39). Then, by thetriangle inequality,

Since Xk -+ x, there exists a number N such that for n > NE

IIx - xnll ~ 2:and therefore, for min(m, n) > N,

IIxm - XII II ~ IIxl1l - x II + IIx - Xn II

which proves the assertion. Unfortunately, the converse is not always true.The interpretation is that it is possible for two members of the sequence tobecome arbitrarily close without the sequence itself approaching a limit inS. A nonned linear space is said to be complete if every Cauchy sequence inthe space convergcs to a vector in the space. The concept of completenessis an important one in what is to follow. Although it is beyond the scope ofthis book to include a detailed treatment, we shall give a brief discussion.

In real analysis, the space of rational numbers is defined [2] as thosenumbers that can be written as p / q, where p and q are integers. It is astandard exercise [3],[4] to produce a sequence of rational numbers that hasthe Cauchy property and yet fails to converge in the space. (We consideran example in Problem 1.15.) This incompleteness is caused by the factthat in between two rational numbers, no matter how close, is an infinitenumber of irrational numbers; often, a sequence of rationals can convergeto an irrational. The solution to this problem is a procedure due to Cantor[5] whereby the irrationals are appended to the rationals in such a mannerso as to produce a complete linear space called the space of real numbersR. We shall assume henceforth that R is complete, and direct the reader tothe literature in real analysis for details.

EXAMPLE 1.16 We can show that Euclidean space Rn is complete. For vectorsa and b in the space, defined by (1.1) and (1.2), we define an inner product by

(1.42)

(1.41 )

(1.39)

(l.40)

(1.43)

lim Xk = x(400

(Xk, h) -+ (x, h)

lim IIxl1l - XII II = 0n,.n--. 00

lim (Xk, h) = (lim :q, h)k---'>oo k---'>oo

(Xli; - x, h) -+ 0By the fonn of the CSB inequality in (1.35), we have

I(Xk -x,h}12 ~ Ilxk _x11 2 11h11 2But, since;n -+ X,

or

This relationship indicates that, given Xb the order of application of thelimit and the inner product with h can be interchanged. .

In 5, a sequence {Xk }~l converges in the Cauchy sense if; givenan E > 0, there exists a number N such that IIxm - Xn II < E whenevermin(m, n) > N. We write

Note that if Xk -+ x, IIx - xkll -+ O.Fundamental to studies of approximation of one vector by. another

vector, to be studied later in this chapter, is the notion of continuity of theinner prod~ct. We show that if {xd~l is a sequence in 5 converging tox E 5, then

IIxk -xlI-+ 0so that (1.41) is verified. We remark that another useful way of writing(1.40) is as follows:

are two fonns whose relationship is crucial to placing finn "boundaries"on the linear space. The type of boundary we seek is one that assurs thatthe limit of a sequence in the linear space also is contained in the space.

In a normed linear space 5, a sequence of vectors {xd~l con}'etgesto a vector x E 5 if, given an > 0, there exists a number N such thatIIx - Xk II < E whenever k > N. We write Xk -+ x or

, .

~ ,

I''''!

IJIt!ij!!

14 Linear Analysis ' Chap. I Sec. 1.5 Hilbert Space 15

1

lIam - apli = It [akmJ - aiPJrj2 ~ Ek=1 I

for min(m, p) > N. Since all the te~s in the sum are nonnegative, we must h~ve\aimJ - akpJI~ E, k = I, 2, ... , n i

for min(m, p) > N. Since the space of real numbers R is complete, tHen as!

which proves that the sequence is Cauchy. However, it is apparent that the sequenceIk converges to the Heaviside function H (~), defined by

lim 111m - lkil = 0m,k"""""'oo

(1.46)~OI0,H(~) = I,

2 (I I)111m - Ikll ~ max ;' k

k = 1,2, ... , nm -700,

Then,

and therefore am -7 a.

But, H (n rf- C(-I ,I). Therefore, the space C(a, f3) is not complete.

where k = 1,2, .... This sequence is continuous, and therefore is in the .linearspace C(-I,I). We show that this sequence is Cauchy. We form the differencebetween two members of the sequehce. For m > k,

EXAMPLE 1.17 We can show that the normed linear space C(a, f3) with normgiven by (1.38) is incomplete. We shall consider a well-known [6],[7] Cauchysequence that fails to converge to a vector in the space. Without loss of gene.rality,let (a, f3) be (-1,1). Consider the sequence

1

11k11m

y

t

(l.45)-I ~ ~ ~ 0

O~~~tt ~ ~ ~ I

We display the two sequence members /k and 1m in Fig. I-I. Note that .thedifference 1m - Ik is always less than unity. It follows that unity is an upper boundon (fm - Ik)2. We therefore must have

111m - Ikll 2 = [II (fm(~) - Ik(~))2d~ ~ ~Although the above result has been obtained for m > k, an interchange of /11 andk gives the general result '

A linear space is a Hilbert space if it is complete in the norm induced bythe inner product. Therefore, in any Hilbert space, Cauchy convergenceimplies convergence. From Example 1.16, Euclidean space Rn is completein the norm induced by the inner product in (1.44). Therefore, Rn is aHilbert space. In a similar manner, it can be shown that unitary space ellis complete. However, from Example 1.17, C(a, fJ) is incomplete. In amanner similar to the completion of the space of rational numbers, Ccan

1m - Ik =

0,

(m - k)~,(l-kn0,:

-I ~ ~ ~ 0

O~~~~..!.. < t

16 Linear Analysis ~hap. 1 Sec. 1.5 Hilbert Space 17

;0 .47)

This process continues until the final member of the sequence is producedby .

j1

'1,',il1

(1.50)

(1.51 )

(1.52)

(1.54)

(1.53)

( 1.55)Znen =--IIZn II

n-1Zn = X n - L(xn , edek

k=J

Z3e3= --

IIz311

and

The next member of the sequence is generated by

Z2e2= --

IIz211For the third member, we form

and

and

be completed. The result is 2(ex, (3), the Hilbert space of real functionsf (~) square integrable on the i~terval (ex, (3), viz.

f3i j2(~)d~ < 00In (1.47), the integration is to be understood in the Lebesgue sen~e [8].Although the Lebesgue theory is essential to the understanding of the proofof completeness, in this book sJch proofs will be omitted. For discussionsof the issues involved, the reader is directed to [9] ,[ 10].

In linear analysis, we are often concerned with subsets of vectors ina linear~pace. One such subset is called a linear manifold. If S is a linearspace and ex, f3 E F, then M is a linear manifold in S, provided that exx + f3yare in M whenever x and y are in M. It is easy to show that M is a linearspace. The proof is left for Problem 1.16.

EXAMPLE 1.18 In R2, the set of all vectors in the first quadrant is not a linearmanifold. Indeed, for x and y in the first quadrant, it is easy to find a, fJ E R suchthat ax + f3y is not in the first quadrant.

We leave the reader to show that the sequence {el, , en} possesses theorthonormal property. In addition, each ek, k = I, , n is a linear com-bination of XI, ... ,Xk. We conclude that any linear combination

The results in Example 1.19 raise an interesting issue. Can th~ same

linear manifold be generated by more than one sequence of vector~? Weshall show that indeed this is the case. We consider the Gram-Schmidtorthogonaiization process. Let {Xl, ... , xn } be a linearly independent se-quence of vectors generating the linear manifold M c H.. The Gram-Schmidt process is a constructive procedure for generating an orthonormalsequence leI, ... , en} from theindependent sequence. To begin, let

iZI = XI :(1.48)

I'

andZI

'el =--. II z. Iii

1.49)

is also a linear combinationn

Lf3kxkk=1

The original sequence and the orthornormal sequence obtained from ittherefore generate the same linear manifold.

EXAMPLE 1.20 Given the sequence {I, r, r 2, ... } E 2 (-I, I), we use theGram-Schmidt procedure to produce an orthonormal sequence. Indeed, we definean inner product by

(f. ~) == [~f(r)g(r)dr

19

(1.58)

(1.59)

k=I,2, ... ,m

em = x - Xmm

=X - L(x, Zk)Zkk=1

Besl ApproximationSec. 1.6

1-l, xk --* X E 1-l. But M is closed, and therefore x E M. We concludethat M is a Hilbert space.

Within the stru:cture of the Hilbert space, it is possible to generalize th~concepts of approximation of vectors and functions. Let x be a vector in aHilbert space 'H and let {ZklZ~1 be an orthonormal set in 1-l. We form thesum

m

Xm = L O'kZk (1.56)k=1

This sum generates a linear manifold M c 1-l. Different members of thelinear manifold are produced by assigning various values to the sequenceof coefficients {O'k }Z~\. We should like to determine what choice of coeffi-cients results in X m being the "best" approximation to x. Specifically, let usmake Xm "close" to x by adjusting the coefficients to minimize IIx - xmll.We expand the square of the norm as follows:

Ilx -xm 11 2 = (x -Xm,X -xm)= (x,x) + (xm,xm) - (xm,x) - (x,xm)

1.6 BEST APPROXIMATION

m m m

= IIxI1 2 + L lad - LO'dx , Zk) - LO'k(X, Zk)k=1 k=1 k=\ '

Completing the square, we obtainm m

Ilx - xm l1 2 = IIxI1 2 + L (O'k - (x, zd) (O'k - (x, Zk) - L !(x, zdl 2k=\ k=1

(1.57)Since the sum in (1.57) containing the coefficients O'k is nonnegative, thenorm-squared (and hence the norm) is minimized by the choice

Expression (1.58) defines the Fourier coefficients associated with orthonor-mal expansions. Note that once we have made the selection given in (1.58),we can define an error vector em by

Chap. I

, Po(r) = 1

ZI('r) = 1el(r) ~ 1/11111 = l/h

Z2(r) = r .....c (r, l/h}(l/h) = re2(r) = j3fi rZ3(~) = r 2 - 1/3

e3(r) = J4S/8 (r 2 - 1/3)

Linear Analysis

, IWe next discuss a characteristic associated with linear manifolds that

plays a central role in approximation theory. A linear manifold M IS saidto be closed if it contains the limits of all sequences that can be constructedfrom the members of M. It is easy to demonstrate that not all' linearmanifolds are closed. For example, the space C(O', (3) is a linear m~nifoldsince a linear combination of two continuous functions is a continuousfunction. However, in Example 1.17, we have given a sequence of vectorsin Cthat fails to converge to a vector in C. The linear manifold C is thereforenot closed. I

An interesting result occUrs if a closed linear manifold is containedin a Hilbert space. Specifically~ if 1-l is a Hilbert space and M is a closedlinear manifold in 1-l, then M is a Hilbert space. Indeed, let {Xk }~1 be aCauchy sequence in M. Then; since M is contained in the Hilbert space

,PI (r) = r, 1 2

P2(i)=2:(3r -1)1 3P3(r) = 2:(Sr - 3r)

1 4 2 3P4(r) = -(3Sr - 30r + )8

1Ps(r) = -(63r 5 -70r3 + lSr)

8The Legendre functions, orthogonal but not orthonormal, are constructed in sucha way that Pn(l) = l.

This process continues for as mahy terms in the orthonormal sequence as wewish to calculate. We remark that the members of the sequence so produced areproportional to the orthogonal sequence of Legendre polynomials, whose fi~stfewmembers an:

18

Then,

Taking the inner product with any member Zj of the orthonormal sequ~nce,we find that I

Let {.j2 sin krr~ J~I be an orthonormal set generating a linear manifold M c H.Then, by (1.56) and (1.58), the "best" approximating function fm E M to f(~) isgiven by :

21Best ApproximationSec. 1.6

(a - fj, b) = 0

b = (131.0)

so that e E Ml., which is therefore closed. The closed linear manifoldM J..IS called the orthogonal complement to M.

We now can produce a result called the Projection Theorem. Let Xbe any vector in the Hilbert space 'H., and let M C 'H. be a closed linearmanifold. Then, there is a unique vector Yo E M c 'H. closest to x in thesense that Ilx - Yo II :::: Ilx - y II for all y in M. Furthermore, the necessary ,and sufficient condition that Yo is the unique minimizing vector is thate = x - Yo is in M J... The proof of this important theorem is deferred toAppendix A.l at the end of the chapter. The vector Yo is called the projectionof x onto M. The vector e is called the projection of x onto MJ... Theideas inherent to the projection theorem have a well-known interpretationin two- and three-dimensional vector spaces, as in the following example.

EXAMPLE 1.22 Let a = (al. (2) be any vector in R2. Let M be the set of allvectors b in R2 with second component equal to zero, viz.

Indeed, let (ek}~l be a sequence in MJ.. converging to a vector e in 'H..~h~ ,

(ek, x) = 0for all X EM. But, by continuity of the inner product,

lim (ek, x) = (e, x) = 0k->oo

Since linear combinations of vectors in M are also in M, the set M is a linearmanifold. In addition, the manifold is closed. Indeed, let b1, b2, be a sequencein M converging to b E H, where

bk = (13lkJ , 0)Since bk ---+ bE H, 13ikJ ---+ 131. Therefore, bE M. By the projection theorem,among all vectors b E M, the vector fj closest to a can be obtained from

wherefj = (~I, 0)

Solving this equation, employing the usual definition of inner product for R2, weobtain

(1.65)

I

(1.64)

(1.63)

d.62)!

Chap. 1I

Linear Analysis

j = 1,2, ... ,m d.60)

X = XIII + em

(j, g) =:. t f(~)g(Od~,Jo

m

fm(~) = :I>k.j2sinbr~'k=1

ak = fal f(17)./isinbr17d17which is the classical Fourier result.

where

b. The error vector em is orthogonal to the approximation vector XIII', I

EXAMPLE 1.21 We consider the Fourier sine series in Example 1.5. Let f(~) E2(0, 1) with inner product

The above results for the approximation of a vector X E 'H. by a vectorXm E M C 11 can be generalized. We shall need the concept of a manifqldthat is orthogonal to a given man'ifold. If M is a linear manifold, theh thevector e E 'H. is a member of a set MJ.. if it is orthogonal to every vector inM. The set .MJ.. is a linear manifold since linear combinations of vectorsin MJ.. are also orthogonal to vectors in M. In fact, MJ.. is also closed.

We summarize these results as follows:

a. The vector x E 'H. has been decomposed into a vector XIII E J0c'H. plus an error vector em, viz. '

m

(em, Zj) = (x, Zj) - I)x, Zk)(Zk, Zj) = 0,k=l '

Since Xm is a linear combination of members of the sequence {Zj };~;, wemust have

(em, XIII) = 0 (1.61)i

20

j .

22 Linear Analysis ~hap. I Sec. 1.6 Best Approximation 23

Substituting (1.66) and rearranging, we have, .

Since f31 is arbitrary, ~ I = al. This result can be visualized (Fig. 1-2) by drawingthe vector a and noting that the vector b lies along the x-axis. The "best" b is thenobtained by dropping a perpendicular from the tip of a to the x-axis. The r~sult isa vector b, called the projection ora onto the x-axis. Note that the error vbctor eis such that it is orthogonal to any ~ector along the x-axis and that a = b 11- e, asrequired by the projection theorem, I

ML (Xj (Yj, yd = (y, Yk),j=l

We write (1.68) in matrix form as follows:

k = 1,2, ... , M (1.68)

y

t....

r(Yl,Yl)(Yl, Y2)

(Yl,YM)

~~::~~~jr~~j r~~:~~~j: : - : (1.69). . .

(YM, YM) (XM (y, YM)

All possible suc~ linear combinations form a closed linear manifold so that theprojection theorem applies. Comparing (1.66), we identify

can have at most N - 1 roots. Therefore, the only solution valid for all r E (0, I)is.B" = 0, n = 1,2, ... , N. We wish to approximate fer) by

(1.70)

(1.71)

N

fer) = Lan rn- tn=l

N

L.B"rn- 1 =0n=1

Inversion of this matrix yields the coefficients (Xj, j = 1, 2, ... , M. Thesecoefficients then determine yin (1.66). The square matrix on the left-handside of (1.69) is the transpose of a matrix called the Gram matrix. Inaddition to its appearance in best approximation, it also finds use in proofsof linear independence (Problem 1.21). Note that the result in (1.69) isa generalization of the Fourier coefficient result in (1.58). Indeed, forcases where the independent sequence Yj, j = 1, 2, ... , M is orthogonal,the matrix in (1.69) diagonalizes. Inversion then produces the Fouriercoefficient result.

One of the classic problems of algebra is the approximation of afunction by a polynomial. This problem is easily cast as best approximationin the following example.

EXAMPLE 1.23 In the Hilbert space .c2(0, 1), consider the approximation ofa function fer) by a polynomial. Let {rn-l}~=1 be a sequence in .c2(O, 1). Thesequence is linearly independent. Indeed, by the fundamental theorem of algebra,the equation

Fig. 1-2 Illustration of the projection theorem in R2, as given in !Example 1.22. I !

In (1.56)-(1.58), we introduced best approximation in terms of or-thonormal expansion functions generating a linear manifold. With the aidof the projection theorem, we next generalize the concept of best approxi-mation to include expansion functions that are linearly independent but notnecessarily orthogonal. Let Y E 11, and let {Yj }~l be a linearly indepen-dent sequence of vectors in 11. We form the sum I

M

Y= I>:jYj (L66)j=l

We wish to approximate Y with y by suitable choice of the coefficients(Xj' We have already indicated in Example 1.19 (Problem 1.17) that linearcombinations of the type in (1.66) form a linear manifold M. In fact,since the limit of sequences of vectors in M must necessarily be in M, themanifold is closed and therefore meets the requirements of the projectiontheorem. Since Yj EM, j = 1, 2, ... , M, the projection theorem gives

(y - y, Yk) =0, k = 1, 2, ... , M 0:67)

24 Linear Analysis 'Chap. I Sec. 1.7 Operators in Hilbert Space 25

1.7 OPERATORS IN HILBERT SPACE

i(1.80)

(1.78)IILull ~ Yllull

L (CXIX] + CX2X2) = CX] LXI + cx2Lx2 0.77)A linear operator L with domain V L C 1i is bounded if there exists a realnumber Y such that

A(cxx] + {>x2) = cxAxl + f3Ax2 = CXZI + f3Z2

The concepts of linearity and inversion for matrices can be generalized tolinear operators in a Hilbert space.

An operator L is a mapping that assigns to a vector x 'E S anothervector Lx E S. We write

Lx = y 0.76)The domain of the operator L is the set of vectors x for which the mappingis defined. The range of the operator L is the set of vectors y resultingfrom the mapping. The operator L is linear if the mapping is such that forany XI and X2 in the domain of L, the vector CXIXI + CX2X2 is also in thedomain and

The solution is given formally by

x=A-Iz

The matrix operation is linear. Indeed, given XI, X2, Zl, and Z2, we have byordinary matrix methods

for all u E V L .

EXAMPLE 1.24 Let Roo be the space of all vectors consisting ~f a countablyinfinite set of re~1 numbers (components), viz.

a = (ai, a2, a3 ...) (1.79)where ak E R. If b = (tiI, tho th. ...), define an inner product for the space by ,

00

(a,b) = Ladhk=1

Let the norm for the space be induced by the inner product. We restrict Roo tothose vectors with finite norm. Define the right shift operator A R in Roo by

ARa = (0, ai, a2, ...)The right shift operator AR is linear. The proof is easy and is omitted. In addition,AR is bounded. Indeed,

: (1.75)

I (1.72)

, d.73)i!I (1.74)I

1"213"

1N+I

[1

where

~I = CXII~I + CX12~2 + CX13~3~2 = CX21 ~ I + CX22~2 + CX23~3

~3 = CX31~1 + CX32~2 + CX33~3Using the usual matrix notation, we let

z = [~I ~2 ~3 fx =: [~I ~2 ~3 f

where T indicates matrix transpose. We then have

Ax = z

Consider the following transfonnation in R3:

Inversion of this matrix equation yields the best approximation.

11 m-l n-Id 1(Ym, Yn) = ,r r r = 1o m+n-(y, Ym) =11 r m- 1 f(r)dr

Substitution of (1.73) and (1.74) into (1.69) gives

We then have

Define an inner product for 2 (0, :1) by

Jol f(r)g(r)dr

"

i;

IIr,l'"

1

'I]Ij1

!

I

26 Linear Analysis Sec. 1.7 Operators in Hilbert Space 27

EXAMPLE 1.25 On the complex Hilbert space 2 (0, I), we consider the! fol-lowing integral equation:

A linear operator L with domain V L C 7{ is continuous if given anE > 0, there exists a 0 > such that, for every Uo E V L, II Luo ~ Lu II < E,for all u E VI. satisfying lIuo - u II < o. We can interpret this definition tomean that when an operator is continuous, Luo is close to Lu whenever Uois close to u. There is an important theorem on interchange of operatorsand limits that follows immediately from the above definition. A linearoperator L with domain V L C 7{ is continuous if and only if for everysequence {un }~1 E V L converging to Uo E V L ,

:.

(1.81)t :10 u(~')k(~, ~/)d( = /(0

i Therefore, the operator A R is boundetl in Roo. Indeed, the least upper bound on y: is unity.

where

(1.84)

n~N

Luo = lim LUn11->00

Luo = L lim Un = lim LUnn---+oo n~oo

II Luo - LUnII < E,and

The proof is in two parts. First, we suppose that L is continuous andE > is given. We may select a 0 according to the definition o(continuityand suppose that lIuo - unll < o. Since Un is a member of a convergingsequence, Iluo -'unll < 0 for all n ~ N. Therefore,

IILuo - Lull < E

This first part of the proof shows that, if an operator is continuous, theoperator and the limit can be interchanged. In the second part of the proof,we must show that if the operator and limit can be interchanged, the operator Iis continuous. This part is not essential to our development and is omitted.The interested reader is referred to [12J. .

We now give a theorem linking the boundedness and continuity ofoperators. A linear operator L with domain V L C 7{ is bounded if andonly if it is continuous. The proof is in two parts. In the first part, we showthat if the operator is bounded, it is continuous. Indeed, if L is boundedand Uo E V L ,

wheneverlIuo - ull < 0

Indeed, the choice 0 = E/ JI is sufficient. In the second part of the proof,we must show that if an operator is continuous, it is bounded. This part is

IILuo - Lull = IIL(uo - u)1I~ YIIuo - ull

for all u E V L . Then, given any E > 0, it is easy to find a 0 :> Osuch that

(1.82)

Lu = /

and finally,

It follows that:

where L is the linear operator given by

L = 11(. )k(~, ~/)d(We shall show that the operator L is bounded if

11f Ik(t ~')12d~d( < 00 (1.83)This property of the kernel k(~,n is:called the Hilbert-Schmidt property, and theoperator L it generates is called a H,ilbert-Schmidt operator. To show thatLisbounded, we form

1/(012 = 111 u(~/)k(~, ~/)d~f:::: 11 lu(S')!2d(f Ik(~. nI2d~'= lIull211 Ik(~, nI2d~'

IIL~II:::: Mllullwhere M is the bound on the double integral.

, This equation can be written in operator notation as follows:

11,

we haveiI;

.ilII)1

11ijilIj

III~IIL

(1.87)

(1.88)

j = 1,2, ...

j = 1,2, ...

"'J [(xIJ [(J,ZI)J... (X2 = (J, Z2)" .

" .

" .

(Lz2,Zt)(Lz2, Z2)

Operators in Hilbert Space

,Illim "" (XdLzk. zJ') = (I z)Il~OO~ , J '

k=l

Sec. 1.7 29

We take the inner product of both sides of (1.86) with a member of thebasis set to gi~e

, Il(IltTlJo I>kLZk. Zj) = (J, Zj),, k=l

By ~ontinuity of the inner product and the rules for inner products, Weobtam

Equa~ion (1.87) is a matrix equation that can be written in standard matrixnotation as follows:

Linear Analysis

11 = 1,2, ...Un =

and therefore,

lim Un = 0n->oo

Lu = L lim Un = lim LUnn->oo n->oo

L lim Un = 0n->oo

But, if we choose Un as a member of the sequence

cosmr~

not essential to our development and is omitted. The interested reader is1

referred to [13].It is straightforward to show that the differential operator L = d/d~

is unbounded. The proof is by contradiction. We suppose that d/d~ isbounded. Then, it is continuous. Therefore, for any Un ~ u, we rhusthave

28

~f this matrix can be inverted to give the coefficients (Xl, (X2, . : . , substitutionmto (1.85) completes the determination of u. 'EXAMPLE 1.26 On the real Hilbert space .c2(0, I), consider the integral equa-tion

j(~) = -IW(~) +l' k(~, nu(nd~'l'

But,lim LU n = lim (-7f sinl17f~)

n---+oo n---+oo

and this limit is undefined. We therefore have arrived at a contradiction,and we conclude that d / d~ is ~nbounded. i

Giveil the concepts ofcontinuity and boundedness of a linear operator,we can show that a bounded linear operator is uniquely determined by amatrix. Indeed, let {Zk}~l be a basis for H. Let L be a bounded linearoperator with

Lu = 1We expand U in the basis as follows:

where

k(~,n =I~(l - $'),nl-n

In operator notation,

O::s~::sf~' ::s ~ ::s I

(1.89)

(1.90)

where / is the identity operator. The operator L - Ji-/ is bounded. We leave theproof for .Problem 1.26: We wish to obtain the matrix representation and therebysolve th.e mtegral equatIOn. We define the inner product for the space as in (1.63).For baSIS functions, we choose

n = 1,2, ...

n

U = lim L(XkZk: n->oo k=l

Since boundedness implies continuity,'n n

Lu = L lim L(XkZk = lim L L(XkZkn->oo k=l n->oo k=l

(1.85) (L - Ji-l)u = j,

Zn = sinmr~,

~ E (0, I) (1.91)

(1.92)By the linearity of the operator L, we then have Then, operating on any member of the basis set, we obtain

:6.86) (I, - p.l)z" = -1 sin mr~ +11 k(~, () silll17r(d( (1.93)

30 Linear Analysis Sec. 1.7 Operators in Hilbert Space 31

I1.i

(1.97)

(1.98)

(1.99)

(1.100)

(1.101)

(1.102)

(1.103)

1IIxil :s -Ixlc

[x, y] = (Lx, y)

Ix 1= J(Lx, x)

Indeed,

Therefore,

With ~his inner product definition, 'DL becomes a Hilbert space HL. TheassocIated energy norm in H L is given by

We.empha~i~e that the operator L must be positive for (1.9~b to satisfy thebaSIC d~finltlOns of a norm. Indeed, the energy inner product and normdefi.ned In (1.97) and (1.98) must be shown in each case to satisfy the rulesfor Inner p~od~cts and norms. For positive-definite operators, we can provethe follOWIng Important relationship between norms:

..A speci~1 inner product and norm [17], associated with positive andposltlve-defimte operators, are useful in relating convergence criteria. De-fine the energy inner product with respect to the operator L by

IIxIl2:s~lxFc

Taking the square root of both sides yields the desired result.Amon~ the many forms ofconvergence criteria, there are several types

that are particularly useful in numerical methods in electromagnetics. Fora sequence {un} C H, Un converges to U is written '

lim IUn - ul = 0n---+oo

eUn~U

and means that

lim lIun - u'l = 0n---+oo

The statement Un converges in energy to U is written

and means that

(1.96)

(1.95)

(1.94)

k = 1,2, ...2Uk = 1 (j, zd,(krr)2 - /1,

Substitution of (1.95) into (1.85) yields the final result, viz.

In the above example of representation of an operator by a matrix, thechoice of the basis functions resulted in diagonalization of the matrix and,therefore, trivial matrix inversion. There are many operators, however, that

I 'do not have properties that result ~n this diagonalization. These concepts arebetter understood after a study of operator properties and resulting Greeh'sfunctions and spectral representations in the next two chapters. ,

An important collection of operators for which there are e~tabl:ishedconvergence criteria are nonnegative, positive, and positive-definite ,oper-ators. The reader is cautioned that there is little uniformity of notationconcerning these operators in the literature. For the purposes herein, anoperator L is nonnegative if (Lx, x) 2: 0, for all x E 'DL. An operator ispositive if (Lx, x) > 0, for all x#-O in 'DL. An operatoris positive-definiteif (Lx, x) 2: c2 11x1l 2 , for c > 0 and x E 'DL. An operator L is symmetricif (Lx, x) = (x, Lx). It is easy to show that nonnegative, positive, andpositive-definite operators are symmetric. In fact, any operator haviqg theproperty that (Lx, x) is real is symmetric. Indeed, '

(x, Lx) == (Lx, x) = (Lx, x)

where the inner product is the usual inner product for 2 (0, I) and 8nm ha~ beendefined in (1.33). The matrix representation in (1.88) therefore diagonalize~, andthe inversion yields i

I

,

After some elementary integrations, we obtain the general matrix element in ,thesquare matrix in (1.88), viz.

L - /1l)Zn, Zm) = [(n~)2 - /1] (Zn, Zm)= ~ [(n~)2 - /1] 811m

But, using (1.90), we find that

11 ' 1~ 11

;k(;,nsinmr(d;' = (1-;) (sinnrr(d( +; (I-nsinnrr,(d(o 0 ~ ,

I(un - u, g)1 ~ lIun - ulIlIgll

. "

33

(1.106)

(1.109)

(1.110)

m=I,2, ... ,n

Lu - f = 0

Ax =b

Method of MomentsSec. 1.8

where U E V L , f E R L . Define the linearly independent sets {cPdk=l CV Land {wdk=l' where cPk and Wk are called expansion functions andweighting functions, respectively. Define a sequence of approximants to uby ,

for g E 'HL. This procedure proves the first half of Property C. The proofof the second half is based on the Hilbert space 'HL being dense in 'H andis omitted. (See [17, p. 24-25].) To prove Property D, we write

1lIun -ull ~ -Iun - ul

c

Taking the limit as n -* 00 yields the desired result.

The purpose of this section is to introduce the Method of Moments in ageneral way and develop various special cases. Emphasis is on convergenceand error minimization.

If L is a linear operator, an approximate solution to Lu = f is givenby the following procedure. For L an operator in 'H, consider

1.8 METHOD OF MOMENTS

n

Un = L akcPk. 1l = 1, 2, . . . (1.107)k=l

A matrix equation is formed in (1.106) by the condition that, upon replace-ment of Uby Un, the left side shall be orthogonal to the sequence {Wk}. Wehave .

(Lun - f, wm ) = 0, m = 1,2, ... , n (1.108)Substitution of (1.107) into (1.108) and use of (1.25) gives the matrixequation of the Method ofMoments [18] ,[19], viz.

where, in usual:matrix form,

nL ak(LcPk. wm ) = (f, wm ),k=l

Note that the exact operator equation (1.106) in a HiIbert space'H has beentransformed in(o an approximate operator equation on Hilbert space C n ,viz. '

IChap. I

(1.105)

d.104)II

Linear Analysis

li~ IUn - up = 0n~oo

To prove Property B, we use the CSB inequality to give

Since, by hypothesis, II Lunll is'bounded and Un -* u, a limiting operationgives

IU - unl2 = I(L(un - u), ~n - u)1 ~ IIL(un - u)lIl1u n - ull= IILu n - Luililun - ull ~ (IiLunll + IILulI) lIun -ull

w:U n -+ U

for any g in 'H. Taking the limit yields the desired result, viz.

lim I(u n - U, g)1 = 0n~oo,

I

To prove Property C, we have

I(Lun - f, g)1 = I(L(un ~ u), g)1 = I[un - u, g]l ~ IUn -c- ulhlwhere we have used the CSB inequality on Hilbert space 'HL. By hypoth-esis, we have convergence in energy. Therefore,

and means that for every g E 1-t

lim I(un - U, g)1 = 011-+00

It is straightforward to show the following relationships among the typesof convergence:

A. If II Lunll is bounded, convergence implies convergence iD energy.~ ,

B. Convergence implies vveak convergence.

C. Convergence in energy implies LUn ~ f. The weak conver-gence is for those g, defined by (1.105), in 'HL. If, however, II Lunllis bounded, then LUn ~ fin 'H.

, ,

D. If L is positive-definite, convergence in energy implies conver-gence.

We first prove Property A. We have

The statement Un converges weakly to U is written

32I .

lim I(Lu" - f, g)1 = 0n-+oo

x = (a] a2 (1.111)

34 Linear Analysis Chap. I Sec. 1.8 Method of Moments 35

Iii

(Ll18)

(1.119)

(1.120)

(Ll22)

(1.121)

n

Un = L[u, k]oo

lim lIu lI - llll = 0n-...oo

Lu = f

(Lu, v) = (u, L*v)

Substitution into (1.107) gives

Let the adjoint! operator L * be defined byI

which is the Fourier series expansion in 7-{L of Un with Fourier coefficientsgiven by (1.117). Therefore,

By Property C, the resuIt in (1.119) implies that LU n ~ f. Uilfortunately,nothing can be said about the nearness of Un to u. If, however, L is positive-definite, Property D states that the approximants converge, vi~.

. ,

for u E D L , V E V L*. Then, if the adjoint L * exists, multiplication of bothsides of (1.121) by L * produces

In the Galerkin procedure, if the operator L is positive and the sequence{d is complete in 7-{L, the method is calIed the Rayleigh-Ritz method.For a classical treatment of the Rayleigh-Ritz method, the reader shouldconsult [17J,[20].

For the more general operators often encountered in electromagnetics,a positive operator can be produced by the following procedure. Consider

(Ll15)

(i.112)I

(1.113)

m=1,2, ... ,n

b = ( (j, WI) . (j, W2)'A = [amd

n :

Lak(L

I 36 Linear Analysis Chap. I\

Sec. A.I Appendix-Proof of Projection Theorem 37

where the energy norm is with respect to the operator L *L. By propertiesof the adjoint, (1.125) can also be written

n

L CXk (L *Lk, m) = (L* f, m), m = 1, 2, ... , n (1,.125)k=1

Since L *L is positive, if the sequence {k} is complete in VL*L, (1.125) isthe Rayleigh-Ritz method and convergence in energy Un ~ Uis assured,viz.

In proving the Projection Theorem, we begin by noting that the firstequality in (A. 1) makes sense. Indeed, IIx - y 11 is bounded below by zero,and therefore has a greatest lower bound. We next show that there existsat least one vector Yo closest to x. We begin by asserting that there existsa vector Yn E M such that by the definition of infimum,

(A.2)

(A.3)lim IIx - Yn II = 8n--->oo

Taking the limit as n --* 00, we find that(1.126)lim IUn - UI= 0

n--->oo

The Galerki:l specialization follows immediately, viz.. ,

.'

Therefore, we can always define a sequence {Yn} E M such that IIx - Yn IIconverges to 8. In (1.36), if we replace x by x - Yn and Y by x - Ym, weobtain [21], afte~ some rearrangement,

llYn - Ym 11 2 = 211x - Ynl1 2 + 211x - Ym 11 2 - 411x - ~(Yn + Ym)1I 2 (AA), 2

0.127)m=I,2, ... ,11n

LcxdLk, Lm} = (f, Lm),k=1

which is the result in the Method of Least Squares, more usually derived[20] by minimization of

so that LUn --* f. Unless the operator L*L is positive-definite, n9thingcan be saidconceming the convergence of Un to u.

It is easy to show that (1.126) implies that

lim IILu n - fII = 0n--->oo .

q.128)Since M is a linear manifold, (Yn + Ym)/2 E M, and we may assert that

Therefore,

,:ii"I

where inf is the greatest lower bound, or infimum. In other words, Yois closest to x in the sense that IIx - Yoll ~ IIx - yll for all Y in M.Furthermore, the necessary and sufficient condition that Yo is the pniqueminimizing vector is that e = x - Yo is in M 1-. The vector Yo is called theprojection ofx onto M. The vector e is called the projection of x ontb M 1-

A.1 APPENDIX-PROOF OF PROJECTIONTHEOREM ,

In this Appendix, we prove the Projection Theorem, considered in S:ection1.6. We restate the theorem here for convenience. Let x be any ve~tor inthe Hilbert space 1t, and let M c 1t be a closed linear manifold. !Then,there is a unique vector Yo E M c 1t closest to x in the sense that I

8 = inf 'lix - YII = Ilx - yollyEM,

i (A.l)I

In the limit as m, n --* 00, the right side goes to 282 + 282 - 482 = 0, andwe conclude that the sequence {Yn} is Cauchy. Since 1t is a Hilbert spaceand M is closed, M is a Hilbert space and Cauchy convergence impliesconvergence. Therefore, Yn --* Yo E M.

We next show that Yo is unique [22]. Suppose it is not unique. Then, ,we must have at least two solutions Yo and Yo satisfying IIx, - Yo IIIIx - Yoll = 8. Then,

lIyo - Yo 11 2 ~ 211x - Yoll 2 + 211x - Yoll 2 - 411x - ~(y~ + yo) 11 22~ 282 + 282 - 482 = 0

(A.6)Therefore, Yo = Yo.

Finally, we show that e =x - Yo E Ml-. We must show that e isorthogonal to every vector in M. Suppose that there exists a vector Z E Mthat is not orthogonal to e. Then, we would have [23]

". A' AIIx - zoll2 = (x - Yo - -zx - Yo - -z)IIzll2 " IIz11 22 A A 1~12

= Ilx - Yo II - IIzll 2 (x - Yo, z) - IIz11 2(z, x - Yo) + 11211 22 IAI2

= IIx - Yo II - IIzll2

39ProblemsChap. 1

and

Ilixil - lIylll .::: IIx - yll

(X, y} + (y, x} = ~ (li x + Yll2 - IIx _ Y1I 2)

First, show that the sequence is Cauchy; next, show that the seqiJence doesnot converge in the space. (Indeed, it converges to e; the details can be foundin [4J.)

I(x, y}1 = IIxlillyll

1.8. Show that in Rn the set of vectorsI

e\ = (1,0, ... ,0), e2 = (0, 1, ... ,0), ... , en = (0,0, ...',1)is linearly independent. Is the same conclusion valid in Cn ? Is the set of'vectors a basis for Cn ?

1.9. Given the basic definition of an inner product space, show thatn n(l:= akXb y} = L adxb y}

k=1 k=\

1.10. Show that C(a, (3) with inner product defined by (1.27) is a real inner product.space.

1.11. Consider the linear space of real continuous twice differentiable functionsover the interval (0, I). As a candidate for an inner product for the space,consider

(j, g} = 11 J"(r)gl/(r)drwhere f and g are members ofthe space and "primes" indicate'differentiation.Determine whether (j, g} is a legitimate inner product.

1.12. Prove the following corollary to the CSB inequality in (1.35):

1.15. Given the following sequence in the space of rational numbers:

n 1X n = {; (k - I)!

if and only if x and yare linearly dependent.1.13. Prove the following identity:

1.14. Consider a complex inner product space with norm induced by the innerproduct. If X and yare members of the space, prove that

i(x, y} - (y, x} = 2' (lix + iYll2 - IIx - ;YIl2)

(A,7)

(A;8)

Chap. 1

(A.9)1

(..\.10)

Problems

zEM

AZo == Yo + \I Z11 2 Z

PROBLEMS

(e, z) = (x - Yo,z) = A =I 0,

Ilx - zoll < Ilx - Yo IIwhich, by (A. I ), is impossible.

Therefore,

We define a vector Zo EM such that

38

Then,

I1.1. Using the rules defining a linear space, show that Oa = 0 and -la = -\-a.1.2. Show that Rn is a linear space.1.3. Show that C(O,I) is a linear space.1.4. As an extension to Example 1.4; in R2 , let Xl = (l, 3), X2 = (2,6.00000001).

Show that XI and X2 are linearly independent. Comment on what mightoccur in solving this problem on a computer with eight-digit accuracy. (Thisproblem is indicative of the difficulties that can arise in establishing linearindependence in numerical experiments in finite length arithmetic.)

1.5. If XI, X2, , Xn is a linearly dependent set, show that at least one member ofthe set can be written as a linear combination of the other members.

1.6. Show that if 0 is a member of the set Xl, X2, . , Xn , the set is linearly depen-~~. !

1.7. Show that if a set containing nvectors is linearly dependent, and if m addi-tional vectors are added to the set, the resulting set of n +m vectors is linearly

idependent. i1

I .

I '

41

n=O

n>O

m =/= n

m=n=/=Om =n =0

-Tf In(2/a)To(~/a),

n 1Un = L - coskTf~

k=1 k

j Ct d~u(O In 11] - ~I = f(1])-ct Ja2_~2

Problems

consider the following integral equation:

This integral equation occurs in diffraction by a slit in a perfectly conductingscreen. It can be shown [14] that the operator

is bounded. Solve the integral equation by using the Chebyshev polynomialsTn (~/a) as a basis for L2(-a, a) and obtaining the matrix representation for,L. Hint: The following are useful integral relations [15]:

It is well-known [16] that

Chap. 1

1.24. Let L = d / d ~ , and consider the sequence of partial sums

C;:hap.lProblems40

a = (ai, a2, ...)

where ai E R. Let M be the set of vectors in Roo with only a finite nOmberof the countably infinite number of components different from zero. Showthat M is a linear manifold. Show that M is not closed. Hint: Consider the

Iconcept of closed as applied specifically to the sequence of vectors I

Xn = (I,,~, ~, ... , ~,O, 0, ...): 2 3 n

1.19. The Legendre functions Pn (~.), n = 0, 1, 2, ... , are orthogonal on ~ E(-1, I),but they are not orthonormal. Create a sequence of orthonormal-ized Legendre functions Pn (n, n = 0, I, 2, . . . . I

1.20. Given that in R3 , XI = (I, 2, 0), X2 = (0, I, 2), X3 = (1, 0, I).(a) Prove that {Xl, X2, X3} is a linearly independent set of vectors.(b) From the linearly indepen~ent set, produce the first two members of the

associate orthonormal set !using the Gram-Schmidt procedure.1.21. Show that the determinant of the Gram matrix in (1.69) is nonzero if and only

if the sequence of vectors {Yk }t,l is linearly independent [II]. '1.22. Let Roo be the space described in Problem 1.18. If b = (f3I, f32, ...), ;define

an inner product for the space by i

,

1.16. Show that if S is a linear space, a linear manifold M c S is also a!linearspace.

1.17. Let Xb k = 1,2, ... , n be a linearly independent sequence of vectors ,in theHilbert space 'H.. Define M to be the set of all linear combinations of the nvectors. Prove that M is a linear manifold. r

1.18. Let Roo be the space of all vectors consisting of a countably infinite set pf realnumbers (components), viz.

00

(a, b) = I>kf3kk=l

Let the norm for the space be induced by the inner product. We rest&t Rooto those ,vectors with finite norm. Define the operator A in Roo by

Aa =: (ai, ~a2' ~a3' ...)Test the operator A for boundedness.

I 1.23. On the Hilbert space L2(-a, a), with inner product

lim Un =-In(2sinTf~)n->oo 2

Show that limn->oo LUn is undefined. The problem is that L is unbounded.This result is an example of the fact that a Fourier series cannot always bedifferentiated term by term.

1.25. Let L = d/d~, and consider the sequence of partial sums

n 1Un = L 2" coskTf~

k=l k

fCt d~(f, g) = .f

1.26. Show that the operator in (1.91) with kernel defined in (1.90) is bounded.1.27. Consider Hilbert space 2 (-1, 1) with inner product I

where all functions are real-valued. Consider the following function f (0:

, I(u,~) = 11 u(Ov(~)d~

43References

(j, g) = 11 f(~)g(~)d~where fU;), g(O E 2(0, 1). Suppose that

Chap. 1

1.31. Consider the real Hilbert space 2(0,1) with inner product '

f(~) = 1 - i2

(a) By the method of best approximation, approximate f(~) by i(~), wherei(~) is a linear combination constructed from the orthonormal set

(JEkcoskn~}t=o in 2(0, 1). In the orthonormili set, Ek is 1 for k = 0and 2 for k # O.

(b) Calculate the norm of the error IIf(O - i(~)II.1.32. Consider Euclidean space R4. Define vectors a and b in R4 by

iChap. 1Problems42

, .

a = (aI, , a4)b = (fJI, , f34)

Define an inner product for the space by

12rr(u, v) = u(Ov(~)d~, 04

(a, b) = L.:>kf3kk=l

Consider those vectors a in R4 restricted by

(a) Show that all vectors with this restriction form a linear manifold M.(b) Find all vectors b that are members of M 1-

REFERENCES[I] Stewart, G.w. (1973), Introduction to Matrix Computations. New

York: Academic Press, 54-56.[2] Hardy, G.H. (1967), A Course in Pure Mathematics. London: Cam~

bridge University Press, 1-2.[3] De Lillo, N.J. (1982), Advanced Calculus with Applications. New

York: Macmillan, 32. '[4] Stakgold, I.: (1967), Boundary Value Problems of Mathematical

Physics. Vol, I. New York: Macmillan, tal-102.[5] Mac Duffee; c.c. (1940), Introduction to Ahstract Algehra. New

York: Wiley. chapter VI.

aER. (~- 2a)f(~) =;sm -2- ,

/

1 d~(j, g) = -1 f(~)g(~)JI=T2

Determine whether this dcfini'tion results in a legitimate inner product.

is an orthogonal sequence.(b) Produce an orthonormal (O.N.) sequence from the orthogonal sequence.(c) Using the members of the O.N. sequence contained on -N ~ ~ ~ N,

where N is a positive integer, find the best approximation to

where members of the space ~re complex functions.(a) Show that the sequence '

in the sense given in Section 1.6, Best Approximation.1.30. Consider the real Hilbert space 2(-1, 1). For f(o. g(O E 2('-1, 1),

define an inner product

44 References , Chap. 1

lr

\

[6] Greenberg, M;D. (1978), Fhundations ofApplied Mathematics.! En-glewood Cliffs, NJ: Prentice-Hall, 319-320.

[7] Helmberg, G. (1975), Int~oduction to Spectral Theory in HilbertSpace. Amsterdam: N0rth-Holland,20--2l.

[8] Ibid., Appendix B. ,[9] Shilov, G.E. (1961), An Introduction to the Theory ofLinear Spaces.

Englewood qiffs, NJ: Prentice-Hall, sect. 83-85. I[10] Stakgold, 1. (1979), Green's Functions and Boundary Vatue 'Prob-

lems. New York: Wiley-~nterscience, 36-41. :[11] Lue)1berger, D.G. (1969),: Optimization by Vector Space Methods.

New iork: Wiley, 5~57. ', ,

[12] op.cit. Helmberg, 73.[13] Akhiezer, N.L, and LM. Glazman (1961), Theory ofLinear Operators

in Hilbert Space. New York: Frederick Ungar, 33,39. '[14] Dudley, D.G. (1985), Error minimization and convergence in n~mer

ical methods, Electromagnetics 5: 89-97.[15] Butler, C.M., and D.R. Wilton (1980), General analysis of narrow

strips and slots, IEEE Trans. Antennas Propagat. AP-28: 42-48.,[16] Abramowitz, M., and LA. 'Stegun (Eds.) (1964), Handbook of Math-

ematical Functions, National Bureau of Standards, Applied Mathe-matics Series, 55, Superintendent of Documents, U.S. GovernmentPrinting Office, Washingtqn, DC 20402, 1005. ,

[17] Mikhlin, S.G. (1965), The Problem of the Minimum of a QuadraticFunctional. San Francisco: Holden-Day.

[18] Harrington, R.E (1968), field Computation by Moment Methods.New York: Macmillan.

[19] Harrington, R.E (1967), "Matrix methods for field problems," hoc.IEEE 55: 13~149.

[20] op.cit. Stakgold (1967), v61. 2, sect. 8.10.[21] Krall,A.M. (1973), Linear Methods ofApplied Analysis. ReadIng,

MA: Addison-Wesley, 181-182.[22] Ibid., !182.[23] op.cit. Akhiezer and Glazman, 10.

2The Green's FunctionMethod

2.1 INTRODUCTION

In this chapter, we begin our study of linear ordinary differential equationsof second order. Our goal is to develop a procedure whereby we cansolve the differential equations using fundamental solutions called Green'sfunctions.

We begin >yith a brief discussion of the delta function. We followwith a description of the Sturm-Liouville operator L and Its properties.We de~ne three types of Sturm-Liouville problems and investigate theirpropertIes. In all three types, we examine the role of the operator L and its 'adjoint operator L *. These operators are used to define the Green's func- :

~ion and the adjoint Green's function, respectively. Our study culminates:10 a procedure fo~ applying the Green's function and/or the adjoint Green's,function in solving the differential equation Lu = f.

2.2 DELTA FUNCTION

The concept of the delta function arises when we wish to fix attention on 'the value of a function f(x) at a given point xo. Mathematically, we seekan operator T that transforms a function f (x), continuous at xo, into f (xo),the value of the function at xo. In equation form, we require T such that

,

, ,

: !

T [f(x)] = f(xo) (2.1)

46 The Green's Function Method Chap. 2 Sec. 2.2 Delta Function 47

We begin by considering the pulse function P (x - xo), defined by ;

~ ,

An important property of the pulse function is that it is even about Xo, viz.

Taking the limit as E ---+ 0, we have

lim lb f(x)P(x - xo)dx = f(xo), Xo E (a, b) (2.7)-+o a

The integration followed by the limiting operation in (2.7) transfonns f(x)to f (xo), the value of the function at Xo.EXAMPLE 2.1 Let f(x) = x 2, Xo = 0, and Xo E (a, b). In this case, f(x) iscontinuous at x = 0 and we have

lim 1" f(x)p,(x - xo)dx = lim[~ l' X2dX],->0 a ,->0 2E _,

= lim(E2

) = 0,-.0 3(2.4)

. (2.3)

: (2.2)Xo - E < X < Xo + Eotherwise

l b . l XO + IP (x - xo)dx = - dx = Ia Xo- 2ENote that, regardless of the value of E, the area under the pulse is 'unity.Indeed, if (a, b) is any interval containing (xo - E, Xo + E),

This property can be proved by interchanging x and Xo in (2.2). The detailsare left for the problems. Multiplying the pulse function by f (x) andintegrating over any interval containing the pulse gives (Fig. 2-1)

1" 1 11+'lim f(x)p,(x - xo)dx = lim - cosx dx,->0 a ,->02E 1-'EXAMPLE 2.2 Let f(x) = cosx, Xo = rr/3, and Xo E (a, b). In this case, If (x) is continuous at x = rr /3 and we have

= !~121E [sin (i + E) - sin (i -E)]I= ~

Since f(O) = 0, w,e have verified (2.7).

Since f (rr /3) = 1/2, we have again verified (2.7).

(2.5)

(2.6)

lb I l xo+f(x)p(x - x.o)dx = - f(x)dxa : 2E Xo-By the mean value theorem for ihtegrals [I], if f is the mean value of f(x)on the interyal x E (xo - E, Xo + E), '

lb f(x)P(x - xo)dx = f

I'II'

\

1/28

~Pe(X-Xo),--------=,

Expression (2.7) fonns the cornerstone of our definition of the deltafunction, as follows:

1h f(x)8(x - xo)dx = lim l b f(x)P(x - xo)dx (2.8)a -+o a

-1-----+----:---+----+-----" x

I

Fi~. 2-1

f(x)

Xo + 8

Pulse function p,(x - xo) and function f(x).

so that l b f(x)8(x - xo)dx = f(xo) (2.9)for any Xo in the interval (a, b). Note in (2.2) that as E becomes smaller, thepulse function becomes narrower and higher while maintaining unit area.If the limit in (2.7) could be taken under the integral, we would have

8(x -xo) 4: lim [p, (x -xo)] (2.10)-+O

i 48 The Green's Function Method Sec. 2.2 Delta Function 49

where (.) indicates the position of the function upon which T operates.From the basic definition of the delta function in (2.9), we obtain

some additional relations. If we set Xo equal to zero, we find

(2.15)

(2.16)

(2.17)

(2.18)

(2.19)

(2.20)

f(x) = L[u(x)]

= L f g(x, ~)f(~)d~=f Lg(x, ~)f(~)d~

u(x) = f g(x, ~)f(~)d~

Lu = f

Suppose we wish to solve the equation

Since L is a differential operator, we shall assume that its inverse is anintegral operator with kernel g(x, ~), so that

or

where L is a diffe~ential operator. Formally, the solution is given by mul-tiplying both sides of (2.15) by the inverse operator, viz.

Substitution into (2.15) gives

where we have assumed, without proof, that we can move the operator Linside the integral. But, from the properties of the delta function, we have

f(x) = l b 8(x - ~)f(S)d~for x E (a, b). Comparing (2.18) and (2.19), we identify

Presumably, if we can solve (2.20), then the solution to (2.15) is givenexplicitly by (2.17). The kernel g(x, ~) is called the Green's function forthe problem.

It is the purpose of this chapter to formalize and structure the introduc-tory ideas above. The result will be the solution to a class of linear ordinarydifferential equations of second order by the Green's function method.

(2.11 )

i

(2.12)I!I

I

(2.13)

bT = 1(.)8(x - xo)dx

a .....

Since this limit does not exist, the interchange of limit and integration in(2.8) is not valid. We have therefore placed an "s" over the equality in(2.10) to indicate symbolic equality only.

The delta function 8(x - xo) has two remarkable properties. Symbol-ically, it is a function that is zero everywhere except at x = xu, where it isundefined. Second, when integrated against a function f that is continuousat xo, it yields the value of the function at xo. We note that (2.9) defines theoperator T we were seeking in (2.1). Indeed, comparing (2.1) and (2.9)yields

l b f(x)8(x)dx = f(O)

l b ,a 8(x - xo)dx = 1Finally, from (2.4) and (2.8), we conclude symbolically that

Also, if in (2.9) we set f(x) = 1, we obtain

i

8(x - xo) :b 8(xo - x) (~.14)II

In concluding our development of the delta function and its properties,we remark that there are certain difficulties with the definitions. Indeed,any function that is zero everywhere except at one point must producci zerowhen Riemann integrated over any interval containing the point. The resultin (2.13), for example, is therefote unacceptable in the Riemann sense. Tointerpret the integral, it seems that we must return to the basic defirtitionin (2.8). The mathematical acceptability of integrals involving the deltafunction have, however, been fdrmalized in the Theory of Distributions,introduced by Schwartz [2]. In the theory, the delta function is called ageneralized function, and the integral in (2.9) is said to exist in the distri-butional sense. Although the theory is beyond the scope of this book, theinterested reader can find introductory treatments in [3],[4]. '

The central role played by'the delta function in the solution to cer-tain differential equations becomes apparent in the following argument.

II

\

50 The Green's Function MethodI

Chap. 2 Sec. 2.3 Sturm-Liouville Operator Theory 512.3 STURM-LIOUVILLE OPERATOR THEORY EXAMPLE 2.3 Consider Bessel's equation of order v, given by

Consider the following linear, ordinary, differential equation of s~condorder:

" 1 I (v2 2)-u - -u + - - k u = f

X x 2(2.28)

Using (2.23)-(2.25), we obtain

(2.29)

q(x) = _(kx)2p(x) = x

1w(x) = -

x

1 I I (V2 2)--(xu) + - - k u = f

X x 2

V2

q(x) = 2"x

p(x) = xw(x) = x

where "prime" indicates differentiation with respect to x. Comparing to (2.21),we identify

ao =-1

al = -O/x)a2 = (v/x)2

To transform to Sturm-Liouville form, we use (2.23)-(2.25) and obtain

so that

2 " I [k 2 2) -x U - xu - ( x) - v u = f (2.30)

We note that (2.30) is obtained simply by multiplying both sides of (2.28) by x 2.In this case, we identify

A = _v2

ao = _x 2

EXAMPLE 2.4 Consider Bessel's equation, given by (2.28), in a slightly dif-ferent form, viz.

(2.27)

(2.26) al = -xa2 = _(kx)2

(2.24)

(2.23)

(2.22)

(L - A)U = f

1 'd[ d]L =--- p(x)- + q(x)w(x) dx dx

1 d [ dU(X)] .---- p(x)-- + q(x)u(x) - AU(X) = f(x)

w(x) dx dx

For the rem~inderof this chapter, without loss of generality, L will alwaysmean the Sturm-Liouville operator in (2.27). I

where we identify the Sturm-Liouville operator L, viz.