Methods of Applied Fourier Analysis ||

Applied and Numerical Harmonic Analysis

Series EditorJohn J. BenedettoUniversity of Maryland

Editorial Advisory Board

Akram AldroubiVanderbilt University

Ingrid DaubechiesPrinceton University

Christopher HeilGeorgia Institute of Technology

James McClellanGeorgia Institute of Technology

Michael UnserSwiss Federal Instituteof Technology, Lausanne

Victor WickerhauserWashington University, St. Louis

Douglas CochranArizona State University

Hans G. FeichtingerUniversity of Vienna

Murat KuntSwiss Federal Instituteof Technology, Lausanne

Wim SweldensLucent TechnologiesBell Laboratories

Martin VetterliSwiss Federal Instituteof Technology, Lausanne

Jayakumar Ramanathan

Methods of Applied Fourier Analysis

Springer Science+Business Media, LLC

Jayakumar Ramanathan Department of Mathematics Eastern Michigan University Ypsilanti, MI 48197

Library of Congress Cataloging-in-Publication Data

Ramanathan, Jayakumar, 1958-Methods of applied fourier analyis / Jayakumar Ramanathan.

p. cm. -- (Applied and numerical harmonic analysis) ISBN 978-1-4612-7267-0 ISBN 978-1-4612-1756-5 (eBook) DOI 10.1007/978-1-4612-1756-5 1. Fourier analysis. 1. Title. II. Series.

QA403.5.R33 1998 515'.2433--dc21

Printed on acid-free paper 1998 Springer Science+Business Media New York Originally published by Birkhliuser Boston in 1998 Softcover reprint ofthe hardcover Ist edition 1998

98-4738 CIP

Copyright is not c1aimed for works of U.S. Government employees. AII rights reserved. No part ofthis publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without prior permission of the copyright owner.

Authorization to photocopy items for internal or personal use, or the internal or personal use of specific c1ients, is granted by Springer Science+Business Media, LLC, provided that the appropriate fee is paid direclly to Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, USA (Telephone: (978) 750-8400), stating the ISBN, the tille ofthe book, and the first and last page numbers of each article copied. The copyright owner's consent does not include copying for general distribution, promotion, new works, or resale. In these cases, specific written permission must first be obtained from the publisher.

ISBN 978-1-4612-7267-0

Typeset by the Author in LArEX.

9 8 765 432 1

Dedicatedto

Beth, Lauren, and Nicole.

Contents

Preface xi

1 Periodic Functions 11.1 The Characters . ..... 11.2 Some Tools of the Trade . 41.3 Fourier Series: LP Theory 81.4 Fourier Series: L2 Theory 151.5 Fourier Analysis of Measures 191.6 Smoothness and Decay of Fourier Series 221.7 Translation Invariant Operators . 231.8 Problems ............ 27

2 Hardy Spaces 312.1 Hardy Spaces and Invariant Subspaces 312.2 Boundary Values of Harmonic FUnctions . 362.3 Hardy Spaces and Analytic FUnctions 422.4 The Structure of Inner FUnctions 452.5 The HI Case ........... 502.6 The Szego-Kolmogorov Theorem 532.7 Problems .... . . . . . . . . . 59

3 Prediction Theory 633.1 Introduction to Stationary Random Processes 633.2 Examples of Stationary Processes . . 683.3 The Reproducing Kernel ....... 713.4 Spectral Estimation and Prediction . 763.5 Problems ............... 84

4 Discrete Systems and Control Theory4.1 Introduction to System Theory4.2 Translation Invariant Operators .4.3 Hoc Control Theory . . . . . . .4.4 The Nehari Problem . . . . . . .4.5 Commutant Lifting and Interpolation4.6 Proof of the Commutant Lifting Theorem4.7 Problems .

5 Harmonic Analysis in Euclidean Space5.1 Function Spaces on Rn .5.2 The Fourier Transform on 1 .5.3 Convolution and Approximation5.4 The Fourier Transform: 2 Theory5.5 Fourier Transform of Measures5.6 Bochner's Theorem .5.7 Problems .

6 Distributions6.1 General Distributions .6.2 Two Theorems on Distributions.6.3 Schwartz Space . . . . .6.4 Tempered Distributions6.5 Sobolev Spaces6.6 Problems .

7 Functions with Restricted Transforms7.1 General Definitions and

the Sampling Formula . . . . . . .7.2 The Paley-Wiener Theorem ....7.3 Sampling Band-Limited Functions7.4 Band-Limited Functions and Information7.5 Problems .

8 Phase Space8.1 The Uncertainty Principle .8.2 The Ambiguity Function . . . . . . .8.3 Phase Space and Orthonormal Bases8.4 The Zak Transform and the Wilson Basis8.5 An Approximation Theorem.8.6 Problems .

viii

87879093103108112119

123123127134138145152156

159159164172175179184

187

187192199203215

219219227236243256260

9 Wavelet Analysis 2639.1 Multiresolution Approximations. 2639.2 Wavelet Bases .......... 2679.3 Examples ............. 2769.4 Compactly Supported Wavelets . 2819.5 Compactly Supported Wavelets II 2909.6 Problems ............ 299

A The Discrete Fourier Transform 301A.l The Analysis of Periodic Sequences . 301A.2 The Cooley-Thkey Algorithm . . 306A.3 The Good-Winograd Algorithm. 310A.4 Fast Computations of Integrals 313

B The Hermite Functions 317

Bibliography 323

Index 328

ix

Preface

From its inception, harmonic analysis has made fundamental connectionswith other scientific disciplines. The aim of this book is to provide a math-ematical introduction to this field with special emphasis on those topicsthat do find direct application in engineering and the sciences. It must bementioned that, even with such a guideline, there is still great latitude inthe choice of possible topics; the particular choices made here are primarilya matter of taste. The material should be accessible to graduate studentsin mathematics with a good background in analysis.The first chapter is an introduction to the basic ideas of trigonometric

series. The material includes a treatment of the 1 and 2 theory togetherwith important ancillary topics such as the Fourier analysis of measures.Chapter 2 is an introduction to the theory of Hardy spaces. The structureof inner and outer functions is presented along with a proof of the Szego-Kolmogorov theorem.With the theory of the first two chapters as a foundation, the next two

chapters develop material that is directly relevant to applications. Chapter3 explores the prediction theory of discrete stationary stochastic processes.This includes a discussion of the spectral theory of stationary processes aswell as prediction theory (including the maximum entropy solution). Thefourth chapter explores connections of Fourier series with discrete controltheory. Ideas and theorems basic to Boo control theory are given, includingNehari's theorem and the commutant lifting theorem.Chapters 5 and 6 are again of a foundational nature. The first of these

mirrors chapter 1 and exposits the basic theory of harmonic analysis in R n.Chapter 6 is an introduction to the theory of distributions. This includesthe theory of tempered distributions as well as a rudimentary treatment ofSobolev spaces.The last three chapters are devoted to application-oriented topics in the

R n setting. Chapter 7 begins with the connection between functions withrestricted Fourier transform and analytic function theory, the main resultbeing the Paley-Wiener theorems. The chapter then turns to the analy-

sis of band-limited functions with the aim of making rigorous connectionswith the ideas of information theory. Chapter 8 is devoted to the analysis offunctions using techniques where the spatial (time) and frequency domainsare treated on an equal footing. The chapter begins with a discussion of theuncertainty principle and the ambiguity transform. The rest is devoted tovarious positive and negative results about the distribution of orthonormalbases within phase space. The subject of the final chapter is wavelet the-ory. After presenting the basic ideas of multiresolution approximations andexamples, the chapter covers the theory of compactly supported wavelets.There are two appendices covering the discrete Fourier transform and

Hermite functions. Problems of varying difficulty are given at the end ofeach chapter. Some ask the reader to fill in details while others exploretopics related to those presented in the text.

Acknowledgment It is a pleasure to thank my family for their encour-agement throughout the duration of this project. Thanks also to WayneYuhasz and Lauren Lavery of Birkhauser for their patient guidance throughthe publication process.

Jayakumar RamanathanAnn Arbor, MI

1998

xii

Chapter 1

Periodic Functions

Fourier analysis has its roots in Fourier's work on the theory of heat wherehe found it necessary to express any periodic function by a trigonometricseries. The issues of how such expansions are to be interpreted and whenthey are possible are surprisingly deep and have motivated much mathe-matics since Fourier's initial contribution. This chapter, an introduction tothis topic, will begin with the formal definition of Fourier series of periodicfunctions as well as a review of the various function spaces essential to aproper study of the convergence of Fourier series. We will then proceed tostudy the convergence of the Fourier series expansions of functions in thesefunctions spaces as well as the relationship of the smoothness of functionsto the decay of the series coefficients.

1.1 The CharactersThe systematic use of trigonometric expansions and integrals began withJoseph Fourier's seminal work [20] on the heat equation. His method,essentially that of separation of variables, is applicable to a variety of partialdifferential equations of importance in physics and engineering - mostnotably the wave and the steady state heat (Laplace's) equation. Althoughseparation of variables nowadays is a routine computational tool, it is thehistorical reason for the great interest in trigonometric series over the lasttwo centuries. We thus begin with an informal and abbreviated discussionof a simple case of this technique; one that effectively motivates the studyof the convergence and existence of Fourier series.Consider a thin circular ring. To keep the notation simple, we will

2 CHAPTER 1. PERiODIC FUNCTIONS

(1.1)

take the length to be 27T. Let () be a variable that designates the positionon the ring. Of course, () and () + 27T give the same location on the ring.The temperature at time t and location () is to be denoted by f(t,O). Asimplified form for Fourier's equation for the flow of heat in the ring is

of o2fot 002

If the initial temperature distribution fo(O) = f(O,O) is given data, thisproblem is called the initial value problem for the heat equation. Observethat the functions

Fn(t,0) = exp(-n2 t)cos(nO)and

Gn(t, ()) = exp(-n2t) sin(n()) n>Oare all solutions of the heat Equation 1.1. Because of the linear natureof the heat equation linear combinations of Fn and Gn are also solutions.Such a solution has the form

n n

where the coefficients an and bn are coefficients. Fourier's idea was toexpand the initial temperature distribution in a trigonometric series in orderto guess these coefficients. Indeed, if

(1.2)

then

n n

n n

is a solution of the initial value problem. (As we will soon see, it is possibleto guess integral formulas for the an and bn.)The program outlined above begs the question of when periodic func-

tions can be expanded in series of the form 1.2. This issue was raised soonafter Fourier first made his ideas public and motivated the development ofanalytic tools well into the twentieth century (see [23]). In this chapter, wepresent an introduction to the modern theory of existence and convergenceof such trigonometric series which is a result of these efforts.Now we proceed to give a nonrigorous derivation of formulas for the

coefficients an and bn in Equation 1.2. However, some changes in notation

1.1. THE CHARACTERS 3

are warranted before we do this concisely. In our general development, wewill study complex valued periodic functions f : R -4 C. Unless otherwisestated, the period will always be taken to be 21r:

f(O + 21r) = f(O) for all O.As the reader can easily check in any particular circumstance, there is noloss of generality in this restriction; formulas for functions with a differentperiod follow easily from this case. It will be algebraically simpler to studyexpansions of periodic functions as linear combinations of the functions

en(O) = exp(mO)as opposed to the functions

nEZ

cos(nO) and sin(nO).Of course, the two schemes are completely equivalent.Of course, it is suggestive to regard 0 as an angle in radians. As such,

odetermines a location on the unit circle in the complex plane

T = {zlz E C and Izl = I}via the formula z = e'o. Since 0 and 0 + 21r both determine the samelocation on T, a periodic function can thus be effectively visualized as afunction on T. Note that if f is a function on T, the integral

l a+21r f(O) dOis independent of a. We will adopt the practice of denoting any such integralas 1f(O) dO.With this preparation, we are in a position to give a working definition

for the Fourier series of a periodic function. Given a function f : T -4 C,we are seeking an expansion of the form

00 00

f(O) = L cnen(O) = L en exp(mO). (1.3)n=-oo n=-

To guess formulas for the coefficients en, we will work formally. We startwith the identity

ifn = m, andotherwise.


By multiplying through Equation 1.3 by emU}) and integrating,1 r -

em = 271" iT f(O)em(O) dO.This nonrigorous calculation motivates the definition of the Fourier coeffi-cients of a periodic function I as

The Fourier series of I is given by00

S[/](O) = I: !(n)e,no.n=-oo

(1.5)

We reiterate that, at this point, these definitions have just a formal signif-icance. After all, the above argument assumes the existence of the seriesexpansion 1.3 which is by no means assured. In addition, we have not yetdiscussed conditions on I which insure that the Fourier coefficients are de-fined. Even after this relatively simple point is clarified, there will still beissues of when and in what sense the series in 1.5 converges to the functionf. All of these objections will take the better part of this chapter to clarify.

1.2 Some Tools of the TradeThe correct integrability conditions needed for expanding periodic functionsin Fourier series are expressed in the definitions of the LP(T) spaces. Inthis section, we provide a brief overview of the central results concerningthese spaces. With one exception, the proofs have been omitted. For thesedetails and further background, we refer the reader to books by Wheedenand Zygmund [67], Riesz and Nagy [53] and Rudin [57].Let L l (T) denote the collection of all measurable functions I defined

on T that are integrable on the unit circle:

1I/IILl := 2~ ll/(O)! dO < 00.More generally, if p E [1,00), the space of functions P(T) consists ofmeasurable functions I whose p-th power is integrable. The norm of afunction in this class is

1I/11LP := (2~ ll/(O)IP dO) lip

1.2. SOME TOOLS OF THE TRADE

To include the p = 00 case, it is useful to define the set

5

E>.(f) = {O : If(O)1 ~ .x}These sets are measurable, and so

for any measurable function f.

m(E>.(f)) = ~ r dO211" JE>.(f)n[-7r,rr]

denotes the Lebesgue measure of the E>.(f) as subsets of T. The classLOO(T) is defined as the set of essentially bounded measurable functions onT:

f E L ooThe Loo-norm is

supP :m(E>.(f)) =f. O} < 00.

IIfllLOO = supP :m(E>.(f)) =f. O}.The basic properties of this class of functions are summarized in the

following three propositions.

Proposition 1.2.1 Fix p E [1,00], and let f,g E LP(T) and.x E C. Then1. IlfllLP ~ 0 with equality if and only if f == 0 almost everywhere,2. IIf + gllLP :::; IIfllLP + IlgIILP, and3. lI.xflILP = 1.xlllfIlLP.

The preceding proposition implies, in particular, that LP(T) is a normedvector space.

Proposition 1.2.2 Suppose fi is a Cauchy sequence of functions in LP(T)Le., Ilfi - hIILP(T) ~ 0 as i,j ~ 00. Then there is a f E LP(T) such thatIIf - filiLP ~ 0 as i ~ 00.Propositions 1.2.1 and 1.2.2 together state that the vector space LP(T)

forms a Banach space under the norm II II LP.Define the translation of a function f(O) on T by 00 E R by

Toof(O) = f(O + (0 ),If f E LP(T), Toof E LP(T) and

IlTooflb = IIflbThe following proposition states that translation is continuous with respectto the LP norm, provided 1 :::; p < 00.


Proposition 1.2.3 If f E LP(T) and 1 ~ p < 00,

Proposition 1.2.3 does not hold when p = 00. Indeed, if f(O) is theindicator function of the interval [-7f/2, 7f/21, then Ilf -lOo fll'x, = 1 for allsufficiently small 00 . This anomaly gives rise to much of the complicationsinvolving the borderline p = 1 and p = 00 cases in Fourier analysis on thecircle.Define the conjugate exponent of p E [1,001 by the equation ~ + 1. = 1.

In particular, the conjugate exponent of 1 is 00 and vice versa. % alsoobserve that 2 is its own conjugate exponent.Proposition 1.2.4 (HOlder's Inequality) Let p and q be conjugate ex-ponents. Then if f E P(T) and g E Lq(T),

One simple consequence of this result is an inclusion result for the LPspaces.

Corollary 1.2.1 Suppose 1 ~ Pi ~ P2 ~ 00. Then, for any f E p2 (T),

and hence LP2 C LPI .

PROOF: We make the following computation, using Holder's inequalitywith the conjugate exponents P2lPi and P2/(P2 - pd:

Ilflltpi 2~ llf(O)IPi dO= 2~ lllf(O)IPi dO< (2~ llf(0)IP2 dO) Pr!P2 (2~ l dO) (P2-pd/PI

Ilfllt~2'The result is immediate.D

1.2. SOME TOOLS OF THE TRADE 7

A basic tool in harmonic analysis is duality. Let B denote a Banachspace with norm II- liB. The dual space of B, denoted by B*, consists ofall bounded linear functionals A : B ---t C. The norm on B* is defined by

IIAIIB* = sup{IA(v)1 :vE Band IIvllB = I}.The vector space B* endowed with this norm is itself a Banach space.Objects in the dual space of various function spaces arise naturally in manyof the important arguments in Fourier analysis. As such theorems thatexplicitly identify the dual spaces of various function spaces are extremelyvaluable.Holder's inequality is really a statement about dual spaces. In fact, if p

and q are conjugate exponents and g E Lq(T), proposition 1.2.4 guaranteesthat the linear functional

is well defined and bounded. The following result characterizes all elementsin the dual space of LP as arising from this construction.

Theorem 1.2.1 Let p satisfy 1 ~ P < 00 and q be its conjugate exponent.Then the dual space of LP(T) can be identified with Lq(T). In particular,if A is an element of the dual space of LP(T), there is a


Theorem 1.2.2 Let >'k be a bounded sequence in B*, Le., a bounded se-quence of linear functionals on a Banach space B. Then there is a subse-quence >'ki and a linear functional>' E B* such that, for any v E B,

as i --+ 00. (1.6)

A sequence of linear functionals converges weakly if the condition inEquation 1.6 holds.The convolution of two measurable functions f, 9 is defined by

f *g(O) = ~Jf(O - w)g(w) dw,21rprovided the integral converges for almost all O. This process is not al-ways well defined, although it certainly is, for example, when f and 9 areboth continuous. Young's theorem provides two other extremely importantsituations when convolution makes sense.

Theorem 1.2.3 (Young's Theorem) Suppose f,g E L1(T). Then, f *9 E L1(T) and

If f E LP and 9 E Lq, then f*9 E LT, where ~ +*+ ~ = 1. Moreover,

1.3 Fourier Series: V TheoryLet f E U(T). As we have seen in the previous section, LP(T) C L1(T).This, together with the observation that Ilenllux> = 1, justifies the estimate

Hence the Fourier coefficients of f E U(T), defined by

j(n) = ~ {21r f(O)e- tnO dO,21r Jo (1.7)

1.3. FOURIER SERIES: LP THEORY

exist. Thus, at a formal level, we define the Fourier series of 1 as00

8[/](0) = I: j(n)etn8 .n=-oo

9

(1.8)

The question of when and in what sense the above series does actuallyconverge to 1 is one of the central questions of Fourier analysis. Define thepartial sums of the series 1.5 by

N

sN(f,O):= I: j(n)etn8 .k=-N

These partial sums need not converge in the L1-norm to f. Even moresurprising is the fact that the partial sums, SN(f, 0), need not convergeuniformly to 1(0) even when 1 is continuous. Given this situation, onecan ask if convergence of the partial sums holds in some weaker sense.Norm convergence of the partial sums is true in the LP spaces as long asp E (1,00), the case of p = 2 being of particular importance. It is naturalto investigate the almost everywhere pointwise convergence behavior of thepartial sums. In an extremely difficult paper [9], L. Carleson proved thatfor 1 E L 2 , the partial sums do converge to 1 almost everywhere. Thisresult was extended by R. A. Hunt [30) to the case 1 E P, where p > 1.Given this complex of results, it is natural to wonder if stronger conclusionscan be drawn if one imposes the additional condition of continuity. Kahaneand Katznelson [31) have shown that given a set of measure zero, Z c T,there is a continuous function 1 : T --+ C whose Fourier series diverges atevery point of Z. In this presentation, we will skirt around the difficultissues arising in convergence questions and present those results that arenecessary in most applications of Fourier series. The reader interested inthe deeper theory of convergence of Fourier series should start with thebook of Katznclson [321 and then the book by Mozzochi [42).A viable theory of LP convergence can be achieved if the method by

which the series is summed is modified. Define the n-th Cesaro mean ofthe partial sums of 1 by

1 NN + 1 I: sk(f; 0)

k=ON

" (1 _ _ 1_kl_)j(k)etk8 .k~N N+1

(1.9)

Evidently, (FN (f; 0) is an average of the first N + 1 partial sums.

10 CHAPTER 1. PERIODIC FUNCTIONS

The following two lemmas are essential to understanding the behaviorof the aN(Jj 0) as N - 00.Lemma 1.3.1 If / E LP(T),

aN(J,O) = 2~JKN(O - w)/(w) dw,where

1 { sin( lY.lO) }2K (0) - 2N - N + 1 sin( ~)

(1.10)

(1.11)

PROOF: Expanding the formula 1.11 using complex exponentials yields theformula

KN(O)_1_ {exP(~O) - exp(_~O)}2N + 1 exp(~~) - exp(-~~)

= _1_ {e-tlfoeXP(~(N + 1)0) - 1}2N + 1 exp(~O) - 1

= _l_le-tNO (t etkO)2N+ k=O1

N + 1 (e_N(O) + 2e_(N-l)(O) + ... + Ne-l(O)+(N + 1) + Nel(O) + ... + eN(O)).

A direct computation easily verifies Equation 1.10. 0The periodic functions K N(0) are the Fejer kernels.

Lemma 1.3.2 For any 0 > 0,

as N - 00.

PROOF: Using the computation in the proof of lemma 1.3.1,

1.3. FOURIER SERIES: L P THEORY

eFigure 1.1: The Fejer kernels K N for N = 1 ... 4

11

On the other hand, Isin( N:i1B)1 is bounded from above by 1, and Isin(B/2)Iis bounded from below by a positive number on any interval of the form[-11", -01 U [0,11"1. Hence KN(B) -+ 0 uniformly on any such interval. Theresult follows. 0Thus the functions Kn(B) are positive functions of unit L1 norm that

are progressively more concentrated near the origin as N -+ 00. See figure1.1 for a plot of the first few Fejer kernels.Theorem 1.3.1 Suppose 1 :=:; p < 00 and f E U(T). Then aN(f; B) -+ fin U(T).PROOF: The integral we are required to estimate is

Since the Fejer kernels are positive, the inner integral above can be writtenas


Holder's inequality together with the fact that the K N are a summabilitykernel imply

(1.12)

whereP(w) = i: If(O) - f(O - w)iP dO.

The function P(w) is continuous at w = 0, by proposition 1.2.3. Fix > 0and choose h > 0 such that 1P(w)1 < for Iwl < h. The last quantity in theestimate 1.12 can be bounded above in the following manner:

~ (2 1)2 ( r KN(w) dw + M r KN(w) dw) .11" Jr-O,oj J[-o,W

where M is a bound for the positive continuous function P on [-h, h]c.Letting N -+ 00 shows that

1 J11" 1limsup -2 If(O) -aN(fjO)IPdO ~ (2 )2'N -+

1.3. FOURIER SERIES: L P THEORY 13

Remark 1.3.2 The properties of the kernels K N essential to the precedingdevelopment are

1. for each N, K N (()) ~ 0 for all w,

2. for each N,

and

3. for each fJ E (0,11'),

as N ~ 00.

Similar results hold for any other kernel with these properties.

Families of kernels with the properties in the preceding remark are calledsummability kernels. The quintessential example (other than Fejer) of asummability kernel is the Poisson kernel

P(r ()) _ 1 - r2, - 1 - 2cos(())r + r 2 O


1t-1t eFigure 1.2: Plot of the Poisson kernels for r = 1/4,3/8,1/2,5/8.

PROOF: Note that1CTn(J; ()) = -1 (so(J; ()) + ... + sn(J; ())). (1.13)

n+

Given t > 0, there is an No such that

for all n ~ No. Using the triangle inequality, we then estimate

II! - gil::; II! - CTn(J; ())I! + IICTn(J; ()) - gil (1.14)In view of theorem 1.3.1, the first term on the right tends to zero as n --+ 00.In particular, II! - CTnll can be estimated from above by t, provided n issufficiently large. The last term can be estimated, by using Equation 1.13and the triangle inequality:

1 nIICTn(J; ()) - gil :::; n + 1 L Sk(J; ()) - g

k=O

< n: 1 (~>.(f; 9) - g)1 n

< n + 1 L II sk(J; ()) - gilk=O

1.4. FOURIER SERIES: L2 THEORY 15

n,

(I - g, en) (I, en) - (g, en)= (SN(Jj ()), en) - (g, en)

(SN(JjO) - g,en ),for N sufficiently large. On the other hand,

Since the last term on the right tends to zero as N ~ 00, (I - g, en), whichis independent of N, must actually vanish. Since n was arbitrary, I - 9 isperpendicular to the span of the en' Then proposition 1.4.2 implies that1- 9 = 0 or I = g.oCertain immediate consequences of theorem 1.4.1 are crucial to point

out.

Theorem 1.4.2 (Parseval's Theorem) II I,g E L 2 (T), then00

(I,g) = L (I,en) (en,g).n=-oo

In the case where I and 9 coincide,00

11/112 = L 1(I,en )12 n=-oo

PROOF: Observe that since IABI ~ (1/2)(A2 + B 2 ),

(1.15)


Bessel's inequality then implies that the series in equation 1.15 convergesabsolutely. Furthermore, a direct calculation confirms the identity

N

(SN(f, B), SN(g, B)} = L (f, en) (en, g).n=-N

On the other hand, we may write

The Cauchy-Schwarz inequality justifies the estimate

l(f - sN(f,B),g} - (SN(f,l)),SN(g,B) - g}1::; l(f - sN(f,B),g}1

+ I(SN(f, B), SN(g, B) - g}1< IIf-sN(f,B)lIl1gll

+lIsN(f, B)II IIsN(g, B) - gilLemma 1.4.1 in conjunction with theorem 1.4.1 forces this last quantity totend to zero as N --t 00. Thus, Parseval's identity (Equation 1.15) holds.The second identity follows by substituting f for g.ORemark 1.4.1 The last three results use very specific pieces of informationabout the functions en, namely, that

for all j, k,

and that the ej span a dense subspace of L2 (T). Thus analogous resultshold for any sequence offunctions in L2(T) that satisfy these two properties.This principle is of sufficient importance to cast in the following generalform. The proof is a verbatim repetition of the previous three arguments.Such a subset of Hilbert spaces is called an orthonormal basis.

Theorem 1.4.3 Let H be a separable Hilbert space and Vn a sequence ofvectors in H satisfying

Suppose, moreover, that the span of the Vn are dense in H.

1.5. FOURIER ANALYSIS OF MEASURES 19

For any element U E H, the vectors SN = Z=:=_N(U, vn)Vn convergeto u .

For vectors UI, U2 E H,

n

Here, we have assumed the vectors V n are indexed over the integers. Ofcourse, there is no problem if the index set is the natural numbers instead.

1.5 Fourier Analysis of MeasuresLet C(T) denote the space of continuous functions on T. The norm onC(T) is defined by

II/I1C(T) = sup 1/(0)1C(T) is a Banach space under this norm. A measure fL on T is a complexreal valued linear function on C(T) such that there is a constant K, > 0such that

IfLU) I ::; K,lI/l1c(T)for all IE C(T). The set of all measures on T forms a vector space denotedby M(T). A measure fL E M(T) is positive if flU) ~ 0 whenever 1(0) ~ 0for all () E T.Example 1.5.1 (Point Masses) Fix 00 in T. Let 1500 : C(T) -+ C denotethe linear function defined by

for any I E C(T). It is straightforward to check that 1500 E M(T) and ispositive. It is called the unit point mass at ()o.

Example 1.5.2 (Absolutely Continuous Measures) Given that w(O)is in L 1(T), the correspondence

11---+ I(O)w(O) dOdefines a linear mapping from C(T) to C. Moreover, the estimate

1 1()w(O) dol::; sup III IIwllLlproves that this correspondence defines a measure.


The Fourier series of a measure /-L E M(T) is defined by00

n=-(X)

Proposition 1.5.1 If /-L, 1/ E M(T) are measures with the same Fourierseries, then /-L = v.

PROOF: By considering the measure /-L - v, it is enough to show that theonly measure whose Fourier coefficients are all zero is the zero measure. Let/-Lo be such a measure. Let f E C(T) and Cn be the n-th Fourier coefficientof f. Note that

N IN +1- kl/-LO(O"N) = L N + 1 Ck/-Lo(ek) = 0,

k=-Nwhere O"N is a temporary abbreviation for the N-th mean O"N(J; 0). On theother hand, since O"N(J; 0) -+ f(O) in the C(T)-norm and

l/-Lo(J) -/-Lo(O"N)1 = l/-Lo(J - O"N)I ~ Csuplf - O"NI,T

/-LO(O"N) -+ /-Lo(J) as N -+ 00. Consequently, /-Lo(J) = 0 for all f E C(T).The proposition follows.DThe main result in the section provides a characterization of the Fourier

series of positive measures.

Definition 1.5.1 A sequence an of complex numbers is positive-definiteif for any sequence of complex numbers ~n E C with only finitely manynonzero terms,

L an-m~n~m 2: o.Proposition 1.5.2 Suppose /-L E M(T) is a positive measure. Then an =/-L(en) is a positive-definite sequence.PROOF: Let ~n be a sequence of complex numbers, with finitely manynon-zero terms. Set

ifJ(O) = L~nen(O).Then, lifJ(OW is a positive function in C(T), and

o < /-L(lifJI 2 )

~ ~ (~Mmen-m(O))L /-L(en-m)~n~m.nm

1.5. FOURIER ANALYSIS OF MEASURES 21

The result follows.DThe following theorem provides a converse to proposition 1.5.2.

Theorem 1.5.1 (Herglotz) Suppose an is a positive-definite sequence.Then, there is a unique positive measure JL E M(T) such that JL(en) = an'PROOF: Let

~ _ {e_n(B) if Inl ~ Nn - 0 otherwise.

Then,L an-m~n~m = L G:kake-k(B)nm k

where G:k is the number of ways k can be written as the difference of twointegers that are less than N in absolute value. A simple calculation showsthat G:k = max(2N + 1 - Ikl, 0). Consequently,

L G:kake-k(B) = (2N + 1)0"2N(a, 0) ~ 0,k

where O"N(a, 0) denotes the average of the first N + 1 partial sums of theseries Eane_n(B). Note also that

l 0"2N(a, B) dB = aofor all N. Define a linear functional JL on the vector space of trigonometricpolynomials by

M

JL(P) = L cnan,n=-]I,,[

where P(O) = E~=-M cnen(O). Note thatJL(P) = 2- lim rP(B)0"2N(a; B) dO.

271' N-+oolT

Now, Ii P(O)0"2N(a; B) dBI ~ sup IPlao.Since the bound on the right is independent of N,

IJL(P)I ~ CsuplPISince trigonometric polynomials are dense in C(T), it follows that J.L extendsto a bounded linear functional on C(T). The only point remaining is to


check that JL is a positive measure. Suppose f E C(T) is a nonnegativefunction. As U2N (f j 0) converges uniformly to f,

JL(f) = lim JL(U2N(fj 0))N-+oo

1 2N= lim 2N "max(O, 2N + 1 - Ikl)(f, ek)ak

N-+oo + 1 L.."k=-2N

= 21lim (f(O)u2N(aj 0) dO.

11" N-+oo iTClearly, each term in the last limit is nonnegative, and hence JL is positive.The uniqueness of JL follows directly from proposition 1.5.1.0

1.6 Smoothness and Decay of Fourier SeriesOne of the basic themes regarding Fourier series is that the smoothness ofthe function is reflected in the decay of the Fourier coefficients at infinity.The Riemann-Lebesgue lemma, given below, is a basic result along theselines.

Theorem 1.6.1 For f E L 1(T), j(n) -+ 0 as n -+ 00.PROOF: Since uN(fj 0) converges to f in Ll(T), the Fourier coefficientsof uN(f; 0) must tend to those of f uniformly. Since each uN(f; 0) hasonly finitely many nonzero Fourier coefficients, j(n) must tend to zero asn -+ 00.0Let Ck (T) denote the class of functions on the circle that have contin-

uous derivatives up to order k.

Theorem 1.6.2 Let f E Ck(T) and

Cn =.2-- { f(O)e- m8 dO211" iTbe the corresponding Fourier coefficients. Then

where (an) E (2.PROOF: The k-th derivative of f is a continuous function and hence in L2 .As a consequence,

1.7. TRANSLATION INVARIANT OPERATORS 23

is in e2 By integrating the above equation by parts and using the period-icity of f(k),

ak = (m)~ rf(k-l)(())e-m8 d().27T iT

The computation can be iterated k - 1 more times to produce the equationin the above theorem. 0

1.7 Translation Invariant OperatorsAn bounded operator T: LP(T) -+ Lq(T) is translation invariant if for any()o E T,

for all f E LPT.Although, the general case is still not well understood, simple characteri-zations of such operators in two special cases, p = q = 1 and p = q = 2, aregiven in this section.

Lemma 1.7.1 Let T : LP(T) -+ Lq(T) be a bounded operator where 1 :::;p, q < 00. T is translation invariant if and only if there exists a series ofnumbers tn such that Ten = tnen for all n E Z. In this case, Itnl :::; IITIIfor all n.PROOF: Suppose, T is translation invariant. For any ()o E T and integern, Tooen = e-t8o en. Upon applying the operator T to both sides,

e-m80Ten = T(e-mOOen) = T(Tooen) = TooTenfor all 00 E T. For the time being fix n, and set 9 = Ten. Then precedingcalculation implies that

e-m801g(O)e-k(()) d() = 1g(O - ()O)e-k(()) d()= e-tk801g(())e-k(O) d().

As a consequence,

for all integers k,

where Ck is the k-th Fourier coefficient of the Lq function g. Since ()o is alsoarbitrary, Ck must be zero for any k distinct from n. Thus we have arguedthat

for all integers n. (1.16)


Now we turn to the converse. Due to Equation 1.16, for any finitetrigonometric polynomial I = I::=-N akek,

N

79oT/(O) = 790 L tkakek(O)k=-N

NL tkakek(O + (0 )k=-N

NL tkaketk60 ek (0)k=-NT(79o f)(O).

Recall that polynomials are dense in P for p E [1,00). Hence an arbitraryI E LP is the limit of finite trigonometric series In. The observations that

andTIn -+ TI in Lq

follow from the continuity of 790 : LP -+ LP and T. Applications of thecontinuity of T and 790 : L'J -+ Lq imply,

T790 In -+ T790 I in LPand

Since T790 In = 790TIn, the same relationship follows for IThe last bound in the lemma follows easily from the definition of the

operator norm. 0First we give a construction of translation invariant operators when

p = q = 1. An argument will then be given on how all such operators arisein this fashion.Let IE C(T) and J.L E M(T). Note that

1* J.L(O) =L1(0 - w)dJ.L(w) = J.L (7_6])where ](w) = I( -w). Since T is compact, the map

01-+7_6 ]

(1.17)

1.7. TRANSLATION INVARIANT OPERATORS 25

is a continuous map from the real line to C(T) (with the uniform topology).In view of Equation 1.17, 1* J.L E C(T). The estimate

L II * J.L((}) Id(} ::; JJ1/(0 - w)ldlJ.Ll(w)d(}::; 11/1IullJ.LIIM

holds due to Fubini's theorem and the translation invariance of the L 1_norm. Since C(T) is dense in L 1(T), the mapping

extends to a bounded linear mapping from L 1(T) to itself. It is easy tocheck that for any I E C(T), Teo I * J.L = TeoU * J.L). Then another densityargument shows that the above condition also holds for I E L 1 (T).Theorem 1.7.1 Let the bounded linear operator T : L 1(T) -+ L1(T) betranslation invariant. There exists a unique measure J.L E M(T) such that

TI = I*J.L

PROOF: For any fixed E C(T) C UXJ, consider the bounded linearfunctional defined by

>.U) = LTI((})(-(})d(}

LT I ((})((}) d(}It is straightforward to calculate that 11>'11 ::; IITII 11110. Since the dualspace of Ll is Loo, the linear functional>. can be expressed as integrationagainst a unique Loo function. This function can be expressed as 1/;( -(}),so

Moreover,

>.U) =J1((})1/;( -(}) dO for all I ELI.

Set S = 1/; and note that S : Loo -+ Loo is a bounded linear transformation.If = en and I = em,


f em(()S(-() d()f e_m(()S(() dOem,

where em now denotes the m-th Fourier coefficient of S. Hence,

for all n.

In particular, S = T on the dense space of finite trigonometric series. Thenanother simple density argument shows that the restriction of T to C(T)maps boundedly to C(T). Consider the measure defined by

J.L(J) = Ti(O)Then for any continuous function f,

for all f E C(T).

Tf(() (79Tf) (0)(T79f) (0)(T'Loi) (0)

= J.L(T-oi)J.L * f(()

The theorem follows.OThe other case where transla.tion operators can be explicitly identified

will be treated next: p = q = 2.

Theorem 1.7.2 An operator T : L2 (T) --+ L2 (T) is translation invariantand bounded if and only if

(1.18)for some t = (tn ) E iOO(Z). Moreover, in this case, IITII = IItllioo PROOF: Suppose T is bounded and translation invariant. Then, lemma1.7.1 insures that Equation 1.18 holds for some sequence t n E C that sat-isfies the following bound

Itnl S IITIIIn particular, IItllioo S IITII.

for each n.

1.8. PROBLEMS 27

Recall that P is the vector space of all finite trigonometric series. Clearly,the functions en forms a basis for this vector space. To establish the con-verse, define a map T : P ~ P by Equation 1.18, where t n is some boundedsequence of complex numbers. Let

N

f(O) = I: Ckek(O)k=-N

be a finite trigonometric series. Then,N

Tf(O) = I: Cktkek(O)k=-N

and

N

< Iltll~oo I: ICkl 2k=-N

- Iltll~oo IIfIlI2.Since P is dense in L2, standard density arguments show that T extends toa bounded operator on L2 . By construction, T is translation invariant.O

1.8 Problems1.1 Let f(O) be a periodic function equal to (11" - 0)/2 on the interval

(0,211"). Use the Fourier series of f to verify the formula

1.2 Prove the identity

1 2~ (kO) = sin((n + 1/2)0)+ 6 cos sin(O/2)'1.3 Let f E Ll(T). Show that

Sn[J](O) = 2~Jf(O - w)Dn(w) dw

28

where

CHAPTER 1. PERIODIC FUNCTIONS

Dn

({}) = sin((n + 1/2){}).sin({}/2)

Is Dn a summability kernel? Why or why not? Show that

2~LDn(o) d{} = 1.(Dn ({}) is called the Dirichlet kernel.)1.4 Suppose F({}) is an integrable function on the interval [0,11"]. Provethat 1" F({}) sin((n + 1/2){}) d{}tends to zero as n -+ 00.

1.5 Let 1 : T -+ C be integrable and, in addition, differentiable at (}oShow that

lim Sn[/]({}o) = 1(00 ),n--+oo

1.6 Suppose 1 : T -+ C is differentiable everywhere and has a continuousderivative. Show that Sn[f] converges uniformly to 1 on T as n -+ 00.(Hint: Show E Icnl converges where the en are the Fourier coefficients ofI.)1.7 Prove theorem 1.3.2.

1.8 Let Cn be a doubly infinite sequence such that

n=-oo

Show that E:=-N cnen converges to a unique element in 2(T) as N -+ 00.1.9 Consider the space 2[0,11"] of square integrable functions on the in-terval [0,11"]. 2[0,11"] is a Hilbert space with inner product

11"(I, g) = - I({})g({}) d{}.11" 0

a. Show that the functions

1, y'2;cos({}), y'2;cos(2{}), ...

form an orthonormal basis for 2[0,11"].

1.8. PROBLEMS

b. Show that the functions

y"2; sin(O), y"2; sin(20), ...

also form an orthonormal basis for L2[0, 11"].1.10 Let j. E M(T) be a measure. Show that for any 1 E C(T),

2~11(0 - w)dj.(w)

29

is a continuous function of O. We designate this function as the convolutionof 1 and j. and denote it by 1 * j..1.11 Let j., v E M(T). Define the convolution of the measures j. and vby

for all 1 E C(T).a. Show that 1Ij. * vIlM(T) ::; 11j.IIM(T) IlvIlM(T).b. Show that j. * v = v * j..c. Let {j be the Dirac measure concentrated at 0 = O. Show that j. * {j = j.

for all measures j..Prove that the Fourier transform of j. * v is p,. D.1.12 A function 1 E L 1(T) has an absolutely convergent Fourier series if

L Ij(n)1 < 00.n

Prove that if 1 and g both have absolutely convergent Fourier series thenso does I g. Show also that

(2.1)

Chapter 2

Hardy Spaces

This chapter will treat the study of Fourier series whose coefficients withnegative index are all zero. From the standpoint of applications, such se-ries are extremely useful due to connections with causal filters and controltheory (see chapter 4). The mathematical richness of the topic arises fromthe interplay between such series and certain classes of analytic functionson the unit disc.

2.1 Hardy Spaces and Invariant SubspacesDefinition 2.1.1 Let 1 ::; P ::; 00. The Hardy space HP consists of thosefunctions f E V(T) such thatI: e'k8 f(O) dO = 0for all k > o.As is straightforward to verify, HP is a closed subspace of LP, and so is

a Banach space under the norm

IIfllHP := IlfllLPoFunctions in HP consists of all those functions in f E LP whose Fourier

series expansion has the form

32 CHAPTER 2. HARDY SPACES

Thus the definition 2.1.1 of HP functions consists of two pieces: an LP inte-grability condition and a requirement on the Fourier coefficients (equation2.1). The interplay between these two aspects is quite subtle, since there isno simple characterization of LP functions in terms of their Fourier coeffi-cients. A notable exception is, of course, when p = 2, where we have theParseval's theorem.Now we discuss the structure of certain types of invariant subspaces of

L2(T) and find that the Hardy spaces H2 and Hoo arise rather naturally.What follows will also be central to our exposition on control theory inchapter 4.The Hilbert space L2 comes with a natural isometry called the shift

operator:Sf(()) = e"ef(())

Clearly,(Sf,S9) = (1,9) for all f,9 E L2

A nontrivial closed subspace V of L2(T) is called shift invariant ifSV := {Sf: f E V} C V.

Examples of shift invariant subspaces are not hard to give.

Example 2.1.1 For a given f E L 2 (T), define AI as the smallest, closedsubspace of L 2 that contains

f, Sf, S2 f,, sn f,.So the set of functions of the form

where P(z) is an analytic polynomial form a dense subspace of At. It iseasy to verify that At is a shift invariant subspace.

Example 2.1.2 Let E C T be a measurable subset, and let

V = {f E L 2(T) : f(()) = 0 a.a. () E E e }.The vector space V is a shift invariant subspace of L2.

The following result states that the preceding two classes of examplesof invariant subspaces are the only ones.

Theorem 2.1.1 Let V C L2 be a nontrivial closed subspace that is shiftinvariant. Then, either

2.1. HARDY SPACES AND INVARiANT SUBSPACES

1.

33

V=A",

for some 4> E Loo with the property that 14>(0)1 = 1 for almost all 0,or

2.

v = {f E L2 : f(O) = 0 a.e. 0 E E e }for some measurable subset E cT.

PROOF: The analysis depends on whether BV = V or if BV is a propersubset of V.

If BV is a proper subset of V, there must be a function 4> E V, of unitnorm, that is orthogonal to BV. As a consequence,

0= (4), Bn 4 = ~ [ e-mo l4>(OW dO21r iT

for all integers n > O. Since 14>(oW is a real-valued function, all of its Fouriercoefficients with a non-zero index must vanish. This says that 14>(OW is aconstant. Since 4> has unit norm, 14>(OW = 1 for almost all O. Since 4> is anelement of the shift-invariant space V, the subspace generated by it mustbe a subset of V: A", C V. Suppose there is a nonzero function 'l/J E Vperpendicular to A",. Then

for all n 2: O.

Once again using the fact that 4> is orthogonal to BV and that B is anisometry,

for any n > O. In other words,

2~ l emo4>(0)'l/J(0) dO = 0for all integers n. Hence 4>i/J == O. Because 14>(0)1 == 1, 'l/J must be the zerofunction. This contradiction forces us to concede that A", = V when BV isa proper subspace of V.Still left is the case when BV = V. As is easily checked (see problem

2.1), under this assumption, B(Vol) = Vol. Thus, if f E V and 'l/J E Vol,we must have


for all integers n. So all of the Fourier coefficients of the Ll function fi[;must be zero. In other words, if a function 1/J is orthogonal to V, then theset

{O: 1/J(O) - O} n {O : f(O) - O}has measure zero for every function f E V. Let

J. = supmeas(X),x

where X varies over all measurable sets X C T such that

1/J E V.L :::} 1/J(O) = 0 a.e. 0 E X.Any set of the form {O : (0) - O} where is a nonzero member of V, is anexample of such a X. As a consequence, J. > O. Now, let X k be a sequenceof such measurable sets such that

and set E = UkXk. We claim that for any E V, the set

has zero measure. Indeed, otherwise, the set

Y = {O : (O) - O} U Ewould have the incompatible properties

1. meas(Y) > J. and2. for any 1/J E V.L,

1/J(O) = 0 for all (} E Y.Therefore, 1/J E V.L if and only if 1/J(O) = 0 for almost all 0 E E. Since(V.L).L = V, we are led to the desired characterization:

V = { E L2 : (O) = 0 a.e. 0 E EC}.OIf the invariant subspace V is a subset of H2 , then SV must be a proper

subset ofV. Indeed, if SV = V, then S-1 V = V also. Thus for any nonzerof E V, s-nf(O) = e-m8 f(O) must be in V C H2 for every positive integern. As this is clearly impossible, the second possibility in theorem 2.1.1 canbe ruled out. In addition, the function is in Loo n H2. So E Hoo. Thuswe have the following theorem at our disposal.

2.1. HARDY SPACES AND INVARIANT SUBSPACES 35

Theorem 2.1.2 Let V C H2 be a nontrivial closed subspace that is invari-ant under the shift operator

SV = {Sf: f E V} C V.Then V = A.'/> for some J E HOC with IJ(O)I = 1 for almost all O.It is a straightforward exercise to check that At/> = {Jf : f E H2} in

the above result.The preceding theorem motivates the following definition.

Definition 2.1.2 A function J E HOC is called an inner function ifIJ(O)I = 1

for almost all O.Example 2.1.3 Functions of the form emo are clearly inner functions, pro-vided n ~ O.

Example 2.1.4 The analytic function

z-(F(z) = --_,1- z(

where ( E C and 1(1 < 1, is an inner function. This is left to the reader asproblem 2.3.

Example 2.1.5 It can be checked that a finite product of inner functionsis also inner.

Complementary to the idea of inner functions is that of outer functions.

Definition 2.1.3 A function u E H2 is outer if Au = H2.Theorem 2.1.3 Let f E H 2 Then f = Ju where J is an inner functionand u is an outer function. This factorization is unique up to multiplicationby unit complex numbers.

PROOF: The space AI is an invariant subspace of H2 By theorem 2.1.2,A I = At/> for an inner function J E Hoc. As a consequence, f = Ju forsome u E H 2 Our next goal is to show that u is outer. Now let 9 be anarbitrary function in H2. Then gJ E A.p = AI' Let Pn(z) be a sequence ofpolynomials such that Pn(etO)f -t gJ in the 2-norm. Since J is inner, it iseasy to verify that Pn (et8 )(fif -t gin 2. Since u = (fif, this says precisely


that 9 E Au. Since 9 was an arbitrary element of H 2 , the function u mustbe outer.The uniqueness of the factorization is next. Suppose that f = u = 'l/Jv

where , 'l/J are inner and u, v are outer. Since v is outer H2 = Av = A,p",u.Since u is outer, A,J;",u = A,J;",. So H 2 = A,J;"" and E H 2 . Since theroles of 'l/J and are symmetrical, we may also argue that 'l/J E H 2 . This isclearly a contradiction, unless is a constant. Since the values of 'l/J areof modulus one almost everywhere, and 'l/J are constant multiples of eachother. Then it follows easily that the same must be true for u and V.OExamples 2.1.3 and 2.1.4 both describe inner functions that arise natu-

rally as the boundary values of analytic functions on the unit disc. Thereis a general relationship between the Hardy spaces and certain analyticfunctions on the unit disc of the complex plane. In fact, this connectionwith analytic function theory is essential to the deeper understanding ofthe structure of inner and outer functions.

2.2 Boundary Values of Harmonic FunctionsA necessary tool for the analysis of Hardy spaces is the theory of harmonicand analytic functions. In this section, we cover some important growthestimates for such functions that are needed later.A harmonic function F(x, y), defined on an open region n c R 2 , is a

C 2-function that satisfies Laplaces equation:

In this chapter, the domain of definition will be the unit disc in the complexplane: n = D = {z E C : Izl < I}. The formula for the Laplacian in polarcoordinates will be useful in this context:

For any r E (0,1), define

Fr(O) = F(r cos(O), r sin(O)).

The theme of this section is that appropriate growth conditions on aharmonic function F(x, y) along circles of radius less than one imply theexistence of boundary values. Then such results will be applied to analyticfunctions on the unit disc, which are, of course, harmonic.

2.2. BOUNDARY VALUES OF HARMONIC FUNCTIONS 37

Proposition 2.2.1 Fixp E [1,00), and let F(x,y) be a harmonic functionon the unit disc D. The function

is increasing on r E (0,1).PROOF: Unfortunately, the proof is complicated by the fact that the func-tion ItIP is nondifferentiable at t = O. To account for this, a smoothingprocedure must be introduced. Set cP,(x, y) = (f + F(x, yW)p/2 where > O. Then cP, is a C2 function on D and, as will be verified by direct com-putation, a subharmonic function. Indeed, let F(x, y) = u(x, y) + w(x, y)where u(x, y) and v(x, y) are real-valued harmonic functions. Then

D..cP, = p(p - 2) (f + u2+ V2/P-4)/21IuV'u + vV'v112+P (f + u2+ v2)(p-2)/2 (lV'uI2 + lV'vI2)

P (f + u2 + v2)(p-4)/2 {(p - 2)luV'u + vV'vl2+(f + u2 + v2)(IV'uI2 + lV'vI2)}

P (f + u2 + V 2)(P-4)/2 AThe term A can be estimated from below by

A (p - 2) luV'u + vV'vl2 + (f + u2 + v2)(IV'uI2 + lV'vI2)> (p - 1) luV'u + vV'vl2 - 2uvV'u . V'v

+ u21V'v12+ v21V'u12

(p - 1) luV'u + vV'vl2 + luV'v - vV'uI2.In view of the assumption that p 2: 1, these computations show that D..cP, 2:O. Now Green's theorem implies

Jfor D..cP, = 2~r1;rcP,(rcos(O),rsin(O dO.Consequently,

;r1cP,(rcos(O), rsin(O dO 2: 0,and the function

r ~LcP,(rcos(O),rsin(O dOis an increasing function of r. Now, letting f -? 0 gives the result.OThe following proposition corresponds to the p = 00 case.

38

Proposition 2.2.2 The function

CHAPTER 2. HARDY SPACES

is increasing on r E (0,1).PROOF: The preceding proposition states that for any p 2: 1,

is an increasing function for r E (0,1). To establish the result, it will beenough to show that, for a fixed r, the integral

approaches M = sUPoIF(reO)1 as P -+ 00. Given 2: 0, let E be thesubset of T on which the continuous function Fr has modulus larger thanM(1 - E). Then

By taking p -+ 00,lim inf Ip 2: M(1 - E).p ....oo

On the other hand, it is straightforward to check that

for all p.

Since was an arbitrary positive number, the above assertion and hencethe proposition hold.OThe standard method of producing harmonic functions on the unit disc

with prescribed boundary values is via the Poisson kernel. This procedurecan be carried out under a variety of assumptions on the boundary values.The Poisson kernel is defined by

P, (0) _ 1 - r 2r - 1 - 2cos(O)r + r2

for r E [0,1) and all 0 E [-71",71"). If the denominator in the above expressionis written as


it is easy to verify that it is positive in the region of interest. Hence thedefining formula is valid in the region of interest and

for all r E [0,1) and all 0 E [-11',11').Lemma 2.2.1 The Poisson kernel satisfies the following properties:

for all r E (0,1), Pr(O) 2': for all 0, for all r E (0,1),

and

for every 6 E (0,11'),

211 Pr(O) dO ~ 1.11' [-6,61In other words, Pr is a summability kernel in the sense of remark 1.3.2.PROOF: See problem 2.4.0Lemma 2.2.1 together with the growth estimates in propositions 2.2.1

and 2.2.2 are the necessary tools for the proof of the ensuing three results.These assert the existence of boundary values of harmonic functions on theunit disc under various growth conditions natural to the HP spaces.

Theorem 2.2.1 Let u(O) be a continuous function defined on the unit cir-cle T. Set

F(rcos(O),rsin(O)) = 2~Lu(w)Pr(O-w)dw.Then F(x, y) is the unique harmonic function on D such that

as r ~ 1

uniformly on T . Moreover,sup IFr(O) I :::; sup lu(O)1OET OET

for all r E (0,1).


PROOF: The Poisson kernel Pr ()) is harmonic everywhere on the unitdisc, and therefore the fact that F(x, y) is harmonic can be verified bydifferentiating the defining formula. That Fr -+ U uniformly, as T -+ 1,follows from lemma 2.2.1 and remark 1.3.2. The uniform convergence ofthe Fr together with proposition 2.2.2 imply the last inequality. If F(x, y)and G(x, y) both converge uniformly to the function u on the unit circle,then h = F - G is a harmonic function such that hr converges uniformly tothe zero function as r -+ 1. A simple application of proposition 2.2.2 forcesh(x,y) == 0.0

Theorem 2.2.2 Let u()) E LP(T), where 1 S p < 00. Define F(x, y) by

Then F(x, y) is the unique harmonic function on the unit disc with

as l' -+ 1

in LP(T). Moreover,

for all l' E (0,1).

PROOF: The Poisson kernel Pr ()) is harmonic everywhere on the unitdisc, and therefore the fact that F(x, y) is harmonic can be verified bydifferentiating the defining formula. That Fr -+ u in LP(T) as l' -+ 1follows from lemma 2.2.1 and remark 1.3.2. This LP convergence of the Frtogether with proposition 2.2.1 imply the last inequality. The uniquenessstatement also follows by an argument analogous to that given in theorem2.2.1.0

Proposition 2.2.3 Suppose f..L is a complex Borel measure on the unit cir-cle T. Set

F(rcos()), rsin())) = Pr * f..L()).Then F(x, y) is the unique harmonic function defined on the unit disc suchthat Fr have uniformly bounded L1-norm and Fr ()) d() converges weakly tothe measure f..L as l' -+ 1. Moreover, one has the estimate


PROOF: Let (()) be a continuous function on T. Then

2~h(w)Pr * J.t(w) dJ.JJ = J.t(Pr *).As r --. 1, Pr * --. uniformly on T, and J.t(Pr * ) --. J.t(). Therefore,Pr*J.t converges weakly to J.t. Since theorem 2.2.1 implies that IlFr*IIC(T) S11llc(T),

IIPr * J.tIlM(T) S 1IJ.tIIM(T)'On the other hand, we know that Pr * J.t E L1(T) and that IIPr * J.tIlM(T) =IlFr * J.tIIl(T)' Thus the inequality in the proposition is true.Only the uniqueness of F subject to the stated conditions has to be

established. Suppose F and G are both harmonic functions such that

Frd(), Grd() --. J.tweakly as r --. 1 and such that Fr and Gr are both uniformly boundedin L1-norm. Then h = F - G is a harmonic function such that hr haveuniformly bounded i-norm and hr converges weakly to the zero measureas r --. 1. For any E C(T) with 11lIc(T) S 1, the function

hr * (()) = 2~hhr(() - w)(w) dJ.JJ = 2~h(() - w)hr(w) dJ.JJis harmonic on the unit disc. We claim that this function converges uni-formly to the zero function as r --. 1. The compactness of T allows us tochoose finitely many numbers ()l, ... , ()n such that any function of the form

w~ (() - w)can be approximated to within a given to by one of the form

w ~ (()i - w).Since hr converges weakly to the zero measure, there is apE (0,1) suchthat 1 > r > p implies

12~h(()i -W)hr(W)dJ.JJ/ S tofor all i = 1,, n. If M is the uniform upper bound of the L1-norms ofthe hr,

for all r sufficiently close to 1. The claim follows. By proposition 2.2.2,hr * == O. Since was an arbitrary continuous function, we must havehr == 0.0


Remark 2.2.1 Let p. E M(T) be a measure on the unit circle. Then

;7[' LemuPr * p.(O) dO = 2~LPr * en(O)dp.(O).On the other hand,

Hence,2~LemuPr * p.(O) dO = r1n1p.(en).

Thus, the Fourier coefficients of Pr *p. are determined naturally from thoseof the measure p..

The machinery just developed will be essential in relating the Hardyspaces to spaces of analytic functions on the unit disc.

2.3 Hardy Spaces and Analytic FunctionsWe begin by defining certain classes of analytic functions on the unit discby prescribing their rate of growth on circles centered at the origin.

Definition 2.3.1 For any p E [1,00), 'liP is the set of analytic functionsF(z) defined on the unit disc such that

11F111lP = sup 21 JIFr(O)IP dO < 00.O

2.3. HARDY SPACES AND ANALYTIC FUNCTIONS 43

Theorem 2.3.1 The normed space 1iP, P E (l,ooJ can be identified withHP. In other words, there is a linear bijection T : 1iP-+ HP such that

IITFIIHP = IIFII'HPfor all F E 1iP PROOF: Let F(z) E 1iPwhere P E (l,ooJ. The functions Fr(O) = F(rel8 )are bounded in LP(T). Under the restriction on p, the Banach spacesLP(T) are duals of the spaces Lq(T), where l/p + l/q = 1. By invokingthe property of weak star compactness, we may conclude that there is asubsequence rn -+ 1 and au E LP(T) such that Frn -+ u weakly as n -+ 00.In particular, for any integer k,

~ { FrJO)e- lk8 dO -+ ~ ( u(O)e-lk8 dO2~ iT 2~ iTas n -+ 00. Let

F(z) = ao +alZ + a2z2 + ...be the power series expansion of the analytic function F(z) at the origin.This series converges on the entire unit disc and uniformly on compactsubsets of the unit disc. The restriction of this formula to the circle ofradius r n shows that the Fourier coefficients of Frn are given by

~1F, (O)e -lk8 dO = a rk2 rn k ,~ T

where is it understood that ak = 0 if k < O. As a result, we l:now that theFourier series of u(O) is expressed by

u(O) = ao + alel8 + a2el28 + ....As we have already seen in remark 2.2.1, the Fourier series expansion ofPr * u is given by

Pr *u(O) = ao +alrelo + a2r2el28 + ... ,which is identical with that of Fr. When p < 00, theorem 2.2.2 enables usto conclude that Fr -+ u in LP(T) as r -+ 1. Then proposition 2.2.1 impliesthat

IIFII'HP= lIuliLP.We claim that this is also true for p = 00. If p = 00, theorem 2.2.2 enablesus to conclude that Fr -+ u in La(T), as r -+ 1, as long as 1 ~ a < 00.Proposition 2.2.1 yields only

l~a


in this situation. However, using the proof of proposition 2.2.2, we caneasily show that

as a --+ 00.

Thusas a --+ 00.

It follows again from the proof of proposition 2.2.2 that u E Loo. Thus ourclaim holds. Since the periodic function u is uniquely determined by F, wemay define T(F) = u. It is easy to verify that T is linear, and we havealready established that it is an isometry.The only remaining issue is ifT is onto. Let u E HP. The Fourier series

of u has the form

Since u E LP C L1, there is a positive constant K such that lenl ~ K forall n ~ O. Immediately, we find that the power series

00

F(z) = LCnznn=O

converges absolutely for any Izl < 1 and that F is analytic on the unit disc.Moreover, since Fr = Pr *u, there is a constant C such that

for all T E (0,1). If p < 00, theorem 2.2.2 assures us that

in LP. If p = 00 the same statement holds in the sense of weak convergence(see 2.2.3). In either case, T(F) = u.DThe proof of theorem 2.3.1 fails when p = 1 since L 1 is not weak-star

compact. Although the result is true in this borderline case, it is consider-ably harder to prove. In this section, we will be content with proving thatfunctions in HI at least have boundary values in the space of measures. Aproof of the fact that HI can actually be identified with HI will be givenin section 5.

Proposition 2.3.1 Let F E HI. There is a measure J.l E M(T) such that

2.4. THE STRUCTURE OF INNER FUNCTIONS 45

PROOF: The hypothesis that F E ?t1 means that the periodic functions Frare uniformly bounded in L 1(T). Since L1(T) C M(T) and

Fr are uniformly bounded in M(T). Now M(T) is a dual space:M(T) = C(T)*.

By applying the weak-star compactness property, we conclude that there isa sequence r n - 1 and a measure J.L such that

weakly in the space of measures. The Fourier coefficients of Frn mustconverge to the corresponding coefficient of J.L:

as n - 00. By following the method in theorem 2.3.1, we can identify Fras Pr * J.L.DBecause of our work in this section, we will automatically associate a

function in ?tv with its boundary value in HV and vice versa, provided that1 < p:::; 00.

2.4 The Structure of Inner FunctionsTheorem 2.1.2 on the structure of invariant spaces points to the importanceof inner functions. In the section, we will explore the structure of suchfunctions in more detail. The first step is to establish the rate at which thezeroes of an inner function approach the unit circle.

Lemma 2.4.1 Let 4> E H OO be an inner function with 4>(0) ::j=. 0 and let


The quotient if!(z)jBn(z) is a bounded analytic function on the closed unitdisc. Bn(z) is an inner function and is continuous on the closed unit disc.Consequently, as r / 1, Bn (ret8 ) is converging uniformly to a continuousfunction on the unit circle whose modulus is everywhere one. This impliesthat

Iif!(ret8) Is~p B

n(ret8 ) -t 1 as r / 1.

Hence, lIif!jBnllHoo = 1. By proposition 2.2.2, Iif!(z)jBn(z)1 ::; lor 1if!(z)1 ::;IBn(z)1 everywhere on the open unit disc. Then we specialize to the casez=Otoget

n

o< 1if!(O)\ ::; IBn(O)1 = II I(mlm=1

Since each (m is of modulus less than one, the product on the right isdecreasing in n. Therefore, the infinite product

converges to a positive number. An equivalent statement is that the seriesE 10g(l(ml) converges to a finite limit. Now for all 0 < x < 1 , log(x) 1. Moreover, it is easy to check that the Fouriercoefficients with negative indices of the limit must all be zero. Hence Fhas boundary values in HI. Thus T is a well-defined operator from 'HI toHI. The only remaining issue is whether T is surjective. Let f E HI. TheFourier series of f has the form

The coefficients are uniformly bounded, so the power series

defines an analytic function F(z) on the open unit disc. In addition, theequation F(re'O) = Pr * f also holds. Since

F E 'HI. It is clear from the preceding analysis that T F = f. 0

2.6 The Szego-Kolmogorov TheoremInner and outer functions provide the building blocks for both HI andH2 functions. Although we have completely characterized inner functions,the structure of outer functions is still left to be determined. This is ac-complished in this section by way of the Szego-Kolmogorov theorem 2.6.2,important in its own right.Let H6 = SH2 The vector space of all trigonometric polynomials of

the form

is dense in H6. Denote this space by Ao.

Theorem 2.6.1 Let h(O) be a nonnegative integrable function on the unitcircle. Then

exp (21 rlog(h) dO) = inf rhe'R(J) dO.1r iT fEAoiT

In particular, log(h) is nonintegrable if the right-hand side equals zero andis integrable otherwise.


(2.7)

PROOF: First we consider the case where log(h) is integrable. Let 9 be anintegrable, real valued function on the unit circle with the property that

1g(O) dO = O.Then Jensen's inequality implies that

exp (2~ lIOg(h) dO) = exp (2~1(log(h) + g(O)) dO) ~ 2~Lhe9 dO.So

exp (2~ llog(h) dO) ~ inf (2~1he9 dO) =: I (2.6)where the infimum is over all integrable, real-valued functions on the unitcircle with zero average value.We will first show that

1= inf ~ ( he~U) dO.IEAo 271" iT

Since the infimum on the right is over a smaller set, it is immediate that thisright-hand side dominates I. The reverse relationship will be establishedby an approximation argument. For a fixed, nonzero, integrable 9 with zeroaverage, let E = {O : g(0) > O} and

A = 2~1g(O) dO.Moreover, set

At = ~ ( min(g(O), N) dO271" iE

andAN = -21 ( max(g(O) , -N) dO.71" iT\E

Define

{min(g(O), N) if 0 E E

gN(O) = ~ max(g(O), -N) otherwise.The following properties of the gN are germane to the ensuing argument:

1. the gN are all in 00,

2. the average value on each gN is zero, and

2.6. THE SZEGO-KOLMOGOROV THEOREM 55

3. IT hegN dO -+ IT heg dO as N -+ 00.Only the last item needs any justification. On the set E, the gN are converg-ing to g monotonically from below. Therefore, by the monotone convergencetheorem, l hegN dO -+ l heg dO as N -+ 00.The observation that At/AN -+ 1 as N -+ 00 implies that

exp(gN (0)) -+ exp(g(0)) as N-+ 00

as N -+ 00.

for almost every 0 E T\E. Since 0 ~ exp(gn(O)) ~ 1 for almost all 0 E T\E,the Lebesgue dominated convergence theorem enables us to conclude

r hegN dO -+ r heg dOJT\E JT\E

By putting all this together, we have the last item.Now the Cesaro means KM * gN satisfy the following properties:

1. for fixed N, the K M * gN are uniformly bounded in M,

2. K M *gN -+ gN as M -+ 00 in the Ll-norm, and

3. each KM *gN is the real part of an element in .40.The first two items are straightforward consequences of our work in chapter2. Only the third item requires explanation. The function gn is real-valuedwith mean zero. Hence, if Cn are the Fourier coefficients of gN,

Co = 0 and

By using this as well as basic facts about the Fejer kernels, we calculate theFourier series of K M * gn:

~ M - Ikl + 1 tkO _ 210 (~ M - Ikl + 1 tkO)L.-t MIcke -:n L.-t M cke .k=-M + k=l + 1

Thus the third item.Now, in view of the second item above, we can choose a subsequence

M i such thatasi-+oo

for almost every O. By Lebesgue's dominated convergence theorem,

l heKMi*gN dO -+ l hegN dO


as i -+ 00. All this leads to the observation that the integral

Jheg dOcan be approximated arbitrarily well by an integral of the form

Jhe~(f) dO f E.4o.Consequently, 2.7 is true.The only point left to clarify is that the two sides in equation 2.6 are

the same. To this end choose > 0, and set .x = 1/21r Jlog(h + ) dO andg(O) = .x - log(h(O) + f). With this choice of g, we compute

2~J(h + )e g dO = exp (2~Jlog(h + ) dO) .Thus

exp (2~Jlog(h + ) dO) = f~nt 2~J(h + )e SRf dO.A simple limit argument establishes the rcsult.DThe last result helps us prove the Szeg6-Kolmogorov theorem.

Theorem 2.6.2 (Szego-Kolmogorov Theorem)Let h(0) be a nonnegative function in L1 (T). Then

inf -21 r11 - f(O)1 2 h(O) dO = exp (-21 rlog(h) dO) .lEAo 1r iT 1r iT

PROOF: First we note that the identity in theorem 2.6.1 holds if f E .40 isreplaced with 2f E .40:

exp (~ rlog(h) dO) = inf -21 rhe2SR(f) dO = inf -21 rhlef l2 dO.21r iT fEAo 1r iT fEAo 1r iT

If f E .40, then el = 1 - g, where 9 is the restriction of an entire functionto the unit circle. As such, the function g can be approximated in the supnorm to arbitrary accuracy by members of .40. The inequality

exp (~ rlog(h) dO) 2: inf 21 rhl1 - 4>1 2 dO (2.8)21r iT EAo 1r iT

is a consequence.

2.6. THE SZEGO-KOLMOGOROV THEOREM 57

To obtain the reverse inequality, we first examine the inequality 2.8 witha different choice of integrable weight: 11 - 91 2 with 9 E .40.

exp (2~ llog(ll - g12) dO) >=

. 1 1mf -2 11- gl211 - fl 2 dO/EAo rr T

inf ~ (ll-g-f-fgI 2 dO./EAo 2rr iT

The last integral can be interpreted as the distance between the function9 + f + f 9 and eo in the L2 norm. Since 9 + f + f 9 E S H 2 , this distanceis at least one. Hence

and log(ll - g12) is Lebesgue integrable with nonnegative mean value.Hence, using theorem 2.6.1 with 11 - gl2h as the weight gives

inf ~ { e1R(f)ll - gl2hdO/EAo 2rr iTexp (2~ llog(ll - gl2h) dO)

> exp (2~ llog(h) dO) .Taking infimum over the left-hand side over all 9 E .40 gives the result.DThe geometric significance of this theorem is that it gives an explicit

formula for the distance between Vii and the closure of the space of func-tions of the form Viif where f E Ao. This formula is important in manycontexts. In the next chapter it will shed light on our work in predictiontheory. It is also an essential tool in understanding the structure of outerfunctions. The next theorem is the first step in this process.

Theorem 2.6.3 Let f E H 2 , and denote by F(z) its analytic extension tothe unit disc. The function f is outer if and only if

2~Llog(lf(O)l) dO = log(IF(O)l).PROOF: It will be useful to bear in mind that if


then the power series expansion for F(z) is

As a function in H 2 , f admits a factorization f = u as a product of aninner function and an outer function u. Recall that this factorization isunique up to multiples of unit complex numbers. Let (z) and U(z) denotethe analytic extensions of and u, respectively. The Szeg6-Kolmogorovtheorem applied to the nonnegative integrable function lu(OW yields

exp (~ llog(IUI) dO) inf 21 r11 - gl21ul2 dOgEAo 7r iT

= inf 21 r lu - ugl 21dO.

gEAo 7r iT

The last infimum can be interpreted as the square of the distance betweenthe function u and the closure of the subspace {uglg E Ao} in H2. Thissubspace is just S(Au). Because u is outer, Au = H2 and S(Au) = S(H2).Since S(Au) is a proper subspace of Au, u is not an element of S(Au) =S(H2 ). If

the distance between u and S(H2 ) is Icol > O. As a consequence,

exp (.!:. r log(lul) dO) = inf 21 r 11 - gl21ul2dO = Icol2 = IU(0)12 > o.7r iT gEAo 7r iT

Now if f is outer, then IF(O)I = IU(O)I, and

exp (~ llog(lul) dO) = IU(OW = 1F(0)12 > O.To establish the converse, suppose that f is not outer. Then is a non-constant inner function and 1(0)1 < 1. In this case

exp (~ llog(lul) dO) = IU(OW > 1(O)U(OW = IF(OWThe result is established.DRemarkably, a little more work actually gives a fairly explicit formula

for outer functions. This line of reasoning is described in the problems.

2.7. PROBLEMS

2.7 Problems

59

2.1 Let Y be a subspace of L2 with BY = Y. Prove that B(Yol) = yol.2.2 Let be an inner function. Show that A = {I : I E H 2 }.2.3 Prove that the function

z-(F(z) = ---1- z(

is an inner function for any ( E C with ( < 1.2.4 Verify that the Poisson kernel satisfies

00

Pr(O) = L rlnletnO,n=-(X)

where the convergence is uniform for all r E [0,1). Prove that Pr(O) is asummability kernel.

2.5 Prove the following identities involving the Poisson kernel:

a. If z = re t is a point within the unit disc,

b.

_~ j'lf OP'(O) dO = ~.21r -'If r 1+ r

2.6 a. Let J.l be a positive measure in M(T), and definea(t) = J.l([-t, t])

for t E [0, 1r).Prove the formula

j 'lf Pr(O)dJ.l(O) = 1 - r J.l(T) _ r P;(O)a(O) dO.-'If 1+ r 10

b. Suppose J.l is a singular, positive measure in M(T). Set X to be thosepoints 0 such that

lim J.l([O - f,O + f]) = O.e-O 2f


Standard results from the differentiation theory of measures (see [57]and [67]) state that the complement of X in T has measure zero.Show that

lim Pr * /L(B) = 0r->l

for all BE X.

2.7 Show directly that the spaces 1{P are Banach spaces.

2.8 For a fixed ( in the unit disc, setz+(l(z) = ---.1- (z

Show that, if f(B) is in BP, then f(l(e tlJ )) is in BP. (Hint: first verify theresult if f has a finite Fourier expansion.)2.9 Let f be an outer function and F(z) its analytic extension to the unitdisc. Show that F(z) has no zeroes.2.10 As in the last problem, let f be an outer function and F(z) itsanalytic extension. For a fixed ( in the unit disc, set

z+(l(z) = --- .1+(z

Show that f(l(e tlJ )) is an outer function with analytic extension given byF(l(z)).2.11 With the same notation as in the previous problem, prove that

1 1 (etO +()- lR ----e;: log If(B)1 dO = logW(()!211" T et -.,

for all ( in the unit disc. Use this to argue that

( 1 retO

+( )exp 211" iT etO _ ( log If(O)1 dO = F(()

for all ( in the unit disc.2.12 Let h(B) be a positive, square integrable function on T with logh E

L1(T). Show that

F(z) = exp (21 ret : + z log Ih(O)1 dO)11" iT et - z

is an outer function.

2.7. PROBLEMS

2.13 Let J.L be a singular, positive measure on T. Show that

( 1 1e'(} + z )F(z) = exp --2 -(}- dJ.L7r T e' - zis a singular inner function. (Hint: see problem 2.6.)

61

Chapter 3

Prediction Theory

Stationary processes model random processes whose statistics are invariantin time. This invariance will be exploited so as to associate a spectralmeasure with any such random process. In practice, only finitely manymoments of this spectral measure can be accurately estimated from a givenstationary time series. The problem of constructing prediction schemesbased on this partial information has no unique answer and often dependson further characteristics of the data. In this chapter, we will give anintroduction to the mathematics behind this problem.

3.1 Introduction to Stationary Random Pro-cesses

Let 0 be a measure space with a measure P of total mass one: P(O) = 1.In the framework of probability theory, 0 is the sample space (or ensemblespace), and P is a probability measure. A random variable X is a complex-valued measurable function on O. If X is integrable, its expectation isdefined as

E(X) = In X dP.A (discrete) random process X n (on 0) is a sequence of random variables.Such a process is wide-sense stationary if

1. the expectations

for all n,

64 CHAPTER 3. PREDICTION THEORY

2. the expectations E(Xi ) are independent of i, and

3. if for each fixed j, the correlations E(Xi+jXi ) are independent of i.

The correlation coefficient of lag j of such a process is defined as

The following observation allows us to apply the discrete Fourier transform.

Proposition 3.1.1 The sequence of correlation coefficients Cn of a wide-sense stationary process X n is positive definite.

PROOF: If ai is a sequence of complex numbers with finitely many nonzeroterms,

o < E (I I>iXiI2)= LaiajE(XiXj)

oThen Herglotz's theorem 1.5.1 implies that there is a measure Jl on the

unit circle T such that

Jl is the spectral measure of the process X n

Theorem 3.1.1 There is a linear transformation Z : C(T) --+ L 2 (S1) suchthat

1. J Z(J)Z(g)dP = Jl(Jg) for all f,g E C(T) and2. Z(en ) = X n for all n.

PROOF: Let D be the linear subspace of C(T) consisting of linear com-binations of the form L akek (0) where only finitely many ak are nonzero.The transformation Z will first be defined on D by

3.1. INTRODUCTION TO STATIONARY RANDOM PROCESSES 65

Lakblck-l

= JL ((L akek)(L b1et ))

Let f E C(T) and Pk((J) be a sequence of functions in D such thatas k ~ 00.

Then

Since Pk is a Cauchy sequence in C(T), Z(Pk ) is a Cauchy sequence inL 2 (0) and converges to a random variable Y E L2 (0). Y is independent ofthe approximating sequence Pk. Indeed, suppose Qk is another sequencein D approximating f. Then,

Since the last term tends to 0 as k ~ 00, Z(Qk) must also converge to Y.Hence, define Z(f) = Y. An approximation argument similar in spirit tothe ones given above yields item 1 of the theorem (see problem 3.1).0

Remark 3.1.1 In fact, somewhat more is true in this regard. The spaceC(T) is dense in L2 (T, dJL) since JL is a positive Borel measure. As a re-sult, on the basis of the identities given in theorem 3.1.1, Z extends to anisometry from L 2(T, dJL) into L 2 (0, dP). The image of this transforma-tion is identical to the closure of the span of the random variables X n inL 2 (0,dP).

Z is the spectml process of the random process X n . It is clear from theabove proof that the image of Z is actually the closure of the span of theX n in L2 (0).Our analysis of stationary processes will be in terms of the correlation

coefficients as well as the spectral process and measure. As a first illustra-tion of this, we end this section with a characterization of certain degeneratestationary processes in terms of the spectral measure.


Definition 3.1.1 A stationary process X n is degenerate if the span of X nis finite dimensional in 2(0, dP).

Theorem 3.1.2 A wide-sense stationary process X k is degenerate if andonly if the spectral measure JL is a finite linear combination of point masses.

This result follows from the next lemma which is interesting in its ownright.

Lemma 3.1.1 A given positive measure J.L is a linear combination of pointmasses if and only if the matrix

(J.L(ei-j))O~i,j~Nis singular for all sufficiently large N.

PROOF: Suppose J.L is a finite linear combination of point masses:

Then

A column vector of the matrix (Ci-j )O~i,j~N has the form

2:~=1 akejo({h)2:~=1 akejo-l({h)2:~=1 akejo-2({h)

N

= L akejo({h)k=l

Hence each of column vectors of the matrix (Ci-j)O~i,j~N is contained inthe N-dimensional subspace generated by the column vectors

1 ~ k ~ N.

3.1. INTRODUCTION TO STATIONARY RANDOM PROCESSES 67

Consequently the determinant

equals zero.To establish the converse, assume that

is singular. This matrix has a Grammian structure:

Hence, the functions en,' .. ,en+N are linearly dependent in L 2(T, j.L) forany fixed integer n. It is a straightforward exercise (see problem 3.2) toshow that the space V spanned by the functions .. ,e-2, e-l, eo, el, e2,' ..is a finite dimensional subspace of L2 (T, j.L). Consider the linear map T :L2 (T,j.L) -> L2 (T,j.L) defined by

T(f)(fJ) = e'of(fJ).Since T is unitary (as can be verified), the spectral theorem for finite dimen-sional vector spaces implies that there is an orthonormal basis VI, ... , V Kof V such that

for 1 ::; i ::; K."KSuppose that eo = L-k=1 akVk. Then

en = Jeneo dj.L = j.L (Tneoeo)KLa~emOk.k=l

Now we construct explicitly a measure with these Fourier coefficients. In-deed, define the measure 1/ := 2::-1 a~60k and verify that I/(en ) = Cn. Bythe uniqueness theorem for the Fourier coefficients of measures (proposition1.5.1), j.L = 1/.0PROOF OF THEOREM 3.1.2: If X k is a degenerate process, the randomvariables X n span a finite dimensional space in L2(0, dP). Hence, theGrammian matrix


is singular for all sufficiently large N. Therefore, by lemma 3.1.1, thespectral measure of the process IL must be the sum of point masses.To establish the converse, assume that IL, the spectral measure of Xi is

a linear combination of point masses. Then for some sufficiently large Nthe matrix

C = (Ci-j)O

3.2. EXAMPLES OF STATIONARY PROCESSES 69

=

=

The spectral process 3, in this example, also has an especially simple struc-ture:

3(en ) = Wn for each n.Stationary processes that are built out of white noise processes in a

tractable way are especially valued in applications. Now we proceed to de-velop one such model known as the autoregressive model. The constructiondepends on a choice of polynomial

Po(z) = ao + alZ + a2z2 + ... + apzPwhose zeroes are all located outside the closed unit disc. In this case, thefunction Fo(z) = 1/Po(z) is analytic on an open set containing the closedunit disc. As such, Fo has a power series expansion of the form

Fo(z) = bo+ bIZ + b2z2+ ... + bnzn + ...that converges on an open set containing the closed unit disc. Define therandom variables

Now

em (} Fo(e-t(}) = boem (} +b1et(n-I)(}+b2e-t(n-2)(} + .. +bket(n-k)(}+... (3.1)is an orthonormal expansion in L2 (T). Recall that the spectral process 3is a bounded linear transformation from L2 (T) to L2 (S1, dP). Applicationof the spectral process to both sides of Equation 3.1 yields

X n = boWn + bl Wn- l + b2Wn- 2+ ... + bkWn-k + ....Now using theorem 3.1.1, we compute

In3(Po~:~t(})3(Po~:~t(}) dP1 1 em (} etn(}

- dO211' T Po(e-t(}) Po(e-t(})2- [ 1 2 dO.211' iT lPo(e-t (})I

This last quantity is finite and independent of n. In an entirely similar vein,we also compute that


Thus X n is a stationary process. As an added bonus, the preceding compu-tation also provides us with the autocorrelation en and the spectral measuredJ-L for this process:

en =

dJ-L =

It is also useful to determine a formula for Wn in terms of the X n . To thisend, we use the formula

emlJ emlJPo(e-tlJ)Fo(e-tlJ)p

= emlJFo(e- tlJ ) L ake- tklJk=O

p= L aket(n-k)1JFo(e- tlJ ).

k=O

This series converges in the sup norm and therefore in L2 (T). Applicationof the spectral process :=: to both sides gives

p

Wn = :=:(emlJ ) = LakXn-k.k=O

Since ao =I- 0, this equation yields

(3.2)

(3.3)It is this relationship that motivates the nomenclature - X n is called anautoregressive process of order p. The white noise process Wn is calledthe innovations process for X n . The rationale behind this terminology isthat Wn is the "new information" required at the n-th stage in order toconstruct the random variable X n .

Example 3.2.1 Let

Po(z) = (3 - z)(2 - 2z + Z2) = 6 - Sz + 5z2 - z3.The series expansion for Fo(z) = 1/Po (z) is

~ ~ ~ 2 ~ 3_~z4 o (Z5)6 + 9z + lOS z + 324 z 1944 + .

3.3. THE REPRODUCING KERNEL 71

0.03

0.02

0.01

1to-1t eFigure 3.1: Plot of the spectral density in example 3.2.1.

If X n is an autoregressive process (of order three) generated by thesechoices, the process defined by

must be the associated innovations process. The spectral measure of X n is

See figure 3.1 for a plot of the spectral density.

3.3 The Reproducing KernelIn practice, a stationary random process X n can be observed only for finitelymany instants in the 'time' variable n. It is not surprising that only finitelymany of the correlation coefficients en can be accurately estimated from thisinformation. This restricts the accuracy with which the spectral measureof the process can be estimated. (We refer the reader to the excellentbook by Percival and Walden [47] for an in-depth discussion of how toestimate the correlation coefficients of time series.) Spectral estimationconcerns the problem of choosing a measure p, such that p,(en ) = en inthe range of values of the index n for which the correlation coefficients areknown. There is no unique way of doing this. Moreover, different solutions


of this problem can be appropriate in different circumstances. Thus, itis important to have a characterization of the set of possible measuressatisfying these constraints as well as an understanding of special solutionsthat satisfy additional desirable properties. The approach that we adoptto this area follows the article of Landau [33].For the remainder of this section, let co, CI, ... ,CN be a finite sequence

of numbers with the property that the matrix

is strictly positive definite. (Here C-n := cn .) Let S denote the set ofpositive measures on T such that

for 0 ::; n ::; N.

Indices will vary from 0 to N unless otherwise stated. Let Pn be the vectorspace of polynomials of degree n. Define

[P, Q] = I: aibjCi-j,O~i,j~N

where P(z) = L:~=o aizi and Q = L:~o bizi are two functions in P".

Proposition 3.3.1 [.,.] satisfies the following properties:

(3.4)

1. [aPo+ bPI,Q] = a[Po, Q] + b[PI, Q] for a,b E C and Po, PI, Q E Pn ,2. [P,Q] = [Q,P] for P,Q E Pn ,3. [zP,zQ] = [P,Q] for P,Q E Pn , and4. [P, P] 2: 0 for any P E Pn with equality exactly when P == O.

PROOF: These properties are routine to verify. In particular, the last itemfollows directly from the stated assumptions on co,' .. ,CN. See problem3.3.0Proposition 3.3.1 states that [.,.] is a positive definite Hermitian form

on the vector spaces Pn for n = 0,1,2,, N. The tower

is a sequence of increasing subspaces where the dimension increases by oneat each stage. Therefore, there exists a Pn (et8 ) E Pn such that Pn J.. Pn - l


for n = 0,, N. In fact, since the dimension increases by one at each step,Pn is unique up to a constant factor. Choose the normalization

for n = 0, ... , N.

Let t" = lPn' Pn]. If Pn(z) = Azn + R(z) where R E Pn- 1 then the abovenormalization implies

t" = IPn,P,,]IAe", P,,] + IR, P,,]

= Alen, Pn]= AHence tn is the leading coefficient of the polynomial Pn . The followingproperty of Pn is also very useful in the sequel.

Proposition 3.3.2 The roots of the polynomials Pn(z) are in the unit disc{z: Izl < I}.PROOF: Let ( be a complex root of .Pr,(z). By the fundamental theorem ofalgebra,

Pn(z) = (z - OQ(z) = zQ(z) - (Q(z)where Q is of degree n - 1. Since Pn 1- Pn - 1 , the equation

Pn +(Q = zQgives an orthogonal decomposition of zQ. Upon taking norms with respectto Ie, e],

lPn, Pn]+ 1(12 IQ, Q] = IzQ, zQ].Because lPn' Pn] > 0 and IzQ, zQ] = IQ, Q],

IQ,Q] > 1(12 IQ,Q]The result follows. 0The orthogonal polynomials Pn are extremely computable and can be

used to give explicit descriptions of certain measures in S. To describesuch constructions, first it is necessary to describe certain auxiliary objectsrelated to the orthogonal polynomials Pn .Consider the linear functional on Pn defined by P 1-4 P((). Since Ie, e]

is a positive definite inner product on the finite dimensional vector spacePn, there is a Kn,dz) E Pn such that

P(() = IP,Kn,d for all P E Pn .The polynomial Kn,((z) = Kn((, z) is the reproducing kernel of degree n.


Proposition 3.3.3 The reproducing kernel of degree n satisfies the follow-ing properties:

1. Kn,c;(() = [Kn,(,Kn,d,2 K (z) = "n P;;(()Pdz) and

n,( L.Jk=O t n '

3. [Kn,(, Kn,d- 1 = infpE'Pn,p(()=dP, Pl.

PROOF: Item 1 follows directly from the definition of Kn,((z). The nor-malized polynomials

PO(z) Pn(z)..;to' , vr.;;

are an orthonormal basis of Pn. The polynomial Kn,c;(z) has the followingexpansion in terms of the above basis:

~[K ~]Pi(Z)Kn,((z) = L..J n,(, If Ifi=O V bi V bi

t PJ()Pi(z) .i=O t i

Hence item 2.Now observe that

{P E Pn : P(() = I} = {P E Pn : [P,Kn,d = I}.Consequently standard geometrical facts about inner product spaces applyto yield the fact that the polynomial achieving the minimum in

inf [P,P]PE'Pn,P(()=l

must be a scalar multiple of K n ,(:

cKn ,(.

This leads to the condition that

or1

c= .Kn,c;(()

Item 3 follows.DThe case ( = 0 will be of special importance in the sequel. The following

lemma gives several useful properties of the associated polynomials.


Lemma 3.3.1

2. Pn(z) = znKn,o(1/z), and3. Kn,o(z) determines CO,"', cn'

PROOF: Let A denote the positive definite (n + 1) x (n + 1) matrix

( C~:n~ cCn~_11 c~~:.: 1 ) .A = (Ci-j)O~i,j~n = '-1JLet v, wEen+! be the row vectors corresponding to the coefficients of thepolynom

Methods of Applied Fourier Analysis ||

Documents

Transcript of Methods of Applied Fourier Analysis ||