1 Introduction to Fourier analysispetersd/464/FourierSeriesh.pdf · 1 Introduction to Fourier...
76
We consider a signal f (x) where x is time. This may be a sound recorded my a microphone, and give something like this: 0 0.5 1 1.5 2 2.5 3 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 In practice we usually are only given function values f (x j ) for points x j = jh with a step size h (“sampling”), e.g., for sound recorded on a CD we have h = 1 44100 sec, here h = 1 32 : 0 0.5 1 1.5 2 2.5 3 0 0.2 0.4 0.6 0.8 1 1.2 Describing a signal in terms of function values is called a representation in the time domain. Note that our sample signal f (x) is periodic: We have for all x ∈ R f (x + L)= f (x) with the period L. Here L = 1. Consider the signal g(x) := f (2x): 0 0.5 1 1.5 2 2.5 3 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 Here the period is L = 1 2 . The frequency is ξ = 1/L, this measures how many periods we have on a unit interval (ξ can be noninteger, e.g., for f (2.5x) we have ξ = 2.5). 1
Transcript of 1 Introduction to Fourier analysispetersd/464/FourierSeriesh.pdf · 1 Introduction to Fourier...
1 Introduction to Fourier analysis
We consider a signal f (x) where x is time.
This may be a sound recorded my a microphone, and give something like this:
0 0.5 1 1.5 2 2.5 3
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
In practice we usually are only given function values f (x j) for points x j = jh with a step size h (“sampling”), e.g., for soundrecorded on a CD we have h = 1
44100 sec, here h = 132 :
0 0.5 1 1.5 2 2.5 3
0
0.2
0.4
0.6
0.8
1
1.2
Describing a signal in terms of function values is called a representation in the time domain.
Note that our sample signal f (x) is periodic: We have for all x ∈ R
f (x+L) = f (x)
with the period L. Here L = 1.
Consider the signal g(x) := f (2x):
0 0.5 1 1.5 2 2.5 3
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
Here the period is L = 12 . The frequency is ξ = 1/L, this measures how many periods we have on a unit interval (ξ can be
noninteger, e.g., for f (2.5x) we have ξ = 2.5).
1
When we hear a sound signal we can directly perceive frequencies, but not sample values in the time domain.
Consider a signal with period L = 1, like our sample signal.
What functions with f (x) = f (x+1) do we know? The functions cos(2πx) and sin(2πx) work, but so do
cos(2πkx), sin(2πkx)
with integer frequencies k ∈ {0,1,2,3, . . .}. For k = 0 we only have cos(2π ·0 · x) = 1 which is constant.
The idea of Fourier analysis is to write the signal f (x) as a superposition of these functions.
Since
eix = cos(x)+ isin(x),
cos(x) = 12
(eix + e−ix) , sin(x) = 1
2i
(eix− e−ix)
we can use instead of the functions
1, cos(2πx), cos(4πx), cos(6πx), . . .sin(2πx), sin(4πx), sin(6πx), . . .
the functions. . . ,e−i6πx,e−i4πx,e−i2πx,1,ei2πx,ei4πx,ei6πx, . . .
i.e., we want to write our 1-periodic signal f (x) as
f (x) =∞
∑k=−∞
fkei2πkx
where fk ∈ C are the so-called Fourier coefficients and describe our signal in the “frequency domain”.
Example: For the signal f (x) = sin(2πx) we have f (x) = 12i
(e2πix− e−2πix
), hence the Fourier coefficients are f−1 =
12 i,
f1 =−12 i, and fk = 0 for all k∈Z\{−1,1}. We see that a real-valued signal may have complex, non-real Fourier coefficients.
We will also consider nonperiodic signals f (x). In this case we will need to use functions ei2πξ x with frequencies ξ ∈ R,and we will write our signal as
f (x) =∫
∞
ξ=−∞
f (ξ )ei2πξ x
The Fourier transform takes us from the time domain to the frequency domain. The inverse Fourier transform takes usfrom the frequency domain to the time domain.
We will consider four different settings for the “Fourier transform”:
time domain frequency domain
1-periodic function, continuous time f (x), x ∈ [0,1) fk, k ∈ Z
1-periodic function, discrete time f j for x j =j
N , j = 0, . . . ,N−1 fk, k = 0, . . . ,N−1
Nonperiodic function, continuous time f (x), x ∈ R f (ξ ), ξ ∈ R
Nonperiodic function, discrete time f j for x j = jh, j ∈ Z f (ξ ), ξ ∈[− 1
2h ,12h
]Warning: There are different conventions for defining “Fourier transforms” in use in books and mathematical software:Some people use eiξ x instead of e2πiξ x, and different factors 2π or
√2π are used.
2
Mathematically speaking, all these definitions are equivalent, as one can go from one convention to a different one byinserting factors 2π or
√2π in appropriate places.
In practice one has to be careful e.g. when using software like Matlab, Mathematica, Maple: You have to check the precisedefinition of “Fourier transform” which the software uses.
In this class I will always represent f (x) as a superposition of terms f (ξ ) · e2πiξ x or fk · e2πikx. I will explain how to useMatlab which uses different definitions.
We will first consider functions with period 1, i.e., f (x+ 1) = f (x) for all x. Once we understand this case, it is easy togeneralize this to functions with period L, i.e., f (x+L) = f (x):
time domain frequency domain
L-periodic function, continuous time f (x), x ∈ [0,L) fk for ξk =kL
, k ∈ Z
L-periodic function, discrete time f j for x j =j
N L, j = 0, . . . ,N−1 fk for ξk =kL
, k = 0, . . . ,N−1
2 Linear algebra: vectors, subspaces, orthogonal basis, orthogonal projection
2.1 Motivation: Fourier series
We are given a 1-periodic function f (x). We want to write this in terms of the functions u(k)(x) := e2πikx:
f =∞
∑k=−∞
fku(k)
This means we have to find the Fourier coefficients fk ∈ C for all k ∈ Z.
It turns out that this is a linear algebra problem, similar to the following:
Example problem in the vector space V = R2: We are given the vector f =[
2.55
], and we want to write this as a linear
combination of the vectors u(1) =[
21
]and u(2) =
[−12
]: Find c1,c2 ∈ R such that
f = c1u(1)+ c2u(2)
-2 -1 0 1 2 3 40
1
2
3
4
5
u(1)
u(2)
f
Solution using results from section 2.3: As(u(1),u(2)
)= 0 the vectors u(1),u(2) are an orthogonal basis of V , hence
c1 =
(f ,u(1)
)(u(1),u(1)
) = 105
= 2, c2 =
(f ,u(2)
)(u(2),u(2)
) = 7.55
= 1.5
3
2.2 Vector spaces, span, inner product
“Vector space” means that we specify a set of vectors, and a set of scalars. We can add two vectors, and we can multiply avector by a scalar.
Example: V = R3 consists of vectors
x1x2x3
with x1,x2,x3 ∈ R3. Scalars are numbers in R.
Example: V = Cn is a vector space: vectors have the form u =
u1...
un
with u j ∈ C, and scalars are numbers in C.
For vectors u(1), . . . ,u(N) and scalars c1, . . . ,cN the linear combination v = c1u(1)+ · · ·+ cNu(N) is again a vector.
If c1u(1)+ · · ·+ cNu(N) can only be the zero vector for c1 = · · · = cN = 0 we say that the vectors u(1), . . . ,u(N) are linearlyindependent.
We denote the set of all linear combinations by
span{
u(1), . . . ,u(N)}={
c1u(1)+ · · ·+ cNu(N) | c j ∈ C}
W = span{
u(1), . . . ,u(N)}
is a subspace of V : W is a vector space with W ⊂V .
Example: W = span
1
20
, 2
30
, 3
40
is the x1,x2-plane in R3. Note that one of the 3 vectors is “redundant”.
If W = span{
u(1), . . . ,u(N)}
and the vectors u(1), . . . ,u(N) are linearly independent:
• the vectors u(1), . . . ,u(N) are called a basis of the subspace W
• N is called the dimension of the subspace W
For the vector space Cn we introduce the inner product
(u,v) =n
∑i=1
uivi
Here vi denotes the complex conjugate: For a complex number z = x+ iy we have z = x− iy.
The norm of a vector is defined as‖u‖= (u,u)1/2. (1)
For a vector u ∈ Cn this means that ‖u‖=√∣∣u1
∣∣2 + · · ·+ ∣∣un∣∣2.
We say that two vectors u,v are orthogonal if (u,v) = 0.
An important property is the Cauchy-Schwarz inequality:
|(u,v)| ≤ ‖u‖‖v‖ (2)
In (1) we defined the norm in terms of the inner product. Conversely, we can also express the inner product (u,v) in terms ofnorms ‖·‖:For real vectors u,v ∈ Rn we have
‖u+ v‖2−‖u− v‖2 =(‖u‖2 +2(u,v)+‖v‖2
)−(‖u‖2−2(u,v)+‖v‖2
)= 4(u,v)
(u,v) =14
[‖u+ v‖2−‖u− v‖2
]For complex vectors u,v ∈ Cn we can use the identity
(u,v) =12
[‖u+ v‖2− i‖u+ iv‖2− (1− i)‖u‖2− (1− i)‖v‖2
](3)
4
2.3 Orthogonal projection on a subspace W
If the nonzero vectors u(1), . . . ,u(N) are orthogonal on each other, i.e.,(u(k),u(l)
)= 0 for k 6= l
we say that they form an orthogonal basis of the subspace
W = span{
u(1), . . . ,u(N)}.
For a given vector v we now want to find the vector w ∈W which is closest to w, i.e., ‖v−w‖ should be minimal. In thiscase we must have that the difference vector w− v is orthogonal on the subspace W , i.e.,(
v−w,u(k))= 0 for k = 1, . . . ,N. (4)
We want to find the coefficients c1, . . . ,cN of the vector v = c1u(1) + · · ·+ cNu(N). Plugging this into (4) gives using theorthogonality of the vectors u(1), . . . ,u(N) that
ck =
(v,u(k)
)(u(k),u(k)
) (5)
For a given vector w we can find the projection v as follows:
1. Compute the coefficients ck =(v,u(k))(u(k),u(k))
for k = 1, . . . ,N
2. Let w = c1u(1)+ · · ·+ cNu(N).
Claim: This vector w gives the minimal error ‖v−w‖ among all w ∈W :Proof: Assume that we use a vector w+d with some d ∈W instead of w:
‖v− (w+d)‖2 = (v−w−d,v−w−d) = (v−w,v−w)− (v−w,d)︸ ︷︷ ︸0
−(d,v−w)︸ ︷︷ ︸0
+(d,d)
as v−w is orthogonal on all vectors in W . Hence we have for nonzero d
‖v− (w+d)‖2 = ‖v−w‖2 +‖d‖2︸︷︷︸0
> ‖v−w‖2 , (6)
i.e., the error of v+d is strictly larger than the error of v.�
Since v−w is orthogonal on w we have the Pythagoras equation
(v,v) = (w+(v−w),w+(v−w)) = (w,w)+(v−w,w)︸ ︷︷ ︸0
+(w,v−w)︸ ︷︷ ︸0
+(v−w,v−w)
‖v‖2 = ‖w‖2 +‖v−w‖2
As ‖v−w‖ ≥ 0 we obviously have‖v‖ ≥ ‖w‖
Since w = c1u(1)+ · · ·+ cNu(N) with orthogonal vectors u(1), . . . ,u(N) we have
‖w‖2 = |c1|2∥∥∥u(1)
∥∥∥2+ · · ·+ |cN |2
∥∥∥u(N)∥∥∥2
(7)
and hence
‖v‖2 = |c1|2∥∥∥u(1)
∥∥∥2+ · · ·+ |cN |2
∥∥∥u(N)∥∥∥2
+‖v−w‖2 (8)
This equation can be used to compute ‖v−w‖. Since ‖v−w‖ ≥ 0 we obviously have
‖v‖2 ≥ |c1|2∥∥∥u(1)
∥∥∥2+ · · ·+ |cN |2
∥∥∥u(N)∥∥∥2
. (9)
5
2.4 Example for R3
Consider the vectors u(1) =
101
, u(2) =
−111
and let W := span{
u(1),u(2)}
. Note that(u(1),u(2)
)= 0, hence u(1),u(2)
are an orthogonal basis of W , and W has dimension 2. Note that W ⊂V = R3 which is a vector space of dimension 3.
Now let v =
123
. We want to find the closest vector w ∈W :
c1 =
(v,u(1)
)(u(1),u(1)
) = 42= 2, c2 =
(v,u(2)
)(u(2),u(2)
) = 43, w = c1u(1)+ c2u(2) =
13
2410
We can check that v−w is orthogonal on both u(1),u(2): v−w = 13
12−1
, so(v−w,u(1)
)= 0,
(v−w,u(2)
)= 0.
We can compute the distance of the point v to the subspace W in two ways:
1. ‖v−w‖=
∥∥∥∥∥∥13
12−1
∥∥∥∥∥∥= [19 · (1+4+1)
]1/2=√
23
2. ‖v−w‖2 = ‖v‖2−|c1|2∥∥u(1)
∥∥2−|c2|2∥∥u(2)
∥∥2= 14−22 ·2− (4
3)2 ·3 = 2
3
2.5 Orthonormal basis u(1), . . . ,u(N)
If the orthogonal vectors u(1), . . . ,u(N) have all length 1, i.e.
(u(k),u(l)
)=
{0 for k 6= l1 for k = l
some of the formulas from the previous section become simpler:
The coefficients ck of the projection v = c1u(1)+ · · ·+ cNu(N) are given by
ck =(
v,u(k))
(10)
and (8), (9) become
‖v‖2 = |c1|2 + · · ·+ |cN |2 +‖v−w‖2 (11)
‖v‖2 ≥ |c1|2 + · · ·+ |cN |2 (12)
2.6 Functions as vectors
Sets of functions can also form a vector space. For example,let V denote the set of all complex-valued continuous functions on the interval [0,1]
with scalars in C. For functions u,v ∈V and a scalar α ∈ C
u+ v, αu
are again functions in V .
6
We can define an inner product as follows: For functions u(x),v(x) on [0,1] let
(u,v) =∫ 1
0u(x)v(x)dx.
Then we have the norm
‖u‖2 = (u,u) =∫ 1
0|u(x)|2 dx.
We can also allow more general functions u as vectors (e.g. piecewise continuous), as long as∫ 1
0 |u(x)|2 dx is finite. The
vector space of these functions (where the integrals exist in the sense of Lebesgue) is called L2([0,1]) , and the norm ‖u‖ iscalled L2-norm.
For functions u(1), . . . ,u(N) which are orthogonal on each other we consider again the subspace
W = span{
u(1), . . . ,u(N)}.
For a given function v we want to find the best approximation w = c1u(1)+ · · ·+ cNu(N) ∈V , in the sense that the error
‖v−w‖2 =∫ 1
0|v(x)−w(x)|2 dx
becomes minimal (“least squares approximation”). We can find v as follows:
1. Compute the coefficients ck =(w,u(k))(u(k),u(k))
for k = 1, . . . ,N
2. Let w = c1u(1)+ · · ·+ cNu(N).
Example: Let u(1)(x) = 1 and u(2)(x) =
{1 x ∈ [0, 1
2 ]
−1 x ∈ (12 ,1]
. Then the subspace W := span{u(1),u(2)} consists of piecewise
constant functions on the partition 0, 12 ,1. Note that we have
(u(1),u(2)
)= 0, i.e., we have an orthogonal basis.
For the given function v(x) = x2 find the best least-squares approximation with a function w ∈W :
We first compute the coefficients
c1 =
(v,u(1)
)(u(1),u(1)
) = ∫ 10 x2dx∫ 10 1dx
=1/3
1=
13, c2 =
(v,u(2)
)(u(2),u(2)
) = ∫ 1/20 x2dx+
∫ 11/2(−x2)dx∫ 1
0 1dx=
13
(18 −1+ 1
8
)1
=−14
and obtain
w(x) = c1u(1)(x)+ c2u(2)(x) =
{1
12 x ∈ [0, 12 ]
712 x ∈ (1
2 ,1]
We can compute ‖v−w‖ using (8):
‖v−w‖2 = ‖v‖2−|c1|2∥∥∥u(1)
∥∥∥2−|c2|2
∥∥∥u(2)∥∥∥2
= 15 − (1
3)2 ·1− (1
4)2 ·1 =
19720
= .02638888...
0 0.5 1
0
0.2
0.4
0.6
0.8
1
7
3 Periodic functions, continuous time: Fourier series
3.1 The space TN of trigonometric functions
We are now interested in periodic functions: We say a function is periodic with period L if
f (x+L) = f (x) for all x ∈ R.
We first consider the case of functions with period L = 1 (then the case with general period L will follow easily). We assumewe have a “signal” f (x) with
f (x+1) = f (x) for all x ∈ R.
The function f (x) may have complex values. We want to approximate this signal using the following trigonometric functionswith period 1
v(x) =a0 ·1+a1 cos(2πx)+b1 sin(2πx)
+ · · ·+aN cos(2πNx)+bN cos(2πNx). (13)
with coefficients ak,bk ∈ C. We denote the space of these functions by TN (trigonometric polynomials with frequencyup to N). Note that v(x) is a sum of terms with different frequencies:
• 1 has frequency 0
• cos(2πx) and sin(2πx) have frequency 1
•...
• cos(2πNx) and sin(2πNx) have frequency N.
We consider the case of a complex valued signal, so we have ak,bk ∈ C.
Recall thateix = cosx+ isinx (14)
implying
cosx =eix + e−ix
2, sinx =
eix− e−ix
2i. (15)
Using (15) in (13) gives the function v(x) in the form
v(x) = f0 ·1 frequency 0
+ f−1e−2πix + f1e2πix frequency 1...
...
+ f−Ne−2πiNx + fNe2πiNx. frequency N (16)
From this form of v(x) we can get to (13) by using (14). The function v(x) is written as a sum of terms with frequencies0,1, . . . ,N.
For a real-valued function f (x) the coefficients ak,bk in (13) are real, but the coefficients f−N , . . . , fN in (16) are complexnumbers which need not be real.
We use the form (16) for finding v(x). We can then easily convert this to the form (13). Therefore we use the functions
u(k)(x) := e2πikx for k ∈ Z
and haveTN = span{u(−N),u(−N+1), . . . ,u(N)}
8
Observations:
1. We have u(k)(x) ·u(`)(x) = u(k+`)(x). Therefore u ∈TN and v ∈TM implies for the product u · v ∈TN+M.
2. We have u(l) = cos(2πkx)− isin(2πikx) = e−2πilx and hence
(u(k),u(l)
)=∫ 1
0e2πikxe−2πilxdx =
{0 for k 6= l1 for k = l
This means that the functions u(−N),u(−N+1), . . . ,u(N) form an orthonormal basis of the space TN .
For a given function f (x) we denote the best approximation in the space TN by f(N)(x) :
f(N)(x) =N
∑k=−N
fke2πikx
The coefficients fk are called the Fourier coefficients of the function f . Since we have an orthonormal basis we can use (10)to compute them:
fk =(
f ,u(k))=∫ 1
0f (x)e−2πikxdx.
Since the functions are 1-periodic we can also use∫ 1/2−1/2 · · · (or any other interval of length 1).
Proposition 1. If the function f is real-valued, then f−k = fk and the function f(N) is real-valued.
Proof. We have for the Fourier coefficients fk and its complex conjugate fk
fk =∫ 1
0f (x)e−2πikxdx, fk =
∫ 1
0f (x)e−2πikxdx =
∫ 1
0f (x)e2πikx = f−k
Hence we get for the complex conjugate of f(N)(x) = ∑Nk=−N fke2πikx
f(N)(x) =N
∑k=−N
fk e2πikx =N
∑k=−N
f−ke2πi(−k)x = f(N)(x)
since this is the same sum, just in reverse order.
Note that this argument works since we use a symmetric range k =−N, . . . ,N of indices. For a real-valued function f (x) thefunction g(x) := ∑
Nk=0 fke2πikx is in general non-real.
For a general complex valued function f (x) we can write the terms of frequency k as
f−ku(−k)+ fku(k) = f−k [cos(2πkx)− isin(2πkx)]+ fk [cos(2πkx)+ isin(2πkx)]
=(
fk + f−k)
cos(2πkx)+ i(
fk− f−k)
sin(2πkx) (17)
For a real valued function f (x) we get complex Fourier coefficients fk = Ak + iBk and f−k = Ak− iBk with real Ak,Bkyielding
f−ku(−k)+ fku(k) = 2Ak cos(2πkx)−2Bk sin(2πkx) (18)
The equation (11) is here‖ f‖2 =
∣∣ f−N∣∣2 + · · ·+ ∣∣ fN
∣∣2 +∥∥ f − f(N)
∥∥2 (19)
which can be used to find∥∥ f − f(N)
∥∥. Because of∥∥ f − f(N)
∥∥≥ 0 we have
‖ f‖2 ≥N
∑k=−N
∣∣ fk∣∣2 .
9
Since this is true for any N = 0,1,2,3, . . . we have the Bessel inequality
‖ f‖2 ≥∞
∑k=−∞
∣∣ fk∣∣2 (20)
where the sum S := ∑∞k=−∞
∣∣ fk∣∣2 ≤ ‖ f‖2 converges since it is bounded from above. Hence we have∥∥ f − f(N)
∥∥→‖ f‖2−S as N→ ∞. (21)
We have S≤ ‖ f‖2. Is S = ‖ f‖2? If yes, the Fourier series f(N) converges to f as N→ ∞.
As ∑∞k=−∞
∣∣ fk∣∣2 converges we must have
fk→ 0 as k→ ∞ or k→−∞ (22)
i.e., the Fourier coefficients decay to zero as the frequency |k| increases.
3.2 Example: Fourier series and convergence
Consider the 1-periodic function f (x) with
f (x) =
{−1 for x ∈ [−1
2 ,0)1 for x ∈ [0, 1
2)(23)
and find the best approximation f(5)(x) in the space T5.
We need to compute the Fourier coefficients fk =(
f ,u(k))
. We start with f0: Since u(0)(x) = 1 we get
f0 =∫ 1/2
−1/2f (x) ·1dx =
∫ 0
−1/2(−1)dx+
∫ 1/2
01dx =−1
2 +12 = 0.
Now we consider k 6= 0:
fk =∫ 1/2
−1/2f (x)e−2πikxdx =
∫ 0
−1/2(−1)e−2πikxdx+
∫ 1/2
01 · e−2πikxdx (24)
For the second integral we get
∫ 1/2
0e−2πikxdx =
[e−2πikx
−2πik
]1/2
x=0=−1
2πik
(e−2πik/2−1
)We have for integer k that
e−2πik/2 =
{1 for even k−1 for odd k
hence ∫ 1/2
0e−2πikxdx =
{0 for even k−1
2πik (−2) =− iπk for odd k
For the first integral in (24) we proceed in the same way and get exactly the same value. Therefore we obtain the Fouriercoefficients
fk =
{0 for even k ∈ Z− 2i
πk for odd k ∈ Z(25)
and the best approximation f(5) ∈T5 is
f(5)(x) =−2iπ
(−1
5e−10πix− 1
3e−6πix + e−2πix + e2πix +
13
e6πix +15
e10πix).
10
Note that we have terms with frequency 1, 3 and 5. We now want to convert this to the form (13): E.g., for the terms offrequency 3 we get with (14)
−2iπ
(−1
3e−6πix +
13
e6πix)=−2i
π·(
i3· sin(6πx)+
i3
sin(6πx))=
43π
sin(6πx)
since the cosine terms cancel. Therefore we get
f(5)(x) =4π
(sin(2πx)+
13
sin(3 ·2πx)+15
sin(5 ·2πx))
For a general odd N we have
f(N)(x) =4π
(11
sin(1 ·2πx)+13
sin(3 ·2πx)+15
sin(5 ·2πx)+ · · ·+ 1N
sin(N ·2πx))
In order to evaluate a Fourier sum ∑Nk=−N fke2πikx in Matlab we use foursum(x,fh0,fhpos,fhneg). Here
fh0= f0, fhpos=[
f1, . . . , fN], fhneg=
[f−1, . . . , f−N
]For a real-valued function we can omit the last argument and use foursum(x,fh0,fhpos).
Here is the m-file foursum.m:
function y = foursum(x,fh0,fhpos,fhneg)
if nargin==3 % foursum(x,fh0,fhpos) uses fhneg=conj(fhpos)
fhneg = conj(fhpos);
end
f = fh0*ones(size(x));
for k=1:length(fhpos)
f = f + fhpos(k)*exp(k*2i*pi*x);
end
for k=1:length(fhneg)
f = f + fhneg(k)*exp(-k*2i*pi*x);
end
We can then plot the functions f (x), f(5)(x) and f(11)(x) on the interval [−12 ,
12 ] together:
x = -.5:.001:.5; % points for plotting
fhpos = -2i/pi./(1:11); fhpos(2:2:11)=0; % Fourier coefficients for even k are zero
plot(x,sign(x),x,foursum(x,0,fhpos(1:5)),x,foursum(x,0,fhpos))
legend(’f’,’f_{(5)}’,’f_{(11)}’); grid on
11
-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
f
f(5)
f(11)
Observations:
• For x = 0 we have f (0) = 1 according to the definition of f , but f(N)(x) = 0 for all N. This is not surprising: If wechange the definition of f (0) e.g. from 1 to −1 the integrals fk =
(f ,u(k)
)do not change. Therefore changing the
definition of f in a single point does not affect f(N)(x).
• There is an “overshoot” near the jump:The smallest local maximum has a y-value of about 1.1789, for any value of N.For each N there exists x ∈ (0, 1
2) with f(N)(x)≥ 1.1789. As N increases, the “overshoot” happens closer and closer tothe jumps, and away from the jump the function f(N) gets closer and closer to f .Since the exact function value is 1 the size of the overshoot is ≈ 0.1789. Note that the jump size at x = 0 is J = 2.
The size of the overshoot is always ≈ .0895J where J denotes the jump size
We will later see that the number .0895 is actually obtained in terms of the “sine integral” function:
>> overshoot = sinint(pi)/pi-.5
overshoot =
0.0894898722360836
We now want to find the error∥∥ f − f(N)
∥∥ by using (19). Here we have
‖ f‖2 =∫ 1/2
−1/2| f (x)|2 dx =
∫ 1/2
−1/21dx = 1
and for odd NN
∑k=−N
∣∣ fk∣∣2 = ( 2
π
)2
2(
112 +
132 +
152 + · · ·+
1N2
).
12
Hence ∥∥ f − f(5)∥∥=√1− 8
π2
(112 +
132 +
152
)≈ 0.25874
∥∥ f − f(11)∥∥=√1− 8
π2
(112 +
132 +
152 + · · ·+
1112
)≈ 0.18357
What will happen as N tends to infinity? One can show that 112 +
132 +
152 + · · · = π2/8. The Matlab symbolic toolbox can
find this symbolic sum:
>> syms k; symsum(1/(2*k-1)^2,k,1,Inf)
ans =
pi^2/8
Hence
S =∞
∑k=−∞
∣∣ fk∣∣2 = 8
π2
(112 +
132 +
152 + · · ·
)︸ ︷︷ ︸
π2/8
= 1
so we get from (21) ∥∥ f − f(N)
∥∥→‖ f‖2−S = 1−1 = 0 as N→ ∞
which means that the Fourier series∞
∑k=−∞
fke2πikx =4π
∑k=1...∞
k odd
sin(k ·2πx)k
(26)
converges to the function f in the mean square sense.
This does not mean that we have “pointwise convergence”: We have f (0) = 1, but f(N)(0) = 0.
If we modify the function f at finitely many points (or even countably many points), the new function f
• satisfies∥∥ f − f
∥∥= 0, i.e., these two functions are considered to be the same vector in L2.
• f has the same Fourier coefficients as f
3.3 The three basic convergence theorems for Fourier series
We defined the Fourier approximation f(N)(x) = ∑Nk=−N fke2πikx. We want to let N→ ∞ and obtain the Fourier series
∞
∑k=−∞
fke2πikx
But this expression only makes sense as a limit N→ ∞. In which sense does this limit exist?
It turns out that there are three different ways to understand this limit. This is the key to understanding the concept of a“Fourier series”.
L2 is the space of 1-periodic functions f (x) where∫ 1
0 | f (x)|2 dx exists (as a Lebesgue integral). Recall that functions f , f
who differ at finitely many points (or countably many points, or on a set of measure zero) have∥∥ f − f
∥∥= 0 and are thereforeconsidered to be the same vector in L2.
Let PW 0 denote the space of piecewise continuous 1-periodic functions: f is continuous at all z∈ [0,1) except for finitelymany breakpoints z1, . . . ,zM ∈ [0,1), let z0 := zM. At each breakpoint z j the left-sided limit f (z j−0) and the right-sided limitf (z j +0) need to exist. Note that functions in PW 0 may have jumps.
Let us denote the “pieces with limits” on [z j−1,z j] by f j(x):
f j(x) :=
f (z j−1 +0) x = z j−1
f (x) z j−1 < x < z j
f (z j−0) x = z j
13
Let PW 1 denote the space of piecewise continuously differentiable 1-periodic functions: Assume f ∈ PW 0 and thateach piece f j(x) has a continuous derivative f ′j(x) on [z j−1,z j]. Note that functions in PW 1 may have jumps.
We will later prove the following three theorems:
(T1) For all f ∈ L2 we have ∥∥ f(N)− f∥∥→ 0 as N→ ∞ mean square convergence (27)
(T2) For all f ∈ PW 1 we have for all x ∈ R:
f(N)(x)→f (x−0)+ f (x+0)
2as N→ ∞ pointwise convergence (28)
I.e., if f does not have a jump at x0 we have f(N)(x0)→ f (x0). If f has a jump at x0 the Fourier series converges to theaverage of left and right-sided limit, irrespective of how f (x0) is defined.
(T3) For all f ∈ PW 1 without jumps we have
maxx∈[0,1)
∣∣ f(N)(x)− f (x)∣∣→ 0 as N→ ∞ uniform convergence (29)
In this case we also have that ∑∞k=−∞
∣∣ fk∣∣< ∞, i.e. we have for each x ∈R “absolute convergence” of the Fourier series
∑∞−∞ fke2πikx. See section 3.4 for an explanation of absolute convergence.
Note that uniform convergence implies pointwise convergence and mean square convergence.
Example 1: Consider the 1-periodic function f (x) = x−1/2 for x ∈ [0,1). We can define the Fourier coefficients fk =∫ 10 x−1/2e−2πikxdx. These integrals exist since
∣∣x−1/2e−2πikx∣∣= x−1/2 which is integrable on [0,1].
But we have∫ 1
0 | f (x)|2 dx =
∫ 10 x−1dx, and this integral does not exist as
∫ 1ε
x−1dx = ln1− lnε→∞ as ε→ 0. Hence f /∈ L2,and we do not have mean square convergence.
The Dini criterion (30) gives pointwise convergence f(N)(x)→ f (x) for all x ∈ (0,1), but not for x = 0.
Example 2: Consider the 1-periodic function f (x) = x−1/3 for x ∈ [0,1). Here we have∫ 1
0 | f (x)|2 dx =
∫ 10 x−2/3dx =[
3x1/3]1
0 = 3, so f ∈ L2 and we have mean square convergence∥∥ f(N)− f
∥∥→ 0 as N→ ∞.
The Dini criterion (30) gives pointwise convergence f(N)(x)→ f (x) for all x ∈ (0,1), but not for x = 0.
Example 3: Consider the 1-periodic function f (x) = x2 for x ∈ [0,1]. Clearly f ∈ PW 1, and therefore we have pointwiseconvergence. For x = 0 we have f(N)(x)→ 1
2 as N→ ∞.
Example 4: Let f denote the 1-periodic function with f (x) = |x| for x ∈ [−12 ,
12). Clearly f ∈ PW 1, and f does not have
jumps. Hence by (T3) we have uniform convergence.
Remarks:
1. Continuity is not enough to guarantee pointwise convergence everywhere :There exists a continuous function f such that supN∈N f(N)(0) =∞. There are even functions where supN∈N f(N)(x) =∞
for all x in an uncountable dense subset of [0,1].However: If f is continuous or even f ∈ L2 there is pointwise convergence f(N)(x)→ f (x) for all x ∈ [0,1] except ona set of measure zero.
2. For f ∈ PW 1 with jumps we have∣∣ fk∣∣= O(|k|−1) and therefore no absolute convergence.
Absolute convergence implies uniform convergence. However, there are Fourier series with uniform convergence, butno absolute convergence.
3. For (T2), (T3) one can weaken the assumption f ∈ PW 1. We don’t need that each piece f j has a continuous derivative.Actually it is enough that each piece f j(x) is “slightly better than continuous”:If∣∣ f j(s)− f j(t)
∣∣≤C |s− t|α with some α > 0 (“Holder continuity”) then (28) still holds.If each f j is additionally piecewise monotonic and there are no jumps then (29) and absolute convergence still holds.
14
4. The fundamental result for pointwise convergence is the “Dini criterion”: If f is integrable and∫ 1/2
0
∣∣∣∣ f (x0− t)+ f (x0 + t)2
−L∣∣∣∣ dt
t< ∞ (30)
we have f(N)(x0)→ L as N→ ∞.
MORAL: There are some evil artificial continuous functions for which pointwise convergence fails in many points. Butfor all practical examples from applications f is piecewise “slightly better than continuous”, and then we have pointwiseconvergence everywhere (28), and in the absence of jumps uniform convergence (29).
3.4 Calculus review: absolute convergence
Example: Consider the series 1− 12 +
13 −
14 + · · · . This is an alternating series ∑
∞k=1 ak with |ak+1| < |ak| and ak → 0 as
k→ ∞. By the alternating series theorem this series converges, and one can show
S = 1−12 +
13−
14 +
15−
16 +
17−
18 +
19 · · ·= ln2
We now reorder the terms in this series: we take two positive terms, one negative term, two positive terms, . . . . Despite thefact that we add the same terms as before (just in a different order) we now get a different value for the sum
S = 1+ 13−
12 +
15 +
17−
14 +
19 +
111−
16 + · · ·=
32 ln2
The problem is that the series is not absolutely convergent: ∑∞k=1 |ak|= 1+ 1
2 +13 +
14 + · · ·= ∞.
A series ∑∞k=1 ak is called absolutely convergent if
∞
∑k=1|ak|< ∞ .
In this case we have
• the series is convergent
• for any reordering we obtain the same sum
For a Fourier series ∑∞k=−∞ fke2πikx we have
∣∣ fke2πikx∣∣= ∣∣ fk
∣∣ · ∣∣e2πikx∣∣= ∣∣ fk
∣∣.Therefore
∞
∑k=−∞
∣∣ fk∣∣< ∞ means that the Fourier series is absolutely convergent.
Example: f (x) =
{1 for x ∈ [−1
2 ,0]0 for x ∈ [0, 1
2)from section 3.2. We found fk =
{0 for even k− 2i
πk for odd k.
We have ∑k=1,...,∞
k odd
1k= ∞,so the Fourier series ∑
∞k=−∞ fke2πikx is not absolutely convergent. Therefore it matters in which order
we add the terms.
If we use f(N)(x) = ∑∞k=−∞ fke2πikx we have for a fixed x pointwise convergence: f(N)(x)→ 1
2 [ f (x−0)+ f (x+0)] as N→∞.
Since the Fourier series is not absolutely convergent, adding the terms in a different order can give different results. Thisis why we had to specify that ∑
∞k=−∞ (· · ·) has to be interpreted as limN→∞ ∑
∞k=−∞ (· · ·). E.g., limN→∞ ∑
2Nk=−N fke2πikx will
in general give a different result. This is also important if we want to approximate the Fourier series numerically by usingfinitely many terms.
The same thing happens for any function f ∈ PW 1 with jumps: then∣∣ fk∣∣= O(|k|−1) and there is no absolute convergence.
We will prove in section 3.11 : ∑∞k=−∞
∣∣ fk∣∣< ∞ implies uniform convergence and that f (x) is continuous.
15
3.5 Convolutions
Convolutions can be used for signal processing. We can e.g. “smoothen” a signal using convolutions.
For 1-periodic functions f , g we define the convolution q = f ∗g by
q(x) =∫ 1
t=0f (t)g(x− t)dt (31)
Properties of the convolution:
• q is also 1-periodic
• f ∗g = g∗ f since using the change of variable s = x− t in (31) gives t = x− s , dt =−ds
q(x) =∫ x−1
s=xf (x− s)g(s)(−ds) =
∫ x
s=x−1f (x− s)g(s)ds =
∫ 1
s=0f (x− s)g(s)ds
(as the integrand is 1-periodic in s, we have∫ x
s=x−1(· · ·)ds =∫ 1
s=0(· · ·)ds.)
• ( f ∗g)′ = f ′ ∗g = f ∗g′: Assume that f ′exists and is continuous, g is continuous. Then we can take the derivative ddx
in (31) inside the integral:
q′(x) =∫ 1
t=0f (t)g′(x− t)dt = f ∗g′.
• If g ∈Tn then also q = f ∗g ∈Tn: For g(x) = ∑Nk=−N gke2πikx we get
q(x) =∫ 1
t=0f (t)
[N
∑k=−N
gke2πik(x−t)
]dt =
N
∑k=−N
(∫ 1
t=0f (t)e−2πiktdt
)︸ ︷︷ ︸
fk
gke2πikx =N
∑k=−N
qke2πikx (32)
withqk = fkgk (33)
A simple example of a convolution is smoothing with a “moving average”: Define the 1-periodic function gn for x ∈ [−12 ,
12)
by
gn(x) =
{n for x ∈ (− 1
2n ,12n)
0 otherwise(34)
Note that∫ 1/2−1/2 gn(x)dx = 1. For a 1-periodic function f the convolution qn = f ∗gn gives
qn(x) = n∫ x+1/(2n)
t=x−1/(2n)f (t)dt
which is the average of the function f over the interval [x− 12n ,x+
12n ].
Example: Here is the function g4(x): -0.4 -0.2 0.2 0.4
1
2
3
4
We consider f from (23) (red) and the convolution q = f ∗g4 (black, dashed):
16
-1.0 -0.5 0.5 1.0
-1.0
-0.5
0.5
1.0
It is easy to see that we must have q(x) = 1 for x ∈ [18 ,
38 ] since we only take the average of function values 1. Similarly,
q(x) =−1 for x ∈ [−38 ,−
18 ] since we only take the average of function values−1. For x ∈ [−1
8 ,18 ] the value of q(x) increases
from −1 to 1, with q(0) = 0. One can show: the convolution of two piecewise constant functions f ,g gives a continuouspiecewise linear function q = f ∗g.
If the function f is continuous the maximal error will converge to 0 (“uniform convergence”):
maxx∈R| f (x)−qn(x)| → 0 as n→ ∞ (35)
Proof: For any ε > 0 there exists δ > 0 so that |y− x| < δ implies | f (y)− f (x)| < ε . Hence we have for n ≥ 1/δ that| f (t)− f (x)|< ε for all t ∈ [x− 1
2n ,x+12n ]. Then qn(x)− f (x) = n
∫ x+1/(2n)t=x−1/(2n) ( f (t)− f (x))dt gives
|qn(x)− f (x)| ≤ n∫ x+1/(2n)
t=x−1/(2n)| f (t)− f (x)|︸ ︷︷ ︸≤ ε
≤ ε �
If max | f −qn| ≤ ε we have ‖ f −qn‖2 ≤∫ 1
0 ε2dx = ε2. I.e., uniform convergence implies ‖ f −qn‖ → 0 (convergence inmean square sense).
The functions gn become “more and more concentrated near 0” in the following sense: We have for n = 1,2,3, . . .
(P1) gn(x)≥ 0
(P2)∫ 1/2−1/2 gn(x)dx = 1
(P3) for every δ > 0: max[− 12 ,
12 ]\[−δ ,δ ]gn(x)→ 0 as n→ ∞
For any sequence of functions gn with these three properties we can prove uniform convergence of f ∗gn:
Lemma 2. Assume that the 1-periodic functions g1,g2,g3, . . . satisfy the properties (P1), (P2), (P3). For a 1-periodiccontinuous function f we have for qn := f ∗gn
maxx∈R| f (x)−qn(x)| → 0 as n→ ∞
Proof. For a given ε > 0 we want to show maxx∈R | f (x)−qn(x)| < ε if n is sufficiently large. Since f is continuous on[−1
2 ,12 ] there exists δ > 0 such that
for all x,y with |x− y|< δ : | f (x)− f (y)|< ε/2 (36)
17
With (P2) we obtain
qn(x)− f (x) =∫ 1/2
−1/2[ f (x− t)− f (x)]gn(t)dt
=∫[− 1
2 ,12 ]\[−δ ,δ ]
[ f (x− t)− f (x)]gn(t)dt︸ ︷︷ ︸À
+∫
δ
−δ
[ f (x− t)− f (x)]gn(t)dt︸ ︷︷ ︸Á
Because of (36) we have
Á <∫
δ
−δ
ε
2gn(t)dt ≤
∫ 1/2
−1/2
ε
2gn(t)dt
(P2)=
ε
2
Let M = max[−1/2,1/2] | f (x)|. Then we can use (P3) to find N such that
for n≥ N: max[− 12 ,
12 ]\[−δ ,δ ]gn(x)≤
ε
4M
which gives with | f (x− t)− f (x)| ≤ 2M that
À≤∫[− 1
2 ,12 ]\[−δ ,δ ]
2Mε
4Mdt ≤
∫[− 1
2 ,12 ]
2Mε
4Mdt =
ε
2
The “limit” of gn for n→∞ is something which is nonzero only for x = 0, but still has integral 1. This object is called Diracdelta. It is not a function, but a so-called generalized function. This means that pointwise evaluation δ (x) does not reallymake sense. However,
∫f (x)δ (x)dx still makes sense and gives f (0). See section 3.15 below for more details.
Here we consider the 1-periodic Dirac delta δper. This can be imagined as the “limit” of the 1-periodic functions gn(x) asn→ ∞, so that we formally have
∫ 1/2−1/2 F(x)δper(x)dx = F(0) for any continuous function F . Therefore we can formally find
the Fourier coefficients of g = δper:
gk =∫ 1/2
−1/2δper(x)e−2πikx︸ ︷︷ ︸
F(x)
dx = F(0) = e−2πi·0 = 1 for all k.
We formally have q = f ∗δper with
q(x) =∫ 1/2
t=−1/2δper(t) f (x− t)︸ ︷︷ ︸
F(t)
dt = F(0) = f (x)
So for g = δper we formally have q := f ∗g = f , and for the Fourier coefficients we have
qk = fk gk︸︷︷︸1
= fk
3.6 The Fourier approximation f(N) as convolution f(N) = f ∗DN with the Dirichlet kernel DN
We define the Dirichlet kernel DN ∈TN by
DN(x) =N
∑k=−N
1 · e2πikx
Therefore (32), (33) give for the convolution q = f ∗DN
q(x) = ∑k=−N
fke2πikx = f(N)(x)
18
Hence the Fourier approximation f(N) is the convolution with the Dirichlet kernel: f(N) = f ∗DN
The Dirichlet kernel is a 1-periodic real function. What does the graph look like?
We claim
DN(x) :=N
∑k=−N
u(k) =
2N +1 for x ∈ Zsin((2N +1)πx)
sin(πx)otherwise
(37)
Proof: For x = 0 we obtain DN(0) = 2N +1. For nonzero x ∈ [−12 ,
12 ] let r := e2πix
DN(x) = r−N + · · ·+ rN = r−N (1+ r+ · · ·+ r2N)= r−N r2N+1−1r−1
=rN+1− r−N
r−1=
rN+12 − r−N−1
2
r1/2− r−1/2 �
The functionsinx
xis defined for all x 6= 0. For x = 0 we have the limit 1. Therefore we define
sincx :=
{sinx
x for x 6= 01 for x = 0
Here is the graph of sincx (blue), together with ±1x (orange):
-6 π -5 π -4 π -3 π -2 π -π π 2 π 3 π 4 π 5 π 6 π
-0.2
-0.1
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.
Warning: The function sinc(x) in Matlab is sincMatlab(x) = sinc(πx). We will use the notations
sincx =sinx
x, sincπ x = sinc(πx) =
sin(πx)πx
We rewrite (37) using sinc and obtain: DN(x) is a 1-periodic function which for x ∈ [−12 ,
12 ] is given by
DN(x) = (2N +1)sincπ ((2N +1)x)
sincπ(x)(38)
For x 6= 0
DN(x) =1
πxsinc(πx)· sin((2N +1)πx)
19
Note that sinc(πx) is an even function which decreases from 1 to 2/π on the interval [0, 12 ]. Hence
G(x) :=1
π |x|sinc(πx)(39)
satisfies on [−12 ,
12 ]
1π |x|
≤ G(x)≤ 12 |x|
Here is the graph of D8(x) (blue) together with ±G(x) (orange, independent of N):
-8
17
-7
17
-6
17
-5
17
-4
17
-3
17
-2
17
-1
17
1
17
2
17
3
17
4
17
5
17
6
17
7
17
8
17
-1
2
1
2
-4
-3
-2
-1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
Since the function DN has negative values it does not satisfy (P1).
But it also does not satisfy (P3) since the orange curves are independent of N.
Note that the upper orange curves behave like c|x| which is not integrable on [−1
2 ,12 ]. We will now show that because of this
we have∫ 1/2−1/2 |DN(x)|dx→ ∞ as N→ ∞:
Consider∫ 1/2
0 |DN(x)|dx: We consider the N “humps” at [ j2N+1 ,
j+12N+1 ] for j = 0, . . . ,N−1: here N = 8
20
01
17
2
17
3
17
4
17
5
17
6
17
7
17
8
17
1
2
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
We have
|DN(x)| ≥|sin((2N +1)πx)|
π |x|
Let a j :=∫ ( j+1)/(2N+1)
j/(2N+1) DN(x)dx. Then the area of the hump on [ j2N+1 ,
j+12N+1 ] is for j = 0, . . . ,N−1
∣∣a j∣∣= ∫ ( j+1)/(2N+1)
j/(2N+1)|DN(x)|dx≥ 1
πj+1
2N+1
∫ ( j+1)/(2N+1)
j/(2N+1)|sin((2N +1)πx)|dx︸ ︷︷ ︸
12N +1
∫ 1
0sin(πx)dx︸ ︷︷ ︸2/π
=2
π2 ·1
j+1
so that ∫ 1/2
0|DN(x)|dx≥ 2
π2
(11+
12+ · · ·+ 1
N
)∫ 1/2
−1/2|DN(x)|dx→ ∞ as N→ ∞
Using this property one can show that there are continuous functions for which f(N)(0)→ ∞.
3.7 Fejer kernel, completeness of trigonometric basis e2πikx,k ∈ Z
In order to obtain nicer convergence properties we use smaller factors for the higher frequencies: We define the Fejer kernelσN ∈TN by
σN(x) =N
∑k=−N
(1− |k|
N +1
)· e2πikx (40)
Note that the factor 1− |k|N+1 decreases linearly from 1 to 0 as k = 0, . . . ,N +1.
Therefore (32), (33) give for the convolution qN = f ∗σN
qN(x) = ∑k=−N
(1− |k|
N +1
)fke2πikx
21
For each k we have 1− |k|N+1 → 1 as N→ ∞ . So we can hope that qN(x)→ ∑
∞k=−∞ fke2πikx = f (x) as N→ ∞.
The Fejer kernel is a 1-periodic real function. What does the graph look like?
We claim
σN(x) :=N
∑k=−N
(1− |k|
N +1
)u(k) =
N +1 for x ∈ Zsin2((N +1)πx)(N +1) · sin2(πx)
otherwise
Proof: For nonzero x ∈ [−12 ,
12 ] let r := e2πix. We use
(1+ r+ · · ·+ rN)2 = 1+2r+ · · ·+NrN−1 +(N +1)rN +NrN+1 + · · ·+1 · r2N
r−N ·
︷ ︸︸ ︷(rN+1−1
r−1
)2
︸ ︷︷ ︸(r
N2 +1− r−
N2
r−1
)2
= r−N +2r−N+1 + · · ·+Nr−1 +(N +1)r0 +Nr1 + · · ·+1 · rN
Hence with r := e2πikx
σN(x) =1
N +1[r−N +2r−N+1 + · · ·+Nr−1 +(N +1)r0 +Nr1 + · · ·+1 · rN]
=1
N +1
(r
N2 +1− r−
N2
r−1
)2
=1
N +1
(r
N+12 − r−
N+12
r12 − r−
12
)2
�
Using the sinc function we can write σN
σN(x) = (N +1)sinc2
π ((N +1)x)sinc2
π(x)
For x 6= 0 we get with G(x) from (39)
σN(x) =1
N +1·
G(x)2︷ ︸︸ ︷1
π2x2 sinc2(πx)·sin2 ((N +1)πx)
Here is the graph of σ8(x) (blue) together with (N +1)−1G(x)2 (orange, goes to zero as N→ ∞):
-4
9
-1
3
-2
9
-1
9
01
9
2
9
1
3
4
9
-1
2
1
2
1
2
3
4
5
6
7
8
9
22
Note that we have
1. σN(x)≥ 0
2. Since in (40) the coefficient for k = 0 is 1:∫ 1/2−1/2 σN(x)dx = 1.
3. We have σN(x)≤ 1N+1 ·
(2x
)2, so for |x| ≥ δ we have σN(x)≤ 1N+1 ·
4δ 2 → 0 as N→ ∞.
As σN satisfies (P1), (P2), (P3) we obtain from Lemma 2 that qN = σN ∗ f converges uniformly to f .
Hence we obtain
Theorem 3. Assume that f is 1-periodic and continuous. For any ε > 0 there exists a trigonometric polynomial qn ∈ Tn
such thatmax
x| f (x)−qn(x)| ≤ ε.
Remarks:
• We did NOT prove that max∣∣ f − f(N)
∣∣→ 0 as N→∞. There are continuous functions f where f(N) does not uniformlyconverge to f .
• However, Theorem 3 shows that the mean square error∥∥ f − f(N)
∥∥ converges to zero: Since f(N) is the best approxi-mation in TN we have
∥∥ f − f(N)
∥∥≤ ‖ f −q‖ for any q ∈TN and hence
∥∥ f − f(N)
∥∥2 ≤ ‖ f −qN‖2 =∫ 1
0| f (x)−qN(x)|2 dx≤ (max | f −qN |)2→ 0 as N→ ∞
Now we want to consider more general functions.
For a piecewise continuous function f with finitely many jumps at points z1, . . . ,zk we can construct a continuous functionfε such that ‖ f − fε‖ ≤ ε: This can be done by cutting out intervals [z j − δ ,z j + δ ] and replacing the function f with astraight line on these intervals.
This means: For any given ε we can first construct a function fε with ‖ f − fε‖ ≤ ε . Then we can use Theorem 3 to constructfor fε a function qn such that | fε(x)−qn(x)| ≤ ε and hence
‖ fε −qn‖2 =∫ 1
0| fε(x)−qn(x)|2 dx≤
∫ 1
0ε
2dx = ε2
so that we obtain‖ f −qn‖ ≤ ‖ f − fε‖+‖ fε −qn‖ ≤ ε + ε = 2ε
i.e., we can find trigonometric functions qn such that the L2-error ‖ f −qn‖ is arbitrarily small.
The same argument works for a square integrable function f ∈ L2, i.e.,∫ 1
0| f (x)|2 dx < ∞
(in the sense of a Lebesgue integral). For any ε > 0 it is possible to construct a continous function fε with ‖ f − fε‖ ≤ ε (seee.g. Theorem 3.14 in [Rudin: Real and Complex Analysis]).
This implies that any square integrable function can be approximated by trigonometric polynomials in the mean square sense:
Theorem 4. Assume that f satisfies∫ 1
0 | f (x)|2 dx < ∞ (in the Lebesgue sense). For any ε > 0 there exists a trigonometric
polynomial qn ∈Tn such that ‖ f −qn‖ ≤ ε .
We now obtain the proof of the convergence result (T1) in section 3.3:
23
Corollary 5. Assume that f satisfies∫ 1
0 | f (x)|2 dx < ∞. Let f(N)(x) = ∑
k=Nk=−N fke2πikx. Then we have∥∥ f − f(N)
∥∥→ 0 as N→ ∞ (41)∫ 1
0| f (x)|2 dx =
∞
∑k=−∞
∣∣ fk∣∣2 “Parseval identity” (42)
Proof. By Theorem 4 we can find for any given ε > 0 a function qN ∈ TN such that ‖ f −qN‖ < ε . Since f(N) is the bestapproximation of f in TN (see (6)) we have ∥∥ f − f(N)
∥∥≤ ‖ f −qN‖ ≤ ε
which gives (41). From (19) we have
‖ f‖2 =∥∥ f − f(N)
∥∥2+
N
∑k=−N
∣∣ fk∣∣2
As N→ ∞ we have∥∥ f − f(N)
∥∥→ 0. Hence ∑Nk=−N
∣∣ fk∣∣2 converges to ‖ f‖2 for N→ ∞.
For functions f ,g ∈ L2 we have the inner product ( f ,g) =∫ 1
0 f (x)g(x)dx and norm ‖ f‖= ( f , f )12 .
Let us denote by ~f =(
fk)
k∈Z the “infinite vector” of all Fourier coefficients.
For ~f , ~g we have the inner product(~f ,~g)= ∑
∞k=−∞ fkgk and the norm
∥∥∥~f∥∥∥= (~f , ~f) 12.
The Parseval identity states that ‖ f‖ =∥∥∥~f∥∥∥. We have seen in (3) that we can express an inner product (u,v) in terms of
norms ‖u+ v‖, ‖u+ iv‖, ‖u‖, ‖v‖. Therefore the Parseval identity implies
for all f ,g ∈ L2: ( f ,g) =(~f ,~g)
∫ 1
0f (x)g(x)dx =
∞
∑k=−∞
fkgk
3.8 Summary: Fourier series for 1-periodic functions
Here we summarize the key properties of the Fourier series for 1-periodic functions f ,g:
fk =∫ 1
0f (x)e−2πikxdx ⇔ f (x) =
∞
∑k=−∞
fke2πikx Inversion
∫ 1
0| f (x)|2 dx =
∞
∑k=−∞
∣∣ fk∣∣2 Parseval
q(x) =∫ 1
0f (t)g(x− t)dt ⇔ qk = fk gk Convolution
(43)
• Since the integrands are 1-periodic we can also use the limits∫ 1
2− 1
2(· · ·)dx in all of the above integrals.
• The statement f (x) = ∑∞k=−∞ fke2πikx is interpreted as limN→∞ ∑
Nk=−N(. . .) , see (T1), (T2), (T3) in section 3.3.
• The Parseval identity actually holds for inner products:∫ 1
0 f (x)g(x)dx = ∑∞k=−∞ fkgk.
24
3.9 Fourier series for L-periodic functions
So far we considered 1-periodic functions. But all results carry over if we have a different period:
Consider a function f (x) with period L > 0, i.e., we have f (x+L) = f (x) for all x ∈ R. Then the function F(x) := f (Lx) is1-periodic. Using our previous results for F we have
f ( Lx︸︷︷︸t
) = F(x) =∞
∑k=−∞
Fke2πikx =∞
∑k=−∞
Fke2πikt/L
With the change of variables t = Lx we obtain
fk = Fk =∫ 1
x=0f ( Lx︸︷︷︸
t)e−2πikx dx︸︷︷︸
L−1dt
=∫ L
t=0f (t)e−2πikt/LL−1dt
fk = L−1∫ L
0f (x)e−2πikx/Ldx ⇔ f (x) =
∞
∑k=−∞
fke2πikx/L Inversion
L−1∫ L
0| f (x)|2 dx =
∞
∑k=−∞
∣∣ fk∣∣2 Parseval
q(x) = L−1∫ L
0f (t)g(x− t)dt ⇔ qk = fk gk Convolution
(44)
Note:
• Because of the change of variables we obtain L−1 ∫ L0 · · · in place of
∫ 10 · · · on the left hand side.
• f is written as a linear combination of the functions e2πiξkx with frequencies ξk =kL
, k ∈ Z.
• Since the integrands are L-periodic we can also use the limits∫ L
2− L
2(· · ·)dx in all of the above integrals.
• The statement f (x) = ∑∞k=−∞ fke2πikx is interpreted as limN→∞ ∑
Nk=−N(. . .) , see (T1), (T2), (T3) in section 3.3.
• The Parseval identity actually holds for inner products: L−1 ∫ L0 f (x)g(x)dx = ∑
∞k=−∞ fkgk.
3.10 Pointwise convergence
We now prove (T2) in section 3.3.
Assume f ∈ PW 1. Since f(N) = f ∗DN and DN is even we have
f(N)(x0) =∫ 1
2
− 12
DN(t) f (x0− t)dt =∫ 1
2
− 12
DN(t) f (x0 + t)dt =∫ 0
− 12
DN(t) f (x0 + t)dt︸ ︷︷ ︸I−N
+∫ 1
2
0DN(t) f (x0 + t)dt︸ ︷︷ ︸
I+N
We claim thatI−N → 1
2 f (x0−0), I+N → 12 f (x0 +0) as N→ ∞
We define for t ∈ (0, 12 ]
g(t) :=f (x0 + t)− f (x0 +0)
sin(πt)
This function is piecewise continuous on (0, 12 ], and the limit at 0 is
limt→0t>0
g(t) = limt→0t>0
f (x0 + t)− f (x0 +0)t
· tsin(πt)
= D+ f (x0) ·1π
25
with the 1-sided derivative from the right D+ f (x0). Let us define g(−t) :=−g(t) so that g is an odd function on [−12 ,
12 ].
Since∫ 1
20 DN(t)dt = 1
2 we obtain for the second integral
I+N − 12 f (x0 +0) =
∫ 12
0DN(t) f (x0 + t)dt− 1
2 f (x0 +0) =∫ 1
2
0[ f (x0 + t)− f (x0−0)]DN(t)dt
=∫ 1
2
0
f (x0 + t)− f (x0−0)sin(πt)
sin((2N +1)πt)dt =∫ 1
2
0g(t)sin((2N +1)πt)dt
= 12
∫ 12
− 12
g(t)sin((2N +1)πt)dt = 12
∫ 12
− 12
g(t)12i
(e2πiNteπit − e−2πiNte−πit)dt
=14i
[F−N− GN
]where F(t) := g(t)eπit and G(t) := g(t)e−πit . These functions are piecewise continuous and hence in L2, so (22) givesF−N → 0 and GN → 0 as N→ ∞.
In the same way we can prove I−N −12 f (x0−0)→ 0.
3.11 Uniform convergence
We want to prove (T3) in section 3.3. We will use the following result:
For sequences uk,vk ∈ C for k ∈ N we have the Cauchy Schwarz inequality:∣∣∣∣∣ ∞
∑k=1
ukvk
∣∣∣∣∣≤(
∞
∑k=1|uk|2
)1/2(∞
∑k=1|vk|2
)1/2
Proposition 6. Assume f ∈ PW 1 without jumps. Then we have∞
∑k=−∞
∣∣ fk∣∣< ∞ , i.e., absolute convergence for the Fourier
series.
Proof. Since f ∈ PW 1 we have the derivative g := f ′ ∈ PW 0 with Fourier coefficients gk. The Bessel inequality (20) gives∑
∞k=−∞
|gk|2 ≤ ‖g‖2.
Using integration by parts we obtain using that f (t) and e−2πikt are continuous and 1-periodic
gk =∫ 1
0f ′(t)e−2πiktdt =
[f (t)e−2πikt
]1
0︸ ︷︷ ︸0
−∫ 1
0f (t)(−2πik)e−2πiktdt (45)
= 2πik fk
Hence fk = gk · (2πik)−1. Now we use Cauchy-Schwarz with uk := |gk| and vk := (2πik)−1
∞
∑k=1
∣∣ fk∣∣ ·1≤( ∞
∑k=1|gk|2
)1/2(∞
∑k=1
14π2k2
)
Since ∑∞k=1
1k2 < ∞ and ∑
∞k=1 |gk|2 < ∞ we obtain ∑
∞k=1
∣∣ fk∣∣< ∞. In the same way we also obtain ∑
−∞
k=−1
∣∣ fk∣∣< ∞
Theorem 7. If ∑∞k=−∞
∣∣ fk∣∣< ∞ we have for f(N)(x) = ∑
Nk=−N fke2πikx and f (x) = ∑
∞k=−∞ fke2πikx
• uniform convergence: maxx∈[0,1]∣∣ f(N)(x)− f (x)
∣∣→ 0 as N→ ∞
• f (x) is continuous
26
Proof. Let Sk := ∑Nk=−N
∣∣ fk∣∣. We have Sk→ S := ∑
∞k=−∞
∣∣ fk∣∣ as k→ ∞.
∣∣ f (x)− f(N)(x)∣∣= ∣∣∣∣∣ ∑
|k|>Nfke2πikx
∣∣∣∣∣≤ ∑|k|>N
∣∣ fk∣∣=: S−SN
We can now take the maximum over all x ∈ [0,1] and obtain maxx∈[0,1]∣∣ f (x)− f(N)(x)
∣∣≤ S−SN → 0 as N→ ∞.
Each of the functions f(N) is continuous. The “uniform limit theorem” from calculus states: If a sequence of continuousfunctions converges uniformly, the limit function is also continuous.
With a similar argument we can actually prove that the maximal error decays like N−1/2:
Proposition 8. Let f denote a 1-periodic function with∫ 1
0 | f ′(x)|2 dx < ∞. Then
maxx∈[0,1]
∣∣ f (x)− f(N)(x)∣∣≤ 1
π√
2
∥∥ f ′∥∥ ·N−1/2→ 0 as N→ ∞
Proof. We obtain for the maximal error
maxx∈[0,1]
∣∣ f (x)− f(N)(x)∣∣= max
x∈[0,1]
∣∣∣∣∣ ∑|k|>N
fke2πikx
∣∣∣∣∣︸ ︷︷ ︸≤ ∑|k|>N
∣∣ fke2πikx∣∣︸ ︷︷ ︸∣∣ fk
∣∣
≤ ∑|k|>N
∣∣ fk∣∣
For real sequences u j,v j we have the Cauchy Schwarz inequality:∣∣∣∣∣ ∞
∑k=1
ukvk
∣∣∣∣∣≤(
∞
∑k=1
u2k
)1/2(∞
∑k=1
v2k
)1/2
yielding
∑|k|>N
∣∣ fk∣∣= ∑
|k|>N
∣∣ fk∣∣ |k| · |k|−1 ≤
(∑|k|>N
∣∣ fk∣∣2 |k|2) 1
2
︸ ︷︷ ︸A
(∑|k|>N|k|−2
) 12
︸ ︷︷ ︸B
Let g := f ′, then∣∣ fk∣∣ · |k|= 1
2π|gk|, hence ∑|k|>N
∣∣ fk∣∣2 |k|2 ≤ 1
4π2 ∑∞k=−∞
|gk|2 and A≤ 12π‖g‖.
For B we use
∞
∑k=N+1
|k|−2 ≤∫
∞
x=Nx−2dx = N−1, B≤ (2N−1)
12 =√
2N−12
yielding
maxx∈[0,1]
∣∣ f (x)− f(N)(x)∣∣≤ 1
π√
2‖g‖ ·N−1/2→ 0 as N→ ∞ (46)
Remark: Here we only used that g = f ′ ∈ L2.For f ∈ PW 1 we can obtain a better result if we additionally assume that each f ′j is piecewise monotonic (or one can assumef ∈ PW 2, i.e., for each piece f ′j , f ′′j are continuous). Then one can show
∣∣ fk∣∣≤ c |k|−2 yielding
maxx∈[0,1]
∣∣ f (x)− f(N)(x)∣∣≤C ·N−1 (47)
This is the actual convergence rate which one observes for piecewise smooth functions without jumps.
27
3.12 D(x) = sinc(πx) and its antiderivative G(x) = 1π
Si(x)
For the function sincx :=
{sinx
x for x 6= 01 for x = 0
we consider the antiderivative
Si(x) :=∫ x
0sinc t dt
This is called the “sine integral function” Si(x). This cannot be expressed in terms of elementary functions. In Matlab youcan use sinint(x).
We will consider the function D(x) := sinc(πx) and its antiderivative G(x) :=1π
Si(πx) . Here are their graphs:
-7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7
-0.5
0
0.5
1
We have the following properties of G(x):
• one can show that∫
∞
−∞D(x)dx = 1. As D(x) = D(−x) we have
∫∞
0 D(x)dx = 12 .
• D(x) is positive on (0,1), negative on (1,2) , positive on (2,3) . . .. . D(x) has zeros at 1,2,3, . . .Hence G(x) has local maximum at 1, a local minimum at 2, a local maximum at 3, . . .
• let ak :=∫ k+1
ksinc(πx)dx . Note that a0 > 0, a1 < 0, a2 > 0, etc.
As∫ 1
0 sin(πx)dx = 2π
and |sinc(πx)| ≤ 1πx we have |ak| ≤ 2
π2 · 1k . We also have |ak+1| ≤ |ak|.
• We have G(k) = a0 +a1 + · · ·+ak−1.As k→ ∞ we obtain the alternating series a0 +a1 +a2 + · · ·= 1
2 .Since we have an alternating series with |ak+1| ≤ |ak| and ak→ 0 as k→ ∞ the “alternating series theorem” gives:G(k) = a0 + · · ·+ak−1 converges to 1
2 as k→ ∞, and the error satisfies∣∣G(k)− 1
2
∣∣≤ |ak| ≤ 2π2 · 1
k
We have for actually for all x > 0 that∣∣G(x)− 1
2
∣∣≤ 2π2 ·
1x
.
28
3.13 The Dirichlet kernel DN(x) and its antiderivative GN(x)
We consider the Dirichlet kernel DN(x) (1-periodic) and its antiderivative GN(x) :=∫ x
0 DN(t)dt. Here are the graphs ofDN/(2N +1) and GN for N = 8:
-.5-8 /17
-7 /17
-6 /17
-5 /17
-4 /17
-3 /17
-2 /17
-1 /17
0 1/17
2/17
3/17
4/17
5/17
6/17
7/17
8/17
.5
-0.5
0
0.5
1
DN
/(2N+1)
GN
We have the following properties of GN(x):
• we saw that∫ 1
2− 1
2DN(x)dx = 1. As DN(−x) = DN(x) we have
∫ 12
0 D(x)dx = 12 .
• DN(x) is positive on (0, 12N+1), negative on ( 1
2N+1 ,2
2N+1) , positive on ( 22N+1 ,
32N+1) . . .. . DN(x) has zeros at
12N+1 ,
22N+1 ,
32N+1 , . . .
Hence GN(x) has local maximum at 12N+1 , a local minimum at 2
2N+1 , a local maximum at 32N+1 , . . .
• let ak :=∫ (k+1)/(2N+1)
k/(2N+1)DN(x)dx for k = 0, . . . ,N−1, aN :=
∫ 1/2N/(2N+1) DN(x)dx. Note that a0 > 0, a1 < 0, a2 > 0, etc.
As∫ 1
0 sin(πx)dx = 2π
and |DN(x)| ≤ 12x we have |ak| ≤ 1
π· 1
k for k = 1,2, . . .. We also have |ak+1| ≤ |ak|.
• We have GN(k
2N+1) = a0 +a1 + · · ·+ak−1 for k = 1, . . . ,N and a0 +a1 + · · ·+aN = 12 .
Since we have an alternating series with |ak+1| ≤ |ak| the “alternating series theorem” gives:For k = 0, . . . ,N the error satisfies
∣∣GN(k
2N+1)−12
∣∣≤ |ak| ≤ 1π· 1
k , i.e. for x = k2N+1 we have
∣∣GN(x)− 12
∣∣≤ 1π· 1(2N+1)x .
We have actually for all x ∈ (0, 12 ] that
∣∣GN(x)− 12
∣∣≤ 1π· 1(2N +1)x
.
• GN
(s
2N +1
)= G(s)+O(N−2) : We get with the change of variables z = (2N +1)t
GN(s
2N +1) = (2N +1)
∫ s/(N+1)
t=0
sinc((2N +1)πt)sinc(πt)
dt =∫ s
z=0
sinc(πz)sinc(πz/(2N +1))
dz
= π−1 Si(πs)+O(N−2)
since sinc(h) = 1+O(h2) as h→ 0.
29
3.14 Gibbs phenomenon: behavior of error f(N)(x)− f (x) for functions with jumps and kinks
Consider a function f ∈ PW 1 with jumps. We want to investigate the behavior of the error
eN(x) := f(N)(x)− f (x)
Gibbs phenomenon for g(x) = 12 − x on [0,1]
We first consider the 1-periodic function g with g(x) = 12 − x for x ∈ [0,1). This function has a jump of size J = 1 at 0.
Here is g and the Fourier approximation g(10) :
-1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1
-0.5
-0.25
0
0.25
0.5
Here are the error functions eN(x) := g(N)(x)−g(x) for x ∈ (0, 12 ] and N = 5,10,20:
0 0.1 0.2 0.3 0.4 0.5-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
e5
(x)
e10
(x)
e20
(x)
We observe:
• for each N we have oscillations which decay like cx away from the jump at 0
• as N increases the amplitude of the oscillations at a point x decreases like Cx/N
• the oscillations have period 1N+ 1
2
• the functions eN( s
2N+1
)are almost the same for N = 5,10,20, and almost the same as G(s)− 1
2 :
30
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5
-0.05
-0.04
-0.03
-0.02
-0.01
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09e
5(s/11)
e10
(s/21)
e20
(s/41)
G(s)-.5
We will prove: Let dx denotes the closest distance from x to the jump, then
|eN(x)| ≤1π· J(2N +1)dx
(48)
Close to the jump at z0=0 we have
eN
(z0 +
s2N +1
)≈ J ·
[G(s)− 1
2 sign(s)]
(49)
where G(s) := π−1 Si(πs). For this specific function g this holds with an error term of O(N−2). But for general functionsf ∈ PW 1 which have several jumps, or different slopes to the left and right of the jump at z0 (49) holds with an error term ofO(N−1).
This means to the right of the jump we have
eN(z0 +0) =−12 J, eN(z0 +
12N +1
)≈ .0895J, eN(z0 +2
2N +1)≈−.0486J
To the left of the jump we have the opposite signs.
Claim: eN(x) = GN(x)− 12 for x ∈ (0, 1
2).
Proof: Note that for s ∈ [−1,1] we have g(s) =−s− 12 +
{1 s > 00 s < 0
.
Hence for x, t ∈ (0,1) we have g(x− t) = t− x− 12 +
{1 t < x0 t > x
and
g(N)(x) =∫ 1
t=0DN(t)g(x− t)dt =
∫ 1
t=0DN(t)(t− x− 1
2)dt +∫ x
t=0DN(t)dt
=∫ 1
t=0DN(t)(t− 1
2)dt︸ ︷︷ ︸0
−x∫ 1
t=0DN(t)dt︸ ︷︷ ︸1
+GN(x)
g(N)(x)− (12 − x)︸ ︷︷ ︸g(x)
= GN(x)− 12
31
Gibbs phenomenon for f ∈ PW 1
For a piecewise continuous differentiable function f ∈ PW 1 with jumps Jk := f (zk +0)− f (zk−0) at zk, k = 1, . . . ,K we seethat
F(x) := f (x)− J1 ·g(x− z1)−·· ·− JK ·g(x− zK)
does not have jumps and F ∈ PW 1 . If each f ′j is piecewise monotonic this also holds for F and (47) gives
maxx
∣∣F(N)(x)−F(x)∣∣≤ cN−1
and for each term J`g(x− z`) we get (48), (49). For f (x) = F(x)+J1 ·g(x− z1)+ · · ·+JK ·g(x− zK) we get the sum of theseterms.
Gibbs phenomenon for function with �kinks�
Assume that f is a piecewise smooth function without jumps. In this case we have (47)
maxx
∣∣ f(N)(x)− f (x)∣∣≤CN−1
The “bad points” are the “kinks” z j where f ′ has a jump of J: denoting the right and left sided derivative by D+,D−
J := D+ f (z j)−D− f (z j)
Example: Let f denote the 1-periodic function with f (x) = (x− x2)/2. Here J = D+ f (0)−D− f (0) = 12 − (−1
2) = 1. Hereare f and f(10) around the kink at 0:
-0.2 -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2
0
0.02
0.04
0.06
0.08
Here are the errors eN := f(N)− f for N = 5,10,20:
0 0.1 0.2 0.3 0.4 0.5-4
-2
0
2
4
6
8
1010
-3
e5(x)
e10
(x)
e20
(x)
At a kink z j we have an error∣∣ f(N)(z j)− f (z j)
∣∣≤ cN−1 whereas at a point x away from kinks we have∣∣ f(N)− f
∣∣≤CxN−2 .Here Cx decays like 1/dx where dx is the distance to the closest kink.
32
We notice that (2N +1)eN(z0 +s
2N+1) look similar for N = 5,10,20
0 1 2 3 4 5
-0.03
-0.02
-0.01
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
11e5
(s/11)
21e10
(s/21)
41e20
(s/41)
H(s)-s/2
Here H(s) is an antiderivative of G(s)H(s) := π
−2 cos(πs)+π−1s ·Si(πs)
and we obtain close to a kink z0 for the error eN := f(N)− f
eN
(z0 +
s2N +1
)≈ J
2N +1·[H(s)− 1
2 |s|]
In particular we have eN(z0)≈ J2N+1 ·
1π2 .
3.15 Generalized functions, derivatives and the Dirac delta
Functions and generalized functions on R
Let f denote a function defined on the real line R. Let e.g. f (x) denote the temperature in a room along a straight line. Fora given position x0 we have a corresponding temperature value f (x0) at this point. However, when we try to measure thetemperature with a thermometer we will not get the value at a single point. As the thermometer has finite size, we will geta weighted average of temperatures: We have a function ϕ(x) which is nonzero only on some small interval (a,b), and weobtain a weighted average
A =∫
∞
−∞
f (x)ϕ(x)dx
We use the notation A = 〈 f ,ϕ〉 which means: “test f with the test function ϕ(x)”. We assume that test functions ϕ satisfy
• ϕ and its derivatives ϕ ′,ϕ ′′,ϕ ′′′, . . . exist and are continuous
• ϕ is nonzero only on some bounded interval
Summary: For a “classical function” f (x) we can perform the following two operations:
(A) Sampling at a point x ∈ R: For any value x ∈ R we obtain an output value y = f (x) ∈ R
(B) Testing with a test function ϕ(x): For any test function ϕ(x) we obtain an output value Y = 〈 f ,ϕ〉 =∫∞
−∞f (x)ϕ(x)dx.
33
We have seen that there are things like the “Dirac Delta” which are not classical functions. Therefore we now define“generalized functions” (a.k.a. “distributions”). For a generalized function F the operation (A) may not make sense. How-ever, the operation (B) is always well defined: For any test function ϕ we obtain an output value Y = 〈 f ,ϕ〉.
Example: For a sequence of functions g(n) with (P1), (P2), (P3) we have for any continuous test function ϕ⟨g(n),ϕ
⟩=∫
∞
−∞
g(n)(x)ϕ(x)dx→ ϕ(0) as n→ ∞
So the “limit object” is the generalized function δ which maps any given test function ϕ to the output value
〈δ ,ϕ〉 := ϕ(0)
This generalized function is called Dirac delta. There is no classical function with this property: we would need a functionwith integral 1 which is only nonzero in the point x = 0.
Similarly we define δb as the Dirac delta concentrated in x = b, i.e.,
〈δb,ϕ〉= ϕ(b)
We can always define the derivative G = F ′ of a generalized function, yielding another generalized function G. For aclassical function f we have for any test function using integration by parts
⟨f ′,ϕ
⟩=∫ b
af ′(x)ϕ(x)dx = [ f (x)ϕ(x)]ba︸ ︷︷ ︸
0
−∫ b
af (x)ϕ ′(x)dx =−
⟨f ,ϕ ′
⟩where ϕ is zero outside of the interval (a,b). Therefore we define the generalized function F ′ by⟨
F ′,ϕ⟩
:=−⟨F,ϕ ′
⟩Example 1: Let F = δ3. Then F ′ is the generalized function which gives for a test function ϕ the output value⟨
F ′,ϕ⟩=−
⟨F,ϕ ′
⟩=−ϕ
′(3)
Example 2: Consider the Heaviside function H(x) =
{0 for x < 01 for x≥ 0
. It does not have a derivative which is a classical
function. However, if we consider H as a generalized function we obtain a derivative H ′: For a test function ϕ we get⟨H ′,ϕ
⟩=−
⟨H,ϕ ′
⟩=−
∫∞
−∞
H(x)ϕ ′(x)dx =−∫
∞
0ϕ′(x)dx =− [ϕ(x)]b0 = ϕ(0)
Hence we have that the derivative of the Heaviside function is the Dirac Delta:
H ′ = δ0
Example 3: Consider a piecewise differentiable function
f (x) =
{f1(x) for x < Af2(x) for x≥ A
where f1(x) has a continuous derivative on (−∞,A] and f2(x) has a continuous derivative on [A,∞). Let g(x) denote the“piecewise derivative”
g(x) :=
{f ′1(x) for x < Af ′2(x) for x≥ A
34
What is the derivative of f ′ as a generalized function? For a test function ϕ we have an interval (a,b) containing A andall points x with ϕ(x) 6= 0. Then
⟨f ′,ϕ
⟩=−
⟨f ,ϕ ′
⟩=−
∫ A
af1(x)ϕ ′(x)dx−
∫ b
Af2(x)ϕ ′(x)dx
= [− f1(x)ϕ(x)]Aa︸ ︷︷ ︸
− f1(A)ϕ(A)
+∫ A
af ′1(x)ϕ(x)dx+[− f2(x)ϕ(x)]
bA︸ ︷︷ ︸
f2(A)ϕ(A)
+∫ b
Af ′2(x)ϕ(x)dx
=∫
∞
−∞
g(x)ϕ(x)dx+( f2(A)− f1(A))ϕ(A)
yieldingf ′ = g+ J ·δA with the "jump size" J := f (A+0)− f (A−0) (50)
Periodic generalized functions
We can now define 1-periodic generalized functions. E.g., the 1-periodic Dirac delta δper is concentrated at the pointsj ∈ Z. Because of its graph it is also known as the “Dirac comb”.
Similarly, for a ∈ R the 1-periodic generalized function δper,a is concentrated at the points j + a with j ∈ Z. E.g., the
generalized function δper, 13
is concentrated at the points . . . ,−53 ,−
23 ,
13 ,
44 ,
73 , . . . .
For a 1-periodic classical function f and a 1-periodic test function ϕ we can define
〈 f ,ϕ〉per =∫
periodf (x)ϕ(x)dx
where the integral goes over a full period of length 1 (e.g. from 0 to 1, or from −12 to 1
2 ). For a 1-periodic generalizedfunction F and a 1-periodic test function ϕ the operation 〈F,ϕ〉per is well defined. E.g., for F = δper, 1
4and the test function
ϕ(x) = sin(2πx) we obtain⟨
δper, 14,ϕ⟩
per= ϕ(1
4) = sin(2π14) = sin(π
2 ) = 1.
3.16 Properties of Fourier coe�cients
If we apply certain operations on the function f , how is this reflected in the Fourier coefficients fk?
Changing x to −x: Let g(x) := f (−x) . With the change of variables s =−t we obtain
gk =∫ 1
2
t=− 12
f (−t)e−2πiktdt =∫ − 1
2
s= 12
f (s)e2πkis(−ds) =∫ 1
2
s=− 12
f (s)e2πkisds = f−k
hence gk = f−k .
Shifting the function by a: Let g(x) := f (x−a) , i.e., we shift the graph of the function a distance a to the right. Withthe change of variables s = t−a we obtain
gk =∫ 1
0f (t−a)e−2πiktdt =
∫ 1−a
−af (s)e−2πik(s+a)ds = e−2πika
∫ 1
0f (s)e−2πiksds = e−2πika fk
Hence gk = e−2πika fk .
35
Taking the derivative: Let g(x) := f ′(x) . Then integration by parts gives
gk =∫ 1
0f ′(x)e−2πikxdx =
[f (x)e−2πikx
]1
0︸ ︷︷ ︸0
−∫ 1
0f (x)(−2πik)e−2πikxdx = (2πik) fk
since f (x) and e−2πikx are 1-periodic functions. Hence gk = 2πik · fk .
Recall that functions with∫ 1
0 | f (x)|2 dx < ∞ correspond to Fourier coefficients fk with ∑
∞k=−∞
∣∣ fk∣∣2 < ∞.
For a generalized function F we can still define the Fourier coefficients Fk by
Fk =⟨
F,e−2πikx⟩
per
since ϕ(x) = e−2πikx is a 1-periodic test function.
For a generalized function F we can define the “reflected” generalized function F(−x), the shifted generalized functionF(x−a), and the derivative F ′, and the above rules for the Fourier coefficients still hold. E.g., for G = F ′ we have
Gk =⟨
F ′,e−2πikx⟩
per=−
⟨F,
ddx
(e−2πikx
)⟩per
=−⟨
F,(−2kπi)e−2πikx⟩
per= 2kπi
⟨F,e−2πikx
⟩per
= 2kπi · Fk
Example: We consider the 1-periodic function f with f (x) = 1 on [0, 12) and f (x) = −1 on [−1
2 ,0). Let us first find thederivative G = f ′ as a generalized function. The function f (x) is constant, except for jumps of size 2 and −2 at x = 0 andx = 1
2 , respectively. Hence we get with (50)
G = f ′ = 0+2δper−2δper, 12
Recall that the Fourier coefficients of g = δper are gk = 1 for all k ∈ Z. The generalized function h = δper, 12
is obtained by
shifting g = δper by a = 12 , hence the rule for the shifted function gives
hk = e−2kπi 12 · gk = (−1)k ·1
Hence G = 2δper−2δper, 12
has the Fourier coefficients
Gk = 2−2 · (−1)k =
{0 for even k4 for odd k
Now we can use G = f ′ to find the Fourier coefficients fk: From the derivative rule we have Gk = 2kπi · fk, hence
for k 6= 0: fk =Gk
2kπi=
{0 for even k
42kπi =−
2ikπ
for odd k
What about f0? Note that f0 = 〈 f ,1〉per is the average of f over a full period of length 1. In our example we have f0 =∫ 12− 1
2f (x)dx = 0. Recall that we obtained the same values for fk earlier in (25).
36
4 Periodic function, discrete time: discrete Fourier transform
4.1 Periodic functions for discrete time values, aliasing
So far we considered 1-periodic functions f (x) defined for x ∈ R, and we wrote this function as a linear combination of thebasis functions u(k)(x) = e2πikx for k ∈ Z.
Now we consider a step size 1/N and the discrete points x j = j/N for j ∈ Z. A discrete time signal is given by values f j atthe points x j, j ∈ Z. As the signal is periodic we have
f j+N = f j for j ∈ Z
Hence we only need to specify N values. By convention we use the values f0, . . . , fN−1:
~f = [ f0, . . . , fN−1]> ∈ CN .
We now want to use the values of the trigonometric functions u(k)(x) at the points x j: This function has the values
~u(k) :=[e2πik·0/N , . . . ,e2πik· j/N
]>We have however so-called aliasing: ~u(k+N) =~u(k)
Proof:u(k+N)(x j) = e2πi(k+N) j/N = e2πi jk/N e2πi j︸︷︷︸
1
= u(k)(x j)
The functions u(k)(x), u(k+N)(x), u(k+2N)(x), . . . have all exactly the same values at the grid points x j (on all the other pointsin R they are different, though):This means that the vectors~u(k) for all k ∈ Z are actually only N different vectors which keeprepeating.
Example: Let e.g. N = 5:. . . ,~u(−2)︸︷︷︸
~u(3), ~u(−1)︸︷︷︸~u(4)
, ~u(0), ~u(1), ~u(2), ~u(3), ~u(4), ~u(5)︸︷︷︸~u(0)
, ~u(6)︸︷︷︸~u(1)
, . . .
In order to represent the vector ~f = [ f0, . . . , f4]> ∈ C5 we need 5 basis functions. By convention we use the vectors
~u(0),~u(1),~u(2),~u(3),~u(4). We could also have picked the vectors ~u(−2),~u(−1),~u(0),~u(1),~u(2), but these are exactly the samevectors, just in a different order.
Given: values of signal in time domain: ~f = [ f0, f1, f2, f3, f4]> ∈ C5
Wanted: write ~f as linear combination of~u(0), . . . ,~u(4):
~f = f0~u(0)+ f1~u(1)+ f2~u(2)+ f3~u(3)+ f4~u(4)
We want to find the vector ~f =[
f0, f1, f2, f3, f4]> ∈ C5 which represents the signal in the frequency domain.
Once we have this vector ~f we can also write ~f as a linear combination of the vectors~u(−2)︸︷︷︸~u(3)
,~u(−1)︸︷︷︸~u(4)
,~u(0),~u(1),~u(2):
~f = f3~u(−2)+ f4~u(−1)+ f0~u(0)+ f1~u(1)+ f2~u(2)
37
4.2 The discrete Fourier transform FN
Given: values of signal in time domain: ~f = [ f0, f1, . . . , fN−1]> ∈ CN .
Wanted: write ~f as linear combination of~u(0), . . . ,~u(N−1):
~f = f0~u(0)+ f1~u(1)+ · · ·+ fN−1~u(N−1)
f j =N−1
∑k=0
fke2πik j/N
We need to find the coefficient vector ~f =[
f0, f1, . . . , fN−1].
We claim that the vectors~u(0), . . . ,~u(N−1) form an orthogonal basis of CN: We have(~u(k),~u(l)
)=
N−1
∑j=0
e2πik j/Ne−2πil j/N =N−1
∑j=0
a j a := e2πi(k−l)/N
For k = l we have a = 1 and hence (~u(k),~u(k)
)= N.
For k, l ∈ {0, . . . ,N−1} with k 6= l we have a 6= 1 and
(~u(k),~u(l)
)=
N−1
∑j=0
a j =aN−1a−1
=e2πi(k−l)−1
a−1=
1−1a−1
= 0.
Now we can use (5) to write a vector ~f = ( f0, . . . , fN−1) ∈ CN as a linear combination of the vectors~u(0), . . . ,~u(N−1):
~f =N−1
∑k=0
fk~u(k) fk =
(~f ,~u(k)
)(~u(k),~u(k)
) = 1N
N−1
∑j=0
f je−2πik j/N (51)
Given a vector ~f = [ f0, . . . , fN−1]> ∈ CN in the “time domain” we obtain a vector ~f =
[f0, . . . , fN−1
]∈ CN in the frequency
domain. This operation is called the discrete Fourier transform FN , its inverse is called F−1N :
~f = FN~f , ~f = F−1N
~f
4.3 Discrete Fourier transform for N = 4,8
Let ω := e2πi/N . This corresponds to a rotation by an angle of 2π/N. Then using the basis vectors
~u(0)=
111...1
, ~u(1)=
1ω
ω2
...ωN−1
, ~u(2)=
1
ω2
ω4
...ω2(N−1)
, ~u(3)=
1
ω3
ω6
...ω3(N−1)
, . . . ,~u(N−1)=
1
ωN−1
ω2(N−1)
...ω(N−1)(N−1)
we want to write a given ~f ∈ CN as ~f = f1~u(1)+ · · ·+ fN−1~u(N−1). We have
f j =N−1
∑k=0
fkωjk, fk =
1N
N−1
∑j=0
f jω− jk
For N = 4 we have ω = e2πi/4 = eiπ/2 = i which is a rotation by π
2 , i.e., 90°.
38
Therefore we get the basis vectors
~u(0) =
1111
, ~u(1) =
1i−1−i
, ~u(2) =
1−11−1
, ~u(3) =
1−i−1i
Imagine a wheel which starts in position at time 0, and rotates with a speed of k rotations per second (k > 0 meanscounterclockwise, k < 0 means clockwise). We take snapshots N times per second: at times 0
4 ,14 ,
24 ,
34 (then it just repeats).
Note that due to aliasing~u(3) is the same as~u(−1). If we show the snapshots of~u(3) as a movie with N = 4 frames per secondthe wheel would appear to rotate backwards with 1 rotation per second, rather than forward with 3 rotations per second. Thisis the reason that in movies wheels appear to rotate backwards sometimes.
~u(0) =
, ~u(1) =
, ~u(2) =
, ~u(3) =
Therefore we can write ~f = F−1N
~f and ~f = FN~f asf0f1f2f3
=
1 1 1 11 i −1 −i1 −1 1 −11 −i −1 i
f0
f1
f2
f3
,
f0
f1
f2
f3
=14
1 1 1 11 −i −1 i1 −1 1 −11 i −1 −i
f0f1f2f3
We get for example
F4
1000
=14
1111
, F4
0123
=
32
−12 +
12 i
−12
−12 −
12 i
For N = 8 we take snapshots 8 times per second at times 0
8 , . . . ,78 and obtain the basis vectors
~u(0) =
,~u(1) =
,~u(2) =
,~u(3) =
,~u(4) =
,~u(5) =
,~u(6) =
,~u(7) =
where ω = = (1+ i)/
√2 , = (−1+ i)/
√2, = (−1− i)/
√2, = (1− i)/
√2 . Note that ~u(0) is a fixed wheel.
~u(1),~u(2),~u(3) are wheels rotating with 1,2,3 rotations per second whereas ~u(7) =~u(−1),~u(6) =~u(−2),~u(5) =~u(−3) are wheelsrotating with −1,−2,−3 rotations per second (i.e., clockwise). For~u(4) =~u(−4) both interpretations are equally valid.
4.4 Discrete Fourier transform in Matlab
The discrete Fourier transform FN maps the vector ~f = [ f0, . . . , fN−1] in the time domain to the vector ~f =[
f0, . . . , fN−1]
in
the frequency domain. The inverse discrete Fourier transform F−1N maps the vector ~f back to the vector ~f :
~f = FN~f , ~f = F−1N
~f
39
As an example consider the vector ~f = [1,1,1,1] in the time domain. Since this corresponds to the function f (x) = 1 theFourier vector is ~f = [1,0,0,0]:
F4[1,1,1,1] = [1,0,0,0], F−14 [1,0,0,0] = [1,1,1,1].
We use the m-files four.m
function fh = four(f)fh = fft(f)/length(f);
and ifour.m
function f = ifour(fh)f = ifft(fh)*length(fh);
Then we can compute the discrete Fourier transform in Matlab as follows:
>> f = [1,1,1,1]f = 1 1 1 1>> fh = four(f)fh = 1 0 0 0>> ifour(fh)ans = 1 1 1 1>> four([1 0 0 0])ans =
0.25 0.25 0.25 0.25>> four([0 1 2 3])ans = 1.5 + 0i -0.5 + 0.5i -0.5 + 0i -0.5 - 0.5i
In Matlab all indices are shifted by 1: You can access components of a Matlab vector v using v(1), v(2), ... . Note thatMatlab begins counting with 1, whereas our signal and Fourier vectors are indexed from 0 to N−1. Therefore in Matlab
f j corresponds to f(j+1)
fk corresponds to fh(k+1)
In our example we have ~f =[
f0, f1, f2, f3]= [1,0,0,0] and f0 corresponds to fh(1) in Matlab.
4.5 Properties of the discrete Fourier transform
Here we summarize the key properties:
fk =1N
N−1
∑j=0
f je−2πik j/N ⇔ f j =N−1
∑k=0
fke2πik j/N Inversion
1N
N−1
∑j=0
∣∣ f j∣∣2 =
N−1
∑k=0
∣∣ fk∣∣2 Parseval
q j =1N
N−1
∑`=0
f`g j−` ⇔ qk = fk gk Convolution
(52)
Recall that (~u(k),~u(`)
)=
{N for k = l0 for k 6= l
This gave us the formula for fk in (51), and (7) gives
(~f , ~f ) =N−1
∑k=0
∣∣ fk∣∣2(~u(k),~u(k))︸ ︷︷ ︸
N
40
yielding the Parseval formula. The Parseval formula implies that inner products in the time domain correspond to inner
products in the frequency domain:1N
(~f ,~g)=(~f ,~g)
, i.e.,
1N
N−1
∑j=0
f jg j =N−1
∑k=0
fkgk
Note that the sums on the left hand side in (52) correspond to the integrals on the left hand side in (43) if we apply a midpointrule to the integrals.
For the convolution we plug in the Fourier representation of g j−` and switch the order of the sums:
q j =1N
N−1
∑`=0
f`g j−` =1N
N−1
∑`=0
f`N−1
∑k=0
gke2πi( j−`)k/N
q j =N−1
∑k=0
(N−1
∑`=0
f`e−2πi`k/N
)︸ ︷︷ ︸
fk
gke2πik j/N
Hence we have q j = ∑N−1k=0 qke2πik j/N with qk = fkgk. Remember that the signals are periodic (wrap-around): g j = g j+N . So
g j−` with j− ` < 0 has to be interpreted in this way.
4.6 Interpretation of f0, . . . , fN−1 and the interpolating function f (x)
Case of odd N
Assume we have for N = 5 a vector ~f = [ f0, . . . , f4] and we compute the discrete Fourier transform ~f =[
f0, f1, f2, f3, f4].
Somehow these coefficients fk tell us something about which frequencies are present in our signal. However, it is not quiteobvious how this works.
First consider the function
F(x) = f0u(0)(x)+ f1u(1)(x)+ f2u(2)(x)+ f3u(3)(x)+ f4u(4)(x)
This function satisfies F(x j) = f j for j = 0, . . . ,N, i.e., it passes through the points of our discrete signal (“the function F(x)interpolates the given data points”). However, this is not very useful:
• Even if the values f0, . . . , f4 are real, the values of F(x) will be complex and non-real for x between the grid points.The problem is that we use only e2πikx for k ≥ 0, so we cannot get terms like cos(2πx) =
(e2πix + e−2πix
).
• We use functions u(k) with frequencies up to k = 4. So we use functions with high frequencies to represent data on afairly coarse grid.
But remember that on the grid points x j = j/N the function u(3) coincides with u(−2), and the function u(4) coincides withu(−1) (for x 6= x j these functions are different though). Hence we can consider the function
f (x) = f0u(0)(x)+ f1u(1)(x)+ f2u(2)(x)+ f3u(−2)(x)+ f4u(−1)(x)
which still passes through the given data points. Now we have k = −2,−1,0,1,2 and therefore f ∈ T2. Therefore we canwrite f in the form
f (x) = a0 +a1 cos(2πx)+b1 sin(2πx)+a2 cos(2π ·2x)+b2 sin(2π ·2x).
This function f (x) is much nicer than F(x):
• If the values f0, . . . , f4 are real, the values of f (x) will be real for all x.
• We use a function in T2 with frequencies only up to k = 2.
41
For general odd N = 2n+1 the Fourier vector[
f0, f1, . . . , fN−1]
yields the interpolating function f ∈Fn given by
f (x) = f0u(0)(x)+ f1u(1)(x)+ · · ·+ fnu(n)(x)+ fN−nu(−n)(x)+ · · ·+ fN−1u(−1)(x) (53)
Therefore the entries of the Fourier vector should be interpreted as follows:
frequency 0 1 2 · · · n n · · · 2 1f0 f1 f2 · · · fn fn+1 · · · fN−2 fN−1
So f0 corresponds to frequency 0 with u(0)(x) = 1, it gives the mean value of our data. Then f1 and fN−1 together tell usabout frequency 1. The highest frequency terms fn, fn+1 for frequency n are in the middle of the vector.
Case of even N
Consider for N = 6 a discrete signal a vector ~f = [ f0, . . . , f5] with the discrete Fourier transform ~f =[
f0, f1, f2, f3, f4, f5].
Here the “bad” function F(x) would be
F(x) = f0u(0)(x)+ f1u(1)(x)+ f2u(2)(x)+ f3u(3)(x)+ f4u(4)(x)+ f5u(5)(x).
On the grid points x j the function u(5) coincides with u(−1), the function u(4) coincides with u(−2) and the function u(3)
coincides with u(−3)(x). So we replace f4u(4)(x)+ f5u(5)(x) with f4u(−2)(x)+ f5u(−1)(x). What should we do with f3u(3)(x)?If we leave it as it is, or if we replace it with f3u(−3)(x) we would get something “unbalanced”, causing a complex functionfor real data. The answer is to replace f3u(3)(x) with f3
12
[u(3)(x)+u(−3)(x)
], yielding the function
f (x) = f0u(0)(x)+ f1u(1)(x)+ f2u(2)(x)+ f312
[u(3)(x)+u(−3)(x)
]+ f4u(−2)(x)+ f5u(−1)(x)
which still passes through the given data points. Note that
12
[u(3)(x)+u(−3)(x)
]= 1
2
[e2πi·3x + e−2πi·3x]= cos(2π ·3x)
so that we can write f in the form
f (x) = a0 +a1 cos(2πx)+b1 sin(2πx)+a2 cos(2π ·2x)+b2 sin(2π ·2x)+a3 cos(2π ·3x)
We denote these functions by T3,cos since we take the functions in T3, but only use the cosine term for frequency 3. Notethat it does not make sense to use sin(2π ·3x) since this function is zero at all grid points x j = j/6.
Tn,cos := span{
1︸︷︷︸freq 0
,cos(2πx),sin(2πx)︸ ︷︷ ︸freq 1
, . . . ,cos(2π(n−1)x) ,sin(2π(n−1)x)︸ ︷︷ ︸freq n−1
,cos(2πnx)︸ ︷︷ ︸freq n
}
For general even N = 2n the Fourier vector ( f0, f1, . . . , fN−1) yields the interpolating function f ∈Tn,cos given by
f (x) = f0u(0)(x)+ f1u(1)(x)+ · · ·+ fn−1u(n−1)(x)+ fn12
(u(n)(x)+u(−n)(x)
)+ fN−n+1u(−n+1)(x)+ · · ·+ fN−1u(−1)(x) (54)
Therefore the entries of the Fourier vector should be interpreted as follows:
frequency 0 1 2 · · · n−1 n n−1 · · · 2 1f0 f1 f2 · · · fn−1 fn fn+1 · · · fN−2 fN−1
42
Summary
Let us define the N dimensional space VN of trigonometric polynomials:
VN :=
{Tn if N = 2n+1 is oddTn,cos if N = 2n is even
Then we obtain
Proposition 9. For given samples f j at x j = j/N for j = 0, . . . ,N−1 there is a unique interpolating function f ∈ VN . We
can find f using ~f := FN~f and
f :=
{f0u(0)+ f1u(1)+ · · ·+ fnu(n)+ fn+1u(−n)+ · · ·+ fN−1u(−1) N = 2n+1 oddf0u(0)+ f1u(1)+ · · ·+ fn−1u(n−1)+ fn
12
[u(n)+u(−n)
]+ fn+1u(−n+1)+ · · ·+ fN−1u(−1) N = 2n even
(55)
Proof. Obviously f ∈VN . Evaluating f at 0N , . . . ,
N−1N gives for each term u(k) the vector ~u(k) . For terms with k < 0 we use
aliasing~u(k) =~u(k+N), and obtain[
f ( 0N ), . . . , f (N−1
N )]= f0~u(0)+ · · ·+ fN−1~u(N−1) = ~f . We have a one-to-one correspondence
of Fourier vectors ~f ∈ CN and functions in VN :
CN FN→ CN ↔ VN
~f FN→ ~f ↔ f
Since the mapping FN : CN→CN is also one-to-one we obtain: For any given vector ~f ∈CN in the time domain there existsa unique interpolating function f ∈VN .
A Fourier vector[
f0, . . . , fN−1]
stands for a function f ∈VN given by the following linear combinations of u(−n), . . . ,u(n):
odd N = 2n+1 :f0 f1 f2 · · · fn fn+1 · · · fN−2 fN−1
u(0) u(1) u(2) · · · u(n) u(−n) · · · u(−2) u(−1) f ∈Tn
even N = 2n :f0 f1 f2 · · · fn−1 fn fn+1 · · · fN−2 fN−1
u(0) u(1) u(2) · · · u(n−1) 12 [u
(n)+u(−n)] u(−n+1) · · · u(−2) u(−1) f ∈Tn,cos
4.7 �Upsampling�: Evaluating f (x) on a �ner grid
Recall that a Fourier vector ~f =∥∥ f0, . . . , fN−1
∥∥ corresponds to a function f ∈VN :
• f ∈Tn given by (53) for odd N = 2n+1
• f ∈Tn−1,cos given by (54) for even N = 2n.
Applying F−1N to ~f gives back the values f0, . . . , fN−1 on the grid points x j = j/N.
Often we would like to evaluate the function f (x) on a finer grid X j = j/M , j = 0, . . . ,M−1 with M > N. We can do this
in the following way: E.g., for a vector ~f =[
f0, f1, f2, f3, f4]
of length N = 5 we have the function
f (x) = f3u(−2)(x)+ f4(x)u(−1)(x)+ f0u(0)(x)+ f1u(1)(x)+ f2(x)u(2)(x) ∈T2
The function does not change if we add zero terms with u(−3)(x) and u(3)(x):
f (x) = 0︸︷︷︸g4
·u(−3)(x)+ f3︸︷︷︸g5
u(−2)(x)+ f4︸︷︷︸g6
u(−1)(x)+ f0︸︷︷︸g0
u(0)(x)+ f1︸︷︷︸g1
u(1)(x)+ f2︸︷︷︸g2
u(2)(x)+ 0︸︷︷︸g3
·u(3)(x)
43
which corresponds to the Fourier vector~g =
[f0, f1, f2, 0, 0, f3, f4
]which is obtained from ~f by “sticking zeros into the middle of the vector”. Now applying the inverse Fourier transform
~g := F−17
~g
gives the values g j = f ( j/M) of the function f on the finer grid.
If we start instead with a vector ~f =[
f0, f1, f2, f3]
of length N = 4 we obtain the interpolating function
f (x) = 12 f2u(−2)(x)+ f3(x)u(−1)(x)+ f0u(0)(x)+ f1u(1)(x)+ 1
2 f2(x)u(2)(x) ∈T2,cos
therefore the vector ~g of length M = 7 which corresponds to the same interpolating function is
~g =[
f0, f1,12 f2, 0, 0, 1
2 f2, f3]
Summary: A vector ~f =[
f0, . . . , fN−1]
corresponds to an interpolating function f (x) ∈
{Tn if N = 2n+1 oddTn,cos if N = 2n even
. A
vector ~g = (g0, . . . , gM−1) of length M > N which corresponds to the same interpolating function is given by
~g =
{[f0, . . . , fn, 0, . . . ,0, fn+1, fN−1
]if N = 2n+1 odd[
f0, . . . , fn−1,12 fn, 0, . . . ,0, 1
2 fn, fn+1, . . . , fN−1 f]
if N = 2n even
This is implemented in the Matlab function fext (see m-file on course web page). We can use this to plot the interpolatingfunction f on a finer grid with M points:
x = (0:5)/6; % grid of given signal
X = (0:511)/512; % fine grid for plotting
f = [0 0 0 1 1 1]; % given signal
fh = four(f);
gh = fext(fh,512); % extend to length 512
g = ifour(gh);
plot(x,f,’o’,X,g)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.2
0
0.2
0.4
0.6
0.8
1
1.2
For M > N we have VN ⊂VM, this corresponds to the function fext for the Fourier vectors. The corresponding operation inthe time domain is called “upsampling”:
CN FN→ CN ↔ VN
upsampling ↓ ↓ fext ∩CM FM→ CM ↔ VM
4.8 �Low-pass �lter�: Least squares approximation with lower frequency functions
We are given the values f j at the points x j =j
N for j = 0, . . . ,N−1. We have seen that we can obtain an interpolating function
f ∈VN =
{Tn for odd N = 2n+1Tn1,cos for even N = 2n
44
If we use a space Tm or Tm,cos with m < n we cannot expect to find a function g(x) which passes through all points. The bestwe can do is to minimize the least squares error
N−1
∑j=0
∣∣g(x j)− f j∣∣2 . (56)
Because of the orthogonality of the vectors~u(k) (see section 4.2) we can obtain the least squares approximation by using onlyterms which correspond to our subspace (see section 2.3): We have a vector ~f ∈CN and using the discrete Fourier transformwe can write it as
~f =N−1
∑k=0
fk~u(k)
Case of odd M = 2m+1: We want to approximate the values f j using functions Tm with M = 2m+1 < N. For exampleconsider T2 where m= 2 and M = 2m+1= 5. In this case we want to use the frequencies−2,−1,0,1,2. Because of aliasingthe corresponding vectors are~u(N−2),~u(N−1),~u(0),~u(1),~u(2). The best approximation ~G ∈ span
{~u(N−2),~u(N−1),~u(0),~u(1),~u(2)
}is obtained as
~G = fN−2~u(N−2)+ fN−1~u(N−1)+ f0~u(0)+ f1~u(1)+ f2~u(2)
and corresponds to the Fourier vector ~G given by[G0, G1, . . . , GN−1
]=[
f0, f1, f2,0, . . . ,0, fN−2, fN−1]
where we replace high frequency coefficients with zeros. The function g(x) corresponds to the Fourier vector ~g of lengthM = 2m+1 = 5 given by
[g0, g1, g2, g3, g4] =[
f0, f1, f2, fN−2, fN−1]
where we “remove the high frequency coefficients from the middle” of the vector ~f .
To approximate the data values using Tm with M = 2m+1 < N we use
[g0, . . . , gM−1] :=[
f0, f1, . . . , fm, fN−m, . . . , fN−1]
Case of even M = 2m: We want to approximate the values f j using functions Tm,cos with M = 2m < N. Here we use forthe highest frequency m only the cosine term. For example consider T2,cos = span{1,cos(2πx),sin(2πx),cos(4πx)} where
m = 2 and M = 2m = 4. The best approximation corresponds to the Fourier vector ~G given by
[G0, G1, . . . , GN−1
]=
[f0, f1,
f2 + fN−2
2,0, . . . ,0,
f2 + fN−2
2, fN−1
].
The function g(x) corresponds to the Fourier vector ~g of length M = 2m = 4 given by
[g0, g1, g2, g3] =[
f0, f1, f2 + fN−2, fN−1].
To approximate the data values using Tm,cos with M = 2m < N we use
[g0, . . . , gM−1] :=[
f0, f1, . . . , fm−1, fm + fN−m, fN−m+1, . . . , fN−1]
Summary: We can approximate a data vector [ f0, . . . , fN−1] of length N using a trigonometric space of dimension M < Nas follows: define the vector~g of length M by
[g0, . . . , gM−1] :=
{[f0, f1, . . . , fm, fN−m, . . . , fN−1
]for odd M = 2m+1[
f0, f1, . . . , fm−1, fm + fN−m, fN−m+1, . . . , fN−1]
for even M = 2m
45
Then the corresponding function
g(x) ∈
{Tm for odd M = 2m+1Tm,cos for even M = 2m
is the best least squares approximation, i.e., it minizes (56).
This operation is implemented in the Matlab function ftrunc (see m-file on course web page).
Example: We consider a discrete signal which has value 0 for x ∈ [0, 12) and value 1 for x ∈ [1
2 ,1). We want to find the bestleast squares approximation using a function g ∈T4, i.e., M = 9.
x = (0:511)/512; % grid
f = (x>=.5); % signal: 1 for x>=.5, 0 else
fh = four(f);
gh = ftrunc(fh,9); % truncate with M=9
Gh = fext(gh,512); % extend for plotting
G = ifour(Gh);
plot(x,f,x,G)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.2
0
0.2
0.4
0.6
0.8
1
1.2
We are given a discrete signal ~f ∈ CN in the time domain. We want to apply a “low-pass filter”. This means: Find the bestleast squares approximation by a discrete signal ~G which corresponds to a function g ∈VM with M < N. “Best least squareapproximation” means that
∥∥∥~f − ~G∥∥∥ is minimal.
~f ∈ CN FN→ CN ↔ VN
↓ ftrunc ↓ orth. projectionlowpass filter ↓ CM ↔ VM
↓ fext ∩~G ∈ CN FN→ CN ↔ VN
4.9 Sampling a function
Sampling a function F at points j/N
We can characterize a 1-periodic function F(x) by its Fourier coefficients Fk, k ∈ Z.
Assume that F ∈ PW 1 without jumps (each piece has continuous derivative, the function may have kinks). Then we showedthat the Fourier series ∑
∞k=−∞ Fke2πikx converges absolutely, i.e., ∑
∞k=−∞
∣∣Fk∣∣< ∞.
Evaluating this function on grid points x j = j/N, j = 0, . . . ,N−1 gives values
f j = F( j/N)
with f j+N = f j. This process which takes us from the function F(x) to the vector ~f = ( f0, . . . , fN−1) is called sampling.Obviously, information about the behavior of f between the grid points gets lost during sampling.
By applying the discrete Fourier transform FN to ~f we obtain the Fourier coefficients(
f0, . . . , fN−1).
How are Fourier coefficients(
f0, . . . , fN−1)
related to the original Fourier coefficients Fk, k ∈ Z?
46
Note that we have F(x) = ∑∞k=−∞ Fku(k)(x) and hence for x j = j/N
F(x j) =∞
∑k=−∞
Fku(k)(x j) (57)
Now remember the “aliasing property” of u(k)(x): On the grid points x j = j/N we have
· · ·= u(k−N)(x j) = u(k)(x j) = u(k+N)(x j) = u(k+2N)(x j) = · · ·
Therefore we can simplify (57) by collecting the identical terms together:
F(x j)︸ ︷︷ ︸f j
=N−1
∑k=0
( ∞
∑`=−∞
Fk+`N
)︸ ︷︷ ︸
fk
u(k)(x j)
Note that we have now reordered the terms in the sum. Since we have absolute convergence ∑∞k=−∞
∣∣Fk∣∣ < ∞ the sum after
reordering is still F(x j).
We see that we obtain the discrete Fourier representation f j = ∑N−1k=0 fku(k)(x j). We have shown
fk =∞
∑`=−∞
Fk+`N (58)
This is how we get from the coefficients Fk, k ∈ Z of the original function to the coefficents f0, . . . , fN−1 of the sampledversion.
Note: Since ∑∞k=−∞
∣∣Fk∣∣< ∞ we have ∑
∞`=−∞
∣∣Fk+`N∣∣< ∞. I.e., the series (58) is absolutely convergent and therefore conver-
gent.
47
This operation in the frequency domain is called “periodization”:
• start with the sequence of Fourier coefficients(Fk)
k∈Z which decay to 0 for k→±∞
• take the shifted sequences(Fk+`N
)k∈Z for `= . . . ,−2,−1,0,1,2, . . . and add them together
• this yields a periodic sequence fk = ∑∞`=−∞ Fk+`N with fk+N = fk, so we only need to specify ~f =
[f0, . . . , fN−1
]
0
0.2
0.4
0.6
0.8
1
-30 -15 0 15 30
0
0.2
0.4
0.6
0.8
1
-30 -15 0 15 30
Conclusion: Sampling in time domain corresponds to periodization in frequency domain.
4.10 Sampling theorem, Nyquist frequency
When we sample a 1-periodic function F(x) at the grid points x j =j
N we obtain the discrete values f j := F( j/N). TheFourier coefficients
(Fk)
k∈Z and(
fk)
k=0,...,N are related by (58). If we only have the sampled values f j we cannot hope toreconstruct the function F(x): we don’t know what happens between the points x j = j/N. For the Fourier coefficients weonly know the sums
f0 = · · ·+ F−N + F0 + FN + F2N + · · · , f1 = · · ·+ F−N+1 + F1 + FN+1 + F2N+1 + · · · , etc.
so we cannot hope to recover Fk.
Assume that the function F(x) is band-limited in the sense that F ∈VN , and we are given the samples f j := F( jN ), j =
0, . . . ,N−1.
How to recover the function F from the samples f j: Let ~f := FN~f and define the interpolating function f ∈VN by (55).
By Proposition 9 the interpolating function in VN is unique, hence F = f .
On the other hand, if we only know that F ∈ VN+1 we cannot reconstruct the function F from the samples F( jN ), j =
0, . . . ,N−1: There is a nonzero function F ∈VN+1 which is zero at all points j/N:
48
For N = 2n even let F(x) = sin(2πnx) ∈Tn =V2n+1. Then F( j/N) = sin(
2πn j2n
)= sin(π j) = 0.
For N = 2n+1 odd let F(x) = cos(2πnx)− cos(2π(n+1)x). Then 2cos(2πnx) = u(−n)+u(n) has the samples
~u(−n)+~u(n)aliasing= ~u(−n+N)+~u(n−N) =~u(n+1)+~u(−n−1)
which are the samples of the function u(n+1)+u(−n−1) = 2cos(2π(n+1)x).
This can be formulated as the sampling theorem:
Proposition 10. Assume that F ∈VM and we are given the samples F( jN ), j = 0, . . . ,N−1. The function F can by uniquely
determined from these samples if and only if N ≥M.
For odd M = 2m+1 this means:
Proposition 11. Assume that F ∈Tm and we are given the samples F( jN ), j = 0, . . . ,N−1. The function F can by uniquely
determined from these samples if and only if N ≥ 2m+1.
Note: Sampling with sampling frequency 2m is not sufficient since sin(2πmx) is zero at the points x = j2m .
If we have samples with sampling frequency N (i.e., spacing h = 1N ) we can resolve signal frequencies less than N
2 = 12h .
This critical frequency N2 = 1
2h is called “Nyquist frequency”.
If the condition is satisfied we haveF(x) = f (x)
where f ∈VN is the interpolating function for the samples f j.
We have an explicit formula for f in terms of the values f j:
Let N = 2n+1 odd and we are given the samples f j = F( jN ), j = 0, . . . ,N−1. Then
F(x) =n
∑j=−n
Fku(k)(x)
Assume that f0 = 1 and f j = 0 if j is not a multiple of N. The discrete Fourier transform gives
FN [1,0, . . . ,0] = N−1[1, . . . , ,1]
Hence we get with the Dirichlet kernel Dn(x) defined in (37)
f (x) = N−1n
∑k=−n
u(k)(x) = (2n+1)−1Dn(x) =
{1 for x ∈ Zsin((2n+1)πx)(2n+1)sin(πx) otherwise
-2/9
-1/9
0 1/9
2/9
3/9
4/9
5/9
6/9
7/9
8/9
1 10/9
11/9
0
0.5
1
49
Here we see the graph of the function 1N Dn(x) for N = 2n+1 with n = 4 (also see the graphs of Dn(x) for n = 8 on pages 20,
29).
Note that the function N−1Dn ∈Tn satisfies N−1Dn(j
N ) =
{1 if j is multiple of N0 otherwise
Now consider ~f = [0,1,0, . . . ,0] where the values are shifted on notch to the right. Now the shifted function N−1Dn(x− 1N )
is the interpolating function.
Consider ~f = [3,−2,0, . . . ,0]. Then N−1[3Dn(x)−2Dn(x− 1
N )]
is the interpolating function.
Hence we obtain an explicit formula for the interpolating function f :
f (x) = N−1N−1
∑j=0
f j ·Dn(x− jN ) (59)
For even N = 2n the Fourier vector [1, . . . , ,1] corresponds to the function
12 u(−n)+u(−n+1)+ · · ·+u(n−1)+ 1
2 u(n) = 12 q−n +q−n+1 + · · ·+qn−1 + 1
2 qn
= 12(q+1)
qn−q−n
q−1= 1
2(q12 +q−
12 )
qn−q−n
q12 −q−
12= cos(πx) · sin(2nπx)
sin(πx)=: Dn,cos(x) (60)
where q := e2πix. Therefore we obtain
f (x) = N−1N−1
∑j=0
f j ·Dn,cos(x− jN ) (61)
for the interpolating function.
4.11 E�cient computation of FN and F−1N : Fast Fourier Transform
We will omit vector arrows in this section and write f , f for the vectors ~f , ~f ∈ CN .
Assume we are given the vector f =[
f0, . . . , fN−1]> ∈CN in the frequency domain. We want to find the vector f =F−1
N f =[ f0, . . . , fN−1]
> ∈ CN in the time domain. If we just use the formula
f j =N−1
∑k=0
fke2πik j/N (62)
we have to compute N multiplications and N−1 additions for each component, in addition to evaluating e2πik j/N . This meansthat the computation of ~f costs at least N2 multiplications (plus a similar number of additions etc.). For a vector of lengthN = 1000 this means that we have at least 106 operations. This means that it is not practical to compute discrete Fouriertransforms of long vectors.
Fortunately there is a way to reorganize the computation in such a way that only N · log2 N operations are required. E.g., forN = 1024 we have
N2 ≈ 106, N · log2 N = 1024 ·10≈ 104
Note that e2πik j/N = ωjk
N where we define
ωN := e2πi/N
50
Assume that N is even. Then we can split up the sum in even and odd terms: Using ω N2= ω2
N we obtain
f j =N−1
∑k=0
ωjk
N fk =
N2 −1
∑k=0
ωj(2k)
N f2k +
N2 −1
∑k=0
ωj(2k+1)
N f2k+1 =
N2 −1
∑k=0
ωjkN2
f2k︸ ︷︷ ︸a j
+ωj
N
N2 −1
∑k=0
ωjkN2
f2k+1︸ ︷︷ ︸b j
(63)
Recall that (62) implies that f j+N = f j. In the same way we get a j+N2= a j and b j+N
2= b j.
Note that a0a1...
a N2 −1
= F−1N2
f0
f2...
fN−2
,
b0b1...
b N2 −1
= F−1N2
f1
f3...
fN−1
After we computed the vectors
a0
...a N
2 −1
, b0
...b N
2 −1
∈ CN2 we can use (63) to find
[f0
...fN−1
]∈ CN : Using a j+N
2= a j, b j+N
2= b j
and ωN2
N =−1 we obtain
f0...
f N2 −1
f N2...
fN−1
=
a0 +ω0Nb0
...
a N2 −1 +ω
N2 −1
N b N2 −1
a N2+ω
N2
N b N2
...aN−1 +ω
N−1N bN−1
=
a0 +ω0Nb0
...
a N2 −1 +ω
N2 −1
N b N2 −1
a0−ω0Nb0
...
a N2 −1−ω
N2 −1
N b N2 −1
=
[a+ ca− c
]where a =
a0...
a N2 −1
, c :=
ω
0Nb0...
ωN2 −1
N b N2 −1
This gives the following algorithm for computing f = FN f :
• let a := F−1N2
f0
f2...
fN−2
, b := F−1N2
f1
f3...
fN−1
, c j := ωj
Nb j for j = 0, . . . , N2 −1
• let f :=[
a+ ca− c
]With this algorithm we can compute FN if N is a power of 2: We use recursion and compute FN
2, FN
4, . . . using this
algorithm and note that F1 f = f .
Here is a Matlab implementation of this algorithm: We assume that fh is a row vector where the length is a power of 2:
function f = ifour_rec(fh)N = length(fh);if N>1if mod(N,2)==1error(’length must be power of 2’)
enda = ifour_rec(fh(1:2:end));c = ifour_rec(fh(2:2:end)).*exp(2i*pi/N*(0:N/2-1));f = [a+c,a-c];
else % case N=1f = fh;
end
51
Let CN denote the number of operations needed to compute the vector F−1N f . We need to compute 2 transforms of length
N2 to compute the vectors a,b. Then we need about N additional operations to compute f0, . . . , fN−1 from a,b. Therefore wehave for even N that
CN = 2CN/2 +N
We obtain the following costs for N = 1,2,4,8, . . .
C1 = 0
C2 = 0 ·2+2 = 1 ·2C4 = 1 ·2 ·2+4 = 2 ·4C8 = 2 ·4 ·2+8 = 3 ·8
C16 = 3 ·8 ·2+16 = 4 ·16
C32 = 4 ·16 ·2+32 = 5 ·32
yielding a cost CN = N · log2 N if N is a power of 2.
The same idea works if N is a multiple of 3: Then we split the sum (62) into three parts: k = 0,3,6,9, . . . and k = 1,4,7, . . .and k = 2,5,8, . . .
f j =N−1
∑k=0
ωjk
N fk =N/3−1
∑k=0
ωj(3k)
N f3k +N/3−1
∑k=0
ωj(3k+1)
N f3k+1 +N/3−1
∑k=0
ωj(3k+2)
N f3k+2 (64)
=N/3−1
∑k=0
ωjk
N/3 f3k +ωj
N
N/3−1
∑k=0
ωjk
N/3 f3k+1 +ω2 jN
N/3−1
∑k=0
ωjk
N/3 f3k+1
Let D :=
1ωN
. . .ω
N/3−1N
and define
a(0) := F−1N/3
f0
f3...
fN−3
, a(1) := DF−1N/3
f1
f4...
fN−2
, a(2) := D2F−1N/3
f2
f5...
fN−1
Then (64) gives using ω
N/3N = ω3 and ω4
3 = ω3 f0...
fN−1
=
a(0)+a(1)+a(2)
a(0)+ω3a(1)+ω23 a(2)
a(0)+ω23 a(1)+ω3a(2)
If N is a multiple of 3 the cost of FN is
CN = 3CN/3 +2N
Matlab’s fft and ifft are implemented using the FFTW library (“Fastest Fourier Transform in the West”) which uses anumber of similar techniques to allow arbitrary values of N.
• this is most efficient if N is a power of 2, or a product of small primes like 2,3,5,7
• even if N is a large prime the cost is O(N logN)
5 Nonperiodic functions
5.1 Band-limited signals with piecewise continuous f (ξ )
The idea of Fourier analysis is to represent a function f (x) as a superposition of functions e2πiξ x with frequency ξ ∈ R.
52
If we only use integer frequencies ξ ∈ Z we obtain a 1-periodic function f (x):
f (x) =∞
∑k=−∞
fke2πikx
If we use only use frequencies ξk =kL with L > 0, k ∈ Z we obtain a function with period L:
f (x) =∞
∑k=−∞
fke2πik kL
If the sum only contains terms e2πiξ x with frequencies |ξ | ≤M with some M ≥ 0 then we call f a band-limited function.
Now we consider a superposition of functions e2πiξ x with all ξ ∈ [−M,M]:Let f (ξ ) denote a piecewise continuous function with f (ξ ) = 0 for |ξ |> M and define
f (x) :=∫
∞
−∞
f (ξ )e2πiξ xdξ
Note that we integrate a piecewise continuous function from ξ =−M to M, so the integral exists.
Example 1: For f (ξ ) =
{1 for ξ ∈ [−1
2 ,12 ]
0 otherwisewe obtain
f (x) =∫ 1
2
− 12
e2iπxξ dξ =
[1
2iπxe2iπxξ
] 12
ξ=− 12
=1
2iπx
[eiπx− e−iπx]= sin(πx)
πx= sinc(πx) = sincπ(x)
where sincπ(x) corresponds to sinc(x) in Matlab.
We see that in this example f (x) has the following properties:
(1.) f is not periodic
(2.) f (x)→ 0 as x→±∞
(3.) f is “smooth”: all derivatives f ′, f ′′, f ′′′, . . . exist and are continuous functions on R
These properties hold for all piecewise continuous functions f (ξ ) which are zero for |ξ |> M.
(1.) is obvious: in order to get a periodic function with period L, only frequencies ξ = k/L with integer k can be present.
Proof of (3.): For f ′(x) we are allowed to move the derivative inside the integral:
ddx
f (x) =∫ M
−Mf (ξ )
ddx
(e2πiξ x
)dξ =
∫ M
−Mf (ξ ) · (2πiξ x) · e2πiξ xdξ(
ddx
)m
f (x) =∫ M
−Mf (ξ )
(ddx
)m(e2πiξ x
)dξ =
∫ M
−Mf (ξ ) · (2πiξ x)m · e2πiξ xdξ
yielding
g(x) = f ′(x) ⇐⇒ g(ξ ) = 2πiξ f (ξ )
g(x) = f (m)(x) ⇐⇒ g(ξ ) = (2πiξ )m f (ξ )(65)
Example 2: For g(ξ ) =
{ξ for |ξ | ≤ 1
2
0 otherwisewe see that g(ξ ) = ξ f (ξ ) with f (ξ ) from Example 1. Let u(ξ ) := 2πig(ξ ) =
2πiξ f (ξ ). Then (65) gives
u(x) = f ′(x) =ddx
[sin(πx)
πx
]=
cos(πx)ππx
+sin(πx)
π
(−1x2
)
53
and g(x) = 12πi g(x).
Proof of (2.): We assume that f (ξ ) is piecewise continuous and zero outside of [−M,M]. Note that f (ξ + h) has a graphwhich is shifted by h to the left. We then claim
Ah :=∫
∞
−∞
∣∣ f (ξ )− f (ξ +h)∣∣dξ → 0 as h→ 0 (66)
To prove this claim let us first consider the case where f (ξ ) =
0 for ξ < Mcontinuous function for ξ ∈ [−M,M]
0 for ξ > M
. Then Ah is the sum
of the yellow areas in this graph:
-M-h -M M-h M
0
The yellow area consists of three parts:
Ah =∫ −M
−M−h
∣∣ f (ξ )︸︷︷︸0
− f (ξ +h)∣∣dξ +
∫ M−h
−M
∣∣ f (ξ )− f (ξ +h)∣∣dξ +
∫ M
M−h
∣∣ f (ξ )− f (ξ +h)︸ ︷︷ ︸0
∣∣dξ
Since f (ξ ) is continuous on [−M,M] we have that∣∣ f (ξ )− f (ξ +h)
∣∣ in the second integral goes uniformly to zero as h→ 0.The first integral is
∫ −M−M−h
∣∣ f (ξ +h∣∣dξ , the third integral is
∫MM−h
∣∣ f (ξ )∣∣dξ , they go to zero as h→ 0 since we have integralsof a continuous function over an interval of length h with h→ 0.
If the function f (ξ ) is piecewise continuous on [−M,M] with additional jumps, then we get additional “vertical yellowregions” of width h at each jump and the above argument can still be applied.
We use the change of variables ξ = t + 12x
f (x) =∫
∞
ξ=−∞
f (ξ )e2πiξ xdξ =∫
∞
t=−∞
f (t + 12x)e
2πi(t+ 12x )xdt =
∫∞
t=−∞
f (t + 12x)e
2πitx eπi︸︷︷︸−1
dt
yielding
f (x) = 12 f (x)+ 1
2 f (x) =∫
∞
ξ=−∞
12
[f (ξ )− f (ξ + 1
2x)]
e2πiξ xdξ
| f (x)| ≤∫
∞
ξ=−∞
∣∣∣12
[f (ξ )− f (ξ + 1
2x)]
e2πiξ x∣∣∣dξ = 1
2
∫∞
−∞
∣∣ f (ξ )− f (ξ + 12x)∣∣dξ = 1
2 A 12x
Now we use (66) with h = 12x and obtain
f (x)→ 0 as x→±∞ (67)
Actually, one can show that (66) holds for any (classical) function f (ξ ) with∫
∞
−∞
∣∣ f (ξ )∣∣dξ < ∞ . Therefore we obtain (67)
for these functions:
Lemma 12 (Riemann-Lebesgue Lemma). If the function f (ξ ) satisfies∫
∞
−∞
∣∣ f (ξ )∣∣dξ < ∞ we have
f (x)→ 0 as x→±∞
54
Here it is crucial that f is a function and not a generalized function: For f = δa with a ∈ R we have∫∞
−∞
∣∣ f ∣∣dξ =∫
∞
−∞
δadξ = 1 < ∞.
But Ah =∫
∞
−∞
∣∣ f (ξ )− f (ξ +h)∣∣dξ =
∫∞
−∞(δa +δa−h)dξ = 2 which does NOT go to zero as h→ 0. For fa = δa we have
f (x) = e2πiax which is periodic and does NOT go to zero as x→±∞.
5.2 Find g(x) for g(ξ ) = f ′(ξ )
Consider a function f (ξ ) which is zero outside of [−M,M]. Assume that f is piecewise differentiable without jumps, so wehave f (M) = f (−M) = 0.
We use integration by parts
g(x) =∫ M
−Mf ′(ξ ) · e2πiξ xdξ =
[f (ξ ) · e2πiξ x
]M
ξ=−M︸ ︷︷ ︸0
−∫ M
−Mf (ξ ) ·2πix · e2πiξ xdξ︸ ︷︷ ︸
2πix · f (x)
g(x) =−2πix · f (x)
Differentiation in the time domain corresponds to multiplication by 2πiξ in the frequency domain,differentiation in the frequency domain corresponds to multiplication by −2πix in the time domain (note the minus sign!).
time domain frequency domain
f (x) f (ξ )
f ′(x) 2πiξ · f (ξ )
−2πix · f (x) f ′(ξ )
5.3 Approximating F−1 f using the trapezoid rule with values f (kh), h = M/n
For a given function f (ξ ) on [−M,M] we would like to find
f (x) =∫ M
ξ=−Mf (ξ )e2πiξ xdξ (68)
Sometimes we may not be able to find a formula for the antiderivative. Sometimes we may only have values f (kh) for asmall step size h and integer k available. In these cases we can approximate the integral using the trapezoid rule.
We divide the interval [−M,M] into 2n subintervals of size h = M/n. The trapezoid rule approximates the integral over asubinterval [kh,(k+1)h] ∫ (k+1)h
khF(t)dt ≈ h
2[F(kh)+F((k+1)h)]
yielding on the whole interval [−M,M] the approximation∫ M
−MF(t)dt ≈ h
[12 F(−nh)+F((−n+1)h)+ · · ·+F((n−1)h)+ 1
2 F(nh)]
Note that the first and last values are multiplied by 12 .
We apply this to (68) with F(ξ ) = f (ξ )e2πiξ x and obtain with vk := f (kh) and s := hx
f (x)≈ h[
12 v−ne2πi(−n)s + v−n+1e2πi(−n+1)s + · · ·+ vn−1e2πi(n−1)s + 1
2 vne2πins]
55
The Fourier vector ~g = [g0, . . . , gN−1] :=[v0, . . . , vn−1,
12 vn,
12 v−n, v−n+1, . . . , v−1
]of length 2n+1 corresponds to the interpo-
lating functiong(s) = 1
2 g−ne2πi(−n)s + g−n+1e2πi(−n+1)s + · · ·+ gn−1e2πi(n−1)s + 12 gne2πins
where for k < 0 we define gk = gk+N .
Note that the function g(s) is 1-periodic. Since s = hx and x = s/h the function g(s) for s ∈ [−12 ,
12 ] gives with
f (s/h)≈ hg(s)
an approximation for f on[− 1
2h ,12h
].
Therefore we obtain the following recipe for approximating the Fourier integral (68) using the values f (kh) with h=M/n:
• pick n and let h := M/n. Larger n gives smaller error, we will get usable approximations for x ∈ [−14 h−1, 1
4 h−1].
• pick L, let ` := bL/2c. We will get approximations for x values with a spacing of 1Lh .
• let ~g :=[
f (0), f (h), . . . , f ((n−h), 12 f (nh), 1
2 f (−nh), f ((−n+1)h), . . . , f (−h)]
• let ~G := fext(~g,L)
• let ~G := F−1L
~G yielding ~G = [G0, . . . ,GL−1]
• then we obtain for j =−`, . . . , ` the approximation f ( jLh)≈
{hG j for j = 0,1, . . . , `hGL− j for j =−1,−2, . . . ,−`
We get approximations of f (x) for x = jLh with j =−`, . . . , `. The leftmost value is x =− `
Lh ≈−12 h−1, the rightmost value
is x = `L ·h
−1 ≈ 12 h−1.
However, the error increases with |x| and we can only expect to get reasonable approximations for x ∈ [−14 h−1, 1
4 h−1]:consider e2πiξ x for fixed x as a function of ξ : this function has frequency |x| and period 1
|x| . In order to get a reasonable
approximation fot the integral we need 4 subintervals per period, i.e., h≤ 14|x| ⇐⇒ |x| ≤
14h .
5.4 Approximating F−1 f using the midpoint rule with values f (kh), h = M/(n+ 12)
In the previous section we divided the interval [−M,M] into an even number of subintervals and used the endpoints of thesubintervals to approximate the integral.
Now we divide the interval [−M,M] into an odd number of subintervals and use the midpoints of the subintervals toapproximate the integral.
We divide the interval [−M,M] into an 2n+ 1 subintervals of size h = 2M2n+1 = M
n+ 12. The midpoint rule approximates the
integral over a subinterval[(k− 1
2)h,(k+12)h]
∫ (k+ 12 )h
(k− 12 )h
F(t)dt ≈ hF(kh)
56
yielding on the whole interval [−M,M] the approximation∫ M
−MF(t)dt ≈ h [F(−nh)+ · · ·+F(nh)]
We apply this to (68) with F(ξ ) = f (ξ )e2πiξ x and obtain with gk := f (kh) and s := hx
f (x)≈ h[g−ne2πi(−n)s + g−n+1e2πi(−n+1)s + · · ·+ gne2πins
]Recall: the Fourier vector ~g = [g0, . . . , gN−1] of odd length N = 2n+1 corresponds to the interpolating function
g(s) = g−ne2πi(−n)s + g−n+1e2πi(−n+1)s + · · ·+ gne2πins
where for k < 0 we define gk = gk+N .
Note that the function g(s) is 1-periodic. Since s = hx and x = s/h the function g(s) for s ∈ [−12 ,
12 ] gives with
f (s/h)≈ hg(s)
an approximation for f on[− 1
2h ,12h
].
Therefore we obtain the following recipe for approximating the Fourier integral (68) using the values f (kh) with h =M/(n+ 1
2):
• pick n and let h := M/(n+ 12). Larger n gives smaller error, we will get usable approximations for x ∈ [−1
4 h−1, 14 h−1].
• pick L, let ` := bL/2c. We will get approximations for x values with a spacing of 1Lh .
• let ~g :=[
f (0), f (h), . . . , f (nh), f (−nh), . . . , f (−h)]
• let ~G := fext(~g,L)
• let ~G := F−1L
~G yielding ~G = [G0, . . . ,GL−1]
• then we obtain for j =−`, . . . , ` the approximation f ( jLh)≈
{hG j for j = 0,1, . . . , `hGL− j for j =−1,−2, . . . ,−`
As in the previous section the error increases with |x| and we can only expect reasonable approximations for x ∈[−1
4 h−1, 14 h−1].
This recipe looks complicated because of the definition of the discrete Fourier transform: For odd N = 2n+ 1 the discreteFourier transform FN maps
[ f0, . . . , fn, f−n, . . . , f−1] 7→[
f0, . . . , fn, f−n, . . . , f−1]
It is convenient for odd N = 2n+1 to have a mapping sfour
[ f−n, . . . , f−1, f0, . . . , fn] 7→[
f−n, . . . , f−1, f0, . . . , fn]
where the two halves of both vectors are swapped into “correct order”. The inverse mapping is isfour, and we also have acorresponding version of fext: the function sfext(fh,L) with odd L gives the following vector of length L[
0, . . . ,0︸ ︷︷ ︸k zeros
, f0, . . . , fn, f−n, . . . , f−1,0, . . . ,0︸ ︷︷ ︸k zeros
]where k := (L−N)/2.
57
As an example consider f (ξ ) =
{1−|x| for |x| ≤ 10 otherwise
. Here we have f (x) = sincπ(x)2.
fh = @(t) 1-abs(t); % define function fh for |t|<=MM = 1;
X = -4:.001:4; % plot exact f(x) = sinc(x)^2plot(X,sinc(X).^2,’g’); hold on
n = 10; el = 35; % choose n and elN = 2*n+1; L = 2*el+1;h = 2*M/N;gh = fh((-n:n)*h); % use these function values of fhG = h*isfour(sfext(fh,L)); % apply fext and ifourx = (-el:el)/(L*h);
plot(x,G,’k.’);hold off; axis tight; xlim([-4 4]);title(’exact f(x) (green) and midpoint rule approximation (black)’)
-4 -3 -2 -1 0 1 2 3 4
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1exact f(x) (green) and midpoint rule approximation (black)
5.5 Nonperiodic signal for discrete times x j = jh
In the introduction in section 1 we presented four cases of signals in the time domain:
(1.) periodic signal, continuous time x ∈ R
(2.) periodic signal, discrete times x j = jh
(3.) nonperiodic signal, continuous time x ∈ R
(4.) nonperiodic signal, discrete times x j = jh
We have covered cases (1.) and (2.) in sections 3 and 4. Now we want to look at case (4.): We have a nonperiodic signal inthe time domain which goes to zero as x→±∞. We are given signal values f j at times x j = jh with j ∈ Z and a step sizeh > 0.
We want to write this signal as a superposition of e2πiξ x with ξ ∈ [−M,M] for a suitable value M.
We want to find a function f (ξ ) on [−M,M] so that we have
f j =∫ M
−Mf (ξ )e2πiξ jhdξ for all j ∈ Z
It turns out that we have already solved this problem: this exactly corresponds to case (1.) if we switch the time andfrequency domain. For case (1.) with an L-periodic signal we obtained in (44). Let us write down these results using
g j := f j and G(x) := f (x) . We also replace i with −i (all results remain the same, we may just as well use e−2πikx/L in
the formula for f (x) and e2πikx/L in the formula for fk, and some books use this setting):
g j = L−1∫ L
2
− L2
G(x)e2πik jx/Ldx ⇔ G(x) =∞
∑j=−∞
g je−2πi jx/L Inversion
L−1∫ L
2
− L2
|G(x)|2 dx =∞
∑j=−∞
∣∣g j∣∣2 Parseval
58
If we now write h := L−1 we obtain with f j := g j/h and f (ξ ) := G(ξ )
f j =∫ 1
2h
− 12h
f (ξ )e2πiξ j/hdξ ⇔ f (ξ ) = h∞
∑j=−∞
f je−2πiξ jh Inversion
∫ 12h
− 12h
∣∣ f (ξ )∣∣2 dx = h∞
∑j=−∞
∣∣ f j∣∣2 Parseval
We see: data f j at x j = jh in the time domain corresponds to a function f (ξ ) on [−M,M] with M =12h
in the frequency
domain.
For given values f j with finite “energy” E = h∑∞j=−∞
∣∣ f j∣∣2 < ∞ we can find f (ξ ) using
f (ξ ) = h∞
∑j=−∞
f je−2πiξ jh (69)
where the sum converges in the mean square sense: for f(N)(ξ ) := h∑Nj=−N f je−2πiξ jh we have
∥∥ f(N)− f∥∥→ 0 as N→ ∞.
Once we have found the function f (ξ ) we can evaluate∫ 1
2h
− 12h
f (ξ )e2πiξ xdξ for any x ∈ R:
The function
f (x) :=∫ 1
2h
− 12h
f (ξ )e2πiξ xdξ (70)
is an interpolating function for the given data points, i.e., f ( jh) = f j. We see that f (x) is band-limited.
Note that MNy := 12h is the Nyquist frequency for data with spacing h.
We obtain: For given data f j with spacing h and ∑∞j=−∞
∣∣ f j∣∣2 < ∞ there exists a unique interpolating function f ∈
L2([−MNy,MNy]).
Theorem 13 (Sampling Theorem). Consider a band-limited signal F with finite energy:
F(x) =∫ M
−MF(ξ )e2πiξ xdξ ,
∫ M
−M
∣∣F(ξ )∣∣2 dξ < ∞
We sample this signal with spacing h≤ 12M and get f j := F( jh) for j ∈ Z.
Then we can reconstruct the signal F from the samples f j: define f (ξ ) by (69) and f (x) by (70), then F(x) = f (x).
Proof. If h≤ 12M we have M ≤ 1
2h . Define F(ξ ) = 0 for |ξ |> M, then we have
f j = f ( jh) =∫ 1
2h
− 12h
F(ξ )e2πiξ j/hdξ
by its definition. With f (ξ ) from (69) we also have
f j =∫ 1
2h
− 12h
f (ξ )e2πiξ j/hdξ
Therefore we must have F = f (in the sense of L2 functions, i.e., “almost everywhere”).
59
Example 1: Consider the signal f j = 0 for all j ∈ Z. Find interpolating functions f which contain only frequencies in[− 1
2h ,12h
].
Obviously f (x) = 0 is an answer. Is there any other answer? The sampling theorem states that this is the only interpolatingfunction with finite energy.
But note that f (x) = sin(πx/h) also satisfies f ( jh) = sin(π j) = 0, and
f (x) = sin(πx/h) =12i
[eiπx/h− e−iπx/h
]=∫
∞
−∞
f (ξ )e2πiξ xdξ with f =12i
[δ 1
2h−δ− 1
2h
],
so this function f contains only the frequencies −12h and 1
2h . Does this contradict the sampling theorem? No: this function is
periodic and has infinite energy. For a function f containing Dirac deltas the expression∫
∞
−∞
∣∣ f (ξ )∣∣2 dξ does not make sense.
Example 2: Consider the signal . . . ,0,0,0,1,0,0,0 . . . given by
f j =
{1 for j = 00 for j 6= 0
Find the Fourier transform f (ξ ) and the interpolating function f (x).
Answer:f (ξ ) = h
∞
∑j=−∞
f je−2πiξ jh = h ·1 ξ ∈[− 1
2h,
12h
]Therefore the interpolating function f (x) is given by
f (x) =∫ 1/(2h)
−1/(2h)f (ξ )e2πiξ xdξ =
∫ 1/(2h)
−1/(2h)h · e2πiξ xdξ = h
[1
2πixe2πixξ
]1/(2h)
ξ=−1/(2h)=
hπx· 1
2i
(eπix/h− e−πix/h
)=
hπx
sin(
πxh
)= sincπ
( xh
)with the sincπ function defined by
sincπ(x) := sinc(πx) =
sin(πx)
πxx 6= 0
1 x = 0
Remember: In Matlab the function sincπ(x) is called sinc(x).
Example 3: Consider the signal f j given
f j =
3 for j = 0−2 for j = 10 otherwise
Find the interpolating function f (x).
Answer: For data
{3 for j = 00 otherwise
we see from the previous example that the answer is 3sincπ(x/h).
For data
{1 for j = 10 otherwise
we see from the previous example that the answer is sincπ(x−h
h ).
Therefore we obtain f (x) = 3sincπ(x/h)−2sincπ(x−h
h ).
RESULT: For given data f j we obtain the following formula for the interpolating function f
f (x) =∞
∑j=−∞
f j sincπ
( xh− j)
(71)
This is an explicit formula how to recover a band-limited function f from its samples f j = f ( jh) if h≤ 12M .
We can use this formula to upsample a signal.
60
5.6 Convolution formula
For discrete signals f j and g j given at times x j = jh we define the discrete convolution
q j = h∞
∑`=−∞
f` ·g j−`
We include the factor h since this should approximate∫
∞
t=−∞f (t)g(x− t)dt for the continuous time case.
Proof of the convolution formula: we plug in the Fourier representation of g j−` and then interchange the order of theintegral and the sum:
q j = h∞
∑`=−∞
f` ·g j−` = h∞
∑`=−∞
fl ·∫ 1/(2h)
−1/(2h)g(ξ )e2πiξ ( j−`)hdξ
=∫ 1/(2h)
−1/(2h)
(h
∞
∑`=−∞
fle−2πiξ `h)
︸ ︷︷ ︸f (ξ )
g(ξ )dξ
So we obtain q j =∫ 1/(2h)
−1/(2h)q(ξ )e2πiξ jhdξ with q(ξ ) = f (ξ )g(ξ ).
Summary for nonperiodic signal, discrete time:
f (ξ ) = h∞
∑j=−∞
f je−2πiξ jh ⇔ f j =∫ 1
2h
− 12h
f (ξ )e2πiξ jhdξ Inversion
h∞
∑j=−∞
∣∣ f j∣∣2 =
∫ 12h
− 12h
∣∣ f (ξ )∣∣2 dξ Parseval
q j = h∞
∑`=−∞
f`g j−` ⇔ q(ξ ) = f (ξ )g(ξ ) Convolution
(72)
5.7 Nonperiodic signal, continuous time: the Fourier transform f = F f
We finally consider the case of a nonperiodic signal f (x) for continuous time x ∈ R. We consider a function f (x) withf (x)→ 0 as x→±∞.
We assume that we have finite “energy” E =∫
∞
−∞| f (x)|2 dx <∞. We want to write f (x) as a superposition of functions e2πiξ x:
f (x) =∫
∞
−∞
f (ξ )e2πiξ xdξ
Recall the periodic case: For a 1-periodic function f (x) and N ∈ N we let f j := f ( j/N) for j = 0, . . . ,N− 1. Then theformulas (52) for the discrete case contain sums N−1
∑N−1j=0 G( j/N) which for N→ ∞ become integrals
∫ 10 G(x)dx in the
formulas (43) for the continuous case.
Now we look at the nonperiodic case: For a function f (x) on R and h > 0 we let f j := f ( jh) for j ∈ Z.Then the formulas(72) for the discrete case contain sums h∑
∞j=−∞ G( jh) which for h→ 0 formally become integrals
∫∞
−∞G(x)dx.
Therefore we would expect the following results for nonperiodic signals and continuous time:
f (ξ ) =∫
∞
−∞
f (x)e−2πiξ xdx ⇔ f (x) =∫
∞
−∞
f (ξ )e2πiξ xdξ Inversion∫∞
−∞
| f (x)|2 dx =∫
∞
−∞
∣∣ f (ξ )∣∣2 dξ Parseval-Plancherel
q(x) =∫
∞
−∞
f (t)g(x− t)dt ⇔ q(ξ ) = f (ξ )g(ξ ) Convolution
(73)
61
Taking this formal limit h→ 0 in (72) is NOT a proof of these results. I will skip the proofs since they require technicalresults about interchanging limits and integrals, changing the order of double integrals and other things. Presenting sketchesof the proofs without these technical details is misleading since it hides the essential difficulties. For now it is more importantto understand the meaning and the applications of (73).
We call the mapping F : f 7→ f Fourier transform, and the inverse mapping F−1 : f 7→ f inverse Fourier transform.
Note that F is the same as F−1, but with i replaced by −i. Taking the complex conjugate in f (ξ ) =∫
∞
−∞f (x)e−2πiξ xdx
givesf = F f ⇐⇒ f = F−1 f , i.e.,F−1 f = F f , F f = F f (74)
5.8 Convergence results for the Fourier integral∫
∞
−∞f (ξ )e2πiξ xdξ
For the Fourier series the statement f (x) = ∑∞k=−∞ fke2πikx has to be interpreted as a limit of ∑
Nk=−N fke2πikx for N → ∞.
Depending on the properties of f we have mean square convergence, pointwise convergence, or uniform convergence.
For the Fourier transform the statement f (x) =∫
∞
−∞f (ξ )e2πiξ xdξ has to be interpreted as a limit of
∫M−M f (ξ )e2πiξ xdξ for
M→ ∞. Depending on the properties of f we have mean square convergence, pointwise convergence, or uniform conver-gence.
Review of Fourier series
For a Fourier series ∑∞k=−∞ fke2πikx we have with f(N)(x) := ∑
Nk=−N fke2πikx the following results:
(a) if∫ 1
0 | f (x)|2 dx < ∞ we have mean square convergence
∥∥ f(N)− f∥∥
L2([0,1])→ 0 as N→ ∞
(b) if ∑∞k=−∞
∣∣ fk∣∣ < ∞ (absolute convergence) we have uniform convergence maxx∈[0,1]
∣∣ f(N)(x)− f (x)∣∣→ 0 as N → ∞,
and f (x) is continuous.
(c) if f ∈ PW 1 (piecewise function with finitely many pieces, for each piece f and f ′ are continuous):we have pointwise convergence everywhere
for all x ∈ [0,1]: f(N)(x)→ 12 [ f (x−0)+ f (x+0)] as N→ ∞
Corresponding results for Fourier transform
For the Fourier integral∫
∞
−∞f (ξ )e2πiξ xdξ we have with f(M)(x) :=
∫M−M f (ξ )e2πiξ xdξ the following results:
(A) if∫
∞
−∞| f (x)|2 dx < ∞ we have mean square convergence:
∥∥ f(M)− f∥∥→ 0 as M→ ∞
(B) if∫
∞
−∞
∣∣ f (ξ )∣∣dξ <∞ we have uniform convergence maxx∈R∣∣ f(M)(x)− f (x)
∣∣→ 0 as M→∞. Also: f (x) is continuousand f (x)→ 0 as x→±∞.
(C) if∫
∞
−∞| f (x)|dx < ∞ and f ∈ PW 1 (piecewise function, on every finite subinterval there are finitely many pieces, for
each piece f and f ′ are continuous):we have pointwise convergence everywhere
for all x ∈ R: f(M)(x)→ 12 [ f (x−0)+ f (x+0)] as M→ ∞
Let f(M) :=∫M−M f (x)e−2πiξ xdx. Note that the Fourier transform F is the same as the inverse Fourier transform F−1, except
for replacing i with −i. Therefore we obtain corresponding results for the convergence f(M)→ f .
62
5.9 Examples for Fourier transforms, approximation with FFT
We will write g ∈ L2 if∫
∞
−∞|g(x)|2 dx < ∞, and g ∈ L1 if
∫∞
−∞|g(x)|dx < ∞.
We illustrate the results from the previous section for some examples:
Example 1: Let f (x) :=
{1 for |x| ≤ 1
2
0 otherwise. We obtain f (ξ ) = sincπ(ξ ).
We have∫
∞
−∞| f (x)|2 dx = 1 < ∞, so we must have
∫∞
−∞
∣∣ f (ξ )∣∣2 dξ = 1 < ∞. Therefore (A) gives f(M)→ f and f(M)→ f inthe mean square sense.
We have∫
∞
−∞| f (x)|dx = 1 < ∞, so (B) gives that we have uniform convergence f(M)→ f . Also f is continuous and tends to
zero as x→±∞ . This is obviously true for f (ξ ) = sincπ(ξ ).
For f (ξ ) = sincπ(ξ ) we have∣∣ f (ξ )∣∣= O(|ξ |−1) . We obtain
∫∞
−∞
∣∣ f (ξ )∣∣dξ = ∞, but∫
∞
−∞
∣∣ f (ξ )∣∣2 dξ < ∞.
Since f is not continuous (B) implies f /∈ L1. We do not have uniform convergence f(M)→ f .
Since f ∈ L1 and f ∈ PW 1 we get from (C)
limM→∞
∫ M
−Mf (ξ )e2πiξ xdξ =
1 for |x|< 1
212 for |x|= 1
2
0 for |x|> 12
Example 2a: f (x) =
{e−x for x≥ 00 for x < 0
gives f (ξ ) =1
1+2πiξ: using the definition of F we get
f (ξ ) =∫
∞
0e−xe−2πiξ xdx =
∫∞
0e(−1−2πiξ )xdx =
1−1−2πiξ
[e(−1−2πiξ )x
]∞
x=0︸ ︷︷ ︸−1
We now want to approximate F−1 f using the midpoint rule: As f is not continuous we have f /∈ L1. Hence we cannotexpect fast convergence as M→ ∞ and we are curious what happens at the jump at x = 0.
Here we choose M = 10 and n = 100, i.e., we are replacing∫
∞
−∞(· · ·)dξ by using N = 2n+1 = 201 points on [−10,10]. The
first graph shows that the truncation to [−M,M] causes a Gibbs phenomenon at the jump at x = 0.
We know from the Fourier series that we can improve the Gibbs phenomenon by averaging over the period of the oscillations,this means to multiply gk by Qk = sincπ(k/(n+ 1
2)). In the second graph we see that the Gibbs phenomenon is much reduced.
63
tl = tiledlayout(2,1) % Requires R2019b or latertitle(tl,’exact f (red) and midpoint rule approximation (black)’)
M = 10;fh = @(t) 1./(1+2i*pi*t); % define function fh for |t|<=MX = -1:.001:1; %fex = exp(-X).*(X>=0); % exact values of f(x)n = 100; l = 600; % choose n and lN = 2*n+1; L = 2*l+1;h = 2*M/N;gh = fh((-n:n)*h); % use these function values of fh
nexttile % 1st graphG = h*isfour(sfext(gh,L)); % apply fext and ifourx = (-l:l)/(L*h);plot(X,fex,’r’,x,G,’k’);xlim([-1 1]); title(’without smoothing’)
nexttile % 2nd graphQ = sinc((-n:n)/(n+.5)) % for averagingGa = h*isfour(sfext(gh.*Q,L)); % apply averaging, fext and ifourplot(X,fex,’r’,x,Ga,’k’);xlim([-1 1]); title(’with smoothing’)
exact f (red) and midpoint rule approximation (black)
-1 -0.5 0 0.5 1
0
0.2
0.4
0.6
0.8
1
without smoothing
-1 -0.5 0 0.5 1
0
0.2
0.4
0.6
0.8
1with smoothing
Example 2b: Now let f (ξ ) =
{e−ξ for ξ ≥ 00 for ξ < 0
and find f (x). Method 1: using the definition of F−1
f (x) =∫
∞
0e−ξ e2πiξ xdξ =
∫∞
0e(−1+2πix)xξ dx =
1−1 = 2πix
[e(−1+2πix)ξ
]∞
ξ=0︸ ︷︷ ︸−1
we obtain f (x) =1
1−2πix.
Method 2: using (74) we get f (x) = f (x) and f (x) =( 1
1+2πix
)= 1
1−2πix .
We now want to approximate F f numerically using the midpoint rule. Again there are two ways.
Method 1: Use (74): Take the complex conjugate of f , then use the code of Example 2a using isfour, then take thecomplex conjugate of the result.
64
Method 2: Recall that FN is the same operation as F−1N , but we have to use −i instead of i and multiply by N. Therefore
we can use the previous code and replace isfour(v) with sfour(v)*length(v)
tl = tiledlayout(2,1) % Requires R2019b or latertitle(tl,’exact fh (red) and midpoint rule approximation (black)’)
M = 10;f = @(x) 1./(1-2i*pi*x); % define function fh for |t|<=MXi = -1:.001:1; %fhex = exp(-Xi).*(Xi>=0); % exact values of fh(xi)n = 100; l = 600; % choose n and lN = 2*n+1; L = 2*l+1;h = 2*M/N;g = f((-n:n)*h); % use these function values of fh
nexttile % 1st graphGh = h*L*sfour(sfext(g,L)); % apply fext and ifourxi = (-l:l)/(L*h);plot(Xi,fhex,’r’,xi,Gh,’k’);xlim([-1 1]); title(’without smoothing’)
nexttile % 2nd graphQ = sinc((-n:n)/(n+.5)) % for averagingGa = h*L*sfour(sfext(g.*Q,L)); % apply averaging, fext and ifourplot(Xi,fhex,’r’,xi,Ga,’k’);xlim([-1 1]); title(’with smoothing’)
exact fh (red) and midpoint rule approximation (black)
-1 -0.5 0 0.5 1
0
0.5
1
without smoothing
-1 -0.5 0 0.5 10
0.5
1with smoothing
Note that we now used Gh = h*L*sfour(sfext(g,L)); with the additional factor L.
Approximating F−1 f using sample values and midpoint rule
• given values Fh=[
f (−nh), . . . , f (nh)]
(vector of length N = 2n+1)
• approximation for f = F−1 f given by x = (n:n)/(N*h) and F = h*isfour(Fh)
• upsampled version with l > n and L := 2l +1: x = (l:l)/(L*h) and F = h*isfour(sfext(Fh,L))
• we only use f in [−nh,nh]. If f (−nh−0) or f (nh+0) are nonzero we will get Gibbs phenomenon for F.averaging filter improves this: use sinc((-n:n)/(n+.5)).*Fh in place of Fh
Approximating F f using sample values and midpoint rule
• given values F= [ f (−nh), . . . , f (nh)] (vector of length N = 2n+1)
• approximation for f = F f given by x = (n:n)/(N*h) and Fh = h*N*sfour(F)
• upsampled version with l > n and L := 2l +1: x = (l:l)/(L*h) and Fh = h*L*sfour(sfext(F,L))
• we only use f in [−nh,nh]. If f (−nh−0) or f (nh+0) are nonzero we will get Gibbs phenomenon for Fh.averaging filter improves this: use sinc((-n:n)/(n+.5)).*F in place of F
Example 3: f (x) = e−πx2gives f (ξ ) = e−πξ 2
f (x) = e−π(ax)2gives f (ξ ) = a−1e−π(ξ/a)2
Proof. We complete the square: x2 +2ixξ = (x+ iξ )2 +ξ2
f (ξ ) =∫
∞
−∞
e−2πiξ xe−πx2dx =
∫∞
−∞
e−π(2iξ x+x2)dx = e−πξ 2∫
∞
−∞
e−π(x+iξ )2dx︸ ︷︷ ︸
=: F(ξ )
65
We want to show that F(ξ ) = 1. We can differentiate F(ξ ) by taking the derivative inside the integral:
F ′(ξ ) =∫
∞
−∞
ddξ
[e−π(x+iξ )2
]dx =
∫∞
−∞
e−π(x+iξ )2(−π)2(x+ iξ )idx
= i∫
∞
−∞
ddx
[e−π(x+iξ )2
]dx = i
[e−π(x+iξ )2
]∞
x=−∞
= 0
Since we can also obtain F ′′(ξ ),F ′′′(ξ ), . . . by differentiating inside the integral, the function F(ξ ) is smooth. Since F ′(ξ ) =
0 the function F(ξ ) is constant: F(ξ ) = F(0) =∫
∞
−∞
e−πx2dx. Using polar coordinates x = r cosϕ and y = r sinϕ with
dydx = r dr dϕ we obtain
F(0)2 =
(∫∞
x=−∞
e−πx2dx)(∫
∞
y=−∞
e−πy2dy)=∫
∞
x=−∞
∫∞
y=−∞
e−π(x2+y2)dydx
=∫ 2π
ϕ=0
∫∞
r=0e−πr2
r dr dϕ = 2π ·[−12π
e−πr2]∞
r=0= 1
so F(0) = 1.
5.10 Properties of the Fourier transform
We already discussed differentiation in the time and frequency domain. For f (xλ ), f (−x), f (x− a) we use a change ofvariable in f (ξ ) =
∫∞
−∞f (x)e−2πiξ xdx to obtain the results below.
Note that all properties also work in the other direction, with i replaced by −i. E.g., for g(ξ ) = f (ξ − a) we get g(x) =e2πiax f (x). For g(ξ ) = f (ξ/λ ) with λ > 0 we get g(x) = λ f (λx).
time domain frequency domain
f (x) f (ξ )
f ′(x) 2πiξ · f (ξ )
f (k)(x) (2πiξ )k · f (ξ )
−2πix · f (x) f ′(ξ )
(−2πix)k f (x) f (k)(ξ )
f (x−a) e−2πiaξ · f (ξ )
f (x/λ ) with λ > 0 λ f (λξ )
f (−x) f (−ξ )
f (x) f (−ξ )
f ∗g f (ξ )g(ξ )
5.11 Gibbs phenomenon for truncated Fourier integral
Let br(x) :=
{1 for |x| ≤ r0 otherwise
. Then br(x) = b 12( x
2r ) and
b 12(ξ ) = sincπ(ξ ), br(ξ ) = 2r sincπ(2rξ )
We get the same for the inverse Fourier transform: for g(ξ ) = br(ξ ) we get g(x) = 2r sincπ(2rx).
Since sincπ(x) =sin(πx)
πx we have|sincπ(x)| ≤ (πx)−1, i.e., slowly decaying oscillations as x→±∞. Since sin(πx) has period2 we consider the averaged function
A := sincπ ∗12 b1, A(x) =
12
∫ x+1
x−1sincπ(t)dt =
12[Siπ (x+1)−Siπ (x−1)]
66
where Siπ(x) :=∫ x
0 sincπ(t)dt = π−1 Si(πx). Here is the graph of sincπ(x) (blue) and A(x) (orange):
-4 -2 2 4
-0.2
0.2
0.4
0.6
0.8
1.0
We have the following decay for sincπ(x) and A(x):
|sincπ(x)| ≤ (πx)−1 for all x
|A(x)| ≤ 1.17(πx)−2 for |x| ≥ 1.4
with c = 1.17 for |x| ≥ 1.4.Here are the corresponding antiderivatives
∫ x0 sincπ(t)dt = Siπ(x) (blue)
and A1(x) :=∫ x
0 A(t)dt = 12 [(x+1)Siπ (x+1)− (x−1)Siπ (x−1)] (orange):
-4 -2 2 4
-0.6
-0.4
-0.2
0.2
0.4
0.6
We have the following decay∣∣Siπ(x)− 12
∣∣≤ π−1(πx)−1 for x > 0∣∣A1(x)− 1
2
∣∣≤ 1.05π−1(πx)−2 for x≥ 1.11
67
Assume we are given the values of f (x) for x ∈ [−M,M], and we want to approximate f (ξ ). The obvious approximation is
f(M)(ξ ) =∫ M
−Me−2πixξ f (x)dx, f(M) = F ( f ·bM) = f ∗ bM
For f = δa we obtain therefore f(M)(ξ ) = bM(ξ ) = 2M sincπ(2Mξ ).
If f (ξ ) has a jump of size J at ξ0 we obtain a term J Siπ (2M(ξ −ξ0)).
In both cases we have decaying oscillations with period M−1. We can filter out these oscillations by convolving with M ·b 12M
:
f av(M) := f(M) ∗
M2
b 1M, f av
(M)(ξ ) = M∫
ξ+ 12M
ξ− 12M
f(M)(ξ )dξ
We have
f av(M) = f(M) ∗
M2
b 1M= f ∗ bM ∗Mb 1
2M= F
[f ·bM · sincπ(
xM)]
f av(M)(ξ ) =
∫ M
−Mf (x)sincπ(
xM )e−2πiξ xdx
Therefore using f (x)sincπ(xM ) instead of f (x) means that for Dirac deltas and jumps in f we obtain the orange curves from
the above graphs instead of the blue curves.
5.12 Finding frequencies in signals using f
Continuous time x ∈ R
An L-periodic signal f (x) has a Fourier series
f (x) =∞
∑k=−∞
fke2πi kL x
Therefore we can formally write f =∫
∞
−∞f (ξ )e2πiξ x using a sum of Dirac deltas:
f =∞
∑k=−∞
fkδ kL
Now consider a signal f = u+v+w which is the sum of an L1-periodic signal u, an L2-periodic signal v and a “finite energy”signal w with
∫∞
−∞|w(x)|2 dx < ∞. For given values f (x), how can we compute the frequencies ξ1 =
1L1
and ξ2 =1L2
?
The idea is to computef = u+ v+ w,
then the Dirac deltas contained in f and g should show up as sharp peaks whereas v is a classical function with∫∞
−∞|w(ξ )|2 dξ < ∞.
Discrete time x j = jh
In practice we have two restrictions:
1. we only have samples Fj = f ( jh) at points with a stepsize h
2. we only have finitely many samples Fj for j = J, . . . ,J2 instead of all j ∈ Z
Let us assume that we have N = 2n+1 samples Fj for j =−n, . . . ,n.
Let us pretend that f (x) and f (ξ ) are nice classical functions.
Then we can approximate f (ξ ) using the midpoint rule using the recipe on page 65:
68
• find the discrete Fourier transform of [F−n, . . . ,Fn], yielding[F−n, . . . , Fn
]• then hNFk approximates f (ξk) for ξk =
kNh
We can obtain values with a finer spacing for ξ using “upsampling”: Let L = 2`+1 > N
• find the discrete Fourier transform of [0, . . . ,0,F−n, . . . ,Fn,0, . . . ,0], yielding[F−`, . . . , F
]• then hLFk approximates f (ξk) for ξk =
kLh
Recall
• the error increases with |ξ |. We can not expect usable approximations for |ξk|> 14h .
• if we have a slow decay for Fk as k→ ±∞ we will observe a Gibbs phenomenon for the approximations. We can
improve this by applying an averaging filter: use sincπ
(j
n+ 12
)Fj in place of Fj.
Finite energy signal w ∈ L2
If w ∈ L1 and w ∈ PW 1 we can expect that the approximations hLWk converge to finite values w(ξ ).
If there is slow decay for w(x) as x→±∞ or if w has jumps we will see a Gibbs phenomenon, so we should use the averagingfilter.
Periodic signals u,v
First consider u = e2πiξ1x so that u = δξ1 . What do we obtain for the midpoint rule approximation?
We obtain
u(M) = F (u ·gM) = u∗ gM, gM = 2M sincπ(2Mξ )
yieldingu(M) = 2M sincπ(2M(ξ −ξ1))
Let us now use the averaging filter:
q(x) = sincπ(x/M) gives q = Mg 12M
F (u ·q ·gM) = u∗ q∗ gM, gM = 2M sincπ(2Mξ )
We can express q∗ gM using the sine integral function. Its properties are:
• decay like O(r−2) away for the interval [− 12M , 1
2M ]
5.13 Sampling and sampling theorem
This is completely analogous to section 4.9. There we needed ∑∞k=−∞
∣∣ fk∣∣< ∞, here we will need
∫∞
∞
∣∣ f (ξ )∣∣dξ < ∞.
Assume that we have a function f (x) where the Fourier transform f (ξ ) satisfies f ∈ L1,i.e.,∫
∞
∞
∣∣ f (ξ )∣∣dξ < ∞.
We use the samples Fj := f ( jh) with a spacing h > 0 and j ∈ Z. The samples (Fj) j∈Z correspond to a function F(ξ ) forξ ∈
[− 1
2h ,12h
](see section 5.5).
In the time domain sampling takes us from the function f to the samples (Fj) j∈Z which is in general a lossy operation.
In the frequency domain this takes us from the function f (ξ ) on R to a function F(ξ ) on[− 1
2h ,1
2h
]. We want to find out:
what is the operation f 7→ F in the frequency domain?
69
Sampling with a spacing of h causes ALIASING: the function u(ξ )(x) := e2πiξ x has at the points x = jh the same values asu(ξ+h−1):
u(ξ+h−1)( jh) = e2πi(ξ+h−1) jh = e2πiξ jh︸ ︷︷ ︸u(ξ )( jh)
e2πi j︸︷︷︸1
Therefore we have for all `, j ∈ Zu(ξ+`h−1)( jh) = u(ξ )( jh)
i.e., if we only have samples at jh, j ∈ Z we cannot distinguish the frequencies . . . ,ξ −h−1,ξ ,ξ +h−1,ξ +2h−1, . . ..
We split R into intervals of length h−1
R= · · ·∪[−3
2 h−1,−12 h−1]∪ [−1
2 h−1, 12 h−1]∪ [1
2 h−1, 32 h−1]∪ [3
2 h−1, 52 h−1]∪·· ·
and obtain for the Fourier integral (since f ∈ L1 we may interchange sum and integration)
Fj = f ( jh) =∫
∞
ξ=−∞
f (ξ )e2πiξ jhdξ =`=∞
∑`=−∞
∫ 12 h−1
ξ=− 12 h−1
f (ξ + `h−1)e2πi(ξ+`h−1) jh︸ ︷︷ ︸e2πiξ jh
dξ
=∫ 1
2 h−1
− 12 h−1
∞
∑`=−∞
f (ξ + `h−1)︸ ︷︷ ︸F(ξ )
e2πiξ jhdξ
Result: In the frequency domain we obtain F by adding all the shifted functions f (ξ + `h−1) with ` ∈ Z:
F(ξ ) =∞
∑`=−∞
f (ξ +`
h)
This takes us from the nonperiodic function f to the h−1-periodic function F . This operation is called “Periodization” andwe write
F = P 1h
f
Note that we only need the function F(ξ ) for ξ ∈[− 1
2h ,12h
].
For a general function f the function f (ξ ) is nonzero everywhere on R, and then periodization is definitely a lossy operationsince we add all the functions f (ξ + `
h) together for ` ∈ Z.
However, if the function f is band-limited so that f (ξ ) = 0 for ξ /∈ [− 12h ,
11h ] we obtain that periodization does NOT change
the function on this interval: We obtain ∑∞`=−∞ f (ξ + `
h) = f (ξ ) for ξ ∈ [− 12h ,
11h ]. Therefore sampling in this case is NOT a
lossy operation, and we can reconstruct f from the samples Fj, see Theorem 13.
If we know that the function f is band-limited (classical) function only contains frequencies in [−M,M] we should choose hso that 1
2h ≥M, i.e., h≤ 12M . In this case we don’t lose any information when sampling.
This is also relevant if we want to detect frequencies contained in a signal f : If we want to be able to find frequencies in[−M,M] we should use h with h≤ 1
2M .
On the other hand: If the signal contains a frequency ξ0 /∈ [− 12h ,
12h ], then aliasing means that this frequency will show up in
F(ξ ) at a different position ξ0 = ξ0 +`h ∈
[− 1
2h ,12h
].
Example: Let f (x) = e2πi10x 21+π2x2 , then f (ξ ) = e−|ξ−10|. We see that this contains mainly frequencies around 10, say in
[5,15]. If we want to capture all these frequencies we should use h with h≤ 12M with M = 15, i.e., h≤ 1
30 .
If on the other hand we use h = 115 we obtain F(ξ ) with ξ ∈ [− 1
2h ,12h ] = [−7.5,7.5]. Therefore the peak frequency ξ0 = 10
will be shifted to ξ0 = ξ0−h−1 = 10−15 =−5. Sampling with h = 115 seems to suggest that the signal contains frequencies
70
around ξ0 =−5 whereas in reality it contains frequencies around ξ0 = 10.
f = @(x) exp(2i*pi*10*x).*2./(1+(pi*x).^2);% signal contains mainly frequencies xi in [8,12]tl = tiledlayout(2,1);for K=[30,15]h = 1/K;n = 10*K; N = 2*n+1;x = (-n:n)*h; % sample at -10, -10+h, ..., 10F = f(x); % samplesFh = sfour(F); % discrete Fourier transformxi = (-n:n)/(N*h);nexttile; plot(xi,N*h*real(Fh))xlabel(’xi’)title(sprintf(’sampling with h=1/%g’,K))xlim([-15,15])
end
-15 -10 -5 0 5 10 15
0
0.5
1
1.5
2sampling with h=1/30
-15 -10 -5 0 5 10 15
0
0.5
1
1.5
2sampling with h=1/15
5.14 Convolution and cross correlation
Convolution of two signals
For functions f ,g on R we defined the convolution q = f ∗g byq(x) =∫
∞
t=−∞f (t)g(x− t)dt. We have
q = f ∗g ⇐⇒ q = f · g
Discrete signals of �nite length
In practice we have samples Fj and G j for values x j = jh and we define Q j = h∑∞`=−∞ F G j−`
Also, all values outside of a finite range are zero: Fj = 0 if j /∈ {a,a+1, . . . ,a+L−1}, G j = 0 if j /∈ {b,b+1, . . . ,b+M−1}.
Then we obtain Q j = 0 if j /∈ {c,c+1, . . . ,c+N−1} with N = L+M−1.
In Matlab we get with the vectors F,G of length L,M respectively the vector Q of length N = L+M−1 using Q=conv(F,G) .
E�cient computation using FFT
For discrete signal vectors F,G of length L,M we need on the order of LM operations to compute the convolution in theobvious way.
Note that the setting of the discrete Fourier transforms uses periodic signals, and the convolution was defined in this “wraparound” sense.
The idea is to use vectors of length N ≥ L+M−1 padded with zeros at the end. We can e.g. choose N as a power of 2 tomake the FFT as efficient as possible. Then we can use
f = zeros(1,N); f(1:L) = F;g = zeros(1,N); g(1:L) = G;q = four(ifour(f).*ifour(g)); Q = q(1:L+M-1)
to compute Q = conv(F,G).
71
Cross correlation of two signals
Note about terminology: “correlation” is used both in signal processing and statistics to describe related but different con-cepts.In signal processing: We have two signals f ,g depending on time, and the cross correlation q = f ?g (note that ? is differentfrom ∗ used for convolution) is another signal depending on time. Also note that there are different definitions used in theliterature and numerical software: where to put the conjugate, and whether to use τ or −τ for the time shift.In statistics: We have two scalar random variables X ,Y . Then the covariance Cov(X ,Y ) and the correlationCov(X ,Y )/(Var(X)Var(Y ))1/2 are numbers. For observations we have data vectors [x1, . . . ,xN ] and [y1, . . . ,yN ], and wecan compute approximations for the covariance and correlation (which are two numbers) from the data.
Assume we have two signals f ,g on R. For example f (t) my be the water level in a stream, and g(t) the rainfall at a differentlocation upstream. We expect that f (t) behaves somewhat like g(t + τ) with a time shift τ , but we don’t know the time shiftτ is.
In general we have complex signals f ,g. In order to find τ we compute the cross correlation q = f ?g defined by
q(x) :=∫
∞
−∞
f (t)g(t + x)dt =∫
∞
−∞
f (−s)g(x− s)ds
If f (t)≈ g(t + τ0) we expect that the function q(x) has a maximum at τ0. We see that with F(x) := f (−x) we have
q = f ?g = F ∗g ⇐⇒ q = F · g = f · g
Note: For the convolution we have f ∗g = g∗ f . For the cross correlation we have q = f ?g , r = g? f and q(x) = r(−x).
Autocorrelation
It can also be useful to have f = g: For example, f may be a sound signal containing echos. In this case q will always havea maximum at zero, but we expect smaller local maxima at locations corresponding to the time shift for the echos.
If a discrete signal is approximately periodic with period L we can also detect this with the autocorrelation: In this case weexpect local maxima at multiples of L. Note:
q = f ? f ⇐⇒ q = f · f =∣∣ f ∣∣2
Note that the autocorrelation is a method to detect periodicity using the time domain. Since we expect peaks for q at multiplesfor L there is some periodic behavior with period L in q. This will show up as peaks in q =
∣∣ f ∣∣2 corresponding to frequencyξ = L−1 and multiples. This explains the connection to the method of looking for peaks in
∣∣ f ∣∣.Discrete signals of �nite length
In practice we have samples Fj and G j for values x j = jh and we define the cross correlation as
Q j = h∞
∑`=−∞
F G`+ j
In applications all values outside of a finite range are zero: Fj = 0 if j /∈ {a,a+ 1, . . . ,a+ L− 1}, G j = 0 if j /∈ {b,b+1, . . . ,b+M−1}.f = zeros(1,N); f(1:L) = F;g = zeros(1,N); g(1:L) = G;q = four(conj(ifour(f)).*ifour(g)); Q = q(1:L+M-1)
In Matlab V=xcorr(F,G) gives [Q(M:-1:1),Q(L+M-1:-1:M+1)].
72
6 Solving di�erential equations
6.1 Review of simple ODE examples
We want to find a function u(t) which satisfies an equation containing u(t) and u′(t).
Example 1: Find a function u(t) for t ≥ 0 such that
u′(t)+au(t) = 0, u(0) = u0
where a,u0 ∈R are given. The condition u(0) = u0 is called initial condition, and the ODE together with the initial conditionforms an initial value problem (IVP).
In this case it is easy to see that the solution of the ODE is u(t) =Ce−at , and the initial condition gives u0 =C:
u(t) = u0e−at (75)
Example 2: Find a function u(t) for t ≥ 0 such that
u′(t)+au(t) = b, u(0) = u0
where a 6= 0, u0 ∈ R are given.
In this case it is easy to see that the solution is u(t) =Ce−at + ba , and the initial condition gives u0 =C+ b
a :
u(t) = u0e−at +ba
(1− e−at) (76)
6.2 The heat equation ut−uxx = 0
We consider a thin rod of infinite length. The initial temperature at time t = 0 and position x ∈ R is given by the functiong(x). We want to find the temperature u(x, t) at position x ∈ R for all times t ≥ 0. We use the notation ut(x, t) = ∂
∂ t u(x, t),
ux(x, t) = ∂
∂x u(x, t), uxx(x, t) =(
∂
∂x
)2u(x, t).
We now have to model heat conduction.
Consider a point x0 on the rod at time t0. If the average temperature in a small neighborhood of x0 is larger than u(x0, t0) heatconduction will lead to increasing temperature at x0, i.e., ut(x0, t0)> 0.Conversely, if the average temperature in a small neighborhood of x0 is less than u(x0, t0) heat conduction will lead todecreasing temperature at x0, i.e., ut(x0, t0)> 0.
This can be modeled using uxx(x0, t0): We want that
ut = K ·uxx(x, t)
where the constant K describes the heat conductivity of the material. We will use K = 1 for simplicity.
Therefore we obtain the following initial value problem: Given an initial temperature function g(x) for x ∈R, find a functionu(x, t) such that for all x ∈ R, t ≥ 0
ut(x, t) = uxx(x, t) and u(x,0) = g(x) (77)
This is a partial differential equation since it contains derivatives with respect to t and with respect to x.
We can get rid of the derivatives with respect to x by applying the Fourier transform to the x-variable: we can write
u(x, t) =∫
∞
−∞
u(ξ , t)e2πiξ xdξ
ut(x, t) =∫
∞
−∞
∂
∂ tu(ξ , t)e2πiξ xdξ
uxx(x, t) =∫
∞
−∞
u(ξ , t)(2πiξ )2e2πiξ xdξ
73
Therefore the initial value problem (77) becomes in the frequency domain
∂
∂ tu(ξ , t) =−4π
2ξ
2u(ξ ), and u(ξ ,0) = g(ξ )
Note that for each fixed ξ ∈ R we obtain an initial value problem of the form v′(t) =−av(t),v(0) = v0 which we discussedin the previous section. Here a = 4π2ξ 2 and the solution is
u(ξ , t) = g(ξ )e−4π2ξ 2t (78)
yielding in the time domainu(x, t) = F−1
ξ 7→x
[g(ξ )e−4π2ξ 2t
].
Let us first consider g(ξ ) = 1 corresponding to g = δ0, i.e., the initial temperature is a sharp peak at x = 0.
Recall that
• F−1ξ 7→x
[e−πξ 2
]= e−πx2
• F−1ξ 7→x
[f (λξ )
]= λ−1 f (λ−1x) for λ > 0
Therefore we obtain for e−4π2tξ 2= e−π(
√4πt·ξ ) with λ =
√4πt
F−1ξ 7→x
[e−4π2ξ 2t
]=
1√4πt
e−π
(x√4πt
)2
=1√4πt
e−x24t
We call this function the fundamental solution Φ(x, t):
Φ(x, t) =1√4πt
e−x24t
For a general function g(x) we have u(x, t) = F−1ξ 7→x
[g(ξ )e−4π2ξ 2t
], hence u(x, t) is the convolution of g and Φ(·, t):
u(·, t) = g∗Φ(·, t)
Note that (78) gives ∫∞
−∞
u(x, t)dt = u(0, t) = g(0) =∫
∞
−∞
g(x)dx
This means that the “total heat” contained in the rod remains constant for all times. This is reasonable since there is no heatflow into the rod, or out of the rod.
6.3 The heat equation ut−uxx = f (x) with source term
Now we assume that we heat or cool the rod: at a point x we have a heat flow f (x) into the rod.
We first assume that f (x) = F ′′(x) for some function F(x) with F(x) → 0 as x → ±∞. Then f (ξ ) = (2πiξ )2F(ξ ) =−4π2ξ 2F(ξ ).
Applying the Fourier transform with respect to x gives
ut(x, t)−uxx(x, t) = f (x), u(x,0) = g(x)
ut(ξ , t)+4π2ξ
2u(ξ , t) = f (ξ ), u(ξ ,0) = g(ξ )
74
Therefore (76) gives
u(ξ , t) = g(ξ )e−4π2ξ 2t +f (ξ )
4π2ξ 2
(1− e−4π2ξ 2t
)= g(ξ )e−4π2ξ 2t + F(ξ )
(e−4π2ξ 2t −1
)yielding
u(·, t) = g∗Φ(·, t)+F ∗Φ(·, t)−F
Now we assume that f (x) is nonzero only in some interval [−M,M], but the second antiderivative F(x) will then be a straightline for x < −M and x > M. For a straight line w(x) = ax+ b we have that Φ(·, t) ∗w = w. Therefore F ∗Φ(·, t)−F willdecay exponentially fast to zero as x→±∞. Therefore we obtain a reasonable solution to our initial value problem.
Note that the function
β (x) :=
{1 for x = 0ex−1
x otherwise
is smooth and we haveu(ξ , t) = g(ξ )e−4π2ξ 2t + t · f (ξ ) ·β
(−4π
2ξ
2t)
6.4 The heat equation in two dimensions
Now we consider an infinite thin plate described by x =[
x1x2
]∈R2. We are given the initial temperature g(x) = g(x1,x2) and
we want to find the solution u(x, t) = u(x1,x2, t). We need to solve the initial value problem
ut = ux1x1 +ux2x2 , u(x1,x2,0) = g(x1,x2)
In order to get rid of the derivatives with respect to x1,x2 we use the Fourier transform with respect to x1 and x2:
u(ξ1,ξ2, t) = Fx 7→ξ u(x1,x2, t) = Fx1 7→ξ1Fx2 7→ξ2u(x1,x2, t)
and have with e2πiξ1x1e2πiξ2x2 = e2πi(ξ1x1+ξ2x2) = e2πi(ξ ,x) and the inner product (ξ ,x) = ξ1x1 +ξ2x2
u(x1,x2, t) =∫
∞
ξ2=−∞
∫∞
ξ1=−∞
u(ξ1,ξ2, t)e2πi(ξ1x1+ξ2x2)dξ1dξ2
u(x, t) =∫∫
ξ∈R2u(ξ , t)e2πi(ξ ,x)dξ1dξ2
yielding to the ODE IVP
∂
∂ tu(ξ1,ξ2, t) =−4π
2(ξ 21 +ξ
22 )u(ξ1,ξ2, t), u(ξ1,ξ2,0) = g(ξ1,ξ2)
which has the solutionu(ξ , t) = g(ξ )e−4π2(ξ 2
1 +ξ 22 )t
We now consider for g the Dirac delta δ~0 concentrated in ~0 =[
00
]: For smooth test functions ϕ(x1,x2) we have∫
∞
−∞
∫∞
−∞δ~0ϕ dx1dx2 = ϕ(0,0). For g = δ~0 we have g(ξ ) = 1 and we obtain the fundamental solution Φ(x1,x2, t):
Φ(x1,x2, t) = F−1ξ2 7→x2
F−1ξ1 7→x1
e−4π2(ξ 21 +ξ 2
2 )t =(F−1
ξ2 7→x2e−4π2ξ 2
2 t)(
F−1ξ1 7→x1
e−4π2ξ 21 t)=
(1√4πt
e−x21
4t
)(1√4πt
e−x22
4t
)Φ(x1,x2, t) =
14πt
e−x21+x2
24t
For functions f (x1,x2), g(x1,x2) we define the two-dimensional convolution q = f ∗g by first convolving with respect to x1and then with respect to x2
q(x1,x2) =∫
∞
t2=−∞
∫∞
t1=−∞
f (t1, t2)g(x1− t1,x2− t2)dt1dt2
75
and haveq(ξ1,ξ2) = f (ξ1,ξ2)g(ξ1,ξ2)
Therefore the solution of our initial value problem with initial data g(x) is given by
u(·, t) = g∗Φ(·, t)
where now “ ·” denotes x = [ x1x2 ] and “∗” denotes the two dimensional convolution over x1,x2.
76
