Introduction to Wavelets and Wavelet Based...
Transcript of Introduction to Wavelets and Wavelet Based...
Introduction to Wavelets andWavelet Based Numerical Homogenization
Olof Runborg
Numerical Analysis,School of Computer Science and Communication, KTH
Summer School on Multiscale Modeling and Simulation in ScienceBosön, 2007-06-06
Wavelet multiresolution decomposition
Wavelet multiresolution decomposition
Wavelet multiresolution decomposition
Wavelet multiresolution decomposition
Wavelet multiresolution decomposition
Wavelet multiresolution decomposition
Wavelet multiresolution decomposition
Wavelet multiresolution decomposition
Mathematical description
An (orthogonal) Multiresolution Analysis (MRA) is a family of closedfunction spaces such that
1 · · · ⊂ Vj ⊂ Vj+1 ⊂ · · · ⊂ L2(R)
2 f (t) ∈ Vj ⇔ f (2t) ∈ Vj+1
3 ∪Vj = L2(R) and ∩Vj = 04 Exists ϕ for which ϕ(x − k)k∈Z is L2-orthonormal basis for V0.
Vj are called scaling spaces and ϕ is the scaling (or shape)function.1+2) gives that Vj contains functions with finer and finer detailswhen j increases, and eventually all of L2 by 3).2+4) gives that ϕj,k := 2j/2ϕ(2jx − k)k∈Z is an ON basis for Vj .(ϕj,k are dilations (2j ) and translations (k ) of the original ϕ.)
Mathematical description
An (orthogonal) Multiresolution Analysis (MRA) is a family of closedfunction spaces such that
1 · · · ⊂ Vj ⊂ Vj+1 ⊂ · · · ⊂ L2(R)
2 f (t) ∈ Vj ⇔ f (2t) ∈ Vj+1
3 ∪Vj = L2(R) and ∩Vj = 04 Exists ϕ for which ϕ(x − k)k∈Z is L2-orthonormal basis for V0.
Remarks:Vj are called scaling spaces and ϕ is the scaling (or shape)function.1+2) gives that Vj contains functions with finer and finer detailswhen j increases, and eventually all of L2 by 3).2+4) gives that ϕj,k := 2j/2ϕ(2jx − k)k∈Z is an ON basis for Vj .(ϕj,k are dilations (2j ) and translations (k ) of the original ϕ.)
Multiresolution Analysis (MRA)More remarks
For numerical calculations:
Good if ϕ(x) is localized spatiallyNeed to replace L2(R) by finite interval, e.g. L2([0,1]). Then
V0 ⊂ · · · ⊂ Vj ⊂ Vj+1 ⊂ · · · ⊂ L2([0,1])
and the spaces Vj are finite dimensional. In fact
Vj ∼ R2j.
Wavelet spaces
The “difference” between two successive scaling spaces Vj ⊂ Vj+1 isthe wavelet space Wj . It is defined as the orthogonal complement to Vjin Vj+1,
Vj+1 = Vj ⊕Wj , Vj ⊥ Wj .
Wj thus contains the details present in Vj+1 but not in Vj .By induction
Vj+1 = Wj ⊕Wj−1 ⊕ · · · ⊕W0 ⊕ V0.
and also ∪Wj = L2(R).
Moreover, there exists a mother wavelet ψ(x), such that
ψ(x − k)k∈Z
is an ON-basis for W0.
Wavelet multiresolution decomposition
Wavelet multiresolution decomposition
Wavelet spacesBasis
Dilations and translations of ψ,
ψj,k := 2j/2ψ(2jx − k)k∈Z,
is an ON-basis for Wj and ψj,k , ϕj,k
k∈Z
is an ON-basis for Vj+1.ψj,kj,k∈Z
is an ON-basis for L2(R).
Wavelet examples
Haar
Scaling function φ(x) Mother wavelet ψ(x)
Wavelet examples
Daubechies 4
Scaling function φ(x) Mother wavelet ψ(x)
Wavelet examples
Daubechies 6
Scaling function φ(x) Mother wavelet ψ(x)
Wavelet examples
Daubechies 8
Scaling function φ(x) Mother wavelet ψ(x)
Approximation Properties
Basic approximation mechanism:vanishing moments + space localization
|ujk | ≤ c2−(j+1/2)M supx∈Ω
∣∣∣u(M)(x)∣∣∣ .
Approximation Properties
Basic approximation mechanism:vanishing moments + space localization
DefinitionThe mother wavelet ψ has M vanishing moments if∫
ψ(x)xmdx = 0, m = 0, . . . ,M − 1.
Note 1: Implies the same for ψjk (x) = 2j/2ψ(2jx − k).
Note 2: There exists compactly supported wavelet systems with arbi-trary many vanishing moments. (Daubechies 88).
|ujk | ≤ c2−(j+1/2)M supx∈Ω
∣∣∣u(M)(x)∣∣∣ .
Approximation Properties
Basic approximation mechanism:vanishing moments + space localization
Suppose u(x) =∑
jk ujkψjk (x), and Ω = suppψjk . Hence,
ujk =
∫Ω
u(x)ψjk (x)dx ,
|ujk | ≤ c2−(j+1/2)M supx∈Ω
∣∣∣u(M)(x)∣∣∣ .
Approximation Properties
Basic approximation mechanism:vanishing moments + space localization
Suppose u(x) =∑
jk ujkψjk (x), and Ω = suppψjk . Hence,
ujk =
∫Ω
u(x)ψjk (x)dx ,
Take x0 ∈ Ω and Taylor expand:
ujk =
∫Ω
M−1∑m=0
u(n)(x0)(x − x0)
m
m!ψjk (x)dx +
∫Ω
R(x)(x − x0)
M
M!ψjk (x)dx
|ujk | ≤ c2−(j+1/2)M supx∈Ω
∣∣∣u(M)(x)∣∣∣ .
Approximation Properties
Basic approximation mechanism:vanishing moments + space localization
Suppose u(x) =∑
jk ujkψjk (x), and Ω = suppψjk . Hence,
ujk =
∫Ω
u(x)ψjk (x)dx ,
Take x0 ∈ Ω and Taylor expand:
ujk =
∫Ω
M−1∑m=0
u(n)(x0)(x − x0)
m
m!ψjk (x)dx +
∫Ω
R(x)(x − x0)
M
M!ψjk (x)dx
M vanishing moments ⇒ First term = 0.
|ujk | ≤ c2−(j+1/2)M supx∈Ω
∣∣∣u(M)(x)∣∣∣ .
Approximation Properties
Basic approximation mechanism:vanishing moments + space localization
ujk =
∫Ω
R(x)(x − x0)
M
M!ψjk (x)dx
|ujk | ≤ c2−(j+1/2)M supx∈Ω
∣∣∣u(M)(x)∣∣∣ .
Approximation Properties
Basic approximation mechanism:vanishing moments + space localization
ujk =
∫Ω
R(x)(x − x0)
M
M!ψjk (x)dx
Noting that supx∈Ω |R(x)| = supx∈Ω |u(M)(x)|
|ujk | ≤ |Ω|M supx∈Ω
∣∣∣u(M)(x)∣∣∣ ∫
Ω|ψjk (x)|dx ≤ |Ω|M+1/2 sup
x∈Ω
∣∣∣u(M)(x)∣∣∣ .
|ujk | ≤ c2−(j+1/2)M supx∈Ω
∣∣∣u(M)(x)∣∣∣ .
Approximation Properties
Basic approximation mechanism:vanishing moments + space localization
ujk =
∫Ω
R(x)(x − x0)
M
M!ψjk (x)dx
Noting that supx∈Ω |R(x)| = supx∈Ω |u(M)(x)|
|ujk | ≤ |Ω|M supx∈Ω
∣∣∣u(M)(x)∣∣∣ ∫
Ω|ψjk (x)|dx ≤ |Ω|M+1/2 sup
x∈Ω
∣∣∣u(M)(x)∣∣∣ .
Compact support of ψ ⇒ |Ω| ∼ 2−j , (since ψjk = 2j/2ψ(2jx − k)).
|ujk | ≤ c2−(j+1/2)M supx∈Ω
∣∣∣u(M)(x)∣∣∣ .
Approximation Properties
Basic approximation mechanism:vanishing moments + space localization
ujk =
∫Ω
R(x)(x − x0)
M
M!ψjk (x)dx
Noting that supx∈Ω |R(x)| = supx∈Ω |u(M)(x)|
|ujk | ≤ |Ω|M supx∈Ω
∣∣∣u(M)(x)∣∣∣ ∫
Ω|ψjk (x)|dx ≤ |Ω|M+1/2 sup
x∈Ω
∣∣∣u(M)(x)∣∣∣ .
Compact support of ψ ⇒ |Ω| ∼ 2−j , (since ψjk = 2j/2ψ(2jx − k)).
|ujk | ≤ c2−(j+1/2)M supx∈Ω
∣∣∣u(M)(x)∣∣∣ .
Approximation Properties
Basic approximation mechanism:vanishing moments + space localization
|ujk | ≤ 2−(j+1/2)M supx∈Ω
∣∣∣u(M)(x)∣∣∣ .
ujk decay rapidly when u(x) smooth locally and M large (manyvanishing moments).
Most fine-scale coefficients can be neglected. Just need to keepthose where u(x) has abrupt changes/discontinuitiesGood approximation also of piecewise smooth functions.
Approximation Properties
Basic approximation mechanism:vanishing moments + space localization
|ujk | ≤ 2−(j+1/2)M supx∈Ω
∣∣∣u(M)(x)∣∣∣ .
ujk decay rapidly when u(x) smooth locally and M large (manyvanishing moments).Most fine-scale coefficients can be neglected. Just need to keepthose where u(x) has abrupt changes/discontinuities
Good approximation also of piecewise smooth functions.
Approximation Properties
Basic approximation mechanism:vanishing moments + space localization
|ujk | ≤ 2−(j+1/2)M supx∈Ω
∣∣∣u(M)(x)∣∣∣ .
ujk decay rapidly when u(x) smooth locally and M large (manyvanishing moments).Most fine-scale coefficients can be neglected. Just need to keepthose where u(x) has abrupt changes/discontinuitiesGood approximation also of piecewise smooth functions.
Time Frequency Representation of Signals
Given a discrete signal f (n) with n = 0,1, . . ..
Time representation — total localization in time
f (n) =∑
j
fjδ(n − j)
Basis functions
Time Frequency Representation of Signals
Given a discrete signal f (n) with n = 0,1, . . ..
Time representation — total localization in time
f (n) =∑
j
fjδ(n − j)
Time Frequency Representation of Signals
Given a discrete signal f (n) with n = 0,1, . . ..
Fourier representation — total localization in frequency
f (n) =∑
j
fjexp(inj2π/N)
Basis functions
Time Frequency Representation of Signals
Given a discrete signal f (n) with n = 0,1, . . ..
Fourier representation — total localization in frequency
f (n) =∑
j
fjexp(inj2π/N)
Time Frequency Representation of Signals
Given a discrete signal f (n) with n = 0,1, . . ..
Wavelet representation — localization in time and frequency
f (n) =∑
ij
wijψij(n/2π)
Basis functions
Time Frequency Representation of Signals
Given a discrete signal f (n) with n = 0,1, . . ..
Wavelet representation — localization in time and frequency
f (n) =∑
ij
wijψij(n/2π)
Time Frequency Representation of Signals
Given a discrete signal f (n) with n = 0,1, . . ..
Wavelet representation — localization in time and frequency
f (n) =∑
ij
wijψij(n/2π)
Time Frequency Representation of SignalsExample
Time representation — total localization in time
Time Frequency Representation of SignalsExample
Fourier representation — total localization in frequency
Time Frequency Representation of SignalsExample
Wavelet representation — localization in time and frequency
Wavelet transforms
MRA system characterized by the filter coefficients gk and hk,
ϕ(x) =∑
k
hkϕ1,k (x) =√
2∑
k
hkϕ(2x − k),
ψ(x) =∑
k
gkϕ1,k (x) =√
2∑
k
gkϕ(2x − k).
(Note, ϕ,ψ ∈ V0 ⊂ V1 which is spanned by ϕ1,k .)Conversely,
ϕj+1,k (x)︸ ︷︷ ︸∈Vj+1
=∑
`
hk−2`ϕj,`(x)︸ ︷︷ ︸∈Vj
+∑
`
gk−2`ψj,`(x)︸ ︷︷ ︸∈Wj
,
In general: Compactly supported wavelets ⇔ gk and hk are finitelength (FIR) filters.Example: Haar has h0 = h1 = 1√
2and g0 = −g1 = 1√
2.
Wavelet multiresolution decomposition
Wavelet transforms, cont.
Suppose u ∈ Vj+1 is given
u(x) =∑
k
ukϕj+1,k (x).
We want to decompose u into its parts in Wj and Vj .Use identity above for ϕj+1,k
u(x) =∑
k
∑`
uk hk−2`ϕj,`(x) +∑
k
∑`
uk gk−2`ψj,`(x)
andu(x) =
∑`
Uc` ϕj,`(x)︸ ︷︷ ︸
∈Vj (“coarse” part)
+∑
`
U f`ψj,`(x)︸ ︷︷ ︸
∈Wj (“fine” part)
,
whereUc
` =∑
k
ukhk−2`, U f` =
∑k
ukgk−2`.
Wavelet transforms, cont.
Simple to extract the coarse and fine part of u = uk:
Wu =
(UfUc
), u ∈ Vj+1 Uf ∈ Wj , Uc ∈ Vj .
For compactly supported wavelets, W is sparse. It is also orthonormal,WTW = I.In Haar basis on [0,1],
W =1√2
1 −1 0 · · ·0 0 1 −1 0 · · ·...
.
.
.. . .
. . .0 0 · · · 0 1 −11 1 0 · · ·0 0 1 1 0 · · ·...
.
.
.. . .
. . .0 0 · · · 0 1 1
∈ R2j+1×2j+1.
Fast Wavelet Transform
Decomposition can be continued hierarchically
Uj+1 → U fj
Ucj → U f
j−1 Uc
j−1 → U fj−2
Ucj−2 → · · ·
O(2j+1) + O(2j) + O(2j−1) +
Total cost = O(N) where N = 2j+1 is the length of the transformedvector.
Wavelets – some contributors
Strömberg – first continuous waveletMorlet, Grossman – "wavelet"Meyer, Mallat, Coifman – multiresolution analysisDaubechies – compactly supported waveletsBeylkin, Cohen, Dahmen, DeVore – PDE methods using waveletsSweldens – lifting, second generation wavelets
. . . and many, many more.
Wavelet based homogenization[Beylkin,Brewster,Engquist,Dorobantu,Levy,Gilbert,O.R.,. . . ]
Suppose
Lj+1u = f , Lj+1 ∈ L(Vj+1,Vj+1) u, f ∈ Vj+1,
is a discretization (e.g. FD, FEM) of a differential equation onscale-level j + 1 where Lj+1 contains small scales.
Want to find an effective discrete operator Lj ′ , with j ′ j that computesthe coarse part of u.
C.f. classical homogenization.
Example (Elliptic eq, Haar)
∂x r(x/ε)∂xuε = f , ⇒ Lj+1 =1h2 ∆+Rε∆−.
where Rε is diagonal matrix sampling r(x/ε), and 2j ∼ 1/ε. Here onecould use
Lj ′ =1h2 ∆+R∆−.
Wavelet based homogenization[Beylkin,Brewster,Engquist,Dorobantu,Levy,Gilbert,O.R.,. . . ]
Suppose
Lj+1u = f , Lj+1 ∈ L(Vj+1,Vj+1) u, f ∈ Vj+1,
is a discretization (e.g. FD, FEM) of a differential equation onscale-level j + 1 where Lj+1 contains small scales.
Want to find an effective discrete operator Lj ′ , with j ′ j that computesthe coarse part of u.
C.f. classical homogenization.
Example (Elliptic eq, Haar)
∂x r(x/ε)∂xuε = f , ⇒ Lj+1 =1h2 ∆+Rε∆−.
where Rε is diagonal matrix sampling r(x/ε), and 2j ∼ 1/ε. Here onecould use
Lj ′ =1h2 ∆+R∆−.
Wavelet based homogenizationWavelet decomposition of operator
Start from equation
Lj+1u = f , Lj+1 ∈ L(Vj+1,Vj+1) u, f ∈ Vj+1.
Decompose equation in coarse and fine part (use WTW = I)
WLj+1WTWu = Wf ⇒(Aj BjCj Lj
)(U f
Uc
)=
(F f
F c
).
Eliminate U f ,(Lj − CjA−1
j Bj)Uc = F c − CjA−1j F f .
Supposing f smooth so F f = 0 and F c = f .
(Lj − CjA−1j Bj)Uc = f .
Wavelet based homogenizationWavelet decomposition of operator
Start from equation
Lj+1u = f , Lj+1 ∈ L(Vj+1,Vj+1) u, f ∈ Vj+1.
Decompose equation in coarse and fine part (use WTW = I)
WLj+1WTWu = Wf ⇒(Aj BjCj Lj
)(U f
Uc
)=
(F f
F c
).
Eliminate U f ,(Lj − CjA−1
j Bj)Uc = F c − CjA−1j F f .
Supposing f smooth so F f = 0 and F c = f .
(Lj − CjA−1j Bj)Uc = f .
Wavelet based homogenizationWavelet decomposition of operator
Start from equation
Lj+1u = f , Lj+1 ∈ L(Vj+1,Vj+1) u, f ∈ Vj+1.
Decompose equation in coarse and fine part (use WTW = I)
WLj+1WTWu = Wf ⇒(Aj BjCj Lj
)(U f
Uc
)=
(F f
F c
).
Eliminate U f ,(Lj − CjA−1
j Bj)Uc = F c − CjA−1j F f .
Supposing f smooth so F f = 0 and F c = f .
(Lj − CjA−1j Bj)Uc = f .
Wavelet based homogenizationNumerically homogenized operator
We call the matrix
Lj = Lj − CjA−1j Bj , Lj ∈ L(Vj ,Vj),
the (numerically) homogenized operator. SinceHalf the size of original Lj+1.Given Lj , f we can solve for coarse part of solution, Uc .Takes influence of fine scales into account.
Compare with classical homogenization:
L = ∇R(x/ε)∇ ⇒ L = ∇∫
R(x)dx∇−∇∫
R(x)∂χ
∂xdx∇
where χ solves the (elliptic) cell problem.
Wavelet based homogenization
Reduction can be repeated,
Lj → Lj−1 → Lj−2 → . . . , Lj ∈ L(Vj ,Vj),
to discard suitably many small scales / to get a suitably coarse grid.Also, condition number improves
κ(Lj) < κ(Lj+1).
Wavelet based homogenization
Problem: L sparse (banded) 6→ L sparse (banded). (Must invert Aj .)
However: Approximation properties of wavelets imply elements of A−1j
decay rapidly away from diagonal.
Therefore: L diagonally dominant in many important cases and can bewell approximated by a banded matrix. (Cf. a (local) differentialoperator.)
Different Approximation Strategies
1 “Crude” truncation to ν diagonals,2 Band projection to ν diagonals, defined by
Mx = band(M, ν)x , ∀x ∈ spanv1,v2, . . . ,vν.
v j = 1j−1,2j−1, . . . ,N j−1T , j = 1, . . . , ν.
C.f. “probing”, [Chan, Mathew], [Axelsson, Pohlman, Wittum].3 The above methods used on the matrix H instead, where e.g.
Lj+1 =1h2 ∆+R∆− ⇒ Lj =
1(2h)2 ∆+H∆−.
H can be seen as the effective material coefficient.4 The above methods used on the matrix A−1
j instead, whereLj = Lj − CjA−1
j Bj . [Levy, Chertock]
Elliptic 1D case
Consider the elliptic one-dimensional problem
∂xaε(x)∂xu = 1, u(0) = u′(1) = 0,
with standard second order discretization.Try two cases:
aε(x) = ”noise” aε(x) = ”narrow slit”
Elliptic 1D case – noiseDifferent approximation strategies
Exact
ν=13 ν=15 ν=17
0 0.5 1−1
−0.8
−0.6
−0.4
−0.2
0trunc(L, ν)
Exact
ν=3 ν=5 ν=7
0 0.5 1−1
−0.8
−0.6
−0.4
−0.2
0trunc(H, ν)
Exact
ν=1 ν=3 ν=5
0 0.5 1−1
−0.8
−0.6
−0.4
−0.2
0band(H, ν)
Exact
ν=3 ν=5 ν=7
0 0.5 1−1
−0.8
−0.6
−0.4
−0.2
0band(L, ν)
Elliptic 1D case – narrow slitDifferent approximation strategies
Exact
ν=13 ν=15 ν=17
0 0.5 1−1
−0.8
−0.6
−0.4
−0.2
0trunc(L,ν)
Exact
ν=3 ν=5 ν=7
0 0.5 1−1
−0.8
−0.6
−0.4
−0.2
0trunc(H, ν)
Exact
ν=1 ν=3 ν=5
0 0.5 1−1
−0.8
−0.6
−0.4
−0.2
0band(H, ν)
Exact
ν=7 ν=9 ν=11
0 0.5 1−1
−0.8
−0.6
−0.4
−0.2
0band(L,ν)
Elliptic 1D case – narrow slitMatrix element size
0
1000
2000
3000
4000
5000
6000
7000
5 10 15 20 25 30
5
10
15
20
25
30
L
0
0.5
1
1.5
2
2.5
3
5 10 15 20 25 30
5
10
15
20
25
30
H
ExamplesHelmholtz 2D case
Simulate a wave hitting a wall witha small opening modeld byHelmholtz
∇a(x , y)∇u + ω2u = 0, 0
0.2
0.4
0.6
0.8
1 00.2
0.40.6
0.81
0
0.2
0.4
0.6
0.8
1
x y
a
Wall with slit
ExamplesHelmholtz 2D case
0
0.5
1 00.5
1
−2
−1
0
1
2
Original
0
0.5
1 00.5
1
−2
−1
0
1
2
1 homogenization
0
0.5
1 00.5
1
−2
−1
0
1
2
2 homogenizations
0
0.5
1 00.5
1
−2
−1
0
1
2
3 homogenizations
ExamplesHelmholtz 2D case
0
0.5
1 00.5
1
−2
0
2
Untruncated operator
x y
0
0.5
1 00.5
1
−10
0
10
ν=5
x y
0
0.5
1 00.5
1
−2
0
2
ν=7
x y
0
0.5
1 00.5
1
−2
0
2
ν=9
x y
Helmholtz 2D caseMatrix element size
0 100 200 300 400 500
0
100
200
300
400
500
nz = 19026