Post on 25-Apr-2018
Wavelets
• Unlike the Fourier transform, whose basis functions are sinusoids, wavelet transforms are based on small waves of limited duration
• Applications – Image compression – Image denoising
• Background we need – Multiresolution image pyramids – Subband coding – Multiresolution analysis and scaling functions
Multiresolution
• Statistics of images such as histograms can vary significantly from one part of the image to another
• Small objects – Analyze at high-resolution
• Large objects – Analyze at low-resolution
• Need to analyze images at multiple resolutions
© 1992–2008 R. C. Gonzalez & R. E. Woods
Image pyramids
N=2j j=log2N
• Level 0 is of little value • Normally truncated at level P
• j=J-P,…,J (P+1 levels)
© 1992–2008 R. C. Gonzalez & R. E. Woods
• Approximation filter • Gaussian • Mean • No filtering
• Interpolation filter • Nearest neighbor • Bilinear interpolation
f2↑ n( ) =f (n / 2) if n is even
0 otherwise"#$
f2↓ n( ) = f (2n)
Approximation and residual pyramids
• Approximation pyramid – 512 x 512 (j=9)
to 64 x 64 (j=6)
• Residual pyramid – 64 x 64
approximation image at top of pyramid, rest are residuals
– Higher compressibility • Fewer bits © 1992–2008 R. C. Gonzalez & R. E. Woods
Reconstruction from residual pyramid
© 1992–2008 R. C. Gonzalez & R. E. Woods
Upsample 64 x 64 approximation image to 128 x 128 Apply interpolation filter to predict 128 x 128 image Add 128 x 128 residual image (Now have 128 x 128 approximation image) Upsample 128 x 128 approximation image from previous step to 256 x 256 Apply interpolation filter to predict 256 x 256 image Add 256 x 256 residual image (Now have 256 x 256 approximation image) Upsample 256 x 256 approximation image from previous step to 512 x 512 Apply interpolation filter to predict 512 x 512 image Add 512 x 512 residual image
Exact reconstruction if the approximation/residual images were not quantized
Functionally related impulse responses
Subband Coding
• Perfect reconstruction filters:
– Biorthogonal filters • Two prototyped required • Cross modulation
constraint:
• Biorthogonality condition
Approximation of f(n)
Detail part of f(n)
g0 n( ) = −1( )n h1 n( )g1 n( ) = −1( )n+1 h0 n( )
f̂ (n) = f (n)
© 1992–2008 R. C. Gonzalez & R. E. Woods
hi 2n− k( ),gj k( ) = δ i− j( )δ n( )
Subband Coding
• Perfect reconstruction filters:
– Orthonormal filters
• Given a single prototype filter g0, remaining 3 can be computed to satisfy the orthonormality constraints
Approximation of f(n)
Detail part of f(n)
g1 n( ) = −1( )n+1 g0 Keven −1− n( )hi n( ) = gi Keven −1− n( )
f̂ (n) = f (n)
© 1992–2008 R. C. Gonzalez & R. E. Woods
gi n( ),gj n+ 2m( ) = δ i− j( )δ m( )
Daubechies filter
g1 n( ) = −1( )n+1 g0 Keven −1− n( )hi n( ) = gi Keven −1− n( )
Two-dimensional subband coding
• Daubechies orthonormal filters – Prototype:
Approximation
Horizontal detail
Vertical detail
Diagonal detail © 1992–2008 R. C. Gonzalez & R. E. Woods
Reconstruction
© 1992–2008 R. C. Gonzalez & R. E. Woods
1. Upsample columns of all subimages
2. Filter a and dH along columns with g0
3. Filter dV and dD along columns with g1
4. Result of step 2: upsample rows, filter with g0
5. Result of step 3: upsample rows, filter with g1
6. Add result of steps 4 and 5
The Haar basis • Recursively keep
replacing approximation image with its decomposition – Stop at some level
and keep approximation
• Properties: – Histograms of all
detail images very similar
– Can reconstruct image at various resolutions
h0 = 1 2 1 2!" #$
h1 = 1 2 −1 2!" #$
© 1992–2008 R. C. Gonzalez & R. E. Woods
Orthonormal Haar basis
Series expansions
• Example Fourier series – Expansion set: Sines and cosines – Span: Periodic functions
f (x) = α kϕk (x)k∑Function
Expansion coefficients
Basis functions
V = Spank
ϕk x( ){ }
Expansion set: The set of basis functions
Span of expansion set: The space of functions expressible with the chosen expansion set
Expansion coefficients
• Orthonormal basis (basis and dual equivalent):
– Computation of expansion coefficients:
α k = !ϕ(x), f (x) = !ϕk
*(x)∫ f (x)dx
Dual of basis function
f (x) = α kϕk (x)k∑
ϕ j (x),ϕk (x) = ϕ j*(x)ϕk (x)∫ dx = δ jk =
0 j ≠ k1 j = k%&'
α k = ϕk (x), f (x)
Inner product Defn. of inner product
Scaling functions
• Width scaling changes shape
• Amplitude scaling makes sure that
ϕ j ,k x( ) = 2 j /2ϕ 2 j x − k( )
Scaling function: Real, square-integrable prototype function
Amplitude scaling
Scaling of width
Position along x-axis
© 1992–2008 R. C. Gonzalez & R. E. Woods
ϕ j ,k ,ϕ j ,k = 1
Example (Haar)::
Example: Haar scaling function
• Vj is the span achieved by fixing j and varying k
• As j increases, the size of Vj increases to include functions with finer detail
© 1992–2008 R. C. Gonzalez & R. E. Woods
ϕ(x) =1 0 ≤ x < 10 otherwise#$%
Shifted j=0,k=1
Shifted & Scaled j=1,k=1
Scaled j=1,k=0
Vj = Spank
ϕ j ,k x( ){ }
f (x) = 0.25ϕ1,0 (x) +ϕ1,1(x) − 0.25ϕ1,4 (x)
Nested function spaces
• All V0 expansion functions are contained in V1
• All Vj expansion functions are contained in Vj+1
• Any function in Vj is also in Vj+1
© 1992–2008 R. C. Gonzalez & R. E. Woods
Shifted j=0,k=1
Shifted & Scaled j=1,k=1
Scaled j=1,k=0
ϕ0,k (x) =12ϕ1,2k (x) +
12ϕ1,2k+1(x)
Multiresolution analysis requirements 1. The scaling function is orthogonal to its integer
translates 2. The subspaces spanned by the scaling function
at low scales are nested within those spanned at higher scales
3. The only function common to all Vj is f(x)=0 4. Any square integrable function can be
represented with arbitrary precision: • Haar scaling function obeys all these
requirements • Under these conditions:
V∞ = L2 R( ){ }
ϕ j ,k x( ) = αnϕ j+1,n (x)n∑
V−∞ = 0{ }
ϕ j ,k x( ) = αnϕ j+1,n (x)n∑
Substitute ϕ j+1,n x( ) = 2j+1
2ϕ 2 j+1x − n( )and change αn to hϕ (n), then
ϕ j ,k x( ) = hϕ (n)n∑ 2
j+12ϕ 2 j+1x − n( )
set j = k = 0,also note that ϕ0,0 (x) = ϕ(x)
ϕ x( ) = hϕ (n)n∑ 2ϕ 2x − n( ) Refinement equation:
Expansion functions can be built from double resolution copies of themselves
Scaling function coefficients
Refinement equation ϕ j ,k x( ) = 2 j /2ϕ 2 j x − k( )Recall
Refinement equation example
• Haar function
• Scaling coefficients
• Refinement equation
ϕ(x) =1 0 ≤ x < 10 otherwise#$%
hϕ (0) = hϕ (1) = 12
ϕ x( ) = hϕ (n)n∑ 2ϕ 2x − n( )
ϕ x( ) = ϕ 2x( ) +ϕ 2x −1( )
Wavelet functions
• Wavelet functions span the difference between adjacent scaling subspaces Vj and Vj+1 – Given a scaling function that meets the requirements
discussed, we can design a wavelet function
ψ j ,k x( ) = 2 j /2ψ 2 j x − k( )Wavelet function
Amplitude scaling
Scaling of width
Position along x-axis
Wj = Spank
ψ j ,k x( ){ }Vj+1 = Vj ⊕Wj
Union of function spaces © 1992–2008 R. C. Gonzalez & R. E. Woods
Function representation
L2 R( ) = V0 ⊕W0 ⊕W1⊕ ...L2 R( ) = V1⊕W1⊕W2 ⊕ ...L2 R( ) = Vjo
⊕Wjo⊕Wjo +1
⊕ ...
© 1992–2008 R. C. Gonzalez & R. E. Woods
Generating wavelet functions
• For the Haar scaling function:
• Then:
ψ(x) = hψ (n)n∑ 2ϕ 2x − n( )
ϕ j,k (x),ψ j,l (x) = 0
hψ (n) = −1( )n hϕ (1− n) Wavelet function coefficients Modulation, time reversal
hϕ (0) = hϕ (1) = 12
hψ (0) = −1( )0 hϕ (1− 0) = 12
hψ (1) = −1( )1 hϕ (1−1) = − 1 2
For any k, l
Vj+1 = Vj ⊕Wjbecause
Haar wavelet function
ψ (x) = hψ (n)n∑ 2ϕ 2x − n( )
Substitute hψ (0) = 12
and hψ (1) = − 12
ψ (x) = ϕ(2x) −ϕ(2x −1)
ψ (x) =1 0 ≤ x < 0.5−1 0.5 ≤ x < 10 otherwise
&
'(
)(
Haar wavelet function
• f(x) – In space V1
• fa(x) approximation – In space V0
• fd(x) difference – In space W0
ψ (x) =1 0 ≤ x < 0.5−1 0.5 ≤ x < 10 otherwise
$
%&
'&
f (x) = fa (x) + fd (x)
V1 = V0 ⊕W0© 1992–2008 R. C. Gonzalez & R. E. Woods
1D Wavelet Transforms
Wavelet Series Fourier Series
Continuous Wavelet Transform
(Continuous) Fourier Transform
Discrete Wavelet Transform
Discrete Fourier Transform
Wavelet Series
• The first sum uses scaling functions to provide an approximation of f(x) at a chosen starting scale jO. This sum is over translations (k) only.
• The second sum is over scales (greater than jO) and over translations (k). It provides ever increasing detail (finer resolution)
f (x) = cjO (k)ϕ jO ,k(x) + dj (k)ψ j ,k (x)
k∑
j= jO
∞
∑k∑
Approximation (scaling) coefficients
Detail (wavelet) coefficients
Wavelet Series Coefficients
• This assumes an orthonormal basis (or a tight frame) which is often the case.
f (x) = cjO (k)ϕ jO ,k(x) + dj (k)ψ j ,k (x)
k∑
j= jO
∞
∑k∑
cjO (k) = f (x)∫ ϕ jO ,k(x)dx
dj (k) = f (x)ψ j ,k (x)∫ dx
Wavelet Series Example
© 1992–2008 R. C. Gonzalez & R. E. Woods
Discrete Wavelet Transform (DWT) • Like the DFT, the DWT operates on discrete
functions (length M=2J).
f (n) = 1M
Wϕ jO,k( )ϕ jO ,k(n)
k=0
2 j0−1
∑ +1M
Wψ j,k( )ψ j,k (n)k=0
2 j−1
∑j= jO
J−1
∑
Sampled scaling function
Sampled wavelet function
Wϕ ( jO,k) = 1M
f (n)ϕ jO ,k (n)n=0
M−1
∑
Wψ ( j,k) = 1M
f (n)ψ j,k (n)n=0
M−1
∑ for j ≥ jO
Haar transform
ϕ(x) =1 0 ≤ x < 10 otherwise#$%
ψ (x) =1 0 ≤ x < 0.5−1 0.5 ≤ x < 10 otherwise
#
$(
%( T = HFHT
H2 =121 11 −1"
#$
%
&'
H4 =12
1 1 1 11 1 −1 −1
2 − 2 0 0
0 0 2 − 2
"
#
$$$$$
%
&
'''''
ϕ jk (x) = 2 j /2ϕ(2 j x − k)sample ϕ jk (x) to get ϕ jk (n)
ψ jk (x) = 2 j /2ψ (2 j x − k)sample ψ jk (x) to get ψ jk (n)
Wϕ (0,k) =1M
f (n)ϕ0,k (n)n∑
Wψ ( j,k) =1M
f (n)ψ j ,k (n)n∑
Fast Wavelet Transform (FWT)
• It can be shown that
• This is a very useful relationship between the DWT coefficients at adjacent levels.
Wψ ( j,k) = hψ (m − 2k)m∑ Wϕ ( j +1,m)
Wϕ ( j,k) = hϕ (m − 2k)m∑ Wϕ ( j +1,m)
Convolution with hψ(-n)
Convolution with hϕ(-n)
© 1992–2008 R. C. Gonzalez & R. E. Woods
Fast Wavelet Transform
© 1992–2008 R. C. Gonzalez & R. E. Woods
Coefficients at highest scale are samples of the function itself
Another iteration would split VJ-2 into 1/8th frequency bands
Fast Wavelet Transform
© 1992–2008 R. C. Gonzalez & R. E. Woods
Coefficients at highest scale are samples of the function itself
Another iteration would split VJ-2 into 1/8th frequency bands
Wϕ ( jO,k) = 1M
f (n)ϕ jO ,k (n)n=0
M−1
∑
Wψ ( j,k) = 1M
f (n)ψ j,k (n)n=0
M−1
∑ for j ≥ jO
Do this:
Instead of this:
Haar FWT example
© 1992–2008 R. C. Gonzalez & R. E. Woods
hϕ (0) = hϕ (1) = 12
hψ (0) = 12
hψ (1) = − 1 2
Inverse FWT
© 1992–2008 R. C. Gonzalez & R. E. Woods
f (n) = 1M
Wϕ jO,k( )ϕ jO ,k(n)
k=0
2 j0−1
∑ +1M
Wψ j,k( )ψ j,k (n)k=0
2 j−1
∑j= jO
J−1
∑
Do this:
Instead of this:
Example: Haar inverse FWT
© 1992–2008 R. C. Gonzalez & R. E. Woods
f (n) = 1M
Wϕ jO,k( )ϕ jO ,k(n)
k=0
2 j0−1
∑ +1M
Wψ j,k( )ψ j,k (n)k=0
2 j−1
∑j= jO
J−1
∑
J= 2, j0=0
Do this:
Instead of this:
2D Wavelet transform
• Separable 2D scaling function – ϕ(x,y)=ϕ(x)ϕ(y)
• Directionally sensitive separable 2D wavelet functions – ψH(x,y)=ψ(x)ϕ(y) : horizontal detail – ψV(x,y)=ϕ(x)ψ(y) : vertical detail – ψD(x,y)=ψ(x)ψ(y) : diagonal detail
Scaled and translated basis functions
• We can enumerate the last 3 equations as
ϕ j ,m,n x, y( ) = 2j2ϕ 2 j x − m,2 j y − n( )
ψ j ,m,nH x, y( ) = 2
j2ψ H 2 j x − m,2 j y − n( )
ψ j ,m,nV x, y( ) = 2
j2ψ V 2 j x − m,2 j y − n( )
ψ j ,m,nD x, y( ) = 2
j2ψ D 2 j x − m,2 j y − n( )
ψ j ,m,ni x, y( ) = 2
j2ψ i 2 j x − m,2 j y − n( ), i = H ,V ,D{ }
2D Wavelet Transform
• Forward transform
• Inverse transform
f (x, y) = 1MN
Wϕ jO,m,n( )ϕ jO ,m,n(x, y)
n=0
2 j0−1
∑m=0
2 j0−1
∑ +
1MN
Wψ j,m,n( )ψ j ,m,n
i (x, y)n=0
2 j0−1
∑m=0
2 j0−1
∑j= jO
J−1
∑i= H ,V ,D{ }∑
Wϕ jO ,m,n( ) = 1MN
f (x, y)ϕ jO ,m,n (x, y)y=0
N −1
∑x=0
M −1
∑
Wψi j,m,n( ) = 1
MNf (x, y)ψ j ,m,n
i (x, y)y=0
N −1
∑x=0
M −1
∑ , i = H ,V ,D{ }
N = M = 2J
j = jO ,.., J −1m = 0,1,2,...,2 j −1n = 0,1,2,...,2 j −1
2D Wavelet Transform Implementation
© 1992–2008 R. C. Gonzalez & R. E. Woods
Example: DWT with various jO
• Top left – 128 x 128
image, J=7
• Top right: – 2D DWT jO=6
• Bottom left: – 2D DWT jO=5
• Bottom right: – 2D DWT jO=4
© 1992–2008 R. C. Gonzalez & R. E. Woods
Inverse Transform
Wavelet image processing
• Similar to processing in frequency domain • Approach
1. Compute 2D DWT of input image 2. Alter the transform 3. Compute inverse 2D DWT to get output image
• First row: edge detection
• Second row: vertical edge detection
Example: thresholding wavelet coefficients
© 1992–2008 R. C. Gonzalez & R. E. Woods
Wavelet image denoising
• Reduce influence of higher scale detail coefficients – Set to 0 all the detail wavelet coefficients that fall
below a threshold level – How many detail levels to threshold? – Should the threshold be the same for all levels? – Soft vs. hard thresholding
Example: Two highest levels thresholded with an interactively chosen global threshold
© 1992–2008 R. C. Gonzalez & R. E. Woods
• Throwing away the highest scale detail coefficients – Noise reduced – Edges preserved
• Throwing away 2 highest scale detail coefficients – Some detail and
edges lost
Throwing away detail coefficients Output Image Residual Image
© 1992–2008 R. C. Gonzalez & R. E. Woods
Wavelet Compression
• Wavelets pack the most important visual information into a small number of coefficients. The remaining coefficients can be quantized coarsely or truncated with little loss of visual information
• JPEG: Quantizes Discrete Cosine Transform on 8x8 blocks. – Need to subdivide into blocks.Creates a blocking
effect • JPEG-2000: Quantized discrete wavelet
transforms. Provides increased compression ratios. – Wavelets are inherently local basis functions. No
need to subdivide image. No blocking effect
Wavelet selection
Decomposition level selection
• Fixed global threshold of 25 applied to detail coefficients (not to approximation coefficients)
• Different levels of wavelet decomposition levels can be used – Initial levels provide the most gain
© 1992–2008 R. C. Gonzalez & R. E. Woods
• Detail images have a very peaked probability distribution around zero which allows for a small number of quantization levels
© 1992–2008 R. C. Gonzalez & R. E. Woods
Quantizing wavelet coefficients
• Coefficients need to be quantized • Can use an uniform quantizer • Better results can be achieved by
– Introducing a larger quantization interval around zero – Adapting the size of the quantization intervals from
scale to scale (JPEG-style scaling)
JPEG
© 1992–2008 R. C. Gonzalez & R. E. Woods
JPEG-2000
© 1992–2008 R. C. Gonzalez & R. E. Woods