Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for...
Transcript of Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for...
![Page 1: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/1.jpg)
Applied Harmonic Analysis meets Compressed Sensing
Gitta Kutyniok
(Technische Universitat Berlin)
ICERM Program “Network Science and Graph Algorithms”February 4, 2014
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 1 / 50
![Page 2: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/2.jpg)
Outline
1 Imaging SciencesInspiring Empirical ResultsGoal for Today
2 Review of Compressed SensingSeparation via Compressed SensingInpainting via Compressed Sensing
3 Algorithmic AspectsShearlet SystemsNumerical Results
4 Theoretical AnalysisGeometrically Clustered SparsityAnalysis of Separation (joint with D. L. Donoho)Analysis of Inpainting (joint with E. King and X. Zhuang)
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 2 / 50
![Page 3: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/3.jpg)
Ill-posed Inverse Problems in Data Analysis
Two Challenges:
Modern Data in general is often composed of two or moremorphologically distinct constituents. Task: Separation of components given the composed data.
Applications often cause loss of information or necessary informationcan not be collected. Task: Recovery of missing data given the observed data.
Novel Approach:
Applied Harmonic Analysis
Compressed Sensing
First empirical results by J. L. Starck, M. Elad, and D. L. Donoho.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 3 / 50
![Page 4: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/4.jpg)
Separating Artifacts in Images, I
+
(Source: Starck, Elad, and Donoho; 2006)
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 4 / 50
![Page 5: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/5.jpg)
Separating Artifacts in Images, II
Neurobiological Imaging:
Detection of characteristics of Alzheimer.
Separation of spines and dendrites.
+
(Source: Brandt, K, Lim, and Sundermann; 2010)
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 5 / 50
![Page 6: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/6.jpg)
Inpainting, I
(Source: Hennenfent and Herrmann; 2008)
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 6 / 50
![Page 7: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/7.jpg)
Inpainting, II
(Source: King, K, Lim, Zhuang; 2012)
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 7 / 50
![Page 8: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/8.jpg)
Applied Harmonic Analysis Approach
Methodology:Exploit a carefully designed representation system (ψλ)λ ⊆ H:
H ⊇ C ∋ f −→ (〈f , ψλ〉)λ −→∑
λ
〈f , ψλ〉ψλ = f
Two Main Goals:
(1) Decomposition
(2) Efficient representations
Main Desiderata:
Multiscale representation system.
Partition of Fourier domain.
Fast decomposition and reconstruction algorithm.
Optimally sparse approximation of the considered class.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 8 / 50
![Page 9: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/9.jpg)
Goal for Today
Challenges for Today:
Methodology to derive the empirical results!◮ Applied Harmonic Analysis.◮ Compressed Sensing.
Improvement of the methodology!◮ Shearlets as sparsifying system.
Analysis of the methodology!◮ Continuum model.◮ Geometrically clustered sparsity.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 9 / 50
![Page 10: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/10.jpg)
Goal for Today
Challenges for Today:
Methodology to derive the empirical results!◮ Applied Harmonic Analysis.◮ Compressed Sensing.
Improvement of the methodology!◮ Shearlets as sparsifying system.
Analysis of the methodology!◮ Continuum model.◮ Geometrically clustered sparsity.
Another Path to Imaging Science are Variational Approaches:
Contributors: Bertozzi, Burger, Chan, Esedoglu, Kang, Osher, Sapiro,Setzer, Shen, Steidl, Vese, Weikert, ...
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 9 / 50
![Page 11: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/11.jpg)
How does Compressed Sensing come into play?
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 10 / 50
![Page 12: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/12.jpg)
Underdetermined Situations
Separation:
Observe a signal x composed of two subsignals x1 and x2:
x = x1 + x2.
Extract the two subsignals x1 and x2 from x , if only x is known.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 11 / 50
![Page 13: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/13.jpg)
Underdetermined Situations
Separation:
Observe a signal x composed of two subsignals x1 and x2:
x = x1 + x2.
Extract the two subsignals x1 and x2 from x , if only x is known.
The two components are geometrically different.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 11 / 50
![Page 14: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/14.jpg)
Underdetermined Situations
Separation:
Observe a signal x composed of two subsignals x1 and x2:
x = x1 + x2.
Extract the two subsignals x1 and x2 from x , if only x is known.
The two components are geometrically different.
Inpainting:
Given a signalx = xK + xM ∈ HK ⊕HM .
Recover x , if only xK is known.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 11 / 50
![Page 15: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/15.jpg)
Underdetermined Situations
Separation:
Observe a signal x composed of two subsignals x1 and x2:
x = x1 + x2.
Extract the two subsignals x1 and x2 from x , if only x is known.
The two components are geometrically different.
Inpainting:
Given a signalx = xK + xM ∈ HK ⊕HM .
Recover x , if only xK is known.
The original signal is sparse within a frame.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 11 / 50
![Page 16: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/16.jpg)
Birth of Separation via Compressed Sensing
Composition of Sinusoids and Spikes sampled at n points:
x = x01 + x02 = Φ1c01 +Φ2c
02 = [ Φ1 | Φ2 ]
[
c01c02
]
,
where
x , c01 , and c02 are n × 1.
Φ1 is the n × n-Fourier matrix ((Φ1)t,k = e2πitk/n).
Φ2 is the n × n-Identity matrix.
0 50 100 150 200 250-1
-0.5
0
0.5
1
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 12 / 50
![Page 17: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/17.jpg)
Sparsity and ℓ1
Assumption: Letting A be an n × N-matrix, n << N, the seeked solutionc0 of x = Ac0 satisfies:
‖c0‖0 = #{i : c0i 6= 0} is ‘small’, i.e., c0 is sparse.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 13 / 50
![Page 18: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/18.jpg)
Sparsity and ℓ1
Assumption: Letting A be an n × N-matrix, n << N, the seeked solutionc0 of x = Ac0 satisfies:
‖c0‖0 = #{i : c0i 6= 0} is ‘small’, i.e., c0 is sparse.
Ideal: Solve...(P0) min
c‖c‖0 subject to x = Ac
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 13 / 50
![Page 19: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/19.jpg)
Sparsity and ℓ1
Assumption: Letting A be an n × N-matrix, n << N, the seeked solutionc0 of x = Ac0 satisfies:
‖c0‖0 = #{i : c0i 6= 0} is ‘small’, i.e., c0 is sparse.
Ideal: Solve...(P0) min
c‖c‖0 subject to x = Ac
Basis Pursuit (Chen, Donoho, Saunders; 1998)
(P1) minc
‖c‖1 subject to x = Ac
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 13 / 50
![Page 20: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/20.jpg)
Sparsity and ℓ1
Assumption: Letting A be an n × N-matrix, n << N, the seeked solutionc0 of x = Ac0 satisfies:
‖c0‖0 = #{i : c0i 6= 0} is ‘small’, i.e., c0 is sparse.
Ideal: Solve...(P0) min
c‖c‖0 subject to x = Ac
Basis Pursuit (Chen, Donoho, Saunders; 1998)
(P1) minc
‖c‖1 subject to x = Ac
Meta-Result: If the solution c0 is sufficiently sparse, and A is sufficientlyincoherent, then c0 can be recovered from x via (P1).
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 13 / 50
![Page 21: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/21.jpg)
First Results of Compressed Sensing
Composition of Sinusoids and Spikes sampled at n points:
x = x01 + x02 = Φ1c01 +Φ2c
02 = [ Φ1 | Φ2 ]
[
c01c02
]
.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 14 / 50
![Page 22: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/22.jpg)
First Results of Compressed Sensing
Composition of Sinusoids and Spikes sampled at n points:
x = x01 + x02 = Φ1c01 +Φ2c
02 = [ Φ1 | Φ2 ]
[
c01c02
]
.
Theorem (Donoho, Huo; 2001)If #(Sinusoids) + #(Spikes) = ‖(c01 )‖0 + ‖(c02 )‖0 < (1 +
√n)/2, then
(c01 , c02 ) = argmin(‖c1‖1 + ‖c2‖1) subject to x = Φ1c1 +Φ2c2.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 14 / 50
![Page 23: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/23.jpg)
First Results of Compressed Sensing
Composition of Sinusoids and Spikes sampled at n points:
x = x01 + x02 = Φ1c01 +Φ2c
02 = [ Φ1 | Φ2 ]
[
c01c02
]
.
Theorem (Donoho, Huo; 2001)If #(Sinusoids) + #(Spikes) = ‖(c01 )‖0 + ‖(c02 )‖0 < (1 +
√n)/2, then
(c01 , c02 ) = argmin(‖c1‖1 + ‖c2‖1) subject to x = Φ1c1 +Φ2c2.
Theorem (Bruckstein, Elad; 2002)(Donoho, Elad; 2003)Let A = (ai)
Ni=1 be an n× N-matrix with normalized columns, n << N,
and let c0 satisfy
‖c0‖0 <1
2
(
1 + µ(A)−1)
,
with coherence µ(A) = maxi 6=j |〈ai , aj〉|. Thenc0 = argmin‖c‖1 subject to x = Ac .
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 14 / 50
![Page 24: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/24.jpg)
Birth of Inpainting via Compressed Sensing
Main Idea:Let
x0 ∈ H be a signal.
Φ be an ONB (x0 = Φc0).
H = HM ⊕HK with orthogonal projections PM and PK .
ℓ1 Minimization Problem (Elad, Starck, Querre, Donoho; 2005):
c = argmin‖c‖1 subject to PKx0 = PKΦc x = Φc
Theorem (Donoho, Elad; 2003)
=⇒ If ‖c0‖0 < 12(1 + µ(PKΦ)
−1), then x0 = x .
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 15 / 50
![Page 25: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/25.jpg)
Two Paths
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 16 / 50
![Page 26: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/26.jpg)
Avalanche of Recent Work
Problem: Solve x = Ac0 with A an n × N-matrix (n < N).
Deterministic World:
Mutual coherence of A = (ak)k .
Bound ‖c0‖0 dependent on µ(A).
Efficiently solve the problem x = Ac0.
Contributors: Bruckstein, Cohen, Dahmen, DeVore, Donoho, Elad,Eldar, Fuchs, Gribonval, Huo, K, Rauhut, Temlyakov, Tropp, ...
Random World:
Restricted isometry constants of a random A = (ak)k .
Bound ‖c0‖0 by n/(2 log(N/n))(1 + o(1)).
Efficiently solve the problem x = Ac0 with high probability.
Contributors: Candes, Cohen, Dahmen, DeVore, Donoho, K,Krahmer, Rauhut, Romberg, Tanner, Tao, Tropp, Ward, ...
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 17 / 50
![Page 27: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/27.jpg)
Novel Direction for Sparsity
Geometric Sparsity (Donoho, K; 2009):
y = Ax0 with A an n × N-matrix (n < N).
Nonzeros of x0 often◮ arise not in arbitrary patterns,◮ but are rather highly structured.
Interactions between columns of A inill-posed problems
◮ is not arbitrary,◮ but rather geometrically driven.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 18 / 50
![Page 28: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/28.jpg)
Novel Direction for Sparsity
Geometric Sparsity (Donoho, K; 2009):
y = Ax0 with A an n × N-matrix (n < N).
Nonzeros of x0 often◮ arise not in arbitrary patterns,◮ but are rather highly structured.
Interactions between columns of A inill-posed problems
◮ is not arbitrary,◮ but rather geometrically driven.
Other results on “structured sparsity”:
Joint sparsity (Fornasier, Rauhut; 2008)
Block sparsity (Eldar, Kuppinger, Bolcskei; 2010)
Fusion frame sparsity (Boufonous, K, Rauhut; 2011)
...
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 18 / 50
![Page 29: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/29.jpg)
Which Sparsifying System to choose...?
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 19 / 50
![Page 30: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/30.jpg)
First: Separation
Main Idea:
Two morphologically distinct components:◮ Points◮ Curves
Choose suitable representation systems which provide optimallysparse representations of
◮ pointlike structures −→ Wavelets◮ curvelike structures −→ ???
Minimize the ℓ1 norm of the coefficients.
This forces◮ the pointlike objects into the wavelets part of the expansion◮ the curvelike objects into the ??? part.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 20 / 50
![Page 31: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/31.jpg)
Review of 2-D Wavelets
Definition (1D): Let φ ∈ L2(R) be a scaling function and ψ ∈ L2(R) be awavelet. Then the associated wavelet system is defined by
{φ(x −m) : m ∈ Z} ∪ {2j/2 ψ(2jx −m) : j ≥ 0,m ∈ Z}.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 21 / 50
![Page 32: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/32.jpg)
Review of 2-D Wavelets
Definition (1D): Let φ ∈ L2(R) be a scaling function and ψ ∈ L2(R) be awavelet. Then the associated wavelet system is defined by
{φ(x −m) : m ∈ Z} ∪ {2j/2 ψ(2jx −m) : j ≥ 0,m ∈ Z}.
Definition (2D): A wavelet system is defined by
{φ(1)(x −m) : m ∈ Z2} ∪ {2jψ(i)(2jx −m) : j ≥ 0,m ∈ Z
2, i = 1, 2, 3},
where ψ(1)(x) = φ(x1)ψ(x2),
φ(1)(x) = φ(x1)φ(x2) and ψ(2)(x) = ψ(x1)φ(x2),
ψ(3)(x) = ψ(x1)ψ(x2).
Theorem: Discrete wavelets provide optimally sparse approximations forfunctions f ∈ L2(R2), which are C 2 apart from point singularities:
‖f − fN‖22 ≍ N−1, N → ∞.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 21 / 50
![Page 33: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/33.jpg)
First: Separation
Main Idea:
Two morphologically distinct components:◮ Points◮ Curves
Choose suitable representation systems which provide optimallysparse representations of
◮ pointlike structures −→ Wavelets◮ curvelike structures −→ ???
Minimize the ℓ1 norm of the coefficients.
This forces◮ the pointlike objects into the wavelets part of the expansion◮ the curvelike objects into the ??? part.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 22 / 50
![Page 34: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/34.jpg)
Second: Inpainting
Main Idea:
Choose suitable representation system which provide optimallysparse representations of the original image.
Minimize the ℓ1 norm of the coefficients.
This fills in the missing part automatically.
Question: What is a good model for an image?
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 23 / 50
![Page 35: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/35.jpg)
Second: Inpainting
Main Idea:
Choose suitable representation system which provide optimallysparse representations of the original image.
Minimize the ℓ1 norm of the coefficients.
This fills in the missing part automatically.
Question: What is a good model for an image?
Field et al., 1993
Require: System to sparsify curvelike structures −→ ???
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 23 / 50
![Page 36: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/36.jpg)
Fitting Model for Anisotropic Structures
Definition (Donoho; 2001):The set of cartoon-like images E2(R2) is defined by
E2(R2) = {f ∈ L2(R2) : f = f0 + f1 · χB},
where B ⊂ [0, 1]2 with ∂B a closed C 2-curve, f0, f1 ∈ C 20 ([0, 1]
2).
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 24 / 50
![Page 37: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/37.jpg)
Fitting Model for Anisotropic Structures
Definition (Donoho; 2001):The set of cartoon-like images E2(R2) is defined by
E2(R2) = {f ∈ L2(R2) : f = f0 + f1 · χB},
where B ⊂ [0, 1]2 with ∂B a closed C 2-curve, f0, f1 ∈ C 20 ([0, 1]
2).
Theorem (Donoho; 2001):Let (ψλ)λ ⊆ L2(R2). Allowing only polynomial depth search, the optimalasymptotic approximation error of f ∈ E2(R2) is
‖f − fN‖22 ≍ N−2, N → ∞, where fN =∑
λ∈IN
cλψλ.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 24 / 50
![Page 38: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/38.jpg)
Beyond Wavelets...
Observation:
Wavelets only achieve ‖f − fN‖22 ≍ N−1, N → ∞.
Wavelets can not approximate curvilinear singularities optimallysparse.
Reason: Isotropic structure of wavelets:
2jψ(
(
2j 00 2j
)
x −m)
Intuitive explanation:
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 25 / 50
![Page 39: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/39.jpg)
Main Goal in Applied Harmonic Analysis
Design a representation system which...
...is generated by one ‘mother function’,
...provides optimally sparse approximation of cartoons,
...allows for compactly supported analyzing elements,
...is associated with fast decomposition algorithms,
...treats the continuum and digital ‘world’ uniformly.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 26 / 50
![Page 40: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/40.jpg)
Main Goal in Applied Harmonic Analysis
Design a representation system which...
...is generated by one ‘mother function’,
...provides optimally sparse approximation of cartoons,
...allows for compactly supported analyzing elements,
...is associated with fast decomposition algorithms,
...treats the continuum and digital ‘world’ uniformly.
Non-exhaustive list of approaches:
Ridgelets (Candes and Donoho; 1999)
Curvelets (Candes and Donoho; 2002)
Contourlets (Do and Vetterli; 2002)
Bandlets (LePennec and Mallat; 2003)
Shearlets (K and Labate; 2006)
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 26 / 50
![Page 41: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/41.jpg)
Scaling and Orientation
Parabolic scaling:
Aj =
(
2j 0
0 2j/2
)
, j ∈ Z.
Historical remark:
1970’s: Fefferman und Seeger/Sogge/Stein.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 27 / 50
![Page 42: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/42.jpg)
Scaling and Orientation
Parabolic scaling:
Aj =
(
2j 0
0 2j/2
)
, j ∈ Z.
Historical remark:
1970’s: Fefferman und Seeger/Sogge/Stein.
Orientation via shearing:
Sk =
(
1 k0 1
)
, k ∈ Z.
Advantage:
Shearing leaves the digital grid Z2 invariant.
Uniform theory for the continuum and digital situation.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 27 / 50
![Page 43: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/43.jpg)
(Cone-adapted) Discrete Shearlet Systems
Definition (K, Labate; 2006):The (cone-adapted) discrete shearlet system SH(φ,ψ, ψ) generated byφ ∈ L2(R2) and ψ, ψ ∈ L2(R2) is the set
{φ(· −m) : m ∈ Z2}
∪{23j/4ψ(SkAj · −m) : j ≥ 0, |k | ≤ ⌈2j/2⌉,m ∈ Z2}
∪{23j/4ψ(Sk Aj · −m) : j ≥ 0, |k | ≤ ⌈2j/2⌉,m ∈ Z2}.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 28 / 50
![Page 44: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/44.jpg)
(Cone-adapted) Discrete Shearlet Systems
Definition (K, Labate; 2006):The (cone-adapted) discrete shearlet system SH(φ,ψ, ψ) generated byφ ∈ L2(R2) and ψ, ψ ∈ L2(R2) is the set
{φ(· −m) : m ∈ Z2}
∪{23j/4ψ(SkAj · −m) : j ≥ 0, |k | ≤ ⌈2j/2⌉,m ∈ Z2}
∪{23j/4ψ(Sk Aj · −m) : j ≥ 0, |k | ≤ ⌈2j/2⌉,m ∈ Z2}.
General Framework:
Parabolic Molecules (Grohs, K; 2013)
α-Molecules (Grohs, Keiper, K, Schafer; 2014)
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 28 / 50
![Page 45: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/45.jpg)
Compactly Supported Shearlets
Theorem (Kittipoom, K, Lim; 2012):
Let φ,ψ, ψ ∈ L2(R2) be compactly supported, and let ψ, ˆψ satisfy certaindecay condition. Then SH(φ,ψ, ψ) forms a shearlet frame withcontrollable frame bounds.
Remark: Exemplary class with B/A ≈ 4.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 29 / 50
![Page 46: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/46.jpg)
Compactly Supported Shearlets
Theorem (Kittipoom, K, Lim; 2012):
Let φ,ψ, ψ ∈ L2(R2) be compactly supported, and let ψ, ˆψ satisfy certaindecay condition. Then SH(φ,ψ, ψ) forms a shearlet frame withcontrollable frame bounds.
Remark: Exemplary class with B/A ≈ 4.
Theorem (K, Lim; 2011):
Let φ,ψ, ψ ∈ L2(R2) be compactly supported, and let ψ, ˆψ satisfy certaindecay condition. Then SH(φ,ψ, ψ) provides an optimally sparseapproximation of f ∈ E2(R2), i.e.,
‖f − fN‖22 ≤ C · N−2 · (logN)3, N → ∞.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 29 / 50
![Page 47: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/47.jpg)
Recent Approaches to Fast Shearlet Transforms
www.ShearLab.org:
Separable Shearlet Transform (Lim; 2009)
Digital Shearlet Transform (K, Shahram, Zhuang; 2011)
2D&3D (parallelized) Shearlet Transform (K, Lim, Reisenhofer; 2013)
Additional Code:
Filter-based implementation (Easley, Labate, Lim; 2009)
Theoretical Approaches:
Adaptive Directional Subdivision Schemes (K, Sauer; 2009)
Shearlet Unitary Extension Principle (Han, K, Shen; 2011)
Gabor Shearlets (Bodmann, K, Zhuang; 2013)
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 30 / 50
![Page 48: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/48.jpg)
Image Separation: Points + Curves
MCALab 120 (52.74 sec) ShearLab (33.75 sec)
(Source: K, Lim; 2011)
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 31 / 50
![Page 49: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/49.jpg)
Inpainting
Original Noisy Version (80% missing)
Curvelets (29.95dB, 182.15sec) Shearlets (31.04dB, 85.18sec)
(Source: Lim; 2012)
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 32 / 50
![Page 50: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/50.jpg)
What is the Fundamental Mathematical Concept
behind the Empirical Success?
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 33 / 50
![Page 51: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/51.jpg)
General Analysis of Separation
Signal Model:x = x01 + x02 ∈ H
Remarks:
Given two tight frames Φ1, Φ2 (Φi(ΦTi x) = x for all x).
Too many decompositions x = Φ1c1 +Φ2c2.
Use x = Φ1(ΦT1 x1) + Φ2(Φ
T2 x2), where x = x1 + x2.
Norm is placed on analysis rather than synthesis side.
Decomposition Technique:
(x⋆1 , x⋆2 ) = argminx1,x2‖Φ
T1 x1‖1 + ‖ΦT
2 x2‖1 subject to x = x1 + x2
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 34 / 50
![Page 52: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/52.jpg)
Relative Sparsity and Cluster Coherence
Let Φ1 = (ϕ1,i )i∈I1 and Φ2 = (ϕ2,i )i∈I2.
Definition:
For each i = 1, 2, x0i is relatively sparse in Φi w.r.t. Λi , if
‖1Λc1ΦT1 x
01‖1 + ‖1Λc
2ΦT2 x
02‖1 ≤ δ.
We call Λ1 and Λ2 sets of significant coefficients.
We define cluster coherence for Λ1 by
µc(Λ1) = maxj∈I2
∑
i∈Λ1
|〈ϕ1,i , ϕ2,j〉|.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 35 / 50
![Page 53: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/53.jpg)
Central Estimate
Theorem (Donoho, K; 2011):Suppose x01 and x02 are relatively sparse with Λ1 and Λ2 sets of significantcoefficients. Then
‖x⋆1 − x01‖2 + ‖x⋆2 − x02‖2 ≤2δ
1− 2µc,
whereµc = max(µc(Λ1), µc (Λ2)).
δ: Relative sparsity measure.
µc : Cluster coherence.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 36 / 50
![Page 54: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/54.jpg)
Continuum Model
Neurobiological Geometric Mixture in 2D:
Point Singularity:
P(x) =P∑
i=1
|x − xi |−3/2
Curvilinear Singularity:
C =
∫
δτ(t)dt, τ a closed C 2-curve.
Observed Signal:f = P + C
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 37 / 50
![Page 55: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/55.jpg)
Scale-Dependent Decomposition
Observed Object:f = P + C.
Subband Decomposition:Wavelets and shearlets use the same scaling subbands!
fj = Pj + Cj , Pj = P ⋆ Fj and Cj = C ⋆ Fj .
ℓ1-Minimization:
(Wj ,Sj) = argmin‖(〈Wj , ψλ〉)λ‖1 + ‖(〈Sj , ση〉)η‖1 s.t. fj = Wj + Sj
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 38 / 50
![Page 56: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/56.jpg)
Application of Previous Result
x : Filtered signal fj (= Pj + Cj).Φ1: Wavelets filtered with Fj .
Φ2: Shearlets filtered with Fj .
Λ1: Significant wavelet coefficients of 〈ψλ,Pj 〉.Λ2: Significant shearlet coefficients of 〈ση, Cj 〉.δ: Degree of approximation by significant coefficients.
µc(Λ1), µc(Λ2): Cluster coherence of wavelets-shearlets.
Estimate of error: 2δ1−2µc
.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 39 / 50
![Page 57: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/57.jpg)
Application of Previous Result
x : Filtered signal fj (= Pj + Cj).Φ1: Wavelets filtered with Fj .
Φ2: Shearlets filtered with Fj .
Λ1: Significant wavelet coefficients of 〈ψλ,Pj 〉?Λ2: Significant shearlet coefficients of 〈ση, Cj 〉?δ: Degree of approximation by significant coefficients.
µc(Λ1), µc(Λ2): Cluster coherence of wavelets-shearlets.
Estimate of error: 2δ1−2µc
= o(‖Pj‖2 + ‖Cj‖2) as j → ∞.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 40 / 50
![Page 58: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/58.jpg)
Microlocal Analysis Heuristics
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 41 / 50
![Page 59: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/59.jpg)
Microlocal Analysis Heuristics
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 41 / 50
![Page 60: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/60.jpg)
Asymptotic Separation
Theorem (Donoho, K; 2011)
‖Wj − Pj‖2 + ‖Sj − Cj‖2‖Pj‖2 + ‖Cj‖2
→ 0, j → ∞.
At all sufficiently fine scales, nearly-perfect separation is achieved!
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 42 / 50
![Page 61: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/61.jpg)
Asymptotic Separation
Theorem (Donoho, K; 2011)
‖Wj − Pj‖2 + ‖Sj − Cj‖2‖Pj‖2 + ‖Cj‖2
→ 0, j → ∞.
At all sufficiently fine scales, nearly-perfect separation is achieved!
Theorem (K; 2013)Using thresholding as separation strategy, we can even prove that
WF (∑
j
Fj ⋆Wj) = WF (P) and WF (∑
j
Fj ⋆ Sj) = WF (C).
Exact separation of the wavefront sets!
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 42 / 50
![Page 62: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/62.jpg)
General Analysis of Inpainting
Signal Model:x0 = PKx
0 + PMx0 ∈ HK ⊕HM ,
where PK (PM) is the orthogonal projection onto HK (HM).
Remarks:
Given a tight frame Φ.
Redundancy Too many decompositions x = Φc .
Use x = Φ(ΦT x).
Norm is placed on analysis rather than synthesis side.
Inpainting Technique:
x⋆ = argminx‖ΦT x‖1 subject to PKx = PKx0.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 43 / 50
![Page 63: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/63.jpg)
Central Estimate
Theorem (King, K, Zhuang; 2013)Suppose x0 is δ-relatively sparse in Φ with Λ a set of significantcoefficients, and HM is the masked subspace. Then
‖x⋆ − x0‖2 ≤2δ
1− 2µc.
δ = δ(x0,Φ,Λ): Relative sparsity measure.
µc = µc(Φ,Λ,HM): Cluster coherence.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 44 / 50
![Page 64: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/64.jpg)
Continuum Model
Curvilinear Singularity:
C =
∫ ρ
−ρw(t)δτ(t)dt,
τ : [−1, 1] → R2 a C 2-curve, ρ < 1, and w : [−ρ, ρ] → R
+0 ‘bump’.
Mask:Mh = {(x1, x2) ∈ R
2 : |x1| ≤ h}, h > 0.
Observed Signal:f = 1R2\Mh
· C.Subband Decomposition:
C 7→ Cj = C ⋆ Fj .
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 45 / 50
![Page 65: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/65.jpg)
Scale-Dependent Decomposition
Asymptotic analysis:
Consider h = hj .
Set fj = 1R2\Mhj· Cj .
Algorithm:
Sj = argmin‖(〈Sj , ση〉)η‖1 s.t. fj = 1R2\Mhj· Sj
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 46 / 50
![Page 66: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/66.jpg)
Scale-Dependent Decomposition
Asymptotic analysis:
Consider h = hj .
Set fj = 1R2\Mhj· Cj .
Algorithm:
Sj = argmin‖(〈Sj , ση〉)η‖1 s.t. fj = 1R2\Mhj· Sj
Microlocal Analysis Heuristics:
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 46 / 50
![Page 67: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/67.jpg)
Asymptotic Inpainting
Theorem (King, K, Zhuang; 2013)For hj = o(2−j/2) as j → ∞, shearlet inpainting satisfies
‖Sj − Cj‖2‖Cj‖2
→ 0, j → ∞.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 47 / 50
![Page 68: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/68.jpg)
Asymptotic Inpainting
Theorem (King, K, Zhuang; 2013)For hj = o(2−j/2) as j → ∞, shearlet inpainting satisfies
‖Sj − Cj‖2‖Cj‖2
→ 0, j → ∞.
Theorem (King, K, Zhuang; 2013)For hj = o(2−j ) as j → ∞, wavelet inpainting satisfies
‖Wj − Cj‖2‖Cj‖2
→ 0, j → ∞.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 47 / 50
![Page 69: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/69.jpg)
Asymptotic Inpainting
Theorem (King, K, Zhuang; 2013)For hj = o(2−j/2) as j → ∞, shearlet inpainting satisfies
‖Sj − Cj‖2‖Cj‖2
→ 0, j → ∞.
Theorem (King, K, Zhuang; 2013)For hj = o(2−j ) as j → ∞, wavelet inpainting satisfies
‖Wj − Cj‖2‖Cj‖2
→ 0, j → ∞.
Theorem (King, K, Zhuang; 2013)“In case of thresholding, if hj = ω(2−j ), wavelets fail.”
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 47 / 50
![Page 70: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/70.jpg)
Let’s conclude...
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 48 / 50
![Page 71: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/71.jpg)
What to take Home...?
Compressed Sensing solves underdetermined linear systems ofequations exactly if the solution is sparse and the matrix is incoherent.
(Compactly supported) Shearlets provide optimally sparseapproximations of anisotropic features with a unified treatment of thecontinuum and digital world.
The Geometric Separation Problem and the Inpainting Problem canbe solved by these methodologies.
Asymptotically optimal performance can be theoretically proved.
Key features of our analysis:◮ Continuum model.◮ Geometrically clustered sparsity and cluster coherence.◮ Microlocal analysis viewpoint.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 49 / 50
![Page 72: Applied Harmonic Analysis meets Compressed Sensing · PDF fileGoalforToday Challenges for Today: Methodology to derive the empirical results! Applied Harmonic Analysis. Compressed](https://reader031.fdocuments.us/reader031/viewer/2022030406/5a7fa1117f8b9aee018bb838/html5/thumbnails/72.jpg)
Technische Universität BerlinApplied Functional Analysis Group
THANK YOU!
References available at:
www.math.tu-berlin.de/∼kutyniokCode available at:
www.ShearLab.org
Related Books:Y. Eldar and G. KutyniokCompressed Sensing: Theory and ApplicationsCambridge University Press, 2012.
G. Kutyniok and D. LabateShearlets: Multiscale Analysis for Multivariate DataBirkhauser-Springer, 2012.
Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 50 / 50