Post on 31-Mar-2015
1IPIM, IST, José Bioucas, 2007
Shrinkage/Thresholding Iterative Methods
• Nonquadratic regularizers• Total Variation• lp- norm• Wavelet orthogonal/redundant representations• sparse regression
• Majorization Minimization revisietd• IST- Iterative Shrinkage Thresolding Methods• TwIST-Two step IST
2IPIM, IST, José Bioucas, 2007
Linear Inverse Problems -LIPs
3IPIM, IST, José Bioucas, 2007
References
J. Bioucas-Dias and M. Figueiredo, "A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration“ Submitted to IEEE Transactions on Image processing, 2007.
M. Figueiredo, J. Bioucas-Dias, and R. Nowak, "Majorization-Minimization Algorithms for Wavelet-Based Image Deconvolution'', Submitted to IEEE Transactions on Image processing, 2006.
4IPIM, IST, José Bioucas, 2007
More References
M. Figueiredo and R. Nowak, “An EM algorithm for wavelet-based image restoration,” IEEE Trans. on Image Processing, vol. 12, no. 8, pp. 906–916, 2003.
J. Bioucas-Dias, “Bayesian wavelet-based image deconvolution: a GEM algorithm exploiting a class of heavy-tailed priors,” IEEE Trans. on Image Processing, vol. 15, pp. 937–951, 2006.
A. Chambolle, “An algorithm for total variation minimization andapplications,” Journal of Mathematical Imaging and Vision, vol. 20,pp. 89-97, 2004.
P. Combettes and V. Wajs, “Signal recovery by proximal forwardbackward splitting,” SIAM Journal on Multiscale Modeling & Simulation vol. 4, pp. 1168–1200, 2005
I. Daubechies, M. Defriese, and C. De Mol, “An iterative thresholding algorithm for linear inverse problems with a sparsity constraint”, Communications on Pure and Applied Mathematics, vol. LVII, pp. 1413-1457, 2004
5IPIM, IST, José Bioucas, 2007
Majorization Minorization (MM) Framework
Let
EM is an algorithm of this type.
Majorization Minorization algorithm:
....with equality if and only if
Easy to prove monotonicity:
Notes: should be easy to maximize
6IPIM, IST, José Bioucas, 2007
MM Algorithms for LIPs
IST Class: Majorize
IRS Class: Majorize
IST/IRS: Majorize and
7IPIM, IST, José Bioucas, 2007
MM Algorithms: IST class
Assume that
8IPIM, IST, José Bioucas, 2007
MM Algorithms: IST class
Majorizer:
Let:
IST Algorithm
9IPIM, IST, José Bioucas, 2007
MM Algorithms: IST class
Overrelaxed IST Algorithm
Convergence: [Combettes and V. Wajs, 2004]
• is convex •
• the set of minimizers, G, of is non-empty• 2 ]0,1]
• Then converges to a point in G
10IPIM, IST, José Bioucas, 2007
Denoising with convex regularizers
Denoising function also known as the Moreou proximal mapping
Classes of convex regularizers:
1- homogeneous (TV, lp-norm (p>1)) 2- p power of an lp norm
11IPIM, IST, José Bioucas, 2007
1-Homogeneous regularizers
Then
where is a closed convex set
and denotes the orthogonal projection on the convex set
12IPIM, IST, José Bioucas, 2007
Total variation regularization
Total variation [S. Osher, L. Rudin, and E. Fatemi, 1992]
is convex (although not strightly) and 1-homogeneous
Total variation is a discontinuity-preserving regularizer
have the same TV
13IPIM, IST, José Bioucas, 2007
Then
Total variation regularization
[Chambolle, 2004]
14IPIM, IST, José Bioucas, 2007
Total variation denoising
15IPIM, IST, José Bioucas, 2007
Total variation deconvolution
2000 IST iterations !!!
16IPIM, IST, José Bioucas, 2007
Weighted lp-norms
is convex (although not strightly) and 1-homogeneous
There is no closed form expression for excepts for some particular cases
Thus
17IPIM, IST, José Bioucas, 2007
Soft thresholding: p=1
Thus
18IPIM, IST, José Bioucas, 2007
Soft thresholding: p=1
19IPIM, IST, José Bioucas, 2007
Soft thresholding: p=1
20IPIM, IST, José Bioucas, 2007
Another way to look at it:
Since L is convex: the point is a global minimum of L iif
where is the subdifferential of L at f’
21IPIM, IST, José Bioucas, 2007
Example: Wavelet-based restoration
Wavelet basis
Wavelet coefficients Detail coefficients (h – high pass filter)
Approximation coefficients (g-low pass filter)
g,h – quadrature mirror filters
DWT, Harr, J=2
22IPIM, IST, José Bioucas, 2007
Example: Wavelet-based restoration
-500 -400 -300 -200 -100 0 100 200 300 400 5000
1
2
3
4
5
6
7
8
9
10
-150 -100 -50 0 50 100 1500
2000
4000
6000
8000
10000
12000
Histogram of coefficients - h Histogram of coefficients – log h