Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of...
-
Upload
buck-francis -
Category
Documents
-
view
219 -
download
1
Transcript of Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of...
![Page 1: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/1.jpg)
Image Denoising using Locally Learned Dictionaries
Priyam ChatterjeePeyman Milanfar
Dept. of Electrical EngineeringUniversity of California, Santa Cruz
Computational Imaging VII – 20 Jan, 2009
![Page 2: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/2.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 2
Overview
Data Model
Kernel Regression for Denoising
Denoising with Locally Learned Dictionaries (K-LLD)
Results
Conclusions
![Page 3: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/3.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 3
Data Model
Pointwise data model
Patchwise model
Locally smooth functionto be estimated
Zero-mean I.I.D. noise
Observation
denoted as
![Page 4: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/4.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 4
Steering Kernel Regression (SKR)
Optimization problem
Solution: Nonlinear filters Polynomial basis
Data-dependent
weights
![Page 5: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/5.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 5
SKR Weights
Weights based on pixel “self-similarity” in a local patch
Covariance matrix takes into account: orientation and strength of edges
Gradient Covariance
![Page 6: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/6.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 7
SKR Weights
Note how the weights adapt to the underlying image structure
Noisy Noise-free
H. Takeda, S. Farsiu, and P. Milanfar, “Kernel Regression for Image Processing and Reconstruction”, IEEE Trans. on Image Processing, vol. 16, no. 2, pp. 349-366, February 2007.
![Page 7: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/7.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 8
Now we extend it …..
is fixed order, everywhere -- not depending on underlying image structure
• Lower orders fit flat regions, higher order for texture and fine details
Global dictionary does not adapt to local image characteristics
• Dictionary atoms should capture underlying local image structure
![Page 8: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/8.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 9
Denoising with Locally Learned Dictionaries (K-LLD)
Identify dictionary which best captures underlying geometric structure
Similar structures will have similar dictionary, similar weights
Cluster image based on geometric similarity (K-Means on the SKR weights)
Learn dictionary and order of regression for each cluster
![Page 9: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/9.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 10
K-LLD: Algorithm Outline
Calculate
weights
Learn
dictionaries
Clustering
Iter
ate
Noisy Image
Kernel Regression
Denoised Image
![Page 10: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/10.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 11
Class 1Class 1
Class KClass KClustering StageClustering Stage
K-LLD : Algorithm Outline
Dictionary Selection StageDictionary Selection Stage
Noisy ImgCalculate Steering
Weights
Calculate SteeringWeights
Coefficient Calculation Stage
Coefficient Calculation StageDenoised
Img
![Page 11: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/11.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 12
SegmentImage
SegmentImage
Clustering Stage
K-MeansK-MeansClass 1Class 1
Class KClass K
K-LLD : Algorithm Outline
Dictionary Selection StageDictionary Selection Stage
Noisy ImgCalculate Steering
Weights
Calculate SteeringWeights
Coefficient Calculation Stage
Coefficient Calculation StageDenoised
Img
![Page 12: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/12.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 13
Clustering Stage
Objective : Cluster image based on geometric similarity of underlying data
• Feature Selection
• What features capture data geometry ?
• Distance Metric
• What metric captures distance between features ?
• Clustering Algorithm
• What algorithm segments the image best ?
![Page 13: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/13.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 14
K-Means for Clustering
Features : normalized steering wts
Distance Metric : L2
Initialization : Randomly initialize cluster centers
Run K-Means multiple times and select result that minimizes within-cluster distance
![Page 14: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/14.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 15
Noi
se-f
ree
Noi
sy
Clustering the noise-free image
Clustering the noisy image
![Page 15: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/15.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 16
SegmentImage
SegmentImage
Clustering Stage
K-MeansK-MeansClass 1Class 1
Class KClass K
K-LLD : Algorithm Outline
Dictionary Selection StageDictionary Selection Stage
Noisy ImgCalculate Steering
Weights
Calculate SteeringWeights
Coefficient Calculation Stage
Coefficient Calculation StageDenoised
ImgDictionary Selection Stage
PCAPCAFormDictionary
FormDictionary
![Page 16: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/16.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 17
Dictionary Selection
Represent each patch in the kth cluster
Variable Proj.
Solved by PCA
Mean patch of k-th cluster
Enforceorthonormality
![Page 17: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/17.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 18
Dictionary Selection
PCA in each cluster to form a dictionary
Describe data without fitting noise
Number of atoms based on cluster geometry • Clusters with flat regions need fewer atoms, finer
details need more
constant
Singular values
patch sizeNo. of atoms
![Page 18: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/18.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 19
Example : House Image
Dictionary atoms for AWGN of std. dev. 15
Clu
ster
Ato
m 1
Ato
m 2
Ato
m 3
![Page 19: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/19.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 21
Example : Noise-free clusteringC
lust
erA
tom
1A
tom
2A
tom
3A
tom
4
Few of the atoms in the dictionaries for different clusters
Brick facade
![Page 20: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/20.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 23
Algorithm Outline
SegmentImage
SegmentImage
Clustering Stage
Class 1Class 1
Class KClass KK-MeansK-Means
Noisy ImgCalculate Steering
Weights
Calculate SteeringWeights
Coefficient Calculation Stage
Coefficient Calculation Stage
Denoised Img
Dictionary Selection Stage
PCAPCAFormDictionary
FormDictionaryKernel RegressionKernel Regression
Denoised Img
![Page 21: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/21.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 24
Kernel Regression
Weighted least squares solution
Final estimate
Coefficient Calculation
center pixel of patch .
![Page 22: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/22.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 25
Kernel RegressionKernel RegressionDenoised
Img
Algorithm Outline
SegmentImage
SegmentImage
Clustering Stage
Class 1Class 1
Class KClass KK-MeansK-Means
Noisy ImgCalculate Steering
Weights
Calculate SteeringWeights
Dictionary Selection Stage
PCAPCAFormDictionary
FormDictionary
![Page 23: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/23.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 26
Iteration
Re-learn weights (features) from denoised image
Perform clustering of updated image using new features
Learn dictionary from updated image
Kernel regression on input noisy image• Preserves edges and finer structures
![Page 24: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/24.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 28
K-LLD: Algorithm Outline
Calculate
weights
Learn
dictionaries
Clustering
Iter
ate
Noisy Image
Kernel Regression
Denoised Image
Original Noisy Image
![Page 25: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/25.jpg)
Results – AWG noise (std dev 25)
K-LLD, MSE 96.95 SSIM 0.825
BM3D, MSE 88.82 SSIM 0.841Original Parrot Image
K-SVD, MSE 101.54 SSIM 0.826
SKR, MSE 99.96 SSIM 0.826
Noisy Image
![Page 26: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/26.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 31
More ResultsM
SE
SS
IM
![Page 27: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/27.jpg)
Results – Real noise & colorISKR K-LLDBM3D
![Page 28: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/28.jpg)
Color Results
ISKR, Order 2
BM3DOriginal Image
K-LLD
![Page 29: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/29.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 35
Thank you
P. Chatterjee and P. Milanfar, “Clustering-based Denoising with Locally LearnedDictionaries”, Accepted for publication in IEEE Trans. Image Processing
Available at:
http://www.ee.ucsc.edu/~milanfar
![Page 30: Image Denoising using Locally Learned Dictionaries Priyam Chatterjee Peyman Milanfar Dept. of Electrical Engineering University of California, Santa Cruz.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649cfa5503460f949cc03d/html5/thumbnails/30.jpg)
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 36
Iterative Scheme
Why iterate ?
• Weights true to underlying structure in presence of lesser noise
• Better weights means better clustering
• Dictionary captures underlying data better when learned on less noisy image