Image Denoising using Locally Learned Dictionaries
Priyam ChatterjeePeyman Milanfar
Dept. of Electrical EngineeringUniversity of California, Santa Cruz
Computational Imaging VII – 20 Jan, 2009
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 2
Overview
Data Model
Kernel Regression for Denoising
Denoising with Locally Learned Dictionaries (K-LLD)
Results
Conclusions
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 3
Data Model
Pointwise data model
Patchwise model
Locally smooth functionto be estimated
Zero-mean I.I.D. noise
Observation
denoted as
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 4
Steering Kernel Regression (SKR)
Optimization problem
Solution: Nonlinear filters Polynomial basis
Data-dependent
weights
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 5
SKR Weights
Weights based on pixel “self-similarity” in a local patch
Covariance matrix takes into account: orientation and strength of edges
Gradient Covariance
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 7
SKR Weights
Note how the weights adapt to the underlying image structure
Noisy Noise-free
H. Takeda, S. Farsiu, and P. Milanfar, “Kernel Regression for Image Processing and Reconstruction”, IEEE Trans. on Image Processing, vol. 16, no. 2, pp. 349-366, February 2007.
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 8
Now we extend it …..
is fixed order, everywhere -- not depending on underlying image structure
• Lower orders fit flat regions, higher order for texture and fine details
Global dictionary does not adapt to local image characteristics
• Dictionary atoms should capture underlying local image structure
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 9
Denoising with Locally Learned Dictionaries (K-LLD)
Identify dictionary which best captures underlying geometric structure
Similar structures will have similar dictionary, similar weights
Cluster image based on geometric similarity (K-Means on the SKR weights)
Learn dictionary and order of regression for each cluster
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 10
K-LLD: Algorithm Outline
Calculate
weights
Learn
dictionaries
Clustering
Iter
ate
Noisy Image
Kernel Regression
Denoised Image
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 11
Class 1Class 1
Class KClass KClustering StageClustering Stage
K-LLD : Algorithm Outline
Dictionary Selection StageDictionary Selection Stage
Noisy ImgCalculate Steering
Weights
Calculate SteeringWeights
Coefficient Calculation Stage
Coefficient Calculation StageDenoised
Img
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 12
SegmentImage
SegmentImage
Clustering Stage
K-MeansK-MeansClass 1Class 1
Class KClass K
K-LLD : Algorithm Outline
Dictionary Selection StageDictionary Selection Stage
Noisy ImgCalculate Steering
Weights
Calculate SteeringWeights
Coefficient Calculation Stage
Coefficient Calculation StageDenoised
Img
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 13
Clustering Stage
Objective : Cluster image based on geometric similarity of underlying data
• Feature Selection
• What features capture data geometry ?
• Distance Metric
• What metric captures distance between features ?
• Clustering Algorithm
• What algorithm segments the image best ?
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 14
K-Means for Clustering
Features : normalized steering wts
Distance Metric : L2
Initialization : Randomly initialize cluster centers
Run K-Means multiple times and select result that minimizes within-cluster distance
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 15
Noi
se-f
ree
Noi
sy
Clustering the noise-free image
Clustering the noisy image
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 16
SegmentImage
SegmentImage
Clustering Stage
K-MeansK-MeansClass 1Class 1
Class KClass K
K-LLD : Algorithm Outline
Dictionary Selection StageDictionary Selection Stage
Noisy ImgCalculate Steering
Weights
Calculate SteeringWeights
Coefficient Calculation Stage
Coefficient Calculation StageDenoised
ImgDictionary Selection Stage
PCAPCAFormDictionary
FormDictionary
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 17
Dictionary Selection
Represent each patch in the kth cluster
Variable Proj.
Solved by PCA
Mean patch of k-th cluster
Enforceorthonormality
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 18
Dictionary Selection
PCA in each cluster to form a dictionary
Describe data without fitting noise
Number of atoms based on cluster geometry • Clusters with flat regions need fewer atoms, finer
details need more
constant
Singular values
patch sizeNo. of atoms
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 19
Example : House Image
Dictionary atoms for AWGN of std. dev. 15
Clu
ster
Ato
m 1
Ato
m 2
Ato
m 3
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 21
Example : Noise-free clusteringC
lust
erA
tom
1A
tom
2A
tom
3A
tom
4
Few of the atoms in the dictionaries for different clusters
Brick facade
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 23
Algorithm Outline
SegmentImage
SegmentImage
Clustering Stage
Class 1Class 1
Class KClass KK-MeansK-Means
Noisy ImgCalculate Steering
Weights
Calculate SteeringWeights
Coefficient Calculation Stage
Coefficient Calculation Stage
Denoised Img
Dictionary Selection Stage
PCAPCAFormDictionary
FormDictionaryKernel RegressionKernel Regression
Denoised Img
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 24
Kernel Regression
Weighted least squares solution
Final estimate
Coefficient Calculation
center pixel of patch .
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 25
Kernel RegressionKernel RegressionDenoised
Img
Algorithm Outline
SegmentImage
SegmentImage
Clustering Stage
Class 1Class 1
Class KClass KK-MeansK-Means
Noisy ImgCalculate Steering
Weights
Calculate SteeringWeights
Dictionary Selection Stage
PCAPCAFormDictionary
FormDictionary
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 26
Iteration
Re-learn weights (features) from denoised image
Perform clustering of updated image using new features
Learn dictionary from updated image
Kernel regression on input noisy image• Preserves edges and finer structures
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 28
K-LLD: Algorithm Outline
Calculate
weights
Learn
dictionaries
Clustering
Iter
ate
Noisy Image
Kernel Regression
Denoised Image
Original Noisy Image
Results – AWG noise (std dev 25)
K-LLD, MSE 96.95 SSIM 0.825
BM3D, MSE 88.82 SSIM 0.841Original Parrot Image
K-SVD, MSE 101.54 SSIM 0.826
SKR, MSE 99.96 SSIM 0.826
Noisy Image
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 31
More ResultsM
SE
SS
IM
Results – Real noise & colorISKR K-LLDBM3D
Color Results
ISKR, Order 2
BM3DOriginal Image
K-LLD
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 35
Thank you
P. Chatterjee and P. Milanfar, “Clustering-based Denoising with Locally LearnedDictionaries”, Accepted for publication in IEEE Trans. Image Processing
Available at:
http://www.ee.ucsc.edu/~milanfar
20 Jan, 2009 Image Denoising using Locally Learned Dictionaries 36
Iterative Scheme
Why iterate ?
• Weights true to underlying structure in presence of lesser noise
• Better weights means better clustering
• Dictionary captures underlying data better when learned on less noisy image
Top Related