Normalized averaging using adaptive applicability functions with applications in image...
-
Upload
tuan-q-pham -
Category
Technology
-
view
799 -
download
1
description
Transcript of Normalized averaging using adaptive applicability functions with applications in image...
![Page 1: Normalized averaging using adaptive applicability functions with applications in image reconstruction from sparsely and randomly sampled data](https://reader035.fdocuments.us/reader035/viewer/2022081401/5563bef2d8b42ad83c8b517a/html5/thumbnails/1.jpg)
April 12, 2023
1
Pattern Recognition Group
Normalized averaging
using adaptive applicability
functions
Presented at SCIA 2003
Tuan Q. Pham and Lucas J. van Vliet
with applications in image reconstruction
from sparsely and randomly sampled data
![Page 2: Normalized averaging using adaptive applicability functions with applications in image reconstruction from sparsely and randomly sampled data](https://reader035.fdocuments.us/reader035/viewer/2022081401/5563bef2d8b42ad83c8b517a/html5/thumbnails/2.jpg)
April 12, 2023 2
Overview
• Normalized averaging
• Local structure adaptive filtering
• Experimental results
• Comparison with diffusion-based image inpainting
• Directions for further research
![Page 3: Normalized averaging using adaptive applicability functions with applications in image reconstruction from sparsely and randomly sampled data](https://reader035.fdocuments.us/reader035/viewer/2022081401/5563bef2d8b42ad83c8b517a/html5/thumbnails/3.jpg)
April 12, 2023 3
Normalized averaging (Knutsson’93)•Weighted average filtering:
•Normalized averaging = weighted average + signal/certainty principle:
•each signal s is associated with a certainty c•s & c have to be processed separately
where s :signal, c :certainty, a :filter, r :result, * :convolution
( . )s c ar
c a
NA with Gaussian applicability (σ =
1)
input with 10% original pixels
Gaussian smoothing (σ = 1)
*r s a
![Page 4: Normalized averaging using adaptive applicability functions with applications in image reconstruction from sparsely and randomly sampled data](https://reader035.fdocuments.us/reader035/viewer/2022081401/5563bef2d8b42ad83c8b517a/html5/thumbnails/4.jpg)
April 12, 2023 4
Normalized averaging: An exampleReconstruction from 10% random pixels
Nearest neighbor interpolation NA with adaptive applicability
![Page 5: Normalized averaging using adaptive applicability functions with applications in image reconstruction from sparsely and randomly sampled data](https://reader035.fdocuments.us/reader035/viewer/2022081401/5563bef2d8b42ad83c8b517a/html5/thumbnails/5.jpg)
April 12, 2023 5
Image reconstruction using Adaptive Normalized Averaging
Input Image (sparsely & randomly sampled)
Normalized
Averaging
Structure Analysis
Output Image (with local structure extended into missing regions)
σ = 1
Adaptive applicability
![Page 6: Normalized averaging using adaptive applicability functions with applications in image reconstruction from sparsely and randomly sampled data](https://reader035.fdocuments.us/reader035/viewer/2022081401/5563bef2d8b42ad83c8b517a/html5/thumbnails/6.jpg)
April 12, 2023 6
Local structure adaptive filtering
•Local structure from the structure tensor
•orientation φ = arg( )•anisotropy A = (λu - λv)/(λu + λv) •curvature κ = ∂φ /∂•scale rdensity = sample density
•Scale-adaptive curvature-bent anisotropic Gaussian kernel with scales in 2 orthogonal directions:
kernel aligns withlocal structure
T uu vvT T Tu vI I
u
v
212
y x
u v
(1 )u densityC A r (1 )v densityC A r
where C ~ SNR ~ degree of structure enhancement
![Page 7: Normalized averaging using adaptive applicability functions with applications in image reconstruction from sparsely and randomly sampled data](https://reader035.fdocuments.us/reader035/viewer/2022081401/5563bef2d8b42ad83c8b517a/html5/thumbnails/7.jpg)
April 12, 2023 7
Sample Density Transform
•Definition: Smallest radius of a pillbox, centered at each pixel, that encompasses total certainty of at least 1
•Role: Automatic scale selection of the applicability in the NA equation to avoid unnecessary smoothing
Adap. Norm. Avg.Lena with missing hole Density transform NA with Gaussian(σ=1)
![Page 8: Normalized averaging using adaptive applicability functions with applications in image reconstruction from sparsely and randomly sampled data](https://reader035.fdocuments.us/reader035/viewer/2022081401/5563bef2d8b42ad83c8b517a/html5/thumbnails/8.jpg)
April 12, 2023 8
4x4 super-resolution from 4 noisy frames• 4 input LowRes captured with fill-factor = 25%, intensity noise
(σ=10), registration noise (σ=0.2 LR pitch)
1 of 4 input 64x64 LR SR using triangulation SR using adaptive NA
• 16 times upsampling from only 4 frames. How is it possible: along linear structures, only 4 samples are enough for 4x super-resolution
![Page 9: Normalized averaging using adaptive applicability functions with applications in image reconstruction from sparsely and randomly sampled data](https://reader035.fdocuments.us/reader035/viewer/2022081401/5563bef2d8b42ad83c8b517a/html5/thumbnails/9.jpg)
April 12, 2023 9
Scale in perpendicular directionScale along linear structuresSample density
Orientation Anisotropy Curvature
![Page 10: Normalized averaging using adaptive applicability functions with applications in image reconstruction from sparsely and randomly sampled data](https://reader035.fdocuments.us/reader035/viewer/2022081401/5563bef2d8b42ad83c8b517a/html5/thumbnails/10.jpg)
April 12, 2023 10
Comparison with image inpainting• Image inpainting (Sapiro) = diffusion with level line evolution
• also extending orientation into the missing regions
• slow due to iterative nature
• poor result for large holes
input inpainting 110 iters (6
min)
inpainting + texture
synthesis
Adapt. Norm. Avg.
0 iters (6 sec)
![Page 11: Normalized averaging using adaptive applicability functions with applications in image reconstruction from sparsely and randomly sampled data](https://reader035.fdocuments.us/reader035/viewer/2022081401/5563bef2d8b42ad83c8b517a/html5/thumbnails/11.jpg)
April 12, 2023 11
Directions for Further Research
• Applications• Image filtering (noise/watermark removal, edge
enhancement...)• Image interpolation from sparsely and randomly sampled
data (image inpainting, image fusion, super-resolution...)
• Further improvements• Scale-space local structure analysis.• Detect multiple orientations using orientation space. • Robust neighborhood operator than the weighted mean.
![Page 12: Normalized averaging using adaptive applicability functions with applications in image reconstruction from sparsely and randomly sampled data](https://reader035.fdocuments.us/reader035/viewer/2022081401/5563bef2d8b42ad83c8b517a/html5/thumbnails/12.jpg)
April 12, 2023 12
Image inpainting of thin scribbles
Adaptive Normalized Averaging (10 sec)
input
inpainting
![Page 13: Normalized averaging using adaptive applicability functions with applications in image reconstruction from sparsely and randomly sampled data](https://reader035.fdocuments.us/reader035/viewer/2022081401/5563bef2d8b42ad83c8b517a/html5/thumbnails/13.jpg)
April 12, 2023 13
Simultaneous geometry/texture inpainting
geometry
Adaptive NA
(1 min)
textureinput
![Page 14: Normalized averaging using adaptive applicability functions with applications in image reconstruction from sparsely and randomly sampled data](https://reader035.fdocuments.us/reader035/viewer/2022081401/5563bef2d8b42ad83c8b517a/html5/thumbnails/14.jpg)
April 12, 2023 14
Inpainting of ambiguous discontinuity
inpainting Adaptive NA (1 sec)
original input