Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya...
-
Upload
rachel-phillips -
Category
Documents
-
view
226 -
download
2
Transcript of Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya...
![Page 1: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/1.jpg)
1
Weighted Joint Bilateral Filter with Slope Depth Compensation Filter
for Depth Map Refinement
Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi
VISAPP 2013 International Conference on Computer Vision Theory and
Application
![Page 2: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/2.jpg)
2
Outline
• Introduction• Related Works• Proposed Method–Weighted Joint Bilateral Filter – Slope Depth Compensation Filter
• Experimental Results• Conclusion
![Page 3: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/3.jpg)
3
INTRODUCTION
![Page 4: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/4.jpg)
4
Introduction
• Goal : Using two filters to get more accurate disparity map in real-time.
• Consideration– Noise reduction – Correct edges – Blurring control
Goal
![Page 5: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/5.jpg)
5
RELATED WORKS
![Page 6: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/6.jpg)
6
Left Image Right Image
Related Works
• Stereo Matching
![Page 7: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/7.jpg)
Related Works
Local Global
Estimate accuracy Low High
Calculation cost Low High
Methods Pixel matching Block matching
(Optimization methods) Graph cuts
Belief propagation
Example
![Page 8: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/8.jpg)
Related Works
• Flow Chart (Local)
1• Matching Cost Computation
2• Cost Aggregation
3• Disparity Map Computation/Optimization
4• Disparity Map Refinement• Disparity Map Refinement
![Page 9: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/9.jpg)
9
Related Works
• Depth map refinement with filter– Median filter– Bilateral filter
Input depth map Output depth map Filter
![Page 10: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/10.jpg)
10
Related Works
• Bilateral filter – Space weight:Near pixels has large weight – Color weight:Similar color pixels has large weight
• Smoothing – Keep edges –Weak in spike noise
•
![Page 11: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/11.jpg)
11
Related Works
• Joint bilateral filter – Add in the reference image – Color weight is calculated by the reference – Keep object edges of the reference
Reference : Low noise Target : High noise Filtered image
![Page 12: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/12.jpg)
12
Related Works
• Joint bilateral filter– Noise reduction O– Correct edge O– Blurring X • Mixed depth values • Spreading error regions
• Multilateral filter– Space + Color + Depth– Boundary recovering X
![Page 13: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/13.jpg)
13
PROPOSED METHOD
![Page 14: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/14.jpg)
14
Proposed Method
• Weighted joint bilateral filter – Noise reduction – Edge correction
• Slope depth compensation filter – Blurring control
![Page 15: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/15.jpg)
15
Weighted Joint Bilateral Filter
•
– 𝐷: Depth value – 𝑝: Coordinate of current pixel – 𝑠: Coordinate of support pixel – 𝑁: Aggregation set of support pixel – 𝑤(), (): Space/color weight 𝑐– 𝜎𝑠,𝜎𝑐: Space/color Gaussian distribution – 𝑅𝑠: Weight map
![Page 16: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/16.jpg)
16
Weighted Joint Bilateral Filter
• Add in the weight map – Controlling amount of influence on a pixel –Weight of the edge and error is small
Joint bilateral filter 𝜎- Mixed depth values 𝜎- Spreading error regions
![Page 17: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/17.jpg)
17
Weighted Joint Bilateral Filter
• Making weight map – Space/color/disparity weight – Sum of nearness of space,
color, and disparity between center pixel and surrounding pixels.
•
![Page 18: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/18.jpg)
18
Weighted Joint Bilateral Filter
• Mask image is made by Speckle Filter– Detecting speckle noise –Weight of speckle region is 0
Red region: speckle noise Weight = 0
![Page 19: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/19.jpg)
19
Weighted Joint Bilateral Filter
![Page 20: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/20.jpg)
20
Slope Depth Compensation Filter
• Weighted joint bilateral filter– Remaining small blurring – Difference between foreground and background
color is small
• Slope depth compensation filter – Reason of blurring is mixed depth value – Convert mixed value to non-blurred candidate
using initial depth map
Removing remaining blur
![Page 21: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/21.jpg)
21
Slope Depth Compensation Filter
•
– X in Dx {INITIAL;WJBF;SDCF}∈
![Page 22: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/22.jpg)
22
Slope Depth Compensation Filter
![Page 23: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/23.jpg)
23
Proposed Method
Initial disparityStereo matching
Noise reduction/ edge correction Weighted Joint Bilateral F.
Blurring control Slope Depth Compensention F.
![Page 24: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/24.jpg)
24
EXPERIMENTAL RESULTS
![Page 25: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/25.jpg)
25
Experimental Results
• Evaluating accuracy improvement for various types of depth maps – Block Matching (BM) – Semi-Global Matching (SGM) – Efficient Large-Scale (ELAS) – Dynamic Programing (DP) – Double Belief Propagation (DBP)
![Page 26: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/26.jpg)
26
Experimental Results
![Page 27: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/27.jpg)
27
Experimental Results
![Page 28: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/28.jpg)
28
![Page 29: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/29.jpg)
29
![Page 30: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/30.jpg)
30
![Page 31: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/31.jpg)
31
Experimental Results
• Comparing proposed method with cost volume refinement(Teddy).
Yang, Q., Wang, L., and Ahuja, N. A constantspace belief propagation algorithm for stereo matching.In Computer Vision and Pattern Recognition(2010).
32 times slower
![Page 32: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/32.jpg)
32
Experimental Results
![Page 33: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/33.jpg)
33
Experimental Results
• Device : Intel Core i7-920 2.93GHz• Comparing running time (ms) of BM plus proposed
filter with selected stereo methods.
![Page 34: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/34.jpg)
34
Experimental Results
![Page 35: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/35.jpg)
35
Experimental Results
• Use the proposed filter for depth maps from Microsoft Kinect.
![Page 36: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/36.jpg)
36
![Page 37: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/37.jpg)
37
CONCLUSION
![Page 38: Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP.](https://reader035.fdocuments.us/reader035/viewer/2022081519/56649e995503460f94b9c5d0/html5/thumbnails/38.jpg)
38
Conclusion
Contribution• The proposed methods can reduce depth noise and
correct object boundary edge without blurring. • Amount of improvement is large when an input
depth map is not accurate.
Future Works• Investigating dependencies of input natural images
and depth maps.