Improved Census Transforms for Resource-Optimized Stereo Vision
description
Transcript of Improved Census Transforms for Resource-Optimized Stereo Vision
![Page 1: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/1.jpg)
Improved Census Transforms for Resource-Optimized Stereo Vision
Wade S. Fife, Member, IEEE, James K. Archibald, Senior Member, IEEE
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 23, NO. 1, JANUARY 2013
![Page 2: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/2.jpg)
2
Outline• Introduction• Related Work• Proposed Algorithm
• Sparse Census Transform• Generalized Census Transform• Hardware Implementation
• Experimental Results• Conclusion
![Page 3: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/3.jpg)
3
Introduction
![Page 4: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/4.jpg)
4
Introduction• The challenges:
• The enormous amount of computation required to identify the corresponding points in the images.
• It is critical to…
• maximize the accuracy and throughput of the stereo system • while minimizing the resource requirements
![Page 5: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/5.jpg)
5
Objective• Propose the sparse census transforms :
• Reduce the resource requirements of census-based systems• Maintain correlation accuracy
• Propose the generalized census transforms :
• A new class of census-like transforms • Increase the robustness and flexibility
![Page 6: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/6.jpg)
6
Related Work
![Page 7: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/7.jpg)
7
Related Work• Census Transform :
• Color• Gradient
![Page 8: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/8.jpg)
8
Related Work• After aggregation step:
Census on colors Census on gradients
![Page 9: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/9.jpg)
9
Related Work• Sparse census[6] :
• Half of the bits
X
[6] C. Zinner, M. Humenberger, K. Ambrosch, and W. Kubinger, “An optimized software-based implementation of a census-based stereo matching algorithm,” in Proc. 4th ISVC, 2008, pp. 216–227.
The computation costs for the hamming distances are quite large.
![Page 10: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/10.jpg)
10
Related Work• Mini-census[8] :
X
[8] N.-C. Chang, T.-H. Tsai, B.-H. Hsu, Y.-C. Chen, and T.-S. Chang,“Algorithm and architecture of disparity estimation with mini-census adaptive support weight,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 6, pp. 792–805, Jun. 2010.
![Page 11: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/11.jpg)
11
Related Work• Mini-census[8] :
• Mini-census adaptive support weight
[8] N.-C. Chang, T.-H. Tsai, B.-H. Hsu, Y.-C. Chen, and T.-S. Chang,“Algorithm and architecture of disparity estimation with mini-census adaptive support weight,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 6, pp. 792–805, Jun. 2010.
![Page 12: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/12.jpg)
12
Related Work• Mini-census[8] :
![Page 13: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/13.jpg)
13
ProposedAlgorithm
![Page 14: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/14.jpg)
14
Sparse Census Transform• Definition :
• N: the set of points within a T T window around p• : a new set of N•
P’
P
![Page 15: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/15.jpg)
15
Transform Point Selection• Goal : minimize the size of the census transform vector
• Challenge: Must quantify how much each point in the transform window contributes to overall correlation accuracy
• Test correlation accuracy:
• Define a sparse census transform consisting of a single point (| | = 1)• Determine how consistently this point leads to correct correlation• 13 13 correlation window (aggregation)
![Page 16: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/16.jpg)
16
Transform Point Selection• Go
Tsukuba Venus Average
Teddy Cones
Bright: Higher correlation accuracy
25 25 neighborhood
![Page 17: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/17.jpg)
![Page 18: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/18.jpg)
18
Transform Point Selection• Further from the center : value decreasing
• Very near the center : less effective
• It is best to choose points that are neither too far from nor too close to the center pixel.
• Optimal distance : 2 pixels• If the image is noisy should be slightly further
from the center
![Page 19: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/19.jpg)
19
Transform Point Selection•
Tsukuba Venus Average
Teddy Cones
Bright: Higher correlation accuracy
37 37 neighborhood
Tsukuba Venus
Teddy Cones
With Gaussian noise( = 5.12)
![Page 20: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/20.jpg)
20
Proposed Sparse Census Transform• Very good correlation accuracy can be achieved using very sparse transforms.
16-point 12-point 8-point
4-point 2-point 1-point
![Page 21: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/21.jpg)
21
Experimental Results
![Page 22: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/22.jpg)
22
Generalized Census Transform• Goal : greater freedom in choosing the census transform design
• Definition : redrawing the transform as a graph
3 3 census
3 3 correlation(aggregation)
![Page 23: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/23.jpg)
23
Generalized Census Transform• As..
• (1)transform neighborhoods become more and more sparse• (2)fewer pixels are used in the correlation process
• selection of points to include in the transform becomes more critical
2-point 2-edge
Horizontal + Vertical
![Page 24: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/24.jpg)
24
Generalized Census Transform
symmetric
![Page 25: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/25.jpg)
25
Proposed Generalized Census Transform• Benefits :
• Often require a smaller census transform window (memory)• Increased robustness under varying conditions (noise)
16-edge 12-edge 8-edge
4-edge 2-edge 1-edge
![Page 26: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/26.jpg)
26
Experimental Results
![Page 27: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/27.jpg)
27
Experimental Results
![Page 28: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/28.jpg)
28
Hardware Implementation• Pipelining : to increase throughput in an FPGA implementation
(Field Programmable Gate Array)
Range : 0~3
3 2 1 0
3 2 1 0
3 2 1 0
3 2 1 0
3 2 1 0
One input pixel per clock cycle &Output one disparity result per clock cycle
![Page 29: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/29.jpg)
29
Hardware Implementation• Correlation window sum (Aggregation) :
![Page 30: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/30.jpg)
30
ExperimentalResults
![Page 31: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/31.jpg)
12-edge 4-edgeFull 7x7 censusGround TruthLeft Image
![Page 32: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/32.jpg)
12-edge 4-edgeFull 7x7 censusGround TruthLeft Image
![Page 33: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/33.jpg)
12-edge 4-edgeFull 7x7 censusLeft Image
![Page 34: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/34.jpg)
34
Experimental Results
LUTs (look-up tables) : the amount of logic required to implement the methodFFs : the number of 1-bit registers (the amount of pipelining used)RAMs : the number of 18-kbit block memoriesFreq. : the maximum operating frequency reported by synthesis
𝟖𝟖% ↓ 𝟔𝟏% ↓
![Page 35: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/35.jpg)
35
Conclusion
![Page 36: Improved Census Transforms for Resource-Optimized Stereo Vision](https://reader035.fdocuments.us/reader035/viewer/2022062218/56816334550346895dd3bb51/html5/thumbnails/36.jpg)
36
Conclusion• Proposed and analyzed in this paper:
• A range of sparse census transforms
• reduce hardware resource requirements• attempting to maximize correlation accuracy.• often better than or nearly as good as the full census
• Generalized census transforms
• increased robustness in the presence of image noise