고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450...
Transcript of 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450...
![Page 1: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/1.jpg)
SEOUL | Oct.7, 2016
Gwang-Soo Hong, Sookmyung women’s university
고속 스테레오 매칭 알고리즘 및 GPU 기반 병렬처리 기법 연구
![Page 2: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/2.jpg)
2
CONTENTS
Stereo vision
Conventional stereo matching
Fast stereo matching
Evaluation
![Page 3: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/3.jpg)
3
STEREO VISON
![Page 4: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/4.jpg)
4
STEREO VISION Vision-Based Driver Assistance
http://ae-plus.com/technology/autoliv-develops-stereo-camera-system
Pedestrian detection
Vehicle detection
Lane detection
Stereo camera
![Page 5: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/5.jpg)
5
STEREO VISION Computer stereo vision
Stereo vision is the extraction of 3D information from images. 3D information can be extracted by examination of the relative positions of objects in the two image planes.
Left image Right image
Disparity
![Page 6: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/6.jpg)
6
STEREO VISION Relationship between disparity and depth
Triangulation
10/11/2016
𝑍: depth 𝑓: focal length 𝑂𝑅 , 𝑂𝑇: center positions of lens 𝐵: baseline 𝑃: object
𝑍: 𝐵 = 𝑍 − 𝑓: 𝑥
disparity
![Page 7: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/7.jpg)
7
STEREO VISION Overview of a stereo vision system
10/11/2016
Stereo matching
Calibration Rectification
Triangulation
Stereo pair
Rectified stereo pair
Disparity map
Depth map
offline
online
![Page 8: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/8.jpg)
8
CONVENTIONAL STEREO MATCHING
![Page 9: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/9.jpg)
9
STEREO MATCHING Taxonomy for stereo algorithms [1]
1) Matching cost computation
2) Cost aggregation
3) Disparity computation / optimization (local and global algorithm)
4) Disparity refinement
10/11/2016
[1] D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms” Int. Jour.
Computer Vision, 47(1/2/3):7–42, 2002
![Page 10: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/10.jpg)
10
STEREO MATCHING Matching cost computation
Basic Idea
Start at pixel 𝑝, consider its neighborhood defined by a square window
Compare with neighborhoods around pixels on the epipolar line for best match of pixel neighborhoods 10/11/2016
𝑑𝑚𝑎𝑥 𝑑𝑚𝑖𝑛
Left image Right image
𝑑𝑚𝑎𝑥
𝑑𝑚𝑖𝑛
Initial matching cost
𝑝
![Page 11: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/11.jpg)
11
STEREO MATCHING Cost aggregation
Basic Idea
Color differences and a variation exist in the depth discontinuous
The variation in the disparity value is small between adjacent pixels
10/11/2016
Initial matching cost Aggregated matching cost
![Page 12: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/12.jpg)
12
STEREO MATCHING Disparity computation and refinement
Disparity computation
Disparity Refinement
10/11/2016
Disparities
Matc
hin
g c
ost
Select minimum matching cost
Winner-take-all [1]
• Left-Right Consistency Check
• Median filtering
![Page 13: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/13.jpg)
13
FAST STEREO MATCHING
![Page 14: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/14.jpg)
14
FAST STEREO MATCHING Cost volume filtering approach for stereo matching
Stereo pair Initial matching cost Matching cost filtering Disparity computation
![Page 15: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/15.jpg)
15
FAST STEREO MATCHING Matching cost computation
10/11/2016
𝑑𝑚𝑎𝑥
𝑑𝑚𝑖𝑛
Initial matching cost
𝑑𝑚𝑎𝑥 𝑑𝑚𝑖𝑛
Left image Right image
𝑝
𝑝
![Page 16: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/16.jpg)
16
FAST STEREO MATCHING Cost aggregation
10/11/2016
𝑑𝑚𝑎𝑥
𝑑𝑚𝑖𝑛
𝑑𝑚𝑎𝑥
𝑑𝑚𝑖𝑛
Initial matching cost (C) Aggregated matching cost (C’)
Filtering
Guided image filtering
Filter input C
Guide G
Filtering output C’
𝐶′ = 𝐶𝑖 − 𝑛𝑖
𝐶′ = 𝑎𝐺𝑖 + 𝑏
Guide G
![Page 17: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/17.jpg)
17
FAST STEREO MATCHING Cost aggregation
10/11/2016
Weighted guided image filtering
Filter input C
Guide G
Filtering output C’
𝐶′ = 𝐶𝑖 − 𝑛𝑖
𝐶′ = 𝑎𝐺𝑖 + 𝑏
𝐸 = ( 𝑎𝑘 ⋅ 𝐺 𝑝 + 𝑏𝑘 − 𝐶 𝑝2 + 𝜆 ⋅ 𝑎𝑘
2)
𝑝∈𝜔𝑘
𝐸 = ( 𝑎𝑘 ⋅ 𝐺 𝑝 + 𝑏𝑘 − 𝐶 𝑝2 +
𝜆
𝑊(𝐺, 𝑝)⋅ 𝑎𝑘2)
𝑝∈𝜔𝑘
Guided image filtering
![Page 18: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/18.jpg)
18
FAST STEREO MATCHING Cost aggregation
10/11/2016
𝐸 = ( 𝑎𝑘 ⋅ 𝐺 𝑝 + 𝑏𝑘 − 𝐶 𝑝2 +
𝜆
𝑊(𝐺, 𝑝)⋅ 𝑎𝑘2)
𝑝∈𝜔𝑘
Linear regression
![Page 19: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/19.jpg)
19
FAST STEREO MATCHING Cost aggregation
10/11/2016
The weights in tsukuba image
![Page 20: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/20.jpg)
20
FAST STEREO MATCHING Cost aggregation
10/11/2016
Integral image technique
![Page 21: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/21.jpg)
21
EXPERIMENTAL RESULTS
![Page 22: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/22.jpg)
22
COMPARATIVE EVALUATIONS OF STEREO MATCHERS
Test data and Performance of stereo matching
KITTI: http://www.cvlibs.net/datasets/kitti/index.php
Middlebury Stereo Vision: http://vision.middlebury.edu/stereo/
10/11/2016
![Page 23: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/23.jpg)
23
EXPERIMENTAL RESULTS Quantitative Middlebury evaluation
10/11/2016
TSUKUBA VENUS CONES TEDDY AVERAGE PERCENT OF
BAD PIXELS NON-OCC
ALL DISC NON-OCC
ALL DISC NON-OCC
ALL DISC NON-OCC
ALL DISC
Prop. Algorightm 1.37 1.79 6.55 0.21 0.48 2.40 5.98 11.2 15.5 2.29 8.51 7.69 5.33
Inforpermeable[2] 1.06 1.53 5.64 0.32 0.88 4.15 5.60 13.0 14.5 2.65 9.16 7.69 5.51
CostAggr[3] 1.38 1.85 6.90 0.71 1.19 6.13 7.88 13.3 18.6 3.97 9.79 8.26 5.54
GeoSup[4] 1.45 1.83 7.71 0.14 0.26 1.90 6.88 13.2 16.1 2.94 8.89 8.32 5.80
AdaptWeight[5] 1.37 1.79 6.55 0.21 0.48 2.40 5.98 11.2 15.5 2.29 8.51 7.69 6.66
[2] C. Ciglaa and A. A. Alatanb, “Information permeability for stereo matching," Signal Processing: Image Communication, vol. 28, no. 9, pp. 1072-1088,October, 2013.
[3] C. Rhemann, A. Hosni, M. Bleyer, C. Rother and M. Gelautz, “Fast cost-volume filtering for visual correspondence and beyond," IEEE Conference on Computer Vision and Pattern
Recognition, pp. 3017-3024, June, 2011.
[5] K. 300 J. Yoon and I. s. Kwoen, “Adaptive support-weight approach for correspondence search," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 4,
pp. 650-656, April, 2006.
[4] A. Hosni, M. Bleyer, M. Gelautz and C. Rhemann, “Local stereo matching using geodesic support weights," IEEE 16th International Conference on Image Processing, pp. 2093-
2096, November, 2009.
![Page 24: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/24.jpg)
24
EXPERIMENTAL RESULTS Qualitative Middlebury evaluation
10/11/2016
![Page 25: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/25.jpg)
25
EXPERIMENTAL RESULTS Qualitative KITTI evaluation
10/11/2016
![Page 26: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/26.jpg)
26
EXPERIMENTAL RESULTS Computational complexity
3.2
0.16 0.11
0
0.5
1
1.5
2
2.5
3
3.5
CPU TK1 TX1
20x
1.4x
Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375)
1.9
0.08 0.06
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
CPU TK1 TX1
1.3x 23x
Experiment environments
CPU: Intel Core i7 3.40GHz
TK1: Jetson TK1 Kepler GPU
TX1: Jetson TX1 Maxwell GPU
![Page 27: 고속 매칭 알고리즘 GPU 기반 병렬처리...CPU TK1 TX1 20x 1.4x Middlebury evaluation (450 x 375) KITTI evaluation (1242 x375) 1.9 0.08 0.06 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6](https://reader033.fdocuments.us/reader033/viewer/2022050506/5f97c44655159f21ef75a28c/html5/thumbnails/27.jpg)
SEOUL | Oct.7, 2016
THANK YOU