RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process...
Transcript of RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process...
![Page 1: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/1.jpg)
2019-04-10 1
RAFSet 3D-2Dmotion estimation
part 1. LiDAR interpolation
Jeon Hyun Ho
![Page 2: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/2.jpg)
Frame to frame motion estimation method
2D-2D motion estimation ( Scale problem )
3D-2D motion estimation ( minimize image reprojection error )
3D-3D motion estimation ( minimize feature position error )
2
Scale problem
Scaramuzza, Davide, and Friedrich Fraundorfer. "Visual odometry [tutorial]." IEEE Robotics & Automation
Magazine 18.4 (2011): 80-92.
Fraundorfer, Friedrich, and Davide Scaramuzza. "Visual odometry: Part II: Matching,
robustness, optimization, and applications." IEEE Robotics & Automation
Magazine 19.2 (2012): 78-90.
Intro
![Page 3: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/3.jpg)
3
Intro
Feature based motion estimation process
Image Sequence
Feature Detection
Feature Matching(or Tracking)
Motion Estimation (2D-2D, 3D-2D, 3D-3D)
Local Optimization(Bundle Adjustment)
![Page 4: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/4.jpg)
3D-2D motion estimation
4
Interpolation
![Page 5: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/5.jpg)
3D-2D motion estimation
Sparse LiDAR data
5
Interpolation
![Page 6: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/6.jpg)
3D-2D motion estimation
Sparse LiDAR data
6
Interpolation
![Page 7: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/7.jpg)
Method 1
Dilation interpolation
7
Interpolation
k
ij
k
ij
PS
BpS
)(
![Page 8: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/8.jpg)
Method 2
Adaptive bilinear interpolation* (𝑀𝑎𝑥 ∆𝑥, 𝑀𝑎𝑥 ∆𝑦)
8
Interpolation
𝑀𝑎𝑥 ∆𝑥
𝐺𝑟𝑎𝑑𝑖𝑒𝑛𝑡 ∶ 𝐺 =𝑃𝑖𝑘 − 𝑃𝑖+1
𝑘
∆𝑥𝑖𝑖+1
𝐼𝑛𝑡𝑒𝑟𝑝𝑜𝑙𝑎𝑡𝑒𝑑 𝑃𝑜𝑖𝑛𝑡 ∶ 𝑃𝑘 = 𝐺 ∗ ∆𝑥 + 𝑃𝑖𝑘
𝑀𝑎𝑥 ∆𝑦
![Page 9: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/9.jpg)
9
Interpolation
𝐺𝑟𝑎𝑑𝑖𝑒𝑛𝑡 ∶ 𝐺 =𝑃𝑖𝑘 − 𝑃𝑖+1
𝑘
∆𝑥𝑖𝑖+1
𝐼𝑛𝑡𝑒𝑟𝑝𝑜𝑙𝑎𝑡𝑒𝑑 𝑃𝑜𝑖𝑛𝑡 ∶ 𝑃𝑘 = 𝐺 ∗ ∆𝑥 + 𝑃𝑖𝑘
Method 2
Adaptive bilinear interpolation* (𝑀𝑎𝑥 ∆𝑥, 𝑀𝑎𝑥 ∆𝑦)
![Page 10: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/10.jpg)
10
Interpolation
𝐺𝑟𝑎𝑑𝑖𝑒𝑛𝑡 ∶ 𝐺 =𝑃𝑖𝑘 − 𝑃𝑖+1
𝑘
∆𝑥𝑖𝑖+1
𝐼𝑛𝑡𝑒𝑟𝑝𝑜𝑙𝑎𝑡𝑒𝑑 𝑃𝑜𝑖𝑛𝑡 ∶ 𝑃𝑘 = 𝐺 ∗ ∆𝑥 + 𝑃𝑖𝑘
Method 2
Adaptive bilinear interpolation* (𝑀𝑎𝑥 ∆𝑥, 𝑀𝑎𝑥 ∆𝑦)
![Page 11: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/11.jpg)
11
Interpolation
Method 3
Plane interpolation
𝐴(𝑋𝐴, 𝑌𝐴, 𝑍𝐴)
𝐵(𝑋𝐵, 𝑌𝐵 , 𝑍𝐵)
𝐶(𝑋𝐶 , 𝑌𝐶 , 𝑍𝐶)
𝑃𝑙𝑎𝑛𝑒 𝐴𝐵𝐶
![Page 12: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/12.jpg)
12
Interpolation
Method 3
Plane interpolation
𝑝𝑖𝑚𝑔
![Page 13: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/13.jpg)
13
Interpolation
Method 3
Plane interpolation
Camera coordinate
𝑂𝑐
𝑍𝑐
𝑋𝑐
𝑌𝑐
Pixel coordinate
𝑦
𝑥
𝑃
𝑝𝑖𝑚𝑔
𝑝𝑖𝑚𝑔2𝐷(𝑥 − 𝑢, 𝑦 − 𝑣)
𝑝𝑖𝑚𝑔2𝐷(𝑥, 𝑦)
𝑝𝑖𝑚𝑔3𝐷(𝑥 − 𝑢, 𝑦 − 𝑣, 𝑓)
𝑙𝑖𝑛𝑒(𝑂𝑐 , 𝑝𝑖𝑚𝑔3𝐷)
![Page 14: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/14.jpg)
14
Interpolation
Method 3
Plane interpolation
𝑝𝑖𝑚𝑔
![Page 15: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/15.jpg)
15
Interpolation
Method 3
Plane interpolation
𝑝𝑖𝑚𝑔
𝑙𝑖𝑛𝑒 𝐴𝐵 ∶ 𝑦 = 𝑎𝐴𝐵 ∗ 𝑥 + 𝑏𝐴𝐵
𝑙𝑖𝑛𝑒 𝐵𝐶 ∶ 𝑦 = 𝑎𝐵𝐶 ∗ 𝑥 + 𝑏𝐵𝐶
𝑙𝑖𝑛𝑒 𝐶𝐴 ∶ 𝑦 = 𝑎𝐶𝐴 ∗ 𝑥 + 𝑏𝐶𝐴
𝑣𝑎𝑙𝑖𝑑 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛:
𝑖𝑓 (𝑝𝑖𝑚𝑔𝑦− 𝑎𝐴𝐵 ∗ 𝑝𝑖𝑚𝑔𝑥
+ 𝑏𝐴𝐵) ∗ 𝐶𝑦 − 𝑎𝐴𝐵 ∗ 𝐶𝑥 + 𝑏𝐴𝐵 < 0 , 𝑐𝑜𝑛𝑡𝑖𝑛𝑢𝑒;
𝑖𝑓 𝑎𝐴𝐵 = ∞, 𝐴 𝑜𝑟 𝐵 𝑥 − 𝑝𝑖𝑚𝑔𝑥∗ 𝐴 𝑜𝑟 𝐵 𝑥 − 𝐶𝑥 < 0, 𝑐𝑜𝑛𝑡𝑖𝑛𝑢𝑒;
![Page 16: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/16.jpg)
16
Result
LiDAR data – dilation interpolation
![Page 17: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/17.jpg)
17
Result
Reprojection error – dilation interpolation
![Page 18: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/18.jpg)
18
Result
Reprojection error – dilation interpolation
![Page 19: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/19.jpg)
19
Result
Outlier
![Page 20: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/20.jpg)
20
Result
LiDAR data – adaptive bilinear interpolation
![Page 21: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/21.jpg)
21
Result
Reprojection error – adaptive bilinear interpolation
![Page 22: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/22.jpg)
22
Result
LiDAR data – plane interpolation
![Page 23: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/23.jpg)
23
Result
Reprojection error - plane interpolation
![Page 24: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/24.jpg)
24
Result
Reprojection error - plane interpolation
![Page 25: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/25.jpg)
25
Result
Reprojection error - plane interpolation
![Page 26: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/26.jpg)
26
Result
Visual Odometry
![Page 27: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/27.jpg)
27
Conclusion
SOFT (Steroe Odometry based on careful feature selection and tracking)
![Page 28: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/28.jpg)
28
Conclusion
SOFT (Steroe Odometry based on careful feature selection and tracking)
![Page 29: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/29.jpg)
29
Conclusion
SOFT (Steroe Odometry based on careful feature selection and tracking)
![Page 30: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/30.jpg)
30
Conclusion
Optimization – iSAM (Inceremetal Smoothing and Mapping)
![Page 31: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/31.jpg)
31
Conclusion
Optimization – iSAM (Inceremetal Smoothing and Mapping)
![Page 32: RAFSet 3D-2D motion estimation...2017/01/09 · 3 Intro Feature based motion estimation process Image Sequence Feature Detection Feature Matching(or Tracking) Motion Estimation (2D-2D,](https://reader036.fdocuments.us/reader036/viewer/2022081411/60ab14be73f04c13f87c6c45/html5/thumbnails/32.jpg)
32
Q&A