Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik,...
-
Upload
jody-powell -
Category
Documents
-
view
212 -
download
0
Transcript of Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik,...
![Page 1: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/1.jpg)
Correspondence-Free Determinationof the Affine Fundamental Matrix
2007. 2. 6 (Tue)Young Ki Baik, Computer Vision Lab.
![Page 2: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/2.jpg)
2
Correspondence-Free Determination of the Affine fundamental Matrix
• References• Correspondence-Free Determination of the
Affine Fundamental Matrix• Stefan Lehmann et. al. PAMI 2007
• Radon-based Structure from Motion Without Correspondences
• Ameesh Makadia et. al. CVPR 2005
• Robust Fundamental Matrix Determination without Correspondences
• Stefan Lehmann et. al. APRS 2005
![Page 3: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/3.jpg)
3
Correspondence-Free Determination of the Affine fundamental Matrix
• Contents• The conventional method of SfM
• Features of the proposed method
• Theory of the proposed algorithm
• Experimental results
• Discussion
![Page 4: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/4.jpg)
4
• Conventional SfM
Image Sequence
Feature Extraction/ Matching
Relating Image
Projective Reconstructi
on
Auto-Calibration
Dense Matching
3D Model Building
Correspondence-Free Determination of the Affine fundamental Matrix
![Page 5: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/5.jpg)
5
Correspondence-Free Determination of the Affine fundamental Matrix
• The Problem of conventional SfM
• The high sensitivity of fundamental matrix
• Noise and outlier correspondences in feature data severely affect the precision of the fundamental matrix
• Incomplete 3D reconstruction
![Page 6: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/6.jpg)
6
Correspondence-Free Determination of the Affine fundamental Matrix
• The Key Feature• Correspondence-free
• Finding Correspondence (X)
• Illumination changes-free (?)• Intensity value (X)• Position of features (O)
• Limitation• Occlusion ? (X)
• Affine camera only!!
![Page 7: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/7.jpg)
7
Correspondence-Free Determination of the Affine fundamental Matrix
• Parallel projection• Orthographic projection
![Page 8: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/8.jpg)
8
Correspondence-Free Determination of the Affine fundamental Matrix
• Mathematical Model• Assumption
• We have 3-dimensional N features. • The 3D feature space is represented by,
N
nnnn zzyyxxzyxf
13 ,,,,
locations feature Individual : ,, nnn zyx
function delta Dirac :
![Page 9: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/9.jpg)
9
Correspondence-Free Determination of the Affine fundamental Matrix
• Mathematical Model• Assumption
• Parallel projection model determines the 2D feature projections along the lines that are running parallel to the view axis (z-axis) of the camera.
• The model considers a continuous projection plane with infinite extent.
• The corresponding 2D projection data is…
N
nnn yyxxdzzyxfyxf
132 ,,,,
R
![Page 10: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/10.jpg)
10
Correspondence-Free Determination of the Affine fundamental Matrix
• Mathematical Model• Fourier spectra
• The Fourier spectra of and can be denoted as
N
n
zyxj
zyxj
N
n
yxjyxj
nnn
nn
e
dxdydzezyxfF
edxdyeyxfF
1
33
122
,,,,
, ,
3
2
R
R
yxf ,2
componentsfrequency 3D : ,,
zyxf ,,3
theoremslice-projection The
0,,, 32 FF
![Page 11: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/11.jpg)
11
Correspondence-Free Determination of the Affine fundamental Matrix
• Mathematical Model• 2-view case
• The 3D correspondence feature point
• Relation between images
• The 3D frequency vector
TT zyxzyx ,,,,, PP
matrixon translatiandrotation 3D eousnonhomogen:, tR
tRPP
T ,,Δ
![Page 12: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/12.jpg)
12
Correspondence-Free Determination of the Affine fundamental Matrix
• Mathematical Model• 2-view case
• Relation between 3D spectrums
ΔRΔ Δt Tj FeFT
33
ΔPΔTjeF 3
tRPP
The equation shows that rotation R also establishes the transformation between corresponding frequency indices in the 3D Fourier spaces of the original and the transformed spectrum or scene.
![Page 13: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/13.jpg)
13
Correspondence-Free Determination of the Affine fundamental Matrix
• Mathematical Model• Matching line
• The magnitudes of two spectra along these lines will be identical, while the phases will show a linear offset dependent upon the translational component of transformation.
0,,, 32 FF
• The proposed method is to detect these matching lines.
![Page 14: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/14.jpg)
14
Correspondence-Free Determination of the Affine fundamental Matrix
• Mathematical Model• Matching line angle pair
• Angle pair of the matching lines with respect to the axes of the frequency spectra F and F’, respectively.
,
,
![Page 15: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/15.jpg)
15
Correspondence-Free Determination of the Affine fundamental Matrix
• Mathematical Model• Analysis of the transformation parameters
• as the corresponding frequency locations along the matching lines of the spectrum F of the first and the spectrum F’ of the second set of 2D features, respectively.
• It follows that,
,,,
000 ,, zyxt ,, 2200 FeF yxj
![Page 16: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/16.jpg)
16
Correspondence-Free Determination of the Affine fundamental Matrix
• Mathematical Model• Analysis of the transformation parameters
000 ,, zyxt ,, 2200 FeF yxj
sin,cos
sin,cos
sin,cossin,cos 22 vvFevvF vj
sincos 00 yx
sin,cos21 vvFvF
![Page 17: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/17.jpg)
17
Correspondence-Free Determination of the Affine fundamental Matrix
• Mathematical Model• Derivation of a 3D rotation matrix
zxz RRRR
100
0cossin
0sincos
zR
cossin0
sincos0
001
xR
100
0cossin
0sincos
zR
angleunknown :
![Page 18: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/18.jpg)
18
Correspondence-Free Determination of the Affine fundamental Matrix
• Estimation of the fundamental matrix• By using 3D rotation matrix, we can obtain the
relation between 2D projection point (x’,y’) of a 3D feature (x,y,z) with translation.
0sinsin
coscossinsincos
sincossincoscos
xz
y
xx
0sincos
coscoscossinsin
sincoscoscossin
yz
y
xy
![Page 19: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/19.jpg)
19
Correspondence-Free Determination of the Affine fundamental Matrix
• Estimation of the fundamental matrix• In the orthographic projection case, all epipolar
lines are parallel.
• Then we can denote the epipolar line of 2D feature point (x,y) as
00]sincossinsin[
]coscoscossinsin
coscossinsincos[
]sincoscoscossin
sincossincoscos[
qypxzqp
yq
p
xq
pc
v)(x,on depends ccyqxp
![Page 20: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/20.jpg)
20
Correspondence-Free Determination of the Affine fundamental Matrix
• Estimation of the fundamental matrix
00]sincossinsin[
]coscoscossinsin
coscossinsincos[
]sincoscoscossin
sincossincoscos[
qypxzqp
yq
p
xq
pc
sin,cos qpcyqxp
yxyx sincossincos
![Page 21: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/21.jpg)
21
Correspondence-Free Determination of the Affine fundamental Matrix
• Estimation of the fundamental matrix
yxyx sincossincos
edc
b
a
F 00
00
0 edycxybxa
sincos
sin00
cos00
F ?,,
![Page 22: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/22.jpg)
22
Correspondence-Free Determination of the Affine fundamental Matrix
• Estimation of matching line angle• For the practical purpose, corresponding discrete
spectra should be defined as follows.
fkFc
fkFb
k
k
2
2
1
1 resolutionfrequency :f
resolutionfrequency circular :2 f
N
nnnk
N
nnnk
yxjkc
yxjkb
1
1
sincosexp
sincosexp
![Page 23: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/23.jpg)
23
Correspondence-Free Determination of the Affine fundamental Matrix
• Estimation of matching line angle• The final object function
• Discrete Fourier-Mellin transformation method• To find out the matching line (According to the well known shift theorem of the FT,
a shift in the space domain corresponds to a phase shift in the frequency domain.)
22
2
1
,,2 maxarg
cb
ecbd
N
k
jkkk
![Page 24: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/24.jpg)
24
Correspondence-Free Determination of the Affine fundamental Matrix
• Overall flow
![Page 25: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/25.jpg)
25
Correspondence-Free Determination of the Affine fundamental Matrix
• Experimental result• test images : telephoto lens
• Feature points : Harris corner detection method
• Ideal epipolar lines are the horizontal lines.
• The proposed method shows us good result relative to conventional methods.
![Page 26: Correspondence-Free Determination of the Affine Fundamental Matrix 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab.](https://reader035.fdocuments.us/reader035/viewer/2022070410/56649ed95503460f94be77e3/html5/thumbnails/26.jpg)
26
Camera Calibration Methods for Wide Angle view
• Discussion• Key feature
• Correspondence-free method for obtaining the fundamental matrix is presented.
Matching line exists between the Fourier transformed data.
• Limitation• Proposed method
Considers only affine projection model Does not treat occlusion problem
• Future work• Applying projective projection model