The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography...
-
Upload
madison-collins -
Category
Documents
-
view
213 -
download
2
Transcript of The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography...
![Page 1: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/1.jpg)
The Trifocal Tensor
Multiple View Geometry
![Page 2: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/2.jpg)
Scene planes and homographies
plane induces homography between two views
0HFFH TT
H'eF
ve'Fe'H
Hee'
![Page 3: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/3.jpg)
6-point algorithm
6655 Hx'xHx'xe'
x1,x2,x3,x4 in plane, x5,x6 out of plane
Compute H from x1,x2,x3,x4
He'F
![Page 4: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/4.jpg)
Three-view geometry
![Page 5: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/5.jpg)
The trifocal tensor
Three back-projected lines have to meet in a single line
Incidence relation provides constraint on lines
Let us derive the corresponding algebraic constraint…
![Page 6: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/6.jpg)
Notations
iii lll
0]|[IP ]a|[AP' 4 ]b|[BP" 4
0
llPπ T
l'al'Al'P'π' T
4
TT
l"bl"Bl"P"π" T
4
TT
![Page 7: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/7.jpg)
Incidence
iii lll
0
llPπ T
l'al'Al'P'π' T
4
TT
l"bl"Bl"P"π" T
4
TT
2rank is ]π"π'[πM ,,
e.g. is part of bundle formed by ’ and ”
l"bl'a0l"Bl'Al]m,m,m[M T
4T
4
TT
321
321 βmαmm "lβb'lαa0 T4
T4
'la T4k
"lb T
4k "lB'la'lA"lbl TT4
TT4
![Page 8: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/8.jpg)
Incidence relation
"lBa'l'lAb"l"lB'la'lA"lbl T4
TT4
TTT4
TT4
"lba'l'lab"l T4
TT4
Tiiil "lba'l"lba'l T
4TT
4T
ii
T4
T4 babaT iii
"lT'l Tiil
![Page 9: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/9.jpg)
The Trifocal Tensor
"lT,T,T'll 321TT
Trifocal Tensor = {T1,T2,T3}
"lT'l,"lT'l,"lT'l 3T
2T
1T
Only depends on image coordinates and is thus independent of 3D projective basis
"lT'll' TTi 'lT"ll" TT
iAlso and but no simple relation
General expression not as simple asT
4T
4 babaT iii
DOF T: 3x3x3=27 elements, 26 up to scale 3-view relations: 11x3-15=18 dof
8(=26-18) independent algebraic constraints on T(compare to 1 for F, i.e. rank-2)
![Page 10: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/10.jpg)
Homographies induced by a plane
![Page 11: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/11.jpg)
Line-line-line relation
Eliminate scale factor:
(up to scale)
![Page 12: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/12.jpg)
Point-line-line relation
![Page 13: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/13.jpg)
Point-line-point relation
note: valid for any line through x”, e.g. l”=[x”]xx”arbitrary
![Page 14: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/14.jpg)
Point-point-point relation
note: valid for any line through x’, e.g. l’=[x’]xx’arbitrary
![Page 15: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/15.jpg)
Overview incidence relations
![Page 16: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/16.jpg)
Non-incident configurationincidence in image does not guarantee incidence in space
![Page 17: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/17.jpg)
Epipolar lines
if l’ is epipolar line, then satisfied for arbitrary l”
inversely,
epipolar lines are right and left null-space of
![Page 18: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/18.jpg)
Epipoles
With points
becomes respectively
Epipoles are intersection of right resp. left null-space of
(e=P’C and e”=P”C)
![Page 19: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/19.jpg)
Extracting F
21Fgood choice for l” is e” (V3
Te”=0)
![Page 20: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/20.jpg)
Computing P,P‘,P“
?
ok, but not
specifically, (no derivation)
![Page 21: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/21.jpg)
matrix notation is impractical
Use tensor notation instead
0" jkikj
i Tllx
jkiT
![Page 22: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/22.jpg)
Definition affine tensor
• Collection of numbers, related to coordinate choice, indexed by one or more indices
• Valency = (n+m)• Indices can be any value between 1
and the dimension of space (d(n+m)
coefficients)
![Page 23: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/23.jpg)
Conventions
0iijbA
Einstein’s summation:(once above, once below)
ii
iji
ij bAbA
Index rule: jbA iij ,0
Contravariant indices
Covariant indices
![Page 24: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/24.jpg)
More on tensors
• Transformations
iji
j xAx
ijji llA
(covariant)
(contravariant)
![Page 25: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/25.jpg)
Some special tensors
• Kronecker delta
• Levi-Cevita epsilon
(valency 2 tensor)
(valency 3 tensor)
![Page 26: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/26.jpg)
![Page 27: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/27.jpg)
Trilinearities
![Page 28: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/28.jpg)
Compute F and P from T
![Page 29: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/29.jpg)
matrix notation is impractical
Use tensor notation instead
0 jkikj
i Tllx
jkiT
![Page 30: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/30.jpg)
Definition affine tensor
• Collection of numbers, related to coordinate choice, indexed by one or more indices
• Valency = (n+m)• Indices can be any value between 1
and the dimension of space (d(n+m)
coefficients)
![Page 31: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/31.jpg)
Conventions
0iijbA
Contraction:(once above, once below)
ii
iji
ij bAbA
Index rule: jbA iij ,0
![Page 32: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/32.jpg)
More on tensors
• Transformations
iji
j xAx
ijji llA
(covariant)
(contravariant)
![Page 33: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/33.jpg)
Some special tensors
• Kronecker delta
• Levi-Cevita epsilon
(valency 2 tensor)
(valency 3 tensor)
![Page 34: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/34.jpg)
![Page 35: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/35.jpg)
Trilinearities
![Page 36: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/36.jpg)
Transfer: epipolar transfer
![Page 37: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/37.jpg)
Transfer: trifocal transfer
Avoid l’=epipolar line
![Page 38: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/38.jpg)
Transfer: trifocal transferpoint transfer
line transfer
degenerate when known lines are corresponding epipolar lines
![Page 39: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/39.jpg)
Computation of Trifocal Tensor
• Linear method (7-point)
• Minimal method (6-point)
• Geometric error minimization method
• RANSAC method
![Page 40: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/40.jpg)
Basic equations
Three points
Correspondence Relation #lin. indep.Eq.
4
Two points, one line
One points, two line
2
1
2Three lines
At=0 (26 equations)
(more equations)min||At|| with ||t||=1
![Page 41: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/41.jpg)
Normalized linear algorithm
At=0
ijlmimk
ljjmk
liilk
mjjlk
mik TxxTxxTxxTxxx 0
03333 ilk
lk
iik
lk
lik TTxTxTxxx 2,1, li
Points
Lines
or
Normalization: normalize image coordinates to ~1
![Page 42: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/42.jpg)
Normalized linear algorithm
ObjectiveGiven n7 image point correspondences across 3 images,or a least 13 lines, or a mixture of point and line corresp., compute the trifocal tensor.Algorithm(i) Find transformation matrices H,H’,H” to normalize 3 images(ii) Transform points with H and lines with H-1
(iii) Compute trifocal tensor T from At=0 (using SVD)(iv) Denormalize trifocal tensor st
r
k
t
j
sri
jki TT ˆ"H'HH 11
![Page 43: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/43.jpg)
Internal constraints
27 coefficients 1 free scale 18 parameters 8 internal consistency constraints
(not every 3x3x3 tensor is a valid trifocal tensor!)
(constraints not easily expressed explicitly)
Trifocal Tensor satisfies all intrinsic constraintsif it corresponds to three cameras {P,P’,P”}
![Page 44: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/44.jpg)
Maximum Likelihood Estimation
i
iiiiii ddd 222 "x̂,x"'x̂,x'x̂,x
iii x"x'x
iiiiii X"P"x̂,XP''x̂,X]0|I[x̂
data
cost function
parameterization
(24 parameters+3N)
also possibility to use Sampson error (24 parameters)
![Page 45: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/45.jpg)
ObjectiveCompute the trifocal tensor between two images
Algorithm
(i) Interest points: Compute interest points in each image
(ii) Putative correspondences: Compute interest correspondences (and F) between 1&2 and 2&3
(iii) RANSAC robust estimation: Repeat for N samples
(a) Select at random 6 correspondences and compute T
(b) Calculate the distance d for each putative match
(c) Compute the number of inliers consistent with T (d<t)
Choose T with most inliers
(iv) Optimal estimation: re-estimate T from all inliers by minimizing ML cost function with Levenberg-Marquardt
(v) Guided matching: Determine more matches using prediction by computed T
Optionally iterate last two steps until convergence
Automatic computation of T
![Page 46: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/46.jpg)
108 putative matches 18 outliers
88 inliers 95 final inliers
(26 samples)
(0.43)(0.23)
(0.19)
![Page 47: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/47.jpg)
additional line matches
![Page 48: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/48.jpg)
Matrix formulation for m-View
Consider one object point X and its m images: ixi=PiXi, i=1, …. ,m:
i.e. rank(M) < m+4 .
![Page 49: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/49.jpg)
Laplace expansions
• The rank condition on M implies that all (m+4)*(m+4) minors of M are equal to 0.
• These can be written as sums of products of camera matrix parameters and image coordinates.
![Page 50: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/50.jpg)
Matrix formulation
for non-trivially zero minors, one row has to be taken from each image (m).
4 additional rows left to choose
![Page 51: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/51.jpg)
lk
jihgfedcba
000000000000000
ihgfedcba
jkl
only interesting if 2 or 3 rows from view
![Page 52: The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.](https://reader037.fdocuments.us/reader037/viewer/2022110205/56649cc95503460f94991b43/html5/thumbnails/52.jpg)
The three different types
1. Take the 2 remaining rows from one image block and the other two from another image block, gives the 2-view constraints.
2. Take the 2 remaining rows from one image block 1 from another and 1 from a third, gives the 3-view constraints.
3. Take 1 row from each of four different image blocks, gives the 4-view constraints.