Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera...
Transcript of Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera...
![Page 1: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/1.jpg)
Camera Calibration Chapter 7
Multiple View Geometry
![Page 2: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/2.jpg)
Camera calibration
![Page 3: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/3.jpg)
ii xX ↔ ? P
Resectioning
![Page 4: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/4.jpg)
Basic equations
ii PXx =
0Ap =
xi ⇥PXi = 0
![Page 5: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/5.jpg)
Basic equations
0Ap =
minimal solution
Over-determined solution
5½ correspondences needed (say 6) P has 11 dof, 2 independent eq./points
n > 6 points
Apminimize subject to constraint
1p =
1p̂3 =3p̂
=P
![Page 6: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/6.jpg)
Degenerate configurations
More complicate than 2D case
(i) Camera and points on a twisted cubic
(ii) Points lie on plane or single line passing through projection center
![Page 7: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/7.jpg)
Less obvious
(i) Simple, as before
(ii) Anisotropic scaling
Data normalization
32
![Page 8: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/8.jpg)
Line correspondences
Extend Direct Linear Transformation (DLT) to lines
ilPT=Π
liTPX1i = 0
(back-project line)
liTPX2i = 0 (2 independent eq.)
![Page 9: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/9.jpg)
Geometric error
![Page 10: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/10.jpg)
Gold Standard algorithm Objective
Given n≥6 3D to 2D point correspondences {Xi↔xi’}, determine the Maximum Likelihood Estimation of P
Algorithm (i) Linear solution:
(a) Normalization: (b) DLT:
(ii) Minimization of geometric error: using the linear estimate as a starting point minimize the geometric error:
(iii) Denormalization:
ii UXX~ = ii Txx~ =
UP~TP -1=
~ ~ ~
![Page 11: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/11.jpg)
Calibration example
(i) Canny edge detection (ii) Straight line fitting to the detected edges (iii) Intersecting the lines to obtain the images corners
typically precision <1/10 (HZ rule of thumb: 5n constraints for n unknowns
![Page 12: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/12.jpg)
Errors in the world
Errors in the image and in the world
ii XPx⌢
=
![Page 13: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/13.jpg)
Geometric interpretation of algebraic error
( )2)x̂,x(ˆ∑i
iiidw
( ) iiii yxw PX1,ˆ,ˆˆ = P)depth(X;p̂ˆ 3±=iw
)X̂,X(~)x̂,x(ˆ iiiii fddw
then1p̂ if therefore, 3 =
note invariance to 2D and 3D similarities given proper normalization
![Page 14: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/14.jpg)
Estimation of affine camera
note that in this case algebraic error = geometric error
![Page 15: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/15.jpg)
Gold Standard algorithm Objective
Given n≥4 3D to 2D point correspondences {Xi↔xi’}, determine the Maximum Likelihood Estimation of P (remember P3T=(0,0,0,1))
Algorithm (i) Normalization: (ii) For each correspondence
(iii) solution is
(iv) Denormalization:
ii UXX~ = ii Txx~ =
UP~TP -1=
bpA 88 =
bAp 88+=
![Page 16: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/16.jpg)
Restricted camera estimation
Minimize geometric error • impose constraint through parametrization • Select a set of parameters that characterize the
camera matrix
Find best fit that satisfies • skew s is zero • pixels are square • principal point is known • complete camera matrix K is known
Minimize algebraic error • assume map from param q to P=K[R|-RC], i.e. p=g(q) • minimize ||Ag(q)||
![Page 17: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/17.jpg)
Reduced measurement matrix
One only has to work with 12x12 matrix, not 2nx12
Ap 2 = pTATAp = A^p
2
![Page 18: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/18.jpg)
Restricted camera estimation
Initialization • Use general DLT • Clamp values to desired values, e.g. s=0, Note: can sometimes cause big jump in error Alternative initialization • Use general DLT • Impose soft constraints
• gradually increase weights
↵x
= ↵y
![Page 19: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/19.jpg)
Exterior orientation
Calibrated camera, position and orientation unkown Pose estimation
6 dof : 3 points minimal (4 solutions in general)
![Page 20: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/20.jpg)
![Page 21: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/21.jpg)
Covariance estimation
ML residual error
Example: n=197, =0.365, =0.37
![Page 22: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/22.jpg)
Covariance for estimated camera
Compute Jacobian at ML solution, then
( )+−Σ=Σ JJ 1x
TP
(variance per parameter can be found on diagonal)
2χ(chi-square distribution =distribution of sum of squares)
cumulative-1
![Page 23: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/23.jpg)
![Page 24: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/24.jpg)
short and long focal length
Radial distortion
![Page 25: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/25.jpg)
![Page 26: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/26.jpg)
![Page 27: Camera Calibration - McMaster Universityshirani/vision/hartley_ch7.pdf · 2015-03-03 · Camera Calibration Chapter 7 Multiple View Geometry . Camera calibration . X i ↔ x i P ?](https://reader034.fdocuments.us/reader034/viewer/2022043010/5fa14cc92474f979e65c25ff/html5/thumbnails/27.jpg)
Correction of distortion
Choice of the distortion function and center
Computing the parameters of the distortion function (i) Minimize with additional unknowns (ii) Straighten lines (iii) …