55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:
description
Transcript of 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:
![Page 1: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/1.jpg)
55:148 Digital Image ProcessingChapter 11 3D Vision, Geometry
Topics:Basics of projective geometry
Points and hyperplanes in projective spaceHomographyEstimating homography from point correspondence
The single perspective cameraAn overview of single camera calibrationCalibration of one camera from the known scene
Scene reconstruction from multiple viewsTriangulationProjective reconstructionMatching constraintsBundle adjustment
Two cameras, stereopsisThe geometry of two cameras. The fundamental matrixRelative motion of the camera; the essential matrixEstimation of a fundamental matrix from image point correspondencesCamera Image rectificationApplications of the epipolar geometry in vision
Three and more camerasStereo correspondence algorithms
![Page 2: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/2.jpg)
Image epipolar rectification
Illustrative description
• Both image planes being coplanar• Baseline along the horizontal axis (x-axis)
of the image plane• Two epipoles at infinity along the
horizontal direction• Horizontal epipolar lines
Before
After
![Page 3: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/3.jpg)
Image epipolar rectification
Illustrative description
• Both image planes being coplanar• Baseline along the horizontal axis (x-axis)
of the image plane• Two epipoles at infinity along the
horizontal direction• Horizontal epipolar lines
• Method: Apply suitable homographic transformation on each image
Before
After
![Page 4: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/4.jpg)
Image rectification (before)
![Page 5: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/5.jpg)
Image rectification (after)
![Page 6: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/6.jpg)
Image rectificationAnalytic description
• Suppose that we know camera matrices and for the two cameras• The objective is to rotate each image planes around their optical centers until their
focal planes becomes coplanar • Thereby, containing the baseline• More specifically, the new x-axis is parallel to the baseline
• Both images have y-axis x-axisThus, new camera matrices
![Page 7: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/7.jpg)
Image rectificationAnalytic description
• Suppose that we know camera matrices and for the two cameras• The objective is to rotate each image planes around their optical centers until their
focal planes becomes coplanar • Thereby, containing the baseline• More specifically, the new x-axis is parallel to the baseline
• Both images have y-axis x-axisThus, new camera matrices
where,
![Page 8: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/8.jpg)
Image rectificationAnalytic description
• Suppose that we know camera matrices and for the two cameras• The objective is to rotate each image planes around their optical centers until their
focal planes becomes coplanar • Thereby, containing the baseline• More specifically, the new x-axis is parallel to the baseline
• Both images have y-axis x-axisThus, new camera matrices
where,
1) The new x-axis is parallel to the baseline
![Page 9: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/9.jpg)
Image rectificationAnalytic description
• Suppose that we know camera matrices and for the two cameras• The objective is to rotate each image planes around their optical centers until their
focal planes becomes coplanar • Thereby, containing the baseline• More specifically, the new x-axis is parallel to the baseline
• Both images have y-axis x-axisThus, new camera matrices
where,
1) The new x-axis is parallel to the baseline 2) The new y-axis is orthogonal to the new x-axis , for some and
![Page 10: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/10.jpg)
Image rectificationAnalytic description
• Suppose that we know camera matrices and for the two cameras• The objective is to rotate each image planes around their optical centers until their
focal planes becomes coplanar • Thereby, containing the baseline• More specifically, the new x-axis is parallel to the baseline
• Both images have y-axis x-axisThus, new camera matrices
where,
1) The new x-axis is parallel to the baseline 2) The new y-axis is orthogonal to the new x-axis , for some and 3) The new z-axis is orthogonal to the new xy-plane
![Page 11: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/11.jpg)
A proof that Image rectification a suitable homographic transformation
![Page 12: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/12.jpg)
A proof that Image rectification a suitable homographic transformation
Let
For any 3D scene point , projected image points using camera matrices are:
![Page 13: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/13.jpg)
A proof that Image rectification a suitable homographic transformation
Let
For any 3D scene point , projected image points using camera matrices are:
Using the relation in homogeneous coordinate system,
![Page 14: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/14.jpg)
A proof that Image rectification a suitable homographic transformation
Let
For any 3D scene point , projected image points using camera matrices are:
Using the relation in homogeneous coordinate system,
![Page 15: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/15.jpg)
A proof that Image rectification a suitable homographic transformation
Let
For any 3D scene point , projected image points using camera matrices are:
Using the relation in homogeneous coordinate system,
i.e.,
![Page 16: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/16.jpg)
A proof that Image rectification a suitable homographic transformation
Let
For any 3D scene point , projected image points using camera matrices are:
Using the relation in homogeneous coordinate system,
i.e.,
: scale factor: 3-by-3 matrix ≈ perspective transformation ≈ homography equivalent
HENCE PROVED
![Page 17: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/17.jpg)
Image rectificationAlgorithmic description
• As we have seen from analytic discussionGiven the knowledge of and , image rectification process is precisely defined So, the first task is to determine camera matrices and
Algorithm Compute camera matrices and Task 1: Solve using corresponding point pairs in two images and algebraic error minimizationInput: A linear system:
![Page 18: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/18.jpg)
Image rectificationAlgorithmic description
• As we have seen from analytic discussionGiven the knowledge of and , image rectification process is precisely defined So, the first task is to determine camera matrices and
Algorithm Compute camera matrices and Task 1: Solve using corresponding point pairs in two images and algebraic error minimizationInput: A linear system: Using Kronecker product identity: , we get
![Page 19: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/19.jpg)
Image rectificationAlgorithmic description
• As we have seen from analytic discussionGiven the knowledge of and , image rectification process is precisely defined So, the first task is to determine camera matrices and
Algorithm Compute camera matrices and Task 1: Solve using corresponding point pairs in two images and algebraic error minimizationInput: A linear system: Using Kronecker product identity: , we get Put together all correspondences
![Page 20: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/20.jpg)
Image rectificationAlgorithmic description
• As we have seen from analytic discussionGiven the knowledge of and , image rectification process is precisely defined So, the first task is to determine camera matrices and
Algorithm Compute camera matrices and Task 1: Solve using corresponding point pairs in two images and algebraic error minimizationInput: A linear system: Using Kronecker product identity: , we get Put together all correspondences
Compute and apply singular value decomposition; choose along the eigenvector corresponding to the smallest eigenvalue
![Page 21: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/21.jpg)
Image rectificationAlgorithmic description
• As we have seen from analytic discussionGiven the knowledge of and , image rectification process is precisely defined So, the first task is to determine camera matrices and
Algorithm Compute camera matrices and Task 1: Solve using corresponding point pairs in two images and algebraic error minimizationInput: A linear system: Using Kronecker product identity: , we get Put together all correspondences
Compute and apply singular value decomposition; choose along the eigenvector corresponding to the smallest eigenvalueFor to be a valid fundamental matrix, its rank must be 2; but it may not satisfy in reality (why?); thus needs correction
![Page 22: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/22.jpg)
Image rectificationAlgorithmic description
• As we have seen from analytic discussionGiven the knowledge of and , image rectification process is precisely defined So, the first task is to determine camera matrices and
Algorithm Compute camera matrices and Task 1: Solve using corresponding point pairs in two images and algebraic error minimizationInput: A linear system: Using Kronecker product identity: , we get Put together all correspondences
Compute and apply singular value decomposition; choose along the eigenvector corresponding to the smallest eigenvalueFor to be a valid fundamental matrix, its rank must be 2; but it may not satisfy in reality (why?); thus needs correctionSVD decomposition: ; set the smallest eigenvalue of to zero giving a new diagonal matrix ; use the new matrix
![Page 23: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/23.jpg)
Image rectificationAlgorithmic description
• As we have seen from analytic discussionGiven the knowledge of and , image rectification process is precisely defined So, the first task is to determine camera matrices and
Algorithm Compute camera matrices and Task 1: Solve using corresponding point pairs in two images and algebraic error minimizationInput: A linear system: Using Kronecker product identity: , we get Put together all correspondences
Compute and apply singular value decomposition; choose along the eigenvector corresponding to the smallest eigenvalueFor to be a valid fundamental matrix, its rank must be 2; but it may not satisfy in reality (why?); thus needs correctionSVD decomposition: ; set the smallest eigenvalue of to zero giving a new diagonal matrix ; use the new matrix
Note that ML estimation method incorporate the validity condition of F into the optimization process
![Page 24: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/24.jpg)
Image rectificationAlgorithmic description
• As we have seen from analytic discussionGiven the knowledge of and , image rectification process is precisely defined So, the first task is to determine camera matrices and
Algorithm Compute camera matrices and Task 1: Solve using corresponding point pairs in two images and algebraic error minimizationInput: A linear system: Output: Fundamental matrix
Task 2: Decompose the fundamental to matrix camera matrices
![Page 25: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/25.jpg)
Image rectificationAlgorithmic description
• As we have seen from analytic discussionGiven the knowledge of and , image rectification process is precisely defined So, the first task is to determine camera matrices and
Algorithm Compute camera matrices and Task 1: Solve using corresponding point pairs in two images and algebraic error minimizationInput: A linear system: Output: Fundamental matrix
Task 2: Decompose the fundamental to matrix camera matrices
![Page 26: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/26.jpg)
Image rectificationAlgorithmic description
• As we have seen from analytic discussionGiven the knowledge of and , image rectification process is precisely defined So, the first task is to determine camera matrices and
Algorithm Compute camera matrices and Task 1: Solve using corresponding point pairs in two images and algebraic error minimizationInput: A linear system: Output: Fundamental matrix
Task 2: Decompose the fundamental to matrix camera matrices
Input of the algorithm : A linear system: Output of the algorithm: camera matrices and
![Page 27: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/27.jpg)
Image rectificationAlgorithmic description
• As we have seen from analytic discussionGiven the knowledge of and , image rectification process is precisely defined So, the first task is to determine camera matrices and
Algorithm Compute camera matrices and Task 1: Solve using corresponding point pairs in two images and algebraic error minimizationInput: A linear system: Output: Fundamental matrix
Task 2: Decompose the fundamental to matrix camera matrices
Input of the algorithm : A linear system: Output of the algorithm: camera matrices and
Now what?
![Page 28: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/28.jpg)
Image rectificationAlgorithmic description
Algorithm Compute image rectification
![Page 29: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/29.jpg)
Image rectificationAlgorithmic description
Algorithm Compute image rectificationInput: and Output: and and two homography transformations: and
![Page 30: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/30.jpg)
Image rectificationAlgorithmic description
Algorithm Compute image rectificationInput: and Output: and and two homography transformations: and
Decompose each camera matrices: and
![Page 31: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/31.jpg)
Image rectificationAlgorithmic description
Algorithm Compute image rectificationInput: and Output: and and two homography transformations: and
Decompose each camera matrices: and Determine two optical centers and using and and their decompositions
![Page 32: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/32.jpg)
Image rectificationAlgorithmic description
Algorithm Compute image rectificationInput: and Output: and and two homography transformations: and
Decompose each camera matrices: and Determine two optical centers and using and and their decompositionsDetermine new rotation matrix common to and
![Page 33: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/33.jpg)
Image rectificationAlgorithmic description
Algorithm Compute image rectificationInput: and Output: and and two homography transformations: and
Decompose each camera matrices: and Determine two optical centers and using and and their decompositionsDetermine new rotation matrix common to and Determine new intrinsic parameters by averaging and and then annulling skew component
![Page 34: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/34.jpg)
Image rectificationAlgorithmic description
Algorithm Compute image rectificationInput: and Output: and and two homography transformations: and
Decompose each camera matrices: and Determine two optical centers and using and and their decompositionsDetermine new rotation matrix common to and Determine new intrinsic parameters by averaging and and then annulling skew componentCompute new projection matrices:
![Page 35: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/35.jpg)
Image rectificationAlgorithmic description
Algorithm Compute image rectificationInput: and Output: and and two homography transformations: and
Decompose each camera matrices: and Determine two optical centers and using and and their decompositionsDetermine new rotation matrix common to and Determine new intrinsic parameters by averaging and and then annulling skew componentCompute new projection matrices: and
![Page 36: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/36.jpg)
Image rectificationAlgorithmic description
Algorithm Compute image rectificationInput: and Output: and and two homography transformations: and
Decompose each camera matrices: and Determine two optical centers and using and and their decompositionsDetermine new rotation matrix common to and Determine new intrinsic parameters by averaging and and then annulling skew componentCompute new projection matrices: and Compute homography transformations
![Page 37: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/37.jpg)
Image rectificationAlgorithmic description
Algorithm Compute image rectificationInput: and Output: and and two homography transformations: and
Decompose each camera matrices: and Determine two optical centers and using and and their decompositionsDetermine new rotation matrix common to and Determine new intrinsic parameters by averaging and and then annulling skew componentCompute new projection matrices: and Compute homography transformations and
![Page 38: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/38.jpg)
Image rectification (before)
![Page 39: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/39.jpg)
Image rectification (after)
![Page 40: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/40.jpg)
Image rectification: application
3D reconstruction becomes easier
a
b
c
![Page 41: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/41.jpg)
Panoramic view
![Page 42: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/42.jpg)
Panoramic view: challenges
![Page 43: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/43.jpg)
Elementary stereo geometry in the rectified image configuration
The depth of a 3D scene point can be calculated using the disparity measure
![Page 44: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/44.jpg)
Elementary stereo geometry in the rectified image configuration
The depth of a 3D scene point can be calculated using the disparity measure
![Page 45: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/45.jpg)
Elementary stereo geometry in the rectified image configuration
The depth of a 3D scene point can be calculated using the disparity measure
![Page 46: 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics:](https://reader036.fdocuments.us/reader036/viewer/2022062520/568161cd550346895dd1bb2e/html5/thumbnails/46.jpg)
Elementary stereo geometry in the rectified image configuration
The depth of a 3D scene point can be calculated using the disparity measure