15 cv mil_models_for_transformations
-
Upload
zukun -
Category
Technology
-
view
216 -
download
0
Transcript of 15 cv mil_models_for_transformations
![Page 1: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/1.jpg)
Computer vision: models, learning and inference
Chapter 15
Models for transformations
Please send errata to [email protected]
![Page 2: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/2.jpg)
Structure
22Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Transformation models
• Learning and inference in transformation models
• Three problems– Exterior orientation
– Calibration
– Reconstruction
• Properties of the homography
• Robust estimation of transformations
• Applications
![Page 3: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/3.jpg)
Transformation models
33Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Consider viewing a planar scene
• There is now a 1 to 1 mapping between points on the plane and points in the image
• We will investigate models for this 1 to 1 mapping– Euclidean
– Similarity
– Affine transform
– Homography
![Page 4: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/4.jpg)
Motivation: augmented reality tracking
44Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 5: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/5.jpg)
• Consider viewing a fronto-parallel plane at a known distance D.
• In homogeneous coordinates the imaging equations are:
Euclidean Transformation
55Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
3D rotation matrixbecomes 2D (in plane)
Plane at known distance D
Point is on plane (w=0)
![Page 6: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/6.jpg)
Euclidean transformation
66Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Simplifying
• Rearranging the last equation
![Page 7: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/7.jpg)
Euclidean transformation
77Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Simplifying
• Rearranging the last equation
![Page 8: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/8.jpg)
Euclidean transformation
88Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Pre-multiplying by inverse of (modified) intrinsic matrix
![Page 9: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/9.jpg)
Euclidean transformation
99Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
or for short:
Homogeneous:
Cartesian:
For short:
![Page 10: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/10.jpg)
Similarity Transformation
1010Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Consider viewing fronto-parallel plane at unknown distance D
• By same logic as before we have
• Premultiplying by inverse of intrinsic matrix
![Page 11: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/11.jpg)
Similarity Transformation
1111Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Simplifying:
Multiply each equation by :
![Page 12: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/12.jpg)
Similarity Transformation
1212Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Simplifying:
Incorporate the constants by defining:
![Page 13: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/13.jpg)
Similarity Transformation
1313Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Homogeneous:
Cartesian:
For short:
![Page 14: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/14.jpg)
Affine Transformation
1414Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Homogeneous:
Cartesian:
For short:
![Page 15: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/15.jpg)
Affine Transform
1515Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Affine transform describes mapping well when the depth variation within the
planar object is small and the camera is far away
When variation in depth is comparable to distance to object then the affine
transformation is not a good model. Here we need the homography
![Page 16: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/16.jpg)
Projective transformation /collinearity / homography
1616Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Start with basic projection equation:
Combining these two matrices we get:
![Page 17: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/17.jpg)
Homography
1717Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Homogeneous:
Cartesian:
For short:
![Page 18: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/18.jpg)
Adding noise
1818Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• In the real world, the measured image positions are uncertain.
• We model this with a normal distribution
• e.g.
![Page 19: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/19.jpg)
Structure
1919Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Transformation models
• Learning and inference in transformation models
• Three problems– Exterior orientation
– Calibration
– Reconstruction
• Properties of the homography
• Robust estimation of transformations
• Applications
![Page 20: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/20.jpg)
Learning and inference problems
2020Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Learning – take points on plane and their projections into the image and learn transformation parameters
• Inference – take the projection of a point in the image and establish point on plane
![Page 21: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/21.jpg)
Learning and inference problems
2121Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 22: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/22.jpg)
Learning transformation models
2222Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Maximum likelihood approach
• Becomes a least squares problem
![Page 23: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/23.jpg)
Learning Euclidean parameters
2323Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Solve for transformation:
Remaining problem:
![Page 24: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/24.jpg)
• This is an orthogonal Procrustes problem. To solve:
• Compute SVD
• And then set
Learning Euclidean parameters
2424Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Has the general form:
![Page 25: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/25.jpg)
Learning similarity parameters
2525Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Solve for rotation matrix as before
• Solve for translation and scaling factor
![Page 26: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/26.jpg)
Learning affine parameters
2626Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Affine transform is linear
• Solve using least-squares solution
![Page 27: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/27.jpg)
Learning homography parameters
2727Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Homography is not linear – cannot be solved in closed form. Convert to other homogeneous coordinates
• Both sides are 3x1 vectors; should be parallel and so cross product will be zero
![Page 28: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/28.jpg)
• Write out these equations in full
• There are only 2 independent equations here – use a minimum of four points to build up a set of equations
Learning homography parameters
2828Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 29: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/29.jpg)
• These equations have the form , which we need to solve with the constraint
• This is a minimum direction problem– Compute SVD
– Take last column of
• Then use non-linear optimization
Learning homography parameters
2929Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 30: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/30.jpg)
Inference problems
3030Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Given point x* in image, find position w* on object
![Page 31: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/31.jpg)
In the absence of noise, we have the relation , or in homogeneous coordinates
Inference
3131Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
To solve for the points, we simply invert this relation
![Page 32: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/32.jpg)
Structure
3232Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Transformation models
• Learning and inference in transformation models
• Three problems– Exterior orientation
– Calibration
– Reconstruction
• Properties of the homography
• Robust estimation of transformations
• Applications
![Page 33: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/33.jpg)
Problem 1: exterior orientation
3333Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 34: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/34.jpg)
Problem 1: exterior orientation
3434Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Writing out the camera equations in full
• Estimate the homography from matched points• Factor out the intrinsic parameters
![Page 35: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/35.jpg)
Problem 1: exterior orientation
3535Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• To estimate the first two columns of rotation matrix, we compute this singular value decomposition
• Then we set
![Page 36: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/36.jpg)
Problem 1: exterior orientation
3636Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Find the last column using the cross product of first two columns
• Make sure the determinant is 1 – if it is -1 then multiply last column by -1.
• Find translation scaling factor between old and new values
• Finally, set
![Page 37: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/37.jpg)
Structure
3737Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Transformation models
• Learning and inference in transformation models
• Three problems– Exterior orientation
– Calibration
– Reconstruction
• Properties of the homography
• Robust estimation of transformations
• Applications
![Page 38: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/38.jpg)
Problem 2: calibration
3838Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 39: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/39.jpg)
Structure
3939Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 40: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/40.jpg)
One approach (not very efficient) is to alternately
• Optimize extrinsic parameters for fixed intrinsic
• Optimize intrinsic parameters for fixed extrinsic
Calibration
40Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Then use non-linear optimization.
![Page 41: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/41.jpg)
Structure
4141Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Transformation models
• Learning and inference in transformation models
• Three problems– Exterior orientation
– Calibration
– Reconstruction
• Properties of the homography
• Robust estimation of transformations
• Applications
![Page 42: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/42.jpg)
Problem 3 - reconstruction
4242Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Transformation between plane and image:
Point in frame of reference of plane:Point in frame of reference of camera
![Page 43: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/43.jpg)
Structure
4343Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Transformation models
• Learning and inference in transformation models
• Three problems– Exterior orientation
– Calibration
– Reconstruction
• Properties of the homography
• Robust estimation of transformations
• Applications
![Page 44: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/44.jpg)
Transformations between images
4444Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• So far we have considered transformations between the image and a plane in the world
• Now consider two cameras viewing the same plane
• There is a homography between camera 1 and the plane and a second homography between camera 2 and the plane
• It follows that the relation between the two images is also a homography
![Page 45: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/45.jpg)
Properties of the homography
4545Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Homography is a linear transformation of a ray
Equivalently, leave rays and linearly transform image plane – all images formed by all planes that cut the same ray bundle are related by homographies.
![Page 46: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/46.jpg)
Camera under pure rotation
4646Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Special case is camera under pure rotation.Homography can be showed to be
![Page 47: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/47.jpg)
Structure
4747Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Transformation models
• Learning and inference in transformation models
• Three problems– Exterior orientation
– Calibration
– Reconstruction
• Properties of the homography
• Robust estimation of transformations
• Applications
![Page 48: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/48.jpg)
Robust estimation
4848Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Least squares criterion is not robust to outliers
For example, the two outliers here cause the fitted line to be quite wrong.
One approach to fitting under these circumstances is to use RANSAC – “Random sampling by consensus”
![Page 49: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/49.jpg)
RANSAC
4949Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 50: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/50.jpg)
RANSAC
5050Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 51: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/51.jpg)
Fitting a homography with RANSAC
5151Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Original images Initial matches Inliers from RANSAC
![Page 52: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/52.jpg)
Piecewise planarity
5252Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Many scenes are not planar, but are nonetheless piecewise planarCan we match all of the planes to one another?
![Page 53: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/53.jpg)
Approach 1 – Sequential RANSAC
5353Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Problems: greedy algorithm and no need to be spatially coherent
![Page 54: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/54.jpg)
Approach 2 – PEaRL(propose, estimate and re-learn)
5454Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Associate label l which indicates which plane we are in• Relation between points xi in image1 and yi in image 2
• Prior on labels is a Markov random field that encourages nearby labels to be similar
• Model solved with variation of alpha expansion algorithm
![Page 55: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/55.jpg)
Approach 2 – PEaRL
5555Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 56: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/56.jpg)
Structure
5656Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Transformation models
• Learning and inference in transformation models
• Three problems– Exterior orientation
– Calibration
– Reconstruction
• Properties of the homography
• Robust estimation of transformations
• Applications
![Page 57: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/57.jpg)
Augmented reality tracking
5757Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 58: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/58.jpg)
Fast matching of keypoints
5858Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 59: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/59.jpg)
Visual panoramas
5959Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 60: 15 cv mil_models_for_transformations](https://reader033.fdocuments.us/reader033/viewer/2022060200/5597df201a28ab5e388b4738/html5/thumbnails/60.jpg)
Conclusions
• Mapping between plane in world and camera is one-to-one
• Takes various forms, but most general is the homography
• Revisited exterior orientation, calibration, reconstruction problems from planes
• Use robust methods to estimate in the presence of outliers
6060Computer vision: models, learning and inference. ©2011 Simon J.D. Prince