Lecture 20: Structure from Motion
description
Transcript of Lecture 20: Structure from Motion
![Page 1: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/1.jpg)
Lecture 20: Structure from Motion
![Page 2: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/2.jpg)
Announcements
Proposals due today or Wednesday if you need an extra day or two
I will schedule project meetings next week
![Page 3: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/3.jpg)
Today
We've talked about finding corresponding points in images
If we know the projection matrices of the cameras, we can recover the locations of points in space
What if we only know the correspondences? Known as “Structure from Motion”
![Page 4: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/4.jpg)
Today
Today we will be talking about solving the problem under a simplifying assumption
To understand the assumption, we'll first talk about a simplified model of imaging
![Page 5: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/5.jpg)
The equation of projection
(Image from Slides by Forsyth)
We know:
so
![Page 6: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/6.jpg)
The equation of projection
(Image from Slides by Forsyth)
We know:
so
Makes things hard!
![Page 7: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/7.jpg)
Weak perspective Issue
perspective effects, but not over the scale of individual objects
collect points into a group at about the same depth, then divide each point by the depth of its group
Adv: easy Disadv: wrong
(Image from Slides by Forsyth)
Effectively dividing by a constant z
![Page 8: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/8.jpg)
Affine Model
The projection equation can be written as
No division! Okay approximation when variation in depth is
small relative to the overall depth of the object
3D Coordinate
![Page 9: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/9.jpg)
Basic problem Given n fixed points observed by m affine
cameras we can say that for each point
For large enough m and n this is solvable Up to an ambiguity If M and P are a solution, so is
2x4 matrix
Invertible 3x3 matrix
![Page 10: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/10.jpg)
Affine Structure and Motion from Two Images
Projection equations
![Page 11: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/11.jpg)
Leads to condition
Take advantage of affine ambiguity (see text), we can rewrite this as
![Page 12: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/12.jpg)
Which is
One equation per set of correspondences Can solve with 4 sets of corresponding (u,v)
and (u',v') Given new correspondence, solve
![Page 13: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/13.jpg)
What if I have multiple images?
Basic Equations
If I stack the m instances across cameras
![Page 14: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/14.jpg)
Since I'm tracking multiple points
Can stack these into a matrix
(from Forsyth and Ponce)
![Page 15: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/15.jpg)
Side Trip: SVD
(from Forsyth and Ponce)
![Page 16: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/16.jpg)
More properties of SVD
![Page 17: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/17.jpg)
Back to Recovering Structure and Motion
D is a product of a 2mx3 matrix and 3xn matrix Rank 3
So, using SVD
![Page 18: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/18.jpg)
Using the SVD
If
We claim that
![Page 19: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/19.jpg)
Results (Tomasi and Kanade '92)
Input
![Page 20: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/20.jpg)
![Page 21: Lecture 20: Structure from Motion](https://reader033.fdocuments.us/reader033/viewer/2022061418/56813a68550346895da260d8/html5/thumbnails/21.jpg)