M N O P Q R S N T OP Q N P P O U P R V W O P P Q X O P N P ...
CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines)...
Transcript of CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines)...
![Page 1: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/1.jpg)
Single-View Modeling, Part 2
CS5670: Computer VisionNoah Snavely
![Page 2: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/2.jpg)
Announcements
• Midterm due today by the beginning of class
• Professor Snavely out Thursday
– Zhengqi will be in NYC to give a guest lecture
• Project 3 coming out soon
![Page 3: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/3.jpg)
Perspective cues
![Page 4: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/4.jpg)
Perspective cues
![Page 5: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/5.jpg)
Perspective cues
![Page 6: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/6.jpg)
Making measurements in images
![Page 7: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/7.jpg)
Comparing heights
VanishingPoint
![Page 8: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/8.jpg)
Measuring height
1
2
3
4
55.4
2.8
3.3
Camera heightHow high is the camera?
![Page 9: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/9.jpg)
q1
Computing vanishing points (from lines)
• Intersect p1q1 with p2q2
v
p1
p2
q2
Least squares version• Better to use more than two lines and compute the “closest” point of
intersection
• See notes by Bob Collins for one good way of doing this:
– http://www-2.cs.cmu.edu/~ph/869/www/notes/vanishing.txt
![Page 10: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/10.jpg)
Measuring heightwithout a ruler
H R
![Page 11: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/11.jpg)
C
Measuring height without a ruler
ground plane
Compute Z from image measurements• Need more than vanishing points to do this
Z
![Page 12: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/12.jpg)
The cross ratio• A Projective Invariant
– Something that does not change under projective transformations (including perspective projection)
P1
P2
P3
P4
1423
2413
PPPP
PPPP
The cross-ratio of 4 collinear points
Can permute the point ordering
• 4! = 24 different orders (but only 6 distinct values)
This is the fundamental invariant of projective geometry
1
i
i
i
iZ
Y
X
P
3421
2431
PPPP
PPPP
![Page 13: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/13.jpg)
vZ
r
t
b
tvbr
rvbt
Z
Z
image cross ratio
Measuring height
B (bottom of object)
T (top of object)
R (reference point)
ground plane
HC
TBR
RBT
scene cross ratio
1
Z
Y
X
P
1
y
x
pscene points represented as image points as
R
H
R
H
R
![Page 14: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/14.jpg)
Measuring height
RH
vz
r
b
t
R
H
Z
Z
tvbr
rvbt
image cross ratio
Hvvx vy
vanishing line (horizon)
b0
t0
![Page 15: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/15.jpg)
vz
r
b
vx vy
vanishing line (horizon)t0
What if the point on the ground plane b0 is not known?
• Here the person is standing on the box, height of box is known
• Use one side of the box to help find b0 as shown above
b0
t1
b1
Measuring height
v
m0
![Page 16: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/16.jpg)
3D Modeling from a photograph
St. Jerome in his Study, H. Steenwick
![Page 17: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/17.jpg)
3D Modeling from a photograph
![Page 18: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/18.jpg)
3D Modeling from a photograph
Flagellation, Piero della Francesca
![Page 19: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/19.jpg)
3D Modeling from a photograph
video by Antonio Criminisi
![Page 20: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/20.jpg)
3D Modeling from a photograph
![Page 21: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/21.jpg)
Camera calibration• Goal: estimate the camera parameters
– Version 1: solve for projection matrix
ΠXx
1************
ZYX
w
wywx
• Version 2: solve for camera parameters separately
– intrinsics (focal length, principle point, pixel size)
– extrinsics (rotation angles, translation)
– radial distortion
![Page 22: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/22.jpg)
Vanishing points and projection matrix
************
Π 4321 ππππ
1π 2π 3π 4π
T00011 Ππ = vx (X vanishing point)
Z3Y2 ,similarly, vπvπ
origin worldof projection10004 T
Ππ
ovvvΠ ZYX
Not So Fast! We only know v’s up to a scale factor
ovvvΠ ZYX cba
• Can fully specify by providing 3 reference points
![Page 23: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/23.jpg)
Calibration using a reference object
• Place a known object in the scene– identify correspondence between image and scene– compute mapping from scene to image
Issues• must know geometry very accurately
• must know 3D->2D correspondence
![Page 24: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/24.jpg)
Chromaglyphs
Courtesy of Bruce Culbertson, HP Labshttp://www.hpl.hp.com/personal/Bruce_Culbertson/ibr98/chromagl.htm
![Page 25: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/25.jpg)
AR codes
ArUco
![Page 26: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/26.jpg)
Estimating the projection matrix
• Place a known object in the scene– identify correspondence between image and scene– compute mapping from scene to image
![Page 27: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/27.jpg)
Direct linear calibration
![Page 28: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/28.jpg)
Direct linear calibration
Can solve for mij by linear least squares• use eigenvector trick that we used for homographies
![Page 29: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/29.jpg)
Direct linear calibration
• Advantage:
– Very simple to formulate and solve
• Disadvantages:
– Doesn’t tell you the camera parameters
– Doesn’t model radial distortion
– Hard to impose constraints (e.g., known f)
– Doesn’t minimize the right error function
For these reasons, nonlinear methods are preferred
• Define error function E between projected 3D points and image positions
– E is nonlinear function of intrinsics, extrinsics, radial distortion
• Minimize E using nonlinear optimization techniques
![Page 30: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/30.jpg)
Alternative: multi-plane calibration
Images courtesy Jean-Yves Bouguet, Intel Corp.
Advantage• Only requires a plane
• Don’t have to know positions/orientations
• Good code available online! (including in OpenCV)– Matlab version by Jean-Yves Bouget:
http://www.vision.caltech.edu/bouguetj/calib_doc/index.html
– Zhengyou Zhang’s web site: http://research.microsoft.com/~zhang/Calib/
![Page 31: CS5670: Computer Vision - Cornell University · q 1 Computing vanishing points (from lines) •Intersect p 1 q 1 with p 2 q 2 v p 1 p 2 q 2 Least squares version • Better to use](https://reader036.fdocuments.us/reader036/viewer/2022081617/60463f7cb75c0c06f45ca36b/html5/thumbnails/31.jpg)
Some Related Techniques
• Image-Based Modeling and Photo Editing– Mok et al., SIGGRAPH 2001– http://graphics.csail.mit.edu/ibedit/
• Single View Modeling of Free-Form Scenes– Zhang et al., CVPR 2001– http://grail.cs.washington.edu/projects/svm/
• Tour Into The Picture– Anjyo et al., SIGGRAPH 1997– http://koigakubo.hitachi.co.jp/little/DL_TipE.html