Camera Projection Matrix - University of Pennsylvania
Transcript of Camera Projection Matrix - University of Pennsylvania
![Page 1: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/1.jpg)
Camera Projection Matrix
Slides by HyunSoo Park
![Page 2: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/2.jpg)
![Page 3: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/3.jpg)
![Page 4: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/4.jpg)
x XR 3 3 tK
=
x x
x y
Xu f s p r r r t
Yz v f p r r r t
Zr r r t
img 11 12 13 1
img 21 22 23 2
31 32 33 31 11
![Page 5: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/5.jpg)
http://www.joshuanava.biz/perspective/in-other-words-the-observer-simply-points-in-the-same-direction-as-the-lines-in-order-
to-find-their-vanishing-point.html
First person coordinate system World coordinate system
![Page 6: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/6.jpg)
Camera Model
![Page 7: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/7.jpg)
Ground plane
Camera Model (1st Person Perspective)Camera
![Page 8: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/8.jpg)
Ground plane
Camera Model (1st Person Perspective)Camera
1 1
x
y
u f p X
v f p Y
Z
Recall camera projection matrix:
+
Camera intrinsic parameter
: metric space to pixel space
![Page 9: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/9.jpg)
Ground plane
Camera Model (1st Person Perspective)Camera
1 1
x
y
u f p X
v f p Y
Z
K
Recall camera projection matrix:
+
Camera intrinsic parameter
: metric space to pixel space
![Page 10: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/10.jpg)
Ground plane
Camera Model (1st Person Perspective)Camera
1 1
x
y
u f p X
v f p Y
Z
K
Recall camera projection matrix:
3D world (metric)2D image (pix)
![Page 11: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/11.jpg)
Ground plane
Camera Model (1st Person Perspective)Camera
1 1
x
y
u f p X
v f p Y
Z
K
Recall camera projection matrix:
3D world (metric)2D image (pix)
Origin at camera
-1
1
K XC
u X
v Y
Z
CX
![Page 12: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/12.jpg)
Ground plane
Camera Model (multiple 1st Person Perspective)Camera
1 1
x
y
u f p X
v f p Y
Z
K
Recall camera projection matrix:
3D world (metric)2D image (pix)
Origin at camera
1
1 1
-1
1 1 C
11
K X
u X
v Y
Z
Where is origin?1C
X
2CX
2
2 2
-1
2 2 C
21
K X
u X
v Y
Z
![Page 13: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/13.jpg)
1 1
x
y
u f p X
v f p Y
Z
K
Recall camera projection matrix:
3D world (metric)2D image (pix)
X
Ground plane
Camera
X3D world
Origin at world coordinate
Camera Model (3rd Person Perspective)
![Page 14: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/14.jpg)
1 1
x
y
u f p X
v f p Y
Z
K
Recall camera projection matrix:
3D world (metric)2D image (pix)
X
Ground plane
Camera
3D world
Origin at world coordinate
Coordinate Transform (Rotation matrix)
X
1C
WR
2C
WR
1
x1 x2 x3
C y1 y2 y3
z1 z2 z3
=
X X
r r r
r r r
r r r
Coordinate transformation from world to camera:
X
3D world Camera
1CX
![Page 15: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/15.jpg)
Coordinate Transform (Rotation)
2D coordinate transform:
World
x
y
= ( , )X x y
![Page 16: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/16.jpg)
Coordinate Transform (Rotation)
= ( , )X x y
2D coordinate transform:
World
Camera
C C C= ( , )X x y
C
C
cos sin=
sin cos
x
y
x
y
?
![Page 17: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/17.jpg)
Coordinate Transform (Rotation)
= ( , )X x y
2D coordinate transform:
World
Camera
C C C= ( , )X x y
C
C
cos sin=
sin cos
x
y
x
y
![Page 18: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/18.jpg)
Coordinate Transform (Rotation)
= ( , )X x y
2D coordinate transform:
World
Camera
C C C= ( , )X x y
C
C
cos sin=
sin cos
x
y
x
y
2 2cos sindet = cos sin 1
sin cos
![Page 19: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/19.jpg)
Coordinate Transform (Rotation)
= ( , )X x y
2D coordinate transform:
World
Camera
C C C= ( , )X x y
C
C
cos sin=
sin cos
x
y
x
y
rX
rX
: x axis of camera seen from the worldrX
![Page 20: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/20.jpg)
Coordinate Transform (Rotation)
= ( , )X x y
2D coordinate transform:
World
Camera
C C C= ( , )X x y
C
C
cos sin=
sin cos
x
y
x
y
rX
rX
: x axis of camera seen from the worldrX
: y axis of camera seen from the worldYr
Yr
rY
![Page 21: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/21.jpg)
Coordinate Transform (Rotation)
= ( , )X x y
2D coordinate transform:
World
Camera
C C C= ( , )X x y
C
C
cos sin=
sin cos
x
y
x
y
: x axis of world seen from the camera1r
: y axis of world seen from the camera2r
r1
r1
2r 2
r
![Page 22: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/22.jpg)
Ground plane
Camera
3D world
Origin at world coordinate
Coordinate Transform (Rotation)
X
1C
WR
Coordinate transformation from world to camera:
= ( , , )X x y z
World
x1 x2 x3
C
y1 y2 y3 W
z1 z
C
2 z3
= =
X XRX
r r r
r r r
r r r
?
![Page 23: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/23.jpg)
Ground plane
Camera
3D world
Origin at world coordinate
Coordinate Transform (Rotation)
X
1C
WR
x1 x2 x3
C
y1 y2 y3 W
z1 z
C
2 z3
= =
X XRX
r r r
r r r
r r r
Coordinate transformation from world to camera:
= ( , , )X x y z
Camera
C C C C= ( , , )X x y z
?
![Page 24: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/24.jpg)
Ground plane
Camera
3D world
Origin at world coordinate
Coordinate Transform (Rotation)
X
1C
WR
x1 x2 x3
C
y1 y2 y3 W
z1 z
C
2 z3
= =
X XRX
r r r
r r r
r r r
Coordinate transformation from world to camera:
= ( , , )X x y z
Camera
C C C C= ( , , )X x y z
![Page 25: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/25.jpg)
Ground plane
Camera
3D world
Origin at world coordinate
Coordinate Transform (Rotation)
X
1C
WR
x1 x2 x3
C
y1 y2 y3 W
z1 z
C
2 z3
= =
X XRX
r r r
r r r
r r r
Coordinate transformation from world to camera:
= ( , , )X x y z
Camera
C C C C= ( , , )X x y z
Degree of freedom?
![Page 26: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/26.jpg)
Ground plane
Camera
3D world
Origin at world coordinate
Coordinate Transform (Rotation)
X
1C
WR
Coordinate transformation from world to camera:
= ( , , )X x y z
Camera
C C C C= ( , , )X x y z
World
RC
WSO(3)
x1 x2 x3
C
y1 y2 y3 W
z1 z
C
2 z3
= =
X XRX
r r r
r r r
r r r
T
C C C
W W 3 W, det 1 R R I R• Orthogonal matrix
Degree of freedom?
![Page 27: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/27.jpg)
Ground plane
Camera
3D world
Origin at world coordinate
Coordinate Transform (Rotation)
X
1C
WR
Coordinate transformation from world to camera:
= ( , , )X x y z
Camera
C C C C= ( , , )X x y z
World
RC
WSO(3)
x1 x2 x3
C
y1 y2 y3 W
z1 z
C
2 z3
= =
X XRX
r r r
r r r
r r r
T
C C C
W W 3 W, det 1 R R I R• Orthogonal matrix
• Right hand rule
rX
Yr
Zr
rX
Yr
Z X Y= r r r
rX
Yr
Zr
![Page 28: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/28.jpg)
Ground plane
Camera
3D world
Origin at world coordinate
Coordinate Transform (Rotation)
X
1C
WR
Coordinate transformation from world to camera:
Camera
World
RC
WSO(3)
x1 x2 x3
C
y1 y2 y3 W
z1 z
C
2 z3
= =
X XRX
r r r
r r r
r r r
T
C C C
W W 3 W, det 1 R R I R• Orthogonal matrix
• Right hand rule
rX
Yr
Zr
rX
Yr
Z X Y= r r r
rX
: camera x axis seen
from the world coord.Zr
![Page 29: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/29.jpg)
Ground plane
Camera
3D world
Origin at world coordinate
Coordinate Transform (Rotation)
X
1C
WR
Coordinate transformation from world to camera:
Camera
World
RC
WSO(3)
x1 x2 x3
C
y1 y2 y3 W
z1 z
C
2 z3
= =
X XRX
r r r
r r r
r r r
T
C C C
W W 3 W, det 1 R R I R• Orthogonal matrix
• Right hand rule
Yr
Zr
rX
Yr
Z X Y= r r r
: camera y axis seen
from the world coord.
rX
Yr
![Page 30: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/30.jpg)
Ground plane
Camera
3D world
Origin at world coordinate
Coordinate Transform (Rotation)
X
1C
WR
Coordinate transformation from world to camera:
Camera
World
RC
WSO(3)
x1 x2 x3
C
y1 y2 y3 W
z1 z
C
2 z3
= =
X XRX
r r r
r r r
r r r
T
C C C
W W 3 W, det 1 R R I R• Orthogonal matrix
• Right hand rule
1r
2r
3 1 2= r r r
1r : world x axis seen
from the camera
coord.
r1 2
r3r
![Page 31: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/31.jpg)
Ground plane
Camera
3D world
Origin at world coordinate
Coordinate Transform (Rotation)
X
1C
WR
Coordinate transformation from world to camera:
Camera
World
RC
WSO(3)
x1 x2 x3
C
y1 y2 y3 W
z1 z
C
2 z3
= =
X XRX
r r r
r r r
r r r
T
C C C
W W 3 W, det 1 R R I R• Orthogonal matrix
• Right hand rule
1r
2r
3 1 2= r r r
1r : world x axis seen
from the camera
coord.
r1 2
r3r
2r : world y axis seen
from the camera
coord.
3r
![Page 32: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/32.jpg)
x1 x2 x3
y1 y2 y3
z1 z2
C
z3
C
C1 1
1
x
y
x
y
u f p
v f p
f p r r r
f p r r r
r
X
Y
Zr r
X
Y
Z
Ground plane
Camera
3D world
Origin at world coordinate
Camera Projection (Pure Rotation)
X
1C
WR
Coordinate transformation from world to camera:
Camera
World
x1 x2 x3
C
y1 y2 y3 W
z1 z
C
2 z3
= =
X XRX
r r r
r r r
r r r
1r : world x axis seen
from the camera
coord.
r1 2
r3r
2r : world y axis seen
from the camera
coord.
3r
K
Camera projection of world point:
![Page 33: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/33.jpg)
x1 x2 x3
y1 y2 y3
z1 z2
C
z3
C
C1 1
1
x
y
x
y
u f p
v f p
f p r r r
f p r r r
r
X
Y
Zr r
X
Y
Z
Ground plane
Camera
3D world
Origin at world coordinate
Camera Projection (Pure Rotation)
X
1C
WR
Coordinate transformation from world to camera:
Camera
World
x1 x2 x3
C
y1 y2 y3 W
z1 z
C
2 z3
= =
X XRX
r r r
r r r
r r r
1r : world x axis seen
from the camera
coord.
r1 2
r3r
2r : world y axis seen
from the camera
coord.
3r
K
Camera projection of world point:
K
![Page 34: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/34.jpg)
Ground plane
Camera
3D world
Origin at world coordinate
Camera Projection (Euclidean Transform)
X
Coordinate transformation from world to camera:
x1 x2 x3
C
y1 y2 y3 W
z1 z
C
2 z3
= =
X XRX
r r r
r r r
r r r
C C
W,R t
World
= ( , , )X x y z
![Page 35: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/35.jpg)
Ground plane
Camera
3D world
Origin at world coordinate
Camera Projection (Euclidean Transform)
X
Coordinate transformation from world to camera:
Camera
x1 x2 x3 x
C C
W y1 y2 y3 y
z1 z2 z3 z
C= =
1
RX
XX t
r r r t
r r r t
r r r t
World
= ( , , )X x y zC C C C
= ( , , )X x y z
C C
W,R t
![Page 36: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/36.jpg)
Ground plane
Camera
3D world
Origin at world coordinate
Camera Projection (Euclidean Transform)
X
Coordinate transformation from world to camera:
Camera
x1 x2 x3 x
C C
W y1 y2 y3 y
z1 z2 z3 z
C= =
1
RX
XX t
r r r t
r r r t
r r r t
World
= ( , , )X x y zC C C C
= ( , , )X x y z
C C
W,R t
Ct
CtWhere is translation from world to camera
seen from camera.
![Page 37: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/37.jpg)
Ground plane
Camera
3D world
Origin at world coordinate
Geometric Interpretation
X
Coordinate transformation from world to camera:
Camera
x1 x2 x3 x
C C
W y1 y2 y3 y
z1 z2 z3 z
C= =
1
RX
XX t
r r r t
r r r t
r r r t
World
= ( , , )X x y zC C C C
= ( , , )X x y z
C C
W,R t
Ct
Ctwhere is translation from world to camera
seen from camera.
Rotate and then, translate.
![Page 38: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/38.jpg)
Ground plane
Camera
3D world
Origin at world coordinate
Geometric Interpretation
X
Coordinate transformation from world to camera:
Camera
x1 x2 x3 x
C C
W y1 y2 y3 y
z1 z2 z3 z
C= =
1
RX
XX t
r r r t
r r r t
r r r t
World
= ( , , )X x y zC C C C
= ( , , )X x y z
C C
W,R t
Ctwhere is translation from world to camera
seen from camera.
Rotate and then, translate.
x1 x2 x3 x
C
W y1 y2 y3 y
z1 z2 z3 z
C
1
= = 11
1
XXR CX
r r r -C
r r r -C
r r r -C
cf) Translate and then, rotate.
where is translation from world to camera
seen from world.C
C
![Page 39: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/39.jpg)
Ground plane
Camera
3D world
Origin at world coordinate
Camera Projection Matrix
X
Coordinate transformation from world to camera:
Camera
x1 x2 x3 x
C C
W y1 y2 y3 y
z1 z2 z3 z
C= =
1
RX
XX t
r r r t
r r r t
r r r t
World
= ( , , )X x y zC C C C
= ( , , )X x y z
C C
W,R t
C
x1 x2 x3 x
y1 y2 y3 y
z1 z2 z3 z
C
C
C1
1
1
1
x
y
x
y
u f p
v f p
f p r r r t
f p r r r t
r r r t
X
Y
Z
X
Y
Z
K
Camera projection of world point:
K C
WR
Ct
![Page 40: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/40.jpg)
Image Projection: Sanity Check
r
= 0
0
C
-r 0 -1 0
= 0 0 -1
1 0 0
R
![Page 41: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/41.jpg)
Image Projection
r
r
= 0
0
C
-r 0 -1 0
= 0 0 -1
1 0 0
R
![Page 42: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/42.jpg)
Image Projection
r
r
= 0
0
C
-r 0 -1 0
= 0 0 -1
1 0 0
R
cos
= sin
0
C
r
r
-sin cos 0
= 0 0 -1
-cos -sin 0
R
![Page 43: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/43.jpg)
Image Projection K = [200 0 100;0 200 100;0 0 1];
radius = 5;
theta = 0:0.02:2*pi;
for i = 1 : length(theta)camera_offset = [radius*cos(theta(i)); radius*sin(theta(i)); 0];camera_center = camera_offset + center_of_mass';
rz = camera_center-center_of_mass';rz = rz / norm(rz);ry = [0 0 1]';rx = Vec2Skew(ry)*rz; % cross productR = [rx'; ry'; rz'];C = camera_center;P = K * R * [ eye(3) -C];
proj = [];for j = 1 : size(sqaure_point,1)
u = P * [sqaure_point(j,:)';1];proj(j,:) = u'/u(3);
end end
![Page 44: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/44.jpg)
Ground plane
Camera
3D world
Geometric Interpretation
C C
W,R t
x1 x2 x3 x
y1 y2 y3 y
z1 z2 z3 z 11 1
x
y
u f p r r r t
v f p r r r t
X
Y
Zr r r t
Camera projection of world point:
K C
WR
Ct1
=
XP
11 12 13 14
21 22 23 24
31 32 33 341
1
Xu p p p p
Yv p p p p
Zp p p p
What does each number mean?
![Page 45: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/45.jpg)
Ground plane
Camera
3D world
Geometric Interpretation
C C
W,R t
x1 x2 x3 x
y1 y2 y3 y
z1 z2 z3 z 11 1
x
y
u f p r r r t
v f p r r r t
X
Y
Zr r r t
Camera projection of world point:
K C
WR
Ct1
=
XP
What is point at infinity in world x direction?
X
0
0
1
X
11 12 13 14
21 22 23 24
31 32 33 34
0
01
1
u p p p p
v p p p p
p p p p
![Page 46: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/46.jpg)
Ground plane
Camera
3D world
Geometric Interpretation
C C
W,R t
x1 x2 x3 x
y1 y2 y3 y
z1 z2 z3 z 11 1
x
y
u f p r r r t
v f p r r r t
X
Y
Zr r r t
Camera projection of world point:
K C
WR
Ct1
=
XP
What is point at infinity in world x direction?
X
0
0
1
X
This point is at infinite but finite in image.
Xu 11 12 13 14
21 22 23 24
31 32 33 34
0
01
1
u p p p p
v p p p p
p p p p
![Page 47: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/47.jpg)
Point at infinity
![Page 48: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/48.jpg)
Ground plane
Camera
3D world
Geometric Interpretation
C C
W,R t
x1 x2 x3 x
y1 y2 y3 y
z1 z2 z3 z 11 1
x
y
u f p r r r t
v f p r r r t
X
Y
Zr r r t
Camera projection of world point:
K C
WR
Ct1
=
XP
What is point at infinity in world x direction?
X
0
0
1
X
This point is at infinite but finite in image.
Xu 11 12 13 14
21 22 23 24
31 32 33 34
0
01
1
u p p p p
v p p p p
p p p p
![Page 49: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/49.jpg)
Ground plane
Camera
3D world
Geometric Interpretation
C C
W,R t
x1 x2 x3 x
y1 y2 y3 y
z1 z2 z3 z 11 1
x
y
u f p r r r t
v f p r r r t
X
Y
Zr r r t
Camera projection of world point:
K C
WR
Ct1
=
XP
X
0
0
1
X
Xu
11 14 11
31 34 31
21 24 21
31 34 31
+lim
+
+lim
+
X
X
p X p pu
p X p p
p X p pv
p X p p
11 12 13 14
21 22 23 24
31 32 33 34
0
01
1
u p p p p
v p p p p
p p p p
![Page 50: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/50.jpg)
Ground plane
Camera
3D world
Geometric Interpretation
C C
W,R t
x1 x2 x3 x
y1 y2 y3 y
z1 z2 z3 z 11 1
x
y
u f p r r r t
v f p r r r t
X
Y
Zr r r t
Camera projection of world point:
K C
WR
Ct1
=
XP
X
0
0
1
X
Xu
11 14 11
31 34 31
21 24 21
31 34 31
+lim
+
+lim
+
X
X
p X p pu
p X p p
p X p pv
p X p p
11 12 13 14
21 22 23 24
31 32 33 34
0
01
1
u p p p p
v p p p p
p p p p
11
X 21
311
u
u p
v p
p
11
X 21
31
u
p
p
p
![Page 51: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/51.jpg)
Ground plane
Camera
3D world
Geometric Interpretation
C C
W,R t
x1 x2 x3 x
y1 y2 y3 y
z1 z2 z3 z 11 1
x
y
u f p r r r t
v f p r r r t
X
Y
Zr r r t
Camera projection of world point:
K C
WR
Ct1
=
XP
Xu
12 14 12
32 34 32
22 24 22
32 34 32
+lim
+
+lim
+
X
X
p Y p pu
p Y p p
p Y p pv
p Y p p
11 12 13 14
21 22 23 24
31 32 33 34
0
01
1
u p p p p
v p p p p
p p p p
12
Y 22
321
u
u p
v p
p
12
Y 22
32
u
p
p
p
Yu
![Page 52: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/52.jpg)
Ground plane
Camera
3D world
Geometric Interpretation
C C
W,R t
x1 x2 x3 x
y1 y2 y3 y
z1 z2 z3 z 11 1
x
y
u f p r r r t
v f p r r r t
X
Y
Zr r r t
Camera projection of world point:
K C
WR
Ct1
=
XP
13 14 13
33 34 33
23 24 23
33 34 33
+lim
+
+lim
+
X
X
p Z p pu
p Z p p
p Z p pv
p Z p p
11 12 13 14
21 22 23 24
31 32 33 34
0
0
11
u p p p p
v p p p p
p p p p
13
Z 23
331
u
u p
v p
p
13
Z 23
33
u
p
p
p
![Page 53: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/53.jpg)
Celestial Navigation
Far far away: point at infinity
Earth My location
![Page 54: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/54.jpg)
Practiceimg
m
ccd
3840= = 0.00153 1224pix
0.0048
Wf f
W
img
x
3840= = 1920pix
2 2
Wp
img
y
2160= = 1080pix
2 2
Hp
0.0070
0.7520
-0.2738
C
-0.8496 0.0498 0.5731
-0.3216 -0.8203 -0.4067
0.4180 -0.5299 0.6835
R
![Page 55: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/55.jpg)
Practiceimg
m
ccd
3840= = 0.00153 1224pix
0.0048
Wf f
W
img
x
3840= = 1920pix
2 2
Wp
img
y
2160= = 1080pix
2 2
Hp
0.0070
0.7520
-0.2738
C
-0.8496 0.0498 0.5731
-0.3216 -0.8203 -0.4067
0.4180 -0.5299 0.6835
R
3
-0.2374 -0.9565 2.0138 1.2723
0.0578 -1.5763 0.2404 1.2508
0.0004 -0.0005 0.0007 0.0006
P= KR I C
![Page 56: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/56.jpg)
img
m
ccd
3840= = 0.00153 1224pix
0.0048
Wf f
W
img
x
3840= = 1920pix
2 2
Wp
img
y
2160= = 1080pix
2 2
Hp
0.0070
0.7520
-0.2738
C
-0.8496 0.0498 0.5731
-0.3216 -0.8203 -0.4067
0.4180 -0.5299 0.6835
R
3
-0.2374 -0.9565 2.0138 1.2723
0.0578 -1.5763 0.2404 1.2508
0.0004 -0.0005 0.0007 0.0006
P= KR I C
R = [-0.8496 0.0498 0.5731-0.3216 -0.8203 -0.40670.4180 -0.5299 0.6835];
C = [0.00700.7520-0.2738];
P = K * R * [eye(3) -C]u_x = P(1:2,1)/P(3,1)u_y = P(1:2,2)/P(3,2)u_z = P(1:2,3)/P(3,3)
[-0.2374 0.0578]/ 0.0004
=
[-593.5 144.5]
![Page 57: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/57.jpg)
Practiceimg
m
ccd
3840= = 0.00153 1224pix
0.0048
Wf f
W
img
x
3840= = 1920pix
2 2
Wp
img
y
2160= = 1080pix
2 2
Hp
0.0070
0.7520
-0.2738
C
-0.8496 0.0498 0.5731
-0.3216 -0.8203 -0.4067
0.4180 -0.5299 0.6835
R
3
-0.2374 -0.9565 2.0138 1.2723
0.0578 -1.5763 0.2404 1.2508
0.0004 -0.0005 0.0007 0.0006
P= KR I C
R = [-0.8496 0.0498 0.5731-0.3216 -0.8203 -0.40670.4180 -0.5299 0.6835];
C = [0.00700.7520-0.2738];
P = K * R * [eye(3) -C]u_x = P(1:2,1)/P(3,1)u_y = P(1:2,2)/P(3,2)u_z = P(1:2,3)/P(3,3)
[-0.9565 -1.5763] / -0.0005
=
[1,913 3,152]
![Page 58: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/58.jpg)
img
m
ccd
3840= = 0.00153 1224pix
0.0048
Wf f
W
img
x
3840= = 1920pix
2 2
Wp
img
y
2160= = 1080pix
2 2
Hp
0.0070
0.7520
-0.2738
C
-0.8496 0.0498 0.5731
-0.3216 -0.8203 -0.4067
0.4180 -0.5299 0.6835
R
3
-0.2374 -0.9565 2.0138 1.2723
0.0578 -1.5763 0.2404 1.2508
0.0004 -0.0005 0.0007 0.0006
P= KR I C
R = [-0.8496 0.0498 0.5731-0.3216 -0.8203 -0.40670.4180 -0.5299 0.6835];
C = [0.00700.7520-0.2738];
P = K * R * [eye(3) -C]u_x = P(1:2,1)/P(3,1)u_y = P(1:2,2)/P(3,2)u_z = P(1:2,3)/P(3,3)
[2.0138 0.2404]/0.0007
=
[2877 343]
![Page 59: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/59.jpg)
Z direction in the world coordinate system
![Page 60: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/60.jpg)
Z direction in the world coordinate system
uv
![Page 61: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/61.jpg)
Z direction in the world coordinate system
z point at infinity
uv
: z vanishing pointvz
z =T
0 0 1 0
![Page 62: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/62.jpg)
Calibration via Vanishing Points
![Page 63: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/63.jpg)
v1 v3
v2
v1
v3
K v1 3
K v1 1
![Page 64: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/64.jpg)
Strong perspectivenessWeak perspectiveness
![Page 65: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/65.jpg)
Orthographic CameraAffine camera:
x1 x2 x1 x2
A y1 y2 y1 y2
0 0 0 0
0 0 0 0
1 0 0 0 1 0 0 0 1
P
x x
y y
f p r r f / d p r r
f p r r d f / d p r r
d
1f
Orthographic camera:
X Y0 p p
![Page 66: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/66.jpg)
Orthographic CameraAffine camera:
x1 x2 x1 x2
A y1 y2 y1 y2
0 0 0 0
0 0 0 0
1 0 0 0 1 0 0 0 1
P
x x
y y
f p r r f / d p r r
f p r r d f / d p r r
d
1f
Orthographic camera:
X Y0 p p
x1 x2 x3
O y1 y2 y3
0
0
0 0 0 1
P
r r r
r r r
![Page 67: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/67.jpg)
![Page 68: Camera Projection Matrix - University of Pennsylvania](https://reader031.fdocuments.us/reader031/viewer/2022013000/61c99563d770463be77724e4/html5/thumbnails/68.jpg)
Lens Radial Distortion