Post on 07-Jul-2020
Camera Model
3D object
Lens
CCD sensor
3D Point Projection (Metric Space) 3D point
mf
( , , )X Y Z
: Focal length in meter
Pinhole -fm
3D Point Projection (Metric Space) 3D point
mf
( , , )X Y Z
: Focal length in meter
Pinhole -fm
X
Ztan
3D Point Projection (Metric Space) 3D point
mf
( , , )X Y Z
ccd ccdu v( , )
: Focal length in meter
Pinhole -fm
X
Ztan
ccd X
u f fZ
m mtan
3D Point Projection (Metric Space) 3D point
mf
( , , )X Y Z
: Focal length in meter
Pinhole -fm
X
Ztan
ccd X
u fZ
m
ccd Y
v fZ
m
ccd ccdu v( , )
3D Point Projection (Metric Space) 3D point
mf
( , , )X Y Z
: Focal length in meter
Pinhole -fm
X
Ztan
ccd X
u fZ
m
ccd Y
v fZ
m
ccd ccd X Y
u v f fZ Z
m m( , ) ( , )
2D projection onto CCD plane
( , , ) X Y Z
ccd ccdu v( , )
3D Point Projection (Metric Space) 3D point
mf
( , , )X Y Z
: Focal length in meter
Pinhole -fm
ccd X
u fZ
m
ccd Y
v fZ
m
ccd ccd X Y
u v f fZ Z
m m( , ) ( , )
2D projection onto CCD plane
( , , ) X Y Z
Projection plane
( , )u vccd ccd
fm
ccd ccdu v( , )
3D Point Projection (Metric Space) 3D point
mf
( , , )X Y Z
: Focal length in meter
Pinhole -fm
ccd X
u fZ
m
ccd Y
v fZ
m
ccd ccd X Y
u v f fZ Z
m m( , ) ( , )
2D projection onto CCD plane
( , , ) X Y Z
Projection plane
( , )u vccd ccd
fm
ccd ccdu v( , )
3D Point Projection (Metric Space) 3D point
mf
( , , )X Y Z
: Focal length in meter ccd
Xu f
Zm
ccd Y
v fZ
m
ccd ccd X Y
u v f fZ Z
m m( , ) ( , )
2D projection onto CCD plane
( , , ) X Y Z
Projection plane
( , )u vccd ccd
fm
3D Point Projection (Metric Space) 3D point
mf
( , , )X Y Z
: Focal length in meter ccd
Xu f
Zm
ccd Y
v fZ
m
ccd ccd X Y
u v f fZ Z
m m( , ) ( , )
2D projection onto CCD plane
( , , ) X Y Z
Projection plane
( , )u vccd ccd
fm
( , )u vimg img
img img ccd ccdu v u v( , ) ( , ) ?
Metric Pixel
3D Point Projection (Pixel Space)
CCD sensor (mm) Image (pixel)
3D Point Projection (Pixel Space)
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
u vimg img( , )
3D Point Projection (Pixel Space)
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
ccdw
ccdh
u vimg img( , )
3D Point Projection (Pixel Space)
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
ccdw
ccdh
u vimg img( , )
3D Point Projection (Pixel Space)
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
ccdw
ccdh
(0,0)
Projection of pinhole
u vimg img( , )
3D Point Projection (Pixel Space)
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
ccdw
ccdh
(0,0)
Projection of pinhole
imgw
imgh
u vimg img( , )
3D Point Projection (Pixel Space)
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
ccdw
ccdh
(0,0)
Projection of pinhole
(0,0) imgw
imgh
x yp p( , ) : Image principal point
u vimg img( , )
3D Point Projection (Pixel Space)
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
u vimg img( , )
ccdw
ccdh
(0,0)
Projection of pinhole
(0,0) imgw
imgh
x yp p( , ) : Image principal point
xu pu
w w
imgccd
ccd img
3D Point Projection (Pixel Space)
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
u vimg img( , )
ccdw
ccdh
(0,0)
Projection of pinhole
(0,0) imgw
imgh
x yp p( , ) : Image principal point
xu pu
w w
imgccd
ccd img
yv pv
h h
imgccd
ccd img
3D Point Projection (Pixel Space)
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
u vimg img( , )
ccdw
ccdh
(0,0)
Projection of pinhole
(0,0) imgw
imgh
x yp p( , ) : Image principal point
xu pu
w w
imgccd
ccd img
yv pv
h h
imgccd
ccd img
x
wu u p
w
img
img ccd
ccd
y
hv v p
h
img
img ccd
ccd
3D Point Projection (Pixel Space)
O
( , )u vccd ccd
mf
Projection plane ( , , )X Y Z
Z
ccd ccd m m( , ) ( , )X Y
u v f fZ Z
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
u vimg img( , )
ccdw
ccdh
(0,0)
(0,0) imgw
imgh
x yp p( , )
: Metric projection
3D Point Projection (Pixel Space)
O
( , )u vccd ccd
mf
Projection plane ( , , )X Y Z
Z
ccd ccd m m( , ) ( , )X Y
u v f fZ Z
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
u vimg img( , )
ccdw
ccdh
(0,0)
(0,0) imgw
imgh
x yp p( , )
: Metric projection
x x
w w Xu u p f p
w w Z
img img
img ccd m
ccd ccd
y y
h h Yv v p f p
h h Z
img img
img ccd m
ccd ccd
Pixel projection
3D Point Projection (Pixel Space)
O
( , )u vccd ccd
mf
Projection plane ( , , )X Y Z
Z
ccd ccd m m( , ) ( , )X Y
u v f fZ Z
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
u vimg img( , )
ccdw
ccdh
(0,0)
(0,0) imgw
imgh
x yp p( , )
: Metric projection
x x
w w Xu u p f p
w w Z
img img
img ccd m
ccd ccd
y y
h h Yv v p f p
h h Z
img img
img ccd m
ccd ccd
Pixel projection
Exercise
324 m
1280 pix
960 pix
fm ?
What f to make the height of Eifel tower appear 960 pixel distance?
21.8 mm
ccd size
1500 m
Exercise
324 m
1280 pix
960 pix
fm ?
What f to make the height of Eifel tower appear 960 pixel distance?
img
img
ccd
m
h Yy
hf
Z
21.8 mm
ccd size
1500 m
Exercise
324 m
1280 pix
960 pix
fm ?
What f to make the height of Eifel tower appear 960 pixel distance?
21.8 mm
ccd size
1280 324960 0.0757m
0.0218 1500 m mf f
1500 m
img
img
ccd
m
h Yy
hf
Z
Focal Length
Exercise
324 m
Z ?
1280 pix
960 pix
What Z to make the height of Eifel tower appear 960 pixel distance?
img
img
ccd
m
h Yy f
h Z
21.8 mm
ccd size
fm = 50 mm
Do I need to move backward or forward?
Exercise
324 m
Z ?
1280 pix
960 pix
What Z to make the height of Eifel tower appear 960 pixel distance?
21.8 mm
ccd size
1280 324960 0.05 990.826m
0.0218 Z
Z
fm = 50 mm
Do I need to move backward or forward?
Forward
img
img
ccd
m
h Yy f
h Z
Exercise
324 m
1500
What Zp to make the height of Eifel tower appear twice of the person?
fm = 50 mm
Z p
1.7 m
Where Was I?
Circa 1984
Where Was I?
Circa 1984
250 pix
670 pix
Where Was I?
Circa 1984
324 m
fm = 50 mm
0.9 m
250 pix
670 pix
Where Was I?
Circa 1984
324 m
Z 1
0.9 m
250 pix
670 pix
fm = 50 mm
img img1 11
ccd cc
1
11 d
1280 0.90.05 8.03m
0.0218 250 m m
h hY Yy f
h yZ
Zf
h
Where Was I?
Circa 1984
324 m
fm = 50 mm
Z 1
0.9 m
250 pix
670 pix
img img1 11
ccd cc
1
11 d
1280 0.90.05 8.03m
0.0218 250 m m
h hY Yy f
h yZ
Zf
h
Where Was I?
Circa 1984
324 m
Z 1
0.9 m
Z 2
250 pix
670 pix
img img2 22
ccd cc
2
22 d
1280 3240.05 1079m
0.0218 670 m m
h hY Yy f
h yZ
Zf
h
fm = 50 mm
img img1 11
ccd cc
1
11 d
1280 0.90.05 8.03m
0.0218 250 m m
h hY Yy f
h yZ
Zf
h
Where Was I?
Circa 1984
250 pix
670 pix
m m
h hY Y Yy f f Z f
Z h Z h y
img img2 2 22 2
2 ccd 2 ccd 2
1280 3240.05 1079m
0.0218 670
200m
400m
600m
800m
1000m
800 m
Where Was I?
3D Point Projection (Pixel Space)
O
( , )u vccd ccd
mf
Projection plane ( , , )X Y Z
Z
ccd ccd m m( , ) ( , )X Y
u v f fZ Z
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
u vimg img( , )
ccdw
ccdh
(0,0)
(0,0) imgw
imgh
x yp p( , )
: Metric projection
x x
w w Xu u p f p
w w Z
img img
img ccd m
ccd ccd
y y
h h Yv v p f p
h h Z
img img
img ccd m
ccd ccd
Pixel projection
3D Point Projection (Pixel Space)
O
Projection plane ( , , )X Y Z
Z
ccd ccd m m( , ) ( , )X Y
u v f fZ Z
x x
w w Xu u p f p
w w Z
img img
img ccd m
ccd ccd
y y
h h Yv v p f p
h h Z
img img
img ccd m
ccd ccd
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
u vimg img( , )
ccdw
ccdh
(0,0)
(0,0) imgw
imgh
x yp p( , )
: Metric projection
Focal length in pixel
Pixel projection
img imgu v( , )
f
3D Point Projection (Pixel Space)
O
Projection plane ( , , )X Y Z
Z
ccd ccd m m( , ) ( , )X Y
u v f fZ Z
x x
w w Xu u p f p
w w Z
img img
img ccd m
ccd ccd
y y
h h Yv v p f p
h h Z
img img
img ccd m
ccd ccd
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
u vimg img( , )
ccdw
ccdh
(0,0)
(0,0) imgw
imgh
x yp p( , )
: Metric projection
Focal length in pixel
xf
yf
img
m
ccd
x
wf f
w
img
m
ccd
y
hf f
hwhere
Pixel projection
img imgu v( , )
f
3D Point Projection (Pixel Space)
O
Projection plane ( , , )X Y Z
Z
ccd ccd m m( , ) ( , )X Y
u v f fZ Z
x x
w w Xu u p f p
w w Z
img img
img ccd m
ccd ccd
y y
h h Yv v p f p
h h Z
img img
img ccd m
ccd ccd
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
u vimg img( , )
ccdw
ccdh
(0,0)
(0,0) imgw
imgh
x yp p( , )
: Metric projection
Focal length in pixel
f
f
img img
m m
ccd ccd
w h
f f fw h
where
Pixel projection
img img
ccd ccd
w h
w hif
img imgu v( , )
f
3D Point Projection (Pixel Space)
O
Projection plane ( , , )X Y Z
Z
img x
Xu f p
Z
img y
Yv f p
Z
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
u vimg img( , )
ccdw
ccdh
(0,0)
(0,0) imgw
imgh
x yp p( , )
img imgu v( , )
f
3D Point Projection (Pixel Space)
O
Projection plane ( , , )X Y Z
Z
img x
Xu f p
Z
img y
Yv f p
Z
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
u vimg img( , )
ccdw
ccdh
(0,0)
(0,0) imgw
imgh
x yp p( , )
img imgu v( , )
f
img xZu fX p Z
img yZv fY p Z
3D Point Projection (Pixel Space)
O
Projection plane ( , , )X Y Z
Z
img x
Xu f p
Z
img y
Yv f p
Z
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
u vimg img( , )
ccdw
ccdh
(0,0)
(0,0) imgw
imgh
x yp p( , )
img imgu v( , )
f
img xZu fX p Z
img yZv fY p Z
img
img
11
x
y
u f p X
Z v f p Y
Z
3D Point Projection (Pixel Space)
O
Projection plane ( , , )X Y Z
Z
img x
Xu f p
Z
img y
Yv f p
Z
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
u vimg img( , )
ccdw
ccdh
(0,0)
(0,0) imgw
imgh
x yp p( , )
img imgu v( , )
f
img xZu fX p Z
img yZv fY p Z
img
img
11
x
y
u f p X
v f p Y
Z
Computer Graphics
img x
Xu f p
Z
img y
Yv f p
Z
img xZu fX p Z
img yZv fY p Z
Graphics
img
img
11
x
y
u f p X
v f p Y
Z
Computer Vision = inv(Computer Graphics)
img x
Xu f p
Z
img y
Yv f p
Z
img xZu fX p Z
img yZv fY p Z
Graphics
Vision
img
img
11
x
y
u f p X
v f p Y
Z
3D Point Projection (Pixel Space)
O
Projection plane ( , , )X Y Z
Z
img x
Xu f p
Z
img y
Yv f p
Z
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
u vimg img( , )
ccdw
ccdh
(0,0)
(0,0) imgw
imgh
x yp p( , )
img imgu v( , )
f
img xZu fX p Z
img yZv fY p Z
img
img
11
x
y
u f p X
v f p Y
Z
Metric space Pixel space
3D Point Projection (Pixel Space)
O
Projection plane ( , , )X Y Z
Z
img x
Xu f p
Z
img y
Yv f p
Z
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
u vimg img( , )
ccdw
ccdh
(0,0)
(0,0) imgw
imgh
x yp p( , )
img imgu v( , )
f
img xZu fX p Z
img yZv fY p Z
img
img
11
x
y
u f p X
v f p Y
Z
Metric space Pixel space
+
Camera intrinsic parameter : metric space to pixel space
3D Point Projection (Pixel Space)
O
Projection plane ( , , )X Y Z
Z
img x
Xu f p
Z
img y
Yv f p
Z
CCD sensor (mm) Image (pixel)
u vccd ccd( , )
u vimg img( , )
ccdw
ccdh
(0,0)
(0,0) imgw
imgh
x yp p( , )
img imgu v( , )
f
img xZu fX p Z
img yZv fY p Z
img
img
11
x
y
u f p X
v f p Y
Z
K
+
Camera intrinsic parameter : metric space to pixel space
Metric space Pixel space
Camera Intrinsic Parameter
O
Projection plane
Z
img
img
11
x
y
u f p X
v f p Y
Z
K
Metric space
( , , )X Y Z
Camera Intrinsic Parameter
O
Projection plane
Z
img
img
11
x
y
u f p X
v f p Y
Z
K
Metric space
( , , )X Y Z
f
Camera Intrinsic Parameter
O
Projection plane
Z
img
img
11
x
y
u f p X
v f p Y
Z
K
Metric space
( , , )X Y Z
f
img imgu v( , )
Pixel space
2D Inverse Projection
O
Projection plane
img
img
11
x
y
u f p X
v f p Y
Z
K
Metric space
f
img imgu v( , )
Pixel space
u X
v Y
Z
K
img
-1img
1
2D Inverse Projection
O
Projection plane
img
img
11
x
y
u f p X
v f p Y
Z
K
Metric space
f
img imgu v( , )
Pixel space
u X
v Y
Z
K
img
-1img
1
u
vK
img
-1img
1
2D point == 3D ray
3D ray
2D Inverse Projection
O
Projection plane
img
img
11
x
y
u f p X
v f p Y
Z
K
Metric space
f
img imgu v( , )
Pixel space
u X
v Y
Z
K
img
-1img
1
u
vK
img
-1img
1
( , , )X Y Z
2D point == 3D ray
3D ray
2D Inverse Projection
O
Projection plane
img
img
11
x
y
u f p X
v f p Y
Z
K
Metric space
f
img imgu v( , )
Pixel space
u X
v Y
Z
K
img
-1img
1
u
vK
img
-1img
1
( , , )X Y Z
2D point == 3D ray
3D ray
The 3D point must lie in the 3D ray passing through the origin and 2D image point.