3D Computer Vision - - TU...
-
Upload
dinhnguyet -
Category
Documents
-
view
222 -
download
2
Transcript of 3D Computer Vision - - TU...
![Page 1: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/1.jpg)
3D Computer Vision
Prof. Didier StrickerDr. Gabriele Bleser
![Page 2: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/2.jpg)
AG “Augmented Vision” Uni-KL and DFKI• Founded July 2008• 30 fulltime researchers• 3 strongly connected research areas
Computer Vision
Sensor Fusion
HCI and Visualizatio
n
![Page 3: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/3.jpg)
3D Scanning & Reconstruction3D Scanning & Reconstruction Hand & Body Tracking Hand & Body Tracking
Object Recognition & TrackingObject Recognition & Tracking Human Computer InteractionHuman Computer Interaction
Research Topics
Research Topics
![Page 4: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/4.jpg)
Application areas
Virtual Engineering &
Training
Virtual Engineering &
Training
Ambient Assisted Living
Ambient Assisted Living
Safety & SecuritySafety & Security
Cultural HeritageCultural Heritage
……
![Page 5: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/5.jpg)
3D Computer Vision:
• Lecture: Wednesday, 8:15 – 9:45• SWS: 2V+1Ü • Credit points: 4 CP
• Language: English
![Page 6: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/6.jpg)
3D Computer Vision: lecture
• Topics:• Introduction: what is a camera?• Camera model and camera calibration• Fitting and parameter estimation• Image point detection and point matching• 2D-image transformation (mapping) and
panorama• Two cameras: epipolar geometry and
triangulation• Multiple view reconstruction• Depth maps and multiple view stereo
reconstruction• Structured light: laser, coded light
images
3D reconstruction
Camera pose
Texturing
![Page 7: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/7.jpg)
3D Computer Vision: Exercises• Homework assignments consisting of
– theoretical part (questions) and – practical part (Matlab implementation)to be solved and handed in in small groups (by email)
• Accompanying supervised exercise sessions: two hours every two weeks (discussion of last exercise, presentation/preparation of current exercise)
• Room: 32/411 (computer pool) SCI account required• Time: Monday, 17:00 to 19:00• Exam: Oral exam. In order to qualify, a minimum average score of
60% in the exercises is required.• Next lecture: 24th October – Matlab introduction (Tobias Nöll,
Johannes Köhler)
![Page 8: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/8.jpg)
Contact
Prof. Didier Stricker
Tobias Nöll: [email protected] Köhler: [email protected]. Gabriele Bleser: [email protected]
All material will be available at:http://av.dfki.de Students Lectures
![Page 9: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/9.jpg)
Example
Input images Structure From Motion
Multiple View Stereo
3D model
![Page 10: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/10.jpg)
![Page 11: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/11.jpg)
• High quality reconstruction with
3D Reconstruction under controlled conditions
• Results based on High-resolution camera devices Controlled lighting conditions Structured light for textureless
regions
![Page 12: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/12.jpg)
Check the 3D models in the 3D web-viewer -http://av.dfki.de/orcam/
![Page 13: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/13.jpg)
![Page 14: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/14.jpg)
Today: The Camera
![Page 15: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/15.jpg)
Overview• The pinhole projection model
• Qualitative properties• Perspective projection matrix
• Cameras with lenses• Depth of focus• Field of view• Lens aberrations
• Digital cameras• Types of sensors• Color
![Page 16: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/16.jpg)
How do we see the world?
Let’s design a camera• Idea 1: put a piece of film in front of an object• Do we get a reasonable image?
Slide by Steve Seitz
![Page 17: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/17.jpg)
Pinhole camera
Add a barrier to block off most of the rays• This reduces blurring• The opening known as the aperture
Slide by Steve Seitz
![Page 18: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/18.jpg)
Pinhole camera model
Pinhole model:• Captures pencil of rays – all rays through a single point• The point is called Center of Projection (focal point)• The image is formed on the Image Plane
Slide by Steve Seitz
![Page 19: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/19.jpg)
Point of observation
Figures © Stephen E. Palmer, 2002
Dimensionality Reduction Machine (3D to 2D)
3D world 2D image
What have we lost?• Angles• Distances (lengths)
Slide by A. Efros
![Page 20: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/20.jpg)
Projection properties• Many-to-one: any point along same ray maps
to same point in image• Points → points
• But projection of points on focal plane is undefined
• Lines → lines (collinearity is preserved)• But line through focal point projects to a point
• Planes → planes (or half-planes)• But plane through focal point projects to line
![Page 21: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/21.jpg)
Projection properties• Parallel lines converge at a vanishing point
• Each direction in space has its own vanishing point• But parallels parallel to the image plane remain parallel• All directions in the same plane have vanishing points on the
same line
How do we construct the vanishing point/line?
![Page 22: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/22.jpg)
One-point perspectiveMasaccio, Trinity, Santa
Maria Novella, Florence, 1425-28
First consistent use of perspective in Western art?
![Page 23: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/23.jpg)
Perspective distortion• Problem for architectural photography:
converging verticals
Source: F. Durand
![Page 24: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/24.jpg)
Perspective distortion• What does a sphere project to?
Image source: F. Durand
![Page 25: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/25.jpg)
Perspective distortion• What does a sphere project to?
![Page 26: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/26.jpg)
Perspective distortion• The exterior columns appear bigger• The distortion is not due to lens flaws• Problem pointed out by Da Vinci
Slide by F. Durand
![Page 27: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/27.jpg)
Perspective distortion: People
![Page 28: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/28.jpg)
Modeling projection
The coordinate system• We will use the pinhole model as an approximation• Put the optical center (O) at the origin• Put the image plane (Π’) in front of O
x
y
z
Source: J. Ponce, S. Seitz
![Page 29: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/29.jpg)
x
y
z
Modeling projection
Projection equations• Compute intersection with Π’ of ray from P = (x,y,z) to O• Derived using similar triangles
)',','(),,( fzyf
zxfzyx
Source: J. Ponce, S. Seitz
• We get the projection by throwing out the last coordinate:
)','(),,(zyf
zxfzyx
![Page 30: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/30.jpg)
Homogeneous coordinates
Is this a linear transformation?
Trick: add one more coordinate:
homogeneous image coordinates
homogeneous scene coordinates
Converting from homogeneous coordinates
• no—division by z is nonlinear
Slide by Steve Seitz
)','(),,(zyf
zxfzyx
![Page 31: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/31.jpg)
divide by the third coordinate
Perspective Projection MatrixProjection is a matrix multiplication using homogeneous
coordinates:
'/1
0'/10000100001
fzyx
zyx
f)','(
zyf
zxf
![Page 32: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/32.jpg)
divide by the third coordinate
Perspective Projection MatrixProjection is a matrix multiplication using homogeneous
coordinates:
'/1
0'/10000100001
fzyx
zyx
f)','(
zyf
zxf
In practice: lots of coordinate transformations…
World to camera coord.
trans. matrix(4x4)
Perspectiveprojection matrix
(3x4)
Camera to pixel coord. trans. matrix
(3x3)
=2D
point(3x1)
3Dpoint(4x1)
![Page 33: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/33.jpg)
Building a real camera
![Page 34: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/34.jpg)
Camera Obscura
• Basic principle known to Mozi (470-390 BCE), Aristotle (384-322 BCE)
• Drawing aid for artists: described by Leonardo da Vinci (1452-1519)
Gemma Frisius, 1558
Source: A. Efros
![Page 35: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/35.jpg)
Home-made pinhole camera
http://www.debevec.org/Pinhole/
Why soblurry?
Slide by A. Efros
![Page 36: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/36.jpg)
Shrinking the aperture
Why not make the aperture as small as possible?• Less light gets through• Diffraction effects…
Slide by Steve Seitz
![Page 37: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/37.jpg)
Shrinking the aperture
![Page 38: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/38.jpg)
The reason for lenses
Slide by Steve Seitz
![Page 39: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/39.jpg)
Adding a lens
A lens focuses light onto the film• Rays passing through the center are not deviated
Slide by Steve Seitz
![Page 40: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/40.jpg)
Adding a lens
A lens focuses light onto the film• Rays passing through the center are not deviated• All parallel rays converge to one point on a plane located at
the focal length f
Slide by Steve Seitz
focal point
f
![Page 41: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/41.jpg)
Adding a lens
A lens focuses light onto the film• There is a specific distance at which objects are “in focus”
– other points project to a “circle of confusion” in the image
“circle of confusion”
Slide by Steve Seitz
![Page 42: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/42.jpg)
Thin lens formula
fDD’
Frédo Durand’s slide
![Page 43: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/43.jpg)
Thin lens formula
fDD’
Similar triangles everywhere!
Frédo Durand’s slide
![Page 44: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/44.jpg)
Thin lens formula
fDD’
Similar triangles everywhere!
y’y
y’/y = D’/D
Frédo Durand’s slide
![Page 45: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/45.jpg)
Thin lens formula
fDD’
Similar triangles everywhere!
y’y
y’/y = D’/Dy’/y = (D’-f)/f
Frédo Durand’s slide
![Page 46: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/46.jpg)
Thin lens formula
fDD’
1D’ D
1 1f+ =
Any point satisfying the thin lens equation is in focus.
Frédo Durand’s slide
![Page 47: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/47.jpg)
Depth of Field
http://www.cambridgeincolour.com/tutorials/depth-of-field.htm
Slide by A. Efros
![Page 48: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/48.jpg)
How can we control the depth of field?
Changing the aperture size affects depth of field• A smaller aperture increases the range in which the object is
approximately in focus• But small aperture reduces amount of light – need to
increase exposureSlide by A. Efros
f / 5.6
f / 32
Flower images from Wikipedia http://en.wikipedia.org/wiki/Depth_of_field
![Page 49: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/49.jpg)
Varying the aperture
Large aperture = small DOF Small aperture = large DOFSlide by A. Efros
![Page 50: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/50.jpg)
Field of View (Zoom)
Slide by A. Efros
![Page 51: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/51.jpg)
Field of View (Zoom)
Slide by A. Efros
![Page 52: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/52.jpg)
f
Field of View
Smaller FOV = larger Focal LengthSlide by A. Efros
f
FOV depends on focal length and size of the camera retina
![Page 53: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/53.jpg)
Field of View / Focal Length
Large FOV, small fCamera close to car
Small FOV, large fCamera far from the car
Sources: A. Efros, F. Durand
![Page 54: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/54.jpg)
Same effect for faces
standardwide-angle telephoto
Source: F. Durand
![Page 55: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/55.jpg)
Real lenses
![Page 56: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/56.jpg)
Lens Flaws: Chromatic AberrationLens has different refractive indices for different
wavelengths: causes color fringing
Near Lens Center Near Lens Outer Edge
![Page 57: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/57.jpg)
Lens flaws: Vignetting
![Page 58: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/58.jpg)
No distortion Pin cushion Barrel
Radial Distortion• Caused by imperfect lenses• Deviations are most noticeable for rays that pass through the edge of
the lens
![Page 59: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/59.jpg)
Distortion
Slides from Seitz
![Page 60: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/60.jpg)
Modeling distortion
To model lens distortion• Use above projection operation instead of standard projection matrix multiplication
Apply radial distortion
Apply focal length translate image center
Project to “normalized”
image coordinates
Radial and Tangential Distortion
image plane
ideal actual
idealactual
![Page 61: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/61.jpg)
Digital camera
A digital camera replaces film with a sensor array• Each cell in the array is light-sensitive diode that converts photons to electrons• Two common types
– Charge Coupled Device (CCD)– Complementary metal oxide semiconductor (CMOS)
• http://electronics.howstuffworks.com/digital-camera.htm
Slide by Steve Seitz
![Page 62: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/62.jpg)
CCD vs. CMOSCCD: transports the charge across the chip and reads it at one corner of the array.
An analog-to-digital converter (ADC) then turns each pixel's value into a digital value by measuring the amount of charge at each photosite and converting that measurement to binary form
CMOS: uses several transistors at each pixel to amplify and move the charge using more traditional wires. The CMOS signal is digital, so it needs no ADC.
http://www.dalsa.com/shared/content/pdfs/CCD_vs_CMOS_Litwiller_2005.pdf
http://electronics.howstuffworks.com/digital-camera.htm
![Page 63: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/63.jpg)
High Dynamic Range Images
• CCD / CMOS have a given dynamic range for caputring the luminance of the scene – LDR cameras (Low Dynamic Range)
• ProblemThe dynamic range of the scene illumination is higher than the dynamic of the camera
• It results• Under exposed areas• Saturation areas
![Page 64: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/64.jpg)
High Dynamic Range Images
High dynamic range image- Standard RGB images are encoded over 8 bit integer (255 values)- HDR: 12 bits floats
+ + + =
See: www.OpenEXR.com
![Page 65: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/65.jpg)
Interlace vs. progressive scan
http://www.axis.com/products/video/camera/progressive_scan.htm Slide by Steve Seitz
![Page 66: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/66.jpg)
Progressive scan
http://www.axis.com/products/video/camera/progressive_scan.htm Slide by Steve Seitz
![Page 67: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/67.jpg)
Interlace
http://www.axis.com/products/video/camera/progressive_scan.htm Slide by Steve Seitz
![Page 68: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/68.jpg)
Color sensing in camera: Color filter array
Source: Steve Seitz
Estimate missing components from neighboring values(demosaicing)
Why more green?
Bayer grid
Human Luminance Sensitivity Function
![Page 69: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/69.jpg)
Color sensing in camera: Prism• 3 CCD camera
• requires three chips and precise alignment• More expensive
CCD(B)
CCD(G)
CCD(R)
![Page 70: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/70.jpg)
Issues with digital camerasNoise
– low light is where you most notice noise– light sensitivity (ISO) / noise tradeoff– stuck pixels
Resolution: Are more megapixels better?– requires higher quality lens– noise issues
In-camera processing– oversharpening can produce halos
RAW vs. compressed– file size vs. quality tradeoff
Blooming– charge overflowing into neighboring pixels
Color artifacts– purple fringing from microlenses, artifacts from Bayer patterns– white balance
More info online:• http://electronics.howstuffworks.com/digital-camera.htm• http://www.dpreview.com/
Slide by Steve Seitz
![Page 71: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/71.jpg)
Historical context• Pinhole model: Mozi (470-390 BCE),
Aristotle (384-322 BCE)• Principles of optics (including lenses):
Alhacen (965-1039 CE) • Camera obscura: Leonardo da Vinci
(1452-1519), Johann Zahn (1631-1707)• First photo: Joseph Nicephore Niepce (1822)• Daguerréotypes (1839)• Photographic film (Eastman, 1889)• Cinema (Lumière Brothers, 1895)• Color Photography (Lumière Brothers, 1908)• Television (Baird, Farnsworth, Zworykin, 1920s)• First consumer camera with CCD:
Sony Mavica (1981)• First fully digital camera: Kodak DCS100 (1990)
Niepce, “La Table Servie,” 1822
CCD chip
Alhacen’s notes
![Page 72: 3D Computer Vision - - TU Kaiserslauternags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws12-13/lec01_camera.pdf · 3D Computer Vision: Exercises • Homework assignments consisting of –](https://reader031.fdocuments.us/reader031/viewer/2022022003/5aa3c0f97f8b9a46238ec127/html5/thumbnails/72.jpg)
Thank You!