Vision, Video and Virtual Reality Omnidirectional Vision Lecture 6 Omnidirectional Cameras CSC...
-
date post
19-Dec-2015 -
Category
Documents
-
view
219 -
download
0
Transcript of Vision, Video and Virtual Reality Omnidirectional Vision Lecture 6 Omnidirectional Cameras CSC...
Vision, Video
and Virtual Reality Omnidirectional VisionOmnidirectional Vision
Lecture 6Omnidirectional Cameras
CSC 59866CDFall 2004
Zhigang Zhu, NAC 8/203Ahttp://www-cs.engr.ccny.cuny.edu/~zhu/
Capstone2004/Capstone_Sequence2004.html
Vision, Video
and Virtual Reality Lecture OutlineLecture Outline Applications
Robot navigation, Surveillance, Smart rooms Video-conferencing/ Tele-presence Multimedia/Visualization
Page of Omnidirectional Vision (Many universities and companies….) http://www.cis.upenn.edu/~kostas/omni.html
Design Requirements 360 degree FOV, or semi-sphere or full sphere in one snapshot Single effective viewpoint Image Resolutions – one or more cameras? Image Sharpness – optics as well as geometry
Several Important Designs Catadioptric imaging : mirror (reflection) + lens ( refraction) Mirrors: Planar, Conic, Spherical, Hyperboloidal, Ellipsoidal, Paraboloidal Systematic design ( S. Nayar’s group)
Calibrations Harder or simpler?
Vision, Video
and Virtual Reality Sensor DesignSensor Design Catadioptric imaging :
mirror (reflection) + lens ( refraction) Theory of Catadioptric Image Formation ( S. Nayar’s group)
"A Theory of Single-Viewpoint Catadioptric Image Formation" , Simon Baker and Shree K. Nayar ,International Journal of Computer Vision, 1999.
Mirrors Planar Conic, Spherical Hyperboloidal, Ellipsoidal Paraboloidal
Cameras (Lens) Perspective (pinhole) or orthogonal (tele-centric lens) projection One or more?
Implementations Compactness - size, support, and installation Optics – Image sharpness, reflection, etc.
Vision, Video
and Virtual Reality Planar MirrorPlanar Mirror
Panoramic camera system using a pyramid with four (or more) planar mirrors and four (or more) cameras (Nalwa96) has a single effective viewpoint
4 camera design and 6 camera prototype:
FullView - Lucent Technology http://www.fullview.com/
6 cameras
Mirror pyramid
Vision, Video
and Virtual Reality Planar MirrorPlanar Mirror
Panoramic camera system using a pyramid with four (or more) planar mirrors and four (or more) cameras (Nalwa96) has a single effective viewpoint
P1
P2
Viewpoint of the Virtual camera
Geometry of 4 camera approach: four separate cameras in 4 viewpoints can generate images with a single effective viewpoint
Vision, Video
and Virtual Reality Planar Mirror ApproachPlanar Mirror Approach
A single effective viewpoint More than one cameras High image resolution
Vision, Video
and Virtual Reality Planar Mirror ApproachPlanar Mirror Approach
A single effective viewpoint More than one cameras High image resolution
Vision, Video
and Virtual Reality Conic MirrorConic Mirror Viewpoints on a circle semispherical view except occlusion Perspective projection in each direction Robot Navigation (Yagi90, Zhu96/98)
viewpoint
pinhole
Vision, Video
and Virtual Reality Spherical MirrorSpherical Mirror
Viewpoints on a spherical-like surface Easy to construct (Hong91 -UMass )
Intersection of incoming rays are along this lineLocus of
viewpoints
Vision, Video
and Virtual Reality Hyperboloidal MirrorHyperboloidal Mirror Single Viewpoint
if the pinhole of the real camera and the virtual viewpoint are located at the two loci of the hyperboloid
Semi-spherical view except the self occlusion
pinhole
P1
viewpoint
P2
Rotation of the hyperbolic curve generates a hyperboloid
Vision, Video
and Virtual Reality Hyperboloidal MirrorHyperboloidal Mirror ACCOWLE Co., LTD, A Spin-off at Kyoto University
http://www.pluto.dti.ne.jp/~accowle1/ Spherical Mirror Hyperbolic Mirror
Image: High res. in the top
Vision, Video
and Virtual Reality Ellipsoidal MirrorEllipsoidal Mirror Single Viewpoint
if the pinhole of the real camera and the virtual viewpoint are located at the two loci of the ellipsoid
Semi-spherical view except the self occlusion
pinhole
viewpoint
P1
P2
Vision, Video
and Virtual Reality Panoramic Annular Lens
panoramic annular lens (PAL)- invented by P. Greguss* 40 mm in diameter, C-mount* view: H: 360, V: -15 ~ +20* single view point (O)
- geometric mathematical model for image transform & calibration
p p1
pinhole
P1
P
B
O
C
Ellipsoidal mirror
Hyperboloidal mirror
Vision, Video
and Virtual Reality Panoramic Annular Lens
panoramic annular lens (PAL)- invented by P. Greguss* 40 mm in diameter, C-mount* view: H: 360, V: -15 ~ +20•single view point (O)•C-Mount to CCD Cameras
Image: High res. In the bottom
Vision, Video
and Virtual Reality Cylindrical panoramic un-warping
Circular to cylindrical transformationafter eliminating radial distortion
Two Steps:
(1). Center determination
(2) Distortion rectification
2-order polynomial approximation
Vision, Video
and Virtual Reality Paraboloidal MirrorParaboloidal Mirror
Semi-spherical view except the self occlusion Single Viewpoint at the locus of the paraboloid, if
Tele-lens - orthographic projection is used Mapping between image, mirror and the world invariant to
translation of the mirror. This greatly simplifies calibration and the computation of perspective images from paraboloidal images
P1
viewpoint
tele-lens
P2
Vision, Video
and Virtual Reality Paraboloidal MirrorParaboloidal Mirror
Remote Reality – A Spin-off at Columbia University
http://www.remotereality.com/
Camcorder Web Camera Back to Back : Full Spherical View
Vision, Video
and Virtual Reality Paraboloidal MirrorParaboloidal Mirror
Remote Reality – A Spin-off at Columbia University
http://www.remotereality.com/
Vision, Video
and Virtual Reality Catadioptric Camera CalibrationCatadioptric Camera Calibration
Omnidirectional Camera Calibration – Harder or Easier? In general, the reflection by the 2nd order surface makes
the calibration procedure harder However, 360 view may be helpful
Paraboloidal mirror + orthogonal projection Mapping between image, mirror and the world invariant to
translation of the mirror. Projections of two sets of parallel lines suffice for intrinsic
calibration from one view C. Geyer and K. Daniilidis, "Catadioptric Camera calibration",
In Proc. Int. Conf. on Computer Vision, Kerkyra, Greece, Sep. 22-25, pp. 398-404, 1999.
Vision, Video
and Virtual RealityImage Properties of Paraboloid System Image Properties of Paraboloid System
The Image of a Line is a circular arc if the line is not parallel to the optical axis Is projected on a (radial) line otherwise
Dual Vanishing Points There are two VPs for each set of parallel lines, which are
the intersections of the corresponding circles Collinear Centers
The center of the circles for a set of parallel lines are collinear
Vanishing Circle The vanishing points of lines with coplanar directions* lie
on a circle ( all the lines parallel to a common plane)
(Assuming aspect ratio = 1)
Vision, Video
and Virtual RealityImage Properties of Paraboloid System Image Properties of Paraboloid System
The Image Center Is on the (“vanishing”) line connecting the dual vanishing
points of each set of parallel lines Can be determined by two sets of parallel lines
Projection of a Line with unknown aspect ratio Is an elliptical arc in the general case
The Aspect Ratio Is determined by the ratio of the lone-short axes of the
ellipse corresponding to a line Intrinsic Calibration
Estimate aspect ratio by the ratio of ellipse Estimate the image center by the intersection of vanishing
lines of two sets of parallel lines in 3-D space
(with aspect ratio)
Vision, Video
and Virtual RealityCalibration of Paraboloid System Calibration of Paraboloid System
The Image Center Is on the (“vanishing”) line connecting the dual vanishing
points of each set of parallel lines Can be determined by two sets of parallel lines
Vision, Video
and Virtual RealityCalibration of Paraboloid System Calibration of Paraboloid System
The Image Center Yellow “vanishing” line of horizontal set of parallel lines Pink “vanishing” line of vertical set of parallel lines
The Vanishing Circle (Red dotted) The vanishing points of lines with coplanar directions ( on a plane in this example)
Projected to the plane of the calibration pattern
Vision, Video
and Virtual Reality NextNext
Next: Features
END