Visualization of Urban Scenes from Video Footages Siyuan Fang Supervised by Dr. Neill Campbell...
-
Upload
jennifer-bell -
Category
Documents
-
view
212 -
download
0
Transcript of Visualization of Urban Scenes from Video Footages Siyuan Fang Supervised by Dr. Neill Campbell...
Visualization of Urban Scenes from Video Footages
Siyuan Fang
Supervised by Dr. Neill Campbell 29/10/2007
Overview
Generate panorama (or image mosaicing) from a video of street scenes
Potential applications: Virtual environment Robot navigation On-line tourist guide
Combine different source frames (or images) together
Outline
• Background • Research Proposal • Achievements • Future Plan
Mosaicing form Single-Viewpoint Images
Source images are captured using a camera with pure rotation movement around a fixed viewpoint, (the intrinsic parameter can also change)
Align (or register) source images and stitch them together, e.g., the general alignment (registration) model: homography for a pair of images.
The homography can be estimated by points (pixels or features) correspondence, robust estimations (e.g., RANSAC) are used to remove the outliers of the point correspondence
Further refinements: pixel value blending bundle adjustment
The limitation of the single-viewpoint image mosaicing--cannot put viewpoint far away enough
Mosaicing form Multi-Viewpoint Images : Perspective Mosaicing
• The difficulty: there isn’t a single uniform alignment --only planar scenes have the same homography if viewpoint is changing
• Traditional approaches: view synthesis and image based rendering e.g., [GORT96], [SHUM99]--picture surface alignment + residual pixel movement [KUMA95]
• User-interactive design: picture surface alignment + user refinement for the off-plane scenes [AGAR06]
• Limitations of these approaches
Strip Panorama: Parallel Projection Mosaicing
A simulation of the push-broom camera. A common setup: sideway translating camcorder and vertical trip scanning. [ZHEN03]
Parallel projection along the horizontal direction, hence, remain the perspective along the vertical direction, cause the aspect ratio distortion:
Further scenes get expanded Closer scenes get squashed
Strip Panorama: Parallel Projection Mosaicing Cont.
Two kinds of distortion, the same reason:
Aspect ratio distortion Blurred-truncated
distortion
A Better Strip Panorama: Crossed-Slit Mosaicing
The Crossed-Slit Cameral Model, and its special case: POX-slit model.
Strict sideway translation camcorder can be used to simulate POX-slit model: the cameral path is presumably considered as the horizontal slit and the vertical slit is determined by the camera path and the width of the image plane [ZOME03]
The Crossed-Slit Mosaicing cont.
Comparison between the Crossed-Slit Mosaicing and the Parallel projection Mosaicing
The Limitations of the Crossed-Slit Mosaicing (Distortions)
Aims and Objectives
Aims and basic requirements: . Use moving pinhole camcorder (normal) Capture real long street scenes Scalable Least distortion Automaticity
The Proposed Research Approach (provisional)
The proposed approach, 3 main tasks:I. Synthesis novel view based on the crossed-slit mosaic, but fully perspective
— remove two kinds of distortion based on geometry information
II. Handle extremely long scenes,
III. * Handle moving objects
Summary of Achievements
The literature survey on image mosaicing Single-viewpoint mosaicing View synthesis and imaged based rendering Approaches of strip mosaicng
A (provisional) research proposal
Some design and implementation work A single-viewpoint mosaic tool A part of the task 1
Future Plan
Further refine (modify) the research proposal
Complete the task 1 A formal math model and an
approximation algorithm Curved camera path and corner
effect Writing up
Start the design work of task 2 The main problem User interactive may be needed The result of task 1 can be
served as the building block
Reference
[AGAR06] A.Agarwala, M.Agrawala, M.Cohen and D.Salesin, R.Szeliski, Photographing long scenes with multi-viewpoint panoramas, ACM Transactions on Graphics, 25(3), 853 - 861, 2006.
[GORT96] S.J.Gortler, R.Grzeszczuk, R. Szeliski and M.F. Cohen, The Lumigraph, The Journal of Computer Graphics, 30, 43--54, 1996.
[KUMA95] R.Kumar, P.Anandan, M.Irani, J.Bergen and K.Hanna, Representation of scenes from collections of images, Proceedings of IEEE Workshop on Representation of Visual Scenes, 10-17 ,1995
[PELE00] S.Peleg, B.Rousso, A.Rav-Acha and A.Zomet, Mosaicing on Adaptive Manifold, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(10), 1144-1154, 2000.
[SHUM99] H.Y.Shum and L.W.He, Rendering with concentric mosaics, The Journal of Computer Graphics, 33, 299--306, 1999.
[ZHEN03] J.Y.Zheng, Digital route panoramas, IEEE Multimedia, 10(3), 57- 67, 2003.
[ZOME03] A.Zomet, D.Feldman, S.Peleg and D.Weinshall, Mosaicing New Views: The Crossed-Slits Projection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 741- 754, 2003.