Towards Human Level AI Jitendra Malik U.C. Berkeley Jitendra Malik U.C. Berkeley.
Convergence of vision and graphics Jitendra Malik University of California at Berkeley Jitendra...
-
date post
22-Dec-2015 -
Category
Documents
-
view
222 -
download
0
Transcript of Convergence of vision and graphics Jitendra Malik University of California at Berkeley Jitendra...
Convergence of vision and graphics
Convergence of vision and graphics
Jitendra Malik
University of California at Berkeley
Jitendra Malik
University of California at Berkeley
Overview Overview
3D capture:Modeling,simulation
Rendering Display
Applications:Simulation
Virtual RealityRemote collaboration
Graphics and VisionGraphics and Vision
• Computer graphics is the forward problem: given scene geometry, reflectances and lighting, synthesize an image.
• Computer vision must address the inverse problem: given an image/multiple images, reconstruct the scene geometry, reflectacnes and illumination.
• Computer graphics is the forward problem: given scene geometry, reflectances and lighting, synthesize an image.
• Computer vision must address the inverse problem: given an image/multiple images, reconstruct the scene geometry, reflectacnes and illumination.
Image-based Modeling Image-based Modeling
– Vary viewpoint– Vary lighting– Vary scene configuration
– Vary viewpoint– Vary lighting– Vary scene configuration
Recover Models of Real World Scenes and Make Possible Various Visual Interactions
Image-based Modeling Image-based Modeling • 1st Generation---- vary viewpoint but not
lighting– Acquire photographs
– Recover geometry (explicit or implicit)
– Texture map
• 1st Generation---- vary viewpoint but not lighting– Acquire photographs
– Recover geometry (explicit or implicit)
– Texture map
Recovering geometryRecovering geometry
• Historical roots in photogrammetry and analysis of 3D cues in human vision
• Single images adequate given knowledge of object class
• Multiple images make the problem easier, but not trivial as corresponding points must be identified.
• Historical roots in photogrammetry and analysis of 3D cues in human vision
• Single images adequate given knowledge of object class
• Multiple images make the problem easier, but not trivial as corresponding points must be identified.
The Taj Mahal
Taj MahalTaj Mahalmodeled frommodeled from
one photographone photographby G. Borshukovby G. Borshukov
Campus Model of UC Berkeley Campus Model of UC Berkeley
Campanile + 40 Buildings (Debevec et al)Campanile + 40 Buildings (Debevec et al)
Image-based ModelingImage-based Modeling• 2nd Generation---- vary viewpoint and
lighting– Recover geometry & reflectance properties
– Render using light transport simulation or local shading
• 2nd Generation---- vary viewpoint and lighting– Recover geometry & reflectance properties
– Render using light transport simulation or local shading
Original Lighting & Viewpoint Novel Lighting & Viewpoint
Inverse Global Illumination (Yu et al)Inverse Global Illumination (Yu et al)
Reflectance Properties
Radiance Maps
Geometry Light Sources
Image-based ModelingImage-based Modeling
• 3rd Generation--Vary spatial configurations in addition to viewpoint and lighting
• 3rd Generation--Vary spatial configurations in addition to viewpoint and lighting
Novel Viewpoint Novel Viewpoint & Configuration
Our FrameworkOur Framework
• Input– Multiple range scans of
a scene
– Multiple photographs of the same scene
• Input– Multiple range scans of
a scene
– Multiple photographs of the same scene
• Output– Geometric meshes of
each object in the scene
– Registered texture maps for objects
• Output– Geometric meshes of
each object in the scene
– Registered texture maps for objects
Continuing Challenges Continuing Challenges
• Finding correspondences automatically
• Optimal estimation of structure from n views under perspective projection
• Models of reflectance and texture for natural materials and objects
• Finding correspondences automatically
• Optimal estimation of structure from n views under perspective projection
• Models of reflectance and texture for natural materials and objects