Shape-from-X Class 11 Some slides from Shree Nayar. others.
-
Upload
melinda-clarke -
Category
Documents
-
view
215 -
download
0
Transcript of Shape-from-X Class 11 Some slides from Shree Nayar. others.
- Slide 1
- Shape-from-X Class 11 Some slides from Shree Nayar. others
- Slide 2
- 3D photography course schedule (tentative) LectureExercise Sept 26Introduction- Oct. 3Geometry & Camera modelCamera calibration Oct. 10Single View MetrologyMeasuring in images Oct. 17Feature Tracking/matching (Friedrich Fraundorfer) Correspondence computation Oct. 24Epipolar GeometryF-matrix computation Oct. 31Shape-from-Silhouettes (Li Guan) Visual-hull computation Nov. 7Stereo matchingProject proposals Nov. 14Structured light and active range sensing Papers Nov. 21Structure from motionPapers Nov. 28Multi-view geometry and self-calibration Papers Dec. 5Shape-from-XPapers Dec. 123D modeling and registrationPapers Dec. 19Appearance modeling and image-based rendering Final project presentations
- Slide 3
- Shape-from-X X =Shading X =Multiple Light Sources (photometric stereo) X =Texture X =Focus/Defocus X =Specularities X =Shadows X =
- Slide 4
- Shape from shading Shading as a cue for shape reconstruction What is the relation between intensity and shape? Reflectance Map
- Slide 5
- Relates image irradiance I(x,y) to surface orientation (p,q) for given source direction and surface reflectance Lambertian case: : source brightness : surface albedo (reflectance) : constant (optical system) Image irradiance: Letthen
- Slide 6
- Lambertian case Reflectance Map (Lambertian) cone of constant Iso-brightness contour Reflectance Map
- Slide 7
- Lambertian case iso-brightness contour Note: is maximum when Reflectance Map
- Slide 8
- Shape from a Single Image? Given a single image of an object with known surface reflectance taken under a known light source, can we recover the shape of the object? Given R(p,q) ( (p S,q S ) and surface reflectance) can we determine (p,q) uniquely for each image point? NO
- Slide 9
- Solution Add more constraints Shape-from-shading Take more images Photometric stereo
- Slide 10
- Stereographic Projection (p,q)-space (gradient space) (f,g)-space Problem (p,q) can be infinite when Redefine reflectance map as
- Slide 11
- Occluding Boundaries and are known The values on the occluding boundary can be used as the boundary condition for shape-from-shading
- Slide 12
- Image Irradiance Constraint Image irradiance should match the reflectance map Minimize (minimize errors in image irradiance in the image)
- Slide 13
- Smoothness Constraint Used to constrain shape-from-shading Relates orientations (f,g) of neighboring surface points : surface orientation under stereographic projection Minimize (penalize rapid changes in surface orientation f and g over the image)
- Slide 14
- Shape-from-Shading Find surface orientations (f,g) at all image points that minimize smoothness constraint weight image irradiance error Minimize
- Slide 15
- Results
- Slide 16
- Slide 17
- Solution Add more constraints Shape-from-shading Take more images Photometric stereo
- Slide 18
- Photometric Stereo
- Slide 19
- We can write this in matrix form: Image irradiance: Lambertian case: Photometric Stereo : source brightness : surface albedo (reflectance) : constant (optical system)
- Slide 20
- Solving the Equations inverse
- Slide 21
- More than Three Light Sources Get better results by using more lights Least squares solution: Solve for as before Moore-Penrose pseudo inverse
- Slide 22
- Color Images The case of RGB images get three sets of equations, one per color channel: Simple solution: first solve for using one channel Then substitute known into above equations to get Or combine three channels and solve for
- Slide 23
- Computing light source directions Trick: place a chrome sphere in the scene the location of the highlight tells you the source direction
- Slide 24
- For a perfect mirror, light is reflected about N Specular Reflection - Recap We see a highlight when Then is given as follows:
- Slide 25
- Computing the Light Source Direction Can compute N by studying this figure Hints: use this equation: can measure c, h, and r in the image N rNrN C H c h Chrome sphere that has a highlight at position h in the image image plane sphere in 3D
- Slide 26
- Depth from Normals Get a similar equation for V 2 Each normal gives us two linear constraints on z compute z values by solving a matrix equation V1V1 V2V2 N
- Slide 27
- Limitations Big problems Doesnt work for shiny things, semi- translucent things Shadows, inter-reflections Smaller problems Camera and lights have to be distant Calibration requirements measure light source directions, intensities camera response function
- Slide 28
- Trick for Handling Shadows Weight each equation by the pixel brightness: Gives weighted least-squares matrix equation: Solve for as before
- Slide 29
- Original Images
- Slide 30
- Results - Shape Shallow reconstruction (effect of interreflections) Accurate reconstruction (after removing interreflections)
- Slide 31
- Results - Albedo No Shading Information
- Slide 32
- Original Images
- Slide 33
- Results - Shape
- Slide 34
- Results - Albedo
- Slide 35
- Results 1.Estimate light source directions 2.Compute surface normals 3.Compute albedo values 4.Estimate depth from surface normals 5.Relight the object (with original texture and uniform albedo)
- Slide 36
- Shape from texture Obtain normals from texture element (or statistics) deformations, Examples from Angie Loh
- Slide 37
- Shape from texture Obtain normals from texture element (or statistics) deformations, Examples from Angie Loh
- Slide 38
- Depth from focus Sweep through focus settings most sharp pixels correspond to depth (most high frequencies)
- Slide 39
- Depth from Defocus More complicates, but needs less images Compare relative sharpness between images
- Slide 40
- Shape from shadows S. Savarese, M. Andreetto, H. Rusmeier, F. Bernardini, P. Perona, 3D Reconstruction by Shadow Carving: Theory and Practical Evaluation, International Journal of Computer Vision (IJCV), vol 71, no. 3, pp. 305-336, March 2007.
- Slide 41
- Shape from specularities Toward a Theory of Shape from Specular Flow Y. Adato, Y. Vasilyev, O. Ben-Shahar, T. Zickler Proc. ICCV07
- Slide 42
- Next class: 3d modeling and registration