Flexible Bump Map Capture From VideoJames A. Paterson and Andrew W. Fitzgibbon

University of Oxford{jamie,awf}@robots.ox.ac.uk

Calibration

•Requirement: Avoid physical measurement

•Use set of calibration images of light reflected in mirror, intersect ray on which light must lie in at least two images to find relative position

Calibration geometry Images used in calibration

•Measurement error makes closed from solution inaccurate. Use a non-linear optimisation scheme to optimise camera positions, focal length and light position, minimizing reprojection error:

Current issues and possible extensions

•The Lambertian lighting approximation is not perfect, but by using more than 3 images we can avoid irregular intensity values caused by specular highlights and self shadowing. Determining exactly which pixels from which images to include in the computation is non-trivial, but we would hope to maximise on the range of light directions used in calculating each normal.

•Currently only planar surfaces can be captured, however by improving the geometrical model of the subject we hope to extend the process to highly curved surfaces.

•The positions of the markers on the subject must be accurately known, however in e.g. archive footage this information will not be available. Given sufficient images we can include marker positions in the non-linear optimisation, removing this restriction

Bump Mapping

…is a generic term grouping techniques which use 2D texture to apply 3D detail to a surface. This includes Displacement mapping and our primary interest Normal mapping. We present an improved method for normal map acquisition from real world objects.

Standard method Equivalent new method(s)

•Standard normal map capture rig is accurate, but bulky.

•Requires static camera and multiple static lights.

•Proposed new version is portable, only requires one light

•Relative camera-light position is calculated, and must be constant

Calculating a surface normal map

To find the normal at a particular point on the surface, we observe the intensity of that point under different lighting conditions. We model lighting effects via the standard Lambertian equations. Normal map capture requires at least 3 different intensity values for a point on the surface, captured with different known light directions. Then we can find the normal at that point as follows:

The Algorithm

Input Images – A video sequence of the object being moved in front of a calibrated camera-light rig, or equivalently a portable camera-light rig moved around whilst viewing a static object

Fronto-parallel Images – By tracking markers on the object we can compute relative camera-light position, and re-render the object using an inverse perspective projection. This is done via 2D homography for planar markers.

Sparse Normal Map Images – We can now compute surface normals as outlined previously. Here we see a sparse set of normals superimposed on input video frames

Dense Normal Map Images – A higher density grid superimposed on a longer sequence of input images as a bump mapped quadrilateral. Non-Lambertian lighting conditions (specular highlights, self shadowing) lead to occasional spurious results.

Limitation: Planar surfaces only at the moment!