Shapelets Correlated with Surface Normals Produce Surfaces Peter Kovesi School of Computer Science &...
-
Upload
sarai-abdon -
Category
Documents
-
view
220 -
download
2
Transcript of Shapelets Correlated with Surface Normals Produce Surfaces Peter Kovesi School of Computer Science &...
Shapelets Correlated with Surface Normals Produce Surfaces
Peter Kovesi
School of Computer Science & Software Engineering
The University of Western Australia
Shape from Shading and Shape from Texture attempt to
• Calculate the surface normal direction at each point in the image, and then
• Infer the surface shape from the surface normals.
Surface normals are usually calculated in terms of slant and tilt
N
slant
Slant is the angle between the viewing direction and the surface normal
surface
eye
Tilt is the angle of the projection of the surface normal on a plane perpendicular to the viewing direction
viewer plane perpendicularto viewing direction
N
slant
tilt
projection of surface normal
Tilt often has an ambiguity of Problem:
Tilt often has an ambiguity of Problem:
Problem:
Regularization schemes are often used to deal with noise. However, they typically involve the minimization of an Energy Term that has mixed units
€
E = α fx − ˆ f x∑2
+β ˆ f xx
2
+ γ I − ˆ n ⋅s2
+ ...
surface gradient error(highly nonlinear with surface angle)
bending energy shading error
Problem:
Regularization schemes are often used to deal with noise. However, they typically involve the minimization of an Energy Term that has mixed units
€
E = α fx − ˆ f x∑2
+β ˆ f xx
2
+ γ I − ˆ n ⋅s2
+ ...
surface gradient error(highly nonlinear with surface angle)
bending energy
• The weighting parameters have to act as unit conversion operators.• The appropriate values will be dependent on scale. • This can be problematic if the regularization is being applied within a
multi-scale framework. • Discontinuities have to be automatically detected and treated specially.
shading error
Alternative approach:
Project the surface gradients onto a set of integrable basis functions and reconstruct the surface from these.
Frankot and Chellappa [1988] first introduced this approach using Fourier basis functions.
Features of work presented here:• Localized, non-orthogonal basis functions are used.• Correlations with basis functions are performed with
respect to slant and tilt rather than in terms of gradient in x and y.
• Ambiguities of in tilt can be considered.• Low quality reconstructions are possible with slant
information alone.
Decomposing and Reconstructing Signals with Basis Functions
signal
filtered at scale 2
filtered at scale 4
filtered at scale 6
Reconstruction obtained by summing band-passed signals over 6 scales
band-pass filters/wavelets
We only have the gradient of the surface…
We only have the gradient of the surface…
Differentiation is linear
€
f ∗ ′ ′ g = ′ f ∗ ′ g
surface band-pass filter
surface gradient
We only have the gradient of the surface…
Differentiation is linear
A band-passed version of the signal can be obtained from a correlation performed between the gradient of the signal and integral of the band-pass filter.
€
f ∗ ′ ′ g = ′ f ∗ ′ g
surface band-pass filter Integral of band-pass filter(shapelet gradient)
surface gradient
Choice of Shapelet
• The gradient of shapelet function must satisfy the admissibility condition of zero mean.
• Must ensure preservation of phase information in the signal.
• The bank of shapelet filters should provide a uniform coverage of the signal spectrum so that it is faithfully reconstructed.
These requirements are satisfied by a Gaussian
Note that correlating the gradient of one shapelet filter with the gradient of the signal will correspond to extracting a band of frequencies from the signal gradient.
Note that correlating the gradient of one shapelet filter with the gradient of the signal will correspond to extracting a band of frequencies from the signal gradient.
The signal gradient spectrum differs from the original signal spectrum in that its amplitudes are scaled by the frequency.
Note that correlating the gradient of one shapelet filter with the gradient of the signal will correspond to extracting a band of frequencies from the signal gradient.
The signal gradient spectrum differs from the original signal spectrum in that its amplitudes are scaled by the frequency.
The sum of the shapelet gradient transfer functions should form a curve that is inversely proportional to frequency to counteract the scaling of the gradient spectrum. The net result will then be an even coverage of the original signal spectrum.
Shapelet bank formed by five Gaussians with height proportional to scale.
Transfer functions of the gradients of the shapelets above.
The sum forms a curve that is approximately inversely proportional to frequency.
A bank of shapelets will not form an orthogonal basis set
This does not matter if we have a strongly redundant basis set. (Achieved by using a continuous wavelet transform.)
A non-orthogonal basis set can be considered to be the sum of several orthogonal basis sets.- We get accurate reconstruction up to a scale factor
• The scaling of the reconstruction due to the redundancy of the basis set can be determined and corrected if required.
• Overcomplete basis sets have the advantage that small changes in local features result in correspondingly small changes in the basis coefficients.
• Overcompleteness also provides robustness to noise.
In Summary:
• Correlate the gradients/normals of the surface with the gradients/normals of a set of basis functions.
• These correlation results correspond to bandpassed versions of the original signal.
• Reconstruct surface by summing correlation results.
Perform correlation in terms of slant and tilt separately
- slant and tilt are calculated by different means.- slant and tilt have differing degrees of uncertainty.
Correlation of Slant
Slant lies in the range 0 - which only provides gradient magnitude information.
The slant correlation at scale i is formed by
With no tilt information this correlation measure matches positive and negative gradients equally. (No distinction between concave or convex shape)
)tan(=∇
€
C∇i = ∇ f ∗∇ si
gradient of surface
gradient of shapelet at scale i
Correlation of Tilt
• If gradient magnitudes match and tilts match correlation is +ve
• If gradient magnitudes match and tilts differ by correlation is -ve
• If tilts differ by /2 correlation is 0
Use the cosine of the tilt difference as a correlation measure
€
Cτi = cos(τ f − τ si)
tilt of surface tilt of shapelet at scale i
Overall Correlation
€
Ci = C∇i ⋅Cτi
Reconstruction
€
R = Ci
i
∑
synthetic test surface
needle diagram ofsurface normals
surface normals reconstruction over 2 scales
reconstruction over 4 scales reconstruction over 6 scales
Shapelet reconstruction using the gradients of the range data
Range data from OSU database
Slant and tilt with Gaussian noise of 0.3 radians standard deviation
Shapelet reconstruction
Terzopoulos multigrid reconstructionFrankot Chellappa reconstruction
Allowing for Tilt Ambiguity of
)(cos2sfC −=
When tilts are aligned value is 1When tilts are orthogonal value is 0When tilts are opposite value is 1
… results in rectification of the band-pass components of the surface
Reconstruction with a tilt ambiguity of
reconstruction oframps surface
bumps surface
reconstruction
Reconstruction with slant data alone
Slant-only reconstruction of bumps Slant-only reconstruction of ramps
Slant-only reconstruction of sincSinc function
Slant-only reconstruction of face …after segmentation and negation
Shapelet reconstruction
Real data: Surface normals from texture
Sand ridges: Slant set to /4 at all pointsTilt set to 0 for white regions, for black regions
Shapelet reconstruction
Shape from Occluding Contours
Manually marked contours
Surface normals at occluding contours are perpendicular to viewing direction.
This is a powerful cue to shape.
At all points slant is set to 0, except along contours where slant is set to , and tilt is set to be perpendicular to contour.
This gradient field is not integrable.
A crude approximation of the surface normals in the scene…
Shapelet reconstruction
Manually drawn contours
Conclusions
• Reconstructions from surface normals using shapelets is simple and the process is robust to noise.
• The use of basis functions implicitly imposes a continuity constraint in the reconstruction.
• Being able to treat slant and tilt separately permits data with tilt ambiguity, or no tilt data, to be considered.
• Much can be made from limited amounts of surface normal data. This allows some some aspects of structure to be deduced from line drawings.