3D Rigid/Nonrigid RegistrationRegistration 1)Known features, correspondences, transformation model...
-
date post
21-Dec-2015 -
Category
Documents
-
view
217 -
download
0
Transcript of 3D Rigid/Nonrigid RegistrationRegistration 1)Known features, correspondences, transformation model...
3D Rigid/Nonrigid Registration
1) Known features, correspondences, transformation model – feature based
2) Specific motion type, unknown correspondences – feature based
3) Known transformation model, unknown correspondences – region based
4) Specific motion model – feature based
5) Unknown motion model, unknown correspondences – region based
Visual Motion
Jim Rehg
(G.Tech)
Motion (Displacement) of Environment
Imageplane dt
tdt
)()(
rv
SceneFlowMotion
Field
)(tr
)()( tt vPw
Visual motion results from the displacement of the scene with respect to a fixed camera (or vice-versa).Motion field is the 2-D velocity field that results from a projection of the 3-D scene velocities
Examples of Visual Motion
Examples of Visual Motion
Examples of Visual Motion
Applications of Motion Analysis
Visual tracking
Structure recovery
Robot (vehicle) navigation
Applications of Motion Analysis
Visual tracking
Structure recovery
Robot (vehicle) navigation
Motion Segmentation
Where are the independently moving objects (and how many are there)?
Optical Flow
2-D velocity field describing the apparent motion in an image sequence
A vector at each pixel indicates its motion (between a pair of frames).
Ground truthHorn and Schunk
Optical Flow and Motion Field
In general the optical flow is an approximation to the motion field.
When the scene can be segmented into rigidly moving objects (for example) the relationship between the two can be made precise.
We can always think of the optical flow as summarizing the temporal change in an image sequence.
Computing Optical Flow
Courtesy of Michael Black
Cost Function for Optical Flow
Ryx
SSD tyxItvyuxIvuE,
2)],,()1,,([),(
Courtesy of Michael Black
Lucas-Kanade Method
Brute-force minimization of SSD error can be inefficient and inaccurateMany redundant window evaluations
Answer is limited to discrete u, v pairs
Lucas-Kanade Method
Problems with brute-force minimization of SSD errorMany redundant window evaluations
Answer is limited to discrete u, v pairs
Related to Horn-Schunk optical flow equations
Several key innovationsEarly, successful use of patch-based model in low-level vision. Today
these models are used everywhere.
Formulation of vision problem as non-linear least squares optimization, a trend which continues to this day.
Optical Flow Estimation
Optical Flow Estimation
Optical Flow Constraint
I_t is one-to-one in the first
Iteration and changes as u,v changes
Optimization
Optimization
Optimization
Quality of Image Patch
Eigenvalues of the matrix contain information about local image structureBoth eigenvalues (close to) zero: Uniform area
One eigenvalue (close to) zero: Edge
No eigenvalues (close to) zero: Corner
Contributions of Lucas-Kanade
Basic idea of patch or template is very old (goes back at least to Widrow)
But in practice patch models have worked much better than the alternatives:Point-wise differential equations with smoothnessEdge-based descriptions
Patches provide a simple compact enforcement of spatial continuity and support (robust) least-squares estimators.
Apparent Motion
Apparent motion of objects on the image planeCaution required!!
Consider a perfectly uniform sphere that is rotating but no change in the light directionOptic flow is zero
Perfectly uniform sphere that is stationary but the light is changingOptic flow exists
Hope – apparent motion is very close to the actual motion
Courtesy: E. Trucco and A. Verri, “Introductory techniques for 3D Computer Vision”
Aperture Problem
1) The aperture problem arises due to
uniformly colored surfaces in the scene. In the
absence of strong lighting effects, a uniform
surface in the scene appears nearly uniform in the
projection. It is then impossible to determine
correspondences within these regions.
2) We are able to only measure the component
Of the optic flow that is in the direction of the
Intensity gradient. (Unable to measure component
In the tangential direction, edge).
Aperture Problem
We can measure Terms that can be measuredTerms to be computedNumber of equations - 1
The component of the motion field that is orthogonal to the spatial image gradient is not constrained by the image brightness constancy assumption
IntuitivelyThe component of the flow in the gradient direction is determinedThe component of the flow parallel to an edge is unknown
Courtesy: E. Trucco and A. Verri, “Introductory techniques for 3D Computer Vision”
tEyExE ,,
dtdydtdx ,
Different physical motion but same measurable motion within a fixed window