Post on 18-Apr-2017
Optical flow (motion Optical flow (motion vector) computationvector) computation
Course: Course: Computer Graphics and Image ProcessingComputer Graphics and Image Processing
Semester:Semester: Fall 2002Fall 2002
Presenter:Presenter: Nilesh Ghubade (nileshg@temple.edu)Nilesh Ghubade (nileshg@temple.edu)
Advisor: Advisor: Dr Longin Jan LateckiDr Longin Jan Latecki
Dept:Dept: Computer and Information Science, Computer and Information Science, Temple University, Philadelphia, PA-19122Temple University, Philadelphia, PA-19122
Motion AnalysisMotion AnalysisThree groups of motion-related problems:Three groups of motion-related problems:
1.1. Motion detectionMotion detection Registers any detected motion. Registers any detected motion. Single static camera.Single static camera. Used for security purposes.Used for security purposes.
2.2. Moving object detection and locationMoving object detection and location Determination of object trajectory.Determination of object trajectory. Static camera, moving objects OR Moving camera, Static camera, moving objects OR Moving camera,
static objects OR Both camera and objects moving.static objects OR Both camera and objects moving.3.3. Deriving 3D propertiesDeriving 3D properties
Use of set of 2D projections acquired at different Use of set of 2D projections acquired at different time instants of object motion.time instants of object motion.
Object motion Object motion assumptionsassumptions
Maximum velocity.Maximum velocity.
Small acceleration.Small acceleration.
Common motion of object points.Common motion of object points. Mutual correspondence.Mutual correspondence.
Cmax * dt
t0t1
t2
Differential motion Differential motion analysisanalysis
Simple subtraction of images acquired at Simple subtraction of images acquired at different instants in time makes motion different instants in time makes motion detection possible, assuming stationary detection possible, assuming stationary camera position and constant illumination.camera position and constant illumination.
Difference image is a Difference image is a binary image binary image subtract two consecutive images.subtract two consecutive images.
Cumulative difference image:Cumulative difference image: Reveals motion direction.Reveals motion direction. Time related motion properties.Time related motion properties. Slow motion and small object motion.Slow motion and small object motion.Constructed from sequence of ‘n’ images taking first Constructed from sequence of ‘n’ images taking first
image as the reference image.image as the reference image.
ExampleExampleMotion in front of a security camera.
Sobel filter edge detection.
Motion Detection- Sobel Motion Detection- Sobel filter filter 10 frames/second 15 frames/second
15 frames/second25 frames/second
Optical FlowOptical Flow Optical FlowOptical Flow reflects the image changes reflects the image changes
due to motion during a time interval due to motion during a time interval dt.dt. Optical flow fieldOptical flow field is the velocity field that is the velocity field that
represents the 3D motion of object points represents the 3D motion of object points across a 2D image.across a 2D image.
It should not be sensitive to illumination It should not be sensitive to illumination changes and motion of unimportant changes and motion of unimportant objects (e.g. shadows)objects (e.g. shadows)
Exceptions:Exceptions:1.1. Non-zero optical flowNon-zero optical flow fixed sphere illuminated by a fixed sphere illuminated by a
moving source.moving source.2.2. Zero optical flow Zero optical flow smooth sphere under constant smooth sphere under constant
illumination, although there is rotational motion and illumination, although there is rotational motion and true non-zero motion field.true non-zero motion field.
Optical Flow Optical Flow (continued…)(continued…)
Aim is to determine optical flow that Aim is to determine optical flow that corresponds with true motion field.corresponds with true motion field.
Necessary Necessary pre-conditionpre-condition of of subsequent higher level motion subsequent higher level motion processing processing stationary or moving stationary or moving camera.camera.
Provides tools to determine motion Provides tools to determine motion parameters, relative distances of parameters, relative distances of objects in the image etc..objects in the image etc..
Example:Example:t1 t2
AssumptionsAssumptionsOptical flow computation is based on Optical flow computation is based on
two assumptions:two assumptions:
1.1. The observed brightness of any The observed brightness of any object point is constant over time.object point is constant over time.
2.2. Nearby points in the image plane Nearby points in the image plane move in a similar manner (the move in a similar manner (the velocity smoothnessvelocity smoothness constraint). constraint).
Optical flow Optical flow computationcomputation
The optical flow field represented in the The optical flow field represented in the form of Velocity vector: form of Velocity vector:
Length of the vector determines the Length of the vector determines the magnitude of velocity.magnitude of velocity.
Direction of the vector determines the Direction of the vector determines the direction of motion.direction of motion.
Global optical flow estimation—Global optical flow estimation— Local constraints are propagated globally.Local constraints are propagated globally. But errors also propagate across the solution.But errors also propagate across the solution.
Local optical flow estimation—Local optical flow estimation— Divide image into smaller regions.Divide image into smaller regions. But inefficient in the areas where spatial But inefficient in the areas where spatial
gradients change slowly gradients change slowly here use global here use global method, neighboring image parts contribute.method, neighboring image parts contribute.
Forms of Forms of motionmotionTranslation at constant Translation at constant
distance from the distance from the observer.observer.
Set of parallel motion Set of parallel motion vectors.vectors.
Translation in depth Translation in depth relative to the relative to the observer.observer.
Set of vectors having Set of vectors having common focus of common focus of expansion.expansion.
Rotation at constant Rotation at constant distance from view distance from view axis.axis.
Set of concentric Set of concentric motion vectors.motion vectors.
Rotation of planar Rotation of planar object perpendicular to object perpendicular to the view axis.the view axis.
One or more sets of One or more sets of vectors starting from vectors starting from straight line straight line segments.segments.
RepresentationRepresentationLocate the position of a pixel (row,col) in the current image by computing shortest Euclidean distance with respect to 5-by-5 neighborhood in the next consecutive frame.
1616 1515 1414 1313 12121717 44 33 22 11111818 55 00 11 10101919 66 77 88 992020 2121 2222 2323 2424
ExperimentsExperiments
3-by-3 neighborhood
Experiments Experiments (contd…)(contd…)
5-by-5 neighborhood
Experiments Experiments (contd…)(contd…)
Experiments Experiments (contd…)(contd…)
Applications of optical Applications of optical flowflow
Object motion detection.Object motion detection.
Action recognition.Action recognition.
Active vision or structure of motion – Active vision or structure of motion – Reconstruction of 3D object by computing depth information.Reconstruction of 3D object by computing depth information. If you have distance (depth) maps, you can reconstruct If you have distance (depth) maps, you can reconstruct surface surface of of
the object.the object.
Facial expression recognition: referenceFacial expression recognition: referencehttp://athos.rutgers.edu/~decarlo/pubs/ijcv-face.pdfhttp://athos.rutgers.edu/~decarlo/pubs/ijcv-face.pdf
Thank youThank you