Slide 1

Slide 2

Optical Flow

Slide 3

Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

Slide 4

Optical Flow: Where do pixels move to?

Slide 5

Motion is a basic cue Motion can be the only cue for segmentation

Slide 6

Motion is a basic cue Motion can be the only cue for segmentation

Slide 7

Motion is a basic cue Even impoverished motion data can elicit a strong percept

Slide 8

Applications tracking structure from motion motion segmentation stabilization compression mosaicing

Slide 9

Optical Flow Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

Slide 10

Definition of optical flow OPTICAL FLOW = apparent motion of brightness patterns Ideally, the optical flow is the projection of the three-dimensional velocity vectors on the image

Slide 11

Caution required ! Two examples : 1. Uniform, rotating sphere O.F. = 0 2. No motion, but changing lighting O.F. 0

Slide 12

Mathematical formulation I (x,y,t) = brightness at (x,y) at time t Optical flow constraint equation : Brightness constancy assumption:

Slide 13

Optical Flow Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

Slide 14

The aperture problem 1 equation in 2 unknowns

Slide 15

Aperture Problem: Animation

Slide 16

The Aperture Problem 0

Slide 17

Slide 18

What is Optical Flow, anyway? Estimate of observed projected motion field Not always well defined! Compare: Motion Field (or Scene Flow) projection of 3-D motion field Normal Flow observed tangent motion Optical Flow apparent motion of the brightness pattern (hopefully equal to motion field) Consider Barber pole illusion

Slide 19

Slide 20

Optical Flow Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

Slide 21

Horn & Schunck algorithm Additional smoothness constraint : besides OF constraint equation term minimize e s + e c

Slide 22

The calculus of variations look for functions that extremize functionals and

Slide 23

Calculus of variations Suppose 1. f(x) is a solution 2. (x) is a test function with (x 1 )= 0 and (x 2 ) = 0 for the optimum :

Slide 24

Calculus of variations Using integration by parts : Therefore regardless of (x), then Euler-Lagrange equation

Slide 25

Calculus of variations Generalizations 1. Simultaneous Euler-Lagrange equations : 2. 2 independent variables x and y

Slide 26

Hence Now by Gausss integral theorem, so that = 0 Calculus of variations

Slide 27

Given the boundary conditions, Consequently, for all test functions . is the Euler-Lagrange equation

Slide 28

Horn & Schunck The Euler-Lagrange equations : In our case, so the Euler-Lagrange equations are is the Laplacian operator

Slide 29

Horn & Schunck Remarks : 1. Coupled PDEs solved using iterative methods and finite differences 2. More than two frames allow a better estimation of I t 3. Information spreads from corner-type patterns

Slide 30

Slide 31

Horn & Schunck, remarks 1. Errors at boundaries 2. Example of regularization (selection principle for the solution of ill-posed problems)

Slide 32

Results of an enhanced system

Slide 33

Optical Flow Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

Slide 34

Slide 35

Slide 36

Slide 37

Optical Flow Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

Slide 38

Slide 39

Slide 40

Slide 41

Optical Flow Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

Slide 42

Slide 43

Slide 44

Slide 45

Slide 46

Slide 47

Slide 48

Slide 49

Slide 50

Slide 51

Slide 52

Slide 53

Slide 54

Slide 55

Slide 56

Slide 57

Slide 58

Slide 59

Slide 60

Slide 61

Slide 62

Slide 63

Slide 64

Slide 65

Slide 66

Slide 67

Slide 68

Slide 69

Slide 70

Optical Flow Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

Slide 71

Slide 72

Slide 73

Slide 74

Optical Flow Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

Slide 75

Slide 76

Slide 77

Slide 78

Slide 79

Slide 80

Optical Flow Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

Slide 81

Slide 82

Slide 83

Slide 84

Slide 85

Slide 86

Slide 87

Textured Motion http://www.cs.ucla.edu/~wangyz/Research/w yzphdresearch.htm http://www.cs.ucla.edu/~wangyz/Research/w yzphdresearch.htm Natural scenes contain rich stochastic motion patterns which are characterized by the movement of a large amount of particle and wave elements, such as falling snow, water waves, dancing grass, etc. We call these motion patterns "textured motion".