Motion based Correspondence for Distributed 3D tracking of multiple dim objects

Ashok Veeraraghavan

Problem Setting

Constraints

R, T

??

R, T

??

R, T ??

R, T ??

OutlineTracking Algorithm

Implemented at each camera node. Correspondence problem for dim targets. Motion-Based Correspondence Algorithm

Implemented at central processor Recovering Camera Position and

Orientation Recovering 3D tracks using triangulation.

Experimental Setup Objective :

Reconstruct the 3D trajectories of the bees so as to study the response of bees to visual stimuli.

Outdoor Bee Tunnel with the surrounding walls texture systematically varied

Study relationship of flight patterns to visual stimulii.

Two Fixed Cameras. Free Flying bees are the targets to be tracked. Typically the bees are about 20-50 meters away from the camera. Multiple Targets: On average each frame contains about 6-8 bees. Occupy about 5-10 pixels at closet range: Low SNR Objective : Reconstruct the 3D trajectories of the bees so as to

study the response of bees to visual stimuli.

Tracking Algorithm Background Subtraction

Background variations are assumed to be much slower than the target.

Dynamic background estimated using a temporal low pass filter for each pixel.

Connected Component Analysis Morphological processing to connect pixels belonging

to same target.

Probabilistic Data Association Blob Tracking algorithm.

Background Subtraction and Connected Component Analysis

Background Subtraction and Connected Component Analysis

Adaptive Velocity Motion Model

v

r

Correspondence Problem for Dim Targets

Correspondence across camera Views Associating the objects found in various views Especially tricky for multiple dim objects

Dim Targets Low SNR Very Small Targets – (order of few pixels ) Features extraction unreliable

Appearance based correspondence Appearance varies with view Unreliable for dim targets

Motion Based Correspondence Rubin and Richards (1985)

Rao, Yilmaz and Shah (2002)- Maxima of spatio-temporal curvature as Dynamic Instants

Courtesy: [Rao2002]

Dynamic Instants Detects any start instant, stop instant, non-

smooth change in speed, maximal curvature of 3D tracks. Eg., Start Instants

Courtesy: [Rao2002]

Detected Dynamic Instants

Courtesy: [Rao2002]

Correspondence Across Views

External Calibration Internal Camera parameters known. External Orientation of the cameras to be

estimated from correspondence data obtained by matching tracks across views.

Simple non-linear optimization implemented (Levenberg-Marquardt).

Distance between cameras (Baseline) approximately known.

Optimization is local. Requires good initial estimate.

3D flight Paths using Triangulation Internal camera parameters known. External camera calibration parameters

estimated from point correspondences. 3D tracks obtained using Triangulation.

3D Flight Paths

3D Flight Paths

Future Work Human Surveillance. Work with multiple (more than 2 cameras)

cameras. Study the trade-off between bandwidth

and efficiency. Especially can we also add some

appearance information to each target so that limited view reconstruction of target is possible?

Thank You