Tracking · 2013. 3. 4. · Time update (a priori estimates) Measurement update (a posteriori...

Introduction Tracking with static camera Tracking with moving camera

Tracking

Hakan Ardo

March 4, 2013

Hakan Ardo Tracking March 4, 2013 1 / 57

Introduction Tracking with static camera Tracking with moving cameraState space Sliding Window Detection

Outline

1 IntroductionState spaceSliding Window Detection

2 Tracking with static cameraGreedyKalman filterParticle filter

3 Tracking with moving cameraSelf-motion



State space

State: position X = (X ,Y ,Z , 1), velocity, v = (vx , vy , vz , 0), ...

Observation: detection in image x = (x , y , 1)

Observation model: λx = PX

Dynamic model: Xt+1 = Xt + vt and vt+1 = vt

Sthocastic dynamic model: Introduce noise, random numbers q and w

Xt+1 = Xt + vt + q

vt+1 = vt + w

Sthocastic observation model: Introduce noise, a random number r

λx = PX + r



Sliding window detectors



Detection probability


Introduction Tracking with static camera Tracking with moving cameraGreedy Kalman filter Particle filter

Greedy tracker



UNC Chapel Hill Computer Science Slide 1

STC Lecture Series

An Introduction to the Kalman FilterGreg Welch and Gary Bishop

University of North Carolina at Chapel Hill

Department of Computer Science

http://www.cs.unc.edu/~welch/kalmanLinks.html



Sanjay Patil1 and Ryan Irwin2

Graduate research assistant1,REU undergrad2

Human and Systems Engineering

URL: www.isip.msstate.edu/publications/seminars/msstate/2005/particle/

HUMAN AND SYSTEMS ENGINEERING:Gentle Introduction to Particle Filtering




Some Intuition




First Estimate

ˆ x 1 = z1

ˆ σ 21 = σ 2z1

Conditional Density Function

14121086420-2

N(z1,σz12 )

z1 σ 2z1

,




Second Estimate


z2 σ 2z2

,

ˆ x 2 = ...?

ˆ σ 22 = ...? 14121086420-2

N(z2,σz22 )




Combine Estimates

= σ z2

2 σ z1

2 +σ z2

2( )[ ] z1+ σ z1

2 σ z1

2 +σ z2

2( )[ ] z2ˆ x 2= ˆ x 1 + K2 z2 − ˆ x 1[ ]

whereK2 = σ z1

2 σ z1

2 +σ z2

2( )




Combine Variances

1 σ 2 = 1 σ z1

2( )+ 1 σ z2

2( )2




Combined Estimate Density

ˆ x =

ˆ σ 2 = σ 22

ˆ x 2

14121086420-2


N( σ 2)ˆ x,ˆ




Add Dynamics

dx/dt = v + w

wherev is the nominal velocityw is a noise term (uncertainty)



Page 7 of 20Particle Filtering – Gentle Introduction and Implementation Demo

Particle filtering algorithm step-by-step (1)




Particle filtering step-by-step (2)




Some Details

xx AAxx wwzz HH xx........

= +=




Discrete Kalman Filter

Maintains first two statistical moments

process state (mean)

error covariance

z

y

x




Necessary Models

measurementmodel

dynamicmodel

previous state next state

statemeasurement

image plane( u , v )




The Process Model

x k+1 = Axk + wk

zk = Hxk + vk

Process Dynamics

Measurement




Process Dynamics

x k+1 = Axk + wk

xk ∈ Rn contains the states of the process

state vector




Process Dynamics

nxn matrix A relates state at time step k to time step k+1

state transition matrix

x k+1 = Axk + wk




Process Dynamics

process noise

wk ∈ Rn models the uncertainty of the process

wk ~ N(0, Q)

x k+1 = Axk + wk




Measurement

zk = Hxk + vk

zk ∈ Rm is the process measurement

measurement vector




Measurement

zk = Hxk + vk

state vector




Measurement

mxn matrix H relates state to measurement

measurement matrix

zk = Hxk + vk




Measurement

measurement noise

zk ∈ Rm models the noise in the measurement

vk ~ N(0, R)

zk = Hxk + vk




State Estimates

a priori state estimate

a posteriori state estimate

ˆ x –k

ˆ xk




Estimate Covariances

a priori estimate error covariance

a posteriori estimate error covariance

Pk– = E[(xk- xk

–)(xk - xk–)T] ˆ ˆ

Pk = E[(xk- xk)(xk - xk)T] ˆ ˆ




Filter Operation

Time update (a priori estimates)

Measurement update (a posteriori estimates)

Project state and covariance forwardto next time step, i.e. predict

Update with a (noisy) measurementof the process, i.e. correct




Time Update (Predict)

state

error covariance




Measurement Update (Correct)




Time Update (Predict)

a priori state and error covariance

ˆ xk+1 = Axk–

Pk+1 = APk A + Q–




Measurement Update (Correct)

a posteriori state and error covariance

Kalman gain

ˆ xk = xk + Kk (zk - Hxk )ˆ ˆ – –

Pk = (I - Kk H)Pk–

Kk = Pk HT(HPk HT + R)-1 – –




Filter Operation

Time Update(Predict)

Measurement

(Correct)Update



Kalman filter tracker



Particle filter



Patricle filter states

Current state {x

(1)k , x

(2)k , x

(3)k , · · · , x (n)

k

}Predicted state (a priori state extimate){

x−(1)k+1 , x

−(2)k+1 , x

−(3)k+1 , · · · , x

−(n)k+1

}Corrected state (a posteriori state extimate){

x(1)k+1, x

(2)k+1, x

(3)k+1, · · · , x

(n)k+1

}



Dynamic model

Any sthocastic model from which we can sample

f (xk+1 |xk )

Example: The dynamic model from the kalman filter

xk+1 = Axk + wk

f (xk+1 |xk ) = N(Axk ,Q)



Particle filter prediction step

Propagate each particle i , separately

The prediction x−(i)k+1 is choosen as a sample from

f(xk+1

∣∣∣x (i)k

)



Measurement model

Any sthocastic model from which we can calculate likelihoods

f (zk |xk )

Example: The measurement model from the kalman filter

zk = Hxk + vk

f (zk |xk ) = N(Hxk ,R)



Particle filter correction step

The measurement model gives a weight for each particle, i

w(i)k = f

(zk

∣∣∣x−(i)k

)Resample the set of particles using the weights

Each corrected particle, x(i)k , is choosen randomly

x(i)k = x

−(j)k for some random j

The probability of choosing sample j is

w(j)k∑

l w(l)k

The same particle may be choosen several times



Particle filter



Particle filter tracker


Introduction Tracking with static camera Tracking with moving cameraSelf-motion

Recording sport events



Self-motion

Detect keypoints

Track each keypoint separately

Use RANSAC to find camera motion

Compensate for camera motion (stiching)



Self-motion



Master Thesis: Automatisk inspelning av sport

Detta projekt syftar till att utveckla mjukvara som automatiskt kan filma,folja och mata tider for ett idrottsevenemang, till exempel ett 100m-lopp ifriidrott. For detta kravs dels kunskaper i matematisk bildanalys, men endel kunskaper i programmering kommer aven att behovas. Vi far hjalp avIFK Lund att spela in film och utvardera resultatet.

Ett examensarbete syftar till att utveckla algoritmer for automatiskfoljning av lopp. Fran borjan ar kameran stilla och efter starten skallden hela tiden folja loparna genom att rotera och zooma. Algortimenmaste kunna kompensera och den egna kamerarorelsen.Ytterligare ett examensarbete behandlar automatisk detektion avloparbanan. Genom kamerakalibrering och banans kanda matt kanloparnas hastighet beraknas och diagram over hur hastighetenforandrats genom loppet presenteras for publik och tranare. Dennainformation efterfragas av aktiva lopare. Automatisk detektion avmallinje ar aven viktigt for tidtagningen ovan.

Contact: Petter Strandmark <[email protected]>Hakan Ardo Tracking March 4, 2013 57 / 57

Tracking · 2013. 3. 4. · Time update (a priori estimates) Measurement update (a posteriori...

Documents

Transcript of Tracking · 2013. 3. 4. · Time update (a priori estimates) Measurement update (a posteriori...