Handover and TrackingHandover and Trackingin a Camera Networkin a Camera Network

Presented byPresented by Dima GershovichDima Gershovich

Decentralized Discovery Decentralized Discovery of Camera Network of Camera Network

TopologyTopology

Ryan Farrell, Larry DavisRyan Farrell, Larry Davis

Motivation - 1Motivation - 1

►Networks of tens and hundreds of Networks of tens and hundreds of cameras are installed for coverage of cameras are installed for coverage of large areaslarge areas

►For collaboration between cameras an For collaboration between cameras an automatic recovery of network topology automatic recovery of network topology is neededis needed

►A particular interest is in problem where A particular interest is in problem where the cameras have a the cameras have a non-overlapping non-overlapping field of view field of view (thus, finding links (thus, finding links between cameras is not trivial because between cameras is not trivial because of the passage time gaps..)of the passage time gaps..)

Motivation - 2Motivation - 2

►Centralized solution is computationally Centralized solution is computationally expensive and scales poorly for large expensive and scales poorly for large networks.networks.

►Distributed solution provides a better Distributed solution provides a better scalability.scalability.

Notions -1Notions -1

►Network Topology Network Topology Two cameras are considered Two cameras are considered adjacent adjacent if if

there exists a path between the cameras there exists a path between the cameras that does not cross through fields of view that does not cross through fields of view of any other camerasof any other cameras

Graph that defines the network topology:Graph that defines the network topology:►Nodes – camerasNodes – cameras►Edges – between adjacent camerasEdges – between adjacent cameras

Notions - 2Notions - 2

►Transition Model Transition Model A probability distribution that describes:A probability distribution that describes: Where objects go when they leave one Where objects go when they leave one

cameracamera How much time it takes to arrive to the How much time it takes to arrive to the

next cameranext camera►Transition model implicitly defines the Transition model implicitly defines the

network topologynetwork topology►Finding the transition model Finding the transition model

automatically is our goalautomatically is our goal

NotationsNotations

►CCii – camera – camera

►f – an appearancef – an appearance►A(f) – the density of the appearanceA(f) – the density of the appearance►DDAA(f) – distinctiveness of appearance f(f) – distinctiveness of appearance f

More can be learned from distinctive More can be learned from distinctive appearances (toxido on a campus) then appearances (toxido on a campus) then from non-distinctive appearances (t-shirt from non-distinctive appearances (t-shirt & jeans on a campus)& jeans on a campus)

Information-Theoretic Information-Theoretic appearance matchingappearance matching

► Distinctiveness weight for appearance f - Distinctiveness weight for appearance f - AIC Weighting (Akaike’s Information AIC Weighting (Akaike’s Information Criterion)Criterion)

► δδAA – weighting coefficient – weighting coefficient

► Weighted match scoreWeighted match score

Algorithm SketchAlgorithm Sketch

►Modelling PhaseModelling Phase Appearance density A(f) => DAppearance density A(f) => DAA(F)(F)

inter-camera delay densities Tinter-camera delay densities Ti,ji,j

►Estimation PhaseEstimation Phase Computing evidence vectors Computing evidence vectors Inferring Transition Model from evidence Inferring Transition Model from evidence

vectors using Modified Multinomial vectors using Modified Multinomial DistributionDistribution

Modelling PhaseModelling Phase

► Inter-Camera delay densities (function of Inter-Camera delay densities (function of time!)time!)

► τ – the temporal window sizeτ – the temporal window size► Ψ = 1 if 0 < t2 – t1 <= τ, otherwise 0Ψ = 1 if 0 < t2 – t1 <= τ, otherwise 0► K – is a smoothing or weighting kernel (such K – is a smoothing or weighting kernel (such

as a truncated Gaussian, a triangle filter, etc.)as a truncated Gaussian, a triangle filter, etc.)► M(fM(f11,f,f22) – match score defined above) – match score defined above

Estimation Phase - 1Estimation Phase - 1

►Observation o(t,f) weightingObservation o(t,f) weighting►Normalized contribution vector wNormalized contribution vector woo, the j-, the j-

th component expressing an estimate of th component expressing an estimate of the probability that the object observed the probability that the object observed in o appeared next in camera s.in o appeared next in camera s.

Estimation Phase - 2Estimation Phase - 2

► After m observations at camera cAfter m observations at camera cii, the node-, the node-specific evidence vector is given by:specific evidence vector is given by:

► Higher weight is given to more distinctive Higher weight is given to more distinctive appearancesappearances

► Last stage is to infer the transition model Last stage is to infer the transition model from the evidence vectors using Modified from the evidence vectors using Modified Multinomial Distribution..Multinomial Distribution..

Modified Multinomial Modified Multinomial DistributionDistribution

►An oral explanationAn oral explanation

Stochastic Adaptive Stochastic Adaptive Tracking in a Camera Tracking in a Camera

NetworkNetwork

Bi Song, Amit K.Roy-Bi Song, Amit K.Roy-ChowdhuryChowdhury

Problem Formulation - 1Problem Formulation - 1

►Network topology is assumed to be Network topology is assumed to be known (Can be the output of our known (Can be the output of our previous algorithm..)previous algorithm..) Connections between camerasConnections between cameras Entry/Exit pointsEntry/Exit points Distribution of travel time between Distribution of travel time between

cameras.cameras.

►We want to be able to We want to be able to track multiple track multiple people routes between cameraspeople routes between cameras

Problem Formulation - 2Problem Formulation - 2

►nniicc – a network node – a network node

i – node’s indexi – node’s index c – camerac – camera

►For any pair of nodes (nFor any pair of nodes (niicc,n,njj

dd), i != j:), i != j:

lli,ji,j = 0 – if the two nodes are linked, 1 = 0 – if the two nodes are linked, 1 otherwise.otherwise.

►Pτ(nPτ(niicc,n,njj

dd) – distribution of travel time ) – distribution of travel time between the two nodes.between the two nodes.

Camera NetworkCamera Network

Problem Formulation - 3Problem Formulation - 3

►FFn,tn,t – observations at each node are – observations at each node are represented as a feature vectorrepresented as a feature vector

►FFAA – Normalized color (appearance feature) – Normalized color (appearance feature)

►FFII – Gait recognition (identity feature) – Gait recognition (identity feature)

►ΤΤni,njni,njt1,t2t1,t2 – travel time between F – travel time between Fni,t1ni,t1 and F and Fnj,t2nj,t2

►FFA, A, FFII, T – independent random variables, T – independent random variables

Framework for Stochastic Framework for Stochastic Adaptive TrackingAdaptive Tracking

Feature Graph Construction - Feature Graph Construction - 11

►The features are vertices on the graph The features are vertices on the graph (see next slide..)(see next slide..)

►s(es(eijij) is the similarity score between F) is the similarity score between Fii and and FFjj

Similarities computation takes into Similarities computation takes into account known geometric and account known geometric and photometric transformations.photometric transformations.

Feature Graph Construction - Feature Graph Construction - 22

► The similarity score sThe similarity score sijij is a is a realization of a random variable srealization of a random variable s

► The distribution of s on eThe distribution of s on eijij is is modeled as a normal distribution modeled as a normal distribution with swith sijij’ as a mean:’ as a mean:N(S’N(S’ijij, σ, σijij

22))

► The variance is learned from theThe variance is learned from thetraining data in an unsupervised training data in an unsupervised manner.manner.

Feature Graph ConstructionFeature Graph Construction

Framework for Stochastic Framework for Stochastic Adaptive TrackingAdaptive Tracking

Optimal Path in Stochastic Optimal Path in Stochastic Feature Graphs - 1Feature Graphs - 1

►Optimal path problem in graphs with Optimal path problem in graphs with weights as normal variablesweights as normal variables

►Von-Neumann and Morgenstern – Von-Neumann and Morgenstern – define a utility function: it has to be define a utility function: it has to be monotonic, affine linear or monotonic, affine linear or exponential.exponential.

►We define a weighting function for We define a weighting function for similarity:similarity:

Optimal Path in Stochastic Optimal Path in Stochastic Feature Graphs - 2Feature Graphs - 2

►We identify the most preferred set of We identify the most preferred set of paths by maximizing the utility function: paths by maximizing the utility function:

►This problem can be formulated as the This problem can be formulated as the maximum matching problem maximum matching problem in ain a weighted bipartite graphweighted bipartite graph: splitting : splitting each vertex into veach vertex into vinin and v and voutout and the and the weights are the utility function scoresweights are the utility function scores

Framework for Stochastic Framework for Stochastic Adaptive TrackingAdaptive Tracking

Path Smoothness FunctionPath Smoothness Function

►PSF is defined PSF is defined on each edge eon each edge eijij along its path along its path λλqq

►The feature The feature vectors before vectors before eeijij and after e and after eijij are treated as are treated as two clusters: two clusters: {X{X(0)(0)} and {X} and {X(1)(1)}}

Closed-Loop Adaptation of Edge Closed-Loop Adaptation of Edge Similarities - 1Similarities - 1

►Whenever there is a peak in PSF Whenever there is a peak in PSF function for some edge along a path, function for some edge along a path, the validity of connections between the validity of connections between features along that path is under doubtfeatures along that path is under doubt

►We adjust weight on the link where We adjust weight on the link where peak occurredpeak occurred by reducing mean, by reducing mean, adjusting variance based on the adjusting variance based on the learned values and recalculate the learned values and recalculate the optimal paths using the new weightsoptimal paths using the new weights

Closed-Loop Adaptation of Edge Closed-Loop Adaptation of Edge Similarities - 2Similarities - 2

►Adaptation steps:Adaptation steps:

Closed-Loop Adaptation of Edge Closed-Loop Adaptation of Edge Similarities - 3Similarities - 3

►The process of weight adaptation and The process of weight adaptation and optimal path computation continues in optimal path computation continues in a closed loop until a local minimum ofa closed loop until a local minimum of

is reachedis reached►This process is repeated for each This process is repeated for each

possible path λpossible path λqq

Framework for Stochastic Framework for Stochastic Adaptive TrackingAdaptive Tracking

Stochastic Adaptive Tracking Stochastic Adaptive Tracking Algorithm - 1Algorithm - 1

►1. Construct a stochastic weighted 1. Construct a stochastic weighted graph where the vertices are the graph where the vertices are the feature vectors and distribution of edge feature vectors and distribution of edge weights are set as described above.weights are set as described above.

►2. Compute the optimal paths λ2. Compute the optimal paths λqq

►3. Compute PSF for each edge of λ3. Compute PSF for each edge of λq q and and adapt the distribution of edge weights.adapt the distribution of edge weights.

►4. Repeat steps 2 and 3 until a local 4. Repeat steps 2 and 3 until a local minimum of minimum of

is reached.is reached.The final set of optimal paths is given The final set of optimal paths is given as:as:

Stochastic Adaptive Tracking Stochastic Adaptive Tracking Algorithm - 2Algorithm - 2

QuestionsQuestions??

Thanks for your attentionThanks for your attention!!