Information Weighted Consensus for Distributed ... - Tashrif
Transcript of Information Weighted Consensus for Distributed ... - Tashrif
eScholarship provides open access, scholarly publishingservices to the University of California and delivers a dynamicresearch platform to scholars worldwide.
Electronic Theses and DissertationsUC Riverside
Peer Reviewed
Title:Information Weighted Consensus for Distributed Estimation in Vision Networks
Author:Kamal, Ahmed Tashrif
Acceptance Date:2013
Series:UC Riverside Electronic Theses and Dissertations
Degree:Ph.D., Electrical EngineeringUC Riverside
Advisor(s):Roy-Chowdhury, Amit K
Committee:Farrell, Jay A, Tuncel, Ertem
Permalink:http://escholarship.org/uc/item/0rz9v80g
Abstract:
Copyright Information:All rights reserved unless otherwise indicated. Contact the author or original publisher for anynecessary permissions. eScholarship is not the copyright owner for deposited works. Learn moreat http://www.escholarship.org/help_copyright.html#reuse
UNIVERSITY OF CALIFORNIARIVERSIDE
Information Weighted Consensus for Distributed Estimation in Vision Networks
A Dissertation submitted in partial satisfactionof the requirements for the degree of
Doctor of Philosophy
in
Electrical Engineering
by
Ahmed Tashrif Kamal
August 2013
Dissertation Committee:
Dr. Amit K. Roy-Chowdhury, ChairpersonDr. Jay A. FarrellDr. Ertem Tuncel
Copyright byAhmed Tashrif Kamal
2013
The Dissertation of Ahmed Tashrif Kamal is approved:
Committee Chairperson
University of California, Riverside
Acknowledgments
In the past few years of my academic endeavor, I had an amazing journey through the
vast ocean of knowledge with some of the most amazing people of my time. I would like
to start by expressing my deepest gratitude to my advisor Dr. Amit Roy-Chowdhury
and co-advisor Dr. Jay Farrell. I was introduced to my research topic by Dr. Roy-
Chowdhury. He was there for me in every ups and downs of my academic endeavor
throughout my graduate life. I was able to achieve my academic goals due to all the
support and mentoring I received from both my advisors.
I am also grateful to my committee member Dr. Ertem Tuncel. He helped me
out with different conceptual issues that arose throughout different graduate courses and
my PhD research. I learnt most of the background materials I needed for my research
from the courses taught by committee members Dr. Roy-Chowdhury, Dr. Farrell and
Dr. Tuncel who are also amazing teachers.
I would also like to express my gratitude to my colleagues Dr. Bi Song and
Chong Ding. I did learn a lot by collaborating with them at the foundation stage of
my PhD research. Due to all the brain-storming with them and my supervisors, I was
able to grasp a good understanding of the challenges in the problem and was able to
successfully formulate it. I would also like to thank my colleague Anirban Chakraborty
for his help in various experiments and brainstorming sessions.
I will be ever grateful to all my teachers throughout my earlier academic life.
Especially, I would like to thank Mr. Mahmud for inspiring me in science and Mr. Nimai
Das for inspiring me in mathematics. I would also like to thank Mr. Delowar Hossain
who was an amazing teacher to me.
Words cannot express the level of my gratitude to my mother Tahmina Begum
iv
and my father Dr. Kamaluddin Ahmed for all their support, guidance, teaching and
unconditional love. I am also ever grateful to my wife Mehnaz Mahbub for all her love
and support that made the tough barriers easy to pass throughout my graduate years.
I would also like to thank my sister Kashfia Mehrin for her love and support.
I would also like to thank the Office of Naval Research for their grant (N000140910666)
supporting my research.
v
To my parents and my wife for all the support.
vi
ABSTRACT OF THE DISSERTATION
Information Weighted Consensus for Distributed Estimation in Vision Networks
by
Ahmed Tashrif Kamal
Doctor of Philosophy, Graduate Program in Electrical EngineeringUniversity of California, Riverside, August 2013
Dr. Amit K. Roy-Chowdhury, Chairperson
Due to their high fault-tolerance, ease of installation and scalability to large networks,
distributed algorithms have recently gained immense popularity in the sensor networks
community, especially in computer vision. Multi-target tracking in a camera network
is one of the fundamental problems in this domain. Distributed estimation algorithms
work by exchanging information between sensors that are communication neighbors.
Since most cameras are directional sensors, it is often the case that neighboring sensors
may not be sensing the same target. Such sensors that do not have information about
a target are termed as “naive” with respect to that target. State-of-the-art distributed
state estimation algorithms (e.g., the Kalman Consensus Filter (KCF)) in the sensor
networks community are not directly applicable to tracking applications in camera net-
works mainly due to this naivety issue. In our work, we propose generalized distributed
algorithms for state estimation in a sensor network taking the naivety issue into account.
For multi-target tracking, along with the tracking framework, a data associa-
tion step is necessary where the measurements in each camera’s view are associated with
the appropriate targets’ tracks. At first, under the assumption that the data association
is given, we develop distributed state estimation algorithms addressing the naivety is-
vii
sue. In this process, first, we propose the Generalized Kalman Consensus Filter (GKCF)
where an information-weighting scheme is utilized to account for the naivety issue. Next,
we propose the Information-weighted Consensus Filter (ICF) which can achieve optimal
centralized performance while also accounting for naivety. This is the core contribution
of this thesis. Next, we introduce the aspect of multi-target tracking where a probabilis-
tic data association scheme is incorporated in the distributed tracking scheme resulting
the Multi-Target Information Consensus (MTIC) algorithm. The incorporation of the
probabilistic data association mechanism makes the MTIC algorithm very robust to
false measurements/clutter.
The aforementioned algorithms are derived under the assumption that the
measurements are related to the state variables using a linear relationship. However, in
general, this is not true for many sensors including cameras. Thus, to account for the
non-linearity in the observation model, we propose non-linear extensions of the previous
algorithms which we denote as the Extended ICF (EICF) and the Extended MTIC
(EMTIC) algorithms. In-depth theoretical and experimental analysis are provided to
compare these algorithms with existing ones.
viii
Contents
List of Figures xi
1 Introduction 11.1 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.3 Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2 Kalman Consensus Filter and its Extension 92.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.3 Average Consensus: Review . . . . . . . . . . . . . . . . . . . . . . . . . 132.4 Kalman Consensus Filter: Review . . . . . . . . . . . . . . . . . . . . . 142.5 Generalized Kalman Consensus Filter . . . . . . . . . . . . . . . . . . . 18
2.5.1 Weighted Average Consensus . . . . . . . . . . . . . . . . . . . . 182.5.2 Covariance/Information Matrix Propagation . . . . . . . . . . . 192.5.3 Two-stage Update . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.6 Theoretical Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.7 Experimental Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.7.1 Experiment 1: Varying K . . . . . . . . . . . . . . . . . . . . . . 262.7.2 Experiment 2: Varying ∆ . . . . . . . . . . . . . . . . . . . . . . 272.7.3 Experiment 3: Varying SR . . . . . . . . . . . . . . . . . . . . . 282.7.4 Experiment 4: Varying NC . . . . . . . . . . . . . . . . . . . . . 28
2.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3 Information-weighted Consensus Filter 313.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.1.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.1.2 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363.3 Centralized MAP Estimation . . . . . . . . . . . . . . . . . . . . . . . . 383.4 Information Consensus based Distributed MAP Estimation (IC-MAP) . 39
3.4.1 Distributed Implementation . . . . . . . . . . . . . . . . . . . . . 423.4.2 Computation of F−i . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.4.2.1 Case 1: Converged Priors . . . . . . . . . . . . . . . . . 443.4.2.2 Case 2: Uncorrelated Prior Errors . . . . . . . . . . . . 45
3.5 Information-weighted Consensus Filter . . . . . . . . . . . . . . . . . . . 46
ix
3.5.1 Initialization and Special Situations . . . . . . . . . . . . . . . . 483.5.2 ICF, GKCF and KCF Comparison . . . . . . . . . . . . . . . . 50
3.6 Experimental Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 523.6.1 Experiment 1: Varying K . . . . . . . . . . . . . . . . . . . . . . 563.6.2 Experiment 2: Varying µ and τ . . . . . . . . . . . . . . . . . . . 573.6.3 Experiment 3: Varying ∆ . . . . . . . . . . . . . . . . . . . . . . 593.6.4 Experiment 4: Varying SR . . . . . . . . . . . . . . . . . . . . . 593.6.5 Experiment 5: Varying NC . . . . . . . . . . . . . . . . . . . . . 603.6.6 Experiment 6: Arbitrary Communication Graph . . . . . . . . . 613.6.7 Experiment 7: Full Observability . . . . . . . . . . . . . . . . . . 623.6.8 Experiment 8: Stability Analysis . . . . . . . . . . . . . . . . . . 623.6.9 Experiment 9: Robustness to inaccurate knowledge of NC . . . . 633.6.10 Experiment 10: Robustness to non-linear state propagation . . . 63
3.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4 Multi-Target Information Consensus 664.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.1.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 684.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 684.3 Joint Probabilistic Data Association Filter: Review . . . . . . . . . . . . 704.4 Data Association: Information Form . . . . . . . . . . . . . . . . . . . . 714.5 Multi-Target Information Consensus . . . . . . . . . . . . . . . . . . . . 724.6 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 744.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5 Information-weighted Consensus with non-linear models 805.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 805.2 Camera Observation Model (Non-linear ) . . . . . . . . . . . . . . . . . 815.3 Extended Information-weighted Consensus Filter . . . . . . . . . . . . . 825.4 Extended Multi-target Information Consensus . . . . . . . . . . . . . . . 855.5 Comparison of KCF, ICF, MTIC, EKCF, EICF and EMTIC . . . . . . 885.6 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
5.6.1 Simulation Experiments . . . . . . . . . . . . . . . . . . . . . . . 915.6.2 Real-life Experiments . . . . . . . . . . . . . . . . . . . . . . . . 94
5.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
6 Conclusions and Future Work 996.1 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
6.1.1 Mobile Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . 1026.1.2 Hardware Implementation . . . . . . . . . . . . . . . . . . . . . . 102
Bibliography 103
A EKF: Information Form 106
B JPDAF: Information Form 108
C EMTIC: Derivation 110
D Single step consensus comparison: 112
x
List of Figures
1.1 Camera network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.1 Camera network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.2 GKCF simulation framework . . . . . . . . . . . . . . . . . . . . . . . . 262.3 GKCF performance comparison varying K (equal priors) . . . . . . . . 272.4 GKCF performance comparison varying K (unequal priors) . . . . . . . 272.5 GKCF performance comparison varying network connectivity . . . . . . 282.6 GKCF performance comparison varying sensing range . . . . . . . . . . 282.7 GKCF performance comparison varying NC . . . . . . . . . . . . . . . . 29
3.1 Camera network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.2 ICF convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503.3 ICF simulation framework . . . . . . . . . . . . . . . . . . . . . . . . . . 553.4 ICF performance comparison varying K (equal priors) . . . . . . . . . . 563.5 ICF performance comparison varying K (uncorrelated priors) . . . . . . 573.6 ICF performance comparison varying K (correlated priors) . . . . . . . 573.7 ICF performance comparison varying communication bandwidth . . . . 583.8 ICF performance comparison varying computation resource . . . . . . . 583.9 ICF performance comparison varying communication bandwidth . . . . 583.10 ICF performance comparison varying computation resource . . . . . . . 583.11 ICF performance comparison varying network connectivity . . . . . . . 603.12 ICF performance comparison varying sensor range . . . . . . . . . . . . 603.13 ICF performance comparison varying NC . . . . . . . . . . . . . . . . . 613.14 ICF performance comparison varying K (imbalanced communication graph) 613.15 ICF performance comparison under unlimited observability . . . . . . . 623.16 ICF stability analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 623.17 ICF robustness to inaccurate knowledge on NC . . . . . . . . . . . . . . 633.18 ICF robustness to model assumption error . . . . . . . . . . . . . . . . . 63
4.1 MTIC performance comparison varying amount of clutter . . . . . . . . 774.2 MTIC Performance comparison varying different parameters. . . . . . . 78
5.1 EICF and EMTIC Simulation setup . . . . . . . . . . . . . . . . . . . . 915.2 EICF performance comparison varying K . . . . . . . . . . . . . . . . . 935.3 EMTIC performance comparison varying K . . . . . . . . . . . . . . . . 945.4 Camera Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 955.5 EMTIC real-life experiments . . . . . . . . . . . . . . . . . . . . . . . . 97
xi
5.6 EMTIC, EICF and EKF tracking results in the ground plane. . . . . . . 985.7 Comparing mean error from EKF . . . . . . . . . . . . . . . . . . . . . . 98
xii
List of Algorithms
1 Kalman Consensus Filter (Review) . . . . . . . . . . . . . . . . . . . . . 15
2 Generalized Kalman Consensus Filter . . . . . . . . . . . . . . . . . . . 20
3 Information-weighted Consensus Filter . . . . . . . . . . . . . . . . . . . 47
4 Multi-Target Information Consensus . . . . . . . . . . . . . . . . . . . . 73
5 Extended Information-weighted Consensus Filter . . . . . . . . . . . . . 84
6 Extended Multi-Target Information Consensus . . . . . . . . . . . . . . 87
xiii
Important Notations
Notation Dimension Description
i, i′ scalar sensor indexj scalar target indexNC scalar total number of sensorsNT scalar total number of targetsNi set set of neighbors of node it scalar time stepk scalar consensus iteration no.p scalar state vector lengthmi scalar Ci’s measurement vector length
m scalar∑NC
i=1mi
x p target’s true statexc, xi p centralized and Ci’s estimateJc, Ji p× p information matrix of xc & xi
X−
NCp stack of estimates from all nodes
P NCp×NCp error covariance of X−
F NCp×NCp information matrix of X−
Fii′ p× p {i, i′}th block element of FF−i p× p
∑NCi′=1 Fi′i
HI NCp× p [Ip, Ip, . . . Ip]T
( )−, ( )+ prior and posterior informationΦ p× p state transition matrixQ p× p process noise covariance
zi mi Ci’s measurementRi mi ×mi covariance matrix of ziHi mi × p observation matrix at Cihi function observation function of CiZ m stack of all measurementsR m×m covariance matrix of ZH m× p stack of all Hi
ε scalar consensus speed parameter∆i scalar degree of node connectivity
vi[k] p consensus weighted informationVi[k],Wi[k] p× p consensus information matrix
βjn scalar probability that zn is T j ’s obs.βj0 scalar probability that T j was not observedλf scalar clutter density
xiv
Chapter 1
Introduction
Due to the availability of modern low-cost sensors, large-scale camera networks
are being used in applications such as wide-area surveillance, disaster response, envi-
ronmental monitoring, etc. Multiple sensors can cover more area, provide views from
different angles and the fusion of all their measurements may lead to robust scene under-
standing. Among different information fusion approaches, distributed schemes are often
chosen over centralized or hierarchical approaches due to their scalability to a large num-
ber of sensors, ease of installation and high tolerance to node failure. In this article, we
focus on the problem of distributed multi-target tracking in a camera network. However,
the proposed methods are much generalized distributed state estimation schemes and
their applications are not limited only to camera networks. We use the term distributed
to mean that each camera processes its own data and arrives at a final solution through
negotiations with its neighbors; there is no central processor. Note that term distributed
has been also used in computer vision to refer to a camera network that is distributed
over a wide area but where the processing is centralized. To motivate the core contribu-
tion of this work, we first describe the inter-relationship between distributed estimation
1
C1
C5
C3C2 C4
C6
Figure 1.1: In this figure, there are six sensing nodes, C1, C2, . . . , C6 observing an area (black
rectangle) consisting of four targets. The solid blue lines show the communication channels
between different nodes. This figure also depicts the presence of “naive” nodes. For example,
C3, C5, C6 get direct measurements about the black target which it shares with its immediate
network neighbors. However, C1 does not have direct access to measurements of that target and
thus is naive w.r.t. that target’s state.
and camera networks.
Most of the work in distributed tracking has been in the multi-agent systems
community [18]. The methods there assume that each target can be viewed by each
sensor which may not be true for many application scenarios, especially for a camera
network (see Fig. 1.1) where each camera can view only a limited portion of the entire
area. This limits the observability of each sensor to a subset of all the targets. In this
work, our goal is to design distributed tracking schemes which are suited for such sensors
2
with limited field-of-view (FOV).
A distributed multi-target tracking problem can be divided into three sub-
problems, namely, distributed information fusion, data association (measurement to
track association) and dynamic state estimation. Among many types of distributed in-
formation fusion approaches, consensus algorithms [18] are schemes where each node,
corrects its own state using information only from its network neighbors. By iteratively
doing so, each node can individually compute a global function of the information from
all the nodes (e.g. average). The important fact is that consensus is reached without
all-to-all communication; thus consensus based frameworks do not require any specific
communication network topology and are generally applicable to any arbitrary, con-
nected network. The consensus estimates asymptotically converge to the global result.
However, in a finite time window, only a limited number of iterations can be performed
due to limited bandwidth. Due to the simplicity and robustness of consensus algo-
rithms, they have been used in many applications, including estimation problems in
sensor networks (e.g., [28, 30, 31]).
In a distributed multi-target tracking scheme, each node may need to maintain
a state estimate of each target even though it is not directly observing the target, since
the nodes will need to collaborate with each other. Each node gets measurements of
the targets and must associate the measurements to the appropriate target’s track. In
a consensus-based scheme, each node maintains its own copy of the state estimates of
all the targets which makes consensus-based approaches inherently appropriate for our
problem.
When applying these approaches to camera networks, we need to be aware
of one particular issue with vision sensors. As exemplified in Fig. 1.1, many of the
cameras may not see a target and it is very much possible that neighboring cameras on
3
the communication graph do not see the same target. We call a node ‘naive’ about a
target when there are no measurements of that target available in its local neighborhood
(consisting of the node and its immediate network neighbors). In such a situation, in
a consensus-based framework, due to limited local observability and limited number of
consensus iterations, the naive node has access to less information about the target’s
state. Naivety relative to the j-th target is likely to be present in the network when
nodes on the vision graph for target j are sparsely connected in the communication
graph. The vision graph for target T j is a graph where there is an edge between each
pair of nodes that are observing T j . The vision graph is usually time varying, different
from the communication graph, and different for different targets.
A well-known consensus-based scheme for distributed state estimation is the
Kalman Consensus Filter (KCF) [17]. The KCF algorithm was originally designed for
the scenario where each node has an observation of the target. The quality of neighbor-
ing node’s prior information was not taken into account in KCF. Thus, naive nodes may
adversely affect the overall performance of the network. Moreover, the cross-covariance
terms between the state estimates at different nodes were not incorporated in the estima-
tion process in KCF as they are usually hard to compute in a distributed environment.
Due to these reasons, the performance of KCF often suffers when applied to a camera
network. Distributed algorithms that are derived under the assumption that each sensor
has full state observability at each time (e.g. KCF), may be adapted to be used in the
presence of naive nodes when communication protocols (e.g. [9]) for translating a known
vision graph into a set of routing paths that connect nodes in the communication graph
are available. However, scalable distributed vision graph discovery algorithms do not
exist. Moreover, distributed algorithms that require the knowledge of the vision graphs
usually require special target handoff protocols. Such issues are important for networks
4
with sparse communication topology. For example, camera networks are often spread
over a wide area which prevents each camera from communicating directly with all other
cameras. There can be many naive nodes in sparse communication topologies.
1.1 Contribution
The presence of these naive nodes motivates us to propose various novel dis-
tributed state estimation frameworks which can be utilized in many applications in
sensor networks including target tracking in camera networks.
First, we propose the Generalized Kalman Consensus Filter (GKCF) [11] where
each node’s confidence in its state estimate is also incorporated in the distributed es-
timation framework. In the presence of naive nodes, the proposed GKCF algorithm
outperforms the standard KCF.
The results of the KCF and the GKCF algorithms are sub-optimal i.e., they
do not converge to the optimal centralized Kalman filter [10] results. Additionally, com-
putation and communication resource constraints are important issues in a distributed
estimation framework because in many application scenarios, the nodes are low powered
wireless devices. Therefore, a distributed state estimation framework that can guaran-
tee convergence to the optimal centralized estimate while maintaining low computation
and communication resource requirements in the presence of naive nodes is desirable.
To achieve this, next we propose the distributed state estimation framework called the
Information-weighted Consensus Filter (ICF) [12, 13]. The ICF is guaranteed to con-
verge to the optimal centralized performance under certain reasonable conditions.
The issue of naivety is handled in the ICF algorithm by proper information
weighting in the estimation framework. Optimality is achieved by proper relative weight-
5
ing between the prior and the measurement information. Both the GKCF and ICF
algorithms support multiple consensus iterations at each time step t to improve per-
formance. In addition, the experimental results show that ICF outperforms other dis-
tributed estimation algorithms such as KCF and GKCF at any given computation and
communication resource limit. The ICF algorithm (and its derivatives that we propose
later) does not require handoff protocols or the knowledge of the vision graph.
The above mentioned methods assume that there is a single target, or for
multiple targets, the measurement-to-track association is provided. For a multi-target
tracking problem, the data association and the tracking steps are highly inter-dependent.
The performance of tracking will affect the performance of data association and vice-
versa. Thus, an integrated distributed tracking and data association solution is required
where the uncertainty from the tracker can be incorporated in the data association
process and vice-versa. Among many single-sensor multi-target data association frame-
works, the Multiple Hypothesis Tracking (MHT) [23] and the Joint Probabilistic Data
Association Filter JPDAF [2] are two popular schemes. MHT usually achieves higher
accuracy at the cost of high computational load. On the other hand, JPDAF achieves
reasonable results at much lower computation cost. As distributed solutions are usu-
ally applied to resource constrained systems, the JPDAF scheme will be utilized in the
distributed multi-target tracking framework we propose in our work.
With the tight integration between the JPDAF algorithm and our previously
derived ICF algorithm, we next propose the Multi-Target Information Consensus (MTIC)
[14] algorithm. The estimation errors in tracking and data association, as well as the
effect of naivety, are jointly addressed in the development of the MTIC algorithm. The
incorporation of the probabilistic data association mechanism makes the MTIC algo-
rithm very robust to false measurements/clutter.
6
The aforementioned methods are derived assuming a linear observation model.
However, camera observation models are non-linear. Thus to apply these distributed
tracking algorithms in a realistic camera network, we also need to extend these algo-
rithms to handle non-linearity in the observation model. To accomplish this, we present
the Extended Information Consensus Filter (EICF) and the Extended Multi-target In-
formation Consensus (EMTIC) algorithms.
1.2 Related Work
The purely decentralized nature of the fusion algorithm differentiates it from
the majority of multi-camera tracking approaches in the computer vision literature.
For example, in [6], a centralized approach for tracking in a multi-camera setup was
proposed where the cameras were distributed spatially over a large area. In [4], an
efficiently target hand-off scheme was proposed but no multi-camera information fusion
was involved. However, in this thesis, we deal with the distributed multi-target tracking
problem where there is no centralized server, the processing is distributed over all the
camera nodes and no target hand-off strategy is required.
Various methods for distributed multi-target tracking have been proposed in
the sensor-networks literature. In [5], a solution to the distributed data association
problem was proposed by means of the message passing algorithm based on graphical
models in which iterative, parallel exchange of information among the nodes viewing
the same target was required. However, in our proposed framework, no special commu-
nication pattern is assumed. In [22], the authors proposed a framework for querying a
distributed database of video surveillance data in order to retrieve a set of likely paths
of a person moving in the area under surveillance using a dynamic Bayesian Model.
7
However, unlike our proposed methods, this method does not deal with the distributed
fusion of the information in the network.
In [21, 26, 28], the distributed multi-target tracking schemes did not account for
naivety or the presence of cross-correlation between the estimates at different nodes. We
propose the ICF to deal with both these issues. The MTIC algorithm, is an extension of
the ICF to deal with data association between multiple targets. The EICF and EMTIC
provide the ability to work with non-linear camera models. Along with proposing these
novel distributed algorithms, we also provide in-depth proof and analysis of all these
algorithms.
1.3 Organization
We organize this article as the following. In Chapter 2, we review the average
consensus and the KCF algorithm and propose the GKCF algorithm. In Chapter 3, we
propose the ICF algorithm. In Chapter 4, we propose the MTIC algorithm. Finally, in
Chapter 5, we propose the non-linear extensions to the previous algorithms resulting the
EICF and the EMTIC algorithms. Thorough theoretical and experimental comparisons
are provided in each chapter for each method.
8
Chapter 2
Kalman Consensus Filter and its
Extension
2.1 Introduction
In this chapter, we propose a consensus-based framework that is capable of
tracking targets throughout the network in a distributed manner. The problem of track-
ing targets in a distributed sensor network has been studied previously. The Kalman
Consensus Filter (KCF) [17] is a state-of-the-art distributed algorithm for fusing mul-
tiple measurements from different sensors. The KCF is a very appropriate framework
for camera networks and has been applied in [27, 28]. However, certain issues that are
specific to video sensors have not been considered in the existing solutions.
A camera is a unidirectional sensor with a limited sensing region which is called
the field-of-view (FOV). Thus, in a realistic camera network, a target would usually be
seen in only a few of the nodes Fig. 2.1. In a distributed decision making process, the
nodes are assumed to have peer-to-peer communication channels. Thus, when a sensor
gets new measurements for a target, say T j , it shares this measurement information with
9
C1 C5
C3
C2C4
T1
T2
T3
T4
T5
T6
T7
Figure 2.1: In this figure, there are five sensing nodes, C1, C2, . . . , C5 observing an area (black
rectangle) consisting of seven targets T 1, T 2, . . . T 7. The solid blue lines show the communication
channels between different nodes. This figure also depicts the presence of “naive” nodes. For
example, C1 gets direct measurements about T 1 which it shares with its immediate network
neighbor, C2. However, the rest of the cameras, i.e., C3, C4, C5 do not have direct access to
measurements of T 1 and thus are naive w.r.t. T 1’s state.
its network neighbors. This measurement information is used to update the estimate
of T j ’s state and error covariance at each node that directly observes T j or receives
measurement of T j from their neighbor(s). At the same time, the nodes also share their
previous state estimate with each other and try to compensate the difference between
their state estimates of T j using a consensus scheme. Thus, at some nodes in the network
that are neither sensing T j directly nor are neighbor to a node sensing T j (termed as
naive nodes for T j), the state estimate for this target is only adjusted by the consensus
scheme and its error covariance is not adjusted; therefore, the error covariance matrices
of each target may diverge. Even if the consensus term maintains consistency of the
state estimates, the different covariance matrices at each node can have a profound
effect on the convergence transients, as each agent uses its local covariance matrix in
the computation for incorporating measurement information.
10
Such issues are important for networks with sparse communication topology.
For example, camera networks are often spread over a wide area which prevents each
camera from communicating directly with all other cameras. There can be many naive
nodes in sparse communication topologies. The presence of these naive nodes motivates
us to propose certain modifications to the KCF framework for application in camera net-
works. In such scenarios, the proposed Generalized Kalman Consensus Filter (GKCF)
outperforms the standard KCF. Although this article is focused on camera networks
(since it is a common scenario where these constraints apply), the GKCF approach is
applicable to other sensor networks that have similar characteristics. If the network is
fully (or close to fully) connected, then the effect of naive nodes is usually very low and
the standard KCF performs well [27, 28].
To allow for a clear discussion of the literature and our motivation, Sec. 2.2
states the problem of interest and related notation. Secs. 2.3 and 2.4 reviews the
related literature, presents a technical description of our motivation and overviews the
contributions of the chapter. In Sec. 2.5, we derive our Generalized Kalman Consensus
Filter algorithm. Finally, Sec. 2.7, shows simulation results comparing the performance
of our approach with other methods.
2.2 Problem Formulation
Consider a sensor network with NC sensors. There are no specific assumptions
on the overlap between the FOVs of the sensors. The communication in the network
can be represented using an undirected connected graph G = (C, E). The set C =
{C1, C2, . . . , CNC} contains the vertices of the graph and represents the sensor nodes.
The set E contains the edges of the graph which represents the available communication
11
channels between different nodes. The set of nodes having direct communication channel
with node Ci (sharing an edge with Ci) is represented by Ni.
The target’s state is represented by the vector x ∈ Rp. For example, for a
tracking application in a camera network, x might be a vector containing ground plane
position and velocity components. The state dynamics of the target are modeled as
x(t+ 1) = Φx(t) + γ(t). (2.1)
Here Φ ∈ Rp×p is the state transition matrix and the process noise γ(t) is modeled as
N (0,Q).
At time t, depending on its FOV and the location of the target, a sensor may
get a measurement. Note that in this chapter, our assumption of the knowledge of data
association implies that if we have a measurement, it belongs to the target. Let us
denote the measurement of the target at Ci as zi. It is assumed that zi was generated
by the following observation model
zi = Hixi + νi. (2.2)
Here, Hi ∈ Rm×p is the observation matrix for node Ci. The noise νi ∈ Rm is modeled
as a zero mean Gaussian random variable with covariance Ri ∈ Rm×m.
Each node also maintains a prior/predicted state estimate x−i (t) (and its covari-
ance P−i (t)) for each target. Throughout this article, the inverse of the state covariance
matrix (information/precision matrix) will be used and denoted as Ji = (Pi)−1. We
assume that the initial prior state estimate and information matrix is available to each
node for the target upon its detection. Our goal is to track the target at each node, i.e.,
find the state estimate for the target at each node by using the prior and measurement
information available in the entire network in a distributed fashion.
12
2.3 Average Consensus: Review
Average consensus [18] is a popular distributed algorithm to compute the arith-
metic mean of some values {ai}NCi=1. Suppose, each node i has a quantity ai. We are
interested in computing the average value of these quantities i.e. 1NC
∑NCi=1 ai, in a dis-
tributed manner.
In average consensus algorithm, each node initializes its consensus state as
ai[0] = ai and iteratively communicates with its neighbors and updates its own state
information. Throughout this article, the index inside a square bracket will indicate
the consensus iteration number as opposed to the index inside a parenthesis that will
typically refer to the time index. At the beginning of iteration k, a node Ci sends its
previous state ai[k − 1] to its immediate network neighbors Ci′ ∈ Ni and also receives
the neighbors’ previous states ai′ [k−1]. Then it updates its own state information using
the following equation
ai[k] = ai[k − 1] + ε∑i′∈Ni
(ai′ [k − 1]− ai[k − 1])
= A(ai[k − 1]) (2.3)
Here A(ai) is a shorthand mathematical operator for a single step of average consensus
(defined as the above). By iteratively doing so, the values of the states at all the
nodes converge to the average of the initial values. The rate parameter ε should be
chosen between 0 and 1∆max
, where ∆max is the maximum degree of the network graph
G. Choosing larger values of ε will result in faster convergence, but choosing values
equal or more than ∆max will render the algorithm unstable. The average consensus
algorithm can be used to compute the average of vectors and matrices by applying it
to their individual elements separately. Average consensus assumes all agents have an
estimate for all elements of a.
13
Consensus algorithms have been extended to perform various tasks in a net-
work of agents such as linear algebraic operations like SVD, least squares, PCA, GPCA
[31]. These distributed estimation frameworks have been applied in various fields includ-
ing camera networks for distributed implementations of 3-D point triangulation, pose
estimation [30], and action recognition [28]. The average consensus algorithm is appli-
cable only for a static parameter estimation problem. For a dynamic state estimation
problem, a predictor-corrector solution approach is needed.
2.4 Kalman Consensus Filter: Review
Now, we present the KCF algorithm [17] in Algorithm 1 and explain the mo-
tivation for proposing the GKCF. It should be noted that the KCF algorithm works
under the assumption that all the sensors have sensed all the targets. The issue of lim-
ited sensing range in the distributed estimation process has been considered previously.
In [20], the authors considered the case where not all sensors get measurements of the
target. However, the solution was not fully distributed; rather it was a hybrid solution
consisting of a distributed and a centralized scheme for information fusion. The nodes
used the KCF algorithm to update their state estimates. These state estimates were sent
along with the state covariance information to a fusion center. For larger networks a hi-
erarchical tree structure of fusion centers was proposed, where the information of all the
nodes reached a root fusion center. Throughout this article, we are interested in solving
the problem using a completely distributed architecture. We will now discuss various
specific conditions that require attention when the KCF is applied to sparse networks
with naive nodes, and next we will propose solution strategies for each of them.
14
Algorithm 1 KCF at Ci at time step t
Given J−i (t), x−i (t) and ε
1) Get measurement zi.
2) Compute information vector and matrix
ui ← HTi R−1
i zi (2.4)
Ui ← HTi R−1
i Hi (2.5)
3) Broadcast ui, Ui, x−i to neighbors and receive ui′ , Ui′ , x−i′ from neighbors.
4) Fuse the information vectors and matrices
bi ←∑
i′∈Ni∪{i}
ui′ (2.6)
Bi ←∑
i′∈Ni∪{i}
Ui′ (2.7)
5) Compute Kalman Consensus estimate
Mi ← (J−i + Bi)−1
(2.8)
x+i ← x−i + Mi
(bi −Bix
−i
)+ γ(J−i )−1
∑i′∈Ni
(x−i′ − x−i
)(2.9)
γ = ε/(1 + ||(J−i )−1||), ||X|| = tr(XTX)12 (2.10)
6) Predict for next time step
J−i (t+ 1) ← (ΦMiΦT + Q)
−1(2.11)
x−i (t+ 1) ← Φx+i (t) (2.12)
15
1) Average vs. weighted average: The basic KCF algorithm uses average consensus
to combine state estimates from neighboring nodes (see Eqn. (2.9)). With average
consensus, the state estimates of all the nodes get the same weight in the summation.
Since naive nodes do not have observations of the target, their estimates are often highly
erroneous. This results in reduced performance in the presence of naive nodes.
2) Covariance/Information Matrix Propagation: The information matrix mea-
surement update of Eqn. (2.8) considers the node’s own information matrix and the
local neighborhood’s measurement covariance. It does not account for cross covariance
between the estimates by the node and its neighbors. In the theoretical proof of op-
timality for KCF, the cross covariances terms between neighbors’ state estimates were
present [17]. It has been stated in [17] that dropping these cross covariance terms is a
valid approximation when the state estimate error covariance matrices are almost equal
in all the nodes.
However, when Ci is naive w.r.t. the target, bi and Bi are both zero. Therefore,
Mi = (J−i )−1 at Eqn. (2.8). Consequently, from Eqn. (2.11) it can be seen that the
diagonal elements of J−i tend to zero at each time update as long as Ci remains naive
with respect to the target. This makes the covariance matrix diverge. From this, it can
be clearly seen that omitting the cross covariances in the covariance update equation
is not valid for sparse networks with naive agents. The correlation between the two
dependent variables is the unknown parameter making this computation difficult. There
has been some work, e.g. [24] and [1], where the authors incorporated cross covariance
information, which should lead to the optimum result. But, no method for computing
these terms were provided and predefined fixed values were used instead.
3) Over-correction of the states: The measurement update term and consensus
term in Eqn. (2.9) are both functions of the prior state estimate x−i . Both terms apply
16
corrections to the prior state estimate, from different information sources. Thus the state
estimate might get over-corrected. This is usually not a big issue in sensor networks
without naive nodes because every node’s state estimate will be close to the consensus.
In sparse networks, the estimates of naive nodes may lag behind by a significant time.
This happens because naive nodes do not have direct access to new observation of a
target, the only way they can get updated information about a target is through a
neighbor’s state estimate which was updated in the previous iteration. Thus a naive
node might be multiple iterations away from getting new information about a target.
This information imbalance can cause large oscillations. In the KCF algorithms this
effect can be decreased by choosing a smaller rate parameter ε. However, decreasing ε
yields slower convergence of the naive node’s state estimate.
The above issues can be problematic for tracking applications involving a cam-
era network with naive nodes. A naive node may associate an observation to a wrong
target. This can affect the tracking performance of nodes that are actually observ-
ing the target by influencing them to drift away from their estimates. Since KCF is
a very appropriate framework to build a distributed tracker in a camera network, we
propose some changes to address the above challenges leading to a Generalized Kalman
Consensus Filter. The following are the main proposed modifications.
1) The consensus portion of the GKCF correction step at each node will take into
account the state covariances of neighbors. The nodes will then converge towards the
weighted mean, instead the unweighted mean.
2) Each node and its neighbors’ state covariance matrices will be used jointly at con-
sensus step to update that node’s error covariance matrix. This will prevent the state
covariance of the naive nodes from monotonically increasing.
3) Weighted average consensus will correct the prior estimate towards the weighted
17
mean. Then the measurement information in the local neighborhood will be utilized
to update this consensus state and covariance, thus preventing the overcorrection issue
mentioned above.
2.5 Generalized Kalman Consensus Filter
Now, we will present the GKCF algorithm which is summarized in Algorithm
2. To derive the GKCF algorithm, we first introduce the weighted average consensus.
Next, we show how to incorporate this consensus scheme into our framework.
2.5.1 Weighted Average Consensus
Let the initial state estimate of all NC agents be x−i with information matrix
J−i . As we use this information matrix term as weights in the weighted average consensus
algorithm, the terms weight and information matrix will be used interchangeably. So,
the centralized weighted average of the initial states is
xc =
(NC∑i=1
J−i
)−1 NC∑i=1
J−i x−i . (2.13)
Define the initial weighted state and weight of each agent as
vi[0] = J−i x−i , (2.14)
Vi[0] = J−i . (2.15)
Weighted average consensus [18] states that if the iterative update in Eqns. (2.16) and
(2.17)
vi[k] = vi[k − 1] + ε∑i′∈Ni
(vi′ [k − 1]− vi[k − 1]) (2.16)
Vi[k] = Vi[k − 1] + ε∑i′∈Ni
(Vi′ [k − 1]−Vi[k − 1]) (2.17)
18
is performed for all i = 1, . . . , NC , then each of the terms Vi[K]−1vi[K] tends to the
weighted average xc as K → ∞. As a by-product, the weights also converge to the
average of the initial weights i.e., Vi[K] → 1
NC
NC∑i=1
J−i . Both these properties of the
weighted average consensus will be utilized in our approach.
2.5.2 Covariance/Information Matrix Propagation
After communicating with its neighbors and prior to using measurement in-
formation, the optimal (in the local neighborhood) state estimate at Ci is a linear
combination of the information from Ci and its neighbors. Since these variables are not
independent, optimal estimation would require knowledge of the cross correlation struc-
ture between each pair of these random variables. Since, it is usually quite difficult to
compute this cross correlation, we need some other way to approximate the covariance
or in this case the information matrix. The update operation of the information matrix
Vi[k] in Eqn. (2.17) can be used as an approximation of the information matrix due to
the incoming information from the neighbors’ states. A property of the weighted average
consensus is that the weights also converge to the average of the weights as the state
estimates converge towards the weighted average. Thus, this kind of covariance/weight
propagation enables the weights to be updated accordingly when informative state esti-
mates arrive at a naive node.
After computing the state and weight estimates with all the available informa-
tion, we need to propagate the weight and state in time. One should note that instead
of propagating the state estimate, we have to propagate the weighted state estimate as
necessitated by the weighted average consensus equations. Thus the weight propagation
equation takes the form of Eqn. (2.28).
19
Algorithm 2 GKCF on sensor Ci a time-step t
Given J−i , x−i , ε and K. Also let,
1) Get measurements zi
2) Compute information vector and matrix
ui ← HTi R−1
i zi (2.18)
Ui ← HTi R−1
i Hi (2.19)
3) Broadcast ui,Ui to neighbors and receive ui′ ,Ui′ from neighbors
4) Fuse the information matrices and vectors in the local neighborhood
bi ←∑
i′∈Ni∪{i}
ui′ (2.20)
Bi ←∑
i′∈Ni∪{i}
Ui′ (2.21)
5) Initialize consensus variables
vi[0] ← J−i x−i (2.22)
Vi[0] ← J−i (2.23)
6) For K iterations run average consensus on vi[k] and Vi[k] and then compute updated estimate
x−i ← Vi[K]−1vi[K] (2.24)
J−i ← Vi[K] (2.25)
7) Compute GKCF estimate
J+i ← J−i + Bi (2.26)
x+i ← x−i + (J+
i )−1 (bi −Bix−i
)(2.27)
8) Propagate for next time step
J−i (t+ 1) ← (ΦJ+i (t)
−1ΦT + Q)
−1(2.28)
x−i (t+ 1) ← Φx+i (t) (2.29)
20
2.5.3 Two-stage Update
To resolve the issue of overcorrection of the states, we divide the estimation
process in two stages. First, as mentioned above, Ci updates its state and information
matrix using its neighbors’ states and information matrices. Next, we further update our
state and information matrix with current measurement information, which we explain
below.
Consider that a node that has completed Step 2 in Algorithm 2. If it did not
have any observation, then zi and R−1i were set to zero. Using the fused information
vector and matrix and the updated prior weight and state estimate (from the weighted
average consensus step of Eqns. (2.24) and (2.25)), we get the final state and weight
estimate at time t. Thus, using Eqns. (2.26) and (2.27) we can estimate the state
and weight which includes the properly weighted innovations from the measurements
and the state estimates of the neighbors. Note that GKCF is a more general algorithm
that at the expense of additional communications can converge even closer to the global
weighted average (by virtue of the weighted average consensus steps).
2.6 Theoretical Comparison
The KCF algorithm in [17] was originally presented using a single consensus
step. This section presents the state estimation step of the GKCF and KCF algorithm
in an equivalent form for a single consensus step (i.e., K = 1) and compares the differ-
ences between the algorithms theoretically. The derivation of the following results are
presented in the appendix.
21
GKCF (see Proposition 5 in Appendix)
x+i = x−i +
(A(J−i ) + Bi
)−1
bi −Bix−i + ε
∑i′∈Ni
J−i′(x−i′ − x−i
) (2.30)
J+i = A(J−i ) + Bi (2.31)
KCF
x+i = x−i +
(J−i + Bi
)−1 (bi −Bix
−i
)+
ε
1 + ||(J−i )−1||(J−i )−1
∑i′∈Ni
(x−i′ − x−i
)(2.32)
J+i = J−i + Bi (2.33)
The following points of comparison are important:
1) In the third term of Eqn. (2.32), KCF gives equal weight to all the neighbors’ priors.
In the presence naivety, this has detrimental effect as the information at different nodes
are different and need to be weighted by their information matrices. This is properly
accounted for in the GKCF algorithm as the innovation from each neighbor’s prior state(x−i′ − x−i
)is properly weighted by their information matrix J−i′ in Eqn. (2.30).
2) In Eqn. (2.30), GKCF uses(A(J−i ) + Bi
)−1to normalize both the innovation from
the measurements and the innovation from the state priors. Whereas, in Eqn. (2.32),
the normalizing terms are not balanced because KCF uses(J−i + Bi
)−1to normalize
the measurement innovation and (J−i )−1 to normalize the innovation from the priors.
3) In Eqn. (2.32), the normalizing term 1 + ||(J−i )−1|| is a design choice [17] to maintain
stability of the algorithm. However, it does not guarantee optimality of KCF. Although
GKCF is also not guaranteed to converge to the optimal centralized estimate, it does
not bear such a design choice.
22
2.7 Experimental Evaluation
In this section, we evaluate the performance of the proposed GKCF algorithm
in a simulated environment and compare it with other methods: the Centralized Kalman
Filter (CKF) and the Kalman Consensus Filter (KCF) [17]. Comparison in a simulated
environment allows an in-depth analysis of the proposed algorithms as parameters are
varied to measure performance under different conditions.
We simulate a camera network containing NT targets randomly moving (with
a fixed model) within a 500× 500 space. Each target’s initial state vector is random. A
set of NC camera sensors monitor the space (we consider that each communication node
consists of one camera). In each experiment, the cameras are randomly distributed in
the space with random orientations resulting in overlapping field-of-views (FOVs).
Target State Parameters
Each target was initialized at the center of the simulation grid. The target’s
state vector was a 4D vector, with the 2D position and 2D velocity components. The
initial speed was set to 2 units per time step and with a random direction uniformly
chosen from 0 to 2π. The targets evolved for 40 time steps using the target dynamical
model of Eqn. (2.1). Only the targets which remained in the simulation grid for the 40
time steps were considered. The process covariance Q is set to diag(10, 10, 1, 1).
For the target state-estimation model, the dynamical model of Eqn. (2.1)
was also used with the same Q defined above. The initial prior state x−i (1) and prior
covariance P−i (1) are set equal at each node. A diagonal matrix is used for P−i (1)
with the main diagonal elements as {100, 100, 10, 10}. The initial prior state x−i (1) is
generated by adding Gaussian noise of covariance P−i (1) to the ground truth state.
23
Sensor Parameters
A total of NC = 15 nodes were generated at uniformly chosen random locations
on the simulation area. The measurement vector length for each sensor is 2. The FOV
was chosen to be equilateral triangles. We define the sensing range, SR, for each sensor
to be the height of this equilateral triangle. SR was chosen to be 300 units for all the
sensors. A sensor can have an observation of a target only if the ground truth position of
the target is within the sensor’s FOV and in that case, a measurement zi was generated
using the linear observation model Eqn. (2.2) with noise covariance Ri = 100I2. The
observation matrix Hi and state transition matrix Φ is given below.
Hi =
1 0 0 0
0 1 0 0
, Φ =
1 0 1 0
0 1 0 1
0 0 1 0
0 0 0 1
.
Network Topology Parameters
A circulant graph was chosen as the communication topology for the sensor
network to highlight the different issues associated with the sparsity of the communi-
cation network. The circulant graph is ∆-regular where the degree of each node is the
same, say ∆. The adjacency matrix of a circulant graph is a circulant matrix. So, we
denote ∆ to be the degree of the communication graph G. In the experiments, ∆ = 2
was used, unless stated otherwise. Note that, in this scenario, the maximum degree
∆max = ∆ because the degree of all the nodes are same.
24
Consensus Parameters
In Algorithm 2, each sensor communicates its measurement information to
its neighbors iteratively. The maximum number of consensus iterations K was set
to 5 unless stated otherwise. The consensus speed parameter was chosen to be ε =
0.65/∆max = 0.65/∆.
Experimental Description
The parameters that were varied in the experiments are, the maximum number
of consensus iterations K, degree of the communication network ∆, sensing range SR
and the number of cameras NC . For each experiment, only one parameter was varied
while the others were kept constant. As a measure of performance, we computed the
estimation error, e, defined as the Euclidean distance between the ground truth position
and the estimated posterior position. An example of the simulation framework is shown
in Fig. 2.2.
For each experiment, 20 different simulation environments differing in camera
poses were randomly generated using the method discussed above. For each environ-
ment, 20 different randomly generated target tracks were used. Thus, for each experi-
ment, the estimation errors e, were averaged over 20 × 20 = 400 random scenarios, 40
time steps and over all sensors NC . The mean errors for different methods are shown in
the following graphs as the results of different experiments. In the graphs, each line (of
a unique color) corresponds to the mean error e for one estimation method.
The KCF algorithm as originally proposed in [17] uses a single consensus step
per measurement update. To compare it with GKCF, which supports multiple iterations,
we extend KCF for multiple iterations. For this, at each time step, the measurement
25
0 100 200 300 400 5000
100
200
300
400
500
Figure 2.2: In this image of an example simulation environment, the red arrows indicate the
locations and orientations of the cameras. The camera FOVs are shown in blue triangles. There
are 7 cameras in this example. The green dotted lines represent the network connectivity. Each
black arrows depict the actual trajectory of a target moving on the grid.
innovation component is set to zero for k > 1 and we consider only the new information
provided by the neighboring nodes’ estimates.
2.7.1 Experiment 1: Varying K
The objective of this experiment is to compare the performance of different
estimation algorithms for different K. Here, K was varied from 1 to 20 at increments
of 1. The other parameters were kept constant at their default values. The priors were
chosen to be equal.
The results of this experiment are shown in Fig. 2.3. The graph shows that
for K = 1, GKCF performs much better than KCF close to CKF. As, the number of
iterations K is increased, the mean error for GKCF decreases. The main reason for
this difference in performance between KCF and GKCF is that KCF does not account
for the different information content in different nodes’ prior information which unlike
KCF, GKCF does. In this simulation all the initial priors were equal.
26
2 4 6 8 10 12 14 16 18 205
10
15
20
25
30
35
40
Number of Consensus Iterations, K
Mea
nEr
ror,
e
CKFKCFGKCF
Figure 2.3: Varying K with equal initial priors
100
150
200
250
300
2 4 6 8 10 12 14 16 18 205
10
15
20
25
30
35
Number of Consensus Iterations, K
Mea
nEr
ror,
e
CKFKCFGKCF
Figure 2.4: Varying K with unequal initial
priors
To show the robustness of the GKCF approach, we conducted the same exper-
iment with unequal and uncorrelated priors (Fig. 2.4) using Algorithm 2. The results
show that the performance of KCF is severely affected by this whereas GKCF is very
robust to unequal initial priors.
2.7.2 Experiment 2: Varying ∆
The objective of this experiment is to compare the performance of different
approaches for different values of the degree ∆ (i.e., vary the sparsity of the network
from sparse connectivity to dense connectivity). For NC = 15, the maximum possible
degree can be ∆ = 14 at full mesh connectivity. In this experiment, ∆ was varied from
2 to 14 at increments of 2.
The results are shown in Fig. 2.5, where the total number of consensus itera-
tions K was set to 5. It can be seen that the GKCF performs much better than KCF
at all ∆ even with very sparse connectivity. For full connectivity, i.e. ∆ = 14, where G
is a complete graph, all the distributed methods achieve centralized performance.
27
2 4 6 8 10 12 148
10
12
14
16
18
20
22
24
26
28
Degree of Communication Graph, ∆
Mea
nEr
ror,
eCKFKCFGKCF
Figure 2.5: Varying the density of connectiv-
ity of the network graph.
150 200 250 300 350 400 450 500 550 6000
10
20
30
40
50
60
Sensor Range, SR
Mea
nEr
ror,
e
CKFKCFGKCF
Figure 2.6: Varying the sensing range of the
sensors.
2.7.3 Experiment 3: Varying SR
The objective of this experiment is to compare the performance of different
approaches as a function of the sensor range SR (i.e., varying the area of coverage of
each sensor.)
Fig. 2.6 shows the performance of each approach as SR is varied and the total
number of iterations K was set to 5. It can be seen that GKCF performed better than
KCF at each sensor range.
2.7.4 Experiment 4: Varying NC
The objective of this experiment is to compare the performance of different
approaches as a function of the total number of sensors, NC , as it was varied from 5 to
31 at increments of 2. Values less than 5 were not used because at such low number of
sensors, even the centralized algorithm cannot perform well due to the high number of
time instants at which the number of measurements is insufficient for the state vector
to be observable, due to the low percentage coverage of the environment.
Fig. 2.7 shows results for variable NC and fixed K = 20. As the number
of sensors is increased, the observability of the targets by the network increases, but
28
5 10 15 20 25 300
10
20
30
40
50
Number of Sensors, N C
Mea
nEr
ror,
e
CKFKCFGKCF
Figure 2.7: Performance comparison of different approaches by varying total number of sensors.
the number of naive nodes also increases. Due to the amount of information about
the targets increasing, the performance of all the approaches (including centralized)
improves. For a small NC , all the distributed methods performed close to the centralized
method, because the number of naive nodes is small and the network distance from a
naive node to an informed node is small. As the number of nodes increases, the number
of consensus iterations required to reach consensus also increases. Thus we can see
that the performance of all distributed algorithms (both GKCF and KCF) deteriorates
more from the centralized performance for high number of sensors. For all NC , GKCF
outperformed KCF.
2.8 Conclusion
In this chapter, we introduced a novel method for distributed state estima-
tion algorithm, the Generalized Kalman Consensus Filter (GKCF). We discussed under
what circumstances the assumptions of KCF are not valid and hence modifications are
necessary. This is especially true in camera networks where each sensor has a limited
FOV and they are geographically separated by distances that do not allow full commu-
nication. Then we proposed a generalized framework, Generalized KCF, which outper-
29
formed the KCF approach under such conditions. We showed the theoretical derivation
of our framework and also showed simulation results to compare the performance of our
algorithm with other approaches.
30
Chapter 3
Information-weighted Consensus
Filter
3.1 Introduction
Among many types of distributed estimation schemes, consensus algorithms
[18] are schemes where each node, by iteratively communicating with its network neigh-
bors, can compute a function of the measurements at each node (e.g. average). The
consensus estimates asymptotically converge to the global result. In practice, only a
limited number of iterations can be performed due to limited bandwidth and target
dynamics. Thus, true convergence may not be always reached. In the presence of state
dynamics, usually a predictor-corrector model is used for state estimation, where a state
prediction is made from the prior information and corrected using new measurements.
The Kalman Consensus Filter (KCF) [17] is a popular distributed state estimation
framework based on the average consensus algorithm. KCF works well in situations
where each node gets a measurement of the target.
31
C1 C5
C3
C2C4
T1
T2
T3
T4
T5
T6
T7
Figure 3.1: In this figure, there are five sensing nodes, C1, C2, . . . , C5 observing an area (black
rectangle) consisting of seven targets T 1, T 2, . . . T 7. The solid blue lines show the communication
channels between different nodes. This figure also depicts the presence of “naive” nodes. For
example, C1 gets direct measurements about T 1 which it shares with its immediate network
neighbor, C2. However, the rest of the cameras, i.e., C3, C4, C5 do not have direct access to
measurements of T 1 and thus are naive w.r.t. T 1’s state.
In a sensor network, a node might have limited observability when it does not
have any measurement of a target available in its local neighborhood (consisting of the
node and its immediate network neighbors). Due to limited observability and limited
number of iterations, the node becomes naive about the target’s state. A naive node (see
Fig. 3.1) contains less information about the state. If a naive node’s estimate is given
an equal weight in the information fusion scheme (as in KCF), the performance of the
overall state estimation framework may decrease. The effect of naivety is severe in sparse
networks where the total number of edges is much smaller than the maximum possible
number of edges. The Generalized Kalman Consensus Filter (GKCF), was discussed in
the previous chapter which overcomes this issue by utilizing a weighted-averaging con-
sensus scheme where the priors of each node were weighted by their covariance matrices.
32
Both KCF and GKCF estimates are suboptimal and the main reason is that
the cross-covariances between the priors across different nodes are not incorporated in
the estimation framework. As the consensus progresses, the errors in the information
at each node become highly correlated with each other. Thus, to compute the optimal
state estimate, the error cross-covariances cannot be neglected. However, it is difficult to
compute the cross-covariance in a distributed framework. We note that in a consensus-
based framework, the state estimates at different nodes achieve reasonable convergence
over multiple iterations. At this point, each node contains almost identical/redundant
information. This fact can be utilized to compute the optimal estimate in a distributed
framework without explicitly computing the cross-covariances.
Motivated by this idea, we propose an information-weighted consensus algo-
rithm for distributed state estimation which is guaranteed to converge to the optimal
centralized estimates as the prior state estimates become equal at different nodes i.e.,
the total number of iterations approach to infinity at the previous time step. We also
show experimentally that even with limited number of iterations, the proposed algorithm
achieves near-optimal performance. The issue of naivety and optimality is handled by
proper information weighting of the prior and measurement information. The commu-
nication bandwidth requirement is also low for the proposed method.
3.1.1 Contributions
The KCF algorithm weights all its neighbors’ prior states x−i′ ’s equally which
causes high estimation error when naive nodes are present. An initial approach to
resolve this issue was proposed in [11]. There the Generalized Kalman Consensus Filter
(GKCF) algorithm was proposed where the neighbors’ prior x−i′ ’s were weighted by
their covariance matrices P−i′ ’s. The GKCF algorithm outperforms the KCF in the
33
presence of naive nodes. However, the effect of correlation between errors of the nodes’
prior estimates was not brought into account in any of the prior methods because it
is usually extremely hard to estimate the cross-covariance across all the nodes in a
distributed framework. Mainly due to this reason, these distributed estimation schemes
are suboptimal.
Naivety relative to the j-th target is likely to be present in the network when
nodes on the vision graph for target j are sparsely connected in the communication
graph. The vision graph (see [30]) for target T j is a graph where there is an edge
between each pair of nodes that are observing T j (see Fig. 3.1). The vision graph is
usually time varying, different from the communication graph, and different for different
targets. Distributed algorithms that are derived under the assumption that each sensor
has full state observability at each time (e.g. KCF), may be adapted to be used in the
presence of naive nodes when communication protocols (e.g. [9]) for translating a known
vision graph into a set of routing paths that connect nodes in the communication graph
are available. However, scalable distributed vision graph discovery algorithms do not
exist. Moreover, distributed algorithms that require the knowledge of the vision graphs
usually require special target handoff protocols.
Additionally, computation and communication resource constraints are impor-
tant issues in a distributed estimation framework because in many application scenarios,
the nodes are low powered wireless devices. Therefore, a distributed state estimation
framework that can guarantee convergence to the optimal centralized estimate while
maintaining low computation and communication resource requirements in the presence
of naive nodes is desirable. In this chapter, we propose such a distributed state esti-
mation framework called the Information-weighted Consensus Filter (ICF). The ICF
is guaranteed to converge to the optimal centralized performance under certain rea-
34
sonable conditions. We also show experimentally that in other conditions it achieves
near-optimal performance.
The issue of naivety is handled in the ICF algorithm by proper information
weighting in the estimation framework. Optimality is achieved by proper relative weight-
ing between the prior and the measurement information. ICF also supports multiple
consensus iterations at each time step t to improve performance. In addition, the ex-
perimental results show that ICF outperforms other distributed estimation algorithms
at any given computation and communication resource limit. The ICF algorithm does
not require handoff protocols or the knowledge of the vision graph.
3.1.2 Related work
Recently, distributed decision and control frameworks have gained immense
popularity. Consensus algorithms [18] are one of the many types of distributed schemes
used for collective decision making. Consensus algorithms are protocols that are run in-
dividually by each agent where each agent communicates with just its network neighbors
and corrects its own information iteratively using the information sent by its neighbors.
The protocol, over multiple iterations, ensures the convergence of all the agents in the
network to a single consensus. The consensus they reach is a predefined function of all
the information available in the network. It is important to note that this consensus
is reached just by peer-to-peer communication without requiring a central fusion node.
For example, the item being estimated may be the arithmetic mean (average consensus)
[18] or the geometric mean [19] of the initial values. The simplicity and scalability of
consensus algorithms makes them extremely useful in distributed estimation tasks in
sensor networks.
35
Consensus algorithms have been extended to perform various linear algebraic
operations such as SVD, least squares, PCA, GPCA in a network of agents [30] It also
has been utilized in distributed state estimation framework such as the Kalman Consen-
sus Filter (KCF) [17] and the Generalized Kalman Consensus Filter (GKCF) [11] that
we discussed in the previous chapter. The KCF algorithm is a popular distributed esti-
mation framework and has reasonable performance in networks where the entire state is
individually observable by each node. However, its performance deteriorates in the pres-
ence of naivety [11]. The idea of information-weighted consensus was introduced in [13];
however, that article did not provide the detailed theoretical analysis of its properties,
comparisons with other approaches, or the implications for its application in wide-area
camera networks. These items are each provided herein. A survey on distributed esti-
mation and control applications using linear consensus algorithms can be found in [8].
Here the authors show how a weighted average can be computed in a distributed frame-
work using two parallel average consensus schemes. Distributed state and parameter
estimation frameworks have been applied in various fields including camera networks
for distributed implementations of 3-D point triangulation, pose estimation [30], track-
ing [28], action recognition [28, 15], collaborative tracking and camera control [27, 7],
camera calibration [29, 3] etc.
3.2 Problem Formulation
The communication network is represented by the undirected connected graph
G = (C, E). The set C = {C1, ..., CNC} contains the vertices of the graph and represents
the communication nodes. The set E contains the edges of the graph, which represent
the available communication channels between different nodes. Also, let Ni be the set of
36
nodes that have a direct communication channel with node Ci (i.e. shares an edge with
Ci). We call the nodes in Ni, the neighbors of Ci. The number of neighbors (degree)
of Ci is represented by ∆i. For simplicity, we will drop the time step index t in places
where the time step is not important to explain the issue under consideration. There
are NT targets in the area and the length of their individual state vectors is q. Thus,
for x ∈ Rp, p would be equal to qNT . The number of targets can be time-varying, but
that will not be a focus of the thesis, as it would distract from the main topic.
A data association protocol is required for a multi-target estimation framework.
However, in this chapter, to analyze the performance of the state estimation approaches
independent of a data association method, we assume that the data association is given
and without errors. Data association will be brought into account in the following
chapters.
At each time step t, a node Ci may get some measurements from its sensors. For
example, for a target state estimation task in a camera network, a measurement might
be the projection of the position of some target onto a camera’s pixel coordinate system.
We denote the measurement at Ci as zi ∈ Rmi . As, the sensors can be heterogeneous
and the number of sensors can be different at each communication node, the length
of the measurement vector mi can vary across different nodes. The measurement zi is
modeled as
zi = Hix + νi. (3.1)
Here Hi ∈ Rmi×p is the observation matrix for node Ci. The matrix Hi can be time-
varying. The noise νi ∈ Rmi in the measurement of Ci is modeled as a zero mean
Gaussian random variableN (0,Ri) where Ri ∈ Rmi×mi . As we will derive the estimator
in the information form, we will generally end up with equation with inverse covariances
37
(information matrix/ precision matrix). If a node Ci does not have a measurement, we
will use (Ri)−1 = 0 ∈ Rmi×mi and zi = 0 ∈ Rmi . Note that Hi is typically not full
column rank (mi < p). In fact, we are not assuming that the state x is observable from
zi. We do assume that x is observable from {zi}NCi=1. The collection of all measurements
from all sensors can be expressed as,
Z = Hx + ν. (3.2)
Here, Z = [zT1 , zT2 , . . . z
TN ]T ∈ Rm is the concatenation of all measurements in the
network where m =∑N
i=1mi and H = [HT1 ,H
T2 , . . .H
TN ]T ∈ Rm×p is the stack of all
the observation matrices. For the measurement noise vector, ν = [νT1 ,νT2 , . . .ν
TN ]T ∈
Rm, we denote its covariance as R ∈ Rm×m. We assume the measurement noise
to be uncorrelated across nodes. Thus, the measurement covariance matrix is R =
diag(R1,R2, . . .RNC).
3.3 Centralized MAP Estimation
Before considering the decentralized solution for the estimation of x it is in-
formative to review the centralized solution, i.e., the centralized maximum a posteriori
(MAP) estimator. The centralized prior state estimate of x is denoted as x−c ∈ Rp
where the error in the prior estimate is ηc = x−c − x with cov(ηc) = P−c ∈ Rp×p, which
is assumed to be nonsingular. The observation model in Eqn. (3.2) and prior state x−c
can be combined into one equation as x−c
Z
=
Ip
H
x +
ηcν
. (3.3)
Here Ip is the p× p identity matrix.
38
Letting, Y =
x−c
Z
, Hc =
Ip
H
and β =
ηcν
, we have Y = Hcx + β,
where β ∼ N (0,C). We assume that the error in the prior state estimate is uncorrelated
to the measurement noise. Thus, we have the block diagonal covariance matrix C =
diag(P−c ,R). Let us define the prior information matrix to be J−c = (P−c )−1 ∈ Rp×p.
Thus, defining J = C−1 we have J = diag(J−c ,R−1). The centralized maximum a
posteriori (MAP) estimate [16] of the state x is
x+c = (HT
c JHc)−1(HT
c JY)
=(J−c + HTR−1H
)−1(J−c x−c + HTR−1Z), (3.4)
J+c =
(J−c + HTR−1H
). (3.5)
where J+c = (cov(x+
c ))−1
quantifies the information about x in x+c . Eqns. (3.4-3.5) are
useful in the discussion of the physical interpretations of the alternative decentralized
algorithms.
3.4 Information Consensus based Distributed MAP Esti-
mation (IC-MAP)
In a distributed estimation framework, each node Ci possesses a prior estimate
of the state vector that we denote as x−i ∈ Rp (unlike a single prior state estimate in a
centralized solution). The objective of the network is to use distributed computations
across the network such that the posterior state estimate x+i ∈ Rp at each node converges
to the centralized estimate x+c . However, due to resource constraints, this convergence
may not be fully achieved at any given time. Therefore, if consensus were performed
directly on the priors, then at the k-th consensus iteration, the estimate of the i-th node
39
can be modeled as having three components
x−i [k] = x + ηc + δi[k],
where x−c = x + ηc with the quantities x and ηc having been previously defined, and
the consensus algorithm ensures that ‖δi[k]‖ approaches zero as k approaches infinity
for all i. The index in the parenthesis after a variable will refer to the time-index and
the index in the square brackets will refer to the consensus iteration index.
The error covariance in the prior estimate at Ci is P−ii [k] = E[ηi[k] (ηi[k])T ] ∈
Rp×p, where ηi[k] = ηc + δi[k]. Similarly, the error cross-covariance between Ci and
Ci′ ’s prior estimates is P−ii′ [k] = E[ηi[k] (ηi′ [k])T ] ∈ Rp×p. As k → ∞, at each node,
consensus forces x−i [k] → x−c . Therefore, for any {i, i′}, x−i [k] and x−i′ [k] becomes
correlated as x−i [k] → x−i′ [k] resulting in P−ii′ [k] 6= 0. In fact, it is straightforward to
show that as the number of consensus iterations k approaches infinity, P−ii′ [k] converges
to P−c for all {i, i′}.
We drop the k and denote the collection of all the state priors from all the nodes
as X−
= [(x−1 )T , (x−2 )T , . . . , (x−NC)T ]T ∈ RNp. The relationship between the state, the
priors and the prior errors can be summarized as,
X−
= HIx + η. (3.6)
Here, x is the true state of the targets, η = [ηT1 ,ηT2 , . . .η
TNC
]T ∈ RNCp is the error
vector, and HI = [Ip, Ip, . . . , Ip]T ∈ RNCp×p.
Combining the measurements with the result of a finite number k of steps of
consensus on the priors yields X−
Z
=
HI
H
x +
ην
. (3.7)
40
Letting, Y ′ =
X−
Z
, H′c =
HI
H
and β′ =
ην
, we have Y ′ = H′cx+β′, where
β′ ∼ N (0,C′). The noise term β′ is Gaussian because it is accumulated through one or
more consensus iterations (which are linear operations) performed on Gaussian random
variables.
Let us denote the prior information matrix F = P−1 where these two matrices
can be expressed as (p× p) blocks,
P =
P−11 P−12 . . . P−1NC
P−21 P−22
...
.... . .
P−NC1 . . . P−NCNC
, F =
F11 F12 . . . F1NC
F21 F22...
.... . .
FNC1 . . . FNN
. (3.8)
Let us define the information matrix of the prior of node i as
J−i = (P−ii )−1 (3.9)
Here, J−i ∈ Rp×p and in general, J−i 6= Fii. Assuming that the error in the prior
state estimates are uncorrelated to the noise in the new measurements, we have the
block diagonal covariance matrix C′ = diag(P ,R). Thus, we get its inverse as J ′ =
diag(F ,R−1).
The centralized maximum a posteriori (MAP) estimate of the state x is
x+c = (H′T
c J ′H′c)−1(H′T
c J ′Y ′)
=(HTI FHI + HTR−1H
)−1(HT
I FX−
+ HTR−1Z), (3.10)
J+c = HT
I FHI + HTR−1H. (3.11)
Defining
F−i =
NC∑i′=1
Fi′i, (3.12)
41
we have
HTI FHI =
NC∑i=1
F−i and HTI FX
−=
NC∑i=1
F−i x−i . (3.13)
Let us define Ui = HTi R−1
i Hi and ui = HTi R−1
i zi. Due to the block diagonal
structure of R−1, we get
HTR−1H =
NC∑i=1
HTi R−1
i Hi =
NC∑i=1
Ui, (3.14)
HTR−1Z =
NC∑i=1
HTi R−1
i zi =
NC∑i=1
ui. (3.15)
Thus, from Eqns. (3.10) and (3.11), we get
x+c =
(NC∑i=1
(F−i + Ui)
)−1 NC∑i=1
(F−i x−i + ui), (3.16)
J+c =
NC∑i=1
(F−i + Ui). (3.17)
This is the centralized solution that fully accounts for the presence of different priors at
each agent, and the cross-correlated errors in the priors between agents, which develop
naturally due to consensus, but which are not known in a decentralized implementation.
In the following, we show how Eqns. (3.16) and (3.17) can be computed in a distributed
manner.
3.4.1 Distributed Implementation
To implement Eqns. (3.16-3.17) in a distributed fashion, define, Vi[0] = F−i +
Ui and vi[0] = F−i x−i + ui, so that
x+c =
(NC∑i=1
Vi[0]
)−1 NC∑i=1
vi[0], (3.18)
J+c =
NC∑i=1
Vi[0]. (3.19)
42
Under the assumption that a node Ci has information about F−i (methods for computing
F−i will be discussed later), Ci could compute Vi[0] = F−i + Ui and vi[0] = F−i x−i + ui
from F−i , its own state prior x−i , measurement zi, measurement information matrix R−1i
and measurement model parameter Hi. Then, each node transmits to its neighbors its
own information matrix Vi[k] ∈ Rp×p and information vector vi[k] ∈ Rp, receives its
neighbors’ information, and uses the average consensus algorithm as described in Sec.
2.3 to converge toward the global averages of these two quantities. Therefore, from
Eqns. (3.18-3.19) we get
x+c = lim
k→∞(NCVi[k])−1 (NCvi[k]) = lim
k→∞(Vi[k])−1 vi[k] (3.20)
J+c = lim
k→∞NCVi[k] (3.21)
From the discussion above, given F−i , it is clear that the centralized MAP estimate
in Eqns. (3.18-3.19) is achieved using a distributed scheme as k → ∞. We call this
distributed approach of computing the MAP estimate, the Information Consensus based
MAP (IC-MAP) estimation framework.
3.4.2 Computation of F−i
To compute the distributed MAP estimate, node Ci needs to have knowledge
of F−i . In general, computation of F−i requires the knowledge of the entire covariance
matrix P defined in Eqn. (3.8) (i.e. the prior covariances (P−ii ’s) of each node and the
prior cross-covariances (P−ii′ ’s) between each pair of nodes). However, computing P at
every time step at each node in a distributed framework is unrealistic as it would require
too much information exchange among the nodes. However, in the following, we show
that for two special cases (which are of great practical importance), F−i can be com-
puted at each node using only a node’s own prior covariance matrix P−ii (or J−i in the
43
information form). The first case is for converged priors which is an important scenario
because in a consensus-based framework, with high-enough number of consensus itera-
tions, the prior state information at all the nodes ultimately converge to the same value.
The second case is when the prior state estimates across the nodes are uncorrelated to
each other. This is generally true at the early steps of consensus when the nodes had no
prior information about the target and initialized their prior information with random
quantities.
The proposed distributed state estimation framework in Sec. 3.5 is based on
these two special cases. The practical significance of these two cases can be seen from
the experimental results in Sec. 3.6 which implies that the proposed algorithm (derived
from these two special cases) is robust even when the assumptions of neither of these
two cases are met.
3.4.2.1 Case 1: Converged Priors
Here we will discuss the case where the estimate of the state vector at each node has
converged to the centralized estimate at the previous time step t − 1. Thus at time t,
the prior information at each node is the same and equal to the prior information of a
centralized framework (i.e., k sufficiently large such that ‖δi[k]‖ = 0 for all i). This case
will be of great significance when we will incorporate target dynamics and additional
measurement steps to our framework. From Eqns. (3.5) and (3.17) we have
J+c = J−c + HTR−1H =
NC∑i=1
(F−i + Ui). (3.22)
as from Eqn. (3.14), HTR−1H =∑NC
i=1 Ui. Thus, from Eqn. (3.22),
NC∑i=1
F−i = J−c =
NC∑i=1
J−cNC
. (3.23)
44
Now, for converged priors, for all i we have J−c = J−i . Using this in Eqn. (3.23), we
have
NC∑i=1
F−i =
NC∑i=1
J−iNC
. (3.24)
Using this in Eqns. (3.16) and (3.17) and using the fact that x−i = x−c for converged
priors, we have
x+c =
(NC∑i=1
(J−iNC
+ Ui
))−1 NC∑i=1
(J−iNC
x−i + ui
), (3.25)
J+c =
NC∑i=1
(J−iNC
+ Ui
). (3.26)
This can be computed in a distributed manner by initializing Vi[0] =1
NCJ−i + Ui
and vi[0] =1
NCJ−i x−i + ui. This is equivalent to using
1
NCJ−i instead of F−i in Eqns.
(3.16-3.17).
The intuition behind this is as follows. After convergence, given the prior at
one node, the other priors do not contain any new information. Upon convergence in
the previous time step, the prior information matrix at each node J−i is equal to J−c :
Each agent has an identical copy (x−i = x−c ) and amount of information (J−i = J−c ).
Thus at the current time step, the prior information matrix J−i should be divided by
NC as shown in the formulation of Eqns. (3.25-3.26), so that the effective total weight
of all the priors in the estimation scheme remains as J−c .
3.4.2.2 Case 2: Uncorrelated Prior Errors
Now, we consider the case where the errors in the priors are uncorrelated with each
other across different nodes. This case can be used at the initial time step if it is known
that the prior errors are uncorrelated across different nodes.
45
When the prior state errors are uncorrelated, the covariance matrix P will be
block diagonal. So, its inverse F will also be block diagonal as, F = diag(F11 , F22,
. . . FNCNC) = diag
((P−11)−1, (P−22)−1, . . . (P−NCNC
)−1)
. In this special case, with J−i
defined in Eqn. (3.9), we have
Fii = (P−ii )−1 = J−i . (3.27)
As the off-diagonal block elements of F are zero, from the definition of F−i in Eqn.
(3.12), we have
F−i =
NC∑i′=1
Fi′i = Fii = J−i . (3.28)
Thus, when the prior errors are uncorrelated across the different nodes, using F−i = J−i
(in Eqns. (3.16-3.17)) and average consensus, we can compute the centralized MAP
estimate in a distributed framework.
3.5 Information-weighted Consensus Filter
In the previous section, we derived the Information Consensus based MAP
(IC-MAP) estimation framework and proved its optimality for two important scenarios.
In this section, we will consider a dynamic model in state space form and extend the
IC-MAP framework in Sec. 3.4 for distributed state estimation. We call this distributed
state estimation framework, the Information-weighted Consensus Filter (ICF). We prove
theoretically that as the number of consensus iteration k → ∞, the ICF estimates
converge to the estimates of the centralized Kalman filter.
Let us consider the following linear dynamical model
x(t+ 1) = Φx(t) + γ(t). (3.29)
Here Φ ∈ Rp×p is the state transition matrix and the process noise is γ(t) ∼ N (0,Q).
46
For this model, for the centralized case, we have the following state prediction
step [16],
J−c (t+ 1) =(ΦJ+
c (t)−1ΦT + Q)−1
, (3.30)
x−c (t+ 1) = Φx+c (t). (3.31)
Combining this with the IC-MAP estimation framework proposed in Sec. 3.4, we get
the Information-weighted Consensus Filter (ICF) in Algorithm 3.
Algorithm 3 ICF at node Ci at time step t
Input: prior state estimate x−i (t), prior information matrix J−i (t), observation matrix Hi, consensus
speed factor ε and total number of consensus iterations K.
1) Get measurement zi and measurement information matrix R−1i
2) Compute consensus proposals,
Vi[0] ← 1
NCJ−i (t) + HT
i R−1i Hi (3.32)
vi[0] ← 1
NCJ−i (t)x−i (t) + HT
i R−1i zi (3.33)
3) Perform average consensus (Sec. 2.3) on vi[0] and Vi[0] independently for K iterations.
4) Compute a posteriori state estimate and information matrix for time t
x+i (t) ← (Vi[K])−1vi[K] (3.34)
J+i (t) ← NCVi[K] (3.35)
5) Predict for next time step (t+ 1)
J−i (t+ 1) ←(Φ(J+
i (t))−1ΦT + Q)−1
(3.36)
x−i (t+ 1) ← Φx+i (t) (3.37)
Output: ICF estimate x+i (t) and information matrix J+
i (t).
47
Note that in Eqns. (3.32) and (3.33), we have used the results for the converged
prior case (Sec. 3.4.2.1), i.e., using1
NCJ−i instead of F−i . Theoretically, at each time
step, if k → ∞, the IC-MAP estimator guarantees that the priors for the next time
step at each node will be equal to the optimal centralized one. This convergence further
guarantees that the optimal centralized estimate will be reached at the next time steps
if F−i =1
NCJ−i is used. This guarantees the optimality of Algorithm 3 with k → ∞ at
each time step.
In practice, due to the fact that the total number of iterations, k, is finite,
convergence will not be achieved fully. Therefore, the distributed implementation will
not perform quite as well as the centralized solution. The simulation results in Sec. 3.6
will demonstrate that the ICF has near-optimal performance as either k increases or t
increases for a fixed k.
While Eqn. (3.20) shows that inclusion of NC is inconsequential in the com-
putation of x in the CKF, the role of NC is critical in the distributed implementation.
At Eqns. (3.32) and (3.33), due to prior consensus steps, all nodes have the same es-
timate with the same information J−i . If the1
NCis neglected, then the measurement
information receives too little relative weighting by a factor of NC .
3.5.1 Initialization and Special Situations
In a practical implementation scenario, at system startup or for the first few
iterations in a naive node, Vi[0] in Eqn. (3.34) can be 0, if there is no prior or mea-
surement information available in the local neighborhood. In this situation, a node will
not perform step 4 and 5 in Algorithm 3 until it receives non-zero information from its
neighbors (through step 3) or gets a measurement itself (through step 1) yielding Vi[0]
to be non-zero. In the target tracking application, this situation occurs when a new
48
target is detected by one or more, but not all of the sensors.
In some situations prior information may be present at system startup. If the
priors are known to be equal, then using the standard ICF algorithm as in Algorithm
3, should give optimal results. However, if the priors across the nodes are known to
be uncorrelated at the initial time step (t = 1), the results for the uncorrelated case
(Sec.3.4.2.2) should be used instead in the first time step. To do this, only at t = 1,
instead of Eqns. (3.32) and (3.33) in Algorithm 3, the following initializations should
be used,
Vi[0] = J−i + HTi R−1
i Hi (3.38)
vi[0] = J−i x−i + HTi R−1
i zi. (3.39)
Thus, at t = 1, with k → ∞, the states would converge to the optimal centralized
estimate. Then for t > 1, using Algorithm 3 would guarantee convergence to the optimal
centralized estimate for the following time steps (i.e. for t > 1).
An example of the estimation results of different methods is shown in Fig. 3.2.
The example includes NC = 15 nodes and a single target. The target state contains
the two dimensional position and velocity. Each line represents the state estimate at
one node after each iteration. For clarity, in each experiment only the first component
of the state vector for the first 3 node’s is shown. State estimation is performed for 3
time steps. At each time step, 30 consensus iterations were performed. The priors were
initially uncorrelated and different at each node. For ICF, at t = 1, the uncorrelated
initializations Eqns. (3.38-3.39) were used. This example shows that the ICF converges
to the centralized estimate at each time step after several iterations. The reason that ICF
performs better than KCF is because KCF’s performance deteriorates in the presence of
naivety and the cross-covariances between the priors are implicitly considered in ICF.
49
0 10 20 30 10 20 30 10 20 30210
220
230
240
250
260
270
Consensus Iterations, k
Stat
e
3=t2=t1=t
TrueCKFKCFGKCFICF
Figure 3.2: An example showing the convergence of different algorithms with multiple consensus
iterations at different time steps.
3.5.2 ICF, GKCF and KCF Comparison
The KCF algorithm in [17] was originally presented using a single consensus
step. This section presents the state estimation step of the ICF, GKCF and KCF
algorithm in an equivalent form for a single consensus step (i.e., K = 1) and compares
the differences between the algorithms theoretically. The derivation of the following
results are presented in the appendix.
ICF (Asymptotically optimal with equal initial priors) (see Proposition 1 in Appendix)
x+i = x−i +
(A(
J−iNC
)+A(Ui)
)−1
A(ui)−A(Ui)x−i + ε
∑i′∈Ni
J−i′
NC
(x−i′ − x−i
) (3.40)
J+i = NC
(A(
J−iNC
)+A(Ui)
)(3.41)
50
GKCF (Suboptimal) (see Proposition 5 in Appendix)
x+i = x−i +
(A(J−i ) + Bi
)−1bi −Bix−i + ε
∑i′∈Ni
J−i′(x−i′ − x−i
) (3.42)
J+i = A(J−i ) + Bi (3.43)
KCF (Suboptimal)
x+i = x−i +
(J−i + Bi
)−1 (bi −Bix
−i
)+
ε
1 + ||(J−i )−1||(J−i )−1
∑i′∈Ni
(x−i′ − x−i
)(3.44)
J+i = J−i + Bi (3.45)
The following points of comparison are important:
1) In GKCF and KCF, the terms Bi and bi are the summation of the measurement
information and the weighted measurements in the local neighborhood. This fusion
mechanism, unlike average consensus, does not guarantee a proper global convergence.
However, in ICF, A(Ui) and A(ui) are used which guarantee convergence to the global
average values.
2) In GKCF and KCF, J−i is used instead ofJ−iNC
as in ICF. If the priors are uncorrelated,
using J−i is appropriate for a single time step. But, as the nodes converge, which is the
goal of consensus, the information becomes redundant at each node and thus dividing
the prior information matrices by NC is required to match the centralized solution.
3) In the information matrix update equation of the ICF, there is a multiplication factor
of NC . The total information in an estimate is the sum of the information matrices of
the priors and the measurements. However, as the average consensus scheme gives us
the average information in the priors and measurements, this should be multiplied by
NC to get the exact measure of the total information.
51
4) In the third term of Eqn. (3.44), KCF gives equal weight to all the neighbors’ priors.
In the presence naivety, this has detrimental effect as the information at different nodes
are different and need to be weighted by their information matrices.
5) In Eqn. (3.40), ICF uses
(1
NCA(J−i ) +A(Ui)
)−1
to normalize both the innovation
from the measurements and the innovation from the state priors. Whereas, in Eqn.
(3.44), the normalizing terms are not balanced because KCF uses(J−i + Bi
)−1to nor-
malize the measurement innovation and (J−i )−1 to normalize the innovation from the
priors.
6) In Eqn. (3.44), the normalizing term 1 + ||(J−i )−1|| is a design choice [17] to maintain
stability of the algorithm. However, it does not guarantee optimality of KCF.
3.6 Experimental Evaluation
In this section, we evaluate the performance of the proposed ICF algorithm in a
simulated environment and compare it with other methods: the Centralized Kalman Fil-
ter (CKF), the Kalman Consensus Filter (KCF) [17] and the Generalized Kalman Con-
sensus Filter (GKCF) [11]. Comparison in a simulated environment allows an in-depth
analysis of the proposed algorithms as parameters are varied to measure performance
under different conditions.
We simulate a camera network containing NT targets randomly moving (with
a fixed model) within a 500× 500 space. Each target’s initial state vector is random. A
set of NC camera sensors monitor the space (we consider that each communication node
consists of one camera). In each experiment, the cameras are randomly distributed in
the space with random orientations resulting in overlapping field-of-views (FOVs).
52
Target State Parameters
Each target was initialized at the center of the simulation grid. The target’s
state vector was a 4D vector, with the 2D position and 2D velocity components. The
initial speed was set to 2 units per time step and with a random direction uniformly
chosen from 0 to 2π. The targets evolved for 40 time steps using the target dynamical
model of Eqn. (3.29). Only the targets which remained in the simulation grid for the
40 time steps were considered. The process covariance Q is set to diag(10, 10, 1, 1).
For the target state-estimation model, the dynamical model of Eqn. (3.29)
was also used with the same Φ and Q defined above. The initial prior state x−i (1) and
prior covariance P−i (1) is set equal at each node. A diagonal matrix is used for P−i (1)
with the main diagonal elements as {100, 100, 10, 10}. The initial prior state x−i (1) is
generated by adding zero-mean Gaussian noise of covariance P−i (1) to the ground truth
state.
Sensor Parameters
A total of NC = 15 nodes were generated at uniformly chosen random locations
on the simulation area. The measurement vector length for each sensor is mi = 2. The
FOV was chosen to be equilateral triangles. We define the sensing range, SR, for each
sensor to be the height of this equilateral triangle. SR was chosen to be 300 units
for all the sensors. A sensor can have an observation of a target only if the ground
truth position of the target is within the sensor’s FOV and in that case, a measurement
zi was generated using the linear observation model Eqn. (3.1) with noise covariance
53
Ri = 100I2. The observation matrix Hi and state transition matrix Φ is given below.
Hi =
1 0 0 0
0 1 0 0
, Φ =
1 0 1 0
0 1 0 1
0 0 1 0
0 0 0 1
.
Network Topology Parameters
A circulant graph was chosen as the communication topology for the sensor
network to highlight the different issues associated with the sparsity of the communi-
cation network. The circulant graph is ∆-regular where the degree of each node is the
same, say ∆. The adjacency matrix of a circulant graph is a circulant matrix. So, we
denote ∆ to be the degree of the communication graph G. In the experiments, ∆ = 2
was used, unless stated otherwise. Note that, in this scenario, the maximum degree
∆max = ∆ because the degree of all the nodes are same.
Consensus Parameters
At step 3 of Algorithm 3, each sensor communicates its measurement infor-
mation to its neighbors iteratively. The maximum number of consensus iterations K
was set to 5 unless stated otherwise. The consensus speed parameter was chosen to be
ε = 0.65/∆max = 0.65/∆.
Experimental Description
The parameters that were varied in the experiments are, the maximum number
of consensus iterations K, communication bandwidth µ, computation unit τ , degree of
the communication network ∆, sensing range SR and the number of cameras NC . For
54
0 100 200 300 400 5000
100
200
300
400
500
Figure 3.3: In this image of an example simulation environment, the red arrows indicate the
locations and orientations of the cameras. The camera FOVs are shown in blue triangles. There
are 7 cameras in this example. The green dotted lines represent the network connectivity. Each
black arrows depict the actual trajectory of a target moving on the grid.
each experiment, only one parameter was varied while the others were kept constant. As
a measure of performance, we computed the estimation error, e, defined as the Euclidean
distance between the ground truth position and the estimated posterior position. An
example of the simulation framework is shown in Fig. 3.3.
For each experiment, 20 different simulation environments differing in camera
poses were randomly generated using the method discussed above. For each environ-
ment, 20 different randomly generated target tracks were used. Thus, for each experi-
ment, the estimation errors e, were averaged over 20 × 20 = 400 random scenarios, 40
time steps and over all sensors NC . The mean errors for different methods are shown in
the following graphs as the results of different experiments. In the graphs, each line (of
a unique color) corresponds to the mean error e for one estimation method.
The KCF algorithm as originally proposed in [17] uses a single consensus step
per measurement update. To compare it with ICF, which supports multiple iterations,
55
2 4 6 8 10 12 14 16 18 205
10
15
20
25
30
35
40
Number of Consensus Iterations, K
Mea
nEr
ror,
e
CKFKCFGKCFICF
Figure 3.4: ICF performance comparison varying K (equal priors)
we extend KCF for multiple iterations. For this, at each time step, the measurement
innovation component is set to zero for k > 1 and we consider only the new information
provided by the neighboring nodes’ estimates.
3.6.1 Experiment 1: Varying K
The objective of this experiment is to compare the performance of different
estimation algorithms for different K. Here, K was varied from 1 to 20 at increments
of 1. The other parameters were kept constant at their default values. The priors were
chosen to be equal.
The results of this experiment are shown in Fig. 3.4. The graph shows that for
K = 1, ICF performs much better than KCF and GKCF performs close to ICF. As, the
number of iterations K is increased, the mean error for ICF decreases and approaches
the performance of the centralized method at about K = 10. The main reason for this
difference in performance is that KCF and GKCF do not account for the redundancy
in the prior information across different nodes, but ICF does. In this simulation all the
initial priors were equal, thus the information in the priors were completely redundant.
KCF and GKCF by not taking into account redundancy in priors, gives more weight
to prior information and less weight to the measurement information than the optimal
56
100
150
200
250
300
2 4 6 8 10 12 14 16 18 205
10
15
20
25
30
35
Number of Consensus Iterations, K
Mea
nEr
ror,
eKCFGKCFICFCKF
Figure 3.5: Uncorrelated Priors
100
200
300
400
500
600
2 4 6 8 10 12 14 16 18 200
10
20
30
40
Number of Consensus Iterations, K
Mea
nEr
ror,
e
KCFGKCFICFCKF
Figure 3.6: Correlated Priors (ρ = .5)
weights.
ICF is guaranteed to converge to the optimal centralized estimated if the initial
priors are equal. However, to show the robustness of the ICF approach, we conducted the
same experiment with unequal and uncorrelated priors (Fig. 3.5) and with unequal and
correlated (with ρ = 0.5 correlation-coefficient between the priors across nodes) (Fig.
3.6) using Algorithm 3. The results show that ICF achieves near-optimal performance
even when the optimality constraints are not met. This is because ICF is a consensus
based approach and irrespective of the initial condition, after several time steps or
consensus iterations, the priors converge. ICF was proved to be optimal with converged
priors. Thus, after a few time steps it achieves near-optimal performance as the system
approaches the optimality conditions.
3.6.2 Experiment 2: Varying µ and τ
The objective of this experiment is to compare the performance of different
algorithms at different amounts of communication bandwidth and computational re-
sources. We define the bandwidth, µ, of each method to be the total number of
scalars sent at each consensus iteration from a node to each neighbor. As, covari-
57
1 1.5 2 2.5 3 3.5 4 4.5 50
20
40
60
80
100
120
140
Number of Consensus Iterations, K
Band
wid
thµ
KCF BWGKCF BWICF BW
Figure 3.7: Bandwidth µ vs. number
of consensus iterations K for different ap-
proaches.
1 1.5 2 2.5 3 3.5 4 4.5 555
60
65
70
75
80
85
90
95
100
Number of Consensus Iterations, K
Com
puta
tion
units
0.1
τ
KCF CUGKCF CUICF CU
Figure 3.8: Computation unit τ vs. num-
ber of consensus iterations K for different
approaches.
10 20 30 40 50 60 70 80 90
10
15
20
25
30
35
40
Bandwidth, µ
Mea
nEr
ror,
e
CKFKCFGKCFICF
Figure 3.9: Performance comparison be-
tween different approaches as a function of
the bandwidth µ, showing that the ICF
performs better than other distributed ap-
proaches for each µ.
600 700 800 900 1000 1100
10
15
20
25
30
35
40
Computation units, τ
Mea
nEr
ror,
e
CKFKCFGKCFICF
Figure 3.10: Performance comparison be-
tween different approaches as a function of
the computational unit τ , showing that the
ICF performs better than other distributed
approaches at each τ .
ance/information matrices are symmetric, only sending the upper/lower triangular por-
tion of the matrix suffices. Using standard convention, the approximate communication
data requirement µ and computation cycle requirement τ was computed for p = 4 and
mi = 2.
Figs. 3.7 and 3.8 show the bandwidth and computational requirements for
different algorithms for different numbers of consensus iterations K. For any given K,
58
ICF always requires half the bandwidth of GKCF. The KCF has lower communication
and computational requirements.
Figs. 3.9 and 3.10 show the performance achieved by each method for different
amounts of bandwidth and computational requirements. At any given bandwidth µ or
computation unit τ , the ICF performs better than the other distributed methods and
the performance approaches that of the CKF with increased resources.
3.6.3 Experiment 3: Varying ∆
The objective of this experiment is to compare the performance of different
approaches for different values of the degree ∆ (i.e., vary the sparsity of the network
from sparse connectivity to dense connectivity). For NC = 15, the maximum possible
degree can be ∆ = 14 at full mesh connectivity. In this experiment, ∆ was varied from
2 to 14 at increments of 2.
The results are shown in Fig 3.11, where the total number of consensus iter-
ations K was set to 5. It can be seen that the ICF performs better than the other
distributed algorithms at all ∆ and almost approaches the centralized performance at
∆ = 4. For full connectivity, i.e. ∆ = 14, where G is a complete graph, all the distributed
methods achieve centralized performance.
3.6.4 Experiment 4: Varying SR
The objective of this experiment is to compare the performance of different
approaches as a function of the sensor range SR (i.e., varying the area of coverage of
each sensor.)
Fig. 3.12 shows the performance of each approach as SR is varied and the
59
2 4 6 8 10 12 148
10
12
14
16
18
20
22
24
26
28
Degree of Communication Graph, ∆
Mea
nEr
ror,
e
CKFKCFGKCFICF
Figure 3.11: Performance comparison of
different approaches as a function of the de-
gree of communication graph, ∆. Increase
in network connectivity increases the per-
formance of all methods.
150 200 250 300 350 400 450 500 550 6000
10
20
30
40
50
60
Sensor Range, SR
Mea
nEr
ror,
e
CKFKCFGKCFICF
Figure 3.12: Performance comparison
of different approaches by varying sensor
range, SR. As the sensor range increases,
the network gets more measurement infor-
mation and has fewer naive nodes increas-
ing overall performance of all methods (in-
cluding centralized).
total number of iterations K was set to 5. It can be seen that ICF performed better
than the other distributed algorithms at each sensor range.
3.6.5 Experiment 5: Varying NC
The objective of this experiment is to compare the performance of different
approaches as a function of the total number of sensors, NC , as it was varied from 5 to
31 at increments of 2. Values less than 5 were not used because at such low number of
sensors, even the centralized algorithm cannot perform well due to the high number of
time instants at which the number of measurements is insufficient for the state vector
to be observable, due to the low percentage coverage of the environment.
Fig. 3.13 shows results for variable NC and fixed K = 20. As the number
of sensors is increased, the observability of the targets by the network increases, but
the number of naive nodes also increases. Due to the amount of information about
60
5 10 15 20 25 300
10
20
30
40
50
Number of Sensors, N C
Mea
nEr
ror,
eCKFKCFGKCFICF
Figure 3.13: Performance comparison of
different approaches by varying total num-
ber of sensors, NC .
2 4 6 8 10 12 14 16 18 20
10
15
20
25
30
35
Number of Consensus Iterations, K
Mea
nEr
ror,
e
CKFKCFGKCFICF
Figure 3.14: Performance comparison for
imbalanced communication graph.
the targets increasing, the performance of all the approaches (including centralized)
improves. For a small NC , all the distributed methods performed close to the centralized
method, because the number of naive nodes is small and the network distance from a
naive node to an informed node is small. As the number of nodes increases, the number
of consensus iterations required to reach consensus also increases. Thus we can see that
the performance of ICF deviates from the centralized performance for high number of
sensors. For all NC , ICF outperformed the alternative distributed methods.
3.6.6 Experiment 6: Arbitrary Communication Graph
In the previous experiments, for the ease of varying different parameters, we
assumed the communication graph to be balanced. To show that ICF is applicable for
any connected graph, we conduct experiments on random connected graphs for NC = 15.
Edges were added between five random pairs of nodes on the balanced graph which was
used on the previous examples. The result of this experiment is shown in Fig. 3.14. It
depicts the fact that as K →∞, for any random connected graph, ICF converges to the
centralized solution.
61
5 10 15 20 25 302.5
3
3.5
4
4.5
5
5.5
6
Number of Consensus Iterations, K
Mea
nEr
ror,
eCKFKCFGKCFICF
Figure 3.15: Performance comparison under
unlimited observability.
0.2 0.4 0.6 0.8 1 1.2 1.4 1.60
20
40
60
80
100
α
Mea
nEr
ror,
e
CKFKCFGKCFICF
Figure 3.16: Stability analysis.
3.6.7 Experiment 7: Full Observability
Limited local observability (i.e., naive nodes) was one of the motivations for
the derivation of the ICF algorithm. To show that ICF is generally applicable even in
scenarios with without locally unobservable states, in this experiment, we assume that
each node can observe all the targets at all times. From the results in Fig. 3.15 it can
be seen that still ICF outperforms the other methods for most cases and achieves the
optimal performance as K →∞.
3.6.8 Experiment 8: Stability Analysis
In this experiment, we experimentally verify the stability of the ICF approach.
In Sec. 2.3, we have mentioned that the average consensus algorithm is stable if the
consensus rate parameter ε is chosen between 0 and 1∆max
. The stability of the ICF
algorithm is directly related to the stability of the average consensus algorithm. We set
ε = α∆max
and vary α to perform the stability analysis. From the results in Fig. 3.16,
it can be seen that ICF is stable for 0 < α < 1 (i.e., 0 < ε < 1∆max
), which is the same
stability conditions for average consensus algorithm.
62
−30 −20 −10 0 10 20 304
6
8
10
12
14
16
18
20
22
Deviation from actual N C , ∆ N C
Mea
nEr
ror,
eCKFKCFGKCFICF
Figure 3.17: Robustness to inaccurate
knowledge of NC .
2 4 6 8 10 12 14 16 18 200
20
40
60
80
100
120
140
Number of Consensus Iterations, K
Mea
nEr
ror,
e
CKFKCFGKCFICF
Figure 3.18: Robustness to model assump-
tion error.
3.6.9 Experiment 9: Robustness to inaccurate knowledge of NC
Unlike CKF, KCF and GKCF; ICF requires the knowledge of the total number
of nodes NC . Total number of nodes NC , can be computed in a distributed framework
[8]. In case of node failure or inclusion of new nodes to the system, each agent might
have a wrong estimate of NC , say NC + ∆NC . In this experiment, to test the sensitivity
of ICF to the error in the value of NC , ∆NC is varied from −NC + 1 to NC . The actual
value of NC for this experiment is 31. Total number of iterations, K was set to 100.
The results are shown in Fig. 3.17. It shows that ICF is highly tolerant to discrepancies
between the actual NC and the estimated NC and performs better than other methods
for most cases.
3.6.10 Experiment 10: Robustness to non-linear state propagation
Finally, it is natural to ask whether the performance demonstrated in the
previous experiments was highly dependent on the fact that the estimator incorporated
the correct model.
In this final experiment, the ground truth tracks were generated for t = 1 to
63
40 using the following non-linear model of Eqn. (3.46).
x(t) = 200e−λ(t−1)
cos(ωt)
sin(ωt)
−λ cos(ωt)− ω cos(ωt)
−λ cos(ωt) + ω cos(ωt)
+
250
250
0
0
(3.46)
A total of 20 tracks were generated by varying ω = π80 to 4π
80 with increments of π80
and λ = 0.02 to 0.1 with increments of 0.02. The rest of the simulation settings were
kept unchanged (similar to that of Experiment 1). Thus, in the estimation step, the
linear dynamical model of Eqn. (3.29) was used. Fig. 3.18 shows the performance of
different algorithms for this simulation. Comparing with Fig. 3.4 it can be seen that the
performance of all the algorithms deteriorated (including CKF) as one would expect.
However, the relative performance of the different algorithms are similar. This shows
that the ICF and the performance comparison results are quite robust, even when the
assumed dynamical model does not match the true state propagation.
3.7 Conclusion
In this chapter we have presented a novel distributed state estimation frame-
work called the Information-weighted Consensus Filter (ICF) which is generally ap-
plicable to almost any connected sensor network, converges to the optimal centralized
estimate under reasonable conditions, does not require target hand-off protocols, and
requires computation and communication resource similar to or less than alternative al-
gorithms. The development of this algorithm was motivated by applications in camera
networks wherein the observability properties of the state by the nodes is distinct across
the nodes. We compared ICF with the state-of-the-art distributed estimation methods
64
such as KCF and GKCF, both theoretically and through simulations. Our simulation
results show that ICF outperforms these methods. The results also indicate that ICF is
robust to situations where optimality conditions are not met. In the next chapters we
will present novel algorithms incorporating data association schemes to ICF to handle
multiple targets.
65
Chapter 4
Multi-Target Information
Consensus
4.1 Introduction
In this chapter, our goal is to design a distributed multi-target tracking scheme
which is suited for sensors with limited field-of-view (FOV), like cameras. A distributed
multi-target tracking problem can be divided into three sub-problems, namely, dis-
tributed information fusion, data association (measurement to track association) and
dynamic state estimation.
The Kalman Consensus Filter (KCF) [17] is a popular distributed dynamic
state estimation framework based on the average consensus algorithm. KCF works well
in situations where each node gets a measurement of the target. However, we have
discussed in the previous chapters that due the issue with naivety, the performance of
KCF deteriorates when there are sensor with limited FOVs. In the previous chapter, the
Information-weighted Consensus Filter (ICF) was proposed which addresses this issue
with naivety and is also guaranteed to converge to the optimal centralized Kalman filter
66
estimate.
The above mentioned methods assume that there is a single target, or for
multiple targets, the measurement-to-track association is provided. For a multi-target
tracking problem, the data association and the tracking steps are highly inter-dependent.
The performance of tracking will affect the performance of data association and vice-
versa. Thus, an integrated distributed tracking and data association solution is required
where the uncertainty from the tracker can be incorporated in the data association
process and vice-versa. Among many single-sensor multi-target data association frame-
works, the Multiple Hypothesis Tracking (MHT) [23] and the Joint Probabilistic Data
Association Filter JPDAF [2] are two popular schemes. MHT usually achieves higher
accuracy at the cost of high computational load. On the other hand, JPDAF achieves
reasonable results at much lower computation cost. As distributed solutions are usually
applied to low-power wireless sensor networks where the computational and communi-
cation power is limited, the JPDAF scheme will be utilized in the proposed distributed
multi-target tracking framework.
The main contribution of this chapter is the tight integration of data asso-
ciation with state-of-the-art distributed single target tracking methods, taking special
care of the issue of naivety, and demonstration of its performance in the case of a camera
network. In Sec. 4.2 the problem formulation is provided, along with a review of differ-
ent consensus-based estimation methods. In Sec. 4.3, the JPDAF approach is reviewed
and extended to a multi-sensor framework. In Sec. 4.5, the Multi Target Information
Consensus (MTIC) tracker is proposed. Finally, in Sec. 4.6, the proposed method is
compared against others experimentally.
67
4.1.1 Related Work
The purely decentralized nature of the fusion algorithm differentiates it from
the majority of multi-camera tracking approaches in the computer vision literature. For
example, in [6], a centralized approach for tracking in a multi-camera setup was proposed
where the cameras were distributed spatially over a large area. In [4], an efficient target
hand-off scheme was proposed but no multi-camera information fusion was involved.
However, in this article, we deal with the distributed multi-target tracking problem
where there is no centralized server, the processing is distributed over all the camera
nodes and no target hand-off strategy is required. Various methods for distributed multi-
target tracking have been proposed in the sensor-networks literature. In [5], a solution
to the distributed data association problem was proposed by means of the message
passing algorithm based on graphical models in which iterative, parallel exchange of
information among the nodes viewing the same target was required. However, in our
proposed framework, no special communication pattern is assumed. In [21, 26, 28], the
distributed multi-target tracking schemes did not account for naivety or the presence of
cross-correlation between the estimates at different nodes. The method proposed herein
is based on the properties of the ICF [13] that was proposed in the previous chapter,
which deals with both these issues.
4.2 Problem Formulation
Consider a sensor network with NC sensors. There are no specific assumptions
on the overlap between the FOVs of the sensors. The communication in the network
can be represented using an undirected connected graph G = (C, E). The set C =
{C1, C2, . . . , CNC} contains the vertices of the graph and represents the sensor nodes.
68
The set E contains the edges of the graph which represents the available communication
channels between different nodes. The set of nodes having direct communication channel
with node Ci (sharing an edge with Ci) is represented by Ni. There are NT targets
({T 1, T 2, . . . TNT }) in the area viewed by the sensors. It is assumed that NC and NT is
known to each sensor.
The state of the jth target is represented by the vector xj ∈ Rp. For example,
for a tracking application in a camera network, xj might be a vector containing ground
plane position and velocity components. The state dynamics of target T j are modeled
as
xj(t+ 1) = Φxj(t) + γj(t). (4.1)
Here Φ ∈ Rp×p is the state transition matrix and the process noise γj(t) is modeled as
N (0,Qj).
At time t, each sensor Ci, depending on its FOV and the location of the targets,
gets li(t) measurements denoted as {zni }li(t)n=1. The sensors do not know a priori, which
measurement was generated from which target. Under the hypothesis that the obser-
vation zni is generated from T j , it is assumed that zni was generated by the following
observation model
zni = Hjixji + νji . (4.2)
Here, Hji ∈ Rm×p is the observation matrix for node Ci for T j . The noise νji ∈ Rm is
modeled as a zero mean Gaussian random variable with covariance Rji ∈ Rm×m.
Each node also maintains a prior/predicted state estimate xj−i (t) (and its co-
variance Pj−i (t)) for each target. The inverse of the state covariance matrix (informa-
tion/precision matrix) will be denoted as Jji = (Pji )−1. We assume that the initial prior
state estimate and information matrix is available to each node for each target upon its
69
detection. Our goal is to track each target at each node, i.e., find the state estimate for
each target at each node by using the prior and measurement information available in
the entire network in a distributed fashion. A critical step in this process is association
of measurements with targets, which is the topic of this chapter.
4.3 Joint Probabilistic Data Association Filter: Review
The KCF, GKCF and ICF algorithms discussed in the previous chapters as-
sume that the data association (which measurement belongs to which target) is known.
For a realistic multi-target state estimation problem, solving data association is itself
a challenging problem even in the centralized case. Here we briefly review the Joint
Probabilistic Data Association Filter (JPDAF) [2] algorithm which is the starting point
of the proposed multi-sensor multi-target distributed tracking algorithm.
The JPDAF is a single sensor algorithm, thus the sensor index i is unnecessary
and will be dropped. A double superscript is required for the hypothesis that measure-
ment zn is associated with target T j . At time t, the measurement innovation zjn and
the innovation covariance Sj of measurement zn for target T j is computed as,
zjn = zn −Hjxj− (4.3)
Sj = HjPj−HjT + Rj (4.4)
The probability that T j is the correct target to associate with zn is βjn and the prob-
ability that none of the measurements belong to T j is βj0. See [2] for details about
computing these probabilities.
The Kalman gain Kj , mean measurement yj and mean measurement innova-
70
tion yj for target T j are defined as
Kj = Pj−HjT (Sj)−1, (4.5)
yj =l∑
n=1
βjnzn, (4.6)
yj =
l∑n=1
βjnzjn = yj − (1− βj0)Hjxj−. (4.7)
The state and its covariance estimate for JPDAF is given as
xj+(t) = xj−(t) + Kjyj (4.8)
Pj+(t) = Pj−(t)− (1− βj0)KjSj(Kj)T
+ KjPj(Kj)T
(4.9)
where,
Pj =
(l∑
n=1
βjnzjn(zjn)T)− yj
(yj)T. (4.10)
4.4 Data Association: Information Form
In the following, we first express the JPDAF algorithm in the information form,
from which we will extend it to the multiple sensor case. This will then be used in the
next section to derive the distributed multi-target tracking algorithm.
The JPDAF estimation Eqns. (4.8-4.9) can be written in the information form
as the following (see Appendix B):
xj+ =(Jj− + Uj
)−1 (Jj−xj− + uj + βj0Ujxj−
)(4.11)
Jj+ = Jj− + Gj (4.12)
where,
Gj = Jj−Kj((
Cj)−1 −KjTJj−Kj
)−1KjTJj− (4.13)
Cj = (1− βj0)Sj − Pj (4.14)
uj = HjTRj−1yj and Uj = HjTRj−1
Hj . (4.15)
71
In Eqn. (4.11), Jj−xj− is the weighted prior information and uj + βj0Ujxj− is the
weighted measurement information (taking data association uncertainty βj0 into ac-
count). The sum of these two terms represents the total information available to us if
we have a single sensor. To incorporate measurement information from an additional
sensor, the weighted measurement information from that sensor has to be added to this
summation. This is a property of estimators in the information form for combining
measurements from multiple sensors, when noise in those measurements is uncorrelated
with each other, which we assume in this work. In a similar fashion, the information
matrices (Uji and Gj
i ) from additional sensors should also be added to the appropriate
terms. This gives us the multi-sensor centralized estimate in the information form as
the following:
xj+ =
(Jj− +
NC∑i=1
Uji
)−1(Jj−xj− +
NC∑i=1
(uji + βj0i Uj
i xj−))
, (4.16)
Jj+ = Jj− +
NC∑i=1
Gji . (4.17)
4.5 Multi-Target Information Consensus
Based on the data association results derived in the previous section and the
ICF, we will now derive a distributed multi-target tracking algorithm. We shall call
this as the Multi Target Information Consensus (MTIC) tracker. As we have multiple
sensors, we will bring back the sensor index in the subscripts.
Now, in a distributed system, each node will have its own prior information
{xj−i ,Jj−i }. However, consensus guarantees that the information at all nodes converge
to the same value. This is an important point that was utilized in the ICF framework
and similarly, it will be utilized here. Assuming that consensus was reached at the
72
Algorithm 4 MTIC for target T j at node Ci at time step t
Input: xj−i (t), Jj−
i (t), Hji , Rj
i .
1) Get measurements: {zni }
li(t)n=1
2) Compute Sji , yj
i , βj0i , Kj
i and Cji
3) Compute information vector and matrices:
uji ← HjT
i Rji
−1yji (4.18)
Uji ← HjT
i Rji
−1Hj
i (4.19)
Gji ← Jj−
i Kji
(Cj
i
−1 −Kji
TJj−i Kj
i
)−1
Kji
TJj−i (4.20)
4) Initialize consensus data
vji [0] ← uj
i +
(Jj−i
NC+ βj0
i Uji
)xj−i (4.21)
Vji [0] ← Jj−
i
NC+ Uj
i (4.22)
Wji [0] ← Jj−
i
NC+ Gj
i (4.23)
5) Perform average consensus (Sec. 2.3) on vji [0], Vj
i [0] and Wji [0] independently for K iterations.
6) Estimate:
xj+i ←
(Vj
i [K])−1
vji [K] (4.24)
Jj+i ← NCWj
i [K] (4.25)
7) Predict:
xj−i (t+ 1) ← Φxj+
i (t) (4.26)
Jj−i (t+ 1) ←
(Φ(Jj+i (t)
)−1
ΦT + Q
)−1
(4.27)
Output: xj+i (t), Jj+
i (t), xj−i (t+ 1), Jj−
i (t+ 1).
73
previous time step, the prior information at each node will be equal, i.e., Jj−i = Jj−c and
xj−i = xj−c ∀i, j. From this, we can rewrite Eqns. (4.16) and (4.17) as follows:
xj+i =
(NC∑i=1
(Jj−iNC
+ Uji
))−1 NC∑i=1
(uji +
(Jj−iNC
+ βj0i Uji
)xj−i
)
=
(NC∑i=1
Vji
)−1 NC∑i=1
vji =
(∑NCi=1 Vj
i
NC
)−1 ∑NCi=1 vjiNC
(4.28)
Jj+i =
NC∑i=1
(Jj−
NC+ Gj
i
)=
NC∑i=1
Wji = NC
∑NCi=1 Wj
i
NC(4.29)
where,
Vji =
Jj−iNC
+ Uji , Wj
i =Jj−iNC
+ Gji
and vji = uji +
(Jj−iNC
+ βj0i Uji
)xj−i (4.30)
The three averaging terms in Eqns. (4.28) and (4.29) can be computed in a distributed
manner using the average consensus algorithm [18]. The algorithm is summarized in
Algorithm 4.
Note that if a sensor does not get any measurement for T j , i.e., βj0i = 1, uji ,
Uji and Gj
i are set to zero vectors and matrices (due to no measurement information
content).
4.6 Experiments
In this section, first we evaluate the performance of the proposed MTIC al-
gorithm in a simulated environment and compare it with other methods: JPDA-KCF,
ICF with ground truth data association (ICF-GT) and centralized Kalman Filter with
ground truth data association (CKF-GT). ICF-GT converges to CKF-GT in several
iterations, thus ICF-GT will provide a performance bound for the other iterative ap-
proaches. Note that ICF-GT and CKF-GT requires the knowledge of the ground truth
74
data association, whereas MTIC and JPDA-KCF do not.
We simulate a camera network with NC = 15 cameras monitoring an area
containing NT = 3 targets roaming randomly in a 500 × 500 area. Each camera has a
rectangular FOV of 200×200 and they are randomly placed in such a way that together
they cover the entire area. A circulant network topology with a degree of 2 (at each
node) was chosen for the network connectivity. Each target was randomly initialized at
a different location with random velocity. The target’s state vector was a 4D vector,
with the 2D position and 2D velocity components. The targets evolved for 40 time steps
using the target dynamical model of Eqn. (4.1). The state transition matrix (used both
in track generation and estimation) Φ and process covariance Q were chosen as
Φ =
1 0 1 0
0 1 0 1
0 0 1 0
0 0 0 1
, Q =
10 0 0 0
0 10 0 0
0 0 1 0
0 0 0 1
.
The initial prior covariance Pj−i (1) = diag(100, 100, 10, 10) was used at each node for
each target. The initial prior state xj−i (1) was generated by adding zero-mean Gaussian
noise of covariance P−i (1) to initial the ground truth state. The observations were
generated using Eqn. (4.2). The observation matrix Hji was set as
Hji =
1 0 0 0
0 1 0 0
.If the ground truth state was within the FOV of a sensor, a measurement was generated
from the ground truth track using the measurement model Eqn. (4.2) with Ri = 100I2.
The consensus rate parameter ε was set to 0.65/∆max where ∆max = 2, as each node was
connected to two other nodes. Total number of consensus iterations per measurement
step, K, was set to 20. The parameters for computing the association probabilities,
75
βjni ’s, were set as follows (see [2] for details). False measurements (clutter) were gen-
erated at each node at each measurement step using a Poisson process with λ = 132 .
Here, λ is the average number of false measurements per sensor per measurement step.
Gate probability PG was set to 0.99. The probability of detecting a target in each cam-
era, PD was computed by integrating the probability density function of the predicted
measurement, (i.e., N (Hji xj−i ,Sji )) over the area visible to the camera.
To measure the performance of different approaches, one of the parameters
was varied while keeping the others to their aforementioned values. As a measure of
performance, we computed the estimation error, e, defined as the Euclidean distance
between the ground truth position and the estimated posterior position. The simulation
results were averaged over multiple simulation runs with 100 randomly generated sets
of tracks. The mean (µe) of the errors for different methods are shown in the following
graphs as the results of different experiments.
First, the amount of clutter was varied and the results are shown in Fig. 4.1.
The average amount of clutter per sensor per measurement step, λ, was varied from 1256
to 8. From the figure it can be seen that both MTIC and JPDA-KCF is very robust
even to a very high amount of clutter. The amount of clutter was kept at λ = 132 for
the other experiments.
Fig. 4.2a shows the performance of different approaches where the proximity of
the tracks were varied. The proximity was defined in terms of average number of overlaps
across all pairs of tracks present in a simulation run. Two tracks were assumed to be
overlapping if the Euclidean distance between their ground truth states was below 50
units at the same time step. From Fig. 4.2a, it can be seen that as the overlap increases,
the performance of different approaches deteriorated. However, MTIC performs better
than JPDA-KCF. It can be seen that for a high overlap, the error did not increase.
76
This is mainly due to the reason that most of the tracks with high overlap were close to
each other after they separated. Thus, although the data association failed, the tracking
error did not grow much.
1/256 1/128 1/64 1/32 1/16 1/8 1/4 1/2 1 2 4 80
10
20
30
40
50
60
70
80
90
100
λ
Mea
n Er
ror,
µ e
MTICJPDA−KCFICF−GT
Figure 4.1: Performance comparison by varying amount of clutter.
To show the convergence of the different methods, the total number of iterations
per measurement step, K was varied. It can be seen from Fig. 4.2b that with an
increased number of iteration, ICF-GT approached the centralized method CKF-GT. It
can also be seen that MTIC outperforms JPDA-KCF for any given K.
Next, the total number of sensors NC and total number of targets NT were
varied and the results are shown in Figs. 4.2c and 4.2d. With more sensors, the total
number of available measurements increases which should increase estimation perfor-
mance. However, with an increase in the number of sensors, the total number of false
measurements also increases which can adversely affect the performance. Due to these
two contradictory issues, the performance remained almost constant with different num-
ber of sensors. With the increase in the number of targets, the problem of data associ-
ation became more challenging which had an adverse effect in the performance of the
77
different algorithms as can be seen in Fig. 4.2d.
1 2 3 4 5 6 7 8 9 100
10
20
30
40
50
60
70
80
Average overlap
Mea
n Er
ror,
µ e
MTICJPDA−KCFICF−GT
(a) Varying proximity of tracks
2 4 6 8 10 12 14 16 18 200
10
20
30
40
50
60
70
80
90
K
Mea
n Er
ror,
µ e
MTICJPDAFJPDA−KCFICF−GTCKF−GT
(b) Varying K
10 12 14 16 18 20 22 24 26 28 300
10
20
30
40
50
60
70
80
90
NC
Mea
n Er
ror,
µ e
MTICJPDA−KCFICF−GT
(c) Varying NC
2 4 6 8 10 12 140
20
40
60
80
100
120
NT
Mea
n Er
ror,
µ eMTICJPDA−KCFICF−GT
(d) Varying NT
Figure 4.2: MTIC Performance comparison by varying different parameters.
4.7 Conclusion
In this chapter, we have proposed the Multi Target Information Consensus
(MTIC) algorithm, which is a generalized consensus-based distributed multi-target track-
ing scheme applicable to a wide-variety of sensor networks. MTIC handles the issues
with naivety which makes it applicable to sensor networks where the sensors may have
limited FOV (which is the case for a camera network). The estimation errors in tracking
and data association, as well as the effect of naivety, are integrated into a single efficient
algorithm. This makes MTIC very robust to false measurements/clutter. Experimental
78
analysis shows the strength of the proposed method over existing ones. In the next
chapter we will extend this algorithm to handle non-linearity in the observation model.
79
Chapter 5
Information-weighted Consensus
with non-linear models
5.1 Introduction
In the previous chapters, we assumed a linear relationship between the observa-
tion and the target state. In many application scenarios, the observation is a non-linear
function of the target state. This is especially true for cameras because in the perspec-
tive camera observation model, the position of a point in the camera’s pixel coordinate
system is non-linearly related to the position of the point in the world coordinate system.
Thus, for the algorithms derived in the previous sections to be applicable in
such scenarios, we need to re-derive them to support non-linearity in the observation
model. In this chapter, we propose non-linear extensions to the ICF and the MTIC
algorithms proposed in the previous chapters. We call these algorithms, the Extended
ICF (EICF) and the Extended MTIC (EMTIC). We will also show comparisons of these
in a simulated framework along with real-life experiments.
80
5.2 Camera Observation Model (Non-linear )
Let us denote the orientation and position of the camera Ci w.r.t. the world
coordinate system as Riw and piw . The number of pixels per unit length is k and fi is
the focal length of Ci. The center of the image plane in the pixel coordinate system
is {ox, oy}. Say, in world and camera coordinate the system, the position of a point
j is pjw = [xw, yw, zw]T and pji = [xi, yi, zi]T respectively. Note that for ground plane
tracking, zw = 0. The target state can be xj = [xw, yw, uw, vw]T i.e., it may consist of
the position and velocity components of a target in the ground plane. Now, pjw and xj
are related as,
pjw =
1 0 0 0
0 1 0 0
0 0 0 0
0 0 0 0
xj (5.1)
From pjw , we can compute pji as,
pji = Riw ( pjw − piw ) (5.2)
Finally, from From pji the position of that point in Ci’s pixel coordinate system zji can
be written using the camera perspective projection model as,
zji =
kfixizi
+ ox
kfiyizi
+ oy
(5.3)
Using the above three equations, we can see that zji is a non-linear function of xj which
can be represented as zji = hi(xj). Using first-order Taylor series approximation, the
linearized observation matrix can be written as,
Hji = ∇xjhi(x
j)|xj=xj−
i(5.4)
Next, we will extend the ICF and MTIC algorithm for non-linear observation models
where both Hji and hi() will be utilized.
81
5.3 Extended Information-weighted Consensus Filter
In this section, we will derive the Extended Information Consensus Filter
(EICF) which we will prove to converge to the centralized Extended Kalman Filter
(EKF) results. Considering the data association is given, the measurement in each
sensor can be expressed using the non-linear relation as,
zi = hi(x) + νi. (5.5)
The collection of all measurements from all sensors can be expressed as,
Z = hc(x) + ν. (5.6)
Here, Z = [zT1 , zT2 , . . . z
TN ]T ∈ Rm is the concatenation of all measurements in the net-
work where m =∑N
i=1mi. The central non-linear observation function hc() is a stack of
all the functions from individual sensors such that hc(x) = {h1(x)T ,h2(x)T , . . .hNC(x)T }T .
We will represent the stack of linearized observation matrices as H = [HT1 ,H
T2 , . . .H
TN ]T ∈
Rm×p. For the measurement noise vector, ν = [νT1 ,νT2 , . . .ν
TN ]T ∈ Rm, we denote
its covariance as R ∈ Rm×m. We assume the measurement noise to be uncorrelated
across nodes. Thus, the measurement covariance matrix is R = diag(R1,R2, . . .RN ).
Let us denote Uc = HTR−1H and uc = HTR−1Z. The subscript c stands for
“centralized”. Let us also denote the centralized predicted measurement, hc(x−c ) =
{h1(x−c )T ,h2(x−c )T , . . .hNC(x−c )T }T .
The state estimation equations from the centralized Extended Kalman Filter
(EKF) algorithm can be written as (see Appendix A),
x+c =
(J−c + Uc
)−1 (J−c x−c + uc + HTR−1(Hx−c − hc(x
−c )))
(5.7)
J+c =
(J−c + Uc
)(5.8)
82
Given that all the nodes have reached consensus on the previous time step,
we have, x−i = x−c and J−i = J−c for all i. This implies, J−c =
NC∑i=1
J−iNC
and J−c x−c =
NC∑i=1
J−iNC
x−i . Also, as Rc is block diagonal, we have, Uc =
NC∑i=1
Ui, uc =
NC∑i=1
ui and
HTR−1(Hx−c − hc(x−c )) =
NC∑i=1
HTi R−1
i (Hix−i − hi(x
−i )).
Thus, from Eqns. (5.7) and (5.8) we get,
x+c =
(NC∑i=1
(J−iNC
+ Ui
))−1 NC∑i=1
(J−i x−i + ui + HT
i R−1i
(Hixi
− − hi(x−i )))
(5.9)
J+c =
NC∑i=1
(J−iNC
+ Ui
)(5.10)
We will call the above equations the Extended Joint Probabilistic Data Asso-
ciation Filter (EJPDAF). This can be computed in a distributed manner by initializing
vi[0] and Vi[0] as the following at each node and running average consensus algorithm
on them.
vi[0] =J−iNC
x−i + ui + HTi R−1
i (Hix−i − hi(x
−i )) (5.11)
Vi[0] =J−iNC
+ Ui (5.12)
Note that for linear observation model, hi(x−i ) = Hix
−i . Using this relation,
the EICF equations reduce to the original ICF (derived for linear model) equations as
the following,
vi[0] =J−iNC
x−i + ui (5.13)
Vi[0] =J−iNC
+ Ui (5.14)
83
Algorithm 5 EICF at node Ci at time step t
Input: prior state estimate x−i (t), prior information matrix J−i (t), consensus speed factor ε and total
number of consensus iterations K.
1) Linearize hi at x−i (t) to compute Hi
2) Get measurement zi with covariance Ri
3) Compute consensus proposals,
Vi[0] ← 1
NCJ−i (t) + Ui (5.15)
vi[0] ← 1
NCJ−i (t)x−i (t) + ui + HT
i R−1i (Hix
−i − hi(x
−i )) (5.16)
4) Perform average consensus (Sec. 2.3) on vi[0], Vi[0] independently for K iterations.
5) Compute a posteriori state estimate and information matrix for time t
x+i (t) ← (Vi[K])−1vi[K] (5.17)
J+i (t) ← NCVi[K] (5.18)
6) Predict for next time step (t+ 1)
J−i (t+ 1) ←(Φ(J+
i (t))−1ΦT + Q)−1
(5.19)
x−i (t+ 1) ← Φx+i (t) (5.20)
Output: EICF estimate x+i (t), J+
i (t), x−i (t+ 1), J−i (t+ 1).
The EICF algorithm is summarized in Algorithm 5. To highlight the simi-
lar type of difference between ICF and EICF; the centralized Kalman filter and EKF
estimate (see appendix) in information form is given in below,
Kalman filter:
x+c =
(J−c + Uc
)−1 (J−c x−c + uc
)(5.21)
84
EKF:
x+c =
(J−c + Uc
)−1 (J−c x−c + uc + HTR−1
(Hx−c − hc
(x−c)))
(5.22)
Note that the EKF has the additional term HTR−1 (Hx−c − hc (x−c )) which is due to
the non-linearity in the observation model. The same difference is observed in ICF Eqn.
(3.33) and EICF Eqn. (5.11).
5.4 Extended Multi-target Information Consensus
In this section, we will extend the MTIC algorithm to handle non-linear sensing
models. For the linear case, the measurement innovation in the JPDAF algorithm is as
the following,
yj = yj − (1− βj0)Hjxj− (5.23)
However, for the non-linear sensing model, h(), the measurement innovation term be-
comes,
yj = yj − (1− βj0)h(xj−) (5.24)
Using this, the state estimation equations can be written as the following (see
Appendix C)
xj+c =
(Jj−c +
NC∑i=1
Uji
)−1
(Jj−i xj−i +
NC∑i=1
(uji + Hj
i
TRji
−1(Hj
i xj−i − (1− βj0i )hi(x
j−i ))
))(5.25)
Jj+c = Jj−i +
NC∑i=1
Gji (5.26)
85
Given that all the nodes have reached consensus on the previous time step, we
have, xj−i = xj−c and Jj−i = Jj−c for all i. This implies, Jj−c =
NC∑i=1
Jj−iNC
and Jj−c xj−c =
NC∑i=1
Jj−iNC
xj−i . Thus we can write,
xj+c =
(NC∑i=1
(Jj−iNC
+ Uji
))−1
NC∑i=1
(Jj−iNC
xj−i + uji + Hji
TRji
−1(Hj
i xj−i − (1− βj0i )hi(x
j−i ))
)(5.27)
Jj+c =
NC∑i=1
(Jj−iNC
+ Gji
)(5.28)
Thus, for EMTIC, the consensus variables are initialized as,
vji [0] =Jj−iNC
xj−i + uji + Hji
TRji
−1(Hji xj−i − (1− βj0i )hi(x
j−i ))
(5.29)
Vji [0] =
Jj−iNC
+ Uji (5.30)
Wji [0] =
Jj−iNC
+ Gji (5.31)
Note that for linear observation model, hi(xj−i ) = Hj
i xj−i . Using this relation,
the non-linear MTIC equations become the original MTIC (derived for linear model)
equations,
vji [0] =Jj−iNC
xj−i + uji + βj0i Uji xj−i (5.32)
Vji [0] =
Jj−iNC
+ Uji (5.33)
Wji [0] =
Jj−iNC
+ Gji (5.34)
The EMTIC algorithm is summarized is Algorithm 6.
86
Algorithm 6 EMTIC for target T j at node Ci at time step t
Input: xj−i (t), Jj−
i (t), hi, Rji .
1) Linearize hi at xj−i (t) to compute Hj
i
2) Get measurements: {zni }
li(t)n=1
3) Compute Sji , yj
i , βj0i , Kj
i and Cji
4) Compute information vector and matrices:
uji ← HjT
i Rji
−1yji (5.35)
Uji ← HjT
i Rji
−1Hj
i (5.36)
Gji ← Jj−
i Kji
(Cj
i
−1 −Kji
TJj−i Kj
i
)−1
Kji
TJj−i (5.37)
5) Initialize consensus data
vji [0] ← Jj−
i
NCxj−i + uj
i + Hji
TRj
i
−1(Hj
i xj−i − (1− βj0
i )hi(xj−i ))
(5.38)
Vji [0] ← Jj−
i
NC+ Uj
i (5.39)
Wji [0] ← Jj−
i
NC+ Gj
i (5.40)
6) Perform average consensus (Sec. 2.3) on vji [0], Vj
i [0] and Wji [0] independently for K iterations.
7) Estimate:
xj+i ←
(Vj
i [K])−1
vji [K] (5.41)
Jj+i ← NCWj
i [K] (5.42)
8) Predict:
xj−i (t+ 1) ← Φxj+
i (t) (5.43)
Jj−i (t+ 1) ←
(Φ(Jj+i (t)
)−1
ΦT + Qj
)−1
(5.44)
Output: xj+i (t), Jj+
i (t), xj−i (t+ 1), Jj−
i (t+ 1).
87
5.5 Comparison of KCF, ICF, MTIC, EKCF, EICF and
EMTIC
We now compare the state estimation equations of KCF, ICF, MTIC, EKCF,
EICF and EMTIC for one particular target and a single consensus iteration step. The
derivation of these particular forms are shown in Appendix D.
KCF: (see [17])
x+i = x−i +
(J−i + Bi
)−1 (bi −Bix
−i
)+
ε
1 + ||(J−i )−1||(J−i )−1
∑i′∈Ni
(x−i′ − x−i
)(5.45)
J+i = J−i + Bi (5.46)
ICF: (see Proposition 1 in Appendix D)
x+i = x−i +
(A(
J−iNC
)+A(Ui)
)−1
(A(ui)−A(Ui)x
−i + ε
∑i′∈Ni
J−i′
NC
(x−i′ − x−i
) )(5.47)
J+i = NC
(A(
J−iNC
)+A(Ui)
)(5.48)
MTIC: (see Proposition 2 in Appendix D)
x+i = x−i +
(A(
J−iNC
) +A(Ui)
)−1 (A(ui)−A(Ui)x
−i
+A(Uix−i − (1− βi0)Uix
−i ) + ε
∑i′∈Ni
J−i′
NC
(x−i′ − x−i
) )(5.49)
J+i = NC
(A(
J−iNC
)+A(Gi)
)(5.50)
88
EKCF: (see [25])
x+i = x−i +
(J−i + Bi
)−1(bi −HT
i R−1i hi(x
−i ) + γ
∑i′∈Ni
(x−i′ − x−i
))(5.51)
J+i = J−i + Bi (5.52)
where γ =ε
1 + ||(J−i + Bi)−1||
EICF: (see Proposition 3 in Appendix D)
x+i = x−i +
(A(
J−iNC
)+A(Ui)
)−1(A(ui)−A(Ui)x
−i
+A(Uix−i −HT
i R−1i hi(x
−i )) + ε
∑i′∈Ni
J−i′
NC
(x−i′ − x−i
))(5.53)
J+i = NC
(A(
J−iNC
)+A(Ui)
)(5.54)
EMTIC: (see Proposition 4 in Appendix D)
x+i = x−i +
(A(
J−iNC
)+A(Ui)
)−1(A(ui)−A(Ui)x
−i
+A(Uix
−i − (1− β0
i )HTi R−1
i hi(x−i ))
+ ε∑i′∈Ni
J−i′
NC
(x−i′ − x−i
))(5.55)
J+i = NC
(A(
J−iNC
)+A(Gi)
)(5.56)
Note that the differences in the prior states between the neighboring nodes,
x−i′ − x−i are weighted by the corresponding neighbor’s prior information matrix J−i′
in ICF, MTIC, EICF and EMTIC. This handles the issue with naivety as the innova-
tion from a naive neighbor’s prior state will be given less weight. However, in KCF
and EKCF, the innovation from each neighbor’s prior is given equal weight which may
deteriorate the performance of KCF and EKCF in the presence of naive nodes.
The term ui, in Eqns. (5.47) and (5.49) are not exactly the same, as ICF
assumes perfect data association and computes ui from the appropriate measurement
89
zji . Whereas, in MTIC, ui is computed from the mean measurement yji . The same
argument holds for EICF Eqn. (5.53) and EMTIC Eqn. (5.55).
In MTIC Eqn. (5.49), the A(Uix−i −(1−βi0)Uix
−i ) term comes due to the data
association error. In EICF Eqn. (5.53) the A(Uix−i −HT
i R−1i hi(x
−i )) term comes from
non-linearity in the observation model. In a similar fashion, in EMTIC Eqn. (5.55), the
A(Uix
−i − (1− β0
i )HTi R−1
i hi(x−i ))
term is a result of both data association error and
non-linearity in the observation model. It can be seen that, with perfect data association
and linear observation model, this term equals to zero as it is not present in the ICF
Eqn. (5.47) equation.
The information matrix update equations, i.e., Eqns. (5.48) and (5.50), are
different for ICF and MTIC as the data association uncertainty is incorporated in Gi
for MTIC. This shows the tight integration of the data association and tracking steps
in MTIC, as the uncertainty of one step is considered in the other. The same argument
holds for EICF Eqn. (5.54) and EMTIC Eqn. (5.56).
5.6 Experiments
In this section, first we evaluate the performance of the proposed EICF algo-
rithm and compare it with the Extended Kalman Consensus Filter (EKCF) (distributed)
[25] and the Extended Kalman Filter (EKF) (centralized) algorithms. Next, we evaluate
the performance of the proposed EMTIC algorithm and compare it with the Extended
Joint Probabilistic Data Association Filter (EJPDAF) (centralized) (see Eqns. (5.9)-
(5.10)), the EICF (with ground truth data association) algorithm and the EKF (with
ground truth data association) algorithm. Finally we provide real-life experiments to
show the performance of the EMTIC algorithm and compare it with the EICF (with
90
−200−100
0100
200300
400500
600700
−200
−100
0
100
200
300
400
500
600
700
0
50
100
C3
C4
C5
C2
C6
C1
C8
C7
Figure 5.1: EICF and EMTIC Simulation setup
ground truth data association) algorithm.
5.6.1 Simulation Experiments
We simulate a camera network with NC = 8 cameras monitoring an area with
targets randomly roaming in its 500 × 500 area. The simulation setup is shown in
Fig. 5.1. Each of the cameras can view approximately 20% of the entire area with
some overlap with other cameras. The FOV of three cameras are shown in the figure.
Together they cover the entire area. A circulant network topology with a degree of
2 (at each node) was chosen for the network connectivity. Each target was randomly
initialized at a different location with random velocity. The target’s state vector was
a 4D vector, with the 2D position and 2D velocity components. The targets evolved
for 40 time steps using the target dynamical model of Eqn. (4.1). The state transition
matrix (used both in track generation and estimation) Φ and process covariance Q were
91
chosen as
Φ =
1 0 1 0
0 1 0 1
0 0 1 0
0 0 0 1
, Q =
10 0 0 0
0 10 0 0
0 0 1 0
0 0 0 1
.
The initial prior covariance Pj−i (1) = diag(100, 100, 10, 10) was used at each node for
each target. The initial prior state xj−i (1) was generated by adding zero-mean Gaussian
noise of covariance P−i (1) to initial the ground truth state. The observations were
generated using Eqn. (5.5). The linearized observation matrix Hji was computed for
each target at each camera at each time step using the prior state estimate xji (t) in Eqn.
(5.4).
If the ground truth state was within the FOV of a sensor, a measurement was
generated from the ground truth track using the measurement model Eqn. (5.5) with
Ri = 100I2. The consensus rate parameter ε was set to 0.65/∆max where ∆max = 2,
as each node was connected to two other nodes. Total number of consensus iterations
per measurement step, K, was varied in the different simulations. The parameters for
computing the association probabilities, βjni ’s, were set as follows (see [2] for details).
False measurements (clutter) were generated at each node at each measurement step
using a Poisson process with λ = 132 . Here, λ is the average number of false measure-
ments per sensor per measurement step. Gate probability PG was set to 0.99. The
probability of detecting a target in each camera, PD was computed by integrating the
probability density function of the predicted measurement, (i.e., N (hi(xj−i ),Sji )) over
the area visible to the camera.
As a measure of performance, we computed the estimation error, e, defined as
the Euclidean distance between the ground truth position and the estimated posterior
92
2 4 6 8 10 12 14 16 18 203
4
5
6
7
8
9
K
Mea
n Er
ror,
µ e
EKFEKCFEICF
Figure 5.2: EICF performance comparison varying K
position. The simulation results were averaged over multiple simulation runs with 100
randomly generated sets of tracks. The mean (µe) of the errors for different methods
are shown in the following graphs as the results of different experiments.
The performance of the EICF algorithm is shown in Fig. 5.2. Here, the total
number of iterations, K was varied from 1 to 20. The simulation was run on 100 ran-
domly generated tracks. It can be seen that EICF performed better than EKCF for any
K. The performance of the EICF algorithm also matched the centralized performance
(i.e., of EKF) as K was increased. The reason that the performance of EKCF was
deteriorated was mainly due to the naivety issue as discussed earlier.
Next, the performance of the EMTIC algorithm is shown in Fig. 5.3. Here the
total number of iterations K, was varied from 1 to 10. It was run over 100 randomly
generated sets of tracks. At each time a set of 3 tracks were generated. As shown
theoretically in this chapter, as K increased, the performance of the EMTIC algorithm
matched its centralized counter-part i.e., the EJPDAF algorithm’s performance. In the
93
1 2 3 4 5 6 7 8 9 100
2
4
6
8
10
12
K
Mea
n Er
ror,
µ e
EJPDAFEMTICEICFGTEKFGT
Figure 5.3: EMTIC performance comparison varying K
same figure, we also show the performance of the EICF and EKF algorithms where the
ground truth data association was provided. It can be seen that the EMTIC algorithm
compares well with the other algorithms, where the other algorithm (EJPDAF, EICF,
EKF) either are centralized or require the knowledge of the data association where
the EMTIC solves both the tracking and data association in a completely distributed
manner.
5.6.2 Real-life Experiments
Next, we show the performance of the EMTIC algorithm on real-life data. Six
cameras with partially overlapping FOV were chosen for the experiment. The FOV
of the cameras are shown in Fig. 5.4. As the cameras were not programmable, the
algorithms were run on a post-processing basis on a collected data-set. The camera
network topology was simulated in the experiment. The topology used in the experiment
is shown in Fig. 5.4. The cameras were calibrated w.r.t. the ground plane. There were
4 persons roaming in the area. A parts-based (utilizing PHOG features) person detector
94
C1
C5
C3C2 C4
C6
Figure 5.4: In this figure, there are six sensing nodes, C1, C2, . . . , C6 observing an area (black
rectangle) consisting of four targets. The solid blue lines show the communication channels
between different nodes. This figure also depicts the presence of “naive” nodes. For example,
C3, C5, C6 get direct measurements about the black target which it shares with its immediate
network neighbors. However, C1 does not have direct access to measurements of that target and
thus is naive w.r.t. that target’s state.
95
was used to detect the persons in each frame. Figs. 5.5 and 5.6 shows a snapshot of the
real-life experiment (after 145 time-steps).
In Fig. 5.5, the detections are shown in magenta bounding boxes. The center
point of the bottom of the bounding boxes were used as observations. There are many
false measurements and also missing detections.
Fig. 5.6 shows the ground plane tracking results in C1 for the four targets. The
total number of consensus iterations, K was set to 2. In Fig. 5.6a the tracking results of
the EMTIC algorithm is shown. In Fig. 5.6b, the tracking results of the EICF algorithm
using the ground truth data association and in Fig. 5.6c, the tracking results of the EKF
algorithm using ground truth data association information are shown. It can be seen
that the EMTIC algorithm performed well compared to the EICF and EKF algorithm.
It can be seen that the EMTIC tracking results are excellent although as opposed to
the other methods, it is not a centralized solution or it does not require ground truth
data association information. The ground plane tracks of the EMTIC algorithm are also
projected and plotted on each camera’s image plane for viewing purpose in Fig 5.5. In
Fig. 5.7, the error from the EKF estimates for the EMTIC and the EICF algorithms
are shown. It can be seen that the errors are very close to the EICF algorithm for which
the ground truth data association was provided.
5.7 Conclusion
In this chapter we have extended the ICF and the MTIC algorithms to handle
non-linearity in the observation model. In this chapter, we also theoretically compared
all the algorithms derived in this article. Next, we showed simulation results of the EICF
96
(a) C1 (b) C2
(c) C3 (d) C4
(e) C5 (f) C6
Figure 5.5: EMTIC Real-life experiments.
and EMTIC algorithm and compared them with other algorithms. We also showed real-
life applications of the EICF and EMTIC algorithm.
97
−300 300−300
300
(a) EMTIC tracking results
−300 300−300
300
(b) EICF-GT tracking results
−300 300−300
300
(c) EKF tracking results
Figure 5.6: EMTIC, EICF (with ground truth data association) and EKF (with ground truth
data association) tracking results in the ground plane.
0 50 100 1500
2
4
6
8
10
12
14
16
time−step, t
Mea
n er
ror f
rom
EKF
est
imat
es,
µ e
EMTICEICF−GT
Figure 5.7: Comparing mean error from EKF
98
Chapter 6
Conclusions and Future Work
In this thesis, we have proposed novel algorithms for distributed state estima-
tion in sensor networks. The core idea is in the proper information-weighting of the
states which enables the very simple and well-understood distributed averaging algo-
rithm known as the Average consensus algorithm to carry out complex state estimation
and data association tasks in a sensor network. The contributions can be broken down
in the following manner.
Generalized Kalman Consensus Filter
In chapter 2, we introduced a novel method for distributed state estimation
algorithm, the Generalized Kalman Consensus Filter (GKCF). We discussed under what
circumstances the assumptions of KCF are not valid and hence modifications are nec-
essary. This is especially true in camera networks where each sensor has a limited FOV
and they are geographically separated by distances that do not allow full communica-
tion. Then we proposed a generalized framework, Generalized KCF, which outperformed
the KCF approach under such conditions. We showed the theoretical derivation of our
framework and also showed simulation results to compare the performance of our algo-
99
rithm with other approaches.
Information-weighted Consensus Filter
In chapter 3, we presented a novel distributed state estimation framework
called the Information-weighted Consensus Filter (ICF) which is generally applicable
to almost any connected sensor network, converges to the optimal centralized estimate
under reasonable conditions, does not require target hand-off protocols, and requires
computation and communication resource similar to or less than alternative algorithms.
The development of this algorithm was motivated by applications in camera networks
wherein the observability properties of the state by the nodes is distinct across the
nodes. We compared ICF with the state-of-the-art distributed estimation methods such
as KCF and GKCF, both theoretically and through simulations. Our simulation results
show that ICF outperforms these methods. The results also indicate that ICF is robust
to situations where optimality conditions are not met.
Multi-Target Information Consensus
In chapter 4, we proposed the Multi Target Information Consensus (MTIC) al-
gorithm, which is a generalized consensus-based distributed multi-target tracking scheme
applicable to a wide-variety of sensor networks. MTIC handles the issues with naivety
which makes it applicable to sensor networks where the sensors may have limited FOV
(which is the case for a camera network). The estimation errors in tracking and data as-
sociation, as well as the effect of naivety, are integrated into a single efficient algorithm.
This makes MTIC very robust to false measurements/clutter. Experimental analysis
shows the strength of the proposed method over existing ones.
100
Extension to Non-linear Models
In chapter 5, we have extended the ICF and the MTIC algorithms to handle
non-linearity in the observation model of the camera. In that chapter, we also theoret-
ically compared all the algorithms derived in this article. Next, we showed simulation
results of the EICF and EMTIC algorithm and compared them with other algorithms.
We also showed real-life applications of the EICF and EMTIC algorithm.
6.1 Future Work
Some of the possible future directions are discussed below.
Generalization to non-Gaussian Bayesian Estimation
The proposed algorithms currently are based on the first and second order
statistics of the states by assuming that the noise and estimation errors follow Gaussian
distribution. It would be interesting to derive the algorithms to be applicable to non-
Gaussian Bayesian estimation frameworks. This will enable the utilization of a particle-
filter-based framework without requiring an assumption on the statistical behavior of
the target or the observation.
Utilization of Communication and Vision Graph Information
In many situations, one might have the knowledge of the communication and/or
the vision graph of sensor network. It will be interesting to derive approaches which can
utilize that information in the proposed algorithms to minimize the communication and
computation resource consumption. That will be very useful for sensor networks with
low-powered devices.
101
6.1.1 Mobile Platforms
Extending the proposed algorithms to perform distributed tracking and path
planning in a network of mobile platforms would be another really interesting prob-
lem. The proposed distributed tracking algorithms work under the assumption that the
camera calibration is available. For dynamic cameras, one might need to dynamically
update the calibration information.
6.1.2 Hardware Implementation
In the real-life experiments presented in this article, real-life data was collected
and processed offline where the communication network was simulated. Implementing
the algorithms on actual smart cameras while using actual camera network for communi-
cation would be another interesting research problem. Issues related to communication
such as packet loss, latency, changes in network topology will pose interesting additional
challenges to the problem.
All the algorithms discussed in this thesis, at their current state require the
knowledge of the total number of targets and a reasonable initialization and termination
of the tracks (in case of the arrival or departure of targets from the scene). Fusion of a
distributed approach to perform these tasks automatically could be another interesting
problem from an implementation perspective.
102
Bibliography
[1] M. Alighanbari and J. P. How. An unbiased Kalman consensus algorithm. InAmerican Control Conference, 2006. 16
[2] Y. Bar-Shalom, F. Daum, and J. Huang. The probabilistic data association filter.IEEE Control Systems, 29(6):82 –100, Dec. 2009. 6, 67, 70, 76, 92
[3] D. Borra, E. Lovisari, R. Carli, F. Fagnani, and S. Zampieri. Autonomous calibra-tion algorithms for networks of cameras. In American Control Conference, pages5126 –5131, June 2012. 36
[4] Q. Cai and J. K. Aggarwal. Tracking human motion in structured environmentsusing a distributed-camera system. IEEE Trans. Pattern Anal. Mach. Intell.,21(11):1241–1247, Nov. 1999. 7, 68
[5] M. Cetin, L. Chen, J. W. F. Iii, E. T. Ihler, O. L. Moses, M. J. Wainwright, and A. S.Willsky. Distributed fusion in sensor networks. IEEE Signal Processing Magazine,23:42–55, July 2006. 7, 68
[6] R.T. Collins, A.J. Lipton, H. Fujiyoshi, and T. Kanade. Algorithms for cooperativemultisensor surveillance. Proceedings of the IEEE, 89(10):1456–1477, 2001. 7, 68
[7] C. Ding, B. Song, A. A. Morye, J. A. Farrell, and A. K. Roy-Chowdhury. Col-laborative sensing in a distributed PTZ camera network. IEEE Trans. on ImageProcessing, 21(7):3282–3295, 2012. 36
[8] F. Garin and L. Schenato. A survey on distributed estimation and control appli-cations using linear consensus algorithms. In Networked Control Systems, volume406 of Lecture Notes in Control and Information Sciences, pages 75–107. Springer,2011. 36, 63
[9] O. Gnawali, R. Fonseca, K. Jamieson, D. Moss, and P. Levis. Collection treeprotocol. In ACM Conference on Embedded Networked Sensor Systems, 2009. 4, 34
[10] R. Kalman. A new approach to linear filtering and prediction problems. Transactionof the ASME - Journal of Basic Engineering, 82(Series D):35–45, 1960. 5
[11] A. T. Kamal, C. Ding, B. Song, J. A. Farrell, and A. K. Roy-Chowdhury. Ageneralized Kalman consensus filter for wide-area video networks. In IEEE Conf.on Decision and Control, 2011. 5, 33, 36, 52
103
[12] A. T. Kamal, J. A. Farrell, and A. K. Roy-Chowdhury. Information weightedconsensus filters and their application in distributed camera networks. IEEE Trans.Automatic Control. 5
[13] A. T. Kamal, J. A. Farrell, and A. K. Roy-Chowdhury. Information weightedconsensus. In IEEE Conf. on Decision and Control, 2012. 5, 36, 68
[14] A. T. Kamal, J. A. Farrell, and A. K. Roy-Chowdhury. Information consensus fordistributed multi-target tracking. In IEEE Conf. on Computer Vision and PatternRecognition, 2013. 6
[15] A. T. Kamal, B. Song, and A. K. Roy-Chowdhury. Belief consensus for distributedaction recognition. In Intl. Conf. on Image Processing, pages 141 –144, Sept. 2011.36
[16] S. M. Kay. Fundamentals of statistical signal processing: estimation theory.Prentice-Hall, Inc., Upper Saddle River, NJ, USA, 1993. 39, 47
[17] R. Olfati-Saber. Kalman-consensus filter: Optimality, stability, and performance.In IEEE Conf. on Decision and Control, 2009. 4, 9, 14, 16, 21, 22, 23, 25, 31, 36,50, 52, 55, 66, 88
[18] R. Olfati-Saber, J. A. Fax, and R. M. Murray. Consensus and cooperation innetworked multi-agent systems. Proceedings of the IEEE, 95(1):215–233, Jan. 2007.2, 3, 13, 18, 31, 35, 74
[19] R. Olfati-Saber, E. Franco, E. Frazzoli, and J. S. Shamma. Belief consensus anddistributed hypothesis testing in sensor networks. In Network Embedded Sensingand Control, pages 169–182. Springer Verlag, 2006. 35
[20] R. Olfati-Saber and N. F. Sandell. Distributed tracking in sensor networks withlimited sensing range. In Proceedings of the American Control Conference, pages3157 – 3162, Seattle, WA, USA, June 2008. 14
[21] T. Onel, C. Ersoy, and H. Delic. On collaboration in a distributed multi-targettracking framework. In IEEE Intl. Conf. on Communications, June 2007. 8, 68
[22] L. L. Presti, S. Sclaroff, and M. L. Cascia. Path modeling and retrieval in distributedvideo surveillance databases. IEEE Trans. on Multimedia, 14(2):346–360, 2012. 7
[23] D. Reid. An algorithm for tracking multiple targets. IEEE Trans. AutomaticControl, 24(6):843–854, 1979. 6, 67
[24] W. Ren, A. W. Beard, and D. B. Kingston. Multi-agent Kalman consensus withrelative uncertainty. In American Control Conference, 2005. 16
[25] A. K. Roy-Chowdhury and B. Song. Camera Networks: The Acquisition and Anal-ysis of Videos over Wide Areas. Synthesis Lectures on Computer Vision. Morgan& Claypool Publishers, 2012. 89, 90
[26] N. F. Sandell and R. Olfati-Saber. Distributed data association for multi-targettracking in sensor networks. In IEEE Conf. on Decision and Control, 2008. 8, 68
104
[27] B. Song, C. Ding, A. T. Kamal, J. A. Farrell, and A. K. Roy-Chowdhury. Dis-tributed camera networks: Integrated sensing and analysis for wide area sceneunderstanding. IEEE Signal Processing Magazine, 3:20–31, May 2011. 9, 11, 36
[28] B. Song, A. T. Kamal, C. Soto, C. Ding, J. A. Farrell, and A. K. Roy-Chowdhury.Tracking and activity recognition through consensus in distributed camera net-works. IEEE Trans. on Image Processing, 19(10):2564–2579, Oct. 2010. 3, 8, 9, 11,14, 36, 68
[29] R. Tron and R. Vidal. Distributed image-based 3-D localization of camera sensornetworks. In IEEE Conf. on Decision and Control, pages 901 –908, Dec. 2009. 36
[30] R. Tron and R. Vidal. Distributed computer vision algorithms. IEEE Signal Pro-cessing Magazine, 28(3):32–45, May 2011. 3, 14, 34, 36
[31] R. Tron and R. Vidal. Distributed computer vision algorithms through distributedaveraging. In IEEE Conf. on Computer Vision and Pattern Recognition, 2011. 3,14
105
Appendix A
EKF: Information Form
Here we will derive the centralized Extended Kalman Filter (EKF) in infor-
mation form, which is required to derive the distributed EICF algorithm. We have,
measurement residual,
Z = Z − hc(x−c ). (A.1)
Kalman gain,
Kc = Pc−HT
(HP−c HT + R
)−1
= P−c HT(R−1 −R−1H
((P−c )−1 + HTR−1H
)−1 HTR−1)
(using Matrix Inversion Lemma)
=(P−c −P−c HTR−1H
((P−c )−1 + HTR−1H
)−1)HTR−1
=(P−c
((P−c )−1 + HTR−1H
)−P−c HTR−1H
) ((P−c )−1 + HTR−1H
)−1 HTR−1
=(Ip + P−c HTR−1H−P−c HTR−1H
) ((P−c )−1 + HTR−1H
)−1 HTR−1
=((P−c )−1 + HTR−1H
)−1 HTR−1. (A.2)
106
State estimate,
x+c = x−c + KcZ
= x−c +((P−c )−1 + HTR−1H
)−1 HTR−1(Z − hc(x
−c ))
= x−c +(J−c + Uc
)−1 (uc −HTR−1hc(x
−c ))
=(J−c + Uc
)−1 (J−c x−c + Ucx
−c + uc −HTR−1hc(x
−c ))
=(J−c + Uc
)−1 (J−c x−c + uc + HTR−1
(Hx−c − hc
(x−c)))
(A.3)
P+c =
((P−c )−1 + HTR−1H
)−1(A.4)
J+c = (P+
c )−1
= J−c + Uc (A.5)
107
Appendix B
JPDAF: Information Form
The JPDAF estimation and covariance update equation are given in the fol-
lowing which are then converted to the equivalent information form below. The time
index t has been dropped for simplicity. Now, Kalman gain,
Kj = Pj−HjTSj−1
= Pj−HjT(HjPj−HjT + Rj
)−1
= Pj−HjT(
Rj−1 −Rj−1Hj(
(Pj−)−1 + HjTRj−1Hj)−1
HjTRj−1)
(using Matrix Inversion Lemma)
=
(Pj− −Pj−HjTRj−1
Hj(
(Pj−)−1 + HjTRj−1Hj)−1
)HjTRj−1
=
(Pj−
((Pj−)−1 + HjTRj−1
Hj)−Pj−HjTRj−1
Hj
)(
(Pj−)−1 + HjTRj−1Hj)−1
HjTRj−1
=(Ip + Pj−HjTRj−1
Hj −Pj−HjTRj−1Hj)
((Pj−)−1 + HjTRj−1
Hj)−1
HjTRj−1
=(
(Pj−)−1 + HjTRj−1Hj)−1
HjTRj−1(B.1)
yj = yj − (1− βj0)Hjxj− (B.2)
108
xj+ = xj− + Kjyj
= xj− +(
(Pj−)−1 + HjTRj−1Hj)−1
HjTRj−1 (yj − (1− βj0)Hjxj−
)= xj− +
(Jj− + Uj
)−1 (uj − (1− βj0)Ujxj−
)=
(Jj− + Uj
)−1 (Jj−xj− + Ujxj− + uj − (1− βj0)Ujxj−
)=
(Jj− + Uj
)−1 (Jj−xj− + uj + βj0Ujxj−
)(B.3)
Pj+ = Pj− − (1− βj0)KjSjKjT + KjPjKjT
= Pj− −Kj(
(1− βj0)Sj − Pj)
KjT (B.4)
where,
Pj =
(l∑
n=1
βjnzjn(zjn)T)− yj
(yj)T
(B.5)
Let,
Cj = (1− βj0)Sj − Pj (B.6)
Thus, using matrix inversion lemma and by definition of Jj+ = (Pj+)−1 and Jj− =
(Pj−)−1 we get,
Jj+ = Jj− + Jj−Kj((
Cj)−1 −KjTJj−Kj
)−1KjTJj−
= Jj− + Gj (B.7)
where
Gj = Jj−Kj((
Cj)−1 −KjTJj−Kj
)−1KjTJj−. (B.8)
Thus, Equ. (B.3) and (B.7) are the JPDAF estimation equations in the information
form.
109
Appendix C
EMTIC: Derivation
For a single sensor, the JPDAF equations can be rewritten for the non-linear
case as the following,
Kj =(
(Pj−)−1 + HjTRj−1Hj)−1
HjTRj−1(C.1)
yj = yj − (1− βj0)h(xj−) (C.2)
The state estimate,
xj+ = xj− + Kjyj
= xj− +(
(Pj−)−1 + HjTRj−1Hj)−1
HjTRj−1 (yj − (1− βj0)h(xj−)
)= xj− +
(Jj− + Uj
)−1(uj − (1− βj0)HjTRj−1
h(xj−))
=(Jj− + Uj
)−1(
Jj−xj− + Ujxj− + uj − (1− βj0)HjTRj−1h(xj−)
)=
(Jj− + Uj
)−1(
Jj−xj− + uj + HjTRj−1(Hjxj− − (1− βj0)h(xj−))
)(C.3)
Pj+ = Pj− − (1− βj0)KjSjKjT + KjPjKjT
= Pj− −Kj(
(1− βj0)Sj − Pj)
KjT (C.4)
110
where,
Pj =
(l∑
n=1
βjnzjn(zjn)T)− yj
(yj)T
(C.5)
Let,
Cj = (1− βj0)Sj − Pj (C.6)
Thus, using matrix inversion lemma and by definition of Jj+ = (Pj+)−1 and Jj− =
(Pj−)−1 we get,
Jj+ = Jj− + Jj−Kj((
Cj)−1 −KjTJj−Kj
)−1KjTJj−
= Jj− + Gj (C.7)
where
Gj = Jj−Kj((
Cj)−1 −KjTJj−Kj
)−1KjTJj−. (C.8)
As, this is in the information form, for multiple sensor, under the assumption that the
additional measurement information are uncorrelated, they have to be added as the
following,
xj+c =
(Jj−c +
NC∑i=1
Uji
)−1
(Jj−i xj−i +
NC∑i=1
(uji + Hj
i
TRji
−1(Hj
i xj−i − (1− βj0i )hi(x
j−i ))
))(C.9)
Jj+c = Jj−i +
NC∑i=1
Gji (C.10)
111
Appendix D
Single step consensus comparison:
Proposition 1 The state estimate of ICF using a single step of consensus iteration can
be expressed as
x+i = x−i +
(A(
J−iNC
)+A(Ui)
)−1
A(ui)−A(Ui)x−i + ε
∑i′∈Ni
J−i′
NC
(x−i′ − x−i
) (D.1)
J+i = NC
(A(
J−iNC
)+A(Ui)
)(D.2)
Proof. Using the shorthand notation A() for a single step of consensus in (3.34), for a
single step we can write,
x+i =
(A(
J−iNC
)+A(Ui)
)−1(A(
J−iNC
x−i
)+A(ui)
)(D.3)
By adding and subtracting x−i in RHS of (D.3), we get,
x+i = x−i +
(A(
J−iNC
)+A (Ui)
)−1
(A(
J−iNC
x−i
)+A(ui)−
(A(
J−iNC
)+A(Ui)
)x−i
)= x−i +
(A(
J−iNC
)+A(Ui)
)−1
(A(ui)−A(Ui)x
−i +A
(J−iNC
x−i
)−A
(J−iNC
)x−i
)(D.4)
112
Now, A(
J−iNC
x−i
)−A
(J−iNC
)x−i
=J−iNC
x−i + ε∑i′∈Ni
(J−i′
NCx−i′ −
J−iNC
x−i
)−
J−iNC
x−i − ε∑i′∈Ni
(J−i′
NCx−i −
J−iNC
x−i
)
= ε∑i′∈Ni
J−i′
NC
(x−i′ − x−i
)(D.5)
Using this result in (D.4), we get,
x+i = x−i +
(A(
J−iNC
)+A(Ui)
)−1A(ui)−A(Ui)x
−i + ε
∑i′∈Ni
J−i′
NC
(x−i′ − x−i
)(D.6)
Similarly, J+i can be written as
J+i = NC
(A(
J−iNC
)+A(Ui)
)(D.7)
Proposition 2 The state estimate of MTIC using a single step of consensus iteration
can be expressed as
x+i = x−i +
(A(
J−iNC
) +A(Ui)
)−1
(A(ui)−A(Ui)x
−i +A(Uix
−i − (1− βi0)Uix
−i ) + ε
∑i′∈Ni
J−i′
NC
(x−i′ − x−i
) )(D.8)
J+i = NC
(A(
J−iNC
)+A(Gi)
)(D.9)
Proof. MTIC estimate equations (4.24) for a single consensus step, can be written with
the A() notation as (dropping the target superscript j),
x+i =
(A(
J−iNC
) +A(Ui)
)−1(A(
JiNC
x−i ) +A(β0Uix−i ) +A(ui)
)(D.10)
By adding and subtracting x−i in RHS we get,
x+i = x−i +
(A(
J−iNC
) +A(Ui)
)−1
(A(ui)−A(Ui)x
−i +A(
JiNC
x−i )−A(JiNC
)x−i +A(β0Uix−i )
)(D.11)
113
Using (D.5) and writing βi0 = 1− (1− βi0) we have,
x+i = x−i +
(A(
J−iNC
) +A(Ui)
)−1
(A(ui)−A(Ui)x
−i +A(Uix
−i − (1− βi0)Uix
−i ) + ε
∑i′∈Ni
J−i′
NC
(x−i′ − x−i
) )(D.12)
Similarly, J+i can be written as
J+i = NC
(A(
J−iNC
)+A(Gi)
)(D.13)
Proposition 3 The state estimate of EICF using a single step of consensus iteration
can be expressed as
x+i = x−i +
(A(
J−iNC
)+A(Ui)
)−1
(A(ui)−A(Ui)x
−i +A(Uix
−i −HT
i R−1i hi(x
−i )) + ε
∑i′∈Ni
J−i′
NC
(x−i′ − x−i
))(D.14)
J+i = NC
(A(
J−iNC
)+A(Ui)
)(D.15)
Proof. Using the shorthand notation A() for a single step of consensus in (5.17), for a
single step we can write,
x+i =
(A(
J−iNC
)+A(Ui)
)−1
(A(
J−iNC
x−i
)+A(ui) +A(Uixi)−A(HT
i R−1i hi(xi))
)(D.16)
By adding and subtracting x−i in RHS we get,
x+i = x−i +
(A(
J−iNC
)+A(Ui)
)−1(A(
J−iNC
x−i
)−A
(J−iNC
)x−i
+A(Uix−i )−A(Ui)x
−i +A(ui)−A(HT
i R−1i hi(xi))
)(D.17)
Using (D.5), we have,
x+i = x−i +
(A(
J−iNC
)+A(Ui)
)−1(A(ui)−A(Ui)x
−i
+A(Uix−i −HT
i R−1i hi(x
−i )) + ε
∑i′∈Ni
J−i′
NC
(x−i′ − x−i
))(D.18)
114
Similarly, J+i can be written as
J+i = NC
(A(
J−iNC
)+A(Ui)
)(D.19)
Proposition 4 The state estimate of EMTIC using a single step of consensus iteration
can be expressed as
x+i = x−i +
(A(
J−iNC
)+A(Ui)
)−1(A(ui)−A(Ui)x
−i
+A(Uix
−i − (1− β0
i )HTi R−1
i hi(x−i ))
+ ε∑i′∈Ni
J−i′
NC
(x−i′ − x−i
))(D.20)
J+i = NC
(A(
J−iNC
)+A(Gi)
)(D.21)
Proof. Using the shorthand notation A() for a single step of consensus in (5.41), for a
single step we can write,
x+i =
(A(
J−iNC
)+A(Ui)
)−1
(A(
J−iNC
x−i
)+A(ui) +A(Uixi)−A((1− β0
i )HTi R−1
i hi(xi))
)(D.22)
By adding and subtracting x−i in RHS we get,
x+i = x−i +
(A(
J−iNC
)+A(Ui)
)−1(A(
J−iNC
x−i
)−A
(J−iNC
)x−i
+A(Uix−i )−A(Ui)x
−i +A(ui)−A((1− β0
i )HTi R−1
i hi(xi))
)(D.23)
Using (D.5), we have,
x+i = x−i +
(A(
J−iNC
)+A(Ui)
)−1(A(ui)−A(Ui)x
−i
+A(Uix
−i − (1− β0
i )HTi R−1
i hi(x−i ))
+ ε∑i′∈Ni
J−i′
NC
(x−i′ − x−i
))(D.24)
Similarly, J+i can be written as
J+i = NC
(A(
J−iNC
)+A(Gi)
)(D.25)
115
Proposition 5 The state estimate of GKCF using a single step of consensus iteration
can be expressed as
x+i = x−i +
(A(J−i ) + Bi
)−1
bi −Bix−i + ε
∑i′∈Ni
J−i′(x−i′ − x−i
) (D.26)
J+i = A(J−i ) + Bi (D.27)
Proof. In the GKCF algorithm, after incorporating neighbors priors with a single step
consensus we get,
x−i =(A(J−i )
)−1A(J−i x−i ) (D.28)
After incorporating measurement information and adding and subtracting x−i (bi and
Bi are defined in (2.6-2.7)):
x+i = x−i − x−i + x−i +
(A(J−i ) + Bi
)−1 (bi −Bix
−i
)(D.29)
= x−i +(A(J−i ) + Bi
)−1 (bi +A(J−i )x−i −A(J−i )x−i −Bix
−i
)= x−i +
(A(J−i ) + Bi
)−1 (bi −Bix
−i +A(J−i x−i )−A(J−i )x−i
)(D.30)
= x−i +(A(J−i ) + Bi
)−1
bi −Bix−i + ε
∑i′∈Ni
J−i′(x−i′ − x−i
) (D.31)
To get (D.30), the relation in (D.28) was used. To get (D.31), the relation
A(J−i x−i )−A(J−i )x−i = ε∑i′∈Ni
J−i′(x−i′ − x−i
). (D.32)
was used which can be derived in a similar way (D.5) was derived. Similarly, J+i can be
written as
J+i = A(J−i ) + Bi. (D.33)
116