Post on 01-Jun-2020
Value of
Information
1st year review. UCLA 2012
VOI
Year 2 review. ARL, Sept 9 2013
ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and Exploitation
Value of
Information
1st year review. UCLA 2012
ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and Exploitation
VOI
Year 2 review. ARL, Sept 9 2013
Information-driven learning, distributed fusion, and planning
Co-PI Alfred Hero University of Michigan
Value of
Information
1st year review. UCLA 2012
VOI
Year 2 review. ARL, Sept 9 2013 1st year review. UCLA 2012
Value of
Information VOI
Year 2 review. ARL, Sept 9 2013
Our main thrust this year
• Progress 1: Information-driven learning: – Learning structure in high dimension: Kronecker PCA – For spatio-temporal sources Kronecker PCA captures information
much more efficiently than standard (low-rank) PCA
• Progress 2: Distributed information fusion: – Distributed inference: local 2nd order nbd marginalization – Performs as well as global fusion w/o message passing
• Progress 3: Human-in-the-loop planning and processing
– Cooperative human-machine 20 questions framework – Human adds early information gain for target detection
Quantify and optimize VoI by: dimensionality reduction, nearest neighbor aggregation, and human-in-the-loop.
Value of
Information
1st year review. UCLA 2012
VOI
Year 2 review. ARL, Sept 9 2013 1st year review. UCLA 2012
Value of
Information VOI
Year 2 review. ARL, Sept 9 2013
Progress 1: Information driven learning Last year: KGlasso – MSE scaling laws
Tsiligkaridis, Hero, Zhou, 2012
Una
chie
vabl
e re
gion
• KGlasso has best scaling law in n,p • Task: Estimate spatio-temporal covariance
0 10 20 30 40 50 60 70 80 90 100
20 uncorrelated sequences
Time index i
Value of
Information
1st year review. UCLA 2012
VOI
Year 2 review. ARL, Sept 9 2013 1st year review. UCLA 2012
Value of
Information VOI
Year 2 review. ARL, Sept 9 2013
Progress 1: Information-driven learning This year: Kronecker PCA (See poster)
[1] Tsiligkaridis & H TSP 2013
Standard PCA Kronecker PCA
||𝐂 − 𝐂𝑟 ||𝐹2 ≤ 𝐦𝐦𝐦𝒓𝒓𝒓𝒓 𝐑 ≤𝒓
||𝐑 − 𝑹 𝐂 ||𝑭𝟐 + 𝑪𝒓 𝒑𝟐 + 𝒒𝟐
𝒓
Theorem [1]: For pq x pq covariance C the MSE of Kronecker PCA approximation Cr to C is bounded
r r
Deficiency: Single KP may not be adequate fit to covariance A Solution: Use a sum of KPs to approximate covariance -> K-PCA
Property (Pistsianis and Van Loan 1992): K-PCA is complete expansion Our K-PCA algorithm: Spectral solution to convex minimization problem
Value of
Information
1st year review. UCLA 2012
VOI
Year 2 review. ARL, Sept 9 2013 1st year review. UCLA 2012
Value of
Information VOI
Year 2 review. ARL, Sept 9 2013
Progress 1: Information-driven learning Kronecker PCA (See poster)
[1] Tsiligkaridis & H TSP 2013
Kronecker spectrum Eigenspectrum
Value of
Information
1st year review. UCLA 2012
VOI
Year 2 review. ARL, Sept 9 2013 1st year review. UCLA 2012
Value of
Information VOI
Year 2 review. ARL, Sept 9 2013
Progress 2: Distributed information fusion in sensor networks
Meng,Wei,Wiesel,H, AISTATS 2013(NP Award)
||𝐂 − 𝐂𝑟 ||𝐹2
≤ 𝐦𝐦𝐦𝒓𝒓𝒓𝒓 𝐑 ≤𝒓
||𝐑 − 𝑹 𝐂 ||𝑭𝟐 + 𝑪𝒓 𝒑𝟐 + 𝒒𝟐
𝒓
Theorem [1]: For pq x pq sample cov C the MSE of Kronecker PCA of rank r is bounded
Objective: predict states measured by SN Model: Gauss-Markov random field:
Standard: Iterative ML by message passing Proposed: Non-iterative by 2-NN relaxation
Value of
Information
1st year review. UCLA 2012
VOI
Year 2 review. ARL, Sept 9 2013 1st year review. UCLA 2012
Value of
Information VOI
Year 2 review. ARL, Sept 9 2013
Progress 3: Human-in-the-loop processing Cooperative localization (See poster)
Tsiligkaridis, Sadler & Hero, ICASSP 2013
Human MSE gain ratio
Optimal queries are equalizing bisection rules
Human advantage: can account for context Human limitation: limited visual accuity
Value of
Information
1st year review. UCLA 2012
VOI
Year 2 review. ARL, Sept 9 2013 1st year review. UCLA 2012
Value of
Information VOI
Year 2 review. ARL, Sept 9 2013
Summary of year 2 activities
• This year’s research directly impacts – Information-driven learning
• Kronecker PCA provides much better fit to spatio-temporal data than standard PCA. VoI for prediction/detection/classification is improved.
– Distributed information fusion • Second-order neighborhood information has nearly as high
value as global information about SN. – Information exploitation
• Inclusion of human-in-the-loop provides up to 15% MSE gain in early iterations. Value of human-provided information characterized by resolution acuity parameter kappa.
Value of
Information
1st year review. UCLA 2012
VOI
Year 2 review. ARL, Sept 9 2013 1st year review. UCLA 2012
Value of
Information VOI
Year 2 review. ARL, Sept 9 2013
Ongoing and future focus areas and collaborations
• Working towards a unified VoI theory of sensing of spatio-temporal processes: – VoI-driven mission planning with target-dependent payoffs
(UM/MIT) – (Poster today - Mu, Newstadt, How, H) – Information theoretic bounds and algorithms for
learning/fusion/planning that account for side-information (human inputs, low rank, sparse, Kronecker structure)
– Information geometric theory of VoI (UM/ASU)
• Applications to anomaly detection in video, SNs and radar (UM/OSU/AFRL).
• Experimental validation studies: – Software defined radar testbed (UM/OSU/ASU/MIT). – Refined human models and experiments (UM/UCSD)
Value of
Information
1st year review. UCLA 2012
VOI
Year 2 review. ARL, Sept 9 2013 1st year review. UCLA 2012
Value of
Information VOI
Year 2 review. ARL, Sept 9 2013
Publications in year 2 • T. Tsiligkaridis, A.O. Hero, S. Zhou, "Convergence properties of
Kronecker graphical lasso algorithms," IEEE Trans on SP, 2013 • T. Tsiligkaridis N and A.O. Hero, ``Covariance Estimation in High
Dimensions via Kronecker Product Expansions,'' IEEE Trans on SP, 2013.
• D. Wei and A.O. Hero, "Multistage adaptive estimation of sparse Signals," IEEE Journal of Selected Topics in Signal Processing, 2013.
• Dennis Wei and Alfred O. Hero, III, "Adaptive spectrum sensing and estimation," ICASSP 2013, Vancouver.
• Z. Meng, D. Wei, A. Wiesel, A.O. Hero, "Distributed Learning of Gaussian Graphical Models via Marginal Likelihoods,“ AISTAT.
• Theodoros Tsiligkaridis and Alfred O. Hero III, "Low Separation Rank Covariance Estimation using Kronecker Product Expansions," IEEE ISIT 2013, Istanbul.
• Theodoros Tsiligkaridis, Brian M. Sadler and Alfred O. Hero III, "A collaborative 20 questions model for target search with human-machine interaction," ICASSP 2013, Vancouver.