Recovery of Clustered Sparse Signals from Compressive Measurements Volkan Cevher [email protected]...
-
Upload
easter-banks -
Category
Documents
-
view
222 -
download
3
Transcript of Recovery of Clustered Sparse Signals from Compressive Measurements Volkan Cevher [email protected]...
Recovery of Clustered Sparse Signals from Compressive Measurements
Volkan [email protected]
Richard BaraniukChinmay HegdePiotr Indyk
The Digital Universe
• Size: ~300 billion gigabytes generated in 2007
digital bits > stars in the universegrowing by a factor of 10 every 5 years > Avogadro’s number (6.02x1023) in 15 years
• Growth fueled by multimedia / multisensor data
audio, images, video, surveillance cameras, sensor nets, …
• In 2007 digital data generated > total storage
by 2011, ½ of digital universe will have no home
[Source: IDC Whitepaper “The Diverse and Exploding Digital Universe” March 2008]
Approaches
• Finite Rate of Innovation
Sketching / Streaming
Compressive Sensing
[Vetterli, Marziliano, Blu; Blu, Dragotti, Vetterli, Marziliano, Coulot; Gilbert, Indyk, Strauss, Cormode, Muthukrishnan; Donoho; Candes, Romberg, Tao; Candes, Tao]
Approaches
• Finite Rate of Innovation
Sketching / Streaming
Compressive Sensing
[Vetterli, Marziliano, Blu; Blu, Dragotti, Vetterli, Marziliano, Coulot; Gilbert, Indyk, Strauss, Cormode, Muthukrishnan; Donoho; Candes, Romberg, Tao; Candes, Tao]
PARSITY
Agenda
• A short review of compressive sensing
• Beyond sparse models
– Potential gains via structured sparsity
• Block-sparse model
• (K,C)-sparse model
• Conclusions
Compressive Sensing 101
• Goal: Recover a sparse orcompressible signal from measurements
• Problem: Randomprojection not full rank
• Solution: Exploit the sparsity/compressibilitygeometry of acquired signal
• Goal: Recover a sparse orcompressible signal from measurements
• Problem: Randomprojection not full rankbut satisfies Restricted Isometry Property (RIP)
• Solution: Exploit the sparsity/compressibility geometry of acquired signal
– iid Gaussian– iid Bernoulli– …
Compressive Sensing 101
• Goal: Recover a sparse orcompressible signal from measurements
• Problem: Randomprojection not full rank
• Solution: Exploit the modelgeometry of acquired signal
Compressive Sensing 101
• Sparse signal: only K out of N coordinates nonzero
– model: union of K-dimensional subspacesaligned w/ coordinate axes
Basic Signal Models
sorted index
• Sparse signal: only K out of N coordinates nonzero
– model: union of K-dimensional subspaces
• Compressible signal: sorted coordinates decay rapidly to zero
well-approximated by a K-sparse signal(simply by thresholding)
sorted index
Basic Signal Models
power-lawdecay
Recovery Algorithms
• Goal:given
recover
• and convex optimization formulations– basis pursuit, Dantzig selector, Lasso, …
• Greedy algorithms– iterative thresholding (IT),
compressive sensing matching pursuit (CoSaMP)subspace pursuit (SP)
– at their core: iterative sparse approximation
Performance of Recovery
• Using methods, IT, CoSaMP, SP
• Sparse signals
– noise-free measurements: exact recovery – noisy measurements: stable recovery
• Compressible signals
– recovery as good as K-sparse approximation
CS recoveryerror
signal K-termapprox error
noise
Sparsity as a Model
• Sparse/compressible signal model captures simplistic primary structure
sparse image
• Sparse/compressible signal model captures simplistic primary structure
• Modern compression/processing algorithms capture richer secondary coefficient structure
Beyond Sparse Models
wavelets:natural images
Gabor atoms:chirps/tones
pixels:background subtracted
images
Model-Sparse Signals
• Defn: A K-sparse signal model comprises a particular (reduced) set of K-dim canonical subspaces
Model-Sparse Signals
• Defn: A K-sparse signal model comprises a particular (reduced) set of K-dim canonical subspaces
• Structured subspaces
<> fewer subspaces
<> relaxed RIP
<> fewer measurements
[Blumensath and Davies]
Model-Sparse Signals
• Defn: A K-sparse signal model comprises a particular (reduced) set of K-dim canonical subspaces
• Structured subspaces
<> increased signal discrimination
<> improved recovery perf.
<> faster recovery
• Motivation
– sensor networks
• Signal model
intra-sensor: sparsity inter-sensor: common sparse supports
– union of subspaces when signals are concatenated
Block-Sparsity
sensors
[Tropp, Eldar, Mishali; Stojnic, Parvaresh, Hassibi; Baraniuk, VC, Duarte, Hegde]
…
*Individual block sizes may vary…
• Sampling bound (B = # of blocks)
• Problem specific solutions
– Mixed -norm solutions
– Greedy solutions: simultaneous orthogonal matching pursuit [Tropp]
• Model-based recovery framework: -norm [Baraniuk, VC, Duarte, Hegde]
Block-Sparsity Recovery
within block
[Eldar, Mishali (provable); Stojnic, Parvaresh, Hassibi]
• Iterative Thresholding
Standard CS Recovery
[Nowak, Figueiredo; Kingsbury, Reeves; Daubechies, Defrise, De Mol; Blumensath, Davies; …]
update signal estimate
prune signal estimate(best K-term approx)
update residual
• Iterative Model Thresholding
Model-based CS Recovery
[VC, Duarte, Hegde, Baraniuk; Baraniuk, VC, Duarte, Hegde]
update signal estimate
prune signal estimate(best K-term model approx)
update residual
• Provable guarantees
Model-based CS Recovery
update signal estimate
prune signal estimate(best K-term model approx)
update residual
[Baraniuk, VC, Duarte, Hegde]
Block-Sparse Signal
target CoSaMP (MSE = 0.723)
block-sparse model recovery (MSE=0.015) Blocks are pre-specified.
Clustered Sparsity
• (K,C) sparse signals (1-D)
– K-sparse within at most C clusters
• Stable recovery
• Recovery:
– w/ model-based framework
– model approximation via dynamic programming(recursive / bottom up)
Simulation via Block-Sparsity
• Clustered sparse <> approximable by block-sparse
*If we are willing to pay 3 × sampling penalty
• Proof by (adversarial) construction
…
Clustered Sparsity in 2D
target Ising-modelrecovery
CoSaMPrecovery
LP (FPC)recovery
• Model clustering of significant pixels in space domain using graphical model (MRF)
• Ising model approximation via graph cuts[VC, Duarte, Hedge, Baraniuk]
Conclusions
• Why CS works: stable embedding for signals with special geometric structure
• Sparse signals >> model-sparse signals
• Greedy model-based signal recovery algorithms
upshot: provably fewer measurementsmore stable recovery
new model: clustered sparsity
• Compressible signals >> model-compressible signals
Volkan Cevher / [email protected]