Compressive Sensing A New Approach to Signal Acquisition and Processing Richard Baraniuk Rice...
-
Upload
aubrey-willis -
Category
Documents
-
view
223 -
download
0
Transcript of Compressive Sensing A New Approach to Signal Acquisition and Processing Richard Baraniuk Rice...
CompressiveSensingA New Approach to Signal Acquisition and Processing
Richard Baraniuk
Rice University
LECTURE THREE
Sparsity and CS: Applications
and Current Trends
Recall: CS
• Sensing: = random linear combinations of the entries of
• Recovery: Recover from via optimization
measurements sparsesignal
nonzeroentries
Gerhard Richter 4096 Farben / 4096 Colours
1974254 cm X 254 cmLaquer on CanvasCatalogue Raisonné: 359
Museum Collection:Staatliche Kunstsammlungen Dresden (on loan)
Sales history: 11 May 2004Christie's New York Post-War and Contemporary Art (Evening Sale), Lot 34US$3,703,500
“Single-Pixel” CS Camera
randompattern onDMD array
DMD DMD
single photon detector
imagereconstructionorprocessing
w/ Kevin Kelly
scene
“Single-Pixel” CS Camera
randompattern onDMD array
DMD DMD
single photon detector
imagereconstructionorprocessing
scene
• Flip mirror array M times to acquire M measurements• Sparsity-based (linear programming) recovery
…
First Image Acquisition
target 65536 pixels
1300 measurements (2%)
11000 measurements (16%)
World’s First Photograph
• 1826, Joseph Niepce• Farm buildings and sky • 8 hour exposure• On display at UT-Austin
Utility?
DMD DMD
single photon detector
Fairchild100Mpixel
CCD
Utility?
DMD DMD
single photon detector
Fairchild100Mpixel
CCD
CS Low-Light Imaging with PMT
true color low-light imaging
256 x 256 image with 10:1 compression
[Nature Photonics, April 2007]
CS Infrared Camera
20% 5%
CS Hyperspectral Imager
spectrometer
hyperspectral data cube450-850nm
1M space x wavelength voxels200k random sums
CS MRI
• Lustig, Pauly, Donoho et al at Stanford
• Goal: Speed up MRI data acquisition by reducing number of samples required for a given image reconstruction quality
• Approach: Design MRI sampling pattern (in frequency/k-space) to be close to random
Multi-Slice Brain Imaging [M. Lustig]
Compressive SensingIn Action
A/D Converters
Analog-to-Digital Conversion
• Nyquist rate limits reach of today’s ADCs
• “Moore’s Law” for ADCs:– technology Figure of Merit incorporating sampling rate
and dynamic range doubles every 6-8 years
• Analog-to-Information (A2I) converter– wideband signals have
high Nyquist rate but are often sparse/compressible
– develop new ADC technologies to exploit
– new tradeoffs amongNyquist rate, sampling rate,dynamic range, …
frequency hopperspectrogram
time
frequency
Streaming Measurements
measurements
streaming requires special
• Streaming applications: cannot fit entire signal into a processing buffer at one time
Streaming Measurements
measurements
streaming requires special
• Streaming applications: cannot fit entire signal into a processing buffer at one time
Streaming Measurements
measurements
streaming requires special
• Streaming applications: cannot fit entire signal into a processing buffer at one time
RIP?
Streaming Measurements
streaming requires special
• Many applications: Signal sparse in frequency
(Fourier transform)
Random Demodulator
A
AB
B
C
CD
D
Random Demodulator
Random Demodulator
Sampling Rate
• Goal: Sample near signal’s (low) “information rate” rather than its (high) Nyquist rate
A2Isampling rate
number oftones /window
Nyquistbandwidth
Sampling Rate
• Theorem [Tropp et al 2007]
If the sampling rate satisfies
then locally Fourier K-sparse signals can be recovered exactly with probability
Empirical Results
Example: Frequency Hopper
20x sub-Nyquist sampling
spectrogram sparsogram
Nyquist rate sampling
More CS In Action• CS makes sense when measurements
are expensive
• Ultrawideband A/D converters[DARPA “Analog to Information” program]
• Camera networks– sensing/compression/fusion
• Radar, sonar, array processing– exploit spatial sparsity of targets
• DNA microarrays– smaller, more agile arrays for
bio-sensing
…
Beyond Sparsity
Structured Sparsity
Beyond Sparse Models
• Sparse signal model captures simplistic primary structure
wavelets:natural images
Gabor atoms:chirps/tones
pixels:background subtracted
images
Beyond Sparse Models
• Sparse signal model captures simplistic primary structure
• Modern compression/processing algorithms capture richer secondary coefficient structure
wavelets:natural images
Gabor atoms:chirps/tones
pixels:background subtracted
images
Sparse Signals
• K-sparse signals comprise a particular set of K-dim subspaces
Structured-Sparse Signals
• A K-sparse signal model comprises a particular (reduced) set of K-dim subspaces[Blumensath and Davies]
• Fewer subspaces <> relaxed RIP <> stable recovery using
fewer measurements M
Wavelet Sparse
• Typical of wavelet transformsof natural signals and images (piecewise smooth)
Tree-Sparse
• Model: K-sparse coefficients + significant coefficients
lie on a rooted subtree
• Typical of wavelet transformsof natural signals and images (piecewise smooth)
Wavelet Sparse• Model: K-sparse coefficients
+ significant coefficients lie on a rooted subtree
• RIP: stable embedding
K-dim subspaces
Tree-Sparse• Model: K-sparse coefficients
+ significant coefficients lie on a rooted subtree
• Tree-RIP: stable embedding
K-dim subspaces
Recall: Iterated Thresholding
update signal estimate
prune signal estimate(best K-term approx)
update residual
Iterated Model Thresholding
update signal estimate
prune signal estimate(best K-term model approx)
update residual
Tree-Sparse Signal Recovery
target signal CoSaMP, (RMSE=1.12)
Tree-sparse CoSaMP (RMSE=0.037)
N=1024M=80
L1-minimization(RMSE=0.751)
[B, Cevher, Duarte, Hegde’08]
Other Useful Models
• Clustered coefficients [C, Duarte, Hegde, B], [C, Indyk, Hegde, Duarte, B]
• Dispersed coefficients [Tropp, Gilbert, Strauss], [Stojnic, Parvaresh, Hassibi], [Eldar, Mishali], [Baron, Duarte et al], [B, C, Duarte, Hegde]
Clustered Signals
target Ising-modelrecovery
CoSaMPrecovery
LP (FPC)recovery
• Probabilistic approach via graphical model
• Model clustering of significant pixels in space domain using Ising Markov Random Field
• Ising model approximation performed efficiently using graph cuts [Cevher, Duarte, Hegde, B’08]
Block-Sparse Model
N = 4096K = 6 active blocksJ = block length = 64M = 2.5JK = 960 msnts
[Stojnic, Parvaresh, Hassibi], [Eldar, Mishali],[B, Cevher, Duarte, Hegde]
target CoSaMP (MSE = 0.723)
block-sparse model recovery (MSE=0.015)
Sparse Spike Trains
• Sequence of pulses
• Simple model: – sequence of Dirac pulses– refractory period
between each pulse
• Model-based RIP if
• Stable recovery viaiterative algorithm(exploit total unimodularity)[Hedge, Duarte, Cevher ‘09]
N=1024 K=50 =10
M=150
originalmodel-based recovery error
CoSaMPrecovery error
Sparse Spike Trains
Sparse Pulse Trains
• More realistic model: – sequence of Dirac pulses * pulse shape of length – refractory period between each pulse of length
• Model-based RIP if
• More realistic model: – sequence of Dirac pulses * pulse shape of length – refractory period between each pulse of length
• Model-based RIP if
N=4076 K=7 =25 =10
M=290
original CoSaMP model-alg
Sparse Pulse Trains
Summary
• Compressive sensing– randomized dimensionality reduction– integrates sensing, compression, processing– exploits signal sparsity information– enables new sensing modalities, architectures, systems– relies on large-scale optimization
• Why it works: preserves information in signals with concise geometric structure
sparse signals | compressible signals | manifolds
Open Research Issues
• Links with information theory– new encoding matrix design via codes (LDPC, fountains)– new decoding algorithms (BP, etc.)– quantization and rate distortion theory
• Links with machine learning– Johnson-Lindenstrauss, manifold embedding, RIP
• Processing/inference on random projections– filtering, tracking, interference cancellation, …
• Multi-signal CS– array processing, localization, sensor networks, …
• CS hardware– ADCs, receivers, cameras, imagers, radars, …
Discussion Session 3
• Discussion:– Model-based CS
• Computer exercises:– Manifold CS and smashed filter for classification– Model-based compressive sensing– Democracy and justice for enhanced robustness
dsp.rice.edu/cs
Open Positions
open postdoc positions in sparsity / compressive sensing at Rice University
dsp.rice.edu [email protected]
Connexions (cnx.org)
• non-profit open publishing project
• goal: make high-quality educational content available to anyone, anywhere, anytime for free on the web and at very low cost in print
• open-licensed repository of Lego-block modules for authors, instructors, and learners to create, rip, mix, burn
• global reach: >1M users monthlyfrom 200 countries
• collaborators: IEEE (IEEEcnx.org), Govt. Vietnam, TI, NI, …
Thanks!
• PCMI organizers and staff
• Chinmay Hegde, TA
• Rice DSP past and present– Mark Davenport, Marco Duarte, Mike Wakin, Volkan Cevher