Compressive Sensing:An Introduction and Survey of Applications

ObjectivesDescription of theoryDiscussion of important resultsStudy of relevant applications

Introduction to the ProblemCS is a new paradigm that makes possible

fast acquisition of data using few number of samples

It tries to bring the number of samples acquired as close to the information content as possible

The classical Shannon-Whittaker sampling theorem has monopolized signal acquisition arena

It applies only to band-limited signalsSays number of samples ~ desired

resolution

Disadvantages of STBut band-limitedness not a universal

assumptionMost real-life signals have huge frequency

extentAlso it is not always a true measure of

information content (e.g. : train of spikes in frequency)

Also for increased resolution we need to increase the rate of sampling( but speed of most devices is limited)

In some other situations(e.g. : MRI) no of samples that can be acquired is inherently limited

Our objectiveWe want to investigate the possibility

of reconstructing a signal from fewer no. of samples than dictated by ST

Signal model assumed: SparsityMuch broader class than BL signalsA nonlinear classSparsity of a signal=no. of non-zero

coefficients = norm of the signal vector

l

0l

Sparsity model holds for most known signals

Most naturally-occurring signals are sparse in a certain basis

This property has been used in compression using transform coding:

Bandlimit

and Sample at

Nyquist Rate

H Thresholding

f

~

f

Even though samples are acquired at max possible rate we get rid of most of them

This strategy is wasteful in many waysCS combines the first and second stages to

acquire signals in the compressed form

The cost incurred is increased computational requirement

HfSome

algorithm for reconstructio

n

Sparse result

Questions:What is the minimum no of

measurements we require?How can reconstruction be done from

the reduced no of samples?The problem can be shown to be

reducible to a problem of solving an underdetermined system of linear equations

Reconstruction is possible because of sparsity assumption

This method is non-adaptive

Some Notation sensing basis sparsifying basis This is a set of orthonormal vectors where the signal is

known to have a sparse representation with S coefficients

• Measurements are linear functionals of the form where ∈ s.t. • It can be shown measurements are of the form where x is a sparse vector

}{ i}{ i Ni 1

, fi i T NMT ||

bUx

Effectively we are doing a projection of an n-dimensional vector into an M dimensional space

Need to make sure unique recovery is possible

i.e.; if Stability: RIP condition= Sub-matrix is well-

conditioned with a good condition number(necessary and sufficient)

For the sensing matrix and number of measurements these conditions should be satisfied

0)( 21 xxA21 xx

ContdRIP condition is related to the uncertainty

principle for the two orthobasesAnother property of interest in incoherenceThe less the coherence the better the resultsIt’s like saying the sensing matrix should be

as different from the sparsifying basis or the measurements should be holographic (non-concentrated).

Should convey the least amt of info abt the signal(20 weights problem)

Possibly random(or complementary bases like time frequency)

Method of reconstruction

Various possibilities existWe need to pick the least sparse solutionThe RIP condition ensures it is the unique solutionCould go for a combinatorial optimisation(directly

sift thru all sparse solutions to pick the actual one)

Then only require S+1 samplesBut it is NP hard –highly intractableAnother choice-l2 norm. Very easy to analyse. But

does not give Sparse solutionsSo a compromise is to go for l1 norm minimisation

Geometrical ArgumentL1 norm

L2 Norm

Why RIP is necessary?

ResultLet’s consider sets of orthogonal bases 𝝍 and 𝜱 If then with probability exceeding x supported on a fixed set can be recovered using the following optimisation problem: Imp requirements : incoherence ,randomness , RIPThis ensures that the set of measurements for which reconstruction fails occurs with very small prob: to take adv take random samples

)/log()(2 nUSM

1

1||min lx yUx

Non-Uniform Sampling Theorem

Signal composed of S discrete frequenciesTake M random measurements in time

domainFrom above theorem M>Slog(n) gives perfect

reconstructionTime and frequency domains are maximally

incoherentThis is also fundamental i.t.s.t. fewer

samples and reconstruction is virtually impossible

Eg Dirac Comb

More on RIPUnder the assumptions for which the

above result holds it can be shown that for x belonging to a fixed set T :

• Ensures any such signal is recovered

uniquely recoverable• To include all sets T we need to

strengthen the above condition (UUP). But then the number of samples required increases; becomes 4-5th power of log(n)

22

22

22 ||

2

3||||

2xmUxx

m

DefinitionRestricted Isometry Constant : It’s

smallest constant such that for every x belonging to T with size S <1 for condition to holdThis is an approximate orthogonality condition• UUP=should hold for all T’s with the same size

22

22

22 ||)1(||||)1( xAxx SS

S

ResultAssuming the existence of a matrix that

satisfies the above property with sufficiently

then

• Here the condition on M depends on the type of matrix we choose

• This is not probabilistic

12 S

11

12

||||

/||||*

*

lsl

lsl

xxCxx

SxxCxx

Two concernsSignals are only approximately sparseMeasurements are noisyOur reconstruction procedure should

be robust against these two casesRIP to the rescue!Has been shown if the

solution to sub to

satisfies

122 S

1||min lx

2|| lyAx

./|||| 1*

12CSxxCxx lsl

SummaryWe need to find sensing matrices that

satisfy the isometry propertySimple choice: random matricesIf m>CSlog(n/δ) random matrices satisfy RIPFor orthobases we need 4-5th power of log(n)Random matrices present storage difficulty

Application: Spectrum SensingSpectrum sensing is the task of detecting the

presence or absence of a carrier in a wideband of freqs

Cognitive Radios should be equipped with such a mechanism to enable efficient utilisation of

channelA major implementation challenge lies in the

very high sampling rates required by conventional spectral estimation methods which have to operate at or above the Nyquist rate.

Because of high rates no of samples is limitedMay not provide sufficient statistic

The situation is appropriate for deployment of CS

The spectrum is sparse because only a relatively small no of users are transmitting

Let be the frequency range in useBandwidth=B

No ff

Our job is to detect the N frequency bands and classify them as black, grey or white regions based on the PSD level

In the analysis we use a vector of time samples sampled at Nyquist rate of To. In the actual implementation only sub-Nyquist sampling is done

So let where is a vector of M values in the duration [0 MTo]. And

This is a generic model. S can be any matrix of basis vectors

Since sparsity under consideration is in freq, time domain is the best sampling domain. then S =

tT rSx tr

KRx

MI

Steps involvedReconstruct from M measurementsFind high resolution fourier transform

Obtain the frequency edges from Estimate the PSD in each bandLater we’ll see it is not necessary to

reconstruct the frequency vector from Only course sensing is done so noise is not

a key factorTo reduce noise effects a wavelet

smoothing operation is done

tr

tMf rFr fr

tr

Edge DetectionProblem similar to edge detection in imagesLet be a wavelet smoothing function with

a compact support. The dilation of by a scale factor s is given by:

For dyadic scales s is a power of twoContinuous wavelet transform of R(f) is given

by:

• Then we do a simple differencing to this function

)( f)( f

From above:

Replacing by its estimate found below:

We get the estimated wavelet transform:

Then we take the derivative of the above vector and find local minima

tr

Below is the vector with derivative values:

Take local peaks and nous sommes doneAlso in each band we can estimate the

average PSDby just averaging the frequency vector in that

band

One major simplification can be done by noting zs is itself a sparse vector. Thus we can eliminate the need to reconstruct

fr

To do this we define:

which is the differentiation matrix• Then we rewrite the above equation:

And finally:

Recovered frequency response

Channel EstimationCS also has applications in sparse channel

estimation Every delay-dominant channel can be

represented as a superposition of the pilot signal sent

It can be shown we can divide the total delay into bins each 1/W in width

Not all these bins are always occupiedEach bin represents a dominant scattererNo of bins that are non-zero<<p(total

bins)=floor(Tm*W)

Two ways to make use of CSUse proper pilot signalsCan be either random bits 1 or -1Or could be a sum of exponentials with

random frequencies.The first corresponds to using a random

sensing basisThe second is random sampling in frequency

using a subset of FT which is maximally incoherent

In both cases we obtain a better MSE less than LS

Another app is channel coding

References Channel Estimation:

.Bajwa, U.W., Haupt, J., Raz, G. and Nowak, R. (2008). “Compressed Channel Sensing”, Proceedings of the Annual Conference on Information Sciences and Systems, March 2008, Pages 1-6

• Spectrum Sensing: Z. Tian and G. B. Giannakis, “Compressed sensing for wideband

cognitive radios,” in Acoustics, Speech and Signal Processing,2007. ICASSP 2007. IEEE International Conference on, vol.4,Honolulu, HI, Apr. 2007, pp. 1357–1360.

CS E. Candès and J. Romberg, “Sparsity and incoherence in

compressive sampling,” Inverse Prob., vol. 23, no. 3, pp. 969–985, 2007.

An Introduction To Compressive Sampling: Emmanuel Candes Michael B Wakin

A Lecture on CS Richard Baraniuk