Wavelets and Signal Processing:

A Match Made in Heaven

Martin Vetterli, EPFL, 29.9.2015with Y.Barbotin, T.Blu, P.Marziliano, H.Pan, R.Parhizkar

Outline

2

• A Tale of Two Communities

• The Golden Age

• The Beauty and the Beast

• Time-Frequency-Scale as a Way of Life

• Signal Processing in the Age of Sparsity

• A Community of Interest and an Interesting Community

• Conclusions

Acknowledgments

3

• My friends from the two communities

• My collaborators

• H.G.Feichtinger and B.Torresani

4

Outline

• A Tale of Two Communities:

– Jean Morlet

– Signal processors

– Harmonic analysts

• The Golden Age

• The Beauty and the Beast

• Time-Frequency-Scale as a Way of Life

• Signal Processing in the Age of Sparsity

• A Community of Interest and an Interesting Community

• Conclusions

5

Ondelettes et/and Wavelets

“Le découverte de Morlet n'a pas reçu un bonaccueil. Peu de temps auparavant, un malfaiteurbelge était arrivé à persuader l'entreprise que l'onpouvait « flairer le pétrole » à l'aide d'« avionsrenifleurs ».Passant de l'extrême crédulité à l'extrême méfiance,Elf-Aquitaine répondit à la découverte de JeanMorlet en le mettant en « préretraite » “*

The story of Jean Morlet, the inventor of wavelets

* M.Nowak, Y.Meyer, La Recherche, Feb. 2005

Jean Morlet, 2001

6

Ondelettes et/and Wavelets

Morlet proposed this inversion formula based on intuition and numerical evidence. The story goes that when he showed it to a mathematician for verification, he was told: “This formula, being so simple, would be known if it were correct...”

The continuous wavelet transform by Morlet (1984)

7

Speech Processing: Subband Coding

This looks quite boring…

8

Claude Galand at Work!

9

Time Frequency Methods in Signal Processing

• Crochiere, Esteban, Galand 1976

• Short-time Fourier transform

• Perfect reconstruction filter banks

• Transform coding and KLT

• Subband speech and image coding

10

Harmonic Analysis: Beautiful and…Esoteric ?

Weierstrass function (1872)

11

Harmonic Analysis: Haar and Fourier bases

• Heisenberg uncertainty, Gabor expansion

• Balian-Low

12

The Meeting of the Minds

• Compactly supported wavelets: Ingrid Daubechies

• Multiresolution analysis: Stéphane Mallat, Yves Meyer

• Many Contributors: Stromberg, Lemarié, Battle, Cohen, ….

• Local cosine bases: Coifman, Meyer, Malvar

13

The Meeting of the Minds

Strömberg wavelet (1983)

14

Algorithms

• Wavelets based on filter banks• Orthogonal • Biorthogonal • Multidimensional

• Wavelets packets• Adaptive bases

• Mallat’s algorithm

Roy Lichtenstein: Magnifying Glass, 1963

15

Outline

• A Tale of Two Communities

• The Golden Age:

– When Wavelets Were Going to Cure Cancer

• The Beauty and the Beast

• Time-Frequency-Scale as a Way of Life

• Signal Processing in the Age of Sparsity

• A Community of Interest and an Interesting Community

• Conclusions

16

Wavelets are looking for applications…… but applications were not waiting for wavelets!

• Wavelets are beautiful• They don’t have to be necessary useful!

• Wavelets create a framework• Many disparate constructions have a common interpretation

• Wavelets and time-frequency-scale is a way of thinking about problems

17

A few stories

• Bell Laboratories Murray Hill• Is a Daubechies’ filter a filter?

• New York Times Science Section• Image compression will be improved, maybe a hundred fold!

• Wright Patterson Airforce Base, Ohio• Speech compression will be improved 10 times!

18

AFIT/AFOSR Wavelets Workshop participants

Greg Wornell, Stephane Mallat, Alexander Grossmann, Alan Oppenheim, Patrick Flandrin, Thomas Barnwell III, Leon Cohen, Gregory Beylkin, Jon Sjogren, Albert Cohen, Ingrid Daubechies, Martin Vetterli, P.P. Vaidyanathan, Robert Tenney, Ronald Coifman, Alan Willsky, Robert Ryan, Bruce Suter, Mark Oxley, John Benedetto, Greg Warhola.

19

Outline

• A Tale of Two Communities

• The Golden Age

• The Beauty and the Beast

– The Story of JPEG 2000

– Contributions of Wavelets

• Time-Frequency-Scale as a Way of Life

• Signal Processing in the Age of Sparsity

• A Community of Interest and an Interesting Community

• Conclusions

20

JPEG vs JPEG 2000

JPEG• Based on KLT• Theory from 60’s• Block based• Fast DCT

JPEG 2000• Based on wavelets• Theory from 80’s• No blocks• Fast WT

21

And the Winner is…

• JPEG is used in 98% of the cases• Mobile phones, digital

cameras

• JPEG 2000• Used in frame based digital

cinema

• Improvement of 1-2 dB does not justify change of standards

• Patent situation is murky…

From wikipedia page of JPEG2000: « However, the JPEG committee has acknowledged that undeclared submarine patents may still present a hazard. »

22

An example patent…

• Goupillaud, Morlet and Grossman patent

23

Main Contributions of Wavelets to SP:

Use of more general norms (getting rid of the tyranny of SNR ;)

Simple and powerful non-linear approximation for piecewise smooth functions

Thus: a piecewise smooth signal expands as:

• phase changes randomize signs, but not decay• a singularity influence only L wavelets at each scale• wavelet coefficients decay fast

25

Applications: Denoising

original

wavelet

13.8 dB

noisy

countourlets

15.4 dB

26

Lessons learned for signal processing

There is life beyond SNR

• Other norms are key (l1, TV, Sobolev)

There are exotic spaces that are actually useful

• Besov spaces

Sparsity is a key principle

• It helps in more ways than we thought

• It is critical for inverse problems

• It helps regularization

Pseudo diagonalization

…

and all this with solid theory and efficient algorithms!

27

Outline

• A Tale of Two Communities

• The Golden Age

• The Beauty and the Beast

• Time-Frequency-Scale as a Way of Life

– Maximally compact sequences

• Signal Processing in the Age of Sparsity

• A Community of Interest and an Interesting Community

• Conclusions

28

Time-Frequency Tilings

Fourier

Wavelets

29

Maximally compact sequences

Continuous time

• Time and frequency spreads: 2nd moments

• Heisenberg uncertainty bound

• Gaussians are maximally compact

Discrete time

• Time and frequency spreads?

• Uncertainty bound?

• What are the minimizers?

Localization for Discrete Sequences

• Option 1: Extension from analog signals

The signal is periodic in the freq. domain but the frequency definitions are not!

• Option 2: Use the first trigonometric moment (circular statistics)

Localization for Discrete Sequences

Localization for Discrete Sequences

• Option 1: Extension from analog signals

Uncertainty Principle for Discrete Sequences

Heisenberg: For sequences with :

Uncertainty Principle for Discrete Sequences

Maximally Compact Sequences

• The most compact sequence in time for a given frequency spread:

Theorem: For finding maximally compact sequences, solve the SDP:

The solution to above SDP is rank-1 and decomposed to .

Maximally Compact Sequences, Example

• Example:

Theorem: Fourier transform of maximally compact sequences are Mathieu functions.

New Uncertainty Bounds for Sequences

Maximally Compact Sequences: Gaussians?

There is a gap!

A New Benchmark

Uncertainty Principle for sequences

43

Outline

• A Tale of Two Communities

• The Golden Age

• The Beauty and the Beast

• Time-Frequency-Scale as a Way of Life

• Signal Processing in the Age of Sparsity

– Sampling 2.0

• A Community of Interest and an Interesting Community

• Conclusions

44

Given a class of objects, like a class of functions (e.g. bandlimited, SISS)

And given a sampling device, as usual to acquire the real world

– Smoothing kernel or low pass filter

– Regular, uniform sampling

Obvious question:

When is there a countable representation?

When does a minimum number of samples uniquely specify the function?

sampling kernel

An example from my garden...The sampling question:

45

From Analog to Digital… and back!

46

Shannon’s Theorem… a Bit of History

Whittaker

Nyquist

Kotelnikov

Whittaker

Raabe

Gabor

Shannon

1915

1928

1933

1935

1938

1946

1948

1949

Someya

47

Classic Case: Subspaces

Shannon bandlimited case

or 1/T degrees of freedom per unit time

But: a single discontinuity, and no more sampling theorem…

Are there other signals with finite number of degrees of freedom per unit of time that allow exact sampling results?

48

Examples of Non-bandlimited Signals

49

Classic Cases and Beyond…

Sampling theory beyond Shannon?

– Shannon: bandlimitedness is sufficient but not necessary

– Shannon bandwidth and shift-invariant subspaces: dimension of subspace

Is there a sampling theory beyond subspaces?

– Finite rate of innovation: Similar to Shannon rate of information

– Non-linear set up

– Position information is key!

Thus, develop a sampling theory for classes of non-bandlimited but sparse signals!

t

x(t)

t0x1

t1

x0…

xN-1

tN-1

Generic, continuous-time sparse signal

50

Signals with Finite Rate of Innovation

The set up:

For a sparse input, like a weighted sum of Diracs

– One-to-one map yn x(t)?

– Efficient algorithm?

– Stable reconstruction?

– Robustness to noise?

– Optimality of recovery?

sampling kernel

51

The sampling theorem (VMB02)

For the class of periodic FRI signals which includes

– Sequences of Diracs

– Non-uniform or free knot splines

– Piecewise polynomials

– …

There are sampling schemes with sampling at the rate of innovation with perfect recovery and polynomial complexity

Variations: finite length, 2D, local kernels etc

52

What’s Maybe Surprising….

Bandlimited

Manifold

53

Current challenges…

The Tyranny of the pixel!

The

Vio

lin, F

elix

Val

lott

on

54

The good old super-resolution problem….

Up-sampled image with mask regularizerUp-sampled image without mask regularizer

The irony of it all…

55

As a signal processor:• I went from l2(Z) to L2(R)Meanwhile:• Compressed sensing. • Beautiful theory of sparsity in RN!

56

Outline

• A Tale of Two Communities

• The Golden Age

• The Beauty and the Beast

• Time-Frequency-Scale as a Way of Life

• Signal Processing in the Age of Sparsity

• A Community of Interest and an Interesting Community

• Conclusions

The Wavelet and SP crowd

57I.Daubechies, Y.Meyer, R.Coifman, D.Donoho, A.Cohen, S.Mallat, M.Unser, R.Malvar,M.Smith, P.P.Vaidyanathan, A.Aldroubi, H.G.Feichtinger, K.H.Groechenig and many others!

58

My Wavelet and SP gang

T. Blu, O. Rioul, C. Herley, J. Kovacevik, K. Ramchandran, T. Nguyen, A. Ortega, V. Goyal, R. Parhizkar, P. Marziliano, Y. Barbotin, H.Pan, and many more….

59

• Outline

• A Tale of Two Communities

• The Golden Age

• The Beauty and the Beast

• Time-Frequency-Scale as a Way of Life

• Signal Processing in the Age of Sparsity

• A Community of Interest and an Interesting Community

• Conclusions: Foundations of Signal Processing

60

It has changed my view of the (signal processing) world!

Wavelets brought a new understanding of

known methods

«Understanding is a lot like sex. It's got a

practical purpose, but that's not why people

do it normally». Frank Oppenheimer

Wavelets raised high expectations and

produced new methods and successes

«As new developments emerge in any field, its

important to let them mature without

generating unrealistic expectations so that the

beautiful and important aspects have time and

good soil in which to blossom”. Al Oppenheim

61

It has changed my view of the (signal processing) world!

62

63

“The math is right. It’s just in poor taste.’’© New Yorker

64

65