A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of...
Transcript of A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of...
![Page 1: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/1.jpg)
A dual certificate analysis of compressive off-the-grid recovery
Nicolas KerivenEcole Normale Supérieure (Paris)
CFM-ENS chair in Data Science
Joint work with Clarice Poon (Cambridge Uni.), Gabriel Peyré (ENS)
Journée GdR MIA, May 3rd 2018
![Page 2: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/2.jpg)
Discrete compressive sensing
1/14
![Page 3: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/3.jpg)
Discrete compressive sensing
1/14
![Page 4: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/4.jpg)
Discrete compressive sensing
1/14
![Page 5: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/5.jpg)
• Signal: vector
Discrete compressive sensing
1/14
![Page 6: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/6.jpg)
• Signal: vector• Sparsity: few non-zeros coefficients
Discrete compressive sensing
1/14
![Page 7: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/7.jpg)
• Signal: vector• Sparsity: few non-zeros coefficients• Dimensionality reduction (often random matrix)
Discrete compressive sensing
1/14
![Page 8: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/8.jpg)
• Signal: vector• Sparsity: few non-zeros coefficients• Dimensionality reduction (often random matrix)• Recovery: convex relaxation LASSO
Discrete compressive sensing
1/14
![Page 9: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/9.jpg)
Off-the-grid recovery: « super-resolution »
2/14
![Page 10: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/10.jpg)
Off-the-grid recovery: « super-resolution »
2/14
![Page 11: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/11.jpg)
- Fourier
- Convolution
Off-the-grid recovery: « super-resolution »
2/14
![Page 12: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/12.jpg)
- Fourier
- Convolution
Off-the-grid recovery: « super-resolution »
2/14
![Page 13: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/13.jpg)
- Fourier
- Convolution
Off-the-grid recovery: « super-resolution »
• Signal: Radon measure
2/14
![Page 14: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/14.jpg)
- Fourier
- Convolution
Off-the-grid recovery: « super-resolution »
• Signal: Radon measure• Sparsity:
2/14
![Page 15: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/15.jpg)
- Fourier
- Convolution
Off-the-grid recovery: « super-resolution »
• Signal: Radon measure• Sparsity:• Dimensionality reduction (e.g. first Fourier coefficients)
2/14
![Page 16: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/16.jpg)
- Fourier
- Convolution
Off-the-grid recovery: « super-resolution »
• Signal: Radon measure• Sparsity:• Dimensionality reduction (e.g. first Fourier coefficients)• Recovery: convex relaxation?
See Keriven 2017, Gribonval 2017
2/14
![Page 17: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/17.jpg)
- Fourier
- Convolution
Off-the-grid recovery: « super-resolution »
• Signal: Radon measure• Sparsity:• Dimensionality reduction (e.g. first Fourier coefficients)• Recovery: convex relaxation? BLASSO [De Castro, Gamboa 2012]
See Keriven 2017, Gribonval 2017
2/14
![Page 18: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/18.jpg)
- Fourier
- Convolution
Off-the-grid recovery: « super-resolution »
• Signal: Radon measure• Sparsity:• Dimensionality reduction (e.g. first Fourier coefficients)• Recovery: convex relaxation? BLASSO [De Castro, Gamboa 2012]
Other approaches: « Prony-like » ESPRIT, MUSIC… (but only 1d noiseless Fourier)
See Keriven 2017, Gribonval 2017
2/14
![Page 19: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/19.jpg)
Example of applications
Fluorescence microscopy[Betzig 2006]
3/14
![Page 20: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/20.jpg)
Example of applications
Fluorescence microscopy[Betzig 2006]
Astronomy[Puschmann 2017]
3/14
![Page 21: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/21.jpg)
Example of applications
Fluorescence microscopy[Betzig 2006]
Compressive k-means(GMM…) [Keriven 2017]
Astronomy[Puschmann 2017]
3/14
![Page 22: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/22.jpg)
Example of applications
Fluorescence microscopy[Betzig 2006]
Compressive k-means(GMM…) [Keriven 2017]
Astronomy[Puschmann 2017]
• Neuro-imaging with EEG [Gramfort 2013]
• 1-layer neural network [Bach 2017]
• Radar• Geophysics• …
3/14
![Page 23: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/23.jpg)
A seminal result [Candès, Fernandez-Granda 2012]
4/14
![Page 24: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/24.jpg)
A seminal result [Candès, Fernandez-Granda 2012]
First Fourier coefficients
4/14
![Page 25: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/25.jpg)
A seminal result [Candès, Fernandez-Granda 2012]
First Fourier coefficients
Inverse Fourier
4/14
![Page 26: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/26.jpg)
A seminal result [Candès, Fernandez-Granda 2012]
First Fourier coefficients
Inverse Fourier
BLASSO
4/14
![Page 27: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/27.jpg)
A seminal result [Candès, Fernandez-Granda 2012]
First Fourier coefficients
• Minimal separation
Inverse Fourier
BLASSO
4/14
![Page 28: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/28.jpg)
A seminal result [Candès, Fernandez-Granda 2012]
First Fourier coefficients
• Minimal separation
• Regular Fourier on the Torus
Inverse Fourier
BLASSO
4/14
![Page 29: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/29.jpg)
A seminal result [Candès, Fernandez-Granda 2012]
First Fourier coefficients
• Minimal separation
• Regular Fourier on the Torus
• Reconstruction: formulated as SDP (other: Frank-Wolfe,
greedy, Prony-like…)
Inverse Fourier
BLASSO
4/14
![Page 30: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/30.jpg)
A seminal result [Candès, Fernandez-Granda 2012]
First Fourier coefficients
• Minimal separation
• Regular Fourier on the Torus
• Reconstruction: formulated as SDP (other: Frank-Wolfe,
greedy, Prony-like…)
• Noise: weak convergence (mass of concentrated
around true Diracs)
Inverse Fourier
BLASSO
4/14
![Page 31: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/31.jpg)
A seminal result [Candès, Fernandez-Granda 2012]
First Fourier coefficients
• Minimal separation
• Regular Fourier on the Torus
• Reconstruction: formulated as SDP (other: Frank-Wolfe,
greedy, Prony-like…)
• Noise: weak convergence (mass of concentrated
around true Diracs)
Inverse Fourier
BLASSO
4/14
Randomness ? Numberof measurements ?
![Page 32: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/32.jpg)
Compressed sensing off-the-grid [Tang, Recht 2013]
5/14
![Page 33: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/33.jpg)
Compressed sensing off-the-grid [Tang, Recht 2013]
Random Fourier coefficients
random
5/14
![Page 34: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/34.jpg)
Compressed sensing off-the-grid [Tang, Recht 2013]
Random Fourier coefficients
BLASSO
random
5/14
![Page 35: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/35.jpg)
Compressed sensing off-the-grid [Tang, Recht 2013]
Random Fourier coefficients
• As in compressive sensing, randomFourier sampling is possible
But:• Limited to 1d Regular Fourier (relies
heavily on previous work by Candès)
• Random signs assumption
BLASSO
random
5/14
![Page 36: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/36.jpg)
Compressed sensing off-the-grid [Tang, Recht 2013]
Random Fourier coefficients
• As in compressive sensing, randomFourier sampling is possible
But:• Limited to 1d Regular Fourier (relies
heavily on previous work by Candès)
• Random signs assumption
BLASSO
Questions:
• More general sampling scheme ?
• Multi-dimensional result ?
• Get rid of random signs ?
random
5/14
![Page 37: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/37.jpg)
Outline
Background on dual certificates
Compressive off-the-grid recovery
Conclusion, outlooks
![Page 38: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/38.jpg)
Dual certificates
Measurements
Hilbert space
6/14
![Page 39: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/39.jpg)
Dual certificates
Measurements
Hilbert space
BLASSO
6/14
![Page 40: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/40.jpg)
Dual certificates
Measurements
Hilbert space
BLASSO
First-order conditions
solution ofBLASSO
6/14
![Page 41: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/41.jpg)
solution of
Dual certificates
Measurements
Hilbert space
BLASSO
First-order conditions
Dual certificate (noiseless case)
solution ofBLASSO
6/14
![Page 42: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/42.jpg)
What is a dual certificate ?
What does it look like ?
7/14
![Page 43: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/43.jpg)
What is a dual certificate ?
What does it look like ?
7/14
![Page 44: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/44.jpg)
What is a dual certificate ?
What does it look like ?
7/14
![Page 45: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/45.jpg)
Case :
What is a dual certificate ?
What does it look like ?
7/14
![Page 46: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/46.jpg)
Case :
What is a dual certificate ?
What does it look like ?
7/14
![Page 47: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/47.jpg)
Case :
What is a dual certificate ?
What does it look like ?
7/14
Ensures uniqueness and robustness…
Non-degenerate dual certif.
![Page 48: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/48.jpg)
Recovery results
8/14
![Page 49: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/49.jpg)
Recovery results
8/14
![Page 50: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/50.jpg)
Recovery results
Theorem (refinement of [Azaïs et al. 2015])
8/14
![Page 51: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/51.jpg)
Recovery results
Theorem (refinement of [Azaïs et al. 2015])
Hyp: there exists a ND dual certif.
8/14
![Page 52: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/52.jpg)
Recovery results
Theorem (refinement of [Azaïs et al. 2015])
Hyp: there exists a ND dual certif.Result: (wrt Bregman divergence)
- not necessarily sparse, but-
8/14
![Page 53: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/53.jpg)
Recovery results
Theorem (refinement of [Azaïs et al. 2015])
Hyp: there exists a ND dual certif.Result: (wrt Bregman divergence)
- not necessarily sparse, but- Mass of concentrated around
8/14
![Page 54: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/54.jpg)
Recovery results
Theorem (refinement of [Azaïs et al. 2015])
Hyp: there exists a ND dual certif.Result: (wrt Bregman divergence)
- not necessarily sparse, but- Mass of concentrated around- Concentration increases when:
noise
8/14
![Page 55: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/55.jpg)
Recovery results
Theorem (refinement of [Azaïs et al. 2015])
Hyp: there exists a ND dual certif.Result: (wrt Bregman divergence)
- not necessarily sparse, but- Mass of concentrated around- Concentration increases when:
noise
Theorem ([Duval Peyré 2015])
8/14
![Page 56: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/56.jpg)
Recovery results
Theorem (refinement of [Azaïs et al. 2015])
Hyp: there exists a ND dual certif.Result: (wrt Bregman divergence)
- not necessarily sparse, but- Mass of concentrated around- Concentration increases when:
noise
Theorem ([Duval Peyré 2015])
Hyp: the minimal norm certificate is non-degenerate
8/14
![Page 57: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/57.jpg)
Recovery results
Theorem (refinement of [Azaïs et al. 2015])
Hyp: there exists a ND dual certif.Result: (wrt Bregman divergence)
- not necessarily sparse, but- Mass of concentrated around- Concentration increases when:
noise
Theorem ([Duval Peyré 2015])
Hyp: the minimal norm certificate is non-degenerateResult: (in the small noise regime)
- is sparse, with the right number of components
-
8/14
![Page 58: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/58.jpg)
Outline
Background on dual certificates
Compressive off-the-grid recovery
Conclusion, outlooks
![Page 59: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/59.jpg)
Goal
Goal: random sampling
Low-dim, random
9/14
![Page 60: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/60.jpg)
Goal
Strategy: Start with « high »-dimensional problemGoal: random sampling
Low-dim, random Ex: full Fourier (high-dim), full convolution (infinite-dim)
9/14
![Page 61: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/61.jpg)
Goal
Strategy: Start with « high »-dimensional problemGoal: random sampling
Step 1: Build ND certificate with full kernel
Low-dim, random Ex: full Fourier (high-dim), full convolution (infinite-dim)
9/14
![Page 62: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/62.jpg)
Goal
Strategy: Start with « high »-dimensional problemGoal: random sampling
Step 1: Build ND certificate with full kernel
Low-dim, random Ex: full Fourier (high-dim), full convolution (infinite-dim)
9/14
![Page 63: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/63.jpg)
Goal
Strategy: Start with « high »-dimensional problemGoal: random sampling
Step 1: Build ND certificate with full kernel
Low-dim, random Ex: full Fourier (high-dim), full convolution (infinite-dim)
9/14
![Page 64: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/64.jpg)
Goal
Strategy: Start with « high »-dimensional problemGoal: random sampling
Step 1: Build ND certificate with full kernel
Step 2: Use Random Features on full kernel to define sampling
Low-dim, random Ex: full Fourier (high-dim), full convolution (infinite-dim)
9/14
![Page 65: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/65.jpg)
Goal
Strategy: Start with « high »-dimensional problemGoal: random sampling
Step 1: Build ND certificate with full kernel
Step 2: Use Random Features on full kernel to define sampling
Assuming
Define
Low-dim, random Ex: full Fourier (high-dim), full convolution (infinite-dim)
9/14
![Page 66: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/66.jpg)
Goal
Strategy: Start with « high »-dimensional problemGoal: random sampling
Step 1: Build ND certificate with full kernel
Step 2: Use Random Features on full kernel to define sampling
Assuming
Define
Then
Low-dim, random Ex: full Fourier (high-dim), full convolution (infinite-dim)
9/14
![Page 67: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/67.jpg)
Goal
Strategy: Start with « high »-dimensional problemGoal: random sampling
Step 1: Build ND certificate with full kernel
Step 2: Use Random Features on full kernel to define sampling
Assuming
Define
Then
Low-dim, random Ex: full Fourier (high-dim), full convolution (infinite-dim)
9/14
![Page 68: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/68.jpg)
Goal
Strategy: Start with « high »-dimensional problemGoal: random sampling
Step 1: Build ND certificate with full kernel
Step 2: Use Random Features on full kernel to define sampling
m=10
Assuming
Define
Then
Low-dim, random Ex: full Fourier (high-dim), full convolution (infinite-dim)
9/14
![Page 69: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/69.jpg)
Goal
Strategy: Start with « high »-dimensional problemGoal: random sampling
Step 1: Build ND certificate with full kernel
Step 2: Use Random Features on full kernel to define sampling
m=10 m=50
Assuming
Define
Then
Low-dim, random Ex: full Fourier (high-dim), full convolution (infinite-dim)
9/14
![Page 70: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/70.jpg)
Goal
Strategy: Start with « high »-dimensional problemGoal: random sampling
Step 1: Build ND certificate with full kernel
Step 2: Use Random Features on full kernel to define sampling
m=500m=10 m=50
Assuming
Define
Then
Low-dim, random Ex: full Fourier (high-dim), full convolution (infinite-dim)
9/14
![Page 71: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/71.jpg)
Step 1: Acceptable full kernels
10/14
![Page 72: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/72.jpg)
Step 1: Acceptable full kernels
10/14
![Page 73: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/73.jpg)
Step 1: Acceptable full kernels
10/14
![Page 74: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/74.jpg)
Step 1: Acceptable full kernels
10/14
![Page 75: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/75.jpg)
Step 1: Acceptable full kernels
Min. Separation
10/14
![Page 76: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/76.jpg)
Step 1: Acceptable full kernels
Min. Separation
1: kernel at each saturation point
10/14
![Page 77: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/77.jpg)
Step 1: Acceptable full kernels
Min. Separation
2: Small adjustements: minimal separation
1: kernel at each saturation point
10/14
![Page 78: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/78.jpg)
Step 1: Acceptable full kernels
Min. Separation
2: Small adjustements: minimal separation
- Multi-d square Féjer kernel (regular Fourier on Torus)
- Multi-d Gaussian kernel
1: kernel at each saturation point
10/14
![Page 79: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/79.jpg)
Step 2: sampling
Thm: Ideal scaling in sparsity: infinite-dim. golfing scheme
11/14
![Page 80: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/80.jpg)
Step 2: sampling
Thm: Ideal scaling in sparsity: infinite-dim. golfing scheme
Depend on kernel
11/14
![Page 81: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/81.jpg)
Step 2: sampling
Thm: Ideal scaling in sparsity: infinite-dim. golfing scheme
• There exists a ND dual certificate (however not the minimal norm certificate)
Depend on kernel
11/14
![Page 82: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/82.jpg)
Step 2: sampling
Thm: Ideal scaling in sparsity: infinite-dim. golfing scheme
• There exists a ND dual certificate (however not the minimal norm certificate)
• No need for random signs
Depend on kernel
11/14
![Page 83: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/83.jpg)
Step 2: sampling
Thm: Ideal scaling in sparsity: infinite-dim. golfing scheme
• There exists a ND dual certificate (however not the minimal norm certificate)
• No need for random signs
• Proof: golfing scheme [Gross 2009, Candès Plan 2011]
Depend on kernel
11/14
![Page 84: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/84.jpg)
Step 2: sampling
Thm: Ideal scaling in sparsity: infinite-dim. golfing scheme
• There exists a ND dual certificate (however not the minimal norm certificate)
• No need for random signs
• Proof: golfing scheme [Gross 2009, Candès Plan 2011]
Thm: Minimal norm certificate (adaptation of [Tang, Recht 2013])
Depend on kernel
With random signs Without random signs
11/14
![Page 85: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/85.jpg)
Step 2: sampling
Thm: Ideal scaling in sparsity: infinite-dim. golfing scheme
• There exists a ND dual certificate (however not the minimal norm certificate)
• No need for random signs
• Proof: golfing scheme [Gross 2009, Candès Plan 2011]
Thm: Minimal norm certificate (adaptation of [Tang, Recht 2013])
Depend on kernel
With random signs Without random signs
11/14
Fun application: convex approach for automatic estimation of number of components in a GMM
![Page 86: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/86.jpg)
Number of measurements in practice ?
Nicolas Keriven
Compressive k-means [Keriven 2017]
Relative number of measurements m/(sd)
12/14
![Page 87: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/87.jpg)
Outline
Background on dual certificates
Compressive off-the-grid recovery
Conclusion, outlooks
![Page 88: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/88.jpg)
Summary, outlooks
• Summary: generalization of existing results on super-resolutionwith random measurements (and minimal separation)
• Beyond Fourier on the Torus (« acceptable » kernels)• Multi-d• No need for random signs for basic recovery result• Support recovery when random signs (or quadratic number of
measurements)
13/14
![Page 89: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/89.jpg)
Summary, outlooks
• Summary: generalization of existing results on super-resolutionwith random measurements (and minimal separation)
• Beyond Fourier on the Torus (« acceptable » kernels)• Multi-d• No need for random signs for basic recovery result• Support recovery when random signs (or quadratic number of
measurements)
13/14
• Outlooks• Other kernels, very different from translation-invariant• More quantified treatment of dimension• Other practical applications (eg 1-layer neural networks with continuum of
neurons [Bach 2017])
![Page 90: A dual certificate analysis of compressive off-the-grid recoveryA dual certificate analysis of compressive off-the-grid recovery Nicolas Keriven Ecole Normale Supérieure (Paris) CFM-ENS](https://reader033.fdocuments.us/reader033/viewer/2022060604/6058ea4f55dfa575bb2bd240/html5/thumbnails/90.jpg)
Poon, Keriven, Peyré. A Dual Certificates Analysis of Compressive Off-the-Grid Recovery. Preprint arxiv:1802.08464
Code: sketchml.gforge.inria.fr,github: nkeriven
Thank you !
14/14
data-ens.github.ioEnter the data challenges!Come to the colloquium!Come to the Laplace seminars!