Theoretical study of steerable homogeneous operatorsAnd applications to sparse stochastic processes

Presentation ndash End of my Internship

Lilian Besson

Advisors Michael Unser and Julien Fageot

Eacutecole polytechnique feacutedeacuterale de LausanneENS de Cachan (Master MVA)

August 26st 2016 | Time 40 minutes

E-mail lilianbessonens-cachanfrOpen-source httplbokvuepfl2016

Grade I got 2020 for my internship

Introduction amp Motivations 01 Subject of my internship

Subject

Functional operators

Mainly about convolution operators 119866(= linear + continuous + translation-invariant ldquoLSIrdquo)

Steerable and homogeneous convolutions

ndash More freedom than if rotation-invariantndash But still easily parametrized

Applications and experiments

Mainly on sparse stochastic processes in 2D

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 1 39

Introduction amp Motivations 01 Subject of my internship

Subject

Functional operators

Mainly about convolution operators 119866(= linear + continuous + translation-invariant ldquoLSIrdquo)

Steerable and homogeneous convolutions

ndash More freedom than if rotation-invariantndash But still easily parametrized

Applications and experiments

Mainly on sparse stochastic processes in 2D

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 1 39

Introduction amp Motivations 02 Motivations

Apply our operators 119866 to

Sparse processes [Unser and Tafti 2014]

ndash To visualize their effectsndash To generate new processesndash and see pretty images

Splines [Unser et al 2016]

ndash One operatorlArrrArr one spline

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 2 39

Introduction amp Motivations 02 Motivations

Apply our operators 119866 to

Sparse processes [Unser and Tafti 2014]

ndash To visualize their effectsndash To generate new processesndash and see pretty images

Splines [Unser et al 2016]

ndash One operatorlArrrArr one spline

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 2 39

Introduction amp Motivations 03 Outline

Outline

1 Reminders on operators theory

2 Steerable operators

3 Scale-invariance for steerable convolutions

4 Decompositions of steerable convolutions

5 Illustrations on sparse stochastic processes

6 Conclusion amp Appendix

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 3 39

1 Reminders on operators theory

1 Reminders on operators theory

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 4 39

1 Reminders on operators theory 11 Reminders on operators

What are operators

An operator 119866

Takes a function 119891 transforms it to another function 119866119891For real-valued 119891 119866 has to give a real-valued 119866119891

Examples in maths

ndash Derivatives 119863119909 and 119863119910 Laplacian Δndash Denoising contour detection etc

Examples in real life

ndash ldquoFiltersrdquo for photos in Instagram or Facebook ndash Rotating photos etc

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 4 39

1 Reminders on operators theory 11 Reminders on operators

What are operators

An operator 119866

Takes a function 119891 transforms it to another function 119866119891For real-valued 119891 119866 has to give a real-valued 119866119891

Examples in maths

ndash Derivatives 119863119909 and 119863119910 Laplacian Δndash Denoising contour detection etc

Examples in real life

ndash ldquoFiltersrdquo for photos in Instagram or Facebook ndash Rotating photos etc

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 4 39

1 Reminders on operators theory 11 Reminders on operators

Different properties for operators

Today our operators 119866 are alwaysndash Linearndash Continuousndash In 2D 119866 119891(119909 119910) ↦rarr 119866119891(119909 119910)ndash Translation-invariance

Geometric properties [Unser and Tafti 2014]

ndash Scale-invariance or 120574-scale-invariance (= homogeneity)ndash Rotation-invariancendash Steerable

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 5 39

1 Reminders on operators theory 11 Reminders on operators

Different properties for operators

Today our operators 119866 are alwaysndash Linearndash Continuousndash In 2D 119866 119891(119909 119910) ↦rarr 119866119891(119909 119910)ndash Translation-invariance

Geometric properties [Unser and Tafti 2014]

ndash Scale-invariance or 120574-scale-invariance (= homogeneity)ndash Rotation-invariancendash Steerable

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 5 39

1 Reminders on operators theory 12 Schwartz theorem for convolution operators

Schwartz theorem and impulse response

Schwartz convolution theorem [Stein and Weiss 1971]Translation-invariant linear continuous operators are exactlyconvolution operators

119866119891 = (119892 119891)

Impulse response 119892 of 119866

119892 = 1198661205750 is a distribution (= generalized function)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 6 39

1 Reminders on operators theory 12 Schwartz theorem for convolution operators

Schwartz theorem and impulse response

Schwartz convolution theorem [Stein and Weiss 1971]Translation-invariant linear continuous operators are exactlyconvolution operators

119866119891 = (119892 119891)

Impulse response 119892 of 119866

119892 = 1198661205750 is a distribution (= generalized function)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 6 39

1 Reminders on operators theory 12 Schwartz theorem for convolution operators

Fourier multiplier 119892 of an operator 119866

Using the Fourier transform ℱ [Stein and Weiss 1971]

ℱ transforms a convolution (119892 119891) to a point-wise productSo

ℱ119866119891

= ℱ119892 119891 = ℱ119892 middot ℱ119891 = 119892 middot 119891

Fourier multiplier 119892And so

119866119891 = ℱminus1119892 middot 119891

119892 = ℱ119892 is a complex-valued function

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 7 39

1 Reminders on operators theory 12 Schwartz theorem for convolution operators

Using the Fourier multiplier 119892Working in the ldquoFourier domainrdquo

Output function119866119891(119909 119910)

Output in Fourier119892(119903 120579) middot 119891(119903 120579)

Input in Fourier119891(119903 120579)Input function

119891(119909 119910)ℱ

Point-wisemultiplicationby 119892(119903 120579)

ℱminus1

In 2D the Fourier variable 120596 is written in polar coordinates 120596 = (119903 120579)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 8 39

1 Reminders on operators theory 13 Geometric properties and characterizations

Property scale-invariance

DefinitionFor 120574 gt 0 119866 is 120574-scale-invariant when

119866119891([119909119886119910119886

]) = 119886120574119866119891(middot119886)(

[119909119910

]) forall scaling 119886 gt 0

Easy characterization on 119892119892(119886119903 120579) = 119886120574 119892(119903 120579) forall119886 gt 0

Separable form for 119892ndash 119892(119903 120579) = 119903120574 119892(1 120579)ndash 119892(1 120579) only depends on the angle 120579

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 9 39

1 Reminders on operators theory 13 Geometric properties and characterizations

Property scale-invariance

DefinitionFor 120574 gt 0 119866 is 120574-scale-invariant when

119866119891([119909119886119910119886

]) = 119886120574119866119891(middot119886)(

[119909119910

]) forall scaling 119886 gt 0

Easy characterization on 119892119892(119886119903 120579) = 119886120574 119892(119903 120579) forall119886 gt 0

Separable form for 119892ndash 119892(119903 120579) = 119903120574 119892(1 120579)ndash 119892(1 120579) only depends on the angle 120579

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 9 39

1 Reminders on operators theory 13 Geometric properties and characterizations

Example 1 derivatives

Derivatives and directional derivativesndash Usual derivatives 119863119909 and 119863119910

ndash Directional derivative 119863120572def= cos(120572)119863119909 + sin(120572)119863119910

They are 1-scale-invariant

Because their Fourier multipliers are

ndash 119863119909(119903 120579) = 119895119909 = 119895119903 cos(120579)ndash 119863119910(119903 120579) = 119895119910 = 119895119903 sin(120579)ndash 119863120572(119903 120579) = 119895119903 cos(120579 minus 120572)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 10 39

1 Reminders on operators theory 13 Geometric properties and characterizations

Property rotation-invariance

Definition119866 is rotation-invariant when

119866119891(1198771205790

[119909119910

]) = 119866119891(1198771205790middot)(

[119909119910

]) forall rotation 1198771205790

Easy characterization on 119892119892(119903 120579 + 1205790) = 119892(119903 120579) forall rotation angle 1205790

Purely radial 119892119892(119903 120579) = 119892(119903 0) only depends on the radius 119903 119892 is purely radial

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 11 39

1 Reminders on operators theory 13 Geometric properties and characterizations

Property rotation-invariance

Definition119866 is rotation-invariant when

119866119891(1198771205790

[119909119910

]) = 119866119891(1198771205790middot)(

[119909119910

]) forall rotation 1198771205790

Easy characterization on 119892119892(119903 120579 + 1205790) = 119892(119903 120579) forall rotation angle 1205790

Purely radial 119892119892(119903 120579) = 119892(119903 0) only depends on the radius 119903 119892 is purely radial

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 11 39

1 Reminders on operators theory 13 Geometric properties and characterizations

Example 2 fractional Laplacians

Laplacian and fractional Laplacians [Unser and Tafti 2014]

For 120574 gt 0 (minusΔ)1205742 has a Fourier multiplier 119892(119903 120579) = 119903120574

So they are

ndash 120574-scale-invariantndash and rotation-invariant

=rArr Simplest example of 120574-SI and RI operators

Theorem 120574-SI + RIhArr Laplacian [Th339 of my report]

In fact

(minusΔ)1205742 is the only 120574-SI and RI convolution

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 12 39

1 Reminders on operators theory 13 Geometric properties and characterizations

Example 2 fractional Laplacians

Laplacian and fractional Laplacians [Unser and Tafti 2014]

For 120574 gt 0 (minusΔ)1205742 has a Fourier multiplier 119892(119903 120579) = 119903120574

So they are

ndash 120574-scale-invariantndash and rotation-invariant

=rArr Simplest example of 120574-SI and RI operators

Theorem 120574-SI + RIhArr Laplacian [Th339 of my report]

(minusΔ)1205742 is the only 120574-SI and RI convolution

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 12 39

2 Steerable operators

2 Steerable operators

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 13 39

2 Steerable operators 21 Definition of steerability

Steerable convolution operators

Definition [Vonesch et al 2015][Unser and Chenouard 2013]

119866 is steerable when

dim 119881119892 = dim Span1205790isin[02120587]

(119903 120579) ↦rarr 119892(119903 120579 + 1205790)

is finite

Order of steerability

119899119866def= dim 119881119892 isin N

Example Null-operator 119866 = 0hArr 119899119866 = 0

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 13 39

2 Steerable operators 21 Definition of steerability

Steerable convolution operators

Definition [Vonesch et al 2015][Unser and Chenouard 2013]

119866 is steerable when

dim 119881119892 = dim Span1205790isin[02120587]

(119903 120579) ↦rarr 119892(119903 120579 + 1205790)

is finite

Order of steerability

119899119866def= dim 119881119892 isin N

Example Null-operator 119866 = 0hArr 119899119866 = 0

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 13 39

2 Steerable operators 21 Definition of steerability

Steerability generalizes rotation-invariance

Theorem [Th422 of my report]

Non-zero steerable of order 119899119866 = 1hArr rotation-invariant

Just a sanity check

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 14 39

2 Steerable operators 22 Characterization for 2D steerable convolutions

First characterization of 2D steerable convolutions

Theorem Hard [Vonesch et al 2015]119866 is a 2D steerable convolutionhArrthere exists an integer 119870119866 and functions 120588119896 R+ rarr C such that

119892(119903 120579) =sum

minus119870119866le119896le119870119866

120588119896(119903) e119895119896120579

ldquoMax frequencyrdquo 119870119866

119870119866 isin N is unique for non-zero 119866

Still too general

The radial functions 120588119896(119903) are completely unspecified

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 15 39

2 Steerable operators 22 Characterization for 2D steerable convolutions

First characterization of 2D steerable convolutions

Theorem Hard [Vonesch et al 2015]119866 is a 2D steerable convolutionhArrthere exists an integer 119870119866 and functions 120588119896 R+ rarr C such that

119892(119903 120579) =sum

minus119870119866le119896le119870119866

120588119896(119903) e119895119896120579

ldquoMax frequencyrdquo 119870119866

119870119866 isin N is unique for non-zero 119866

Still too general

The radial functions 120588119896(119903) are completely unspecified

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 15 39

2 Steerable operators 22 Characterization for 2D steerable convolutions

First characterization of 2D steerable convolutions

Theorem Hard [Vonesch et al 2015]119866 is a 2D steerable convolutionhArrthere exists an integer 119870119866 and functions 120588119896 R+ rarr C such that

119892(119903 120579) =sum

minus119870119866le119896le119870119866

120588119896(119903) e119895119896120579

ldquoMax frequencyrdquo 119870119866

119870119866 isin N is unique for non-zero 119866

Still too general

The radial functions 120588119896(119903) are completely unspecified

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 15 39

3 Scale-invariance for steerable convolutions

3 Scale-invariance for steerable convolutions

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 16 39

3 Scale-invariance for steerable convolutions 31 Scale-invariance and steerability

Steerable scale-invariant convolutions

What does scale-invariance adds [Th438 of my report]

=rArr 120588119896(119903) = 119886119896119903120574 forall119896

And so 119866 is 120574-scale-invariant and steerable

hArr 119892(119903 120579) = 119903120574sum

minus119870119866le119896le119870119866

119886119896e119895119896120579

=rArr Separable form between 119903 and 120579 great

Strong result with an easy proof

On the white board Z

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 16 39

3 Scale-invariance for steerable convolutions 31 Scale-invariance and steerability

Steerable scale-invariant convolutions

What does scale-invariance adds [Th438 of my report]

=rArr 120588119896(119903) = 119886119896119903120574 forall119896

And so 119866 is 120574-scale-invariant and steerable

hArr 119892(119903 120579) = 119903120574sum

minus119870119866le119896le119870119866

119886119896e119895119896120579

=rArr Separable form between 119903 and 120579 great

Strong result with an easy proof

On the white board ZLilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 16 39

3 Scale-invariance for steerable convolutions 31 Scale-invariance and steerability

Steerable scale-invariant convolutions

What does scale-invariance adds [Th438 of my report]

=rArr 120588119896(119903) = 119886119896119903120574 forall119896

And so 119866 is 120574-scale-invariant and steerable

hArr 119892(119903 120579) = 119903120574sum

minus119870119866le119896le119870119866

119886119896e119895119896120579

=rArr Separable form between 119903 and 120579 great

Parametrization of steerable 120574-SI convolutions new With 119870119866 isin N and 2119870119866 + 1 complex parameters 119886119896 = 120588119896(1)And their polar part is just a trigonometric polynomial (in e119895120579)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 16 39

3 Scale-invariance for steerable convolutions 32 And for real convolution operators

Hermitian-symmetric Fourier multiplier

Hermitian-symmetric 119892 [Stein and Weiss 1971]

119866 is realhArr forall119891 119866119891 is real-valuedhArr 119892 is Hermitian-symmetric

119892(119903 120579 + 120587) = 119892(119903 120579)

Consequence on the coefficients (119886119896)

119886minus119896 = (minus1)119896119886119896forall119896

Parametrization better =rArr With 119870119866 isin N and parameters 1198860 119886119870119866

only

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 17 39

3 Scale-invariance for steerable convolutions 32 And for real convolution operators

Hermitian-symmetric Fourier multiplier

Hermitian-symmetric 119892 [Stein and Weiss 1971]

119866 is realhArr forall119891 119866119891 is real-valuedhArr 119892 is Hermitian-symmetric

119892(119903 120579 + 120587) = 119892(119903 120579)

Consequence on the coefficients (119886119896)

119886minus119896 = (minus1)119896119886119896forall119896

Parametrization better =rArr With 119870119866 isin N and parameters 1198860 119886119870119866

only

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 17 39

3 Scale-invariance for steerable convolutions 33 Important examples differential operators

Example with 119899119866 = 1 the fractional Laplacian

119866 = (minusΔ)1205742 is steerable of order 119899119866 = 1

Fourier multiplier119892(119903 120579) = 119903120574 is Hermitian-symmetric RI and 120574-SI

With our parametrization

With 119870119866 = 0 and 1198860 = 1Check 1198860 = (minus1)01198860

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 18 39

3 Scale-invariance for steerable convolutions 33 Important examples differential operators

Example with 119899119866 = 2 directional derivatives

119866 = 119863120572 is steerable of order 119899119866 = 2

Fourier multiplier [Chaudhury and Unser 2010]119892(119903 120579) = 119895119903 cos(120579 minus 120572) is Hermitian-symmetric not RI and 1-SI

With our parametrization

With 119870119866 = 1 and 119886minus1 = eminus1198951205722 1198860 = 0 1198861 = e1198951205722Check 119886minus1 = (minus1)11198861

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 19 39

3 Scale-invariance for steerable convolutions 33 Important examples differential operators

Example with 119899119866 = 2 directional derivatives

119866 = 119863120572 is steerable of order 119899119866 = 2

Fourier multiplier [Chaudhury and Unser 2010]119892(119903 120579) = 119895119903 cos(120579 minus 120572) is Hermitian-symmetric not RI and 1-SI

Sanity check

And it has 119899119866 = 2Good steerability is more general than rotation-invariant

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 19 39

3 Scale-invariance for steerable convolutions 33 Important examples differential operators

Example with 119899119866 = 3 the ldquoMondrianrdquo 119863119909119863119910

119866 = 119863119909119863119910 is steerable of order 119899119866 = 3

Fourier multiplier119892(119903 120579) = minus1199032 cos(120579) sin(120579) is Hermitian-sym not RI and 2-SI

With our parametrization

With 119870119866 = 2 and 119886minus2 = minus1198954 119886minus1 = 1198860 = 1198861 = 0 1198862 = 1198954Check 119886minus2 = (minus1)21198862

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 20 39

4 Decompositions of steerable convolutions

4 Decompositions of steerable convolutions

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 21 39

4 Decompositions of steerable convolutions 41 Decomposition as a sum not so great

First decomposition with a trigonometric polynomial

Steerable 120574-SI convolutions [Th438]

119892(119903 120579) = 119903120574sum

minus119870119866le119896le119870119866

119886119896e119895119896120579

Already interesting and useful

ndash Simple parametrizationndash Easy to implement

ButB Sums are not easy to invert if we want 119866minus1

Can we do better Yes we can

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 21 39

4 Decompositions of steerable convolutions 41 Decomposition as a sum not so great

First decomposition with a trigonometric polynomial

Steerable 120574-SI convolutions [Th438]

119892(119903 120579) = 119903120574sum

minus119870119866le119896le119870119866

119886119896e119895119896120579

Already interesting and useful

ndash Simple parametrizationndash Easy to implement

ButB Sums are not easy to invert if we want 119866minus1

Can we do better Yes we can

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 21 39

4 Decompositions of steerable convolutions 42 First decomposition as a product

Decomposition as a product with 1198660

Theorem Partly factorized decomposition [Th449]

119866 is a 2D steerable 120574-scale-invariant convolutions

hArr 119866 prop (minusΔ)(120574minus119898)2 ∘1198631205721 ∘ middot middot middot ∘119863120572119898 ∘1198660

ndash 1198660 invertible and 0-SIndash With 1198920(119903 120579) = 0 a trigonometric polynomial of degree 119870119866 minus119898

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 22 39

4 Decompositions of steerable convolutions 42 First decomposition as a product

Decomposition as a product with 1198660

Theorem Partly factorized decomposition [Th449]

119866 is a 2D steerable 120574-scale-invariant convolutions

hArr 119866 prop (minusΔ)(120574minus119898)2 ∘1198631205721 ∘ middot middot middot ∘119863120572119898 ∘1198660

ndash 1198660 invertible and 0-SIndash With 1198920(119903 120579) = 0 a trigonometric polynomial of degree 119870119866 minus119898

Great ndash Now we can think of inverting 119892(119903 120579) ndash If we know how to inverse (minusΔ)(120574minus119898)2 and 119863120572119894

=rArr We can use 119866minus1

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 22 39

4 Decompositions of steerable convolutions 42 First decomposition as a product

Decomposition as a product with 1198660

Sketch of the proof Long cf p60-63 of my reportWork on the roots 119911119894 isin 119885119875 of the complex trigonometric polynomial119875 (e119895120579) = e119895119870119866120579 sum

119886119896(e119895120579)119896 = 119886119870119866

prod119911119894

(e119895120579 minus 119911119894)By Hermitian symmetry 119911119894 isin 119885119875 hArrminus1119911119894 isin 119885119875

Let 119911119894 = 119903119894e119895120573119894 Zndash 0 is not a root 119903119894 = 0

ndash Some roots are on the unit circle 119903119894 = 1e119895120573119894 and minus1e119895120573119894 = minuse119895120573119894 = e119895(120573119894+120587)

Group them by pairs add a 119903 term (and some computations)=rArr gives a derivative 119863120573119894+ 120587

2

ndash For the roots of modulus 119903119894 = 1 (e119895120579 minus 119911119894) is non-canceling=rArr collect all of them in 1198920(119903 120579)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 22 39

4 Decompositions of steerable convolutions 42 First decomposition as a product

Decomposition as a product with 1198660

Sketch of the proof Long cf p60-63 of my reportWork on the roots 119911119894 isin 119885119875 of the complex trigonometric polynomial119875 (e119895120579) = e119895119870119866120579 sum

119886119896(e119895120579)119896 = 119886119870119866

prod119911119894

(e119895120579 minus 119911119894)By Hermitian symmetry 119911119894 isin 119885119875 hArrminus1119911119894 isin 119885119875

Let 119911119894 = 119903119894e119895120573119894 Zndash 0 is not a root 119903119894 = 0ndash Some roots are on the unit circle 119903119894 = 1

e119895120573119894 and minus1e119895120573119894 = minuse119895120573119894 = e119895(120573119894+120587)

Group them by pairs add a 119903 term (and some computations)=rArr gives a derivative 119863120573119894+ 120587

2

ndash For the roots of modulus 119903119894 = 1 (e119895120579 minus 119911119894) is non-canceling=rArr collect all of them in 1198920(119903 120579)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 22 39

4 Decompositions of steerable convolutions 42 First decomposition as a product

Decomposition as a product with 1198660

Sketch of the proof Long cf p60-63 of my reportWork on the roots 119911119894 isin 119885119875 of the complex trigonometric polynomial119875 (e119895120579) = e119895119870119866120579 sum

119886119896(e119895120579)119896 = 119886119870119866

prod119911119894

(e119895120579 minus 119911119894)By Hermitian symmetry 119911119894 isin 119885119875 hArrminus1119911119894 isin 119885119875

Let 119911119894 = 119903119894e119895120573119894 Zndash 0 is not a root 119903119894 = 0ndash Some roots are on the unit circle 119903119894 = 1

e119895120573119894 and minus1e119895120573119894 = minuse119895120573119894 = e119895(120573119894+120587)

Group them by pairs add a 119903 term (and some computations)=rArr gives a derivative 119863120573119894+ 120587

2

ndash For the roots of modulus 119903119894 = 1 (e119895120579 minus 119911119894) is non-canceling=rArr collect all of them in 1198920(119903 120579)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 22 39

4 Decompositions of steerable convolutions 42 First decomposition as a product

Decomposition as a product with 1198660

Theorem Partly factorized decomposition [Th449]

119866 is a 2D steerable 120574-scale-invariant convolutions

hArr 119866 prop (minusΔ)(120574minus119898)2 ∘1198631205721 ∘ middot middot middot ∘119863120572119898 ∘1198660

ndash 1198660 invertible and 0-SIndash With 1198920(119903 120579) = 0 a trigonometric polynomial of degree 119870119866 minus119898

Great But can we do even better Yes we can

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 22 39

4 Decompositions of steerable convolutions 43 Decomposition as product of elementary blocks 119866120582120572

Decomposition as product of elementary blocks 119866120582120572

Can we do better Sure Finish the factorization of the trigonometric polynomial

Finishing the proof

The roots of modulus 119903119894 = 1 can also be grouped by pairs (e119895120579 minus 119911119894)(e119895120579 + 1119911119894) = Let 119911119894 = 119903119894e119895120573119894 we obtain

= 2119895 cos(120579 minus (120573119894 minus 1205872)) + (1119903119894 minus 119903119894)⏟ ⏞ = 0

= 0

Adding a 119903 gives a convex combination of 119863120572119894and (minusΔ)12

119903(2119895 cos(120579minus120572119894)+(1119903119894 minus 119903119894)

)prop

(120582119894

119863120572119894(119903 120579)+(1minus 120582119894) (minusΔ)12(119903 120579)

)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 23 39

4 Decompositions of steerable convolutions 43 Decomposition as product of elementary blocks 119866120582120572

Decomposition as product of elementary blocks 119866120582120572

Can we do better Sure Finish the factorization of the trigonometric polynomial

Finishing the proof

The roots of modulus 119903119894 = 1 can also be grouped by pairs (e119895120579 minus 119911119894)(e119895120579 + 1119911119894) = Let 119911119894 = 119903119894e119895120573119894 we obtain

= 2119895 cos(120579 minus (120573119894 minus 1205872)) + (1119903119894 minus 119903119894)⏟ ⏞ = 0

= 0

Adding a 119903 gives a convex combination of 119863120572119894and (minusΔ)12

119903(2119895 cos(120579minus120572119894)+(1119903119894 minus 119903119894)

)prop

(120582119894

119863120572119894(119903 120579)+(1minus 120582119894) (minusΔ)12(119903 120579)

)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 23 39

4 Decompositions of steerable convolutions 43 Decomposition as product of elementary blocks 119866120582120572

Decomposition as product of elementary blocks 119866120582120572

Can we do better Sure Finish the factorization of the trigonometric polynomial

Finishing the proof

The roots of modulus 119903119894 = 1 can also be grouped by pairs (e119895120579 minus 119911119894)(e119895120579 + 1119911119894) = Let 119911119894 = 119903119894e119895120573119894 we obtain

= 2119895 cos(120579 minus (120573119894 minus 1205872)) + (1119903119894 minus 119903119894)⏟ ⏞ = 0

= 0

Adding a 119903 gives a convex combination of 119863120572119894and (minusΔ)12

119903(2119895 cos(120579minus120572119894)+(1119903119894 minus 119903119894)

)prop

(120582119894

119863120572119894(119903 120579)+(1minus 120582119894) (minusΔ)12(119903 120579)

)Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 23 39

4 Decompositions of steerable convolutions 43 Decomposition as product of elementary blocks 119866120582120572

Decomposition as product of elementary blocks 119866120582119894120572119894

Theorem Fully factorized decomposition [Th51 of my report]

119866 is a 2D steerable 120574-scale-invariant convolution

hArr 119866 prop (minusΔ)(120574minus119870119866)2 ∘119870119866

119894=1

(120582119894119863120572119894 + (1minus 120582119894)(minusΔ)12)⏟ ⏞

def= 119866120582119894120572119894

ndash With convex weights 1205821 120582119870119866isin (0 1]

ndash And angles 1205721 120572119870119866isin [0 2120587]

ndash 119866120582119894120572119894is a 1-SI convolution steerable of order 119899119866 le 2 and 119870119866 le 1

Great ndash Simpler parametrizationndash Now we can more easily invert 119892(119903 120579) (for 119903 = 0)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 24 39

4 Decompositions of steerable convolutions 43 Decomposition as product of elementary blocks 119866120582120572

Decomposition as product of elementary blocks 119866120582119894120572119894

Theorem Fully factorized decomposition [Th51 of my report]

119866 is a 2D steerable 120574-scale-invariant convolution

hArr 119866 prop (minusΔ)(120574minus119870119866)2 ∘119870119866

119894=1

(120582119894119863120572119894 + (1minus 120582119894)(minusΔ)12)⏟ ⏞

def= 119866120582119894120572119894

ndash With convex weights 1205821 120582119870119866isin (0 1]

ndash And angles 1205721 120572119870119866isin [0 2120587]

ndash 119866120582119894120572119894is a 1-SI convolution steerable of order 119899119866 le 2 and 119870119866 le 1

Natural interpretation

ndash 120572 is the direction of the derivativendash 120582 is a ldquodegree of directionalityrdquo

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 24 39

4 Decompositions of steerable convolutions 43 Decomposition as product of elementary blocks 119866120582120572

Decomposition as product of elementary blocks 119866120582119894120572119894

Theorem Fully factorized decomposition [Th51 of my report]

119866 is a 2D steerable 120574-scale-invariant convolution

hArr 119866 prop (minusΔ)(120574minus119870119866)2 ∘119870119866

119894=1

(120582119894119863120572119894 + (1minus 120582119894)(minusΔ)12)⏟ ⏞

def= 119866120582119894120572119894

ndash With convex weights 1205821 120582119870119866isin (0 1]

ndash And angles 1205721 120572119870119866isin [0 2120587]

ndash 119866120582119894120572119894is a 1-SI convolution steerable of order 119899119866 le 2 and 119870119866 le 1

Great ndash Simpler parametrizationndash Now we can more easily invert 119892(119903 120579) (for 119903 = 0)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 24 39

5 Illustrations on sparse stochastic processes

5 Illustrations on sparse stochastic processes

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 25 39

5 Illustrations on sparse stochastic processes 51 Computing 119866minus1

Computing 119866minus1120582120572

Fourier multiplier of 119866minus1120582120572

Obvious but maybe not always well defined

1119892120582120572(119903 120579)

Impulse response of 119866minus1120582120572

120588119866120582120572= ℱminus1

1119892120582120572(119903 120579)

ndash Known for Laplacians and derivatives (120582 = 0 or 1) ndash Harder for our ldquopartly directionalrdquo block 119866120582120572 (0 lt 120582 lt 1)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 25 39

5 Illustrations on sparse stochastic processes 51 Computing 119866minus1

Computing 119866minus1120582120572

Fourier multiplier of 119866minus1120582120572

Obvious but maybe not always well defined

1119892120582120572(119903 120579)

Impulse response of 119866minus1120582120572

120588119866120582120572= ℱminus1

1119892120582120572(119903 120579)

ndash Known for Laplacians and derivatives (120582 = 0 or 1) ndash Harder for our ldquopartly directionalrdquo block 119866120582120572 (0 lt 120582 lt 1)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 25 39

5 Illustrations on sparse stochastic processes 51 Implementing 119866minus1

Implementing 119866minus1120582120572

Implementation

ndash In the discrete Fourier domainndash ldquoSimplerdquo point-wise division by

119892120582120572[mn] = 119903[mn](120582119895 cos(120579[mn]minus 120572) + (1minus 120582)

)ndash Not too hard to implement With Virginie

But ndash Discretization errors fft2 = ℱ and ifft2 = ℱminus1

ndash Our choice if 119892120582120572[mn] = 0 force it = 1G_inv(isinf(G_inv)) = 1

=rArr Approximations

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 26 39

5 Illustrations on sparse stochastic processes 51 Implementing 119866minus1

Implementing 119866minus1120582120572

Implementation

ndash In the discrete Fourier domainndash ldquoSimplerdquo point-wise division by

119892120582120572[mn] = 119903[mn](120582119895 cos(120579[mn]minus 120572) + (1minus 120582)

)ndash Not too hard to implement With Virginie

But ndash Discretization errors fft2 = ℱ and ifft2 = ℱminus1

ndash Our choice if 119892120582120572[mn] = 0 force it = 1G_inv(isinf(G_inv)) = 1

=rArr Approximations

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 26 39

5 Illustrations on sparse stochastic processes 51 Implementing 119866minus1

Implementing 119866minus1120582120572 with fft2 and ifft2

Apply 119866minus1120582120572 to a real 2D image 119891[mn]

Output image119866minus1

120582120572119891[mn]Output in Fourier119891[mn] 119892120582120572[mn]

Input in Fourier119891[mn]Input image

119891[mn]fft2

119892120582120572[mn]

ifft2

B Small approximation error if 119892120582120572[mn] can be zero

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 27 39

5 Illustrations on sparse stochastic processes 53 Sparse stochastic processes

Gaussian and Poisson-Gaussian sparse processes

Quick reminders [Unser and Tafti 2014]Two examples of realizations

(a) Gaussian white noise(iid Gaussian on every pixels)

(b) Compound Poisson-Gaussian(iid Gaussian on the ldquofiringrdquo pixels)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 28 39

5 Illustrations on sparse stochastic processes 54 Illustrations with one inverse block

One block 119866minus11205820 increasing 120582

Example 1 with 120572 = 0 and 120582 = 0 025 05 075 1

(a) On a Gaussian white noise (b) On a ldquolow-firingrdquo Poisson

Figure 119866minus100 purely isotropic

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 29 39

5 Illustrations on sparse stochastic processes 54 Illustrations with one inverse block

One block 119866minus11205820 increasing 120582

Example 1 with 120572 = 0 and 120582 = 0 025 05 075 1

(a) On a Gaussian white noise (b) On a ldquolow-firingrdquo Poisson

Figure 119866minus10250 not yet directional

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 29 39

5 Illustrations on sparse stochastic processes 54 Illustrations with one inverse block

One block 119866minus11205820 increasing 120582

Example 1 with 120572 = 0 and 120582 = 0 025 05 075 1

(a) On a Gaussian white noise (b) On a ldquolow-firingrdquo Poisson

Figure 119866minus10500 not much directional

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 29 39

5 Illustrations on sparse stochastic processes 54 Illustrations with one inverse block

One block 119866minus11205820 increasing 120582

Example 1 with 120572 = 0 and 120582 = 0 025 05 075 1

(a) On a Gaussian white noise (b) On a ldquolow-firingrdquo Poisson

Figure 119866minus10750 more directional alongrarr

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 29 39

5 Illustrations on sparse stochastic processes 54 Illustrations with one inverse block

One block 119866minus11205820 increasing 120582

Example 1 with 120572 = 0 and 120582 = 0 025 05 075 1

(a) On a Gaussian white noise (b) On a ldquolow-firingrdquo Poisson

Figure 119866minus110 purely directional along minusrarr

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 29 39

5 Illustrations on sparse stochastic processes 54 Illustrations with one inverse block

One block 119866minus11205821205876

Example 2 with 120572 = 1205876 on a Gaussian white noise

(a) 120582 = 025 (b) 120582 = 05 (c) 120582 = 075

More directionalminusminusminusminusminusminusminusminusminusminusminusminusrarr

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 30 39

5 Illustrations on sparse stochastic processes 54 Illustrations with one inverse block

One block 119866minus1075120572 turning 120572

Example 3 with 120582 = 075 and 120572 = 0 1205874 1205872 on a Gaussian

(a) 120572 = 0 (b) 120572 = 1205874 (c) 120572 = 1205872

Rotation by +1205874 xminusminusminusminusminusminusminusminusminusminusminusminusminusrarr

Very directional

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 31 39

5 Illustrations on sparse stochastic processes 54 Illustrations with one inverse block

One block 119866minus1095120572 turning 120572

Example 4 with 120582 = 095 and 120572 = 0 1205874 1205872 on a Gaussian

(a) 120572 = 0 (b) 120572 = 1205874 (c) 120572 = 1205872

Rotation by +1205874 xminusminusminusminusminusminusminusminusminusminusminusminusminusrarr

Almost purely directional

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 32 39

5 Illustrations on sparse stochastic processes 55 Illustrations with two inverse blocks

Two purely-directional blocks 119866minus110119866

minus111205872

Example 5 with the inverse ldquoMondrianrdquo derivative 119863minus1119909 119863minus1

119910

(a) On a Gaussian white noise (b) On a ldquolow-firingrdquo Poisson

Figure Purely directional two orthogonal integrations = ldquoMondrianrdquo process

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 33 39

5 Illustrations on sparse stochastic processes 55 Illustrations with two inverse blocks

Two partly-directional blocks 119866minus11205820119866

minus11205821205872

Example 6 with a partly-directional ldquoMondrianrdquo

(a) 120582 = 05 (b) 120582 = 075

Figure Partly directional ldquoMondrianrdquo process on a ldquolow-firingrdquo Poisson

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 34 39

5 Illustrations on sparse stochastic processes 55 Illustrations with two inverse blocks

Two blocks same angle 119866minus11205821205873119866

minus11205821205873

Example 7 with 1205721 = 1205722 = 1205873 on a Gaussian white noise

(a) 120582 = 05 (b) 120582 = 075 (c) 120582 = 1

More directionalminusminusminusminusminusminusminusminusminusminusminusminusrarr

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 35 39

5 Illustrations on sparse stochastic processes 55 Illustrations with two inverse blocks

Two blocks two angles 119866minus112058231205874119866

minus112058251205874

Example 8 with 1205721 = 31205874 1205722 = 5120587

4 on a ldquolow-firingrdquo Poisson

(a) 120582 = 03 (b) 120582 = 05 (c) 120582 = 08

More directionalminusminusminusminusminusminusminusminusminusminusminusminusrarr

≃ Cone of opening 1205872direction 120587 (alonglarr)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 36 39

6 Conclusion

6 Conclusion amp Appendix

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 37 39

6 Conclusion 61 Technical conclusion

Quick sum-up

First we presented

ndash Convolution operators 119866

ndash Fourier multipliers 119892ndash Geometrical properties (120574-SI RI) on 119866hArr on 119892ndash And the notion of steerability

dim 119881119892 = dim Span1205790isin[02120587]

(119903 120579) ↦rarr 119892(119903 120579 + 1205790)

is finite

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 37 39

6 Conclusion 61 Technical conclusion

Quick sum-up

Then we found and proved

ndash Characterization of 2D 120574-SI steerable convolutions as a sumndash Parameters 119870119866 and 1198860 119886119870119866

but not used in practice

ndash And also a decomposition as a product of simple blocks 119866120582120572

119866120582119894120572119894=

(120582119894119863120572119894

+ (1minus 120582119894)(minusΔ)12)

ndash The blocks are exactly the 1-SI steerable of order 119899119866 le 2ndash And are convex combinations of

- a directional derivative 119863120572119894(order 2)

- and the half-Laplacian (minusΔ)12 (order 1)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 37 39

6 Conclusion 61 Technical conclusion

Quick sum-up

And experimentally we applied

On a Gaussian white noise and a compound Poisson noise

ndash Purely isotropic (minusΔ)1205742 (120582 = 1)ndash Or purely directional 119863120572 (120582 = 0)ndash And partly directional 119866120582120572 (120582 isin (0 1))

ndash And two blocks 119866120582120572 with different 120582 and 120572

=rArr Interesting patterns

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 37 39

6 Conclusion 62 Future work

Future work

For the theoretical part

ndash More general theorem of decomposition(general case for 119892 isin Sprime(R2))

ndash Study steerability for higher dimensions 119889 gt 2 (harder)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 38 39

6 Conclusion 62 Future work

Future work

Applications to sparse processes

ndash Visualize our operatorsndash But also generate new processes

=rArr Future publication with Julien and Virginie

Other possibilities of applications in the lab

ndash Generate new splines (an operator 119866hArr a spline)

ndash New data recovery algorithms (an operator 119866hArr a penalization term 119866119891)

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 38 39

6 Conclusion 63 Thank you

Thank you

Thank you for your attention

andinfin thanks to all of you for the last 4 months

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 39 39

6 Conclusion 63 Questions

Questions

Want to know more rarr˓ Read my master thesis internship report

httpsgooglxPzw4A

rarr˓ And e-mail me if needed lilianbessonens-cachanfr

rarr˓ Or consult the references

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 39 39

6 Conclusion 63 Questions

Questions Want to know more rarr˓ Read my master thesis internship report

httpsgooglxPzw4A

rarr˓ And e-mail me if needed lilianbessonens-cachanfr

rarr˓ Or consult the references

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 39 39

Appendix Outline of the appendix

Appendix

Outline of the appendix

ndash Some proofsndash Main references given belowndash Code figures results from our experiments etc minusrarr lbokvuepfl2016

ndash Everything here is open-source under the CC-BY 40 License

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 39 39

Appendix A1 Extra proofs

Proof of steerability of order 1 = rotation-invariance

Theorem [Th422 of my report]

Non-zero steerable of order 119899119866 = 1hArr rotation-invariant

Quick proof

lArr Obvious 119881119892 = Span119892 = R119892 has dimension 119899119866 = 1

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 39 39

Appendix A1 Extra proofs

Proof of steerability of order 1 = rotation-invariance

Theorem [Th422 of my report]

Non-zero steerable of order 119899119866 = 1hArr rotation-invariant

Quick proof

rArr Less obvious but not too hard ndash 119899119866 = 1 so there exists 120582(1205790) isin R such that 1198771205790119892 = 120582(1205790)119892ndash 1198771205790 (119903 120579) ↦rarr (119903 120579 + 1205790) is a bijective change of variablendash so 1198771205790119892 and 119892 have same L2 norm on the circle 120579 isin [0 2120587]

(for a fixed 119903)ndash and so 120582(1205790) = plusmn1ndash But 1198771205790 = 1198771205790211987712057902 so 120582(1205790) = 120582(12057902)2 gt 0ndash and so finally 1198771205790119892 = 119892

So 119899119866 = 1 =rArr 119866 is rotation-invariant as wanted

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 39 39

Appendix A2 Main references

Previous works and references I

Chaudhury K and Unser M (2009)The fractional Hilbert transform and Dual-Tree Gabor-likewavelet analysisIn 2009 IEEE International Conference on Acoustics Speech andSignal Processing pages 3205ndash3208 IEEE

Chaudhury K and Unser M (2010)On the Shiftability of Dual-Tree Complex WavelettransformsIEEE Transactions on Signal Processing 58(1)221ndash232

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 39 39

Appendix A2 Main references

Previous works and references II

Rudin W (1991)Functional AnalysisMcGraw-Hill Inc New York

Stein E M and Weiss G L (1971)Introduction to Fourier analysis on Euclidean spaces volume 1Princeton University Press

Unser M and Chenouard N (2013)A unifying parametric framework for 2D steerable wavelettransformsSIAM Journal on Imaging Sciences 6(1)102ndash135

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 40 39

Appendix A2 Main references

Previous works and references III

Unser M Fageot J and Ward J (2016)Splines are Universal Solutions of Linear Inverse Problemswith Generalized-TV RegularizationarXiv preprint arXiv160301427

Unser M and Tafti P (2014)An Introduction to Sparse Stochastic ProcessesCambridge University Press

Vonesch C Stauber F and Unser M (2015)Steerable PCA for Rotation-Invariant Image RecognitionSIAM Journal on Imaging Sciences 8(3)1857ndash1873

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 41 39

Appendix A3 Open-Source - CC-BY 40 License

Open-Source

LicenseThese slides and the reporta are open-sourced under the CC-BY40 License

Copyright 2016 copy Lilian BessonaAnd the additional resources ndash including code figures etc

Lilian Besson (ENS Cachan) MVA Internship Presentation August 26st 2016 39 39