Discrete Orthogonal Harmonic...

Discrete Orthogonal Harmonic Transforms

Speaker: Chun-Lin Liu,Advisor: Soo-Chang Pei Ph. D

Image Processing Laboratory, EEII 530,Graduate Institute of Communication Engineering,

National Taiwan University.

May 26, 2012

CL-Liu (GICE, NTU) Discrete Orthogonal Harmonic Transforms May 26, 2012 1 / 37

Outline

1 Introduction

2 Computation of 1D Discrete Orthogonal FunctionsProposed method to the 1D discrete orthogonal functionsSimulation results

3 Discrete implementation of 2D Fourier Transform EigenfunctionsThe general form of 2D FT eigenfunctions and the discreteimplementationSimulation Results

4 Conclusions

Outline

1 Introduction

4 Conclusions

Introduction

Some well-known discrete orthogonal transforms areI Discrete Fourier transform (DFT),I Discrete Sine/Cosine/Hartley transforms,I Discrete wavelet transforms.

Properties of orthogonal transformsI More coefficients lead to smaller reconstruction error.I Perfect reconstruction.

ProblemI Explicit definitions of the transform kernels. For instance, the DFT is

defined as

N−1∑n=0

fne−j 2πnk

N , n, k = 0, 1, 2, . . . , N − 1. (1)

A simple method to other discrete transforms

Choose a continuous orthogonal transform and take samples of thetransform kernel.

Pros:I Good approximation to the continuous transforms.

Cons:I Not orthogonal. For instance, assume that the continuous transform

kernel {ψn(x)}∞n=0 satisfies the orthogonal relation:∫ ∞−∞

ψ∗m(x)ψn(x) dx = δm,n. (2)

However, taking the samples x = l∆x does not ensure orthogonality:∑l

ψ∗m (l∆x)ψn (l∆x) ∆x 6= δm,n. (3)

Problem formulation

The 1D discrete orthogonal transform is characterized by thetransform matrix Ψ, as the following relationship

...FN−1

=[ψ0 . . . ψN−1

]H f0...

fN−1

= ΨHf , (4)

I f is the input signal, F is the transformed signal,I {ψn}

N−1n=0 are the discrete transform kernels,

I f , F, and ψn are all N -by-1 column vectors.

Problem statement

We want to the discrete orthogonal transform to be

General ( there are general ways to find entries of Ψ).

Good approximation (ψn is close to the samples of ψn(x)).

Orthogonal (Ψ is an unitary matrix).

Outline

1 Introduction

4 Conclusions

Hermite Gaussian functions and Hermite transforms

The Hermite Gaussian functions (HGFs) are defined by

hn(x) =

2nn!√π

Hn(x)e−x2/2, (5)

where Hn(x) is the Hermite polynomial.

Differential equation (Dx = d / dx is the differential operator):(D2x − x2

)hn(x) = −(2n+ 1)hn(x). (6)

Properties:I hn(x) is the eigenfunction of the Fourier transform with eigenvalue

(−j)n.I Complete and orthonormal basis for L2(R).

Hermite transforms are defined by

∫ ∞−∞

f(x)hn(x) dx. (7)

Outline

1 Introduction

4 Conclusions

Implement discrete HGFs by differential equations

Implement the differential equation as the matrix eigen-problem.1 Convert L into L.2 Solve the eigenvectors and eigenvalues of L numerically.

Continuous:

L︷︸︸︷(D2x − x2

)hn(x) = −(2n+ 1) hn(x),

Discrete:

· . . . ·.... . .

...· . . . ·

︸︷︷︸

·...·

︸︷︷︸hn

·...·

︸︷︷︸hn

Three basic operations1 Multiplied by a constant c. (cI) f(x) = cf(x).2 Multiplied by x. Xf(x) = xf(x).3 Differentiate with respect to x, Dx = d / dx.

Discrete version of cI and X

1 cIN is the discrete version of cI. IN is the identity matrix of size N .

Continuous: cI f(x) = cf(x),

Discrete:

c 0 . . . 00 c . . . 0...

.... . .

...0 0 . . . c

f1f2...

fN−1

cf1cf2...

cfN−1

.2 Assume that the functions are sampled uniformly on x = n∆x,

n = [−N−12 ,−N−3

2 , . . . , N−12 ]T . X = diag(x) represents the discreteversion of X .

X f(x) = xf(x),

−N−1

2 0 . . . 0

0 −N−32 . . . 0

......

. . ....

0 0 . . . N−12

f1f2...

fN−1

−N−1

2 ∆xf1−N−3

2 ∆xf2...

N−12 ∆xfN−1

.CL-Liu (GICE, NTU) Discrete Orthogonal Harmonic Transforms May 26, 2012 11 / 37

Discrete version of differential operators

In numerical analysis, there are forward difference, backwarddifference, and central difference. However, they are simple butinaccurate.

We derive the differential operator from Fourier transforms.

Dxf(x) = F−1FDxf(x) = F−1 (jX )Ff(x), (8)

where F denotes the Fourier transform operator.

Proposed discrete differential operator

3 The discrete differential operator is obtained by

D = F−1 (jX) F, (9)

where F is the DFT matrix and X is the discrete multiplied-by-xoperator.

Outline

1 Introduction

4 Conclusions

Simulation on the Discrete HGFs

The number of discrete points N = 101.

The sample interval ∆x =√

2π/N ≈ 0.2494.

The eigenvalues are all negative and sorted in the descent order.

We want to verify whether The eigenvectors are close to thecontinuous samples.

Note that the results are identical to those of the n2-matrix1.

1S. C. Pei, J. J. Ding, W. L. Hsue and K. W. Chang, “Generalized commutingmatrices and their eigenvectors for DFTs, offset DFTs, and other periodic operations,”IEEE Trans. on Signal Processing, vol.56, No.8, pp.3891-3904, Aug. 2008

First four Discrete HGFs

−10 −5 0 5 10−0.1

Discrete

Continuous

−10 −5 0 5 10−0.4

−0.3

−0.2

−0.1

Discrete

Continuous

−10 −5 0 5 10−0.3

−0.2

−0.1

Discrete

Continuous

−10 −5 0 5 10−0.4

−0.3

−0.2

−0.1

Discrete

Continuous

Outline

1 Introduction

4 Conclusions

2D discrete orthogonal functions

Another question is raised: How do we implement high dimensionaldiscrete orthogonal functions while the orthogonality is kept?

Possible approachI Take the samples of the continuous functions.

F Very simple but not orthogonal.

I Start from the differential equations.F Partial differential equations require a very huge matrix.F Large scale numerical eigen-decomposition is very difficult.F Different coordinate systems lead to multiple solutions.

Due to these difficulties, we narrow our discussion to theeigenfunctions of 2D Fourier transforms2.

2S. C. Pei and C. L. Liu, “A general form of 2D Fourier transformeigenfunctions,” 2012 IEEE International Conference on Acoustics, Speech andSignal Processing, Kyoto, Japan,, March 2012.

Known eigenfunctions of 2D FT

There are three known eigenfunctions for 2D FT.

Separable HGFs (SHGFs) in Cartesian coordinates

hm,n(x, y) = hm(x)hn(y). (10)

Rotated HGFs (RHGFs) in rotated Cartesian coordinates

hm,n(α;x, y) = hm,n(x cosα+ y sinα,−x sinα+ y cosα). (11)

Laguerre-Gaussian functions (LGFs) in polar coordinates

lm,n(r, θ) = Np,lr|l|L|l|p

(r2)e−r

2/2e−jlθ, (12)

where p = min {m,n}, l = m− n, Np,l is the normalization factor,and Llp (·) is the associated Laguerre polynomial.

All of them form a complete and orthonormal basis for L2(R2).

Observations on the 2D Fourier transform eigenfunctions

Their differential equations have identical operators, identicaleigenvalues, but different eigenfunctions.

∂x2+

∂y2− x2 − y2

)hm,n(x, y) = −2 (m+ n+ 1)hm,n(x, y),(

∂x2+

∂y2− x2 − y2

)hm,n(α;x, y) = −2 (m+ n+ 1)hm,n(α;x, y),(

∂rr∂

r2∂2

∂θ2− r2

)lm,n(r, θ) = −2 (m+ n+ 1) lm,n(r, θ).

Eigenvalues of the 2D FT are the same, (−j)m+n, for each solution.

Our question

Is there a general form to represent these eigenfunctions?

Outline

1 Introduction

4 Conclusions

The general form of the 2D FT eigenfunctions

The 2D Fourier transform eigenfunctions ψm,n(x, y) is

ψm,n(x, y) =m+n∑p=0

cm,np hp,m+n−p(x, y), (13)

ψ0,m+n(x, y)ψ1,m+n−1(x, y)

...ψm+n,0(x, y)

c0,m+n0 c0,m+n

1 . . . c0,m+nm+n

c1,m+n−10 c1,m+n−1

1 . . . c1,m+n−1m+n

......

. . ....

cm+n,00 cm+n,0

1 . . . cm+n,0m+n

︸︷︷︸

h0,m+n(x, y)h1,m+n−1(x, y)

...hm+n,0(x, y)

m,n = 0, 1, 2, . . . . cm,np are the combination coefficients.

ψm,n(x, y) corresponds eigenvalue (−j)m+n of the 2D FT.

Mathematical interpretation: linear combination of the eigenfunctionswith the same eigenvalue/eigenspace gives another eigenfunctions.

ψm,n(x, y) are orthogonal if and only if T is unitary.

Known combination coefficients

The SHGFs, RHGFs, and LGFs all satisfy the general for of 2DFourier transform eigenfunctions.

SHGFs hm,n(x, y) and cm,np = δ[p−m].

RHGFs hm,n(α;x, y)3,

cm,np =

√p!(m+ n− p)!

m!n!(sinα)m−p (cosα)n−p P (m−p,n−p)

p (cos 2α) ,

where P(α,β)n (·) is the Jacobi polynomial.

LGFs lm,n(r, θ) 4,

cm,np (LGF) = jp cm,np (RHGF)∣∣α=π/4

. (14)

3A. Wunsche, “Hermite and Laguerre 2D polynomials,” Journal of Computationaland Applied Mathematics, vol. 133, pp. 665-678, 2001.

4M. W. Beijersbergen, L. Allen, H. van der Veen, and J. P. Woerdman,“Astigmatic laser mode converters and transform of orbital angular momentum,” Opt.Comm., vol. 96, pp. 123-132, 1993.

Discrete 2D Fourier transform eigenfunctions

Implementation of discrete 2D Fourier transform eigenfunctions

1 Solve the 1D discrete Hermite Gaussian functions.

2 Construct discrete separable HGFs by multiplying 1D discrete HGFsin each dimension.

3 Compute combination coefficients cm,np .

4 Obtain discrete 2D Fourier transform eigenfunctions according to

ψm,n(x, y) =

m+n∑p=0

cm,np hp,m+n−p(x, y).

Advantages:I More general than the explicit expression.I Different combination coefficients yield different eigenfunctions.I Samples are always on the Cartesian grids.I Orthogonal, as long as the matrix composed of cm,n

p is unitary.

Problem on very high order discrete eigenfunctions

An illustrative example

The number of discrete points in each dimension N = 6.

We want to get the discrete function of h1,5(α;x, y).

The implementation steps indicates1 We have only 6 1D discrete HGFs: h0,h1,h2,h3,h4,h5.2 Construct separable HGFs. Then compute the combination coefficients.3 We have a problem in the last step. Write out the general form

c2,40 h6hT0 + c2,41 h5h

T1 + c2,42 h4h

T2 + c2,43 h3h

+c2,44 h2hT4 + c2,45 h1h

T5 + c2,46 h0h

The problem is the absence of h6 in the 1D discrete HGFs.

Two possible solutions to high-order discrete eigenfunctions

1 Truncate the combination coefficients.

h1,5(α;x, y)⇒ c2,40 h6hT0 + c2,41 h5h

T1 + · · ·+ c2,45 h1h

T5 + c2,46 h0h

I Not orthogonalI High accuracy

2 Replace the combination coefficients with those of lower-ordereigenfunctions. The concept is to mirror the coefficients.

h0,4(α;x, y)⇒ d2,40 h4hT0 + · · ·+ d2,44 h0h

↓ ↓h1,5(α;x, y)⇒ c2,40 h6h

T0 +c2,41 h5h

T1 + · · ·+ c2,45 h1h

T5 + c2,46 h0h

I OrthogonalI Low accuracy

The discrete Laguerre Gaussian transform

The Laguerre Gaussian transform is

Lm,n =

f(x, y)l∗m,n(r, θ) d2 r. (15)

According to the general form, (13), we have

Lm,n =

m+n∑p=0

(cm,np

)∗ ∫R2

f(x, y)hp,m+n−p(x, y) d2 r︸︷︷︸2D separable Hermite transform

. (16)

The Laguerre Gaussian transform is composed of1 2D separable Hermite transform2 Linear transformation associated with cm,n

p .In terms of discrete implementation:

1 Separable transforms and the linear transformation are faster thannon-separable transforms.

2 High-order Lm,n are adjusted either by truncating or by mirroring thetransformation coefficients cm,n

Outline

1 Introduction

4 Conclusions

Simulation results

1 Compare the continuous-sampled Laguerre Gaussian functions withthe discrete Laguerre Gaussian functions.

2 Compare the high-order continuous-sampled LGFs with the discreteLGFs with either truncated coefficients or mirrored coefficients.

3 Image expansion and reconstruction using discrete Laguerre Gaussiantransforms.

Simulation 1: Accuracy of the discrete eigenfunctions

l8,3(r, θ) Magnitude Phase

−10 −5 0 5 10

Number of pointsin each dimensionN = 101.

Sampling interval∆x = ∆y =√

2π/N .

Error comes fromvery small values(≈ 10−14) on theboundaries.

Simulation 2: High-order discrete LGFs

Continuous samples

l63,1(r, θ)

Truncate Mirror

Number of points in eachdimension N = 64.

Sampling interval∆x = ∆y =

√2π/N .

Compare the magnitudes oflm,n(r, θ). The number of ringsshould be min {m,n}+ 1.

For the mirrored discretefunction, coefficients areborrowed from those of l62,0(r, θ).

Error comes fromI Inaccurate combination terms.I Inaccurate high-order Hermite

Gaussian functions.

Simulation 3: Image expansion and reconstruction

The input image I

Reconstruction schemes

Image expansion: Convert inputimage I into its LGT Lm,n.

Image reconstruction: partial Lm,nand the inverse LGT yields thereconstructed image I.

N = 90, ∆x = ∆y =√

2π/N .

The input image I is the 2Dsquared image.

We have two type ofreconstruction schemes, shown onthe left:

I Fix L, increase K gradually.I Fix K, increase L gradually.

Image reconstruction along K (details in θ-direction)

The input image I

discrete LGT=⇒

Reconstruction scheme

K = 0, 2, 4, 6

K = 8, 10, 12, 14

Image reconstruction along L (details in r-direction)

The input image I

discrete LGT=⇒

Reconstruction scheme

n Lm,n

L = 0, 2, 4, 6

L = 8, 10, 12, 14

Outline

1 Introduction

4 Conclusions

Conclusion on 1D discrete orthogonal functions

In the first part, we proposed an alternative method to the discreteorthogonal functions.

As long as the differential equations are known, we can replace thecontinuous operators L with discrete operator L. This approach ismore general and the explicit form is not required.

We can apply this procedure to solve any linear differential equations.I Classical orthogonal polynomialsI Schrodinger equations in quantum mechanicsI Scale transformsI Fractional Fourier transformsI Linear canonical transforms and their eigenfunctions

We utilize the 1D results to implement the 2D case.

Conclusion on 2D Fourier transform eigenfunctions

In the section part, we implement the 2D Fourier transformeigenfunctions along with the transform derived from theseeigenfunctions.

I The general form of 2D Fourier transform eigenfunctions.I Deal with high-order discrete eigenfunctions. We can truncate or

mirror the combination coefficients.I Convert the non-separable transform (LGT) into 2D separable Hermite

transforms and then a linear transformation (fast).

The concept can be generalized to three-dimensional eigenfunctions.

Thank you for your attention!

Q & A Time

Discrete Orthogonal Harmonic...

Documents

Transcript of Discrete Orthogonal Harmonic...

Discrete Dynamical Systems - Dipartimento di Informaticapages.di.unipi.it/.../02-DiscreteDynamicalSystems.pdf · Discrete Dynamical Systems Computational Models for Complex Systems

Dispense di Matematica Arsen Palestini Dispense di ... · discrete Variabili aleatorie ... Spazi di probabilit a III A ogni evento va associata una funzione probabilit a che quanti

OPTIMAL DISCRETE-TIME DESIGN OF MAGNETIC Tiziano … · Tiziano Pulecchi⁄ Marco Lovera⁄ Andras Varga⁄⁄ ⁄ Dipartimento di Elettronica e Informazione Politecnico di Milano

DI-4108 High Speed USB Data Acquisition...DI-4108 Description 330-668-1444 Overview The DI-4108 is suitable for a wide range of analog and discrete measurements. When used in tandem

Notes on Discrete Exterior Calculus - University of Arizonamath.arizona.edu/~agillette/research/decNotes.pdf · 1 Introduction 1.1 Di erential Forms for Physical Models Di erential

Discrete adjoint for CFD - ONERA€¦ · Discrete gradient calculation method Outline 1 Discrete gradient calculation method 2 Discrete gradient methods, intuition and checks 3 Discrete

DISCRETE MOVING FRAMES AND DISCRETE INTEGRABLE …maribeff/discrete-moving-frames-integr… · DISCRETE MOVING FRAMES AND DISCRETE INTEGRABLE SYSTEMS ELIZABETH MANSFIELD, GLORIA MAR

Discrete Di erence Equations - QUBES · Discrete Di erence Equations 2 Escherichia coli: Abbreviated as E. coli, this bacteria lives in the lower intestines of endotherms (warm blooded

Discrete Fourier Series & Discrete Fourier Transformhcso/ee5410_7.pdf · transform, discrete-time Fourier transform (DTFT), discrete Fourier series (DFS) and discrete Fourier transform

Parametric inference for discrete observations of …Parametric inference for discrete observations of di usion processes with mixed e ects Maud Delattre, Valentine Genon-Catalot,

Geometry II - Discrete Di erential Geometrypage.math.tu-berlin.de/~bobenko/Lehre/Skripte/DDG07.pdf2.2 Curvature of planar discrete curves In the smooth case, the curvature of a planar

Flexible discrete chapter - University of Texas at Austin · Flexible Discrete Choice Structures 3 In this chapter the focus is on two classes of di screte choice models that relax

Discrete Systems with Undecidable Dynamics Marco Giunti Marco Giunti Università di Cagliari giunti@unica.it 1Bochum,

High Order Super Nested Arrays - California Institute of ...systems.caltech.edu/dsp/students/clliu/SuperNested/SN_Slides_SAM_2016.pdf1 Minimum redundancy arrays2 2 Nested arrays3 3

electrorig.narod.ru/olderfiles/1/CATERPILLAR_EMCP3... · caterpillar z (shoe dn rtn - reiurn di 2 — c4screte input52 intiate do - discrete

New Cramér-Rao Bound Expressions for Coprime and Other ...systems.caltech.edu/dsp/students/clliu/CRB/CRB_Slides_SAM_2016.pdfNew Cramer-Rao Bound Expressions´ for Coprime and Other

Introduction to discrete functional analysis techniques ... · Introduction to discrete functional analysis techniques for the numerical study of di usion equations with irregular

Discrete Maths - Imperial College Londonyg/Discrete/notes.pdf · Discrete Maths Philippa Gardner ... Discrete Mathematics and its Applications, McGraw Hill ... J.K. Truss. Discrete

Discrete Spectral Hashing for Efficient Similarity Retrieval · Discrete Spectral Hashing for Efﬁcient Similarity Retrieval Di Hu , Feiping Nie , and Xuelong Li , Fellow, IEEE Abstract—To

Arnold Di usion of the Discrete Nonlinear Schr odinger ...faculty.missouri.edu/~liyan/Arnold-Diffusion.pdfArnold di usion, Darboux transformation, isospectral theory, ... mathematical