GigaNumerics

in Deep Structure

Analysis of Im

ages

Employing M

athLinkan

d the Parallel C

omputing Toolkit

Bart Janssen

LucFlorackand Bart ter Haar Rom

eny

2

•Scale

Space

•Toppoints

•Im

age Rec

onstru

ction

•M

athem

atica Im

plemen

tation

•Scale

Space

•Toppoints

•Im

age Rec

onstru

ction

•M

athem

atica Im

plemen

tation

Outline

3

The problem of scale:

Objects live at different scales

Gala looking into the Mediterranean Sea

Salvador Dali

4

The problem of scale:

Objects live at different scales

Solution?

Look at all scales

simultaneously

Gala looking into the Mediterranean Sea

Salvador Dali

5

Sca

le Space

in Human Vision

•The human

visual system is

a multi-scale sampling device

•The retina co

ntains receptive

fields

of va

rying size.

6

Sca

le for Calculating Deriva

tive

s•To calcu

late derivatives w

e nee

d smooth

(continuous) data.

•Im

age data is not sm

ooth.

Pixel data

1stOrd

er D

erivative

Intensity

0

7

•At the original

scale of a

dithered

imag

e we cannot

calculate a

derivative.

•W

e nee

d to

observe the

imag

e at a certain

scale.

BLUR

8

Practical Implementation

•Scale-Space Axioms

–Linea

rity

–Spatial sh

ift inva

rian

ce

–Isotropy

–Cau

sality

–Separability

•Lea

d to the Gau

ssian K

ernel

2

22 1

2/

2

2

)2(

1)

,(

σ

πσ

σ

D

D

xx

ex

G

++

−

=

K

9

Calculating Deriva

tive

s with Gauss

ians

•The derivative of the data at a scale σ

is

defined

as

•Due to nice properties of the Gau

ssian this

can be rewritten

as

)};

()

({

01

σx

Gx

Lx

⊗∂∂

);

()

(1

0σ

xG

xL

x∂∂

⊗

10

Calculating Deriva

tive

s of Im

ages

•Differentiationbec

omes

Integration!

...ListConvolve/ FFT-method

(Lap

lacian

)

11

Gauss

ian Sca

le Space

s

x

y

12

•Scale

Space

•Toppoints

•Im

age Rec

onstru

ction

•M

athem

atica Im

plemen

tation

•Scale

Space

•Toppoints

•Im

age Rec

onstru

ction

•M

athem

atica Im

plemen

tation

Outline

13

Singular points of a

Gaussian scale space image

Fold C

atastrophe

14

Singular points of a

Gaussian scale space image

Fold C

atastrophe

15

Stability of To

p Points

•W

e can calcu

late the

varian

ce of the

displacemen

t of top

points under noise.

•W

e nee

d 4

thord

er

derivatives in the top-

points for that.

16

Applica

tions

•Opticflow

estimation

•M

atch

ing

•Im

age Editing/ Seg

men

tation?

17

OpticFlow

Estim

ation

18

Optic Flow Estim

ation

19

20

Rotate and scale

acco

rding to the

cluster m

eans.Match

ing

21

Differential Inva

riants

•Fea

tures are

irreducible 3

rd

ord

er differential

inva

rian

ts.

•These features

are rotation and

scale inva

rian

t.

22

In this exa

mple

we hav

e tw

o

clusters of

correc

tly match

ed

points.

C1

C2

23

•Scale

Space

•Toppoints

•Im

age Rec

onstru

ction

•M

athem

atica Im

plemen

tation

•Scale

Space

•Toppoints

•Im

age Rec

onstru

ction

•M

athem

atica Im

plemen

tation

Outline

24

Image Reco

nstruction

Given

features

Selec

t from

metam

eric

class

such

that

(consisten

t features)

25

Reco

nstructionfrom

SingularPoints

Use

differential

stru

cture

in singularpoints

as fea

tures.

=

26Variational

Approac

h

27

The solutionis anA-orthogonal

projectionof onto

Gen

eralisationusinggelfandtriples (R

. Duits)

Minim

isationof

under

the co

nstraints

28

Prior and DualFilters

29

Reco

nstructionfrom

SingularPoints

This

mea

ns

Gramm

matrix:

Projection:

30

31

32

•Scale

Space

•Toppoints

•Im

age Rec

onstru

ction

•M

athem

atica Im

plemen

tation

•Conclusions

•Scale

Space

•Toppoints

•Im

age Rec

onstru

ction

•M

athem

atica Im

plemen

tation

•Conclusions

Outline

33

Basicim

plementation

•BuildGramm

Matrix

•In

versionof Gramm

Matrix

•Building and Sam

pling R

econstru

ction

34

makeGramm@kernel_,orders_,points_,8xsize_,ysize_<D

:=Module@8

Φtensor,gramm,signmx,signmy,subΦ,signs<,

subΦ@8st_,dx_,dy_<D

:=

submatrixofinnerproducts;

Φtensor=

Map@

subΦ,

Outer@Plus,points,81,−1,−1< #&ê@

points,1D,

82<

D ;

gramm=Chop@Flatten@

Map@

Flatten,

Transpose@Φtensor,82,4,1,3<D,

82<

D, 1 DD;

gramm

D

35

subΦ@8st_,dx_,dy_<D:=If@Negative@dxD,

If@Negative@dyD,

signmxsignmysubΦ@8st,

−dx,

−dy<D,

signmxsubΦ@8st,

−dx,

dy<D

D,

If@Negative@dyD,

signmysubΦ@8st,dx,

−dy<D,

subΦ@8st,dx,dy<D

=submatrixofinnerproducts;

D

D;

Dyn

amic

Programming

36Parallel Im

plementation

Φtensor

=

Map@

subΦ,

Outer@Plus,

points,

81,

−1,

−1< #&

ê@

points,

1D,

82<

D;

ExportEnvironment@GaussianDerivativeAt,xsize,ysize,

signmy,signmx,kernel,orders,subΦD;

Φtensor

=

ParallelMap@

Map@subΦ,#D&,

Outer@Plus,points,

81,

−1,

−1< #

&ê@points,1D

D;

37

Sampling the Reco

nstruction

adva

ntageof sy

mbolicpower

FourierK@gamma_,scale_,order_,8wx_,wy_<,8xi_,yi_<D:=ModuleA8<,

Exp@�Hwxxi

+wyyiLD

H−�wx

Lorder@@1DDH−

�wy

Lorder@@2DD

1

1+gamma2 Hwx2+wy2L

Exp@−scaleHwx2

+wy2 LD

1

E FourierReconstructionFunction

=

Compile@

88x,_Real<,

8y,_Real<<,

Evaluate@rfD

D;

38

Parallel Sampling

ParallelSampleReconstruction@8xsize_,ysize_<D

:=Module@8x,y,FourierimageData,image,newfeaturepoints,n,wlist<,

wlist=listoffrequencies;

ExportEnvironment@FourierReconstructionFunctionD;

FourierimageData=ParallelMap@Apply@FourierReconstructionFunction,#D&,wlist,82

<D;

image=Re@InverseFourier@FourierimageData,FourierParameters→

81,1<D

DPRange@xsizeD,

Range@ysizeDT;

image

D

39Mathem

atica Dem

o 1

40

Mathlinkforbetterperform

ance

(sometimes)

:Begin:

:Function:MLSobolevReconstruction

:Pattern:MLSobolevReconstruction[X__?(VectorQ[#1,NumberQ]&), Y__?(VectorQ[#1,NumberQ]&),...

:Arguments:{X,Y,OX,OY,T,F,gridsize,gamma}

:ArgumentTypes:{RealList,RealList,IntegerList,...

:ReturnType:Manual

:End:

#inc

lude

<m

ath.

h>...

#inc

lude

"m

athl

ink.

h"

#inc

lude

"y

our

own

stuf

f.h"

void

ML

Sobo

levR

econ

stru

ctio

n(do

uble

*X,lo

ng X

coun

t,do

uble

*Y,lo

ng Y

coun

t,...

.

{

mal

loc(

);

do

som

e co

mp

utat

ions

;

ML

Put

Rea

lLis

t(st

dlin

k,da

ta,s

ize)

;

fre

e();

}

41Mathem

atica Dem

o 2

42

(State of the)Art

43

Questions?

Topological A

bduction of Europe -Hom

age to Rene Thom

Salvador Dali

MathVisionTools

http://w

ww.bmi2.bmt.tue.nl/im

age-an

alysis/

44

Ack

nowledgements

•Bart ter Haa

r Romen

y

•Evg

uen

iaBalmachnova

•Rem

coDuits

•LucFlorack

•Frans Kan

ters

•Bram Platel