Post on 21-Jun-2020
GigaNumerics
in Deep Structure
Analysis of Im
ages
Employing M
athLinkan
d the Parallel C
omputing Toolkit
Bart Janssen
LucFlorackand Bart ter Haar Rom
eny
2
•Scale
Space
•Toppoints
•Im
age Rec
onstru
ction
•M
athem
atica Im
plemen
tation
•Scale
Space
•Toppoints
•Im
age Rec
onstru
ction
•M
athem
atica Im
plemen
tation
Outline
3
The problem of scale:
Objects live at different scales
Gala looking into the Mediterranean Sea
Salvador Dali
4
The problem of scale:
Objects live at different scales
Solution?
Look at all scales
simultaneously
Gala looking into the Mediterranean Sea
Salvador Dali
5
Sca
le Space
in Human Vision
•The human
visual system is
a multi-scale sampling device
•The retina co
ntains receptive
fields
of va
rying size.
6
Sca
le for Calculating Deriva
tive
s•To calcu
late derivatives w
e nee
d smooth
(continuous) data.
•Im
age data is not sm
ooth.
Pixel data
1stOrd
er D
erivative
Intensity
0
7
•At the original
scale of a
dithered
imag
e we cannot
calculate a
derivative.
•W
e nee
d to
observe the
imag
e at a certain
scale.
BLUR
8
Practical Implementation
•Scale-Space Axioms
–Linea
rity
–Spatial sh
ift inva
rian
ce
–Isotropy
–Cau
sality
–Separability
•Lea
d to the Gau
ssian K
ernel
2
22 1
2/
2
2
)2(
1)
,(
σ
πσ
σ
D
D
xx
ex
G
++
−
=
K
9
Calculating Deriva
tive
s with Gauss
ians
•The derivative of the data at a scale σ
is
defined
as
•Due to nice properties of the Gau
ssian this
can be rewritten
as
)};
()
({
01
σx
Gx
Lx
⊗∂∂
);
()
(1
0σ
xG
xL
x∂∂
⊗
10
Calculating Deriva
tive
s of Im
ages
•Differentiationbec
omes
Integration!
...ListConvolve/ FFT-method
(Lap
lacian
)
11
Gauss
ian Sca
le Space
s
x
y
12
•Scale
Space
•Toppoints
•Im
age Rec
onstru
ction
•M
athem
atica Im
plemen
tation
•Scale
Space
•Toppoints
•Im
age Rec
onstru
ction
•M
athem
atica Im
plemen
tation
Outline
13
Singular points of a
Gaussian scale space image
Fold C
atastrophe
14
Singular points of a
Gaussian scale space image
Fold C
atastrophe
15
Stability of To
p Points
•W
e can calcu
late the
varian
ce of the
displacemen
t of top
points under noise.
•W
e nee
d 4
thord
er
derivatives in the top-
points for that.
16
Applica
tions
•Opticflow
estimation
•M
atch
ing
•Im
age Editing/ Seg
men
tation?
17
OpticFlow
Estim
ation
18
Optic Flow Estim
ation
19
20
Rotate and scale
acco
rding to the
cluster m
eans.Match
ing
21
Differential Inva
riants
•Fea
tures are
irreducible 3
rd
ord
er differential
inva
rian
ts.
•These features
are rotation and
scale inva
rian
t.
22
In this exa
mple
we hav
e tw
o
clusters of
correc
tly match
ed
points.
C1
C2
23
•Scale
Space
•Toppoints
•Im
age Rec
onstru
ction
•M
athem
atica Im
plemen
tation
•Scale
Space
•Toppoints
•Im
age Rec
onstru
ction
•M
athem
atica Im
plemen
tation
Outline
24
Image Reco
nstruction
Given
features
Selec
t from
metam
eric
class
such
that
(consisten
t features)
25
Reco
nstructionfrom
SingularPoints
Use
differential
stru
cture
in singularpoints
as fea
tures.
=
26Variational
Approac
h
27
The solutionis anA-orthogonal
projectionof onto
Gen
eralisationusinggelfandtriples (R
. Duits)
Minim
isationof
under
the co
nstraints
28
Prior and DualFilters
29
Reco
nstructionfrom
SingularPoints
This
mea
ns
Gramm
matrix:
Projection:
30
31
32
•Scale
Space
•Toppoints
•Im
age Rec
onstru
ction
•M
athem
atica Im
plemen
tation
•Conclusions
•Scale
Space
•Toppoints
•Im
age Rec
onstru
ction
•M
athem
atica Im
plemen
tation
•Conclusions
Outline
33
Basicim
plementation
•BuildGramm
Matrix
•In
versionof Gramm
Matrix
•Building and Sam
pling R
econstru
ction
34
makeGramm@kernel_,orders_,points_,8xsize_,ysize_<D
:=Module@8
Φtensor,gramm,signmx,signmy,subΦ,signs<,
subΦ@8st_,dx_,dy_<D
:=
submatrixofinnerproducts;
Φtensor=
Map@
subΦ,
Outer@Plus,points,81,−1,−1< #&ê@
points,1D,
82<
D ;
gramm=Chop@Flatten@
Map@
Flatten,
Transpose@Φtensor,82,4,1,3<D,
82<
D, 1 DD;
gramm
D
35
subΦ@8st_,dx_,dy_<D:=If@Negative@dxD,
If@Negative@dyD,
signmxsignmysubΦ@8st,
−dx,
−dy<D,
signmxsubΦ@8st,
−dx,
dy<D
D,
If@Negative@dyD,
signmysubΦ@8st,dx,
−dy<D,
subΦ@8st,dx,dy<D
=submatrixofinnerproducts;
D
D;
Dyn
amic
Programming
36Parallel Im
plementation
Φtensor
=
Map@
subΦ,
Outer@Plus,
points,
81,
−1,
−1< #&
ê@
points,
1D,
82<
D;
ExportEnvironment@GaussianDerivativeAt,xsize,ysize,
signmy,signmx,kernel,orders,subΦD;
Φtensor
=
ParallelMap@
Map@subΦ,#D&,
Outer@Plus,points,
81,
−1,
−1< #
&ê@points,1D
D;
37
Sampling the Reco
nstruction
adva
ntageof sy
mbolicpower
FourierK@gamma_,scale_,order_,8wx_,wy_<,8xi_,yi_<D:=ModuleA8<,
Exp@�Hwxxi
+wyyiLD
H−�wx
Lorder@@1DDH−
�wy
Lorder@@2DD
1
1+gamma2 Hwx2+wy2L
Exp@−scaleHwx2
+wy2 LD
1
E FourierReconstructionFunction
=
Compile@
88x,_Real<,
8y,_Real<<,
Evaluate@rfD
D;
38
Parallel Sampling
ParallelSampleReconstruction@8xsize_,ysize_<D
:=Module@8x,y,FourierimageData,image,newfeaturepoints,n,wlist<,
wlist=listoffrequencies;
ExportEnvironment@FourierReconstructionFunctionD;
FourierimageData=ParallelMap@Apply@FourierReconstructionFunction,#D&,wlist,82
<D;
image=Re@InverseFourier@FourierimageData,FourierParameters→
81,1<D
DPRange@xsizeD,
Range@ysizeDT;
image
D
39Mathem
atica Dem
o 1
40
Mathlinkforbetterperform
ance
(sometimes)
:Begin:
:Function:MLSobolevReconstruction
:Pattern:MLSobolevReconstruction[X__?(VectorQ[#1,NumberQ]&), Y__?(VectorQ[#1,NumberQ]&),...
:Arguments:{X,Y,OX,OY,T,F,gridsize,gamma}
:ArgumentTypes:{RealList,RealList,IntegerList,...
:ReturnType:Manual
:End:
#inc
lude
<m
ath.
h>...
#inc
lude
"m
athl
ink.
h"
#inc
lude
"y
our
own
stuf
f.h"
void
ML
Sobo
levR
econ
stru
ctio
n(do
uble
*X,lo
ng X
coun
t,do
uble
*Y,lo
ng Y
coun
t,...
.
{
mal
loc(
);
do
som
e co
mp
utat
ions
;
ML
Put
Rea
lLis
t(st
dlin
k,da
ta,s
ize)
;
fre
e();
}
41Mathem
atica Dem
o 2
42
(State of the)Art
43
Questions?
Topological A
bduction of Europe -Hom
age to Rene Thom
Salvador Dali
MathVisionTools
http://w
ww.bmi2.bmt.tue.nl/im
age-an
alysis/
44
Ack
nowledgements
•Bart ter Haa
r Romen
y
•Evg
uen
iaBalmachnova
•Rem
coDuits
•LucFlorack
•Frans Kan
ters
•Bram Platel