FINITE ELEMENT BASED SEQUENTIAL BAYESIANNON-RIGID STRUCTURE FROM MOTION

ANTONIO AGUDO, BEGOÑA CALVO AND J. M. M. MONTIEL aagudo, bcalvo, josemari@unizar.es

CONTRIBUTIONS

Non-Rigid Structure from Motion embeddingNavier’s equations coded as FEM (Finite Ele-ment Method) within Bayesian sequential esti-mator coded as EKF (Extended Kalman Filter):

DA (Data Association) is computed. Clas-sically, DA assumed as prior (2D tracking)Potential 30 Hz real-time performance

Full-perspective camera, no orthographicsimplified cameraDealing with isometric and non-isometricscenes just by tuningCan cope with partial occlusionsNo boundary points have to be identifiedAble to perform even with small size maps

EXPERIMENTAL RESULTS

The proposed method is validated with real im-age sequences 320 × 240. The first sequenceframes are used to estimate structure at rest bymeans of rigid EKF-SLAM

Multiply Deformed Silicone Sequence

Non-Isometric deformation

Accurate and consistent wrt ground truth

Waving hand-held camera

Material Parameter Tuning

Parameter Silicone Paperh (m) 1.5 · 10−3 1.0 · 10−4

ν 0.499 0.499∆fEh (m) (std.) 1.5 · 10−5 2.0 · 10−9

Paper Sequences

Frame #274Frame #274

0.65

0.7

Z [m

]

0 10.150.05

0.10.6

Z

-0.1-0.05

00.05

0.1

-0.1

-0.05

0

X [m]Y [m]

Frame #413Frame #413

0.65

0.7

Z [m

]

0 10.150.05

0.10.6

Z

-0.1-0.05

00.05

0.1

-0.1

-0.05

0

X [m]Y [m]

Frame #443Frame #443

0.65

0.7

Z [m

]

0 10.150.05

0.10.6

Z

-0.1-0.05

00.05

0.1

-0.1

-0.05

0

X [m]Y [m]

Frame #495Frame #495

0.65

0.7

Z [m

]

0 10.150.05

0.10.6

Z

-0.1-0.05

00.05

0.1

-0.1

-0.05

0

X [m]Y [m]

Frame #256

0.75

Frame #256

0.7

Z [m

]

0 150 050.1

0.65

Z

-0.1-0.05

00.05

0.10.15

-0.1-0.05

00.05

X [m]Y [m]

Frame #412

0.75

Frame #412

0.7

Z [m

]

0 150 050.1

0.65

Z

-0.1-0.05

00.05

0.10.15

-0.1-0.05

00.05

X [m]Y [m]

Frame #596

0.75

Frame #596

0.7

Z [m

]

0 150 050.1

0.65

Z

-0.1-0.05

00.05

0.10.15

-0.1-0.05

00.05

X [m]Y [m]

Isometric deformation

Natural landmarks

Irregular triangles

Static or waving camera

Computation Time

≈ 1 secframe ≈ 100 map points Matlab

REFERENCES

[1] Antonio Agudo, Begoña Calvo and J. M. M. Montiel.FEM models to code non-rigid EKF monocular SLAM.In International Workshop 4DMOD (ICCV) 2011

[2] J. A. Castellanos, J. Neira and J. D. Tardós. Limits to theconsistency of EKF-based SLAM. In IFAC Symposium onIntelligent Autonomous Vehicles 2004

CONCLUSIONSEKF+FEM can solve NRSfMLow cost linear FEM is accurate enoughRigid boundary points removedComplexity depends on map size but noton number of frames

FUTURE WORK30 Hz Real-time performanceComparison with respect to state-of-the-art NRSfM methodsDetermining extreme deformations limitsProcessing medical endoscope sequences

ACKNOWLEDGMENTSThis work was supported by the SpanishMICINN DIP2009-07130 and DPI2011-27939-C02-01 grants. Thanks to J. Civera and Óscar G.Grasa for fruitful discussion and software assis-tance

EKF+FEM NON-RIGID STRUCTURE SEQUENTIAL ESTIMATION

cam kcam k+m

35

trajectoryk cam k+n

∆SW

∆SW

25 20

25

30

35

0

20253035∆S

0

10

20

05

1015

200

10

20

30-5

05

1015

1020

3040

50 -20

-10

0

10

20

boundary pointsnon-rigid non-rigid structure k+n

non-rigid structure k+m

30

40

50-15

-10-5

0 40

50-15

-105

non-rigid pointsboundary pointsg

structure kk+n

Thin-plate FEM model. Tuning: Young’smodulus E and Poisson’s ratio ν

FEM linear system:

K a = f

No prior boundary points: K+ matrixMoore-Penrose (MP) pseudoinverse

CVPR‐12

xv xv

W W a

a = anr

CVPR‐12 CVPR 12

xvar xv

xr

Waaa = anr a

xr = xv +ar

a anr

a = anr + ar

Camera-centric pose prediction:

gr =

rCk

k+1

qCk

k+1

vCk

k+1

ωCk

k+1

=

rCk

k + (vCk

k + VC)∆t

qCk

k × q ((ωCk

k + ΩC)∆t)

vCk

k + VC

ωCk

k + ΩC

Non-rigid FEM-based scene prediction:

yCk

k+1 = yCk

k + K+

k∆SC

Normalized forces:

∆SCi =

1

Eh

(∆fCxi, ∆f

Cyi, ∆f

Czi

)>O(n3)

computational complexity:

• EKF O(n3)

• MP pseudoinverse O(12n3

)DA EKF prediction elliptical 3σ search re-gion