N -R S M · 2012-07-18 · Thin-plate FEM model. Tuning: Young’s modulus E and Poisson’s ratio...
1
F INITE E LEMENT BASED S EQUENTIAL B AYESIAN N ON -R IGID S TRUCTURE FROM M OTION A NTONIO A GUDO ,B EGOÑA C ALVO AND J. M. M. M ONTIEL {aagudo, bcalvo, josemari}@unizar.es C ONTRIBUTIONS Non-Rigid Structure from Motion embedding Navier’s equations coded as FEM (Finite Ele- ment Method) within Bayesian sequential esti- mator coded as EKF (Extended Kalman Filter): DA (Data Association) is computed. Clas- sically, DA assumed as prior (2D tracking) Potential 30 Hz real-time performance Full-perspective camera, no orthographic simplified camera Dealing with isometric and non-isometric scenes just by tuning Can cope with partial occlusions No boundary points have to be identified Able to perform even with small size maps E XPERIMENTAL R ESULTS The proposed method is validated with real im- age sequences 320 × 240. The first sequence frames are used to estimate structure at rest by means of rigid EKF-SLAM Multiply Deformed Silicone Sequence Non-Isometric deformation Accurate and consistent wrt ground truth Waving hand-held camera Material Parameter Tuning Parameter Silicone Paper h (m) 1.5 · 10 -3 1.0 · 10 -4 ν 0.499 0.499 Δf Eh (m) (std.) 1.5 · 10 -5 2.0 · 10 -9 Paper Sequences Frame #274 Frame #274 0.65 0.7 Z [m] 01 0.15 0.05 0.1 0.6 Z -0.1 -0.05 0 0.05 0.1 -0.1 -0.05 0 X [m] Y [m] Frame #413 Frame #413 0.65 0.7 Z [m] 01 0.15 0.05 0.1 0.6 Z -0.1 -0.05 0 0.05 0.1 -0.1 -0.05 0 X [m] Y [m] Frame #443 Frame #443 0.65 0.7 Z [m] 01 0.15 0.05 0.1 0.6 Z -0.1 -0.05 0 0.05 0.1 -0.1 -0.05 0 X [m] Y [m] Frame #495 Frame #495 0.65 0.7 Z [m] 01 0.15 0.05 0.1 0.6 Z -0.1 -0.05 0 0.05 0.1 -0.1 -0.05 0 X [m] Y [m] Frame #256 0.75 Frame #256 0.7 Z [m] 0 15 0 05 0.1 0.65 Z -0.1 -0.05 0 0.05 0.1 0.15 -0.1 -0.05 0 0.05 X [m] Y [m] Frame #412 0.75 Frame #412 0.7 Z [m] 0 15 0 05 0.1 0.65 Z -0.1 -0.05 0 0.05 0.1 0.15 -0.1 -0.05 0 0.05 X [m] Y [m] Frame #596 0.75 Frame #596 0.7 Z [m] 0 15 0 05 0.1 0.65 Z -0.1 -0.05 0 0.05 0.1 0.15 -0.1 -0.05 0 0.05 X [m] Y [m] Isometric deformation Natural landmarks Irregular triangles Static or waving camera Computation Time ≈ 1 sec frame ≈ 100 map points Matlab R EFERENCES [1] Antonio Agudo, Begoña Calvo and J. M. M. Montiel. FEM models to code non-rigid EKF monocular SLAM. In International Workshop 4DMOD (ICCV) 2011 [2] J. A. Castellanos, J. Neira and J. D. Tardós. Limits to the consistency of EKF-based SLAM. In IFAC Symposium on Intelligent Autonomous Vehicles 2004 C ONCLUSIONS EKF+FEM can solve NRSfM Low cost linear FEM is accurate enough Rigid boundary points removed Complexity depends on map size but not on number of frames F UTURE W ORK 30 Hz Real-time performance Comparison with respect to state-of-the- art NRSfM methods Determining extreme deformations limits Processing medical endoscope sequences A CKNOWLEDGMENTS This work was supported by the Spanish MICINN DIP2009-07130 and DPI2011-27939- C02-01 grants. Thanks to J. Civera and Óscar G. Grasa for fruitful discussion and software assis- tance EKF+FEM N ON -R IGID S TRUCTURE S EQUENTIAL E STIMATION cam k cam k+m 35 trajectory k cam k+n ∆S W ∆S W 5 20 25 30 35 0 20 25 30 35 ∆S 0 10 20 0 5 10 15 0 0 10 20 30 -5 0 5 10 15 10 20 30 40 50 -20 -10 0 boundary points non-rigid non-rigid structure k+n non-rigid structure k+m 30 40 50 -15 -10 -5 0 40 50 -15 -10 5 non-rigid points boundary points structure k k+n Thin-plate FEM model. Tuning: Young’s modulus E and Poisson’s ratio ν FEM linear system: Ka = f No prior boundary points: K + matrix Moore-Penrose (MP) pseudoinverse x v W a a = a nr x v a r x r a a = a nr x r = x v + a r a a nr Camera-centric pose prediction: g r = r C k k +1 q C k k +1 v C k k +1 ω C k k +1 = r C k k +(v C k k + V C )Δt q C k k × q ((ω C k k +Ω C )Δt) v C k k + V C ω C k k +Ω C Non-rigid FEM-based scene prediction: y C k k +1 = y C k k + K + k ΔS C Normalized forces: ΔS C i = 1 Eh ( Δf C xi , Δf C yi , Δf C zi ) > O ( n 3 ) computational complexity: • EKF O ( n 3 ) • MP pseudoinverse O ( 12n 3 ) DA EKF prediction elliptical 3σ search re- gion
Transcript of N -R S M · 2012-07-18 · Thin-plate FEM model. Tuning: Young’s modulus E and Poisson’s ratio...
FINITE ELEMENT BASED SEQUENTIAL BAYESIANNON-RIGID STRUCTURE FROM MOTION
ANTONIO AGUDO, BEGOÑA CALVO AND J. M. M. MONTIEL aagudo, bcalvo, [email protected]
CONTRIBUTIONS
Non-Rigid Structure from Motion embeddingNavier’s equations coded as FEM (Finite Ele-ment Method) within Bayesian sequential esti-mator coded as EKF (Extended Kalman Filter):
DA (Data Association) is computed. Clas-sically, DA assumed as prior (2D tracking)Potential 30 Hz real-time performance
Full-perspective camera, no orthographicsimplified cameraDealing with isometric and non-isometricscenes just by tuningCan cope with partial occlusionsNo boundary points have to be identifiedAble to perform even with small size maps
EXPERIMENTAL RESULTS
The proposed method is validated with real im-age sequences 320 × 240. The first sequenceframes are used to estimate structure at rest bymeans of rigid EKF-SLAM
Multiply Deformed Silicone Sequence
Non-Isometric deformation
Accurate and consistent wrt ground truth
Waving hand-held camera
Material Parameter Tuning
Parameter Silicone Paperh (m) 1.5 · 10−3 1.0 · 10−4
ν 0.499 0.499∆fEh (m) (std.) 1.5 · 10−5 2.0 · 10−9
Paper Sequences
Frame #274Frame #274
0.65
0.7
Z [m
]
0 10.150.05
0.10.6
Z
-0.1-0.05
00.05
0.1
-0.1
-0.05
0
X [m]Y [m]
Frame #413Frame #413
0.65
0.7
Z [m
]
0 10.150.05
0.10.6
Z
-0.1-0.05
00.05
0.1
-0.1
-0.05
0
X [m]Y [m]
Frame #443Frame #443
0.65
0.7
Z [m
]
0 10.150.05
0.10.6
Z
-0.1-0.05
00.05
0.1
-0.1
-0.05
0
X [m]Y [m]
Frame #495Frame #495
0.65
0.7
Z [m
]
0 10.150.05
0.10.6
Z
-0.1-0.05
00.05
0.1
-0.1
-0.05
0
X [m]Y [m]
Frame #256
0.75
Frame #256
0.7
Z [m
]
0 150 050.1
0.65
Z
-0.1-0.05
00.05
0.10.15
-0.1-0.05
00.05
X [m]Y [m]
Frame #412
0.75
Frame #412
0.7
Z [m
]
0 150 050.1
0.65
Z
-0.1-0.05
00.05
0.10.15
-0.1-0.05
00.05
X [m]Y [m]
Frame #596
0.75
Frame #596
0.7
Z [m
]
0 150 050.1
0.65
Z
-0.1-0.05
00.05
0.10.15
-0.1-0.05
00.05
X [m]Y [m]
Isometric deformation
Natural landmarks
Irregular triangles
Static or waving camera
Computation Time
≈ 1 secframe ≈ 100 map points Matlab
REFERENCES
[1] Antonio Agudo, Begoña Calvo and J. M. M. Montiel.FEM models to code non-rigid EKF monocular SLAM.In International Workshop 4DMOD (ICCV) 2011
[2] J. A. Castellanos, J. Neira and J. D. Tardós. Limits to theconsistency of EKF-based SLAM. In IFAC Symposium onIntelligent Autonomous Vehicles 2004
CONCLUSIONSEKF+FEM can solve NRSfMLow cost linear FEM is accurate enoughRigid boundary points removedComplexity depends on map size but noton number of frames
FUTURE WORK30 Hz Real-time performanceComparison with respect to state-of-the-art NRSfM methodsDetermining extreme deformations limitsProcessing medical endoscope sequences
ACKNOWLEDGMENTSThis work was supported by the SpanishMICINN DIP2009-07130 and DPI2011-27939-C02-01 grants. Thanks to J. Civera and Óscar G.Grasa for fruitful discussion and software assis-tance
EKF+FEM NON-RIGID STRUCTURE SEQUENTIAL ESTIMATION
cam kcam k+m
35
trajectoryk cam k+n
∆SW
∆SW
25 20
25
30
35
0
20253035∆S
0
10
20
05
1015
200
10
20
30-5
05
1015
1020
3040
50 -20
-10
0
10
20
boundary pointsnon-rigid non-rigid structure k+n
non-rigid structure k+m
30
40
50-15
-10-5
0 40
50-15
-105
non-rigid pointsboundary pointsg
structure kk+n
Thin-plate FEM model. Tuning: Young’smodulus E and Poisson’s ratio ν
FEM linear system:
K a = f
No prior boundary points: K+ matrixMoore-Penrose (MP) pseudoinverse
CVPR‐12
xv xv
W W a
a = anr
CVPR‐12 CVPR 12
xvar xv
xr
Waaa = anr a
xr = xv +ar
a anr
a = anr + ar
Camera-centric pose prediction:
gr =
rCk
k+1
qCk
k+1
vCk
k+1
ωCk
k+1
=
rCk
k + (vCk
k + VC)∆t
qCk
k × q ((ωCk
k + ΩC)∆t)
vCk
k + VC
ωCk
k + ΩC
Non-rigid FEM-based scene prediction:
yCk
k+1 = yCk
k + K+
k∆SC
Normalized forces:
∆SCi =
1
Eh
(∆fCxi, ∆f
Cyi, ∆f
Czi
)>O(n3)
computational complexity:
• EKF O(n3)
• MP pseudoinverse O(12n3
)DA EKF prediction elliptical 3σ search re-gion
