Presentation on Bayesian Structure from Motion

Nonrigid Structure-from-Motion: Estimating Shape andMotion with Hierarchical Priors

Aaron Karperpaper by

Lorenzo Torresani Aaron Hertzmann Christoph Bregler

October 22, 2013paper: May 2008

Aaron Karper paper by Lorenzo Torresani, Aaron Hertzmann, Christoph Bregler ()Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical PriorsOctober 22, 2013paper: May 2008 1 / 15

1 Goal

2 Primer in Bayesian statistics

3 Modelrefinement: PPCAfurther refinement: Dynamic model

4 Solving for the model

5 Evaluation

6 Questions


Goal

Goal

Given a series of tracking points pj ∈ R2, we want to estimate

shape of the tracked object,

pose of the tracked object,

movement of the camera,

be robust to missing tracking points (e.g. because of occlusion),

be robust to noisy coordinates of tracking points.

The main example is tracking the movement of a face.


Primer in Bayesian statistics


probability as a measure of (un-)certainty.

we are certain about our data

we are uncertain about how it was produced.




p(M|D)︸︷︷︸posterior

=

prior︷︸︸︷p(M)

likelihood︷︸︸︷p(D|M)

p(D)︸︷︷︸model evidence

∝ p(M) p(D|M)

M is a model and is usually described by some parameters.

D is the observed data.




A hierarchical model can be built with hidden/latent variables Z :

D ←− Z ←− θ

p(θ|D) ∝ p(D|θ) p(θ)

= p(D|Z ) p(Z |θ) p(θ)

D ← Z ← θ means p(D|θ,Z ) = p(D|Z )1

1The variables form a Markov chainAaron Karper paper by Lorenzo Torresani, Aaron Hertzmann, Christoph Bregler ()Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical PriorsOctober 22, 2013paper: May 2008 6 / 15



estimation (distribution) for all variables.

marginalizing for better estimates of remaining variables

p(θ|X ) =

∫p(θ|X ,Y = y) dy


Model

Model

pj,t = cj Rt (sj,t + dt) + nj,t

pj,t projected 2d point.

cj scaling.

Rj orthographic projection.

sj,t shape of object.

dt movement of object.

nj,t noise in recognition ∼ N (0, σ).


Model

Model

Estimate all points at the same time:

pt = Gt (st + Dt) + Nt


Model

Model

st = s̄ + Vt zt + mt

Vt basic shapes.

zt description of object in terms of basic shapes.

mt noise in model.


Model refinement: PPCA

Model – refinement: PPCA

Vt zt describes a shape in low dimensions and blows it up into k points in R3.

zt ∼ N (0, I)

More restricted than PCA, because it assumes shapes vary only a little over thebasic shapes.zt are marginalized out.


Model further refinement: Dynamic model

Model – further refinement: Dynamic model

Assume time line:

z1 ∼ N (0, I)

zt = Φ zt−1 + vt

vt ∼ N (0,Q)


Solving for the model

Solving for the model

Squared loss for model to observed mathbfp.

EM2 to find maximum likelihood.

2estimate-maximize, alternate between estimating variables in model andmaximizing


Evaluation

Evaluation

More robust to noise in motion capture than Xiao et al.3 and Brand4

Will not recover correct solution in synthetic data.

3J. Xiao, J. Chai, and T. Kanade, “A Closed-Form Solution to Non- Rigid Shapeand Motion Recovery,”

4M. Brand, “A Direct Method for 3D Factorization of Nonrigid Motion Observed in2D”


Questions

Questions


Presentation on Bayesian Structure from Motion

Education

Transcript of Presentation on Bayesian Structure from Motion