CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐

*Computer Vision Laboratory (CVLab), EPFL, 1015 Lausanne, Switzerland °Department Computer Science, University of Toronto, M5S 3H5, Canada

ÉCOLE POLYTECHNIQUE FÉDÉRALE DE LAUSANNE

CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐ CVLAB.EPFL.CH⏐

Monocular 3-D Tracking of the Golf Swing Raquel Urtasun*, David J. Fleet° and Pascal Fua*

Problem: Model-based 3D people tracking from monocular video.

Approach: Deterministic nonlinear optimization based on   subspace model for pose trajectories   detection of key poses for automatic initialization   background subtraction   3-frame correspondence

Result: With generic optimization methods the tracker fits smooth motions to monocular video. A multiple hypothesis tracker (e.g., particle filter) is not needed to resolve ambiguities.

This work was supported (in part) by the Swiss National Science Foundation. http://cvlab.epfl.ch/~rurtasun

⊥

Tracking as Deterministic Hill-Climbing

Minimize

With respect to

Tracking Objective function

1) Consistency with 2D joint detection

2) Silhouette consistency: Reprojection inside mask

3) 2D Correspondences in a 3 frame bundle adjustment

[ ]ffm GG ,...,,,...,,,..., 111 µµααφ =

€

∂F∂α i

=∂θ j

∂α i

∂F∂θ jj=1

ndof

∑ ∂F∂µt

=∂θ j

∂µt

∂F∂θ jj=1

ndof

∑∂θ j

∂µt

= α i

∂Θij

∂µti=1

m

∑∂θ j

∂α i

=Θij

Tracking a golf swing

Conclusion

Deterministic approach Fully 3D motion Lower cost than probabilistic Applications: Sport, Orthopedics Motion synthesis

Complementarity of the function terms

Initialization

Mean motion

Tracking an approach swing

Future work

Larger database Non linear models Multi-activity database

Prior Motion Model PCA is used to learn a subspace model for pose trajectories (ie. sequences of joint angles from kinematic human model).  10 examples same subject CMU database  PCA: An eigenvector is the whole motion

€

ψ = ψµ1,…,ψµN[ ]T ≈ Θ0 + α ii=1

m

∑ Θi

[ ] )()()(1

0 ti

m

iit

Ttt µαµψµψ µ Θ+Θ≈= ∑

=

α: style coefficient

µ: virtual time G: global motion

€

F = wtypei

i=1

nobs

∑ Obstypei (xi,S)2

+ wG Gt − ˆ G t2

+ wµ µt − ˆ µ t( )2+ wµ µt − ˆ µ t( )2

+ wα α t − ˆ α t( )2

i=1

m

∑

Motion:

Pose:

Key Pose Detection

4 key poses: