Yi Yang & Deva Ramanan University of California, Irvine.

43
Articulated Pose Estimation with Flexible Mixtures of Parts Yi Yang & Deva Ramanan University of California, Irvine

Transcript of Yi Yang & Deva Ramanan University of California, Irvine.

  • Slide 1
  • Yi Yang & Deva Ramanan University of California, Irvine
  • Slide 2
  • Goal Articulated pose estimation ( ) recovers the pose of an articulated object which consists of joints and rigid parts
  • Slide 3
  • Applications ActionHCIGaming SegmentationObject Detection
  • Slide 4
  • Unconstrained Images
  • Slide 5
  • Classic Approach Part Representation Head, Torso, Arm, Leg Location, Rotation, Scale Marr & Nishihara 1978
  • Slide 6
  • Classic Approach Part Representation Head, Torso, Arm, Leg Location, Rotation, Scale Fischler & Elschlager 1973 Felzenszwalb & Huttenlocher 2005 Marr & Nishihara 1978 Pictorial Structure Unary Templates Pairwise Springs
  • Slide 7
  • Classic Approach Part Representation Head, Torso, Arm, Leg Location, Rotation, Scale Fischler & Elschlager 1973 Felzenszwalb & Huttenlocher 2005 Marr & Nishihara 1978 Andriluka etc. 2009 Eichner etc. 2009 Johnson & Everingham 2010 Sapp etc. 2010 Singh etc. 2010 Tran & Forsyth 2010 Epshteian & Ullman 2007 Ferrari etc. 2008 Lan & Huttenlocher 2005 Ramanan 2007 Sigal & Black 2006 Wang & Mori 2008 Pictorial Structure Unary Templates Pairwise Springs
  • Slide 8
  • Problem: Wide Variations In-plane rotationForeshortening
  • Slide 9
  • Problem: Wide Variations In-plane rotationForeshortening ScalingOut-of-plane rotation Intra-category variationAspect ratio
  • Slide 10
  • Problem: Wide Variations In-plane rotationForeshortening ScalingOut-of-plane rotation Intra-category variationAspect ratio Nave brute-force evaluation is expensive
  • Slide 11
  • Our Method Mini-Parts Key idea: mini part model can approximate deformations
  • Slide 12
  • Example: Arm Approximation
  • Slide 13
  • Example: Torso Approximation
  • Slide 14
  • Key Advantages Flexibility: General affine warps (orientation, foreshortening, ) Speed: Mixtures of local templates + dynamic programming
  • Slide 15
  • Pictorial Structure Model
  • Slide 16
  • Slide 17
  • Slide 18
  • Our Flexible Mixture Model
  • Slide 19
  • Slide 20
  • Slide 21
  • Slide 22
  • Co-occurrence Bias
  • Slide 23
  • Slide 24
  • Slide 25
  • Inference & Learning Inference
  • Slide 26
  • Inference & Learning Learning Inference
  • Slide 27
  • Benchmark Datasets PARSE Full-body http://www.ics.uci.edu/~drama nan/papers/parse/index.html BUFFY Upper-body http://www.robots.ox.ac.uk/~v gg/data/stickmen/index.html
  • Slide 28
  • Solution: Cluster relative locations of joints w.r.t. parents How to Get Part Mixtures?
  • Slide 29
  • Articulation
  • Slide 30
  • Slide 31
  • Slide 32
  • Qualitative Results
  • Slide 33
  • Quantitative Results on PARSE All previous work use explicitly articulated models Image Parse Testset MethodTotal Ramanan 200727.2 Andrikluka 200955.2 Johnson 200956.4 Singh 201060.9 Johnson 201066.2 Our Model74.9 % of correctly localized limbs
  • Slide 34
  • Quantitative Results on PARSE 1 second per image Image Parse Testset MethodHeadTorsoU. LegsL. LegsU. ArmsL. ArmsTotal Ramanan 200752.137.531.029.017.513.627.2 Andrikluka 200981.475.663.255.147.631.755.2 Johnson 200977.668.861.554.953.239.356.4 Singh 201091.276.671.564.950.034.260.9 Johnson 201085.476.173.465.464.746.966.2 Our Model97.693.283.975.172.048.374.9 % of correctly localized limbs
  • Slide 35
  • More Parts and Mixtures Help 14 parts (joints)27 parts (joints + midpoints)
  • Slide 36
  • Quantitative Results on BUFFY Our algorithm = 5 seconds -vs- Next best = 5 minutes Subset of Buffy Testset MethodTotal Tran 201062.3 Andrikluka 200973.5 Eichner 200980.1 Sapp 2010a85.9 Sapp 2010b85.5 Our Model89.1 % of correctly localized limbs
  • Slide 37
  • Quantitative Results on BUFFY All previous work use explicitly articulated models Subset of Buffy Testset MethodHeadTorsoU. ArmsL. ArmsTotal Tran 2010 --- 62.3 Andrikluka 200990.795.579.341.273.5 Eichner 200998.797.982.859.880.1 Sapp 2010a100 91.165.785.9 Sapp 2010b10096.295.363.085.5 Our Model10099.696.670.989.1 % of correctly localized limbs
  • Slide 38
  • Human Detection
  • Slide 39
  • Conclusion Model affine warps with a part-based model
  • Slide 40
  • Conclusion Model affine warps with a part-based model Exponential set of pictorial structures
  • Slide 41
  • Conclusion Model affine warps with a part-based model Exponential set of pictorial structures Flexible vs rigid relations
  • Slide 42
  • Conclusion Model affine warps with a part-based model Exponential set of pictorial structures Flexible vs rigid relations Supervision helps
  • Slide 43
  • Thank you