Probabilistic Structure
1
Probabilis tic Structure Body Localization in Still Images Using Hierarchical Models and Hybrid Search Jiayong Zhang 1 , Jiebo Luo 2 , Robert Collins 3 , Yanxi Liu 1 1 Carnegie Mellon University 2 Kodak Research Lab 3 Pennsylvania State University Hierarchical Model Decomposition (I) View-independent tree-structured model. (II) Mixture model, with eight view-dependent components from angles uniformly distributed in [0,2]. (III) Expanded mixture model with increased landmark density. Experiments Trained on CMU Mobo Tested on 340 images, success rate ≈ 40% Bottom-up Detection of Partial Bodies Combining Top-down & Bottom-up Examples assembled from the “preferred” modes. Dynam ic Programming Reweighted SMC w /M C M C Local Optim iz. M ode Analysis & Fusion Edge Skin Tree-structured M odel M ixture ofView -dependentM odels Boundary M odel Single Image Candidate M odes & H yper-m odes Proposal M aps Fg M asks Shape Samples Shape Sam ples Level I Level II Level III Complete candidate sets clustered into hyper-modes (sorted by likelihood scores). The ideal automated scoring function would have the top-ranked mode (leftmost in each row) coinciding with the preferred mode (with yellow frame). Foreground masks from {Q} Face Scal e = 2 Scal e = 3 Scal e = 4 ) ( max Q scale ) ( max Q orient Calf Scal e = 2 Scal e = 3 Scal e = 4 ) ( max Q scale ) ( max Q orient Visualize {Q} Highlights Generic setting: single image, arbitrary pose & viewpoint 3-level model decomposition (2D, landmark- based) Combination of deterministic and stochastic search Assurance of both robustness and accuracy Assumptions Torso ║ imaging plane No external occlusion Hybrid Strategy Combining DP & SMC Step 1. Fit (I) by Dynamic Programming (DP). The search starts from the bottom (feet) to the top (head). The output is a series of proposal maps, together with foreground masks for different body parts. Step 2. Fit (II) by Sequential Monte Carlo (SMC). The search starts from the top (head) to the bottom (feet). Proposal maps from DP are combined with the prior terms of the mixture model into an improved proposal function for the SMC search, while the foreground masks generated from DP are utilized in the computation of SMC importance weights. Step 3. Fit (III) by local optimization, initialized by the SMC output. Physical Topology Marginal Posteriors: (Proposal Maps) SMC DP Proposal map Q k is computed from line segments in the lower part of the body that have not been visited by SMC (e k+1:K ). Prior (proposal) k and likelihood (weight) k are computed from line segments in the upper part that have already been visited (e 1:k-1 ).
description
Level I. Level III. Level II. SMC. Proposal map Q k is computed from line segments in the lower part of the body that have not been visited by SMC (e k+1:K ). - PowerPoint PPT Presentation
Transcript of Probabilistic Structure
Probabilistic Structure
Body Localization in Still Images Using Hierarchical Models and Hybrid SearchJiayong Zhang1, Jiebo Luo2, Robert Collins3, Yanxi Liu1
1Carnegie Mellon University 2Kodak Research Lab 3Pennsylvania State University
Hierarchical Model Decomposition
(I) View-independent tree-structured model.
(II) Mixture model, with eight view-dependent components from angles uniformly distributed in [0,2].
(III) Expanded mixture model with increased landmark density.
Experiments
Trained on CMU Mobo Tested on 340 images, success rate ≈ 40%
Bottom-up Detection of Partial Bodies
Combining Top-down & Bottom-up
Examples assembled from the “preferred” modes.
Dynamic Programming
Reweighted SMC
w/ MCMC
Local Optimiz.
Mode Analysis & Fusion
Edge
Skin
Tree-structured Model Mixture of View-dependent Models Boundary Model
SingleImage
Candidate Modes &
Hyper-modes
Proposal Maps
Fg Masks
Shape Samples
Shape Samples
Level I Level II Level III
Complete candidate sets clustered into hyper-modes (sorted by likelihood
scores). The ideal automated scoring function would have the top-ranked mode (leftmost in each row) coinciding with the preferred mode (with yellow frame).
Foreground masks from {Q}Face
Scale = 2
Scale = 3
Scale = 4
)(max Qscale
)(max Qorient
Calf
Scale = 2
Scale = 3
Scale = 4
)(max Qscale
)(max Qorient
Visualize {Q}
Highlights Generic setting: single image, arbitrary pose & viewpoint
3-level model decomposition (2D, landmark-based)
Combination of deterministic and stochastic search
Assurance of both robustness and accuracy
Assumptions
Torso ║ imaging plane No external occlusion
Hybrid Strategy Combining DP & SMC
Step 1. Fit (I) by Dynamic Programming (DP). The search starts from the bottom (feet) to the top (head). The output is a series of proposal maps, together with foreground masks for different body parts.
Step 2. Fit (II) by Sequential Monte Carlo (SMC). The search starts from the top (head) to the bottom (feet). Proposal maps from DP are combined with the prior terms of the mixture model into an improved proposal function for the SMC search, while the foreground masks generated from DP are utilized in the computation of SMC importance weights.
Step 3. Fit (III) by local optimization, initialized by the SMC output.
Physical Topology
Marginal Posteriors: (Proposal Maps)
SMC
DP
Proposal map Qk is computed from line segments in the lower part of the body that have not been visited by SMC (ek+1:K).
Prior (proposal) k and likelihood (weight) k are computed from line segments in the upper part that have already been visited (e1:k-1).
