Self-Validated Labeling of Markov Random Fields for Image segmentation

Tzu-Ting Liao

ADVISOR: SHENG-JYH WANG

W. Feng, J. Y. Jia, and Z. Q. Liu, “Self-validated labeling of Markov random fields for image segmentation,” IEEE Transactions on PAMI, 2010.

K-labeling• Image labeling

• Number of label K is known.

40 percent Gaussian NoiseK-labeling segmentationNormalize Cut(NCut)

Labeling qualityLabeling Cost

Self-validated labeling

Self-labeling segmentation(Previous methods)

Self-labeling segmentation(This methods)

Split-and-merge

Graph theoretic approach• Greig: binary labeling problem(s-t graph mincut/maxflow)

• Boykov, Kolmogorov: K-labeling problem

Graph formulation of MRF-based segmentation

• Image I

• MRF • Observation

• Graph formulation

• Undirected graph .

• • second order neighborthood system

Graph formulation of MRF-based segmentation

• Optimal Labeling

• Minimizing the Gibbs Energy

Graph formulation of MRF-based segmentation

• Minimizing the Gibbs Energy

•(K-way) graph cut problemNP-complete

•K=2graph mincut / maxflow problem

𝑆1

𝑆2

𝑆𝑘

Optimal Binary Segmentation• Feature space representation

• Energy assignment

Optimal Binary Segmentation• Potts model

•

Optimal Binary Segmentation• Mincut/Maxflow problem

Graduated Graph Cuts(GGC)• Four types of segment-level operation

• Retaining• Splitting• Merging• Regrouping

Tree-structured graph cuts(TSGC)• Segment Retaining Energy

• Segment Splitting Energy

• Splittable

𝑆

𝑌𝐶0 (𝑆 )

𝐶1 (𝑆 )

𝑌

Tree-structured graph cuts(TSGC)• Algorithm

Input Image I One Segment

Splitting or Retaining

Feature Model

Final Segment Seg(I)

unchange

change

Overpatitioning problem

Net-structured graph cuts(NSGC)• Nearest neighbor of

• Segment merging energy

𝑆 𝑖 𝑆 𝑗

𝑌 𝑖 𝑌 𝑗

𝐶0 (𝑆𝑖 )

𝐶1 (𝑆 𝑖 )

𝐶0 (𝑆 𝑗 )

𝐶1 (𝑆 𝑗 )(𝐷𝑖𝑠𝑡 (𝐶0 ,𝐶1)=𝑚𝑖𝑛𝑖𝐷 (𝑀 𝑖

𝑡 ,𝐶1−𝑡 ))

Net-structured graph cuts(NSGC)

Input Image I One Segment

Splitting, Retaining

or marging

Feature Model

Final Segment Seg(I)

unchange

change

𝑆 𝑖 𝑆 𝑗 𝑆 𝑖 𝑆 𝑗

𝑆 𝑖 𝑆 𝑗 𝑆 𝑖 𝑆 𝑗

Hierarchical Graph Cuts(HGC)• Complexity of s-t graph cut is O(mn2) .(n vertices, m

arcs)

• Image pyramid(efficiency)

Graduated Graph Cuts(GGC)

Experimental results• Eight methods

• Efficient graph-based segmentation(GBS)• MeanShift• Two K-way graph cuts methods: KGC-I (K-means

+expansion) and KGC-II (K-means + swap)• Isoperimetric graph partitioning (IsoCut)• Tree-structured MRF (TS-MRF)• Data-driven Markov Chain Monte Carlo (DDMCMC)• Normalized cut (NCut)

• CIE L*u*v*

Ro

bu

stn

ess

to

no

ise

Pre

serv

atio

n o

f lo

ng

-ra

ng

e s

oft

bo

un

da

ries

Ro

bu

stn

ess

to

no

ise

Pre

serv

atio

n o

f lo

ng

-ra

ng

e s

oft

bo

un

da

ries

Experimental results

F-measure score to ground truth Average number of segments

Experimental results

Conclusion• Automatically determine the number of labels.

• Balance the labeling accuracy, spatial coherence, and the labeling cost.

• Computationally efficient.

• Independent to initialization.

• Converge to good local minima of the objective energy function.