Slide 1

OP2: Information Theoretical Region Merging Approaches and Fusion of Hierarchical Image Segmentation Results Felipe Calderero, Thesis Advisor: Ferran Marqus Barcelona Forum on Ph.D. Research in Communications, Electronics and Signal Processing 21 st October 2010

Slide 2

Outline 1. Introduction 2. Information Theoretical Region Merging 3. Cooperative Region Merging 4. Conclusions Barcelona Forum on Ph.D. Researh in Communications, Electronics and Signal Processing 2

Slide 3

1. What is image segmentation? Image Segmentation Partition of the image into regions (disjoint sets of spatially contiguous pixels) Key step in image analysis Semantically, first level of abstraction Practically, reduction of primitives But image segmentation is a difficult task Barcelona Forum on Ph.D. Researh in Communications, Electronics and Signal Processing 3

Slide 4

1. An ill-posed problem Image Segmentation is an ill-posed problem A unique solution may not exist Different levels of detail Same level of detail Barcelona Forum on Ph.D. Researh in Communications, Electronics and Signal Processing 4

Slide 5

1. An ill-posed problem Image Segmentation is an ill-posed problem A unique solution may not exist Different levels of detail Same level of detail Hierarchical Segmentation Approaches Fusion of (Hierarchical) Segmentation Results Barcelona Forum on Ph.D. Researh in Communications, Electronics and Signal Processing 5

Slide 6

1. PhD Thesis Objectives Objective 1: Hierarchical Segmentation Approaches Objective 2: Fusion of (Hierarchical) Segmentation Results Provide an unsupervised hierarchical solution to the segmentation of generic images Design a generic and scalable segmentation scheme to fuse in an unsupervised manner hierarchical segmentation results Barcelona Forum on Ph.D. Researh in Communications, Electronics and Signal Processing 6

Slide 7

1. PhD Thesis Approach Solutions to Objective 1 and Objective 2 Generic No a priori information Hierarchical solution Unsupervised Bottom-up hierarchy [Marr82] Region Merging Techniques Hierarchy of most representative partitions at different levels of detail Barcelona Forum on Ph.D. Researh in Communications, Electronics and Signal Processing 7

Slide 8

1. PhD Thesis Contributions Contribution 1: Hierarchical Segmentation Approaches Contribution 2: Fusion of (Hierarchical) Segmentation Results Information Theoretical Region Merging Techniques (IT-RM) Cooperative Region Merging Scheme (CRM) Barcelona Forum on Ph.D. Researh in Communications, Electronics and Signal Processing 8

Slide 9

2. Region Merging Techniques Region Merging Hierarchical bottom-up segmentation approaches Specified by Region Model Merging Criterion Merging Order Partition Selection Criterion G A A B B C C D D E E F F G G Binary Partition Tree (BPT) [Garrido99] Efficiency of computation and representation Selection Criterion Hierarchy creation Relevant partition extraction Unsupervised mode Barcelona Forum on Ph.D. Researh in Communications, Electronics and Signal Processing 9

Slide 10

2. Information Theory Region Merging Information Theoretical Region Merging (IT-RM) Statistical and information theoretical framework 1.Region Model i.i.d / Markov region model 2.Merging Criteria Kullback-Leibler / Bhattacharyya Criteria 3.Merging Order Classical / Scale-based 4.Unsupervised mode: Multiple partition selection criterion (statistically relevant) Barcelona Forum on Ph.D. Researh in Communications, Electronics and Signal Processing 10

Slide 11

2. IT-RM Applications Semantic image analysis Original 1 st Significant Partition 2 nd Significant Partition Barcelona Forum on Ph.D. Researh in Communications, Electronics and Signal Processing 11

Slide 12

2. IT-RM Applications Semantic image analysis (textures) Original 1 st Significant Partition 2 nd Significant Partition Barcelona Forum on Ph.D. Researh in Communications, Electronics and Signal Processing 12

Slide 13

2. IT-RM Applications Object-based representation and analysis Barcelona Forum on Ph.D. Research in Communications, Electronics and Signal Processing 13

Slide 14

3. Cooperative Region Merging Motivation: Objective 2 Most IT-RM techniques have similar and accurate performance Instead of selecting, why not combining the set of techniques? Cooperative Region Merging (CRM) Similar to a negotiation process in decision making Barcelona Forum on Ph.D. Research in Communications, Electronics and Signal Processing 14

Slide 15

Characteristics: parallel, scalable, hierarchical, unsupervised, flexible 3. Cooperative Region Merging Cooperative Region Merging 1. Segmentation results are computed independently by each technique (RM step) 2. A basic consensus or agreement is established between the set of techniques (FUSION step) 3. Steps 1 and 2 are repeated while further consensus is possible Barcelona Forum on Ph.D. Research in Communications, Electronics and Signal Processing 15

Slide 16

3. CRM Applications Accuracy and robustness improvement Combining different segmentation techniques 1 st Median Partition 2 nd Median Partition 3 rd Median Partition Original Human Partition Barcelona Forum on Ph.D. Research in Communications, Electronics and Signal Processing 16

Slide 17

3. CRM Applications Fusion of heterogeneous information channels Combining color and depth for object-based segmentation Color image Disparity map 1 st Sign. Partition 2 nd Sign. Partition 3 rd Sign. Partition Barcelona Forum on Ph.D. Research in Communications, Electronics and Signal Processing 17

Slide 18

3. CRM Applications Scalability and flexibility of the fusion scheme Fusion of multispectral band and vegetation classification Vegetation extraction using bands: B, G, R, IRNDVIRGB composition Vegetation extraction using bands: B, G, R, IR + PAN Barcelona Forum on Ph.D. Research in Communications, Electronics and Signal Processing 18

Slide 19

4. Conclusions Unsupervised segmentation of generic images: a challenge Hierarchical Region Merging approach as a possible solution Info. Theoretical Region Merging Segmentation State-of-the-art results in unsupervised manner Relevant image explanations at different levels of detail Application independent object-based semantic tool Cooperative Region Merging Information Fusion Accuracy and robustness improvement in unsupervised manner Scalable, flexible and generic scheme Fusion of homogeneous/heterogeneous information channels Barcelona Forum on Ph.D. Research in Communications, Electronics and Signal Processing 19

Slide 20

Information Theoretical Region Merging Approaches and Fusion of Hierarchical Image Segmentation Results Felipe Calderero felipe.calderero@upf.edu Image Processing Group Pompeu Fabra University (UPF) Barcelona, Spain Ferran Marqus ferran.marques@upc.edu Image Processing Group Universitat Politcnica de Catalunya Barcelona, Spain felipe.calderero@upf.edu ferran.marques@upc.edu

Slide 21

Barcelona Forum on Ph.D. Research in Communications, Electronics and Signal Processing 21

Slide 22

4. Applications CRM: Fusion of color and depth information Barcelona Forum on Ph.D. Research in Communications, Electronics and Signal Processing 22

Slide 23

4. Applications IT-RM: Semantic image analysis Original 1 st Significant Partition 2 nd Significant Partition Barcelona Forum on Ph.D. Research in Communications, Electronics and Signal Processing 23

Slide 24

5. Conclusions Information Theoretical Region Merging State-of-the-art segmentation results without any assumption about the nature of the region Unsupervised extraction of most relevant image explanations at different levels of detail Application independent accurate tool for object-based representation and semantic analysis of generic images Barcelona Forum on Ph.D. Research in Communications, Electronics and Signal Processing 24

Slide 25

5. Conclusions Cooperative region merging Global improvement of the accuracy and the stability of the segmentation results by combining different segmentation approaches Parameter removal solution Fusion of heterogeneous information channels Flexibility to incorporate specificities of the fusion problem A priori information about the fused sources (e.g. channel priority) Joint segmentation and classification stages Barcelona Forum on Ph.D. Research in Communications, Electronics and Signal Processing 25