1

Object Recognition as Ranking Holistic Figure-Ground

Hypotheses

Fuxin Li and Joao Carreira and Cristian Sminchisescu

2

Outline Introduction Method Overview Segment Categorization Segment Post-Processing Experiment Conclusion

3

Introduction Object detector : Top-down approaches

4

Introduction Semantic segmentation results

produced by our algorithm

5

Introduction Ideally segment1. Can model entire object

2. At least sufficiently distinct parts of them

6

7

Introduction Constrained Parametric Min Cuts

algorithm (CPMC) [6]

8

Method Overview

This paper focus

CPMC

Number of segment

9

Method Overview Recognition framework1. Segment categorization

2. Segment post-processing

10

Segment categorization1. Scoring function2. Sort3. Combine high-rank segment

11

12

Segment post-processing

COW

13

Segment Categorization Multiple Segment and Features Learning Scoring Functions with

Regression Learning the Kernel Hyperparameters Compare with Structural SVM

14

Multiple Segment and Features

Model object appearance ： 1. Extracted four bag of words of SIFT 2. Two on foreground 3. Two on Background, aim to improve

recognition

15

Multiple Segment and Features

Encode shape information ：1. A bag of word of local sharp

contexts [2] : measure similarity between shapes

2. Three pyramid HOGs [5] : classifying images by the object

categories they contain

16

Multiple Segment and Features

Chi-square kernel ：

Computed from each histogram feature and use a weighted sum of such kernel for regression

17

Learning Scoring Functions with Regression

: Image I with ground truth segments

: Segmentation algorithm provides a set

of segment : Denote the K object

categories : Indicator function

18

Learning Scoring Functions with Regression

Quality function :

Measure overlap with all denote

the value for and is the maximal overlap with ground truth segments belonging to , and do not appear

19

Learning Scoring Functions with Regression

Learn the function for each :

use nonlinear SVR(Support Vector Regression) to regress against ,the features extracted from

20

Learning Scoring Functions with Regression

Use kernel trick : : support vector from training set : obtained by the SVR optimizer : maximal score of the

segment : final class of the

segment

21

Learning the Kernel Hyperparameters

Fundamental equation (3) is infeasible to estimate all kernel hyperparameter via grid search

Use subset of data comprised segments that best overlap each ground truth segment

22

Compare with Structural SVM The structural SVM(in [3])

formulation for sliding window prediction is :

23

Connections with Structural SVM

Our algorithm VS Structural SVM Structural SVM score the bounding box

and Our algorithm score the segment

Important advantage1. Guarantee the highest rank for the

ground truth2. Correct ranking for all segment

24

Segment Post-Processing Simple decision rule : avoid the post-

processing and direct choose the segment , cannot detect multiple objects

Our methodology : weighted consolidation of segment and sequential interpretation strategy

25

Segment Post-Processing

26

Segment Post-Processing To decide which segments to

combine

Consider segment with intersection

> 0.75 for combination

27

Segment Post-Processing1. Highest-scoring segment as seed2. Group segments that intersect it3. Generated a final mask 4. Proceed with the next higher rank

segment5. Choose segment that are not

overlapping with 3

28

Segment Post-Processing Generate the score for the pixels in

the mask by (9), only pixels with score > 0.65 are displayed in the mask.

29

Experiments Classification : Caltech-101 Detection : ETHZ Shape classes Segmentation : VOC 2009

30

Classification

31

Classification

32

Detection

33

Detection

34

Detection

35

OWT-UCM Masks

36

Segmentation

Bounding box

37

Conclusion CPMC Categorization Post-

processing Achieve good performance Future work : improve the scalability

to be able to process hundreds of thousands of image