1 CVR05University of California Berkeley

Cue Integration in Figure/Ground LabelingCue Integration in Figure/Ground Labeling

Xiaofeng Ren, Charless Fowlkes, Jitendra MalikXiaofeng Ren, Charless Fowlkes, Jitendra Malik

2 CVR05University of California Berkeley

IntroductionIntroduction

CRF

Conditional Random Fields on triangulated images, trained to integrate low/mid/high-level grouping cues

Approach:

3 CVR05University of California Berkeley

Joint Contour/Region InferenceJoint Contour/Region Inference

Xe

Xe

Xe

Xe

Xe

Xe

Xe

Xe

Xe

XeXeXe

Xe

Xe

Xe

Xe

Xe

Xe

Yt

Yt

Yt

Yt

Yt

Yt

Yt

Yt

YtYt

Contour variables {Xe}

Region variables {Yt}

Object variables {Z}

Z

Integrating {Xe},{Yt} and{Z}: low/mid/high-level cues

4 CVR05University of California Berkeley

Grouping CuesGrouping Cues• Low-level Cues

– Edge energy along edge e

– Brightness/texture similarity between two regions s and t

• Mid-level Cues– Edge collinearity and junction frequency at

vertex V

– Consistency between edge e and two adjoining regions s and t

• High-level Cues– Texture similarity of region t to exemplars

– Compatibility of region shape with pose

– Compatibility of local edge shape with pose

• Low-level Cues– Edge energy along edge e

– Brightness/texture similarity between two regions s and t

• Mid-level Cues– Edge collinearity and junction frequency at

vertex V

– Consistency between edge e and two adjoining regions s and t

• High-level Cues– Texture similarity of region t to exemplars

– Compatibility of region shape with pose

– Compatibility of local edge shape with pose

L1(Xe|I)L2(Ys,Yt|I)

M1(XV|I)

M2(Xe,Ys,Yt)

H1(Yt|I)H2(Yt,Z|I)H3(Xe,Z|I)

5 CVR05University of California Berkeley

Conditional Random Fields for Cue IntegrationConditional Random Fields for Cue Integration

,,|,exp),(

1,,, IZYXE

IZIZYXP

ts

tse

e IYYLIXLE,

21 |,|

ts

etsV

V XYYMIXM,

21 ,,|

e

et

tt

t IZXHIZYHIYH |,|,| 321

Estimate the marginal posteriors of X, Y and Z

6 CVR05University of California Berkeley

H3(Xe,Z|I): local shape and poseH3(Xe,Z|I): local shape and pose

shapeme i(horizontal line) distribution ON(x,y,i)

shapeme j(vertical pairs) distribution ON(x,y,j)

Let S(x,y) be the shapeme at image location (x,y); (xo,yo) be the object location in Z. Compute average log likelihood SON(e,Z) as:

eyx

oo yxSyyxxONe ,

),(,,log1

eOFFOFF

eONONe

XZeS

XZeSIZXH

),(

),(|,3

Then we have:

SOFF(e,Z) is defined similarly.

7 CVR05University of California Berkeley

Training/TestingTraining/Testing• Trained on half (172) of the grayscale horse images from

the [Borenstein & Ullman 02] Horse Dataset.

• Use human-marked segmentations to construct groundtruth labels on both CDT edges and triangles.

• Uses loopy belief propagation for approximate inference; takes < 1 second to converge for a typical image.

• Parameter estimation with gradient descent for maximum likelihood; converges in 1000 iterations.

• Tested on the other half of the horse images in grayscale.

• Quantitative evaluation against groundtruth: precision-recall curves for both contours and regions.

• Trained on half (172) of the grayscale horse images from the [Borenstein & Ullman 02] Horse Dataset.

• Use human-marked segmentations to construct groundtruth labels on both CDT edges and triangles.

• Uses loopy belief propagation for approximate inference; takes < 1 second to converge for a typical image.

• Parameter estimation with gradient descent for maximum likelihood; converges in 1000 iterations.

• Tested on the other half of the horse images in grayscale.

• Quantitative evaluation against groundtruth: precision-recall curves for both contours and regions.

8 CVR05University of California Berkeley

9 CVR05University of California Berkeley

10 CVR05University of California Berkeley

ResultsResults

Input Input Pb Output Contour Output Figure

11 CVR05University of California Berkeley

Input Input Pb Output Contour Output Figure

12 CVR05University of California Berkeley

Input Input Pb Output Contour Output Figure

13 CVR05University of California Berkeley

ConclusionConclusion

14 CVR05University of California Berkeley

Thank You

15 CVR05University of California Berkeley