Graph-Cut Algorithm with Application to Computer Vision Presented by Yongsub Lim Applied Algorithm...

Graph-Cut Algorithm with Appli-cation to Computer VisionPresented by Yongsub Lim

Applied Algorithm Laboratory

Applied Algorithm Laboratory, KAIST 2

Contents

• Image Labeling Problem

• Maxflow-Mincut Theorem (remind)

• Energy Minimization Problem–Graph Construction

• Conclusion

• Our Related Work


Image Labeling Problem

• We want to identify an object in an image

• It can be done by labeling a binary value, 0 or 1, to each pixel

• The goal is to find a labeling minimizing an energy function defined on an image, it is generally NP-Hard


Image Labeling Problem


Maxflow-Mincut Theorem

• Given a directed graph• Definition (Graph-Cut). Two disjoint sub-

sets of nodes such that• s-t cut has a node s in S and t in T• Min-cut is a cut minimizing the capacity

TS and

),( EVG

VTS

TvSu

vuwTSc,

),(),(



• Definition (Flow Network). A directed graph with a source and a sink where an edge has nonnegative capacity

• Definition (Flow). A real-valued function that satisfies the following constraints

–Capacity Constraint: for all –Skew Symmetry: for all–Flow Conservation: for all

),(),(,, vucvufVvu

),(),(,, uvfvufVvu

u ofneighbor

0),(},,{\v

vuftsVu where s is a source and t is a sink

f



• t• s

• 1/2

• 2/5

• 0/4

• 2/2

• 1/5

• 0/3

• 3/3



• The value of , , is

• Maxflow-Mincut Theorem. If is a maxi-mum flow, where is a s-t min-cut

• Thus, we can find a min-cut in graphs by solving the max-flow problem

f

f

sv

vsf of neighbors

),(

|| f

),(|| TScf ),( TS



s

t

100100

100

1

100

1

100100

3

This line deter-minesmin-cut and max-flow, too


Energy Minimization Problem

• To minimize an energy function defined on a pixel-grid graph

• Definition (Submodular). is a submodular function if

Evu

vuuvVv

vv xxxxH),(

),()()(

2}1,0{:

)0,1()1,0()1,1()0,0(



• When all pairwise functions ‘s are sub-modular, we can find a labeling minimiz-ing by a Min-Cut of the graphH

uv

Evu

vuuvVv

vv xxxxH),(

),()()(

all of these are submodular



0 0 0

1 1 1

1 0 1

Min-Cut



• Our goal is to find a binary labeling minimiz-ing an energy function defined on a pixel-grid graph

• An labeling minimizing an energy function is found using a min-cut

• By Maxflow-Mincut theorem, we can com-pute a min-cut using the max-flow algorithm


Graph Construction

• Remaining is how we construct a graph from given an energy function and an im-age

• We should define sets of nodes and edges, and weights of edges


Graph Construction

• A set of nodes–Nodes each of which corresponds to each pixel, a

source and a sink

• A set of edges–If two pixels are adjacent, there is a bidirectional

edge between corresponding nodes–Edges from a source to all nodes except a sink–Edges from all nodes except a source to a sink


Graph Constructions

t


Graph Construction

• Defining weights

• Assume that there is a two-pixel image

4)1(,3)0(

2)1(,1)0(

22

11

xx

xx

7)1,1(,8)0,1(

5)1,0(,3)0,0(

2121

2121

xxxx

xxxx

Evu

vuuvVv

vv xxxxH),(

),()()(


Graph Construction

21212121

221121

7385

432),(

xxxxxxxx

xxxxxxE

4)1(,3)0(

2)1(,1)0(

22

11

xx

xx

7)1,1(,8)0,1(

5)1,0(,3)0,0(

2121

2121

xxxx

xxxx


Graph Construction

4)1(,3)0(

2)1(,1)0(

22

11

xx

xx

7)1,1(,8)0,1(

5)1,0(,3)0,0(

2121

2121

xxxx

xxxx

121 ),( xxxE

s

t

1x 2x

1


Graph Construction

4)1(,3)0(

2)1(,1)0(

22

11

xx

xx

7)1,1(,8)0,1(

5)1,0(,3)0,0(

2121

2121

xxxx

xxxx

1121 2),( xxxxE

s

t

1x 2x

1

2


Graph Construction

4)1(,3)0(

2)1(,1)0(

22

11

xx

xx

7)1,1(,8)0,1(

5)1,0(,3)0,0(

2121

2121

xxxx

xxxx

221121 432),( xxxxxxE

3

s

t

1x 2x

1

2 4


Graph Construction

4)1(,3)0(

2)1(,1)0(

22

11

xx

xx

7)1,1(,8)0,1(

5)1,0(,3)0,0(

2121

2121

xxxx

xxxx

2121221121 85432),( xxxxxxxxxxE

s

t

1x 2x

1 3

2 45

8


Graph Construction

4)1(,3)0(

2)1(,1)0(

22

11

xx

xx

7)1,1(,8)0,1(

5)1,0(,3)0,0(

2121

2121

xxxx

xxxx

Note that21221121

21221121

xxxxxxxx

xxxxxxxx

Now, we can embed all terms of an energy function to a graph as weights


Graph Construction

• Finally we get the following graph

s

t

1x 2x

c d

a be

f


Graph Construction

• We need two verifications

• We should verify that the capacity of a s-t min-cut is same with the minimum energy

• And that all weights are non-negative for the max-flow algorithm


Graph Construction

s

t

1x 2x

)1(1x

)0(1x

)1(2x

)0(2x

)1,0(21xx

)0,1(21xx

21212121221121 7853432),( xxxxxxxxxxxxxxE

The graph, which we construct, has a s-t min-cut capacity same with the minimum of an energy function


Graph Construction

s

t

1x 2x

)1(1x

)0(1x

)1(2x

)0(2x

)1,0(21xx

)0,1(21xx




Graph Construction

s

t

1 1

)1(1x

)0(1x

)1(2x

)0(2x

)1,0(21xx

)0,1(21xx




Graph Construction

s

t

1 1

)1(1x

)0(1x

)1(2x

)0(2x

)1,0(21xx

)0,1(21xx



7 where

)(

)(7

21

2122

211121

nn

xxxn

xxxnxx


Graph Construction

s

t

1x 2x

)1(1x

)0(1x

)1(2x

)0(2x

)1,0(21xx

)0,1(21xx




Graph Construction

s

t

0 1

)1(1x

)0(1x

)1(2x

)0(2x

)1,0(21xx

)0,1(21xx




Graph Construction

• Are all weights non-negative?

• First, weights from unary terms may nega-tive

• It does not change a minimum labeling even if we add a constant term to the en-ergy function

4)1(,3)0(

2)1(,1)0(

22

11

xx

xx

7)1,1(,8)0,1(

5)1,0(,3)0,0(

2121

2121

xxxx

xxxx


Graph Construction

s

t

1x 2x

c d

a be

f

)(xE

b or d is negative


Graph Construction

• Second, weights from pairwise terms are always non-negative because of the sub-modularity

4)1(,3)0(

2)1(,1)0(

22

11

xx

xx

7)1,1(,8)0,1(

5)1,0(,3)0,0(

2121

2121

xxxx

xxxx


Graph Construction

2122112121 xxkxxkxnxxxm

nnnmmm

xxxnxxxnxnx

xxxmxxxmxxm

2121

2122211121

2122211121

and where

)( )(

)( )(

2122221111 )()( xxnmkxxnmk


Graph Construction

2122112121 xxkxxkxnxxxm

itysubmodularby 21 kknm

2122221111 )()( xxnmkxxnmk

2121 and where nnnmmm

11111 such that and fix We knmnm

222Then knm ∴ All of weights from pairwise terms are always non-negative


Graph Construction

• All of weights are non-negative• We can compute a min-cut through

Maxflow-Mincut Theorem• And obtain a labeling minimizing an en-

ergy functions

t

1x 2x

c d

a be

f


Conclusion

• Given an energy function defined on an image

• Minimizing is generally NP-Hard• For the class of energy functions whose

pairwise functions are all submodular, we can compute a minimum labeling through a graph min-cut in polynomial time

Evu

vuuvVv

vv xxxxH),(

),()()(

H


Our Related Work

• We want a minimum labeling with fixed 1’s

–Label count = # of 1’s labeled

• Previous work computes minimum label-ing for some label counts

• Our recent work computes approximate minimum labeling for almost all label counts (submitted to CVPR10)


Our Related Work

original Nodecomposition

3x3decomposition

5x5decomposition

Used image is of size 300x300. Results for label count 54118.


Our Related Work

No decomposition 3x3 decomposition 5x5 decomposition

The average energy value 1 1.00141 1.00308

The number of label counts 0.176454 0.998678 0.9999944


Q&A


Thanks

Graph-Cut Algorithm with Application to Computer Vision Presented by Yongsub Lim Applied Algorithm...

Documents

Transcript of Graph-Cut Algorithm with Application to Computer Vision Presented by Yongsub Lim Applied Algorithm...