1

Fast Approximate EnergyMinimization via Graph CutsPresenter Kuang-Jui HsuDate 2011/3/24(Thur.)Outline1.Important Journal & Conference

2.Paper Outline

Important Journal & ConferenceJournal Rankings:

1. TPAMI: IEEE Transactions on Pattern Analysis and Machine Intelligence

2. IJCV: International Journal of Computer Vision

Conference Rankings :

1. CVPR: IEEE Conference on Computer Vision and Pattern Recognition

2. ICCV: International Conference on Computer Vision

3. ECCV: European Conference On Computer Vision

Paper Outline1. Energy minimization in early vision

2. Energy minimization via graph cuts

3. Finding the optimal swap move

4. Finding the optimal expansion move

5. Experimental resultsEnergy minimization in early visionGoal: find a labeling f that assigns each pixel where f is both piecewise smooth and consistent with the observed data.

Energy function: : the extent to which is not piecewise smooth. : the disagreement between f and the observed data.

Date energy: : how appropriate a label is for the pixel p given the observed data

: the set of pairs of adjacent pixels In this paper, authors allow to be arbitrary

Energy minimization in early visionSmooth energy: a critical issue, and many different function have been proposed

Major difficult: the enormous computation cost, but typically these energy functions have many local minima.

Simulated annealing was popularized in computer vision and is widely used since it can optimize an arbitrary energy function.

Minimizing an arbitrary requires exponential time, and as consequence simulated annealing is very slow.

Annealing is inefficient partly because at each step it changes the value of a single pixel.

In special cases such energies can be minimized exactly, but the problem is NP-hard in general.

Energy minimization in early visionAuthors develop algorithms that approximately minimize energy E(f) for an arbitrary finite set of labels under two fairly general classes of interaction potentials V :

a) Semi-metric: on the space , for any pair of labels , , it satisfies two properties. 1. 2. b) Metric: the condition of the semi-metric plus the triangle inequality for any , , in

Compute a labeling which is a local minimum even when very large moves are allowed.Move : gives an arbitrary label to a single pixel

Energy minimization in early visionThe authors algorithms are based on graph cuts.

The authors two algorithms : a) First algorithm: based on --swap moves and works for any semi-metric b) Second algorithm: based on -expansion moves but works only for metric

Note that -expansion moves produce a solution within a known factor of the global minimum of E

Energy minimization via graph cuts Partitions and moves

Algorithms and properties

Graph cutsPartitions and moves Any labeling f can be uniquely represented by a partition of image pixels P= , where is a subset of pixels assigned label l.

One to one correspondence between labelings f and partitions P

- swap: given a pairs of labels , , a move from a partition P (labeling f ) to a new partition (labeling ) if

Difference between P and : some pixels that were labeled in P are now labeled in , and some pixels that were labeled in P are now labeled in .

-expansion: given a label , a move from a partition P (labeling f ) to a new partition (labeling )

An -expansion move allows any set of image pixels to change their labels to .

Algorithms and properties

- swap move algorithm-expansion move algorithm8

11Algorithms and propertiesThree important properties:

Guarantee to terminate in a finite number of cycles; Under the fairly general assumptions, terminate in cycles.

Once the algorithm has terminated, the energy of the resulting labeling is a local minimum with respect to a swap or an expansion move.

Expansion move algorithm produces a labeling f such that where is the global minimum and

Graph cutsKey part: step 3.1, where graph cuts are used to efficiently find Let be a weighted graph with two distinguished vertices called terminals. A cut is a set of edges such that the terminals are separated in the induced graph The cost of cut C, denoted |C|, equals the sum of its edge weights.Step 3.1 uses a single minimum cut on a graph whose size is .

, are terminals. , and are weighted edges

the complete bipartite graphBconsisting of the nodes ofon one side, and the nodes ofon the other, plus the special nodes(on's side) and(on's side). We callBtheinduced graphf of and(http://infolab.stanford.edu/c3/papers/html/bbdiff/node6.html#SECTION00031000000000000000)

Acomplete bipartite graph,G:= (V1+V2,E), is a bipartite graph such that for any two vertices,v1V1andv2V2,v1v2is an edge inG. The complete bipartite graph with partitions of size |V1|=mand |V2|=n, is denotedKm,n.http://en.wikipedia.org/wiki/Complete_bipartite_graph13Finding the optimal swap moveCritical step: Given an input labeling f (partition P) and a pair of labels , , we wish to find a labeling that minimizes E over all labelings within one - swap of f .

The technique is based on computing a labeling corresponding to a minimum cut on a graph

Finding the optimal swap move

1.include two terminals , 2.3.The set consists of , 4.5.Each pixel is connected to the terminals , by edge6.t-links: terminal links( )7.n-links: neighbor links, each pair of pixels which are neighbor connected by an edge 8.The set of edges thus consists of (the t-links) and (the n-links)

Finding the optimal swap move

Weights assigned to the edges are Finding the optimal swap moveAny cut C on must sever (include) exactly one t-link for any pixel

Define a natural labeling corresponding to a cut C on

Lemma 1: A labeling corresponding to a cut on is one - swap away from the initial labeling f.

Finding the optimal swap move Property 1: For any cut C and for any n-links

Finding the optimal swap moveLemma 2 : For any cut C and for any n-link

Finding the optimal swap moveTheorem 1: There is a one to one correspondence between cuts C on and labelings that are one - swap from f . Moreover, the cost of a cut C on is |C| = plus a constant.

Proof : The severed t-links uniquely determine the labels assigned to pixels p and n-links that must to be cut. Compute the cost of a cut C |C|=

Finding the optimal swap moveCorollary 1 : The optimal - swap from f is where C is the minimum cut on a graph

Finding the optimal expansion moveGiven an input labeling f (partition P) and a label , we wish to find a lebeling that minimizes E over all labelings within one -expansion of f .

Solve the problem assuming each is metric, and thus satisfies the triangle inequality.

Based on computing a labeling corresponding to a minimum cut on a graph

Finding the optimal expansion moveThe structure of graphThe set of vertices includes the two terminals and as well as all image pixels For each pair of neighboring pixels separated in the current partition(i.e. ), create an auxiliary vertex Auxiliary nodes are introduced at the boundaries between partition sets .The set of vertices :

Each pixel is connected to the terminals and by t-links , correspondingly. Each pair of neighboring pixels which are not separated by the partition P (i.e. ) is connected by an n-link .For each pair of neighboring pixels {p , q} such that ,create a triplet of edgesWhere a = is the corresponding auxiliary node.

The edges and connected pixels p and q to and the t-link connects the auxiliary node to the terminal

The set of all edges

Finding the optimal expansion move

This defines a natural labeling corresponding to a cut C on

The weights:Finding the optimal expansion moveAny cut C on must sever(include) exactly one t-link for any pixel

Lemma 3 A labeling corresponding to a cut C on is one -expansion away from the initial labeling f.

Finding the optimal expansion moveAssume that is an auxiliary node between the corresponding pair ofneighboring pixels.Property 2 A minimum cut C on satisfies:

Finding the optimal expansion moveLemma 4 If and then the minimum cut C on satisfies ( )

Property 1 holds when we substitute for . We will refer to this as property 1( ).

property 1( ) :

Property 1( ) holds for any cut, and Property 2 holds for a minimum cut. Elementary cut on : A cut that satisfies Properties 1( ) and 2.

Finding the optimal expansion moveTheorem 2 Let be constructed as above given f and . Then there is a one to one correspondence between elementary cuts on and labelings within one -expansion of f. Moreover, for any elementary cut C we have

Proof : The label at the pixel p determines which of the t-links to p is in C . Property 1( ) shows which n-links between pairs of neighboring pixels {p , q} such that should be severed. Similarly , Property 2 determines which of the links in corresponding to {p , q} such that should be cut. The cost of an elementary cut C :

For any pixel =>

Finding the optimal expansion move Lemmas 2 and 4 hold for elementary cuts, since they were based on properties1 and 2

=>

=> The minimum cut is an elementary cut.

Corollary 2 The optimal -expansion from f is where C is the minimum cut on .

Experimental resultsUse three energy function, each with a quadratic

The First energy function , called E1, uses the truncated quadratic as its smoothness term. The second(E2) and third (E3) energy function use the Potts model and the truncated L2 distance, correspondingly, as their smoothness penalty V. Potts model: Truncated distance:

Experimental results

Simulated annealingAuthors algorithm Experimental results

Experimental results