Comp 775: Graph Cuts and Continuous Maximal Flows

Marc Niethammer, Stephen PizerDepartment of Computer Science

University of North Carolina, Chapel Hill

2

Representations

BackgroundGraph Cuts and Continuous Maximal Flows

Classifying individual pixels versus finding an optimal separatingcurve/surface between object and background.

3

From curve evolution to pixel/voxel labeling

BackgroundGraph Cuts and Continuous Maximal Flows

Example: Chan-Vese model Indicator function

Putting it all together: Relaxed indicator function

4

Convex vs. Non-Convex

BackgroundGraph Cuts and Continuous Maximal Flows

Non-convex

Initial contour

Initial contour Final contour

Final contour

Images: Bresson et al.

Convex

Convex

Prone to locallyoptimal solutions.

Independent ofinitial condition,optimal solution isguaranteed.

5

From Curve Evolution to Pixel/Voxel Labeling

BackgroundGraph Cuts and Continuous Maximal Flows

Convex, continuous, constrained optimization problems.

... can also include region-based terms, appearance information, orientation-dependency

Advantages:- Convex -> Globally optimal- No metrication artifacts- Straightforward parallel

implementations

convex

non-convex

6

Continuous Maximal Flow [Appleton]

Continuous Maximal FlowGraph Cuts and Continuous Maximal Flows

Continuous version of maximal flow [Appleton].Results in a PDE and can be solved as such.

Energy to be minimized:

More general form (can easily include local classifiers):

7

Segmentation with Continuous Max-Flow

Continuous Maximal FlowGraph Cuts and Continuous Maximal Flows

Images: Unger

Thresholding

Continuous max-flowwith seeds.

8

Continuous Maximal Flow

Continuous Maximal FlowGraph Cuts and Continuous Maximal Flows

Energy to be minimized:

Introduce the auxiliary variable p, which we maximize for:

Now solve this by gradient descent.

9

Continuous Maximal Flow

Continuous Maximal FlowGraph Cuts and Continuous Maximal Flows

The variation is

The gradient descent scheme becomes

10

Chan-Vese (=Otsu-Thresholding w/ spatial regularity)

Continuous Max-FlowGraph Cuts and Continuous Maximal Flows

Iterative solution method is related to solving a wave equation.

11

Ex.: Segmentation with Continuous Max-Flow

Continuous Max-FlowGraph Cuts and Continuous Maximal Flows

12

3D Example

Continuous Max-FlowGraph Cuts and Continuous Maximal Flows

13

Segmentation by Graph-Cut

Graph-CutGraph Cuts and Continuous Maximal Flows

Example: Binary Segmentation

Images from ECCV Tutorial, Kumar/Kohli

Need to partition the picture into foreground and background.

Alternative approach through graph construction (vs. PDE)

14

Segmentation by Graph-Cut

Graph-CutGraph Cuts and Continuous Maximal Flows

Graph G=(V,E)

Images from ECCV Tutorial, Kumar/Kohli

Approach: Interpret the image as a graph,where pixels are connected to its neighbors.

Goal: Cut the graph into pieces to obtain the desired image partition. Assign a label to every pixel.

15

Segmentation by Graph-Cut

Graph-CutGraph Cuts and Continuous Maximal Flows

Optimization Problem: Minimize

Looking at it on a pixel-by-pixel basis (where f is the labeling):

Problem: We cannot try all possible pairings for the labels f. Need an efficient algorithm to solve this problem.

16

Maximum Flow and Minimum Cut

Graph-CutGraph Cuts and Continuous Maximal Flows

Solution: 1) Transform problem to maximizing nework flow 2) Use algorithms for network flows on images.

Preview: Graph structure for binary labeling

Pixels are connected to neighbors (pairwise interaction cost) and to source and target vertices (data cost).

Images: Boykov

Cut separating source (s) from sink (t) gives the segmentation.

17

Max-Flow/Min-Cut Theorem

Graph-CutGraph Cuts and Continuous Maximal Flows

For any network having a single origin and a single destination node, the maximum possible flow from origin to destination equals the minimum cut value for all the cuts in the network.

18

Example: Ford & Fulkerson

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

Pixel weights: s to node flow: high if near source (bkg) class, low if notPixel weights: Node to t flow: high if near target class, low if notNode to neighbor flow: high in both directions if near intensities

19

Example: Ford & Fulkerson

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

One way to compute maximal flow: 1) Pick any viable path (no zero flow)2) Subtract minimum flow from each segment on path3) Add minimum flow on reverse path

20

Example: Ford & Fulkerson

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

1) Pick path (no zero flow)

21

Example: Ford & Fulkerson

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

2) Subtract minimum flow from each segment on path3) Add minimum flow on reverse path

22

Example: Ford & Fulkerson

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

23

Example: Ford & Fulkerson

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

1) Pick path (no zero flow)

24

Example: Ford & Fulkerson

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

2) Subtract minimum flow from each segment on path3) Add minimum flow on reverse path

25

Example: Ford & Fulkerson

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

26

Example: Ford & Fulkerson

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

1) Pick path (no zero flow)

27

Example: Ford & Fulkerson

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

2) Subtract minimum flow from each segment on path3) Add minimum flow on reverse path

28

Example: Ford & Fulkerson

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

29

Example: Ford & Fulkerson

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

1) Pick path (no zero flow)

30

Example: Ford & Fulkerson

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

2) Subtract minimum flow from each segment on path3) Add minimum flow on reverse path

31

Example: Ford & Fulkerson

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

32

Example: Ford & Fulkerson

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

- Cut at zeros. - Cost of min-cut is 4. - Divides the nodes (pixels) into

two groups.- Sum of original values of zeros equals the maximum flow.

33

Example: Ford & Fulkerson

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

Look at the difference between initial and final capacities.Ignore negative capacities.This is where everything flows. Flow is conserved at nodes.

34

Why update the backward flow?

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

Image: John Chinneck

Can undo flows. Final flow does not include the central edge.

35

Image Segmentation

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

Allows (amongst many things) to compute binary segmentations.

Image: Boykov

Original Noisy Reconstructed

36

Multi-Label Case

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

Image: Boykov

37

Multi-Label Case

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

Image: Boykov

38

Influence of Neighborhood Choice

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

Minimum cost cut (standard 4-neighborhoods)

Minimum length geodesic contour (image-based Riemannian metric)

Images: Boykov

Can choose different weighted neighborhood to reduce metrication errors. (Or use continuous maximal flow.)

39

Interactive Segmentations

Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

Images: Boykov