Random Walks

for Image Segmentation

Group Learning Presentation

Xiaowei Zhou, 2010-12-10

1

Outline

• Problem Description and Basic Idea

• Solution of Random Walk Probability

• Properties and Results

• Relation to Other Methods

• Discussion

2

Problem Description• Interactive Segmentation

• Requirements:▫ fewer interactions

▫ fast computation, fast editing

Partially labeled image Segmented image

3

Previous Methods

• Interactive segmentation

▫ Intelligent Scissors (live wire)

fail in the presence of noise and weak boundary

too much interaction needed for high accuracy

▫ Active Contour

local minimum solution

inconvenient for editing

▫ Graph Cuts

small cut bias

no global optimal solution for multi-way cut

4

Basic Idea

A pixel p belongs to

which class?

A random walker starts

from p, which seeds he

will arrive first ?

Green Red Yellow Blue

The pixel intensity means: starting at this pixel the probability that a random walker will arrive the

target seed before arriving other three seeds

5

Why formulate in this way?

1. Fast solution

▫ solving a sparse linear system

▫ ability of multiple segmentation

▫ fast editing

2. Nice properties

▫ location of weak/missing boundaries

▫ noise robustness

▫ avoidance of trivial solutions

6

Algorithm Overview

1. Allocate region seeds si for each region i

2. Calculate ui(x,y):

the probability of first arriving si for a

random walker starting from (x,y)

3. Assign (x,y) to Label k

if uk(x,y) is the largest among ui(x,y)

for i = 1…N

7

Outline

• Problem Description and Basic Idea

• Solution of Random Walk Probability

• Properties and Results

• Relation to Other Methods

• Discussion

8

Definition of Image Graph

2 4 6 8 10 12 14 16

2

4

6

8

10

12

200 400 600 800 1000 1200

100

200

300

400

500

600

700

800

900

Graph nodes: pixels

Edge weights: similarity between neighboring pixels

Figures are borrowed from

http://www.cs.yale.edu/homes/spielman/sgta/

9

2

2

( )i jI I

ijw e

Graph Terminology

• Adjacency Matrix

• Degree Matrix

• Laplacian Matrix

[ ] ij n nW w

1

D di ij

j

n n n

d

w

d

L D W

10

2

2

( )

0

i jI Iij

i j

w

e i j

i

ij

ij

d i jl

w i j

Random walk probability

Let pij be the the

probability of walking

from node i to node j:

Let xi be the probability,

starting at node i, of a

random walker reaching

the red node before

reaching the blue nodes

( , )

ij ij

ij

ik i

i k E

w wp

w d

11

Random walk probability

• How to compute X = [x1 x2 … xn ]T?

• xi has following property

▫ xred = 1 for the red node

▫ xblue = 0 for the blue nodes

▫ for other nodesi ij j

j

x p x

12

• If we define the transition matrix to be

• except for labeled nodes.

• Reorder the node vector X to make labeled nodes in

front of unlabeled :

1

( , )

ij

ij n nik

i k E n n

wP p D W

w

0

0

L L L

T

U U U

X D W BX D W

X D B W

13

PX X

1 1

1 1

L L L

T

U U U

D W D BP

D B D W

• The properties can be described by:

• XL are known and we need to calculate XU

• Finally we can compute XU by solving:

1 1

0L L

T

U U U U U

X I X

X D B D W X

1 1

1 1( )

T

U U L U U

T

U U U U L

X D B X D WX

I D W X D B X

( ) T

U U U LD W X B X

14

• is positive definite for fully connected graph,

thus the linear system is nonsingular.

• Proof:

15

U UD W

2 2

,

( )

( )

( )

( ) ( )

U

U U

T

U U U U

T s s

U U U U

T T s s

U U U U U

ij i ij i j

i U j L i j U

x D W x

x D D W x

x D x x D W x

w x w x x

U ij

j L i U

D diag w

U

s

ij

j U i U

D diag w

Degree Matrix

for the subgraph

containing

unlabeled nodes

s

U UW W

Adjacency Matrix

for the subgraph

containing

unlabeled nodes

Minimization Point of View

2

( , )

1 1( )

2 2

. . 1, 0

T

ij i j

i j E

red blue

X LX w x x

s t x x

• It can be shown that the random walk

solution also minimizes following energy:

16

Continuous case

• In continuous case, it is called the Dirichlet integral

• The Eular-Lagrange equation of the above functional

is the Laplace equation. Thus the solution of random

walk probability in the continuous case is a harmonic

function with predefined boundary conditions

21( ) | |

2D u u d

2 0 ( ) 1 ( ) 0red blueu u s u s

17

Outline

• Problem Description and Basic Idea

• Solution of Random Walk Probability

• Properties and Results

• Relation to Other Methods

• Discussion

18

Locate weak boundary

19

Noise Robustness

20

The secretes of nice properties

• Property of Harmonic function

▫ Maximum Principle:

the maximum and minimum are only attained

on the boundary.

▫ Mean value property:

the function value is an average(or weighted

average) of neighboring values.

21

Outline

• Problem Description and Basic Idea

• Solution of Random Walk Probability

• Properties and Results

• Relation to Other Methods

• Discussion

22

Graph Cut / Min-Cut

• The energy for Graph Cut segmentation is:

• It can be proved that the constraint is slack,

which means the Min-cut problem is a linear

programming problem which minimize the energy

with L1 penalty.

( , )

| |

. . {0,1}, 1, 0

ij i j

i j E

red blue

w x x

s t x x x

{0,1}x

23

Normalized Cut (I)

• Revisit the Normalized Cut

• Relax y to be real numbers, the problem becomes:

( , ) ( , )( , )

deg( ) deg( )

( )... with {1, }, 1 0

TT

iT

cut A B cut A BNcut A B

A B

y D W yy b y D

y Dy

1

( )min

T

y D T

y D W y

y Dy

24

Normalized Cut (II)

• Let , finally we want to solve

• is called normalized Laplace matrix.

• The energy minimized in the Normalized Cut is

similar to the random walk energy. They both have

quadratic form of a Laplace matrix, while the

constraints are different.

1

2z D y

1/2

1/2

* 1/2 1/2

1

1/2 1/2

1

arg min ( )

arg min

T

z D

T

z D

z z D D W D z

z D LD z

1/2 1/2D LD

25

Random Walk Graph Cut Normalized Cut

Objective

function

Constraints

Solution. (k=2) Linear system Maximal FlowEigenvector of the

normalized Laplace

Solution. (k>2)Linear system Alpha-expansion

(approximated)Spectral clustering

Uniqueness Unique Not unique Unique

Result biasShift related to

seed placementSmall-cut bias

No bias

Comparison

( , )

| |ij i j

i j E

w x x

TX LX1/2 1/2Tz D LD z

1, 0red bluex x 1/2

1z D1, 0red bluex x

26

Outline

• Problem Description and Basic Idea

• Solution of Random Walk Probability

• Properties and Results

• Relation to Other Methods

• Discussion

27

Conclusions

• The advantages of random walker

segmentation are mainly due to

1. It describe the segmentation problem with a

simple physical process, whose solution has

well-studied probabilistic interpretation and

nice mathematical properties.

2. The discrete calculus formulation of the

problem can be solved analytically and

efficiently.

28

Why Use Graph?

• Capture the discrete nature of digital images

naturally model the spatial coherence of pixels

can be easily extended to high dimension

• Continuous differential operators can be replaced by

combinatorial operators, which can be directly

implemented.

• The linear algebra representation of the energy often

has analytical solution.

29

Discrete calculus

30

Key References

[1] L. Grady, "Random walks for image segmentation," PAMI, pp.

1768-1783, 2006.

[2] A. K. Sinop and L. Grady, "A seeded image segmentation

framework unifying graph cuts and random walker which yields a

new algorithm," ICCV 2007, pp. 1-8.

[3] J. Shi and J. Malik, "Normalized cuts and image

segmentation," PAMI, vol. 22, pp. 888-905, 2002.

[4] U. Von Luxburg, "A tutorial on spectral clustering," Statistics and

Computing, vol. 17, pp. 395-416, 2007.

31