Minimum Spanning Trees

Prim’s Algorithm

Kruskal’s Algorithm

Wed, July 9th

Outline For Today

1. Graph Theory Part 1: Trees & Properties of Trees

2. Minimum Spanning Trees

3. Prim’s Algorithm And Run-time Analysis

4. Graph Theory Part 2: Cuts & Properties of Cuts

5. Prim’s Algorithm’s Correctness & “The Cut Property”

6. Kruskal’s Algorithm

Outline For Today

1. Graph Theory Part 1: Trees & Properties of Trees

2. Minimum Spanning Trees

3. Prim’s Algorithm And Run-time Analysis

4. Graph Theory Part 2: Cuts & Properties of Cuts

5. Prim’s Algorithm’s Correctness & “The Cut Property”

6. Kruskal’s Algorithm

Outline For Today

1. Graph Theory Part 1: Trees & Properties of Trees

2. Minimum Spanning Trees

3. Prim’s Algorithm And Run-time Analysis

4. Graph Theory Part 2: Cuts & Properties of Cuts

5. Prim’s Algorithm’s Correctness & “The Cut Property”

6. Kruskal’s Algorithm

Tree

Definition: An undirected graph G(V, E) is a tree iff

1. G is connected

2. G is acyclic

Example: Tree (1)

Example: Not a Tree (1)

Example: Not a Tree (2)

Example: Tree (2)

Example: Not a Tree (3)

Example: Tree (3)

Example: Not a Tree (3)

Breaking a Tree Lemma

Removing any edge (u, v) from a tree

T disconnects T!

Why?No u v path can exist! Assume it did…⤳

u x1 x2 v

Contradicts acyclicity of T!

Reverse is Also True

Let G be a connected graph and assume

removing any edge from G disconnects it.

Then G is acyclic and hence a tree.

Why?

x1

xi xi+1

xkClaim: G cannot

contain a cycle

C

Proof That G Cannot Contain a Cycle C

x1

xi xi+1xk

Breaking a Cycle Lemma: Removing any

edge from a cycle cannot disconnect a

connected graph!

Take (xi xi+1), and any path P that was

using it S txi xi+1

Proof That G Cannot Contain a Cycle C

x1

xi xi+1xk

Breaking a Cycle Lemma: Removing any

edge from a cycle cannot disconnect a

connected graph!

Take (xi xi+1), and any path P that was

using it S txi xi+1

Proof That G Cannot Contain a Cycle C

x1

xi xi+1xk

Breaking a Cycle Lemma: Removing any

edge from a cycle cannot disconnect a

connected graph!

Take (xi xi+1), and any path P that was

using it S txi xi+1

xi-1

x1

xks and t are

still

connected!

Cycle Creation Lemma

Adding any edge (u, v) to a tree T

creates a cycle!

Cycle Creation Lemma

Adding any edge (u, v) to a tree T

creates a cycle!

Cycle Creation Lemma

Adding any edge (u, v) to a tree T

creates a cycle!

Proof: B/c T is connected, ∃path P from u to

v.

u v

adding (u, v) closes the cycle.

Theorems

Every tree of n vertices contains n-1

edges!

Every n-1 acyclic set of edges is a tree

=> i.e., they connect V!

Outline For Today

1. Graph Theory Part 1: Trees & Properties of Trees

2. Minimum Spanning Trees

3. Prim’s Algorithm And Run-time Analysis

4. Graph Theory Part 2: Cuts & Properties of Cuts

5. Prim’s Algorithm’s Correctness & “The Cut Property”

6. Kruskal’s Algorithm

Minimum Spanning Tree

Input: undirected G(V, E) and arbitrary weights on the

edges

Output: A tree T* of V such that w(T*) ≤ any other T of

V

w(T) = sum of the weights of all n-1 edges in T

“spanning” tree of G(V, E) means T* has to connect

all of V

Assumptions:

1. G is connected (minor)

2. Edge weights are distinct (all of our algorithms

work w/o this assumption)

Spanning Tree Example

A

D

B

C

1

2

4

2.5

3

5

Spanning Tree Example

A

D

B

C

1

2

4

2.5

3

5

Spanning Tree Example

A

D

B

C

1 4

2.5

W((A,C), (C,B), (B,D)) = 1 + 2.5 + 4 = 7.5

Spanning Tree Example

A

D

B

C

1

2

4

2.5

3

5

Spanning Tree Example

A

D

B

C

1

2

4

W((A,C), (C,D), (B,D)) = 1 + 2 + 4 = 7

MST Applications

Designing all kinds of networks:

datacenter networks

road networks

phone networks

Circuit Design

Clustering

Image Segmentation

…

Outline For Today

1. Graph Theory Part 1: Trees & Properties of Trees

2. Minimum Spanning Trees

3. Prim’s Algorithm And Run-time Analysis

4. Graph Theory Part 2: Cuts & Properties of Cuts

5. Prim’s Algorithm’s Correctness & “The Cut Property”

6. Kruskal’s Algorithm

Prim’s Algorithm Simulation (1)

?

Prim’s Algorithm Simulation (1)A

B C

1

46

D

Prim’s Algorithm Simulation (1)

B C

1

46

2

5

A D E

Prim’s Algorithm Simulation (1)

B C

1

46

2

5

A D E

F

3

Prim’s Algorithm Simulation (1)

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

Prim’s Algorithm Simulation (1)

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

Prim’s Algorithm Simulation (1)

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

Prim’s Algorithm Simulation (1)

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

9

Prim’s Algorithm Simulation (1)

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

9

Prim’s Algorithm Simulation (1)

B C

1

4

2A D E

F

3

G

2.5

7.5

H

7

Final Tree!

Prim’s Algorithm

procedure prim(G(V, E)): choose any s∈V let L={s}, R=V-{s} T = {} // final tree edges

for i = 1 to n: 1. let (u, v) be the min cost edge crossing L-R 2. remove v from R and add to L

3. add (u, v) to T return T

Prim’s Algorithm Simulation (2)

?

Prim’s Algorithm Simulation (2)

∞

∞∞

A

B C

1

46

D

Prim’s Algorithm Simulation (2)

1

64

A

B C

1

46

D

Prim’s Algorithm Simulation (2)

64

∞

B C

1

46

2

5

A D E

Prim’s Algorithm Simulation (2)

54

2

B C

1

46

2

5

A D E

Prim’s Algorithm Simulation (2)

54

B C

1

46

2

5

A D E

F∞

3

Prim’s Algorithm Simulation (2)

54

B C

1

46

2

5

A D E

F3

3

Prim’s Algorithm Simulation (2)

54

B C

1

46

2

5

A D E

F

3

∞ G

2.5

7.5

Prim’s Algorithm Simulation (2)

2.54

B C

1

46

2

5

A D E

F

3

7.5 G

2.5

7.5

Prim’s Algorithm Simulation (2)

4

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

∞H

8

7

7.5

Prim’s Algorithm Simulation (2)

4

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

7H

8

7

7.5

Prim’s Algorithm Simulation (2)

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

7H

8

7

7.5

Prim’s Algorithm Simulation (2)

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

9 7.5

Prim’s Algorithm Simulation (2)

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

9

Prim’s Algorithm Simulation (2)

B C

1

4

2A D E

F

3

G

2.5

7.5

H

7

Final Tree!

Prim’s Algorithm procedure prim(G(V, E)): choose any s∈V let L={s}, R=V-{s} let mWE store for each vertex in R, its min weight edge to L. T = {} // final tree edges

for i = 1 to n: 1. let v be min mWE vertex let (u, v) be the edge minimizing v’s mWE 2. remove v from R & mWE;add to L

3. add (u, v) to T 4. update mWE for each neighbor of v in Rreturn T

Prim’s Algorithm Running Time

Exactly the same As Dijkstra’s

Algorithm! O((n+m)log(n)) = O(mlog(n)) when using a

priority queue backed with a heap to store “mWE”

values

When a vertex y is moved from R to L, For each (x,

y):

Dijkstra Update: val[y]=min{val[y],

shortDist[x] +

w((x, y))}

Prim Update: val[y]=min{val[y], w((x, y))}

Outline For Today

1. Graph Theory Part 1: Trees & Properties of Trees

2. Minimum Spanning Trees

3. Prim’s Algorithm And Run-time Analysis

4. Graph Theory Part 2: Cuts & Properties of Cuts

5. Prim’s Algorithm’s Correctness & “The Cut Property”

6. Kruskal’s Algorithm

Cuts

A cut of G(V, E) is a slice of V into 2 non-empty

sets

A

D

B

C

1

2

4

2.5

3

5

Cuts

A cut of G(V, E) is a slice of V into 2 non-empty

setsA

D

B

C

1

2

4

2.5

3

5

Cuts

A cut of G(V, E) is a slice of V into 2 non-empty

setsA

D

B

C

1

2

4

2.5

3

5

Empty Cut Lemma

A graph G(V,

E) is not

connected

⇔∃ cut (X, Y)

with no

crossing edges

[prove as

exercise]

Double Cut Crossing Lemma of Cycles

Suppose a cycle C⊆ E has an edge e

crossing a cut (X, Y)

Claim: Then there is another e` ≠ e of

C that also crosses (X, Y)

Double Cut Crossing Lemma of Cycles

u ve

u`

v`

e`

X Y

Lonely Cut Corollary

Suppose there is a cut (X, Y) which

has only one edge e crossing it

Claim: e cannot be part of any cycle

C!

Lonely Cut Corollary

u ve

Claim: e cannot

be part of any

cycle C!

X Y

Outline For Today

1. Graph Theory Part 1: Trees & Properties of Trees

2. Minimum Spanning Trees

3. Prim’s Algorithm And Run-time Analysis

4. Graph Theory Part 2: Cuts & Properties of Cuts

5. Prim’s Algorithm’s Correctness & “The Cut Property”

6. Kruskal’s Algorithm

For Correctness We Need To Prove 2 Things1. Outputs a Spanning Tree Tprim

2. Tprim is a minimum spanning tree

1: Prim Outputs a Spanning Tree

Proof: At each iteration i, Tprim,i spans Li AND is

acyclic

By induction: Holds for base case s

Ind. Hyp. (IH).: Tprim,k spans L after kth iteration

Prim picks the min edge (u, v) from Lk to Rk.

By IH: u has a path to every vertex in Lk and vice

versa

=> with addition of (u, v), v has a path to

every

vertex in Lk and vice versa => Tprim,k+1 is

spanning

Tprim,k+1 is acyclic b/c (u, v) is the first edge from Lk

to Rk (by the lonely cut corollary)

Q.E.D

Plan for 2: Tprim is a Minimum Spanning Tree1. “Cut Property” of Edges

2. Argue that the Cut Property Implies Tprim is an MST

Cut Property of Edges

Consider any cut (X, Y) of G, and

suppose e is the minimum edge

crossing (X, Y)

Claim: e belongs to every MST

Cut Property of Edges

1

8

3

4

5

X Y

Cut Property of Edges

u v1

8

3

4

5

(u,v) belongs to

every MST!X Y

Cut Property => Tprim is a MST

?

Cut Property => Tprim is a MSTA

B C

1

46

D

Cut Property => Tprim is a MST

B C

1

46

2

5

A D E

Cut Property => Tprim is a MST

B C

1

46

2

5

A D E

F

3

Cut Property => Tprim is a MST

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

Cut Property => Tprim is a MST

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

Cut Property => Tprim is a MST

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

Cut Property => Tprim is a MST

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

9

Cut Property => Tprim is a MST

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

9

Cut Property => Tprim is a MST

Prim looks at n-1 different cuts.

Add n-1 edges that every MST

includes.

Since any spanning tree include n-1

edges

=> Tprim is “the” MST.

(with distinct edge weights this proof

also proves the uniqueness of MST)

Proof of Cut Property (Exchange Argument)

u ve=min

X Y

internal edges and vertices of X

internal edges and vertices of Y

Proof of Cut Property (Exchange Argument)

u ve=min

X YLet T be any MST that

doesn’t contain e

internal edges and vertices of X

internal edges and vertices of Y

Proof of Cut Property (Exchange Argument)

u v

X YLet T be any MST that

doesn’t contain e

internal edges of T in Y

internal edges of T in X

Proof of Cut Property (Exchange Argument)

u v

X Yadd e to T

internal edges of T in Y

internal edges of T in X

By cycle creation lemma we created a cycle.

Proof of Cut Property (Exchange Argument)

u

a

b

v

X Yadd e to T

internal edges of T in Y

internal edges of T in X

By Double Crossing Lemma ∃(a, b) that also crosses (X, Y)

Proof of Cut Property (Exchange Argument)

u

a

b

v

X Y

internal edges of T in Y

internal edges of T in X

Claim: Can safely remove (a, b) and still get a spanning tree. Why?

Proof of Cut Property (Exchange Argument)

u

a

b

v

X Y

internal edges of T in Y

internal edges of T in X

By Breaking a Cycle Lemma we can’t disconnect T ∪ (u, v)!

Proof of Cut Property (Exchange Argument)

u

a

b

v

X YT` = T + (u, v) – (a,

b)

internal edges of T in Y

internal edges of T in X

T` is still a spanning

tree and w(T`) < w(T)

Proof of Cut Property (Exchange Argument)Started with (X, Y) cut, and e=(u, v): min edge crossing

(X, Y)

Took any minimum spanning tree T that did not include

e.

Added e to T => created a cycle C (by cycle-creation-

lemma)

By DCL, there is another edge e` that crosses (X, Y)

Removed e` => T`

By Breaking A Cycle Lemma: T` is still connected

Therefore w(T`) < w(T) => T was not an MST.

=> Every MST has to include e!Q.E.D.

Summary of Prim’s Algorithm’s CorrectnessFirst proved Tprim is a spanning tree

By induction on Li

Second proved Tprim is a an minimum spanning tree

Note this version of Prim’s Correctness Proof contains:

• Exchange Argument: Cut Property

• Greedy Stays Ahead Argument: Prim adds n-1 edges

and stay ahead at each addition.

Outline For Today

1. Graph Theory Part 1: Trees & Properties of Trees

2. Minimum Spanning Trees

3. Prim’s Algorithm And Run-time Analysis

4. Graph Theory Part 2: Cuts & Properties of Cuts

5. Prim’s Algorithm’s Correctness & “The Cut Property”

6. Kruskal’s Algorithm

Kruskal’s Algorithm Simulation

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

9

1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9

Kruskal’s Algorithm Simulation

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

9

1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9

Kruskal’s Algorithm Simulation

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

9

1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9

Kruskal’s Algorithm Simulation

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

9

1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9

Kruskal’s Algorithm Simulation

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

9

1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9

Kruskal’s Algorithm Simulation

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

9

1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9

Creates a cycle

Kruskal’s Algorithm Simulation

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

9

1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9

Creates a cycle

Kruskal’s Algorithm Simulation

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

9

1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9

Kruskal’s Algorithm Simulation

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

9

1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9

Kruskal’s Algorithm Simulation

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

9

1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9

Creates a cycle

Kruskal’s Algorithm Simulation

B C

1

46

2

5

A D E

F

3

G

2.5

7.5

H

8

7

9

1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9

Creates a cycle

Kruskal’s Algorithm Simulation

B C

1

4

2A D E

F

3

G

2.5

7.5

H

7

Final Tree!

Same as Tprim

Kruskal’s Algorithm Pseudocode

procedure kruskal(G(V, E)): sort E in order of increasing

weights rename E so w(e1) < w(e2) < … < w(em) T = {} // final tree edges for i = 1 to m: if T ∪ ei=(u,v) doesn’t create cycle add ei to T return T

For Correctness We Need To Prove 2 Things1. Outputs a Spanning Tree Tkrsk

2. Tkrsk is a minimum spanning tree

1: Kruskal Outputs a Spanning Tree (1)

Need to prove Tkrsk is spanning AND is acyclic

Acyclic is by definition of the algorithm.

Why is Tkrsk spanning (i.e., connected)?

Recall Empty Cut Lemma:

A graph is not connected iff ∃ cut (X, Y) with no

crossing edges

If all cuts have a crossing edge -> graph is

connected!

1: Kruskal Outputs a Spanning Tree (2)Consider any cut (X, Y)

Since input G is connected, ∃ edges crossing (X, Y)

Let e* be the min-weight edge

Kruskal inspects all edges

Consider the time when Kruskal inspects e*

Claim: No edge crossing (X, Y) is in Tkrsk

And adding e* to Tkrsk cannot create a cycle =>

Why?

(because of lonely cut corallary)

Therefore Kruskal will add e*, so for each cut, there

is an edge in Tkrsk crossing it => Tkrsk spans V.

Plan for 2: Tkrsk is a Minimum Spanning Tree1. Argue that the Cut Property Implies Tkrsk is an

MST

In particular we will argue that each edge

(u, v) added to Tkrsk is a min-edge in a cut (X,

Y) in G.

Cut Property => Tkrsk is a MST (1)Let (u, v) be any edge added by Kruskal’s Algorithm.

u and v are in different comp. (b/c Kruskal checks for

cycles)

ux

y

v

t

zw

Claim: (u, v) is min-edge crossing this cut!

Cut Property => Tkrsk is a MST (2)

(u, v) is the min edge crossing the cut b/c

Kruskal looks at edges in increasing weights.

By the Cut Property => (u, v) is in every MST.

Therefore all edges added to Tkrsk is in every

MST

=> Tkrsk is “the” MST.

Q.E.D.

Summary of Kruskal’s Correctness

First proved Tkrsk is a spanning tree

Acyclicity was by definition

Connectedness followed from showing that there is

an edge of Tkrsk crossing any cut (X, Y) of G, which

by the empty cut lemma implied that Tkrsk was

connected

Second proved Tkrsk was a minimum spanning tree by

arguing that any edge (u, v) if Tkrsk is a min-edge in

some cut (X, Y)

Note this version of Kruskal’s Correctness Proof

contains:

• Exchange Argument: Cut Property

• Greedy Stays Ahead Argument: Krusk adds n-1

edges and all are justified.

Kruskal’s Algorithm Pseudocode

procedure kruskal(G(V, E)): sort E in order of increasing

weights rename E so w(e1) < w(e2) < … < w(em) T = {} // final tree edges for i = 1 to m: if T ∪ ei=(u,v) doesn’t create cycle add ei to T return T

Kruskal’s Algorithm Naïve Run-time

O(mlog(n)) to sort the edges

m iterations

O(n) time to check for a cycle (BFS/DFS)

Total: O(mn) total time

On Monday will improve to O(mlog(n))