Minimum Spanning Trees
Prim’s Algorithm
Kruskal’s Algorithm
Wed, July 9th
Outline For Today
1. Graph Theory Part 1: Trees & Properties of Trees
2. Minimum Spanning Trees
3. Prim’s Algorithm And Run-time Analysis
4. Graph Theory Part 2: Cuts & Properties of Cuts
5. Prim’s Algorithm’s Correctness & “The Cut Property”
6. Kruskal’s Algorithm
Outline For Today
1. Graph Theory Part 1: Trees & Properties of Trees
2. Minimum Spanning Trees
3. Prim’s Algorithm And Run-time Analysis
4. Graph Theory Part 2: Cuts & Properties of Cuts
5. Prim’s Algorithm’s Correctness & “The Cut Property”
6. Kruskal’s Algorithm
Outline For Today
1. Graph Theory Part 1: Trees & Properties of Trees
2. Minimum Spanning Trees
3. Prim’s Algorithm And Run-time Analysis
4. Graph Theory Part 2: Cuts & Properties of Cuts
5. Prim’s Algorithm’s Correctness & “The Cut Property”
6. Kruskal’s Algorithm
Tree
Definition: An undirected graph G(V, E) is a tree iff
1. G is connected
2. G is acyclic
Example: Tree (1)
Example: Not a Tree (1)
Example: Not a Tree (2)
Example: Tree (2)
Example: Not a Tree (3)
Example: Tree (3)
Example: Not a Tree (3)
Breaking a Tree Lemma
Removing any edge (u, v) from a tree
T disconnects T!
Why?No u v path can exist! Assume it did…⤳
u x1 x2 v
Contradicts acyclicity of T!
Reverse is Also True
Let G be a connected graph and assume
removing any edge from G disconnects it.
Then G is acyclic and hence a tree.
Why?
x1
xi xi+1
xkClaim: G cannot
contain a cycle
C
Proof That G Cannot Contain a Cycle C
x1
xi xi+1xk
Breaking a Cycle Lemma: Removing any
edge from a cycle cannot disconnect a
connected graph!
Take (xi xi+1), and any path P that was
using it S txi xi+1
Proof That G Cannot Contain a Cycle C
x1
xi xi+1xk
Breaking a Cycle Lemma: Removing any
edge from a cycle cannot disconnect a
connected graph!
Take (xi xi+1), and any path P that was
using it S txi xi+1
Proof That G Cannot Contain a Cycle C
x1
xi xi+1xk
Breaking a Cycle Lemma: Removing any
edge from a cycle cannot disconnect a
connected graph!
Take (xi xi+1), and any path P that was
using it S txi xi+1
xi-1
x1
xks and t are
still
connected!
Cycle Creation Lemma
Adding any edge (u, v) to a tree T
creates a cycle!
Cycle Creation Lemma
Adding any edge (u, v) to a tree T
creates a cycle!
Cycle Creation Lemma
Adding any edge (u, v) to a tree T
creates a cycle!
Proof: B/c T is connected, ∃path P from u to
v.
u v
adding (u, v) closes the cycle.
Theorems
Every tree of n vertices contains n-1
edges!
Every n-1 acyclic set of edges is a tree
=> i.e., they connect V!
Outline For Today
1. Graph Theory Part 1: Trees & Properties of Trees
2. Minimum Spanning Trees
3. Prim’s Algorithm And Run-time Analysis
4. Graph Theory Part 2: Cuts & Properties of Cuts
5. Prim’s Algorithm’s Correctness & “The Cut Property”
6. Kruskal’s Algorithm
Minimum Spanning Tree
Input: undirected G(V, E) and arbitrary weights on the
edges
Output: A tree T* of V such that w(T*) ≤ any other T of
V
w(T) = sum of the weights of all n-1 edges in T
“spanning” tree of G(V, E) means T* has to connect
all of V
Assumptions:
1. G is connected (minor)
2. Edge weights are distinct (all of our algorithms
work w/o this assumption)
Spanning Tree Example
A
D
B
C
1
2
4
2.5
3
5
Spanning Tree Example
A
D
B
C
1
2
4
2.5
3
5
Spanning Tree Example
A
D
B
C
1 4
2.5
W((A,C), (C,B), (B,D)) = 1 + 2.5 + 4 = 7.5
Spanning Tree Example
A
D
B
C
1
2
4
2.5
3
5
Spanning Tree Example
A
D
B
C
1
2
4
W((A,C), (C,D), (B,D)) = 1 + 2 + 4 = 7
MST Applications
Designing all kinds of networks:
datacenter networks
road networks
phone networks
Circuit Design
Clustering
Image Segmentation
…
Outline For Today
1. Graph Theory Part 1: Trees & Properties of Trees
2. Minimum Spanning Trees
3. Prim’s Algorithm And Run-time Analysis
4. Graph Theory Part 2: Cuts & Properties of Cuts
5. Prim’s Algorithm’s Correctness & “The Cut Property”
6. Kruskal’s Algorithm
Prim’s Algorithm Simulation (1)
?
Prim’s Algorithm Simulation (1)A
B C
1
46
D
Prim’s Algorithm Simulation (1)
B C
1
46
2
5
A D E
Prim’s Algorithm Simulation (1)
B C
1
46
2
5
A D E
F
3
Prim’s Algorithm Simulation (1)
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
Prim’s Algorithm Simulation (1)
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
Prim’s Algorithm Simulation (1)
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
Prim’s Algorithm Simulation (1)
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
9
Prim’s Algorithm Simulation (1)
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
9
Prim’s Algorithm Simulation (1)
B C
1
4
2A D E
F
3
G
2.5
7.5
H
7
Final Tree!
Prim’s Algorithm
procedure prim(G(V, E)): choose any s∈V let L={s}, R=V-{s} T = {} // final tree edges
for i = 1 to n: 1. let (u, v) be the min cost edge crossing L-R 2. remove v from R and add to L
3. add (u, v) to T return T
Prim’s Algorithm Simulation (2)
?
Prim’s Algorithm Simulation (2)
∞
∞∞
A
B C
1
46
D
Prim’s Algorithm Simulation (2)
1
64
A
B C
1
46
D
Prim’s Algorithm Simulation (2)
64
∞
B C
1
46
2
5
A D E
Prim’s Algorithm Simulation (2)
54
2
B C
1
46
2
5
A D E
Prim’s Algorithm Simulation (2)
54
B C
1
46
2
5
A D E
F∞
3
Prim’s Algorithm Simulation (2)
54
B C
1
46
2
5
A D E
F3
3
Prim’s Algorithm Simulation (2)
54
B C
1
46
2
5
A D E
F
3
∞ G
2.5
7.5
Prim’s Algorithm Simulation (2)
2.54
B C
1
46
2
5
A D E
F
3
7.5 G
2.5
7.5
Prim’s Algorithm Simulation (2)
4
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
∞H
8
7
7.5
Prim’s Algorithm Simulation (2)
4
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
7H
8
7
7.5
Prim’s Algorithm Simulation (2)
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
7H
8
7
7.5
Prim’s Algorithm Simulation (2)
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
9 7.5
Prim’s Algorithm Simulation (2)
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
9
Prim’s Algorithm Simulation (2)
B C
1
4
2A D E
F
3
G
2.5
7.5
H
7
Final Tree!
Prim’s Algorithm procedure prim(G(V, E)): choose any s∈V let L={s}, R=V-{s} let mWE store for each vertex in R, its min weight edge to L. T = {} // final tree edges
for i = 1 to n: 1. let v be min mWE vertex let (u, v) be the edge minimizing v’s mWE 2. remove v from R & mWE;add to L
3. add (u, v) to T 4. update mWE for each neighbor of v in Rreturn T
Prim’s Algorithm Running Time
Exactly the same As Dijkstra’s
Algorithm! O((n+m)log(n)) = O(mlog(n)) when using a
priority queue backed with a heap to store “mWE”
values
When a vertex y is moved from R to L, For each (x,
y):
Dijkstra Update: val[y]=min{val[y],
shortDist[x] +
w((x, y))}
Prim Update: val[y]=min{val[y], w((x, y))}
Outline For Today
1. Graph Theory Part 1: Trees & Properties of Trees
2. Minimum Spanning Trees
3. Prim’s Algorithm And Run-time Analysis
4. Graph Theory Part 2: Cuts & Properties of Cuts
5. Prim’s Algorithm’s Correctness & “The Cut Property”
6. Kruskal’s Algorithm
Cuts
A cut of G(V, E) is a slice of V into 2 non-empty
sets
A
D
B
C
1
2
4
2.5
3
5
Cuts
A cut of G(V, E) is a slice of V into 2 non-empty
setsA
D
B
C
1
2
4
2.5
3
5
Cuts
A cut of G(V, E) is a slice of V into 2 non-empty
setsA
D
B
C
1
2
4
2.5
3
5
Empty Cut Lemma
A graph G(V,
E) is not
connected
⇔∃ cut (X, Y)
with no
crossing edges
[prove as
exercise]
Double Cut Crossing Lemma of Cycles
Suppose a cycle C⊆ E has an edge e
crossing a cut (X, Y)
Claim: Then there is another e` ≠ e of
C that also crosses (X, Y)
Double Cut Crossing Lemma of Cycles
u ve
u`
v`
e`
X Y
Lonely Cut Corollary
Suppose there is a cut (X, Y) which
has only one edge e crossing it
Claim: e cannot be part of any cycle
C!
Lonely Cut Corollary
u ve
Claim: e cannot
be part of any
cycle C!
X Y
Outline For Today
1. Graph Theory Part 1: Trees & Properties of Trees
2. Minimum Spanning Trees
3. Prim’s Algorithm And Run-time Analysis
4. Graph Theory Part 2: Cuts & Properties of Cuts
5. Prim’s Algorithm’s Correctness & “The Cut Property”
6. Kruskal’s Algorithm
For Correctness We Need To Prove 2 Things1. Outputs a Spanning Tree Tprim
2. Tprim is a minimum spanning tree
1: Prim Outputs a Spanning Tree
Proof: At each iteration i, Tprim,i spans Li AND is
acyclic
By induction: Holds for base case s
Ind. Hyp. (IH).: Tprim,k spans L after kth iteration
Prim picks the min edge (u, v) from Lk to Rk.
By IH: u has a path to every vertex in Lk and vice
versa
=> with addition of (u, v), v has a path to
every
vertex in Lk and vice versa => Tprim,k+1 is
spanning
Tprim,k+1 is acyclic b/c (u, v) is the first edge from Lk
to Rk (by the lonely cut corollary)
Q.E.D
Plan for 2: Tprim is a Minimum Spanning Tree1. “Cut Property” of Edges
2. Argue that the Cut Property Implies Tprim is an MST
Cut Property of Edges
Consider any cut (X, Y) of G, and
suppose e is the minimum edge
crossing (X, Y)
Claim: e belongs to every MST
Cut Property of Edges
1
8
3
4
5
X Y
Cut Property of Edges
u v1
8
3
4
5
(u,v) belongs to
every MST!X Y
Cut Property => Tprim is a MST
?
Cut Property => Tprim is a MSTA
B C
1
46
D
Cut Property => Tprim is a MST
B C
1
46
2
5
A D E
Cut Property => Tprim is a MST
B C
1
46
2
5
A D E
F
3
Cut Property => Tprim is a MST
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
Cut Property => Tprim is a MST
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
Cut Property => Tprim is a MST
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
Cut Property => Tprim is a MST
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
9
Cut Property => Tprim is a MST
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
9
Cut Property => Tprim is a MST
Prim looks at n-1 different cuts.
Add n-1 edges that every MST
includes.
Since any spanning tree include n-1
edges
=> Tprim is “the” MST.
(with distinct edge weights this proof
also proves the uniqueness of MST)
Proof of Cut Property (Exchange Argument)
u ve=min
X Y
internal edges and vertices of X
internal edges and vertices of Y
Proof of Cut Property (Exchange Argument)
u ve=min
X YLet T be any MST that
doesn’t contain e
internal edges and vertices of X
internal edges and vertices of Y
Proof of Cut Property (Exchange Argument)
u v
X YLet T be any MST that
doesn’t contain e
internal edges of T in Y
internal edges of T in X
Proof of Cut Property (Exchange Argument)
u v
X Yadd e to T
internal edges of T in Y
internal edges of T in X
By cycle creation lemma we created a cycle.
Proof of Cut Property (Exchange Argument)
u
a
b
v
X Yadd e to T
internal edges of T in Y
internal edges of T in X
By Double Crossing Lemma ∃(a, b) that also crosses (X, Y)
Proof of Cut Property (Exchange Argument)
u
a
b
v
X Y
internal edges of T in Y
internal edges of T in X
Claim: Can safely remove (a, b) and still get a spanning tree. Why?
Proof of Cut Property (Exchange Argument)
u
a
b
v
X Y
internal edges of T in Y
internal edges of T in X
By Breaking a Cycle Lemma we can’t disconnect T ∪ (u, v)!
Proof of Cut Property (Exchange Argument)
u
a
b
v
X YT` = T + (u, v) – (a,
b)
internal edges of T in Y
internal edges of T in X
T` is still a spanning
tree and w(T`) < w(T)
Proof of Cut Property (Exchange Argument)Started with (X, Y) cut, and e=(u, v): min edge crossing
(X, Y)
Took any minimum spanning tree T that did not include
e.
Added e to T => created a cycle C (by cycle-creation-
lemma)
By DCL, there is another edge e` that crosses (X, Y)
Removed e` => T`
By Breaking A Cycle Lemma: T` is still connected
Therefore w(T`) < w(T) => T was not an MST.
=> Every MST has to include e!Q.E.D.
Summary of Prim’s Algorithm’s CorrectnessFirst proved Tprim is a spanning tree
By induction on Li
Second proved Tprim is a an minimum spanning tree
Note this version of Prim’s Correctness Proof contains:
• Exchange Argument: Cut Property
• Greedy Stays Ahead Argument: Prim adds n-1 edges
and stay ahead at each addition.
Outline For Today
1. Graph Theory Part 1: Trees & Properties of Trees
2. Minimum Spanning Trees
3. Prim’s Algorithm And Run-time Analysis
4. Graph Theory Part 2: Cuts & Properties of Cuts
5. Prim’s Algorithm’s Correctness & “The Cut Property”
6. Kruskal’s Algorithm
Kruskal’s Algorithm Simulation
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
9
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9
Kruskal’s Algorithm Simulation
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
9
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9
Kruskal’s Algorithm Simulation
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
9
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9
Kruskal’s Algorithm Simulation
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
9
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9
Kruskal’s Algorithm Simulation
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
9
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9
Kruskal’s Algorithm Simulation
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
9
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9
Creates a cycle
Kruskal’s Algorithm Simulation
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
9
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9
Creates a cycle
Kruskal’s Algorithm Simulation
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
9
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9
Kruskal’s Algorithm Simulation
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
9
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9
Kruskal’s Algorithm Simulation
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
9
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9
Creates a cycle
Kruskal’s Algorithm Simulation
B C
1
46
2
5
A D E
F
3
G
2.5
7.5
H
8
7
9
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9
Creates a cycle
Kruskal’s Algorithm Simulation
B C
1
4
2A D E
F
3
G
2.5
7.5
H
7
Final Tree!
Same as Tprim
Kruskal’s Algorithm Pseudocode
procedure kruskal(G(V, E)): sort E in order of increasing
weights rename E so w(e1) < w(e2) < … < w(em) T = {} // final tree edges for i = 1 to m: if T ∪ ei=(u,v) doesn’t create cycle add ei to T return T
For Correctness We Need To Prove 2 Things1. Outputs a Spanning Tree Tkrsk
2. Tkrsk is a minimum spanning tree
1: Kruskal Outputs a Spanning Tree (1)
Need to prove Tkrsk is spanning AND is acyclic
Acyclic is by definition of the algorithm.
Why is Tkrsk spanning (i.e., connected)?
Recall Empty Cut Lemma:
A graph is not connected iff ∃ cut (X, Y) with no
crossing edges
If all cuts have a crossing edge -> graph is
connected!
1: Kruskal Outputs a Spanning Tree (2)Consider any cut (X, Y)
Since input G is connected, ∃ edges crossing (X, Y)
Let e* be the min-weight edge
Kruskal inspects all edges
Consider the time when Kruskal inspects e*
Claim: No edge crossing (X, Y) is in Tkrsk
And adding e* to Tkrsk cannot create a cycle =>
Why?
(because of lonely cut corallary)
Therefore Kruskal will add e*, so for each cut, there
is an edge in Tkrsk crossing it => Tkrsk spans V.
Plan for 2: Tkrsk is a Minimum Spanning Tree1. Argue that the Cut Property Implies Tkrsk is an
MST
In particular we will argue that each edge
(u, v) added to Tkrsk is a min-edge in a cut (X,
Y) in G.
Cut Property => Tkrsk is a MST (1)Let (u, v) be any edge added by Kruskal’s Algorithm.
u and v are in different comp. (b/c Kruskal checks for
cycles)
ux
y
v
t
zw
Claim: (u, v) is min-edge crossing this cut!
Cut Property => Tkrsk is a MST (2)
(u, v) is the min edge crossing the cut b/c
Kruskal looks at edges in increasing weights.
By the Cut Property => (u, v) is in every MST.
Therefore all edges added to Tkrsk is in every
MST
=> Tkrsk is “the” MST.
Q.E.D.
Summary of Kruskal’s Correctness
First proved Tkrsk is a spanning tree
Acyclicity was by definition
Connectedness followed from showing that there is
an edge of Tkrsk crossing any cut (X, Y) of G, which
by the empty cut lemma implied that Tkrsk was
connected
Second proved Tkrsk was a minimum spanning tree by
arguing that any edge (u, v) if Tkrsk is a min-edge in
some cut (X, Y)
Note this version of Kruskal’s Correctness Proof
contains:
• Exchange Argument: Cut Property
• Greedy Stays Ahead Argument: Krusk adds n-1
edges and all are justified.
Kruskal’s Algorithm Pseudocode
procedure kruskal(G(V, E)): sort E in order of increasing
weights rename E so w(e1) < w(e2) < … < w(em) T = {} // final tree edges for i = 1 to m: if T ∪ ei=(u,v) doesn’t create cycle add ei to T return T
Kruskal’s Algorithm Naïve Run-time
O(mlog(n)) to sort the edges
m iterations
O(n) time to check for a cycle (BFS/DFS)
Total: O(mn) total time
On Monday will improve to O(mlog(n))
Top Related