Chapter 23Minimum Spanning Trees

Kruskal Prim

[Source: Cormen et al. textbook except where noted]

Minimum Spanning Trees

This refers to a number of problems (many of practical importance) that can be reduced to the following abstraction: let G = (V, E) be a connected, undirected graph. You are also given a function w : E R (usually non-negative, but not always so required). You are then asked to find an acyclic subset T Í E that connects all the vertices and whose total weight is minimized. The acyclicity of T implies that T is a tree; the requirement that it connect all the vertices implies that it is a spanning tree; the minimization implies that it is a minimum such object.€

w T( ) = w u,v( )u,v( )∈T

∑

Minimum Spanning Trees

MST Example

Minimum Spanning Trees

Algorithms

We will construct two algorithms as variants of a “generic” one. Both will be greedy algorithms (requiring that we show all necessary greedy properties are satisfied). We start with a loop invariant: let A be a set of edges; prior to each iteration, A is a subset of some minimum spanning tree. At each step we determine an edge (u, v) (a safe edge) that we can add to A without violating the invariant.

Minimum Spanning Trees

The Generic Algorithm

Minimum Spanning Trees

The Generic AlgorithmHow do we find a safe edge? It exists, because of the assumption on A (part of a minimum spanning tree)…

Def.: a cut (S, V – S) of an undirected graph G = (V, E) is a partition of V.

Def.: an edge (u, v) crosses the cut (S, V – S) if one of its endpoints is in S and the other in V.

Def.: a cut respects a set A of edges if no edge in A crosses the cut.

Def.: An edge is a light edge crossing a cut if its weight is minimum among all edges crossing the cut. Uniqueness is not required. This can be generalized to properties other than crossing.

Minimum Spanning Trees

The Generic Algorithm

Minimum Spanning Trees

The Generic AlgorithmTheorem 23.1. Let G = (V, E) be a connected undirected graph with a real-valued weight function w defined on E. Let A be a subset of E that is included in some minimum spanning tree for G, let (S, V – S) be any cut of G that respects A, and let (u, v) be a light edge crossing (S, V – S). Then edge (u, v) is safe for A.

Proof: let T be an MST, A Í T, A ≠ T. Assume T does not contain the light edge (u, v). If it does, we are done. If not, we construct another MST T’ that contains both A and (u, v).

T {(u, v)} must contain a cycle, with edges on a simple path p from u to v in T. u and v are on opposite sides of the cut (S, V – S), and at least one edge in T lies on p and crosses the cut.

€

∪

Minimum Spanning Trees

The Generic AlgorithmProof (cont.) Let (x, y) be any such edge – it cannot be in A, because the cut respects A. Since (x, y) is on the unique simple path from u to v in T, removing (x, y) breaks T into two components. Adding (u, v) reconnects them to form a new spanning tree T’ = (T – {(x, y)}) {(u, v)}. But T’ is also an MST: since (u, v) is a light edge crossing (S, V – S) and (x, y) also crosses the cut, w(u, v) ≤ w(x, y) and w(T’) = w(T) – w(x, y) + w(u, v) ≤ w(T). The minimality of T implies w(T) ≤ w(T’), so T’ must be minimal, also.

Is (u, v) safe? Since A Í T and (x, y) Ï A, we have A Í T’.

Thus A {(u, v)} Í T’. Since T’ is an MST, (u, v) is safe for A.

€

∪

€

∪

Minimum Spanning Trees

The Generic AlgorithmCorollary 23.2. Let G = (V, E) be a connected undirected graph with a real-valued weight function w defined on E. Let A be a subset of E that is included in some minimum spanning tree for G, let C = (VC, EC) be a connected component (tree) in the forest GA = (V, A). If (u, v) is a light edge connecting C to some other component in GA, then (u, v) is safe for A.

Proof: the cut (VC, V – VC) respects A, and (u, v) is a light edge for this cut. Therefore (u, v) is safe for A.

Minimum Spanning Trees

The Generic AlgorithmWe now move to realizations of the generic algorithm.

We choose two distinct ones, one based on a sorted (non-decreasing) list of edges and the set operations MAKE-SET, FIND-SET and UNION; the other based on priority queues of edges and the operation EXTRACT-MIN.

Minimum Spanning Trees

Kruskal’s Algorithm

Minimum Spanning Trees

Kruskal’s AlgorithmThis algorithm uses the disjoint-set datatype introduced in Ch. 21. The user-accessible operations are MAKE-SET(x), UNION(x, y) and FIND-SET(x), where x and y stand for either singletons or sets of which they are representative elements.

We can show (easily) that a sequence of m MAKE-SET, UNION and FIND-SET operations (with an easy implementation), n of which are MAKE-SET takes O(m lg n) time. A more complex implementation (with union-by-rank and path compression) and analysis gives a bound O(m•a(n)), where a(n) ≤ 4 for any value of n relevant to our universe.

Minimum Spanning Trees

Kruskal’s AlgorithmLine 1 takes O(1); lines 2 and 3 take O(n), but we will account for it later; line 4 takes O(|E| lg |E|); the for loop (ll. 5-8) performs O(|E|) FIND-SET and UNION operations, which, with the |V| MAKE-SET operations take a total of O((|V| + |E|)a(|V|)) time. The assumption that G is connected implies |E| ≥ |V| - 1, so that the disjoint-set operations take O(|E|a(|V|)). Since a(|V|) = O(lg |V|) = O(lg |E|), the total time of Kruskal’s algorithm is O(|E| lg |E|) and |E| < |V|2 reduces this to O(|E| lg |V|).

Minimum Spanning Trees

Kruskal’s AlgorithmWe observe that the set A is a forest, whose vertices are all those of the given graph. A safe edge is added to A: it is always a least-weight edge that connects two distinct components.

Minimum Spanning Trees

Prim’s Algorithm

Minimum Spanning Trees

Prim’s Algorithm The edges in the set A always make up a single tree.The tree starts from an arbitrary root vertex r and grows until it spans all vertices in V. Each step adds to the tree A a light edge that connects A to an isolated vertex (one on which no edge of A is incident) By Cor. 23.2, this process this adds only edges that are safe for A, so that, at the end, A forms an MST. The strategy is greedy, since at each step it adds to the tree an edge of minimum weight.

Minimum Spanning Trees

Prim’s Algorithm: Invariant

Prior to each iteration of the while loop (ll. 6-11):

1. A = {(v, v.p) : v Î V – {r} – Q}

2. The vertices already placed into the minimum spanning tree are those in V – Q

3. For all vertices v Î Q, if v.p ≠ NIL, then v.key < ¥ and v.key is the weight of a light edge (v, v.p) connecting v to some vertex already placed into the MST.

Minimum Spanning Trees

Prim’s Algorithm: Maintenance L. 7 extracts a vertex u in Q incident on a light edge that crosses the cut (V – Q, Q) (except for the first iteration, which extracts r). Removing u from Q adds it to V – Q of vertices in the tree, adding (u, u.p) to A. The for loop updates the key and p attributes of each vertex adjacent to u, but not in the tree, thus maintaining the third part of the loop invariant.

Minimum Spanning Trees

Prim’s Algorithm: Time

This will depend on the implementation of the priority queue. But;

1. Creating a priority queue is O(|V|).

2. Extracting elements is O(lg |V|) for each extraction.

3. Decrease-Key operations, necessary because the keys of elements in the queue can change, will also take time O(lg |V|). Since this can happen for each edge we examine, total time can be O(|E| lg |V|).

Minimum Spanning Tree:Greedy Algorithms

A B

CD

E

F

G

2

2

1 1

3

4

4

5 6

68

7

source: 91.503 textbook Cormen et al.

for Undirected, Connected, Weighted Graph

G=(V,E)

Produces minimum weight tree of edges that includes every vertex.

Invariant: Minimum weight spanning forest

Becomes single tree at end

Invariant: Minimum weight tree

Spans all vertices at end

Time: O(|E|lg|E|) given fast FIND-SET, UNION

Time: O(|E|lg|V|) = O(|E|lg|E|) slightly faster with fast priority queue

Minimum Spanning Trees

Review problem: For the undirected, weighted graph below, show 2 different

Minimum Spanning Trees. Draw each using one of the 2 graph copies below. Thicken an edge to make it part of a spanning tree. What is the sum of the edge weights for each of your Minimum Spanning Trees?

A B

CD

E

F

G

2

2

1 1

3

4

4

5 6

68

7

Chapter 24Shortest Paths

Dijkstra

[Source: Cormen et al. textbook except where noted]

BFS as a Basis for Shortest Path Algorithms

Source/Sink Shortest Path

Problem: Given 2 vertices u, v, find the shortest path in G from u to v.

Solution: BFS starting at u. Stop at v.

Single-Source Shortest Paths

Problem: Given a vertex u, find the shortest path in G from u to each vertex.

Solution: BFS starting at u. Full BFS tree.

source: based on Sedgewick, Graph Algorithms

BFSfor unweighted, undirected graph G=(V,E)

Time: O(|V| + |E|) adj list

O(|V|2) adj matrix

Time: O(|V|(|V| + |E|)) adj list

O(|V|3) adj matrix

All-Pairs Shortest Paths

Problem: Find the shortest path in G from each vertex u to each vertex v.

Solution: For each u: BFS starting at u; full BFS tree.

Shortest Path Applications

Road maps Airline routes Telecommunications network routing VLSI design routing

Weight ~ Cost ~ Distance

source: based on Sedgewick, Graph Algorithms

for weighted, directed graph G=(V,E)

Shortest Path Trees

source: Sedgewick, Graph Algorithms

Shortest Path Tree gives shortest path from root to each other vertex

Shortest Path Principles: Relaxation

Graph Initialization

We start with a weighted, directed graph with a weight function w defined on its edges. We will generally consider a distinguished vertex, called the source, and usually denoted by s.

Shortest Path Principles: Relaxation

“Relax” a constraint to try to improve solution “Rubber band” analogy [Sedgewick]

Relaxation of an Edge (u,v): test if shortest path to v [found so far] can be improved

by going through u A B

C D

E

F

G

2

2

1 1

3

4

4

5 6

68

7

Shortest Path Principles: Relaxation

Shortest Path Principles: Bellman-Ford

DAG-SHORTEST-PATHS

Single Source Shortest Paths Dijkstra’s Algorithm

source: 91.503 textbook Cormen et al.

for (nonnegative) weighted, directed graph G=(V,E)

Single Source Shortest Paths Dijkstra’s Algorithm

source: 91.503 textbook Cormen et al.

for (nonnegative) weighted, directed graph G=(V,E)

Single Source Shortest Paths Dijkstra’s Algorithm

See separate ShortestPath 91.404 slide show

A B

C D

E

F

G

2

2

1 1

3

4

4

5 6

68

7

source: 91.503 textbook Cormen et al.

for (nonnegative) weighted, directed graph G=(V,E)

Single Source Shortest Paths Dijkstra’s Algorithm

Review problem: For the directed, weighted graph below, find the shortest

path that begins at vertex A and ends at vertex F. List the vertices in the order that they appear on that path. What is the sum of the edge weights of that path?

A B

C D

E

F

G

2

2

1 1

3

4

4

5 6

68

7

Why can’t Dijkstra’s algorithm handle negative-weight edges?

Single Source Shortest Paths Dijkstra’s Algorithm

source: Sedgewick, Graph Algorithms

for (nonnegative) weighted, directed graph G=(V,E)

Shortest Path Algorithms

source: Sedgewick, Graph Algorithms