Graph Algorithms Shortest path problems [Adapted from K.Wayne]
Shortest path problems and algorithms
Transcript of Shortest path problems and algorithms
Shortest path problems and algorithms1. Shortest path problems
Shortest path problem Single-source shortest path problem Single-destination shortest path problem All-pairs shortest path problem
2. Algorithms Dijkstra’s algorithm for single source shortest path problem
(positive weight) Bellman–Ford algorithm for single source shortest path problem
(allow negative weight) Floyd–Warshall algorithm for all pairs shortest paths Johnson's algorithm for all pairs shortest paths
CP412 ADAII
1. Shortest path problems
Shortest path problem:Input: weight graph G = (V, E, W),
source vertex s and destination vertex t. Output: a shortest path of G from s to t.
Single-source shortest path problem:Output: shortest paths from source vertex s to all other vertices of G.
Single-destination shortest path problem: Output: shortest paths from all vertices of G to a single destination t.
All-pairs shortest path problem:Output: shortest paths between every pair of vertices u, v in the graph.
CP412 ADAII
History of SPP and algorithms
Shimbel (1955). Information networks.
Ford (1956). RAND, economics of transportation.
Leyzorek, Gray, Johnson, Ladew, Meaker, Petry, Seitz (1957). Combat Development Dept. of the Army Electronic Proving Ground.
Dantzig (1958). Simplex method for linear programming.
Bellman (1958). Dynamic programming.
Moore (1959). Routing long-distance telephone calls for Bell Labs.
Dijkstra (1959). Simpler and faster version of Ford's algorithm.
CP412 ADAII
Dijkstra’s algorithms
Edger W. Dijkstra’s quotes The question of whether computers can think is
like the question of whether submarines can swim. In their capacity as a tool, computers will be but a ripple on
the surface of our culture. In their capacity as intellectual challenge, they are without precedent in the cultural history of mankind.
Do only what only you can do. (https://en.wikipedia.org/wiki/Turing_Award)
Dijkstra’s algorithm: solve the single-source shortest path problemGREEDY strategy Starting from source s, set visited set R = {s} Always pick the next closest vertex and add to R Use priority queue to store unvisited vertices by distance from s After deleteMin v, update distances of remaining vertices adjacent
to v using decreaseKey
CP412 ADAII
Example
CP412 ADAII4
5
0
1 2 3
48
7 6 5
4
8
8 7
117
1
2
6
4 14
9
10
2
0
1 2 3
48
7 6 5
4
8
8 7
117
1
2
6
4 14
9
10
4
8
12
2
0
1 2 3
48
7 6 5
4
8
8 7
117
1
2
6
4 14
9
10
4
8
12
9
15
2
0
1 2 3
48
7 6 5
4
8
8 7
117
1
2
6
4 14
9
10
4
8
12
9
15
2 11
0
0 0
Example
CP412 ADAII4
6
0
1 2 3
48
7 6 5
4
8
8 7
117
1
2
6
4 14
9
10
4
8
12
9
15
2 11
0
1 2 3
48
7 6 5
4
8
8 7
117
1
2
6
4 14
9
10
4
8
12
9
15
2 11
25
21
0
1 2 3
48
7 6 5
4
8
8 7
117
1
2
6
4 14
9
10
4
8
12
9
14
2 11
19
21
0
1 2 3
48
7 6 5
4
8
8 7
117
1
2
6
4 14
9
10
4
8
12
9
14
2 11
19
21
0 0
0 0
Example
Shortest Path Tree (SPT)
CP412 ADAII4
7
0
1 2 3
48
7 6 5
4
8
8 7
117
1
2
6
4 14
9
10
4
8
12
9
14
2 11
19
21
0
1 2 3
48
7 6 5
4
8
8 7
117
1
2
6
4 14
9
10
4
8
12
9
14
2 11
19
210 0
Correctness of Dijkstra’s algorithmLemma: (optimal substructure property) A least-cost path from X to Y contains least-cost paths from X to every vertex on the path to Y. (prove by contradiction)
Let d(v) be the label found by the algorithm and let δ(v) be the shortest path distance from s-to-v. We want to show that d(v) = δ(v) for every vertex v at the end of the algorithm, showing that the algorithm correctly computes the distances. We prove this by induction on |R| via the following lemma
Lemma: For each x ∈ R, d(x) = δ(x).
CP412 ADAII
Correctness of Dijkstra’s algorithmProof by Induction on |R|. Base case (|R| = 1): Since R only grows in size, the only time |R| = 1 is when R = {s} and d(s) = 0 = δ(s), which is correct.
Suppose it is true for |R| < k. Now assume |R| = k. Let u be the last vertex added to R. We need only show that d(u) = δ(u). Suppose for a contradiction that the shortest path from s-to-u is Q and has length w(Q) < d(u). Let R0 = R \ {u}. Q starts in R0 and at some leaves R0 (to get to u which is not in R0 ). Let xy be the first edge along Q that leaves R0 . Let Qx be the s-to-x subpath of Q. Clearly: w(Qx) + w(xy) ≤ w(Q). Since d(x) is the length of the shortest s-to-x path, d(x) ≤ w(Qx), d(x) + w(xy) ≤ w(Q). Since y is adjacent to x, d(y) must have been updated by the algorithm, so d(y) ≤ d(x) + w(xy), and d(y) < d(u). Finally, since u was picked by the algorithm, u must have the smallest distance label: d(u) ≤ d(y). Combining these inequalities gives us the contradiction that d(u) < d(u). Therefore, no such shorter path Q must exist and so d(u) = δ(u).
CP412 ADAII
Implementation and analysis
CP412 ADAII
CP412 ADAII
BuildHeap: O(|V|)
DeleteMin: O(|V| log |V|)
DecreaseKey: O(|E| log |V|)
Total running time: O(|E| log |V|)
What about graphs with negative edges?
CP412 ADAII
Do not mark any vertex as “known”.Instead allow multiple updates.
Bellman-Ford Algorithm
CP412 ADAII
The idea of Bellman-Ford algorithm is from dynamic programming.
Consider problem of shortest path using k number of edges for given source s and any destination v. Let dk(s, v) denote the weight of shortest path from s to v using at most k edges.
We can start by determining d0(s, v) for all v ϵ V, which is infinite for all vertices except s itself, i.e., d0(s, s) = 0, d0(s, v) = ∞, v≠s. Then we can use this to determine d1(s, v) for all v ϵ V. In general we want to determine dk+1(s, v) based on all dk(s, v).
Observation: if a dk+1(s, v) shortest path uses at most k edges, then dk+1(s, v) = dk(s, v), otherwise it uses k+1 edges, then it reaches of v’s neighbor x with k edges and plus edge (x, v). Therefore,
dk+1(s, v) = min{ dk(s, v), min{dk(s, x)+w(x, v) : x ϵ N-(v)} }
Bellman-Ford algorithm and analysis
CP412 ADAII
distance[|V|], predecessor[|V|], source vertex s // Step 1: initialize graph
for each vertex v in V: // O(|V|)distance[v] := inf // At the beginning , all vertices have a weight of infinitypredecessor[v] := null // And a null predecessor
distance[s] := 0
// Step 2: relax edges repeatedlyfor i from 1 to |V|-1:
for each edge (u, v) with weight w in edges: // O(|V||E|)if distance[u] + w < distance[v]:
distance[v] := distance[u] + wpredecessor[v] := u
// Step 3: check for negative-weight cyclesfor each edge (u, v) with weight w in edges:
if distance[u] + w < distance[v]:error "Graph contains a negative-weight cycle"
return distance[], predecessor[]
Time: O(|V||E|), Space : O(|V|)
Example
CP412 ADAII
Correctness of Bellman-Ford Algorithm
CP412 ADAII
Theorem. Given a directed weighted graph G = (V, E, w), w : E → R, and a source s, the BellmanFord algorithm returns the shortest path length from s to every vertex or indicates that there is a negative weight cycle in G reachable from s.
Proof. By induction on the number of edges k in a path. The base case is correct since d0(s,s) = 0. For all v in V \ s, on each step a shortest (s, v) path with up to k +1 edges must consist of a shortest (s, u) path of up to k edges followed by a single edge (u, v). Therefore if we take the minimum of these we get the overall shortest path with up to k+1 edges.
The algorithm only proceeds to |V| rounds (step 3). If there is a reachable negative-weight cycle, error returns; otherwise shortest path to every v is simple and consist of at most n vertices and hence |V|-1 edges.
CP412 ADAII
Also known as Floyd's algorithm, the Roy–Warshall algorithm, the Roy–Floyd algorithm, is an algorithm for finding all pair shortest paths in a weighted graph with positive or negative edge weights (but with no negative cycles). Based on the idea of dynamic programming.
Let G be graph with vertices numbered 1 through N. Consider a function sp(i,j,k) that returns the shortest possible path from i to j using vertices only from the set {1,2, …,k} as intermediate points along the way. • Now, given this function, find the shortest path from each i to each j
using only vertices in {1, …, k+1}. • For each of these pairs of vertices, the true shortest path could be
either (1) a path that only uses vertices in the set {1,… ,k} or (2) a path that goes from i to k+1 and then from k+1 to j.
Floyd–Warshall algorithm
CP412 ADAII 18
• The best path from i to j that only uses vertices 1 through k is defined by sp(i,j,k). If there were a better path from i to k+1 to j, then the length of this path is the concatenation of the shortest path from i to k+1 using vertices in {1, … ,k} and the shortest path from k+1 to j (also using vertices in {1, … ,k}.
Then sp(i, j, k+1) satisfies the following recursion formula:
sp(i,j,0) = w(i, j)
sp(i,j,k+1)=min{sp(i,j,k), sp(i,k+1,k)+sp(k+1,j,k) }.
Floyd–Warshall algorithm and analysis
CP412 ADAII 19
D[|V|] [|V|] = { ∞ }
for each vertex v in VD[v][v] ← 0for each edge (u,v)
D[u][v] ← w(u,v) // the weight of the edge (u,v) // O(|V|2)
for k from 1 to |V|for i from 1 to |V|
for j from 1 to |V|if D[i][j] > D[i][k] + D[k][j]
D[i][j] ←D[i][k] + D[k][j] // O(|V|3)end if
Time: O(|V|3), Space: O(|V|2)
Floyd–Warshall algorithm and recovery pathD[|V|] [|V|] = { ∞ }, n[|V|] [|V|] = { null }FloydWarshall()
for each edge (u,v)D[u][v] ← w(u,v) n[u][v] ← u
for k from 1 to |V| // standard Floyd-Warshall implementationfor i from 1 to |V|
for j from 1 to |V|if D[i][k] + D[k][j] < D[i][j] then
D[i][j] ← D[i][k] + D[k][j]n[i][j] ← n[i][k]
procedure Path(u, v)if n[u][v] = null then
return []path = [u]while u ≠ v
u ← n[u][v]path.append(u)
return path
20CP412 ADAII
Example
CP412 ADAII 21
Initialization: (k = 0)
CP412 ADAII 22
Iteration 1: (k = 1) Shorter paths from 2 ↝ 3 and 2 ↝ 4 are found through vertex 1
Iteration 2: (k = 2) Shorter paths from 4 ↝ 1, 5 ↝ 1, and 5 ↝ 3 are found through vertex 2
CP412 ADAII 23
Iteration 3: (k = 3) No shorter paths are found through vertex 3
CP412 ADAII 24
Iteration 4: (k = 4) Shorter paths from 1 ↝ 2, 1 ↝ 3, 2 ↝ 3, 3 ↝ 1, 3 ↝ 2, 5 ↝ 1, 5 ↝ 2, 5 ↝ 3, and 5 ↝ 4 are found through vertex 4
Iteration 5: (k = 5) No shorter paths are found through vertex 5