CSCE 510 - Systems Programming Lecture 02 - System Calls CSCE 510 Jan 16, 2013.
CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411,...
-
Upload
archibald-mccormick -
Category
Documents
-
view
222 -
download
2
Transcript of CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411,...
![Page 1: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/1.jpg)
CSCE 411Design and Analysis
of Algorithms
Background ReviewProf. Jennifer Welch
CSCE 411, Background Review 1
![Page 2: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/2.jpg)
Outline Algorithm Analysis Tools Sorting Graph Algorithms
CSCE 411, Background Review 2
![Page 3: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/3.jpg)
Algorithm Analysis Tools Big-Oh notation Basic probability
CSCE 411, Background Review 3
![Page 4: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/4.jpg)
How to Measure Efficiency Machine-independent way:
analyze "pseudocode" version of algorithm assume idealized machine model
one instruction takes one time unit "Big-Oh" notation
order of magnitude as problem size increases Worst-case analyses
safe, often occurs most often, average case often just as bad
CSCE 411, Background Review 4
![Page 5: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/5.jpg)
Faster Algorithm vs. Faster CPU A faster algorithm running on a slower machine will
always win for large enough instances Suppose algorithm S1 sorts n keys in 2n2 instructions Suppose computer C1 executes 1 billion instruc/sec
When n = 1 million, takes 2000 sec Suppose algorithm S2 sorts n keys in 50nlog2n
instructions Suppose computer C2 executes 10 million instruc/sec
When n = 1 million, takes 100 sec
CSCE 411, Background Review 5
![Page 6: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/6.jpg)
Caveat No point in finding fastest algorithm
for part of the program that is not the bottleneck
If program will only be run a few times, or time is not an issue (e.g., run overnight), then no point in finding fastest algorithm
CSCE 411, Background Review 6
![Page 7: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/7.jpg)
Review of Big-Oh Notation Measure time as a function of input size
and ignore multiplicative constants lower order terms
f(n) = O(g(n)) : f is “≤” g f(n) = Ω(g(n)) : f is “≥” g f(n) = (g(n)) : f is “=” g f(n) = o(g(n)) : f is “<” g f(n) = (g(n)) : f is “>” g
CSCE 411, Background Review 7
![Page 8: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/8.jpg)
Big-Oh f(n) = O(g(n)) means, informally, that g is an
upper bound on f Formally: there exist c > 0, n0 > 0 s.t.
f(n) ≤ cg(n) for all n ≥ n0. To show that f(n) = O(g(n)), either find c and
n0 that satisfy the definition, or use this fact: if limn->∞ f(n)/g(n) = some nonnegative constant,
then f(n) = O(g(n)).
CSCE 411, Background Review 8
![Page 9: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/9.jpg)
Big-Omega f(n) = Ω(g(n)) means, informally, that g is an
lower bound on f Formally: there exist c > 0, n0 > 0 s.t.
f(n) ≥ cg(n) for all n ≥ n0. To show that f(n) = Ω(g(n)), either find c and
n0 that satisfy the definition, or use this fact: if limn->∞ f(n)/g(n) = some positive constant or ∞,
then f(n) = Ω(g(n)).
CSCE 411, Background Review 9
![Page 10: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/10.jpg)
Big-Theta f(n) = Θ(g(n)) means, informally, that g is a
tight bound on f Formally: there exist c1 > 0, c2 > 0, n0 > 0 s.t.
c1g(n) ≤ f(n) ≤ c2g(n) for all n ≥ n0.
To show that f(n) = Θ(g(n)), either find c1, c2, n0 that satisfy the definition, or use this fact: if limn->∞ f(n)/g(n) = some positive constant,
then f(n) = Θ(g(n)).
CSCE 411, Background Review 10
![Page 11: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/11.jpg)
Tying Back to Algorithm Analysis Algorithm has worst case running time O(g(n))
if there exist c > 0 and n0 > 0 s.t. for all n ≥ n0, every execution (including the slowest) on an input of size n takes at most cg(n) time.
Algorithm has worst case running time Ω(g(n)) if there exist c > 0 and n0 > 0 s.t. for all n ≥ n0, at least one execution (including the slowest) on an input of size n takes at least cg(n) time.
CSCE 411, Background Review 11
![Page 12: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/12.jpg)
More Definitions f(n) = o(g(n)) means, informally, g is a
non-tight upper bound on f limn->∞ f(n)/g(n) = 0 pronounced “little-oh”
f(n) = ω(g(n)) means, informally, g is a non-tight lower bound on f limn->∞ f(n)/g(n) = ∞ pronounced “little-omega”
CSCE 411, Background Review 12
![Page 13: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/13.jpg)
CSCE 411, Spring 2012: Set 3 13
How To Solve Recurrences Ad hoc method:
expand several times guess the pattern can verify with proof by induction
Master theorem general formula that works if recurrence has the form
T(n) = aT(n/b) + f(n) a is number of subproblems n/b is size of each subproblem f(n) is cost of non-recursive part
![Page 14: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/14.jpg)
CSCE 411, Spring 2012: Set 10 14
Probability Every probabilistic claim ultimately refers to
some sample space, which is a set of elementary events
Think of each elementary event as the outcome of some experiment Ex: flipping two coins gives sample space
HH, HT, TH, TT
An event is a subset of the sample space Ex: event "both coins flipped the same" is HH, TT
![Page 15: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/15.jpg)
CSCE 411, Spring 2012: Set 10 15
Sample Spaces and Events
S
A
HH THTT
HT
![Page 16: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/16.jpg)
CSCE 411, Spring 2012: Set 10 16
Probability Distribution A probability distribution Pr on a
sample space S is a function from events of S to real numbers s.t. Pr[A] ≥ 0 for every event A Pr[S] = 1 Pr[A U B] = Pr[A] + Pr[B] for every two
non- intersecting ("mutually exclusive") events A and B
Pr[A] is the probability of event A
![Page 17: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/17.jpg)
CSCE 411, Spring 2012: Set 10 17
Probability DistributionUseful facts: Pr[Ø] = 0 If A B, then Pr[A] ≤ Pr[B] Pr[S — A] = 1 — Pr[A] // complement Pr[A U B] = Pr[A] + Pr[B] – Pr[A B] ≤ Pr[A] + Pr[B]
![Page 18: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/18.jpg)
CSCE 411, Spring 2012: Set 10 18
Probability Distribution
A
B
Pr[A U B] = Pr[A] + Pr[B] – Pr[A B]
![Page 19: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/19.jpg)
CSCE 411, Spring 2012: Set 10 19
Example Suppose Pr[HH] = Pr[HT] = Pr[TH] =
Pr[TT] = 1/4. Pr["at least one head"]
= Pr[HH U HT U TH] = Pr[HH] + Pr[HT] + Pr[TH] = 3/4. Pr["less than one head"]
= 1 — Pr["at least one head"] = 1 — 3/4 = 1/4
HH THTT
HT1/4
1/4
1/4
1/4
![Page 20: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/20.jpg)
CSCE 411, Spring 2012: Set 10 20
Specific Probability Distribution discrete probability distribution: sample
space is finite or countably infinite Ex: flipping two coins once; flipping one coin
infinitely often uniform probability distribution: sample
space S is finite and every elementary event has the same probability, 1/|S| Ex: flipping two fair coins once
![Page 21: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/21.jpg)
CSCE 411, Spring 2012: Set 10 21
Flipping a Fair Coin Suppose we flip a fair coin n times Each elementary event in the sample space is
one sequence of n heads and tails, describing the outcome of one "experiment"
The size of the sample space is 2n. Let A be the event "k heads and nk tails occur". Pr[A] = C(n,k)/2n.
There are C(n,k) sequences of length n in which k heads and n–k tails occur, and each has probability 1/2n.
![Page 22: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/22.jpg)
CSCE 411, Spring 2012: Set 10 22
Example n = 5, k = 3 Event A is
HHHTT, HHTTH, HTTHH, TTHHH,HHTHT, HTHTH, THTHH,HTHHT, THHTH, THHHT
Pr[3 heads and 2 tails] = C(5,3)/25
= 10/32
![Page 23: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/23.jpg)
CSCE 411, Spring 2012: Set 10 23
Flipping Unfair Coins Suppose we flip two coins, each of
which gives heads two-thirds of the time
What is the probability distribution on the sample space?
HH THTT
HT4/9
2/9
2/9
1/9
Pr[at least one head] = 8/9
![Page 24: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/24.jpg)
CSCE 411, Spring 2012: Set 10 24
Independent Events Two events A and B are independent if
Pr[A B] = Pr[A]·Pr[B] I.e., probability that both A and B
occur is the product of the separate probabilities that A occurs and that B occurs.
![Page 25: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/25.jpg)
CSCE 411, Spring 2012: Set 10 25
Independent Events ExampleIn two-coin-flip example with fair coins: A = "first coin is heads" B = "coins are different"
HH THTT
HT1/4
1/4
1/4
1/4
A B Pr[A] = 1/2Pr[B] = 1/2Pr[A B] = 1/4 = (1/2)(1/2)so A and B are independent
![Page 26: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/26.jpg)
CSCE 411, Spring 2012: Set 10 26
Discrete Random Variables A discrete random variable X is a function
from a finite or countably infinite sample space to the real numbers.
Associates a real number with each possible outcome of an experiment
Define the event "X = v" to be the set of all the elementary events s in the sample space with X(s) = v.
Pr["X = v"] is the sum of Pr[s] over all s with X(s) = v.
![Page 27: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/27.jpg)
CSCE 411, Spring 2012: Set 10 27
Discrete Random Variable
X=v
X=v
X=v
X=vX=v
Add up the probabilities of all the elementary events inthe orange event to get the probability that X = v
![Page 28: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/28.jpg)
CSCE 411, Spring 2012: Set 10 28
Random Variable Example Roll two fair 6-sided dice. Sample space contains 36 elementary
events (1:1, 1:2, 1:3, 1:4, 1:5, 1:6, 2:1,…) Probability of each elementary event is 1/36 Define random variable X to be the
maximum of the two values rolled What is Pr["X = 3"]? It is 5/36, since there are 5 elementary
events with max value 3 (1:3, 2:3, 3:3, 3:2, and 3:1)
![Page 29: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/29.jpg)
CSCE 411, Spring 2012: Set 10 29
Independent Random Variables It is common for more than one random
variable to be defined on the same sample space. E.g.: X is maximum value rolled Y is sum of the two values rolled
Two random variables X and Y are independent if for all v and w, the events "X = v" and "Y = w" are independent.
![Page 30: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/30.jpg)
CSCE 411, Spring 2012: Set 10 30
Expected Value of a Random Variable Most common summary of a random
variable is its "average", weighted by the probabilities called expected value, or expectation, or
mean
Definition: E[X] = ∑ v Pr[X = v]
v
![Page 31: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/31.jpg)
CSCE 411, Spring 2012: Set 10 31
Expected Value Example Consider a game in which you flip two fair coins. You get $3 for each head but lose $2 for each tail. What are your expected earnings? I.e., what is the expected value of the random
variable X, where X(HH) = 6, X(HT) = X(TH) = 1, and X(TT) = —4?
Note that no value other than 6, 1, and —4 can be taken on by X (e.g., Pr[X = 5] = 0).
E[X] = 6(1/4) + 1(1/4) + 1(1/4) + (—4)(1/4) = 1
![Page 32: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/32.jpg)
CSCE 411, Spring 2012: Set 10 32
Properties of Expected Values E[X+Y] = E[X] + E[Y], for any two
random variables X and Y, even if they are not independent!
E[a·X] = a·E[X], for any random variable X and any constant a.
E[X·Y] = E[X]·E[Y], for any two independent random variables X and Y
![Page 33: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/33.jpg)
CSCE 411, Spring 2012: Set 10 33
Conditional Probability Formalizes having partial knowledge about
the outcome of an experiment Example: flip two fair coins.
Probability of two heads is 1/4 Probability of two heads when you already know
that the first coin is a head is 1/2
Conditional probability of A given that B occurs, denoted Pr[A|B], is defined to be
Pr[AB]/Pr[B]
![Page 34: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/34.jpg)
CSCE 411, Spring 2012: Set 10 34
Conditional Probability
A
B
Pr[A] = 5/12Pr[B] = 7/12Pr[AB] = 2/12Pr[A|B] = (2/12)/(7/12) = 2/7
![Page 35: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/35.jpg)
CSCE 411, Spring 2012: Set 10 35
Conditional Probability
Definition is Pr[A|B] = Pr[AB]/Pr[B]
Equivalently, Pr[AB] = Pr[A|B]·Pr[B]
![Page 36: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/36.jpg)
Sorting Insertion Sort Heapsort Mergesort Quicksort
CSCE 411, Background Review 36
![Page 37: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/37.jpg)
CSCE 411, Spring 2012: Set 2 37
Insertion Sort Review How it works:
incrementally build up longer and longer prefix of the array of keys that is in sorted order
take the current key, find correct place in sorted prefix, and shift to make room to insert it
Finding the correct place relies on comparing current key to keys in sorted prefix
Worst-case running time is (n2)
![Page 38: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/38.jpg)
CSCE 411, Spring 2012: Set 2 38
Heapsort Review How it works:
put the keys in a heap data structure repeatedly remove the min from the heap
Manipulating the heap involves comparing keys to each other
Worst-case running time is (n log n)
![Page 39: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/39.jpg)
CSCE 411, Spring 2012: Set 2 39
Mergesort Review How it works:
split the array of keys in half recursively sort the two halves merge the two sorted halves
Merging the two sorted halves involves comparing keys to each other
Worst-case running time is (n log n)
![Page 40: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/40.jpg)
CSCE 411, Spring 2012: Set 3 40
Mergesort Example
1 32 42 5 66
2 64 5 1 2 63
5 2 64 1 3 62
5 2 64
2 5 64
1 3
1 3
62
62
5 62 4
5 62 14 3 62
1 3 62
![Page 41: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/41.jpg)
CSCE 411, Spring 2012: Set 3 41
Recurrence Relation for Mergesort Let T(n) be worst case time on a sequence
of n keys If n = 1, then T(n) = (1) (constant) If n > 1, then T(n) = 2 T(n/2) + (n)
two subproblems of size n/2 each that are solved recursively
(n) time to do the merge Solving recurrence gives T(n) = (n log
n)
![Page 42: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/42.jpg)
CSCE 411, Spring 2012: Set 2 42
Quicksort Review How it works:
choose one key to be the pivot partition the array of keys into those keys
< the pivot and those ≥ the pivot recursively sort the two partitions
Partitioning the array involves comparing keys to the pivot
Worst-case running time is (n2)
![Page 43: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/43.jpg)
Graphs Mathematical definitions Representations
adjacency list, adjacency matrix
Traversals breadth-first search, depth-first search
Minimum spanning trees Kruskal’s algorithm, Prim’s algorithm
Single-source shortest paths Dijkstra’s algorithm, Bellman-Ford algorithm
CSCE 411, Background Review 43
![Page 44: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/44.jpg)
Graphs Directed graph G = (V,E)
V is a finite set of vertices E is the edge set, a binary relation on E (ordered pairs of
vertice) In an undirected graph, edges are unordered pairs of
vertices If (u,v) is in E, then v is adjacent to u and (u,v) is
incident from u and incident to v; v is neighbor of u in an undirected graph, u is also adjacent to v, and (u,v) is
incident on u and v Degree of a vertex is number of edges incident from
or to (or on) the vertex
CSCE 411, Spring 2012: Set 4 44
![Page 45: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/45.jpg)
More on Graphs A path of length k from vertex u to vertex u’ is
a sequence of k+1 vertices s.t. there is an edge from each vertex to the next in the sequence In this case, u’ is reachable from u
A path is simple if no vertices are repeated A path forms a cycle if last vertex = first vertex A cycle is simple if no vertices are repeated
other than first and last
CSCE 411, Spring 2012: Set 4 45
![Page 46: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/46.jpg)
Graph Representations So far, we have discussed graphs purely as
mathematical concepts How can we represent a graph in a
computer program? We need some data structures.
Two most common representations are: adjacency list – best for sparse graphs (few edges) adjacency matrix – best for dense graphs (lots of
edges)
CSCE 411, Spring 2012: Set 4 46
![Page 47: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/47.jpg)
Adjacency List Representation
Given graph G = (V,E) array Adj of |V| linked lists, one for each vertex in V The adjacency list for vertex u, Adj[u], contains all
vertices v s.t. (u,v) is in E each vertex in the list might be just a string,
representing its name, or it might be some other data structure representing the vertex, or it might be a pointer to the data structure for that vertex
Requires Θ(V+E) space (memory) Requires Θ(degree(u)) time to check if (u,v) is in E
CSCE 411, Spring 2012: Set 4 47
![Page 48: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/48.jpg)
CSCE 411, Spring 2012: Set 8 48
Adjacency List Representation
a
c d
b
e
a
b
c
d
e
b c
a d e
a d
b c e
b d
Space-efficient for sparse graphs
![Page 49: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/49.jpg)
Adjacency Matrix Representation
Given graph G = (V,E) Number the vertices 1, 2, ..., |V| Use a |V| x |V| matrix A with (i,j)-entry of A
being 1 if (i,j) is in E 0 if (i,j) is not in E
Requires Θ(V2) space (memory) Requires Θ(1) time to check if (u,v) is in E
CSCE 411, Spring 2012: Set 4 49
![Page 50: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/50.jpg)
CSCE 411, Spring 2012: Set 8 50
Adjacency Matrix Representation
a
c d
b
e
a
b
c
d
e
Check for an edge in constant time
a b c d e0 1 1 0 0
1 0 0 1 1
1 0 0 1 0
0 1 1 0 1
0 1 0 1 0
![Page 51: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/51.jpg)
More on Graph Representations Read Ch 22, Sec 1 for
more about directed vs. undirected graphs how to represented weighted graphs
(some value is associated with each edge) pros and cons of adjacency list vs.
adjacency matrix
CSCE 411, Spring 2012: Set 4 51
![Page 52: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/52.jpg)
CSCE 411, Spring 2012: Set 8 52
Graph Traversals Ways to traverse/search a graph
Visit every vertex exactly once
Breadth-First Search Depth-First Search
![Page 53: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/53.jpg)
CSCE 411, Spring 2012: Set 8 53
Breadth First Search (BFS) Input: G = (V,E) and source s in V for each vertex v in V do
mark v as unvisited mark s as visited enq(Q,s) // FIFO queue Q while Q is not empty do
u := deq(Q) for each unvisited neighbor v of u do
mark v as visited enq(Q,v)
![Page 54: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/54.jpg)
CSCE 411, Spring 2012: Set 8 54
BFS Example start at s and consider in alphabetical
order
a
c
d
b
s
![Page 55: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/55.jpg)
CSCE 411, Spring 2012: Set 8 55
BFS Tree We can make a spanning tree rooted
at s by remembering the "parent" of each vertex
![Page 56: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/56.jpg)
CSCE 411, Spring 2012: Set 8 56
Breadth First Search #2 Input: G = (V,E) and source s in V for each vertex v in V do
mark v as unvisited parent[v] := nil
mark s as visited parent[s] := s enq(Q,s) // FIFO queue Q
![Page 57: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/57.jpg)
CSCE 411, Spring 2012: Set 8 57
Breadth First Search #2 while Q is not empty do
u := deq(Q) for each unvisited neighbor v of u do
mark v as visited parent[v] := u enq(Q,v)
![Page 58: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/58.jpg)
CSCE 411, Spring 2012: Set 8 58
BFS Tree Example
a
c
d
b
s
![Page 59: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/59.jpg)
CSCE 411, Spring 2012: Set 8 59
BFS Trees BFS tree is not necessarily unique for
a given graph Depends on the order in which
neighboring vertices are processed
![Page 60: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/60.jpg)
CSCE 411, Spring 2012: Set 8 60
BFS Numbering During the breadth-first search, assign
an integer to each vertex Indicate the distance of each vertex
from the source s
![Page 61: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/61.jpg)
CSCE 411, Spring 2012: Set 8 61
Breadth First Search #3 Input: G = (V,E) and source s in V for each vertex v in V do
mark v as unvisited parent[v] := nil d[v] := infinity
mark s as visited parent[s] := s d[s] := 0 enq(Q,s) // FIFO queue Q
![Page 62: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/62.jpg)
CSCE 411, Spring 2012: Set 8 62
Breadth First Search #3 while Q is not empty do
u := deq(Q) for each unvisited neighbor v of u do
mark v as visited parent[v] := u d[v] := d[u] + 1 enq(Q,v)
![Page 63: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/63.jpg)
CSCE 411, Spring 2012: Set 8 63
BFS Numbering Example
a
c
d
b
sd = 0
d = 1
d = 1
d = 2
d = 2
![Page 64: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/64.jpg)
CSCE 411, Spring 2012: Set 8 64
Shortest Path Tree Theorem: BFS algorithm
visits all and only vertices reachable from s sets d[v] equal to the shortest path
distance from s to v, for all vertices v, and sets parent variables to form a shortest
path tree
![Page 65: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/65.jpg)
CSCE 411, Spring 2012: Set 8 65
BFS Running Time Initialization of each vertex takes O(V) time Every vertex is enqueued once and
dequeued once, taking O(V) time When a vertex is dequeued, all its
neighbors are checked to see if they are unvisited, taking time proportional to number of neighbors of the vertex, and summing to O(E) over all iterations
Total time is O(V+E)
![Page 66: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/66.jpg)
CSCE 411, Spring 2012: Set 8 66
Depth-First Search Input: G = (V,E) for each vertex u in V do
mark u as unvisited
for each unvisited vertex u in V do call recursiveDFS(u)
recursiveDFS(u): mark u as visited for each unvisited neighbor v of u do
call recursiveDFS(v)
![Page 67: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/67.jpg)
CSCE 411, Spring 2012: Set 8 67
DFS Example start at a and consider in alphabetical
order a
c
d
b
e
f g h
![Page 68: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/68.jpg)
CSCE 411, Spring 2012: Set 8 68
Disconnected Graphs What if graph is disconnected or is
directed? call DFS on several vertices to visit all
vertices purpose of second for-loop in non-recursive
wrapper a
cb
d
e
![Page 69: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/69.jpg)
CSCE 411, Spring 2012: Set 8 69
DFS Tree Actually might be a DFS forest
(collection of trees) Keep track of parents
![Page 70: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/70.jpg)
CSCE 411, Spring 2012: Set 8 70
Depth-First Search #2 Input: G = (V,E) for each vertex u in
V do mark u as unvisited parent[u] := nil
for each unvisited vertex u in V do parent[u] := u // a
root call recursive DFS(u)
recursiveDFS(u): mark u as visited for each unvisited
neighbor v of u do parent[v] := u call recursiveDFS(v)
![Page 71: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/71.jpg)
CSCE 411, Spring 2012: Set 8 71
More Properties of DFS Need to keep track of more
information for each vertex: discovery time: when its recursive
call starts finish time: when its recursive call
ends
![Page 72: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/72.jpg)
CSCE 411, Spring 2012: Set 8 72
Depth-First Search #3 Input: G = (V,E) for each vertex u in V do
mark u as unvisited parent[u] := nil
time := 0 for each unvisited
vertex u in V do parent[u] := u // a root call recursive DFS(u)
recursiveDFS(u): mark u as visited time++ disc[u] := time for each unvisited
neighbor v of u do parent[v] := u call recursiveDFS(v)
time++ fin[u] := time
![Page 73: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/73.jpg)
CSCE 411, Spring 2012: Set 8 73
Running Time of DFS initialization takes O(V) time second for loop in non-recursive wrapper
considers each vertex, so O(V) iterations one recursive call is made for each vertex in recursive call for vertex u, all its
neighbors are checked; total time in all recursive calls is O(E)
Total time is O(V+E)
![Page 74: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/74.jpg)
CSCE 411, Spring 2012: Set 4 74
Minimum Spanning Tree
7
1645
6 8
11
15
14
17
10
13
3
12
29
18
Given a connected undirected graph with edge weights,find subset of edges that spans all the nodes,creates no cycle, and minimizes sum of weights
![Page 75: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/75.jpg)
CSCE 411, Spring 2012: Set 4 75
Facts About MSTs There can be many spanning trees of
a graph In fact, there can be many minimum
spanning trees of a graph But if every edge has a unique
weight, then there is a unique MST
![Page 76: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/76.jpg)
CSCE 411, Spring 2012: Set 9 76
Generic MST Algorithm input: weighted undirected graph
G = (V,E,w) T := empty set while T is not yet a spanning tree of G
find an edge e in E s.t. T U e is a subgraph of some MST of G
add e to T return T (as MST of G)
![Page 77: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/77.jpg)
CSCE 411, Spring 2012: Set 4 77
Kruskal's MST algorithm
7
1645
6 8
11
15
14
17
10
13
3
12
29
18
consider the edges in increasing order of weight,add in an edge iff it does not cause a cycle
![Page 78: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/78.jpg)
CSCE 411, Spring 2012: Set 4 78
Implementing Kruskal's Alg. Sort edges by weight
efficient algorithms known How to test quickly if adding in the
next edge would cause a cycle? use disjoint set data structure, later
![Page 79: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/79.jpg)
CSCE 411, Spring 2012: Set 9 79
Kruskal's Algorithm as a Special Case of Generic Algorithm Consider edges in increasing order of
weight Add the next edge iff it doesn't cause
a cycle At any point, T is a forest (set of
trees); eventually T is a single tree
![Page 80: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/80.jpg)
CSCE 411, Spring 2012: Set 4 80
Another MST Algorithm Kruskal's algorithm maintains a forest
that grows until it forms a spanning tree Alternative idea is keep just one tree and
grow it until it spans all the nodes Prim's algorithm
At each iteration, choose the minimum weight outgoing edge to add greedy!
![Page 81: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/81.jpg)
CSCE 411, Spring 2012: Set 9 81
Idea of Prim's Algorithm Instead of growing the MST as possibly
multiple trees that eventually all merge, grow the MST from a single vertex, so that there is only one tree at any point.
Also a special case of the generic algorithm: at each step, add the minimum weight edge that goes out from the tree constructed so far.
![Page 82: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/82.jpg)
CSCE 411, Spring 2012: Set 9 82
Prim's Algorithm input: weighted undirected graph G = (V,E,w) T := empty set S := any vertex in V while |T| < |V| - 1 do
let (u,v) be a min wt. outgoing edge (u in S, v not in S) add (u,v) to T add v to S
return (S,T) (as MST of G)
![Page 83: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/83.jpg)
CSCE 411, Spring 2012: Set 9 83
Prim's Algorithm Example
a
b
h
i
c
g
d
f
e
4
8 7
8
11
2
7 6
1 2
4 14
9
10
![Page 84: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/84.jpg)
CSCE 411, Spring 2012: Set 9 84
Implementing Prim's Algorithm How do we find minimum weight
outgoing edge? First cut: scan all adjacency lists at
each iteration. Results in O(VE) time. Try to do better.
![Page 85: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/85.jpg)
CSCE 411, Spring 2012: Set 9 85
Implementing Prim's Algorithm Idea: have each vertex not yet in the
tree keep track of its best (cheapest) edge to the tree constructed so far.
To find min wt. outgoing edge, find minimum among these values use a priority queue to store the best edge
info (insert and extract-min operations)
![Page 86: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/86.jpg)
CSCE 411, Spring 2012: Set 9 86
Implementing Prim's Algorithm When a vertex v is added to T, some
other vertices might have their best edges affected, but only neighbors of v add decrease-key operation to the priority
queueu
vTi
Ti+1 w
w's best edge to Ti
check if this edge is cheaper for w
x
x's best edge to Ti
v's best edge to Ti
![Page 87: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/87.jpg)
CSCE 411, Spring 2012: Set 9 87
Details on Prim's AlgorithmAssociate with each vertex v two fields: best-wt[v] : if v is not yet in the tree, then
it holds the min. wt. of all edges from v to a vertex in the tree. Initially infinity.
best-node[v] : if v is not yet in the tree, then it holds the name of the vertex (node) u in the tree s.t. w(v,u) is v's best-wt. Initially nil.
![Page 88: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/88.jpg)
CSCE 411, Spring 2012: Set 9 88
Details on Prim's Algorithm input: G = (V,E,w)// initialization initialize priority queue Q to contain all
vertices, using best-wt values as keys let v0 be any vertex in V decrease-key(Q,v0,0)
// last line means change best-wt[v0] to 0 and adjust Q accordingly
![Page 89: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/89.jpg)
CSCE 411, Spring 2012: Set 9 89
Details on Prim's Algorithm while Q is not empty do
u := extract-min(Q) // vertex w/ smallest best-wt
if u is not v0 then add (u,best-node[u]) to T for each neighbor v of u do
if v is in Q and w(u,v) < best-wt[v] then best-node[v] := u decrease-key(Q,v,w(u,v))
return (V,T) // as MST of G
![Page 90: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/90.jpg)
CSCE 411, Spring 2012: Set 9 90
Running Time of Prim's AlgorithmDepends on priority queue implementation. Let Tins be time for insert Tdec be time for decrease-key Tex be time for extract-minThen we have |V| inserts and one decrease-key in the
initialization: O(VTins+Tdec) |V| iterations of while
one extract-min per iteration: O(VTex) total
![Page 91: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/91.jpg)
CSCE 411, Spring 2012: Set 9 91
Running Time of Prim's Algorithm Each iteration of while includes a for
loop. Number of iterations of for loop varies,
depending on how many neighbors the current vertex has
Total number of iterations of for loop is O(E).
Each iteration of for loop: one decrease key, so O(ETdec) total
![Page 92: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/92.jpg)
CSCE 411, Spring 2012: Set 9 92
Running Time of Prim's Algorithm O(V(Tins + Tex) + ETdec) If priority queue is implemented with
a binary heap, then Tins = Tex = Tdec = O(log V) total time is O(E log V)
(Think about how to implement decrease-key in O(log V) time.)
![Page 93: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/93.jpg)
CSCE 411, Spring 2012: Set 9 93
Shortest Paths in a Graph Let's review two important single-
source shortest path algorithms: Dijkstra's algorithm Bellman-Ford algorithm
![Page 94: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/94.jpg)
CSCE 411, Spring 2012: Set 9 94
Single Source Shortest Path Problem Given: directed or undirected graph G
= (V,E,w) and source vertex s in V Find: For each t in V, a path in G from
s to t with minimum weight Warning! Negative weights are a
problem:
s t4
5
![Page 95: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/95.jpg)
CSCE 411, Spring 2012: Set 9 95
Shortest Path Tree Result of a SSSP algorithm can be
viewed as a tree rooted at the source Why not use breadth-first search? Works fine if all weights are the same:
weight of each path is (a multiple of) the number of edges in the path
Doesn't work when weights are different
![Page 96: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/96.jpg)
CSCE 411, Spring 2012: Set 9 96
Dijkstra's SSSP Algorithm Assumes all edge weights are nonnegative Similar to Prim's MST algorithm Start with source vertex s and iteratively
construct a tree rooted at s Each vertex keeps track of tree vertex that
provides cheapest path from s (not just cheapest path from any tree vertex)
At each iteration, include the vertex whose cheapest path from s is the overall cheapest
![Page 97: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/97.jpg)
CSCE 411, Spring 2012: Set 9 97
Prim's vs. Dijkstra's
s
5
4
1
6
Prim's MST
s
5
4
1
6
Dijkstra's SSSP
![Page 98: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/98.jpg)
CSCE 411, Spring 2012: Set 9 98
Implementing Dijkstra's Alg. How can each vertex u keep track of its
best path from s? Keep an estimate, d[u], of shortest path
distance from s to u Use d as a key in a priority queue When u is added to the tree, check each of
u's neighbors v to see if u provides v with a cheaper path from s: compare d[v] to d[u] + w(u,v)
![Page 99: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/99.jpg)
CSCE 411, Spring 2012: Set 9 99
Dijkstra's Algorithm input: G = (V,E,w) and source vertex
s// initialization d[s] := 0 d[v] := infinity for all other vertices v initialize priority queue Q to contain
all vertices using d values as keys
![Page 100: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/100.jpg)
CSCE 411, Spring 2012: Set 9 100
Dijkstra's Algorithm while Q is not empty do
u := extract-min(Q) for each neighbor v of u do
if d[u] + w(u,v) < d[v] thend[v] := d[u] + w(u,v)decrease-key(Q,v,d[v])parent(v) := u
![Page 101: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/101.jpg)
CSCE 411, Spring 2012: Set 9 101
Dijkstra's Algorithm Example
a b
d e
c
2
84 9
2
4
12
10 6 3
source is vertex a
0 1 2 3 4 5Q abcd
ebcde cde de d Ø
d[a]
0 0 0 0 0 0
d[b]
∞ 2 2 2 2 2
d[c] ∞ 12 10 10 10 10
d[d]
∞ ∞ ∞ 16 13 13
d[e]
∞ ∞ 11 11 11 11
iteration
![Page 102: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/102.jpg)
CSCE 411, Spring 2012: Set 9 102
Running Time of Dijkstra's Alg. initialization: insert each vertex once
O(V Tins)
O(V) iterations of while loop one extract-min per iteration => O(V Tex) for loop inside while loop has variable number of
iterations…
For loop has O(E) iterations total one decrease-key per iteration => O(E Tdec)
Total is O(V (Tins + Tex) + E Tdec)
![Page 103: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/103.jpg)
CSCE 411, Spring 2012: Set 9 103
Using Different Heap Implementations O(V(Tins + Tex) + ETdec) If priority queue is implemented with a
binary heap, then Tins = Tex = Tdec = O(log V) total time is O(E log V)
There are fancier implementations of the priority queue, such as Fibonacci heap: Tins = O(1), Tex = O(log V), Tdec = O(1) (amortized) total time is O(V log V + E)
![Page 104: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/104.jpg)
CSCE 411, Spring 2012: Set 9 104
Using Simpler Heap Implementations O(V(Tins + Tex) + ETdec) If graph is dense, so that |E| = (V2), then it
doesn't help to make Tins and Tex to be at most O(V).
Instead, focus on making Tdec be small, say constant.
Implement priority queue with an unsorted array: Tins = O(1), Tex = O(V), Tdec = O(1) total is O(V2)
![Page 105: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/105.jpg)
CSCE 411, Spring 2012: Set 9 105
What About Negative Edge Weights? Dijkstra's SSSP algorithm requires all
edge weights to be nonnegative even more restrictive than outlawing
negative weight cycles Bellman-Ford SSSP algorithm can
handle negative edge weights even "handles" negative weight cycles by
reporting they exist
![Page 106: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/106.jpg)
CSCE 411, Spring 2012: Set 9 106
Bellman-Ford Idea Consider each edge (u,v) and see if u
offers v a cheaper path from s compare d[v] to d[u] + w(u,v)
Repeat this process |V| - 1 times to ensure that accurate information propgates from s, no matter what order the edges are considered in
![Page 107: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/107.jpg)
CSCE 411, Spring 2012: Set 9 107
Bellman-Ford SSSP Algorithm input: directed or undirected graph G = (V,E,w)//initialization initialize d[v] to infinity and parent[v] to nil for all v in V
other than the source initialize d[s] to 0 and parent[s] to s// main body for i := 1 to |V| - 1 do
for each (u,v) in E do // consider in arbitrary order if d[u] + w(u,v) < d[v] then
d[v] := d[u] + w(u,v) parent[v] := u
![Page 108: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/108.jpg)
CSCE 411, Spring 2012: Set 9 108
Bellman-Ford SSSP Algorithm// check for negative weight cycles for each (u,v) in E do
if d[u] + w(u,v) < d[v] then output "negative weight cycle exists"
![Page 109: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/109.jpg)
CSCE 411, Spring 2012: Set 9 109
Running Time of Bellman-Ford O(V) iterations of outer for loop O(E) iterations of inner for loop O(VE) time total
![Page 110: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/110.jpg)
CSCE 411, Spring 2012: Set 9 110
Bellman-Ford Example
s
c
a
b3
—44
2
1
process edges in order(c,b)(a,b)(c,a)(s,a)(s,c)
<board work>
![Page 111: CSCE 411 Design and Analysis of Algorithms Background Review Prof. Jennifer Welch CSCE 411, Background Review 1.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697bf801a28abf838c84f18/html5/thumbnails/111.jpg)
CSCE 411, Spring 2012: Set 9 111
Speeding Up Bellman-Ford The previous example would have
converged faster if we had considered the edges in a different order in the for loop move outward from s
If the graph is a DAG (no cycles), we can fully exploit this idea to speed up the running time