1

CPSC 320: Intermediate AlgorithmDesign and Analysis

July 16, 2014

2

Course Outline

• Introduction and basic concepts

• Asymptotic notation

• Greedy algorithms

• Graph theory

• Amortized analysis

• Recursion

• Divide-and-conquer algorithms

• Randomized algorithms

• Dynamic programming algorithms

• NP-completeness

3

Union-Find Data Structure

4

Union-Find Data Structure

• Assume any graph algorithm that needs to maintain connected components

• Operations:

• Create: build a new set where each node is in its own component

• Find: return which component a node is in

• Union: join two components

• Objective: create should run in , find and union should run in or better

5

Tree implementation

• Idea: each node points to a “parent”, nodes are identified by the parent

• Create: pointers are initialized to nothing

• Find: return the root of the tree that contains the node

• Union: set the parent of one node to be the parent of the other

• Problem: union is , but find is , can we improve?

• Union: the shallowest becomes child of the deepest

• Each node keeps track of its depth (“rank”)

• Path compression:

• When find is performed, every node in the path is updated to point to root

6

Cost claims

• Claim: Find takes in average over calls

• is the (binary) iterated logarithm function

• Number of times lg needs to be applied until 1 (or less) is reached

• Example: (because , lg was applied 5 times)

7

Amortized Analysis

8

Amortized cost

• Although the worst case for one call has a large bound, we are interested in the total cost of a sequence of calls

• In other words, calls to a function may be better than

• The amortized cost of the function takes the total amount in consideration

9

Potential Method

• Each operation receives a potential cost, which if larger than actual cost can be “saved” towards future operations

• Operations that take longer can use “saved” value

• Total cost of sequence of operations is the sum of potential costs

• Properties:

• Assume is state of algorithm after iterations

• (you cannot use more than what you saved)

• (you start with no saved cost – usually)

• Alternative notation:

• for real cost, for amortized cost

10

Union-Find with Path Compression

• Property: For every node N, rank(N)<rank(parent(N))

• Initial update: trivial; after compression: property is maintained

• Property: A node of rank has at least nodes in its tree

• Proof by induction on

• Property: If there are elements, then there are at most nodes of rank

• This implies the maximum rank is

• Theorem: The amount of time required for find+union operations on a union-find structure with elements is

11

Proof – Potential Function

• Potential function depends on depth, but also on comparing with parents

• Assume is a node in the tree, is its parent, and is its rank

• Divide nodes based on the comparison of ranks between node and parent:

• root: no parent

• close:

• far: (but not close)

• really far: everything else

• Observation:

• if is close

• if is far

12

Proof – Potential Function

• Based on previous observations, , and thus

• Now let’s analyse the cost of each operation

13

Proof – Create operation

• Create runs in linear time, as expected.

14

Proof – Union operation

• If we add tree rooted in to tree rooted in :

• decreases (changes from root to some other rule)

• may increase by 3 if rank is incremented

• If rank of increases, if then may decrease

• All other nodes are unaltered

• So , and by consequence

15

Proof – Find operation

• Suppose that we are looking for node , and to find its root we look at (and update) nodes

• Then real cost of find is

• What about ? Let’s track for which nodes changes

• does not change

• does not change

• nodes not in do not change

• For all nodes new parent rank is now higher, so may change from:

• really far to really far (does not change)

• far to far or really far

• close to close, far or really far

16

Proof – Find operation

• If changes from close or far to really far:

• in , in , so decreases

• If changes from close to close:

• increases by at least , so decreases

• If changes from close to far:

• In ,

• In , , so decreases

• If changes from far to far:

• For at most one node, , so may be the same

• For all other nodes, , so decreases

17

Proof – Find operations

• So, which nodes did not decrease?

• , , at most one far node, and nodes that were really far before

• In total: at most

• So, how many nodes did decrease (by at least 1)?

• at least

18

Proof – Conclusion

• Since and , a sequence of operations is

• CQD

19

Recurrence Relations

20

Recursive Functions

• Complexity analysis of iterative functions has been covered

• How can we analyse the complexity of a recursive function?

• Example:

Algorithm MergeSort(, , )

If () Then

MergeSort(, , )

MergeSort(, , )

Merge(, , , )

21

Recurrence Relations

• The complexity of MergeSort can be computed as:

• How can we figure out what the complexity is for the function above?

22

Guess and Test

• First approach: guess one expression and test if it works

• Test usually involves strong induction

• Example: prove that

• Induction step:

23

Guess and Test

• Base case:

• :

doesn’t work!

• :

2 is not enough, since

24

Guess and Test

• Base case (cont.):

• :

• So, for and ,

25

Guess and Test

• Prove that

• Induction step:

• Prove didn’t reach expected result, disproved

• Careful, even though c+d is a constant, we needed the constant to be c only.

26

Guess and Test

• Prove that

• Induction step:

• Prove didn’t reach expected result, but we got close

• Let’s try a stronger claim:

27

Guess and Test

• Prove that

• Induction step:

• Base case,

• Proved that for and