Ch.4

Decrease and Conquer

Decrease-by-One Quiz

Hint: How can one soldier traverse, with all other conditions restored to where they were at the beginning?

4.1 Insertion Sort

Sort the array of nelements by:first sorting the sub-array of its first n-1 elements, and then inserting the last element in the correct position.

As presented, this is a top-down strategy, to be implemented recursively.

However, we can also work in reverse, bottom-up. This allows the simpler, iterative solution - see next slide.

What exactly gets done in the first pass?Second pass?

AnalysisIf already sorted:

If reverse-sorted:

Average?Proof not in text, but left for Ex. 11 – see next slide

Conclusion: Insertion Sort is better than Selection and Bubble Sort!

Beyond Insertion Sort: Shell Sort• Donald Shell (1959)• Starts by sorting pairs of elements far apart from each other, then

progressively reducing the gap between elements to be compared. By starting with far apart elements, it can move some out-of-place elements into position faster than a simple nearest neighbor exchange (Bubble Sort).

• https://www.codingeek.com/algorithms/shell-sort-algorithm-explanation-implementation-and-complexity/

• It is proved that with some carefully selected gap sequences, the average complexity is Θ(N∙N2/3) and Θ(N∙N8/15).

• It is conjectured that, with some carefully selected gap sequences, the average complexity is Θ(N∙N1/4), and even Θ(N∙lnN∙ln(lnN)).

4.1 Individual work for next time (in notebook!)

End-of-section exercises 2, 7.

4.2 Topological SortingNote: This refers to sorting of graph nodes, not arrays!

Review of the concept of a digraph (directed graph):

Unlike undirected graphs, the DFS (and BFS) traversal of a digraph can have forward, back and cross edges!

Digraphs can model prerequisites

Topological Sorting problem: List the graph vertices in such an order that for every edge in the graph, the vertex where the edge starts is listed before the vertex where the edge ends.

Note: It is impossible if a cycle exists!

New concept: DAG = Directed Acyclic Graph

Cycles in digraphsWe have a cycle iff there is a back edge in DFS:

DFS alg. for Topological Sorting

QUIZ: DFS alg. for Topo. Sorting

Consider the graph where the direction of C2 –C4 is reversed. Is it still a DAG?Apply the algorithm, showing all the steps!

QUIZ: DFS alg. for Topo. Sorting

Consider the graph where the direction of C2 –C4 is reversed. Is it still a DAG? Yes – no cycles!Apply the algorithm, showing all the steps!

No edges pointing back!

QUIZ: DFS alg. for Topo. Sorting

QUIZ: DFS alg. for Topo. Sorting

A: No. In the text example, what is wrong with the order C1 C3 C4 C5 C2?

Decrease-and-Conquer for Topo. SortSource Removal Alg.

Break ties in alphabetical order!

Note that the solution is different from that obtained with the DFS algorithm:

QUIZ: Decrease-and-Conquer Topo. Sort

Solution with DFS Solution with source removal

QUIZ: Decrease-and-Conquer Topo. Sort

Hint: Reason by contradiction: What if no vertex is a source?

4.2 Individual work for next time (in notebook!)

End-of-section exercises 1, 5, 6.

4.3 Generating Combinatorial ObjectsNote: For simplicity, we assume that the elements to be permuted are the integers 1 to n. If not …

Decrease-by-one: • Solve P(n-1) first: generate all (n − 1)! Permutations of size n-1. • We can get a solution to P(n) by inserting n in each of the n

possible positions among elements of every permutation of n − 1 elements.

Practice: Use this alg. for n=4.

Minimal change: each permutation can be obtainedfrom its immediate predecessor by exchanging just two elements in it. (Similar to Gray code!)

Tweak: start with inserting n into 12 . . . (n − 1) by moving right to left and then switch direction every time a new permutation of {1, . . . , n − 1} needs to be processed.

Generating only permutations of length n: Johnson-Trotter alg.

Its arrow points to a smaller element

QUIZ: Johnson-Trotter alg.

Show the operation for n = 4.

Note: The order of the permutations in Johnson-Trotter is the same as in the bottom-up with minimal change!

SKIP Lexicographic Permute

Generating subsetsDecrease-by-one: • Solve S(n-1) first: generate all (n − 1)! subsets of size n-1. • We can get a solution to S(n) by inserting or not inserting an in

each of the subsets from S(n-1).

Practice: Use this alg. for n=4.

Subsets as binary numbers

Generating subsets is as easy as counting in binary!

Variation: Squashed order• any subset involving aj can be listed only after all the subsets

involving a1, . . . , aj−1,• Is the order above the squashed order?

• What simple change would produce the squashed order? (End-of-section Ex. 6) Hint: Think of binary numbers!

Subsets as binary numbers

Variation: Minimal change order, a.k.a. Gray code• Every subset differs from its immediate predecessor by either an

addition or a deletion of a single element but not both.• Every binary number differs from its immediate predecessor by

only a single bit.• Is the order above a minimal change/Gray code order?

Subsets in Minimal change order, a.k.a. Gray code

Subsets in Minimal change order, a.k.a. Gray code

This is for Gray code what ____________ is for ________________.

4.3 Individual work for next time (review for midterm!)

End-of-section exercises 1, 2 (no part c), 4, 5, 6(already discussed in today's lecture), 7.

4.4 Decrease by Constant Factor

Binary Search

Is it recursive or iterative?

Worst-case analysis

If n is a power of 2:

If n is general:

See the Smoothness Rule on pp.488-90 in App.B

Average-case analysis

, which is still

Needs probabilistic analysis to prove! (Not covered in our text.)

Practice: Write a recursive version of this algorithm!

Fake Coin

Algorithm: Divide n coins into two piles of floor(n/2) coins each, leaving one extra coin aside if n is odd, and put the two piles on the scale. If the piles weigh the same, the coin put aside must be fake;otherwise, we can proceed in the same manner with the lighter pile, which must be the one with the fake coin.

Among n identical-looking coins, one is fake.With a balance scale, we can compare any two sets of coins: by tipping to the left, to the right, or staying even, the scale will tell whether the sets weigh the same or which of the sets is heavier thanthe other but not by how much. Design an efficient algorithm for detecting the fake coin. (An easier version of the problem, the one we discuss here, assumes that the fake coin is known to be, say, lighter than the genuine one.)

Fake CoinAlgorithm: Divide n coins into two piles of floor(n/2) coins each, leaving one extra coin aside if n is odd, and put the two piles on the scale. If the piles weigh the same, the coin put aside must be fake;otherwise, we can proceed in the same manner with the lighter pile, which must be the one with the fake coin.

Worst-case analysis

Solution:

SKIP from 4.4:• Russian Peasant Multiplication (p.153)• Josephus Problem (p.154)

4.4 Individual work for next time (review for midterm!)

End-of-section exercises 4, 5, 10.Hint for #10:

4.5 Decrease by variable Factor

The size reduction pattern varies from one iteration of the algorithm to another• Example: In Euclid’s alg., the remainder of a/b can be

anywhere in between 0 and b-1.

Selecting the kth order statisticDefinition: The kth order statistic is the kth smallest element of a list of n elements. If k = 1, we have the minimum If k = n, we have the maximum If k = n/2, we have the median. Do not confuse the median with the mean (a.k.a. average)!

Image source: https://keydifferences.com/difference-between-mean-and-median.html

k starts at 1

Selecting the kth order statistic If k = n/2, we have the median.

Why is the ceiling function needed?

Consider two cases: n is odd and n is even!• Note that k starts at 1, so the array index is actually k-1!

Selecting the kth order statistic

Profound idea: Partitioning!The pivot could be chosen in many ways (including randomly!), but here we simply take it to be the first element:

There are two well-known algorithms to produce the partitioning:

Hoare - this was the original alg. used by Tony Hoare in Quicksort (1959) and Quickselect (1961).Lomuto (attributed to Nico Lomuto), simpler to program and to analyze, but less efficient.

LomutoPartitioning

i first index of “unseen” segments first index of “<“ segmentInitially, i = l+1, and s = l, because both “<” and “≥” segments are empty.

Note well: All these three segments can

be empty!

OK, we found the partition – now what?

A: We continue searching on the appropriate side of the pivot!

Array index 4

Now we can stop, as the new pivot is in the desired index 4.

Array index 4

Since we have “tail recursion”, it is easy to convert to an iterative alg. – write it for practice!

Efficiency analysis:• We count the comparisons A[i] < p in LomutoPartition• One call to LomutoPartition always takes exactly r – l

comparisons• In the first call, this is n – 1 This is the best case!

Efficiency analysis:• In the worst case, in each of the n − 1 iterations we have

an extremely unbalanced partition of the current sub-array, with one part being empty and the other containing r – l elements.

Efficiency analysis:• In the average case, it can be proven that the number of

comparisons is linear!

Interpolation SearchArray must be sorted!

Same idea as BSearch, but, instead of going to the middle, we estimate the position, assuming a linear distribution of the values of the array elements.

Instead of the index m from BSearch

we now have

Interpolation Search

Analysis:• Best-case Θ(1)• Worst-case Θ(N), if distribution of array elements is extremely

non-linear, e.g. first element is 0, and all others 42, and search key is 41.5.

• Average case, can be shown to be very good:

Searching and Insertion in a BST

Analysis:• Best-case Θ(1)• Worst-case Θ(n), if distribution of array elements is extremely

non-linear, e.g. first element is 0, and all others 42, and search key is 41.5.

• Average case, can be shown to be logarithmic:

Read FYI The Game of Nim(not required for exam!)

Homework for Ch.4, due Tue, Feb. 20 before exam: End-of-section exercises4.4 3 (p.156)4.5 6 and 7 (p.166).