Transform and ConquerTransform and Conquer

Chapter 6Chapter 6

Transform and ConquerTransform and Conquer

Solve problem by transforming into:Solve problem by transforming into:

a more convenient instance of the same problem a more convenient instance of the same problem ((instance simplificationinstance simplification))– presortingpresorting– Gaussian eliminationGaussian elimination

Transform and ConquerTransform and Conquer

Solve problem by transforming into:Solve problem by transforming into:

a different representation of the same instance a different representation of the same instance ((representation changerepresentation change))– balanced search treesbalanced search trees– heaps and heapsortheaps and heapsort– polynomial evaluation by Horner’s rulepolynomial evaluation by Horner’s rule– Fast Fourier TransformFast Fourier Transform

Transform and ConquerTransform and Conquer

Solve problem by transforming into:Solve problem by transforming into:

a different problem altogether (a different problem altogether (problem problem reductionreduction))– reductions to graph problemsreductions to graph problems– linear programming linear programming

Instance simplification - Instance simplification - PresortingPresorting

Solve instance of problem by transforming Solve instance of problem by transforming into another simpler/easier instance of the into another simpler/easier instance of the same problemsame problem

Instance simplification - Instance simplification - PresortingPresorting

Presorting:Presorting:Many problems involving lists are easier Many problems involving lists are easier

when list is sorted.when list is sorted.

searching searching

computing the median (selection problem)computing the median (selection problem)

computing the modecomputing the mode

finding repeated elementsfinding repeated elements

Selection ProblemSelection Problem

Find the kFind the kth th smallest element in A[1],…A[smallest element in A[1],…A[nn]. ]. Special cases:Special cases:– minimumminimum: k : k = 1= 1– maximummaximum: k : k = = nn– medianmedian: k : k = = nn/2/2

Presorting-based algorithm Presorting-based algorithm – sort listsort list– return A[return A[kk]]

Selection ProblemSelection Problem

Partition-based algorithm (Variable Partition-based algorithm (Variable decrease & conquer):decrease & conquer):– pivot/split at A[pivot/split at A[ss] using partitioning algorithm ] using partitioning algorithm

from quicksortfrom quicksort– if if ss==k k return A[ return A[ss] ] – else if else if s<k s<k repeat with sublist A[repeat with sublist A[ss+1],…A[+1],…A[nn]. ]. – else if s>k else if s>k repeat with sublist A[1],…A[repeat with sublist A[1],…A[ss-1]. -1].

ΘΘ(n) average case (n) average case

Notes on Selection ProblemNotes on Selection Problem

Presorting-based algorithm: Presorting-based algorithm: ΘΘ((n n lglgnn) + ) + ΘΘ(1) = (1) = ΘΘ((n n lglgnn) )

Partition-based algorithm: Partition-based algorithm: ΘΘ(n)(n)

Conclusion:Conclusion: Pre-sorting does NOT yield an Pre-sorting does NOT yield an advantageadvantage

Finding repeated elementsFinding repeated elements

Presorting-based algorithm: Presorting-based algorithm: – use mergesort (or quicksort): use mergesort (or quicksort): ΘΘ((n n lglgnn) ) – scan array to find repeated scan array to find repeated adjacentadjacent

elements: elements: ΘΘ((nn) )

Brute force algorithm: Brute force algorithm: ΘΘ((nn22) )

Conclusion:Conclusion: Presorting yields Presorting yields significantsignificant improvementimprovement

Taxonomy of Searching Taxonomy of Searching AlgorithmsAlgorithms

Elementary searching algorithmsElementary searching algorithms– sequential searchsequential search– binary searchbinary search– binary tree searchbinary tree search

Taxonomy of Searching Taxonomy of Searching AlgorithmsAlgorithms

Balanced tree searchingBalanced tree searching– AVL treesAVL trees– red-black treesred-black trees– multiway balanced trees (2-3 trees, 2-3-4 multiway balanced trees (2-3 trees, 2-3-4

trees, B trees)trees, B trees)

Taxonomy of Searching Taxonomy of Searching AlgorithmsAlgorithms

HashingHashing– separate chainingseparate chaining– open addressingopen addressing

Balanced trees: AVL treesBalanced trees: AVL trees

For every node, difference in height For every node, difference in height between left and right sub-tree is at most between left and right sub-tree is at most 11AVL property is maintained through AVL property is maintained through rotationsrotations, each time the tree becomes , each time the tree becomes unbalancedunbalancedlg lg nn ≤≤ hh ≤ 1.4404 lg (≤ 1.4404 lg (n n + 2) - 1.3277 + 2) - 1.3277 average:average: 1.01 lg 1.01 lg n + n + 0.1 0.1 for large for large nn

Balanced trees: AVL treesBalanced trees: AVL trees

Disadvantage: needs extra storage for Disadvantage: needs extra storage for maintaining node balancemaintaining node balanceA similar idea: red-black trees (height of A similar idea: red-black trees (height of subtrees is allowed to differ by up to a subtrees is allowed to differ by up to a factor of 2)factor of 2)

General case: single R-rotationGeneral case: single R-rotation

Double LR-rotationDouble LR-rotation

Balance factorBalance factor

Algorithm maintains Algorithm maintains balance factorbalance factor for for each node. For example:each node. For example:

HeapsHeaps

Definition:Definition:A A heapheap is a binary tree with the following is a binary tree with the following

conditions:conditions:

it is essentially complete:it is essentially complete:

The key at each node is The key at each node is ≥ keys at its ≥ keys at its childrenchildren

Definition implies:Definition implies:

Given Given n,n, there exists a unique binary tree there exists a unique binary tree with with nn nodes that nodes that

is essentially complete, with is essentially complete, with hh= lg = lg nn

The root has the largest keyThe root has the largest key

The subtree rooted at any node of a heap The subtree rooted at any node of a heap is also a heapis also a heap

Heap constructionHeap construction

Insert elements in the order given Insert elements in the order given breadth-first in a binary treebreadth-first in a binary tree

Starting with the last (rightmost) parental Starting with the last (rightmost) parental node, fix the heap rooted at it, if it does node, fix the heap rooted at it, if it does not satisfy the heap condition:not satisfy the heap condition:1.1. exchange it with its largest childexchange it with its largest child

2.2. fix the subtree rooted at it (now in the child’s fix the subtree rooted at it (now in the child’s position)position)

Root deletionRoot deletion

The root of a heap can be deleted and the heap The root of a heap can be deleted and the heap fixed up as follows:fixed up as follows:

exchange the root with the last leaf exchange the root with the last leaf

compare the new root (formerly the leaf) with each compare the new root (formerly the leaf) with each of its children and, if one of them is larger than the of its children and, if one of them is larger than the root, exchange it with the larger of the two. root, exchange it with the larger of the two.

continue the comparison/exchange with the continue the comparison/exchange with the children of the new root until it reaches a level of children of the new root until it reaches a level of the tree where it is larger than both its children the tree where it is larger than both its children

RepresentationRepresentation

Use an array to store breadth-first Use an array to store breadth-first traversal of heap tree:traversal of heap tree:

Example:Example:

Left child of node Left child of node jj is at 2 is at 2jj

Right child of node Right child of node jj is at 2 is at 2jj+1+1

Parent of node Parent of node jj is at is at j j /2/2

9

1

5 3

4 2

1 2 3 4 5 6

9 5 3 1 4 2

Priority queuesPriority queues

A A priority queue priority queue is the ADT of an ordered is the ADT of an ordered set with the operations:set with the operations:– find element with highest priority find element with highest priority

– delete element with highest priority delete element with highest priority – insert element with assigned priority insert element with assigned priority

Heaps are very good for implementing Heaps are very good for implementing priority queuespriority queues

Insertion of a new elementInsertion of a new element

Insert element at last position in heap. Insert element at last position in heap.

Compare with its parent and if it violates Compare with its parent and if it violates heap conditionheap condition

exchange themexchange them

Continue comparing the new element with Continue comparing the new element with nodes up the tree until the heap condition nodes up the tree until the heap condition is satisfiedis satisfied

Bottom-up vs. Top-down heap Bottom-up vs. Top-down heap constructionconstruction

Top down:Top down: Heaps can be constructed by Heaps can be constructed by successively inserting elements into an successively inserting elements into an (initially) empty heap(initially) empty heap

Bottom-up:Bottom-up: Put everything in and then fix it Put everything in and then fix it

Which one is better?Which one is better?