RULES-6: a simple rule induction algorithm for handling large data
Algorithm Design Techniques: Induction
-
Upload
dana-collins -
Category
Documents
-
view
50 -
download
1
description
Transcript of Algorithm Design Techniques: Induction
![Page 1: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/1.jpg)
Algorithm Design Techniques: Induction
• Chapter 5 (Except Sections 5.6 and 5.7)
![Page 2: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/2.jpg)
Induction
• Main objective: Using induction or inductive reasoning as a “recursive” algorithm design technique
• Why recursion– Concise algorithms for complex problems can be
developed– The algorithms are easy to comprehend– Development time– Proof of correctness of the designed algorithm is
usually simple.
![Page 3: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/3.jpg)
How Does Induction Produce Recursive Algorithms?
• Having a problem with input size n, it is sometimes easier to start with a solution to the problem with a smaller size and extend the solution to include the input size n.
![Page 4: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/4.jpg)
Problems to Discuss
• Selection sort
• Insertion sort
• Radix sort
• Integer exponentiation
• Evaluating polynomials
![Page 5: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/5.jpg)
Recursive Selection Sort
• Induction Hypothesis: We know how to sort A[2..n]
• Inductive Reasoning: We sort A[1..n] as follows– Find the minimum A[j], 1 j n– Swap(A[1],A[j])– Sort A[2..n] // induction hypothesis
• What is the base case?• What is the initial call to the sorting algorithm?• This kind of recursion is called tail-recursion.
![Page 6: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/6.jpg)
Recursive Selection Sort AlgorithmAlgorithm SELECTIONSORTREC
Input: An array A[1..n] of n elements.
Output: A[1..n] sorted in nondecreasing order.Procedure sort(i) {Sort A[i..n]}
1. if i < n then
Begin
2. k := i;
3. for j := i + 1 to n
4. if A[j] < A[k] then k := j;
5. if k != i then Swap(A[i], A[k]);
6. sort(i + 1);
End
![Page 7: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/7.jpg)
Complexity Analysis of Recursive Selection Sort
• What is the recurrence relation that describes the number of comparisons carried out by the algorithm?
• What is the solution to the recurrence?
![Page 8: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/8.jpg)
Recursive Insertion Sort
• Induction Hypothesis: We know how to sort A[1..n-1]
• Inductive Reasoning: We sort A[1..n] as follows:– Sort A[1..n-1] // induction hypothesis– Insert A[n] in its proper position in A[1..n]
• This may involve copying zero or more elements one position ahead in order to insert A[n]
![Page 9: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/9.jpg)
Recursive Insertion Sort AlgorithmAlgorithm INSERTIONSORTRECInput: An array A[1..n] of n elements.Output: A[1..n] sorted in non-decreasing order.
Procedure sort(i) {Sort A[1..i]}1. if i >1 then2. sort(i – 1) {Recursive Call}3. x := A[i]4. j := i – 15. while j > 0 and A[j] > x6. A[j + 1] := A[j]7. j := j – 1 end while8. A[j + 1] := x end if
![Page 10: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/10.jpg)
Complexity Analysis of Recursive Insertion Sort
• Unlike selection sort, the analysis has to differentiate between the best case and the worst case. Why?
• Recurrence of the best case:
• Solution to the best case:
• Recurrence of the worst case:
• Solution to the worst case:
![Page 11: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/11.jpg)
Radix Sort
• Treats keys as numbers in a particular radix or base.
• This is NOT an elements comparison-based sorting algorithm
• It only works under the following assumption:
![Page 12: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/12.jpg)
Radix Sort Derivation
• Assume that our keys are of the form
dk dk – 1 … d1
• Induction Hypothesis: Suppose that we know how to sort numbers lexicographically according to their least k – 1 digits, dk – 1, dk – 2, …, d1 , k > 1.
• Inductive reasoning: We sort the numbers based on their first k digits as follows:– Use induction hypothesis to sort the numbers based on their
1..k-1 digits.
– Sort the numbers based on their corresponding kth digits
![Page 13: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/13.jpg)
Description of Radix Sort Algorithm
1. Distribute the input numbers into 10 sublists L0, L1,…, L9 according to the least significant digit, d1.
2. Form a new list, we denote by main list, by removing the input from the 10 lists, starting from L0, L1, …etc. in order.
3. Distribute the main list into the 10 sublists according to the second digit, d2, and then repeat step 2.
4. Repeat step 3 for the rest of the digits in order of d3, d4, …, dk.
![Page 14: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/14.jpg)
Example
• Sort the following numbers using radix sort:
467 1247 3275 6792 9187
9134 4675 39 7 6644
![Page 15: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/15.jpg)
Radix Sort Algorithm
Algorithm RADIXSORTInput: A linked list of numbers L = {a1, a2,…,an} and k, the maximum number of
digits.Output: L sorted in nondecreasing order.
1. for j ←1 to k do2. Prepare 10 empty lists L0, L1, …, L9.3. while L is not empty4. a ← next element in L. Delete a from L.5. i ← jth digit in a. Append a to list Li.6. end while7. L ← L0
8. for i ←1 to 9 do9. L ← L . Li {append list Li to L}10. end for11. end for12. return L
![Page 16: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/16.jpg)
Time and Space Complexity Analysis
• Time Complexity
• Space Complexity
![Page 17: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/17.jpg)
Integer Exponentiation
• What is the straightforward algorithm to raise x to the power n?
• Such an algorithm is exponential in input size!!!!!!!!
![Page 18: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/18.jpg)
Algorithm Derivation
• Assume we have integer n which is represented in binary as (bkbk-1…b1b0)2
• Induction Hypothesis: Assume that we know how to compute xn/2.
• Inductive Reasoning: We raise x to the power n as follows:– Use induction hypothesis to compute
y=xn/2 . xn/2
– If n is even, xn = y
– If n is odd, xn = x . y
![Page 19: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/19.jpg)
Integer Exponentiation AlgorithmALGORITHM EXPRECURSIVE
Input: A real number x and a nonnegative integer n.
Output: xn. /* the first call to the algorithm is power(x,n) */
Procedure power(x, m) {Compute xm }
1. if m=0 then y ← 1
2. else
3. y ← power(x,m/2)4. y ← y2
5. if m is odd then y ← x . y
end if
7. return y
![Page 20: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/20.jpg)
Complexity Analysis of the Algorithm
• When n = 0, the number of multiplications is Therefore, T(0)=
• Best Case Analysis:– When does the best case occur?– What is the recurrence equation? Solution?
• Worst Case Analysis:– When does the worst case occur?– What is the recurrence equation? Solution?
![Page 21: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/21.jpg)
Iterative Exponentiation Algorithm
• Assume we have integer n which is represented in binary as (bkbk-1…b1b0)2
Starting with y = 1, scan the binary digits of the number n from left to right (j = k down to 0):– If bj = 0, square y
– If bj = 1, square y and multiply it by x
• Example: Compute 212 using the iterative exponentiation algorithm
![Page 22: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/22.jpg)
Polynomial Evaluation
• A polynomial of degree n is generally written as:
where, a0, a1, …, an and x, is a sequence of n + 2 real numbers. Our objective here is to evaluate this general polynomial at the point x.
011
1 ...)( axaxaxaxP nn
nnn
![Page 23: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/23.jpg)
Two Straightforward Solutions
1. Evaluate each term aixi separately• What is its time complexity?
2. Compute xi by multiplying x by the previously computed value xi-1
• How many multiplications, assignments, and additions do we have?
1. ;0aP
2. ;1t 3. do to1for ni 4. xtt *
5. taPP i *
![Page 24: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/24.jpg)
Horner’s Rule
01
01321
011
1
)(
))...)))((...(((
...)(
axxP
axaxaxaxaxa
axaxaxaxP
n
nnnn
nn
nnn
![Page 25: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/25.jpg)
Horner’s Rule• Induction Hypothesis: Suppose we know how to
evaluate the following:
• Inductive Reasoning: We evaluate Pn(x) as follows:– Using induction hypothesis, we know how to
evaluate Pn-1(x)
– Pn(x) = x Pn-1(x) + a0
• What is the base step?
122
11
1 ...)( axaxaxaxP nn
nnn
![Page 26: Algorithm Design Techniques: Induction](https://reader036.fdocuments.us/reader036/viewer/2022062301/568137c0550346895d9f5d4a/html5/thumbnails/26.jpg)
Horner’s Algorithm
• Horner’s Algorithm
Input: A sequence of n+2 real numbers a0, a1, …, an, x
Output: Pn(x)=anxn+an-1xn-1+…+a1x+a0
p ← an
for j ← 1 to n do
p ← x . p + an-j
return p
• How many multiplications, additions, and assignments do we have?