J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.
-
Upload
brandon-craig -
Category
Documents
-
view
213 -
download
0
Transcript of J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.
![Page 1: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/1.jpg)
COSC 3101N J. Elder
Thinking about Algorithms Abstractly
Lecture 1. Introduction
![Page 2: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/2.jpg)
COSC 3101N J. Elder
So you want to be a computer scientist?
![Page 3: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/3.jpg)
COSC 3101N J. Elder
Is your goal to be a mundane programmer?
![Page 4: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/4.jpg)
COSC 3101N J. Elder
Or a great leader and thinker?
![Page 5: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/5.jpg)
COSC 3101N J. Elder
Original Thinking
![Page 6: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/6.jpg)
COSC 3101N J. Elder
Boss assigns task:
Given today’s prices of pork, grain, sawdust, …
Given constraints on what constitutes a hotdog.
Make the cheapest hotdog.
Everyday industry asks these questions.
![Page 7: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/7.jpg)
COSC 3101N J. Elder
Um? Tell me what to code.
With more suffocated software engineering systems,the demand for mundane programmers will diminish.
Your answer:
![Page 8: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/8.jpg)
COSC 3101N J. Elder
Your answer:
I learned this great algorithm that will work.
Soon all known algorithms will be available in libraries.
![Page 9: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/9.jpg)
COSC 3101N J. Elder
Your answer:
I can develop a new algorithm for you.
Great thinkers will always be needed.
![Page 10: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/10.jpg)
COSC 3101N J. Elder
Content: An up to date grasp of fundamental problems and solutions
Method: Principles and techniques to solve the vast array of unfamiliar problems that arise in a rapidly changing field
The future belongs to the computer scientist who has
Rudich www.discretemath.com
![Page 11: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/11.jpg)
COSC 3101N J. Elder
Course Content
A list of algorithms.
Learn their code.
Trace them until you are convinced that they work.
Implement them. class InsertionSortAlgorithm extends SortAlgorithm {
void sort(int a[]) throws Exception {
for (int i = 1; i < a.length; i++) {
int j = i;
int B = a[i];
while ((j > 0) && (a[j-1] > B)) {
a[j] = a[j-1];
j--; }
a[j] = B;
}}
![Page 12: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/12.jpg)
COSC 3101N J. Elder
Course Content
A survey of algorithmic design techniques.
Abstract thinking.
How to develop new algorithms for any problem that may arise.
![Page 13: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/13.jpg)
COSC 3101N J. Elder
Study:
Many experienced programmers were asked to code up binary search.
![Page 14: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/14.jpg)
COSC 3101N J. Elder
Study:
Many experienced programmers were asked to code up binary search.
80% got it wrong
Good thing is was not for a nuclear power plant.
![Page 15: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/15.jpg)
COSC 3101N J. Elder
What did they lack?
![Page 16: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/16.jpg)
COSC 3101N J. Elder
What did they lack?
Formal proof methods?
![Page 17: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/17.jpg)
COSC 3101N J. Elder
What did they lack?
Formal proof methods?
Yes, likely
Industry is starting to realize that formal methods are important.
But even without formal methods …. ?
![Page 18: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/18.jpg)
COSC 3101N J. Elder
What did they lack?
Fundamental understanding of algorithmic design techniques.
Abstract thinking.
![Page 19: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/19.jpg)
COSC 3101N J. Elder
Course Content
Notations, analogies, and abstractions
for developing,
thinking about,
and describing algorithms
so correctness is transparent
![Page 20: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/20.jpg)
COSC 3101N J. Elder
Slide Credits
Some materials also due to Prof. A. Mirzaian (York).
These in turn include some slides created by Prof. S. Rudich of Carnegie Mellon University.
The slides for these 12 lectures have been adapted from slides created by Prof. J. Edmonds (York).
![Page 21: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/21.jpg)
COSC 3101N J. Elder
Please feel free to ask questions!
![Page 22: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/22.jpg)
COSC 3101N J. Elder
Useful Learning Techniques
![Page 23: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/23.jpg)
COSC 3101N J. Elder
Read Ahead
You are expected to read the textbook and lecture notes before the lecture.
This will facilitate more productive discussion during class.
![Page 24: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/24.jpg)
COSC 3101N J. Elder
Explaining
We are going to test you on your ability to explain the material.
Hence, the best way of studying is to explain the material over and over again out loud to yourself, to each other, and to your stuffed bear.
![Page 25: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/25.jpg)
COSC 3101N J. Elder
Day Dream
Mathematics is not all linear thinking.
Allow the essence of the material to seep into your subconscious
Pursue ideas that percolate up and flashes of inspiration that appear.
While going along with your day
![Page 26: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/26.jpg)
COSC 3101N J. Elder
Be Creative
Ask questions.
Why is it done this way and not that way?
![Page 27: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/27.jpg)
COSC 3101N J. Elder
Guesses and Counter Examples
Guess at potential algorithms for solving a problem.
Look for input instances for which your algorithm gives the wrong answer.
![Page 28: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/28.jpg)
COSC 3101N J. Elder
Refinement:
Rudich www.discretemath.com
The best solution comes from a process of repeatedly refining and inventing alternative solutions
![Page 29: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/29.jpg)
COSC 3101N J. Elder
Grade School Revisited:How To Multiply Two Numbers
2 X 2 = 5
A Few Example A Few Example AlgorithmsAlgorithms
Rudich
![Page 30: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/30.jpg)
COSC 3101N J. Elder
Complex Numbers
Remember how to multiply 2 complex numbers?
(a+bi)(c+di) = [ac –bd] + [ad + bc] i
Input: a,b,c,d Output: ac-bd, ad+bc
If a real multiplication costs $1 and an addition cost a penny. What is the cheapest way to obtain the output from the input?
Can you do better than $4.02?
![Page 31: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/31.jpg)
COSC 3101N J. Elder
Input: a,b,c,d Output: ac-bd, ad+bc
m1 = ac
m2 = bd
A1 = m1 – m2 = ac-bd
m3 = (a+b)(c+d) = ac + ad + bc + bd
A2 = m3 – m1 – m2 = ad+bc
Gauss’ $3.05 Method:
![Page 32: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/32.jpg)
COSC 3101N J. Elder
Question:
The Gauss method saves one multiplication out of four. It requires 25% less work.
Could there be a context where performing 3 multiplications for every 4 provides a more dramatic savings?
![Page 33: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/33.jpg)
COSC 3101N J. Elder
Odette Bonzo
![Page 34: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/34.jpg)
COSC 3101N J. Elder
How to add 2 n-bit numbers.
**
**
**
**
**
**
**
**
**
**
**
+
![Page 35: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/35.jpg)
COSC 3101N J. Elder
How to add 2 n-bit numbers.
**
*
**
**
**
**
**
* **
**
**
**
**
+
![Page 36: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/36.jpg)
COSC 3101N J. Elder
How to add 2 n-bit numbers.
**
*
**
**
**
**
* **
* **
*
**
**
**
**
+
![Page 37: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/37.jpg)
COSC 3101N J. Elder
How to add 2 n-bit numbers.
**
*
**
**
**
* **
* **
*
* **
*
**
**
**
**
+
![Page 38: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/38.jpg)
COSC 3101N J. Elder
How to add 2 n-bit numbers.
**
*
**
**
* **
* **
*
* **
*
* **
*
**
**
**
**
+
![Page 39: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/39.jpg)
COSC 3101N J. Elder
How to add 2 n-bit numbers.
**
*
* **
*
* **
*
* **
*
* **
*
* **
*
* **
*
* **
*
* **
*
* **
*
***
*
+*
*
![Page 40: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/40.jpg)
COSC 3101N J. Elder
Time complexity of grade school addition
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
*
**
*
+*
*
*
**
*
T(n) = The amount of time grade school addition uses to add two n-bit numbers
= θ(n) = linear time.
On any reasonable computer adding 3 bits can be done in constant time.
![Page 41: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/41.jpg)
COSC 3101N J. Elder
# of bits in numbers
tim
ef = θ(n) means that f can be
sandwiched between two lines
![Page 42: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/42.jpg)
COSC 3101N J. Elder
Is there a faster way to add?
QUESTION: Is there an algorithm to add two n-bit numbers whose time grows sub-linearly in n?
![Page 43: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/43.jpg)
COSC 3101N J. Elder
Any algorithm for addition must read all of the input bits
Suppose there is a mystery algorithm that does not examine each bit
Give the algorithm a pair of numbers. There must be some unexamined bit position i in one of the numbers
If the algorithm is not correct on the numbers, we found a bug
If the algorithm is correct, flip the bit at position i and give the algorithm the new pair of numbers. It give the same answer as before so it must be wrong since the sum has changed
![Page 44: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/44.jpg)
COSC 3101N J. Elder
So any algorithm for addition must use time at least linear in
the size of the numbers.
Grade school addition is essentially as good as it can be.
![Page 45: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/45.jpg)
COSC 3101N J. Elder
How to multiply 2 n-bit numbers.
X* * * * * * * *
* * * * * * * *
* * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * *
* * * * * * * * * * * * * * * * *
n2
![Page 46: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/46.jpg)
COSC 3101N J. Elder
How to multiply 2 nHow to multiply 2 n--bit numbers.bit numbers.
X* * * * * * * * * * * * * * * *
* * * * * * * ** * * * * * * *
* * * * * * * ** * * * * * * *
* * * * * * * ** * * * * * * *
* * * * * * * ** * * * * * * *
* * * * * * * * * * * * * * * *
n2
I get it! The total time is bounded by
cn2.
![Page 47: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/47.jpg)
COSC 3101N J. Elder
Grade School Addition: Linear timeGrade School Multiplication: Quadratic time
No matter how dramatic the difference in the constants the quadratic curve will eventually dominate the linear curve
# of bits in numbers
tim
e
![Page 48: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/48.jpg)
COSC 3101N J. Elder
Neat! We have demonstrated that as things scale multiplication is a
harder problem than addition.
Mathematical confirmation of our common sense.
![Page 49: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/49.jpg)
COSC 3101N J. Elder
Don’t jump to conclusions!We have argued that grade school multiplication uses more time than
grade school addition. This is a comparison of the complexity of two
algorithms.
To argue that multiplication is an inherently harder problem than
addition we would have to show that no possible multiplication algorithm runs in
linear time.
![Page 50: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/50.jpg)
COSC 3101N J. Elder
Grade School Addition: θ(n) timeGrade School Multiplication: θ(n2) time
Is there a clever algorithm to multiply two numbers
in linear time?
![Page 51: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/51.jpg)
COSC 3101N J. Elder
Despite years of research, no one knows! If you resolve this question,
York will give you a PhD!
![Page 52: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/52.jpg)
COSC 3101N J. Elder
Is there a faster way to multiply two numbers
than the way you learned in grade school?
![Page 53: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/53.jpg)
COSC 3101N J. Elder
Divide And Conquer (an approach to faster algorithms)
DIVIDE a problem into smaller subproblems
CONQUER them recursively
GLUE the answers together so as to obtain the answer to the larger problem
![Page 54: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/54.jpg)
COSC 3101N J. Elder
Multiplication of 2 n-bit numbers
X =
Y =
X = a 2n/2 + b Y = c 2n/2 + d
XY = ac 2n + (ad+bc) 2n/2 + bd
a b
c d
![Page 55: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/55.jpg)
COSC 3101N J. Elder
Multiplication of 2 n-bit numbers
X =
Y =
XY = ac 2n + (ad+bc) 2n/2 + bd
a b
c d
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
RETURN
MULT(a,c) 2n + (MULT(a,d) + MULT(b,c)) 2n/2 + MULT(b,d)
![Page 56: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/56.jpg)
COSC 3101N J. Elder
Time required by MULT
T(n) = time taken by MULT on two n-bit numbers
What is T(n)? What is its growth rate? Is it θ(n2)?
![Page 57: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/57.jpg)
COSC 3101N J. Elder
Recurrence Relation
T(1) = k for some constant k
T(n) = 4 T(n/2) + k’ n + k’’ for some constants k’ and k’’
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
RETURN
MULT(a,c) 2n + (MULT(a,d) + MULT(b,c)) 2n/2 + MULT(b,d)
![Page 58: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/58.jpg)
COSC 3101N J. Elder
Let’s be concrete
T(1) = 1
T(n) = 4 T(n/2) + n
How do we unravel T(n) so that we can determine its growth rate?
![Page 59: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/59.jpg)
COSC 3101N J. Elder
Technique 1: (Substitution)
Recurrence Relation:
T(1) = 1 & T(n) = 4T(n/2) + n
Guess: G(n) = 2n2 – n
Verify: Left Hand Side Right Hand Side
T(1) = 2(1)2 – 1
T(n) = 2n2 – n
1
4T(n/2) + n
= 4 [2(n/2)2 – (n/2)] + n
= 2n2 – n
![Page 60: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/60.jpg)
COSC 3101N J. Elder
Technique 2: Recursion Tree
• T(n) = n + 4 T(n/2)
•T(1) = 1
n
T(n/2) T(n/2) T(n/2) T(n/2)
= n
T(n/2) T(n/2) T(n/2) T(n/2)
=T(n)T(n)
T(1) 1=
![Page 61: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/61.jpg)
COSC 3101N J. Elder
n
T(n/2) T(n/2) T(n/2) T(n/2)
T(n) =
![Page 62: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/62.jpg)
COSC 3101N J. Elder
n
T(n/2) T(n/2) T(n/2)
T(n) =
n/2
T(n/4)T(n/4)T(n/4)T(n/4)
![Page 63: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/63.jpg)
COSC 3101N J. Elder
nT(n) =
n/2
T(n/4)T(n/4)T(n/4)T(n/4)
n/2
T(n/4)T(n/4)T(n/4)T(n/4)
n/2
T(n/4)T(n/4)T(n/4)T(n/4)
n/2
T(n/4)T(n/4)T(n/4)T(n/4)
![Page 64: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/64.jpg)
COSC 3101N J. Elder
nT(n) =
n/2 n/2 n/2n/2
11111111111111111111111111111111 . . . . . . 111111111111111111111111111111111
n/4 n/4 n/4n/4n/4n/4n/4n/4n/4n/4n/4n/4n/4n/4n/4 n/4
![Page 65: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/65.jpg)
COSC 3101N J. Elder
Level i is the sum of 4i copies of n/2i
1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1
. . . . . . . . . . . . . . . . . . . . . . . . . .
n/2 + n/2 + n/2 + n/2
n
n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4
0
1
2
i
log n2
![Page 66: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/66.jpg)
COSC 3101N J. Elder
Level i is the sum of 4 i copies of n/ 2 i
1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1
. . . . . . . . . . . . . . . . . . . . . . . . . .
n/ 2 + n/ 2 + n/ 2 + n/ 2
n
n/ 4 + n/ 4 + n/ 4 + n/ 4 + n/ 4 + n/ 4 + n/ 4 + n/ 4 + n/ 4 + n/ 4 + n/ 4 + n/ 4 + n/ 4 + n/ 4 + n/ 4 + n/ 4
0
1
2
i
lo g n2
=1n
= 4n/2
= 16n/4
= 4i n/2i
Total: θ(nlog4) = θ(n2)
= 4lognn/2logn
=nlog41
![Page 67: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/67.jpg)
COSC 3101N J. Elder
Divide and Conquer MULT: θ(n2) time Grade School Multiplication: θ(n2) time
All that work for nothing!
![Page 68: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/68.jpg)
COSC 3101N J. Elder
MULT revisited
MULT calls itself 4 times. Can you see a way to reduce the number of calls?
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
RETURN
MULT(a,c) 2n + (MULT(a,d) + MULT(b,c)) 2n/2 + MULT(b,d)
![Page 69: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/69.jpg)
COSC 3101N J. Elder
Gauss’ Idea: Input: a,b,c,d Output: ac, ad+bc, bd
A1 = ac
A3 = bd
m3 = (a+b)(c+d) = ac + ad + bc + bd
A2 = m3 – A1- A3 = ad + bc
![Page 70: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/70.jpg)
COSC 3101N J. Elder
Gaussified MULT (Karatsuba 1962)
T(n) = 3 T(n/2) + n
Actually: T(n) = 2 T(n/2) + T(n/2 + 1) + kn
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN e2n + (MULT(a+b, c+d) – e - f) 2n/2 + f
![Page 71: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/71.jpg)
COSC 3101N J. Elder
nT(n) =
n/2
T(n/4)T(n/4)T(n/4)T(n/4)
n/2
T(n/4)T(n/4)T(n/4)T(n/4)
n/2
T(n/4)T(n/4)T(n/4)T(n/4)
n/2
T(n/4)T(n/4)T(n/4)T(n/4)
![Page 72: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/72.jpg)
COSC 3101N J. Elder
n
T(n/2) T(n/2) T(n/2)
T(n) =
![Page 73: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/73.jpg)
COSC 3101N J. Elder
n
T(n/2) T(n/2)
T(n) =
n/2
T(n/4)T(n/4)T(n/4)
![Page 74: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/74.jpg)
COSC 3101N J. Elder
nT(n) =
n/2
T(n/4)T(n/4)T(n/4)
n/2
T(n/4)T(n/4)T(n/4)
n/2
T(n/4)T(n/4)T(n/4)
![Page 75: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/75.jpg)
COSC 3101N J. Elder
Level i is the sum of 3i copies of n/2i
1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1
. . . . . . . . . . . . . . . . . . . . . . . . . .
n/2 + n/2 + n/2
n
n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4
0
1
2
i
log n2
![Page 76: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/76.jpg)
COSC 3101N J. Elder
=1n
= 3n/2
= 9n/4
= 3i n/2i
Total: θ(nlog3) = θ(n1.58..)
Level i is the sum of 3 i copies of n/ 2 i
1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1
. . . . . . . . . . . . . . . . . . . . . . . . . .
n/ 2 + n/ 2 + n/ 2
n
n/ 4 + n/ 4 + n/ 4 + n/ 4 + n/ 4 + n/ 4 + n/ 4 + n/ 4 + n/ 4
0
1
2
i
lo g n2 = 3lognn/2logn
=nlog31
![Page 77: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/77.jpg)
COSC 3101N J. Elder
Dramatic improvement for large n
Not just a 25% savings!
θ(n2) vs θ(n1.58..)
![Page 78: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/78.jpg)
COSC 3101N J. Elder
Multiplication Algorithms
Kindergarten n2n
Grade School n2
Karatsuba n1.58…
Fastest Known n logn loglogn
![Page 79: J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 1. Introduction.](https://reader037.fdocuments.us/reader037/viewer/2022110321/56649f425503460f94c61aba/html5/thumbnails/79.jpg)
COSC 3101N J. Elder
End