CSCE 310J: Data Structures & Algorithms
-
Upload
dorian-garrison -
Category
Documents
-
view
60 -
download
2
description
Transcript of CSCE 310J: Data Structures & Algorithms
http://www.cse.unl.edu/~goddard/Courses/CSCE310J
Fundamentals of Algorithm Analysis
Dr. Steve [email protected]
CSCE 310J: Data Structures & Algorithms
Most of the slides in this lecture were originally created by
Jeff Edmonds
from York University
Relevant Mathematics
Jeff Edmonds
York University
Classifying FunctionsThe Time Complexity of an AlgorithmAdding Made EasyRecurrence Relations
Start With Some Math
Input Size
Tim
eClassifying Functions
f(i) = n(n)
Recurrence Relations
T(n) = a T(n/b) + f(n)
Adding Made Easy∑i=1 f(i).
Time Complexityt(n) = (n2)
Assumed Knowledge
• Logarithms and Exponentials
• Existential and Universal Quantifiers, g b Loves(b,g)
b g Loves(b,g)
Quantifiers: all, some
• “for all x” x P(x) is true iff P(x) is true for all x– universal quantifier (universe of discourse)
• “there exist x” x P(x) is true iff P(x) is true for some value of x– existential quantifier
x A(x) is logically equivalent to x(A(x)) x A(x) is logically equivalent to x(A(x)) x (A(x) B(x))
“For all x such that if A(x) holds then B(x) holds”
Classifying Functions
Giving an idea of how fast a function grows without going into too much detail.
Which are more alike?
Which are more alike?
Mammals
Which are more alike?
Which are more alike?
Dogs
Classifying Animals
Vertebrates
Birds
Mam
mals
Reptiles
Fish
Dogs
Giraffe
Classifying Functions
T(n) 10 100 1,000 10,000
log n 3 6 9 13 amoeba
n1/2 3 10 31 100 bird
10 100 1,000 10,000 human
n log n 30 600 9,000 130,000 my father
n2 100 10,000 106 108 elephantn3 1,000 106 109 1012 dinosaur2n 1,024 1030 10300 103000 the universe
Note: The universe contains approximately 1050 particles.
n
Which are more alike?
n1000 n2 2n
Which are more alike?
Polynomials
n1000 n2 2n
Which are more alike?
1000n2 3n2 2n3
Which are more alike?
Quadratic
1000n2 3n2 2n3
Classifying Functions?
Functions
Classifying Functions
Functions
Constant
Logarithm
ic
Poly L
ogarithmic
Polynom
ial
Exponential
Exp
Double E
xp
(log n)5 n5 25n5 log n5 2n5 25n
2
Classifying Functions?
Polynomial
Classifying Functions
Polynomial
Linear
Quadratic
Cubic
?
5n25n 5n3 5n4
Logarithmic
• log10n = # digits to write n
• log2n = # bits to write n = 3.32 log10n
• log(n1000) = 1000 log(n)
Differ only by a multiplicative constant.
Poly Logarithmic
(log n)5 = log5 n
Logarithmic << Polynomial
For sufficiently large n
log1000 n << n0.001
Linear << Quadratic
For sufficiently large n
10000 n << 0.0001 n2
Polynomial << Exponential
For sufficiently large n
n1000 << 20.001 n
Ordering Functions
Functions
Constant
Logarithm
ic
Poly L
ogarithmic
Polynom
ial
Exponential
Exp
Double E
xp
(log n)5 n5 25n5 log n5 2n5 25n
2
<< << << << << <<
<< << << << << <<
Which Functions are Constant?
• 5• 1,000,000,000,000• 0.0000000000001• -5• 0• 8 + sin(n)
YesYesYes
No
YesYes
Which Functions are “Constant”?
• 5• 1,000,000,000,000• 0.0000000000001• -5• 0• 8 + sin(n)
YesYesYesNo
NoYes
Lie in between
7
9
The running time of the algorithm is a “Constant” It does not depend significantly
on the size of the input.
Which Functions are Quadratic?
• n2
• … ?
Which Functions are Quadratic?
• n2
• 0.001 n2
• 1000 n2
Some constant times n2.
Which Functions are Quadratic?
• n2
• 0.001 n2
• 1000 n2
• 5n2 + 3n + 2log n
Which Functions are Quadratic?
• n2
• 0.001 n2
• 1000 n2
• 5n2 + 3n + 2log n
Lie in between
Which Functions are Quadratic?
• n2
• 0.001 n2
• 1000 n2
• 5n2 + 3n + 2log n
Ignore low-order termsIgnore multiplicative constants.
Ignore "small" values of n. Write θ(n2).
Which Functions are Polynomial?
• n5
• … ?
Which Functions are Polynomial?
• nc
• n0.0001
• n10000
n to some constant power.
Which Functions are Polynomial?
• nc
• n0.0001
• n10000
• 5n2 + 8n + 2log n • 5n2 log n • 5n2.5
Which Functions are Polynomial?
• nc
• n0.0001
• n10000
• 5n2 + 8n + 2log n • 5n2 log n • 5n2.5
Lie in between
Which Functions are Polynomials?• nc
• n0.0001
• n10000
• 5n2 + 8n + 2log n • 5n2 log n • 5n2.5
Ignore low-order termsIgnore power constant.
Ignore "small" values of n. Write nθ(1)
Which Functions are Exponential?
• 2n
• … ?
Which Functions are Exponential?
• 2n
• 20.0001 n
• 210000 n
n times some constant powerraises to the power of 2.
Which Functions are Exponential?
• 2n
• 20.0001 n
• 210000 n
• 8n
• 2n / n100 •2n · n100
too small?too big?
Which Functions are Exponential?
• 2n
• 20.0001 n
• 210000 n
• 8n
• 2n / n100 •2n · n100
= 23n
> 20.5n
< 22n
Lie in between
Which Functions are Exponential?
• 2n
• 20.0001 n
• 210000 n
• 8n
• 2n / n100 •2n · n100
= 23n
> 20.5n
< 22n
20.5n > n100
2n = 20.5n · 20.5n > n100 · 20.5n
2n / n100 > n0.5n
Which Functions are Exponential?
Ignore low-order termsIgnore base.
Ignore "small" values of n. Ignore polynomial factors.
Write 2θ(n)
• 2n
• 20.0001 n
• 210000 n
• 8n
• 2n / n100 •2n · n100
Classifying Functions
Functions
Constant
Logarithm
ic
Poly L
ogarithmic
Polynom
ial
Exponential
Exp
Double E
xp
(log n)θ(1) nθ(1) 2θ(n)θ(log n)θ(1) 2nθ(1) 2θ(n)
2
Notations
Theta f(n) = θ(g(n)) f(n) ≈ c g(n)
BigOh f(n) = O(g(n)) f(n) ≤ c g(n)
Omega f(n) = Ω(g(n)) f(n) ≥ c g(n)
Little Oh f(n) = o(g(n)) f(n) << c g(n)
Little Omega f(n) = ω(g(n)) f(n) >> c g(n)
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
f(n) = θ(g(n))
Definition of Theta
f(n) is sandwiched between c1g(n) and c2g(n)
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
f(n) = θ(g(n))
Definition of Theta
f(n) is sandwiched between c1g(n) and c2g(n)
for some sufficiently small c1 (= 0.0001)
for some sufficiently large c2 (= 1000)
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
f(n) = θ(g(n))
Definition of Theta
For all sufficiently large n
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
f(n) = θ(g(n))
Definition of Theta
For all sufficiently large n
For some definition of “sufficiently large”
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
f(n) = θ(g(n))
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
c1·n2 3n2 + 7n + 8 c2·n2? ?
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·n2 3n2 + 7n + 8 4·n23 4
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·12 3·12 + 7·1 + 8 4·121False
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·72 3·72 + 7·7 + 8 4·727False
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·82 3·82 + 7·8 + 8 4·828True
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·92 3·92 + 7·9 + 8 4·929True
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·102 3·102 + 7·10 + 8 4·10210True
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·n2 3n2 + 7n + 8 4·n2?
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·n2 3n2 + 7n + 8 4·n2n 88
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·n2 3n2 + 7n + 8 4·n2
Truen 8
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·n2 3n2 + 7n + 8 4·n2
7n + 8 1·n2
7 + 8/n 1·n
n 8
True
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n2)
3·n2 3n2 + 7n + 8 4·n23 4 8 n 8
True
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
n2 = θ(n3)
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
n2 = θ(n3)
c1 · n3 n2 c2 · n3? ?
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
n2 = θ(n3)
0 · n3 n2 c2 · n30True, but ?
Definition of Theta
c c n n n1 0 2 0 0 0, , , , . . .
n2 = θ(n3)
0 Constants c1 and c2 must be positive!
False
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
c1·n 3n2 + 7n + 8 c2·n? ?
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3·n 3n2 + 7n + 8 100·n3 100
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3·n 3n2 + 7n + 8 100·n?
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3·100 3·1002 + 7·100 + 8 100·100100
False
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3·n 3n2 + 7n + 8 10,000·n3 10,000
Definition of Theta
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3·10,000 3·10,000 2 + 7·10,000 + 8 100·10,00010,000
False
Definition of Theta
3n2 + 7n + 8 = θ(n)
What is the reverse statement?
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
Understand Quantifiers!!!
Sam Mary
Bob Beth
JohnMarilin Monro
Fred Ann
Sam Mary
Bob Beth
JohnMarilin Monro
Fred Ann
b, Loves(b, MM)
b, Loves(b, MM)
b, Loves(b, MM)]
b, Loves(b, MM)]
Definition of Theta
c c n n n c g n f n o r f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
The reverse statement
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
Definition of Theta
c c n n n c g n f n o r f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3n2 + 7n + 8 > 100·n3 100 ?
Definition of Theta
c c n n n c g n f n o r f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3·1002 + 7·100 + 8 > 100·1003 100
True100
Definition of Theta
c c n n n c g n f n o r f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3n2 + 7n + 8 > 10,000·n3 10,000 ?
Definition of Theta
c c n n n c g n f n o r f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3·10,0002 + 7·10,000 + 8 > 10,000·10,0003 10,000
True10,000
Definition of Theta
c c n n n c g n f n o r f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3n2 + 7n + 8 > c2·nc1 c2 ?
Definition of Theta
c c n n n c g n f n o r f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3·c22 + 7 · c2 + 8 > c2·c2c1 c2
Truec2
Definition of Theta
c c n n n c g n f n o r f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3n2 + 7n + 8 > c2·nc1 c2 ?n0
Definition of Theta
c c n n n c g n f n o r f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( ) ( )
3n2 + 7n + 8 = θ(n)
3·c22 + 7 · c2 + 8 > c2·c2c1 max(c2,n0 )
Truec2 n0
True
Definition of Theta
3n2 + 7n + 8 = g(n)θ(1)
c1, c2, n0, n n0 g(n)c1 f(n) g(n)c2
3n2 + 7n + 8 = nθ(1)
Definition of Theta
3n2 + 7n + 8 = nθ(1)
nc1 3n2 + 7n + 8 nc2
c1, c2, n0, n n0 g(n)c1 f(n) g(n)c2
? ?
Definition of Theta
3n2 + 7n + 8 = nθ(1)
n2 3n2 + 7n + 8 n3
c1, c2, n0, n n0 g(n)c1 f(n) g(n)c2
2 3
Definition of Theta
3n2 + 7n + 8 = nθ(1)
n2 3n2 + 7n + 8 n3
c1, c2, n0, n n0 g(n)c1 f(n) g(n)c2
2 3 ? n ?
Definition of Theta
3n2 + 7n + 8 = nθ(1)
n2 3n2 + 7n + 8 n3
c1, c2, n0, n n0 g(n)c1 f(n) g(n)c2
2 3 5 n 5
Definition of Theta
3n2 + 7n + 8 = nθ(1)
n2 3n2 + 7n + 8 n3
True
True
c1, c2, n0, n n0 g(n)c1 f(n) g(n)c2
2 3 5 n 5
Order of Quantifiers
n n n c c c g n f n c g n0 0 1 2 1 2, , ( ) ( ) ( ), ,
f(n) = θ(g(n))
?
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
g b lo ves b g, , ( , ) b g lo ves b g, , ( , )
Understand Quantifiers!!!
One girl Could be a separate girl for each boy.
Sam Mary
Bob Beth
JohnMarilin Monro
Fred Ann
Sam Mary
Bob Beth
JohnMarilin Monro
Fred Ann
Order of Quantifiers
No! It cannot be a different c1 and c2
for each n.
n n n c c c g n f n c g n0 0 1 2 1 2, , ( ) ( ) ( ), ,
f(n) = θ(g(n))
c c n n n c g n f n c g n1 2 0 0 1 2, , , , ( ) ( ) ( )
Other Notations
Theta f(n) = θ(g(n)) f(n) ≈ c g(n)
BigOh f(n) = O(g(n)) f(n) ≤ c g(n)
Omega f(n) = Ω(g(n)) f(n) ≥ c g(n)
Little Oh f(n) = o(g(n)) f(n) << c g(n)
Little Omega f(n) = ω(g(n)) f(n) >> c g(n)
BigOh Notation
• n2 = O(n3)
• n3 = O(n2)
BigOh Notation
• O(n2) = O(n3)
• O(n3) = O(n2)Odd Notation
f(n) = O(g(n)) Standard f(n) ≤ O(g(n)) Stresses one function dominating another.
f(n)O(g(n)) Stress function is member of class.
The Time Complexity of an Algorithm
Specifies how the running time depends on the size of the input.
Purpose?
Purpose
• To estimate how long a program will run.
• To estimate the largest input that can reasonably be given to the program.
• To compare the efficiency of different algorithms.
• To help focus on the parts of code that are executed the largest number of times.
• To choose an algorithm for an application.
Time Complexity Is a Function
Specifies how the running time depends on the size of the input.
A function mapping
“size” of input
“time” T(n) executed .
Definition of Time?
Definition of Time
• # of seconds (machine dependent).
• # lines of code executed.
• # of times a specific operation is performed (e.g., addition).
Which?
Definition of Time
• # of seconds (machine dependent). • # lines of code executed. • # of times a specific operation is performed
(e.g., addition).
These are all reasonable definitions of time, because they are within a constant factor of
each other.
Size of Input Instance?
83920
Size of Input Instance
• Size of paper• # of bits• # of digits• Value
- n = 2 in2
- n = 17 bits
- n = 5 digits
- n = 83920
2’’
83920
5
1’’
Which are reasonable?
Size of Input Instance
• Size of paper• # of bits• # of digits• Value
- n = 2 in2
- n = 17 bits
- n = 5 digits
- n = 83920
• Intuitive
2’’
83920
5
1’’
Size of Input Instance
• Size of paper• # of bits• # of digits• Value
- n = 2 in2
- n = 17 bits
- n = 5 digits
- n = 83920
• Intuitive• Formal
2’’
83920
5
1’’
Size of Input Instance
• Size of paper• # of bits• # of digits• Value
- n = 2 in2
- n = 17 bits
- n = 5 digits
- n = 83920
• Intuitive• Formal• Reasonable
# of bits =
3.32 * # of digits
2’’
83920
5
1’’
Size of Input Instance
• Size of paper• # of bits• # of digits• Value
- n = 2 in2
- n = 17 bits
- n = 5 digits
- n = 83920
• Intuitive• Formal• Reasonable• Unreasonable
# of bits = log2(Value) Value = 2# of bits
2’’
83920
5
1’’
Two Example Algorithms
• Sum N array entries: A(1) + A(2) + A(3) + …
• Factor value N: Input N=5917 & Output N=97*61.
Algorithm N/2, N/3, N/4, …. ?
Time?
Two Example Algorithms
• Sum N array entries: A(1) + A(2) + A(3) + …
• Factor value N: Input N=5917 & Output N=97*61.
Algorithm N/2, N/3, N/4, …. ?
Time = N
Time = N
Two Example Algorithms
• Sum N array entries: A(1) + A(2) + A(3) + …
• Factor value N: Input N=5917 & Output N=97*61.
Algorithm N/2, N/3, N/4, …. ?
Is this reasonable?
Time = N
Time = N
Two Example Algorithms
• Sum N array entries: A(1) + A(2) + A(3) + …
• Factor value N: Input N=5917 & Output N=97*61.
Algorithm N/2, N/3, N/4, …. ?
No!
Time = N
Time = N
One is considered fast and the other slow!
Two Example Algorithms
• Sum N array entries: A(1) + A(2) + A(3) + …
• Factor value N: Input N=5917 & Output N=97*61.
Algorithm N/2, N/3, N/4, …. ?
Size of Input Instance?
Two Example Algorithms
• Sum N array entries: A(1) + A(2) + A(3) + …
• Factor value N: Input N=5917 & Output N=97*61.
Algorithm N/2, N/3, N/4, …. ?
size n = N
size n = log N
Two Example Algorithms
• Sum N array entries: A(1) + A(2) + A(3) + …
• Factor value N: Input N=5917 & Output N=97*61.
Algorithm N/2, N/3, N/4, …. ?
Time as function of input size?
size n = N
size n = log N
Two Example Algorithms
• Sum N array entries: A(1) + A(2) + A(3) + …
• Factor value N: Input N=5917 & Output N=97*61.
Algorithm N/2, N/3, N/4, …. ?
Time = N = n
Time = N = 2n
size n = N
size n = log N
Two Example Algorithms
• Sum N array entries: A(1) + A(2) + A(3) + …
• Factor value N: Input N=5917 & Output N=97*61.
Algorithm N/2, N/3, N/4, …. ?
Linear vs Exponential Time!
Time = N = n
Time = N = 2n
size n = N
size n = log N
Size of Input Instance?
14,23,25,30,31,52,62,79,88,98
Size of Input Instance
• # of elements - n = 10 elements
14,23,25,30,31,52,62,79,88,98
10
Size of Input Instance
• # of elements - n = 10 elements
14,23,25,30,31,52,62,79,88,98
10
Is this reasonable?
Size of Input Instance
• # of elements - n = 10 elements ~ Reasonable
14,23,25,30,31,52,62,79,88,98
10
If each element has size c
# of bits = c * # of elements
Size of Input Instance
• # of elements - n = 10 elements Reasonable
14,23,25,30,31,52,62,79,88,98
10
If each element is in [1..n]
each element has size log n
# of bits = n log n ≈ n
Time Complexity Is a Function
Specifies how the running time depends on the size of the input.
A function mapping
# of bits n needed to represent the input
# of operations T(n) executed .
Which Input of size n?
There are 2n inputs of size n.Which do we consider
for the time T(n)?
Which Input of size n?
Typical Input
Average Case
Worst Case
Which Input of size n?
Typical Input But what is typical?
Average Case For what distribution?
Worst Case •Time for all inputs is bound.
•Easiest to compute
Time Complexity of Algorithm
O(n2): Prove that for every input of size n, the algorithm takes no more than cn2 time.
Ω(n2): Find one input of size n, for which the algorithm takes at least this much time.
θ (n2): Do both.
The time complexity of an algorithm isthe largest time required on any input of size n.
Time Complexity of Problem
O(n2): Provide an algorithm that solves the problem in no more than this time.
Ω(n2): Prove that no algorithm can solve it faster.θ (n2): Do both.
The time complexity of a problem is the time complexity of the fastest algorithm that solves the problem.
Adding The Classic Techniques
Evaluating ∑i=1 f(i).n
Gauss ∑i=1..n i = 1 + 2 + 3 + . . . + n
= ?
Arithmetic Sum
1 + 2 + 3 + . . . + n-1 + n = S
n + n-1 + n-2 + . . . + 2 + 1 = S
(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S
n (n+1) = 2S
1 + 2 + 3 + . . . + n-1 + n = S
n + n-1 + n-2 + . . . + 2 + 1 = S
(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S
n (n+1) = 2S
1 + 2 + 3 + . . . + n-1 + n = S
n + n-1 + n-2 + . . . + 2 + 1 = S
(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S
n (n+1) = 2S
1 + 2 + 3 + . . . + n-1 + n = S
n + n-1 + n-2 + . . . + 2 + 1 = S
(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S
n (n+1) = 2S
2
1)(n n S
1 + 2 + 3 + . . . + n-1 + n = S
n + n-1 + n-2 + . . . + 2 + 1 = S
(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S
n (n+1) = 2S
Let’s restate this argument using a
geometric representation
Algebraic argument
1 2 . . . . . . . . n
= number of white dots.1 + 2 + 3 + . . . + n-1 + n = S
n + n-1 + n-2 + . . . + 2 + 1 = S
(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S
n (n+1) = 2S
1 + 2 + 3 + . . . + n-1 + n = S
n + n-1 + n-2 + . . . + 2 + 1 = S
(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S
n (n+1) = 2S
1 2 . . . . . . . . n
= number of white dots
= number of yellow dots
n . . . . . . . 2 1
1 + 2 + 3 + . . . + n-1 + n = S
n + n-1 + n-2 + . . . + 2 + 1 = S
(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S
n (n+1) = 2S
n+1 n+1 n+1 n+1 n+1
= number of white dots
= number of yellow dots
n
n
n
n
n
n
There are n(n+1) dots in the grid
1 + 2 + 3 + . . . + n-1 + n = S
n + n-1 + n-2 + . . . + 2 + 1 = S
(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S
n (n+1) = 2S
n+1 n+1 n+1 n+1 n+1
= number of white dots
= number of yellow dots
n
n
n
n
n
n
2
1)(n n S
Note = (# of terms · last term))
Gauss ∑i=1..n i = 1 + 2 + 3 + . . . + n
= (# of terms · last term)
Arithmetic Sum
True when ever terms increase slowly
∑i=0..n 2i = 1 + 2 + 4 + 8 +. . . + 2n
= ?
Geometric Sum
Geometric Sum
∑i=0..n 2i = 1 + 2 + 4 + 8 +. . . + 2n
= 2 · last term - 1
Geometric Sum
∑i=0..n ri = r0 + r1 + r2 +. . . + rn
= ?
Geometric Sum
S
Sr r r r ... r r
S Sr 1 r
Sr 1
r 1
2 3 n n 1
n 1
n 1
1 r r r ...2 3 rn
Geometric Sum
∑i=0..n rir 1
r 1
n 1
Geometric SumWhen r>1?
∑i=0..n rir 1
r 1
n 1
Geometric Sum
θ(rn)Biggest TermWhen r>1
∑i=0..n ri = r0 + r1 + r2 +. . . + rn
= (biggest term)
Geometric Increasing
True when ever terms increase quickly
∑i=0..n ri
Geometric SumWhen r<1?
1 r
1 r
n 1
∑i=0..n ri
Geometric Sum
1 r
1 r
n 1
θ(1)When r<1
Biggest Term
∑i=0..n ri = r0 + r1 + r2 +. . . + rn
= (1)
Bounded Tail
True when ever terms decrease quickly
∑i=1..n 1/i
= 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + …+ 1/n
= ?
Harmonic Sum
f(i) = 1∑i=1..n f(i) = n
n
Sum of Shrinking Function
f(i) = 1/2i
∑i=1..n f(i) = 2
Sum of Shrinking Function
f(i) = 1/i∑i=1..n f(i) = ?
n
Sum of Shrinking Function
Harmonic Sum
∑i=1..n 1/i
= 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + …+ 1/n
= (log(n))
Harmonic Sum
Approximating Sum by Integrating
The area under the curveapproximates the sum
∑i=1..n f(i) ≈ ∫x=1..n f(x) dx
Approximating Sum by Integrating
∫x=1..n xc dx = 1/c+1 nc+1∑i=1..n ic ≈
Arithmetic Sums
θ(# of terms · last term)True when ever terms increase slowly
= θ(nc+1)
Approximating Sum by Integrating
∫x=1..n bx dx = 1/ln(b) bn∑i=1..n bi ≈
Geometric Sums
θ(last term)True when ever terms increase quickly
= θ(bn)
Harmonic Sum
Approximating Sum by Integrating
Approximating Sum by Integrating
Problem: Integrating may be hard too.
Outline
II) Math proofs of sums
V) Ability to know the answer fast
I) What is the sumEg. ∑i=1..n f(i) = ∑i=1..n 1/i
0.9999 =θ(n0.0001)
III) Pattern of sums ∑i=1..n f(i) = θ(n · f(n))
IV) Intuition why this is the sum
I flash it upIf you follow great.If not don’t panic.
Adding Made Easy
We will now classify (most) functions f(i)
into four classes:
–Geometric Like–Arithmetic Like–Harmonic–Bounded Tail
Adding Made Easy
We will now classify (most) functions f(i)
into four classes
For each class, we will give
an easy rule for approximating
it’s sum
θ( ∑i=1..n f(i) )
Classifying Animals
Vertebrates
Birds
Mam
mals
Reptiles
Fish
Dogs
Giraffe
Mammal Complex social networks Dog Man’s best friend
Classifying Functions
Functions
Poly.
Exp.
2θ(n) ∑i=1..n f(i) = θ(f(n))
8·2n / n100 + n3
θ(2n / n100)
8·2n + n3
nθ(1) 2θ(n)
θ(2n)
f(n) = 8·2n / n100 + n3
θ(2n / n100) ∑i=1..n f(i) = θ(2n / n100)
20.5n < < 2n8· 2n / n100 + n3
Significant Less significant
Irrelevant
Adding Made Easy
11
1
2
34
Four Classes of Functions
Adding Made Easy
If the terms f(i) grow sufficiently quickly, then the sum will be
dominated by the largest term.
Silly example:1+2+3+4+5+ 1,000,000,000 ≈ 1,000,000,000
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
If the terms f(i) grow sufficiently quickly, then the sum will be
dominated by the largest term.
Classic example:
?
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
If the terms f(i) grow sufficiently quickly, then the sum will be
dominated by the largest term.
Classic example:∑i=1..n 2i = 2n+1-1 ≈ 2 f(n)
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
If the terms f(i) grow sufficiently quickly, then the sum will be
dominated by the largest term.
For which functions f(i) is this true?
How fast and how slow can it grow?
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
r 1r 1
n 1
θ( )r n Last Term
when r>1.
∑i=1..n ri 1 r r r . . .2 3 r n
θ(f(n))
∑i=1..n (1.0001)i ≈ 10,000 (1.0001)n
= 10,000 f(n)Lower Extreme:
Upper Extreme: ∑i=1..n (1000)i ≈ 1.001(1000)n
= 1.001 f(n)
Recall, when f(n) = ri = 2θ(n)
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
Because the constant is sooo big, the statement is just barely true.
∑i=1..n (1.0001)i ≈ 10,000 (1.0001)n
= 10,000 f(n)Lower Extreme:
Upper Extreme: ∑i=1..n (1000)i ≈ 1.001(1000)n
= 1.001 f(n)
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
∑i=1..n (1.0001)i ≈ 10,000 (1.0001)n
= 10,000 f(n)Lower Extreme:
Upper Extreme: ∑i=1..n (1000)i ≈ 1.001(1000)n
= 1.001 f(n)
Even bigger?
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
2n2i
∑i=1..n 22 ≈ 22 = 1f(n)
No Upper Extreme:
Even bigger?
∑i=1..n (1.0001)i ≈ 10,000 (1.0001)n
= 10,000 f(n)Lower Extreme:
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
2n2i
∑i=1..n 22 ≈ 22 = 1f(n)
No Upper Extreme:
Functions in between?
∑i=1..n (1.0001)i ≈ 10,000 (1.0001)n
= 10,000 f(n)Lower Extreme:
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
∑i=1..n 2i
= 2·2n –2
= 21 + 22 + 23 + 24 +…+ 2n
8·[ ]
8· 8· 8· 8· 8·
8·
= θ( 2n )
1100 2100 3100 4100 n100
i100
n100
+ i3
+ 13 + 23 + 33 + 43 + n3
+ n3
Dominated by the largest term.
= θ(f(n))
f(n) = 2n8·n100
+ n3 = θ( 2n )n100
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
n100≈
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
f(n) = 2n8·n100
∑i=1..n f(i)f(n) c·f(n)f(i) ·f(n)f(i) ·f(i+1)
Easy Hard
(i+1)100
i1002
= (1+1/i)100
12 1.9
=8·2(i+1)
(i+1)100f(i+1)f(i) 8·2i
i100
=
1/.51
·f(i+1)f(i)
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
f(n) = 2n8·n100
∑i=1..n f(i)f(n) c·f(n)f(i) ·f(n)
·f(i+2)
·f(i+1)f(i)
·f(i+1)f(i) ·f(i+3).51(n-i)·f(i+(n-i)).51(n-i)·f(n)
f(i) .51(n-i)·f(n)
Eg. i = n, f(n) .51(n-n)·f(n)1·f(n)
i = 0, f(0) .51n·f(n)
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
f(n) = 2n8·n100
∑i=1..n f(i)f(n) c·f(n)
∑i=1..n f(i) ∑i=1..n .51(n-i)·f(n)
= θ(1) · f(n)
f(i) ·f(n)·f(i+1)f(i)
f(i) .51(n-i)·f(n)
= [∑i=1..n .51(n-i)] · f(n)
Functions
Poly.
Exp.
2θ(n) ∑i=1..n f(i) = θ(f(n))
8·2n / n100 + n3
θ(2n / n100)
8·2n + n3
nθ(1) 2θ(n)
θ(2n)
f(n) = 8·2n / n100 + n3
θ(2n / n100) ∑i=1..n f(i) = θ(2n / n100)
20.5n < < 2n8· 2n / n100 + n3
Significant Less significant
Irrelevant
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
If f(n) = c ban nd loge (n)c ?a ?b ?d ?e ?
f(n) = Ω, θ ?
then ∑i=1..n f(i) = θ(f(n)).
2Ω(n)
> 0 0 1
(-,)(-,)
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
All functions in 2Ω(n)? Maybe not.
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
Functions that oscillate continually do not have the property.
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
Functions expressed with +, -, , , exp, log
do not oscillate continually.
They are well behaved for sufficiently large n.
These do have the property.
f(i) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:
Adding Made Easy
Done
Adding Made Easy
If most of the terms f(i) have roughly the same value, then the sum is roughly the
number of terms, n, times this value.
Silly example:1,001 + 1,002 + 1,003 + 1,004 + 1,005
≈ 5 · 1,000
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
If most of the terms f(i) have roughly the same value, then the sum is roughly the
number of terms, n, times this value.
Another silly example:∑i=1..n 1 = n · 1
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
If most of the terms f(i) have roughly the same value, then the sum is roughly the
number of terms, n, times this value.
Is the statement true for this function?
∑i=1..n i = 1 + 2 + 3 + . . . + n
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
If most of the terms f(i) have roughly the same value, then the sum is roughly the
number of terms, n, times this value.
Is the statement true for this function?
∑i=1..n i = 1 + 2 + 3 + . . . + n
The terms are not roughly the same.
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
But half the terms are roughly the same.
∑i=1..n i =
1 + . . . + n/2 + . . . + n
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
But half the terms are roughly the same
and the sum is roughly the number
terms, n, times this value
∑i=1..n i =
1 + . . . + n/2 + . . . + n
∑i=1..n i = θ(n · n)
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
Is the statement true for this function?
∑i=1..n i2 = 12 + 22 + 32 + . . . + n2
Even though, the terms are not roughly the same.
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
Again half the terms are roughly the same.
∑i=1..n i =
12 + . . . + (n/2)2 + . . . + n2
1/4 n2
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
Again half the terms are roughly the same.
∑i=1..n i =
12 + . . . + (n/2)2 + . . . + n2
1/4 n2
∑i=1..n i2 = θ(n · n2)
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
∑i=1..n f(i) ≈ area under curve
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
area of small square ∑i=1..n f(i)
≈ area under curve area of big square
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
n/2 · f(n/2)
= area of small square ∑i=1..n f(i)
≈ area under curve area of big square
= n · f(n)
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
θ(n · f(n)) = n/2 · f(n/2)
= area of small square ∑i=1..n f(i)
≈ area under curve area of big square
= n · f(n)
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
θ(n · f(n)) = n/2 · f(n/2)
= area of small square ∑i=1..n f(i)
≈ area under curve area of big square
= n · f(n)
?
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
∑i=1..n i2 = 12 + 22 + 32 + . . . + n2
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
f(i) = n2
f(n/2) = θ(f(n))
The key property is
= ?
The key property is
f(n/2) = (n/2)2 = 1/4 n2 = θ(n2) = θ(f(n))
∑i=1..n i2 = 12 + 22 + 32 + . . . + n2
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
f(i) = n2
= θ(n·f(n)) = θ(n · n2)
∑i=1..n f(i) = ?
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
f(i) = nθ(1)
∑i=1..n ir = 1r + 2r + 3r + . . . + nr
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
f(i) = nθ(1)
f(n/2) = θ(f(n))
The key property is
The key property is
f(n/2) = (n/2)r = 1/2r nr = θ(nr) = θ(f(n))
∑i=1..n ir = 1r + 2r + 3r + . . . + nr
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
= θ(n·f(n)) = θ(n · nr)
f(i) = nθ(1)
∑i=1..n 2i = 21 + 22 + 23 + . . . + 2n
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
f(i) = nθ(1)
f(n/2) = θ(f(n))
The key property is
The key property is
∑i=1..n 2i = 21 + 22 + 23 + . . . + 2n
f(n/2) = 2(n/2) = θ(2n) = θ(f(n))
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
f(i) = nθ(1)
= θ(n·f(n)) = θ(n · 2n)
∑i=1..n 1 = n · 1 = n · f(n)
Middle Extreme:
Upper Extreme: ∑i=1..n i1000 = 1/1001 n1001
= 1/1001 n · f(n)
All functions in between.
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
Adding Made Easy
Half done
f(i) = 1∑i=1..n f(i) = n
n
Sum of Shrinking Function
f(i) = ?∑i=1..n f(i) = n1/2
n
Sum of Shrinking Function
1/i1/2
f(i) = 1/i∑i=1..n f(i) = log n
n
Sum of Shrinking Function
f(i) = 1/2i
∑i=1..n f(i) = 2
Sum of Shrinking Function
If most of the terms f(i) have roughly the same value, then the sum is roughly the
number of terms, n, times this value.
Does the statement hold for functions f(i) that shrink?
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
If most of the terms f(i) have roughly the same value, then the sum is roughly the
number of terms, n, times this value.
∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n
Does the statement hold for the Harmonic Sum?
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
Does the statement hold for the Harmonic Sum?
∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n
∑i=1..n f(i) = ?
θ(n · f(n))
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
Does the statement hold for the Harmonic Sum?
∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n
∑i=1..n f(i)
θ(n · f(n))
= ∑i=1..n 1/i = ?
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
Does the statement hold for the Harmonic Sum?
∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n
∑i=1..n f(i)
θ(n · f(n))
= ∑i=1..n 1/i = θ(log n)
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
Does the statement hold for the Harmonic Sum?
∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n
∑i=1..n f(i)
= θ(n · f(n))
= ∑i=1..n 1/i = θ(log n)
?
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
Does the statement hold for the Harmonic Sum?
∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n
∑i=1..n f(i)
= θ(n · f(n))
= ∑i=1..n 1/i = θ(log n)
θ(1) = θ(n · 1/n)
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
Does the statement hold for the Harmonic Sum?
∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n
∑i=1..n f(i)
= θ(n · f(n))
= ∑i=1..n 1/i = θ(log n)
θ(1) = θ(n · 1/n)≠
No the statement does not hold!
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
Adding Made Easy
not included
Does the statement hold for the almost Harmonic Sum?
∑i=1..n 1/i0.9999
Shrinks slightly slower than harmonic.
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
Does the statement hold for the almost Harmonic Sum?
∑i=1..n 1/i0.9999
∑i=1..n f(i)
= θ(n · f(n))
= ∑i=1..n 1/i0.9999
= ?
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
Does the statement hold for the almost Harmonic Sum?
∑i=1..n 1/i0.9999
∑i=1..n f(i)
= θ(n · f(n))
= ∑i=1..n 1/i0.9999
= θ(n0.0001)
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
Does the statement hold for the almost Harmonic Sum?
∑i=1..n 1/i0.9999
∑i=1..n f(i)
= θ(n · f(n))
= ∑i=1..n 1/i0.9999
= θ(n0.0001)
?
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
Does the statement hold for the almost Harmonic Sum?
∑i=1..n 1/i0.9999
∑i=1..n f(i)
= θ(n · f(n))
= ∑i=1..n 1/i0.9999
= θ(n0.0001)
θ(n0.0001) = θ(n · 1/n0.9999)
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
Does the statement hold for the almost Harmonic Sum?
∑i=1..n 1/i0.9999
∑i=1..n f(i)
= θ(n · f(n))
= ∑i=1..n 1/i0.9999
= θ(n0.0001)
θ(n0.0001) = θ(n · 1/n0.9999)
=
The statement does hold!
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
∑i=1..n 1 = n · 1 = n · f(n)
Middle Extreme:
Lower Extreme: ∑i=1..n 1/i0.9999
= θ(n0.0001) = θ(n · f(n))
All functions in between.
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
∑i=1..n 1 = n · 1 = n · f(n)
Middle Extreme:
Lower Extreme: ∑i=1..n 1/i0.9999
= θ(n0.0001) = θ(n · f(n))
Upper Extreme: ∑i=1..n i1000 = 1/1001 n1001
= 1/1001 n · f(n)
f(i) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:
Arithmetic Like
If f(n) = c ban nd loge (n)c ?a ?b ?d ?e ?
f(n) = Ω, θ ?
then ∑i=1..n f(i) = θ(n · f(n)).
n-1+θ(1)
> 0 0
or b 1> -1
(-,)
(For +, -, , , exp, log functions f(n))
Conclusion
Adding Made Easy
Done
Adding Made Easy
Harmonic
Harmonic Sum
∑i=1..n 1/i
= 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + …+ 1/n
= (log(n))
Adding Made Easy
If the terms f(i) decrease towards zerosufficiently quickly,
then the sum will be a constant.
The classic example ∑i=0..n 1/2i = 1 + 1/2 + 1/4 + 1/8 + … = 2.
f(n) n-1-Ω(1) ∑i=1..n f(i) = θ(1)Bounded Tail:
If f(i) decays even faster, then the tail of the sum is more bounded.
2i
∑i=1..n 22 = θ(1).
1
f(n) n-1-Ω(1) ∑i=1..n f(i) = θ(1)Bounded Tail:
Upper Extreme: ∑i=1..n 1/i1.0001
= θ(1)
f(n) n-1-Ω(1) ∑i=1..n f(i) = θ(1)Bounded Tail:
Upper Extreme: ∑i=1..n 1/i1.0001
= θ(1)
No Lower Extreme:2i
∑i=1..n 22 = θ(1).
1
All functions in between.
f(n) n-1-Ω(1) ∑i=1..n f(i) = θ(1)Bounded Tail:
Bounded Tail
If f(n) = c ban nd loge (n)c ?a ?b ?d ?e ?
f(n) = Ω, θ ?
then ∑i=1..n f(i) = θ(1).
> 0
(-,)
(-,)
Conclusion
or b 1
c nd loge (n)
c a b
ban
Bounded Tail
If f(n) = c nd loge (n)c ?a b = 1d ?e ?
f(n) = Ω, θ ?
then ∑i=1..n f(i) = θ(1).
nθ(1)-1
> 0
< -1(-,)
Conclusion
Adding Made Easy
Done
Adding Made EasyMissing Functions
n n
n nlo g
1/nlogn
logn/n
Adding Made Easy • Geometric Like: If f(n) 2Ω(n), then ∑i=1..n f(i) = θ(f(n)).
• Arithmetic Like: If f(n) = nθ(1)-1, then ∑i=1..n f(i) = θ(n · f(n)).
• Harmonic: If f(n) = 1/n , then ∑i=1..n f(i) = logen + θ(1).
• Bounded Tail: If f(n) n-1-Ω(1), then ∑i=1..n f(i) = θ(1).
(For +, -, , , exp, log functions f(n))
This may seem confusing, but it is really not.
It should help you compute most sums easily.
Recurrence Relations
T(1) = 1
T(n) = a T(n/b) + f(n)
Recurrence Relations Time of Recursive Program
procedure Eg(int n)
if(n1) then
put “Hi”
else
loop i=1..f(n)
put “Hi”
loop i=1..a
Eg(n/b)
Recurrence relationsarise from the timing of recursive programs.
Let T(n) be the # of “Hi”s on an input of “size” n.
procedure Eg(int n)
if(n1) then
put “Hi”
else
loop i=1..f(n)
put “Hi”
loop i=1..a
Eg(n/b)
Given size 1,the program outputs
T(1)=1 Hi’s.
Recurrence Relations Time of Recursive Program
procedure Eg(int n)
if(n1) then
put “Hi”
else
loop i=1..f(n)
put “Hi”
loop i=1..a
Eg(n/b)
Given size n,the stackframe outputs
f(n) Hi’s.
Recurrence Relations Time of Recursive Program
procedure Eg(int n)
if(n1) then
put “Hi”
else
loop i=1..f(n)
put “Hi”
loop i=1..a
Eg(n/b)
Recursing on a instances of size n/bgenerates T(n/b) “Hi”s.
Recurrence Relations Time of Recursive Program
procedure Eg(int n)
if(n1) then
put “Hi”
else
loop i=1..f(n)
put “Hi”
loop i=1..a
Eg(n/b)
Recursing a times
generates a·T(n/b) “Hi”s.
Recurrence Relations Time of Recursive Program
procedure Eg(int n)
if(n1) then
put “Hi”
else
loop i=1..f(n)
put “Hi”
loop i=1..a
Eg(n/b)
For a total of T(1) = 1 T(n) = a·T(n/b) + f(n)
“Hi”s.
Recurrence Relations Time of Recursive Program
X = 2133Y = 2312ac = bd = (a+b)(c+d) = XY =
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
Stack Frame: A particular execution of one routine on one particular input instance.
X = 2133Y = 2312
ac = ?bd = (a+b)(c+d) = XY =
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312
ac = ?bd = (a+b)(c+d) = XY =
X = 21Y = 23ac = bd = (a+b)(c+d) = XY =
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312
ac = ?bd = (a+b)(c+d) = XY =
X = 21Y = 23
ac = ?bd = (a+b)(c+d) = XY =
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312
ac = ?bd = (a+b)(c+d) = XY =
X = 21Y = 23
ac = ?bd = (a+b)(c+d) = XY =
X = 2Y = 2XY=
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312
ac = ?bd = (a+b)(c+d) = XY =
X = 21Y = 23
ac = ?bd = (a+b)(c+d) = XY =
X = 2Y = 2XY = 4
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312
ac = ?bd = (a+b)(c+d) = XY =
X = 21Y = 23ac = 4bd = (a+b)(c+d) = XY =
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312
ac = ?bd = (a+b)(c+d) = XY =
X = 21Y = 23ac = 4
bd = ?(a+b)(c+d) = XY =
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312
ac = ?bd = (a+b)(c+d) = XY =
X = 21Y = 23ac = 4
bd = ?(a+b)(c+d) = XY =
X = 1Y = 3XY = 3
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312
ac = ?bd = (a+b)(c+d) = XY =
X = 21Y = 23ac = 4bd = 3(a+b)(c+d) = XY =
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312
ac = ?bd = (a+b)(c+d) = XY =
X = 21Y = 23ac = 4bd = 3
(a+b)(c+d) = ?XY =
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312
ac = ?bd = (a+b)(c+d) = XY =
X = 21Y = 23ac = 4bd = 3
(a+b)(c+d) = ?XY =
X = 3Y = 5XY = 15
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312
ac = ?bd = (a+b)(c+d) = XY =
X = 21Y = 23ac = 4bd = 3(a+b)(c+d) = 15
XY = ?
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312
ac = ?bd = (a+b)(c+d) = XY =
X = 21Y = 23ac = 4bd = 3(a+b)(c+d) = 15XY = 483
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312ac = 483bd = (a+b)(c+d) = XY =
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312ac = 483
bd = ?(a+b)(c+d) = XY =
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312ac = 483
bd = ?(a+b)(c+d) = XY =
X = 33Y = 12
ac = ?bd = (a+b)(c+d) = XY = 15
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312ac = 483
bd = ?(a+b)(c+d) = XY =
X = 33Y = 12
ac = ?bd = (a+b)(c+d) = XY = 15
X = 3Y = 1XY = 3
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312ac = 483
bd = ?(a+b)(c+d) = XY =
X = 33Y = 12ac = 3
bd = ?(a+b)(c+d) = XY = 15
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312ac = 483
bd = ?(a+b)(c+d) = XY =
X = 33Y = 12ac = 3
bd = ?(a+b)(c+d) = XY = 15
X = 3Y = 2XY = 6
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312ac = 483
bd = ?(a+b)(c+d) = XY =
X = 33Y = 12ac = 3bd = 6
(a+b)(c+d) = ?XY = 15
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
X = 2133Y = 2312ac = 483
bd = ?(a+b)(c+d) = XY =
X = 33Y = 12ac = 3bd = 6
(a+b)(c+d) = ?XY = 396
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Stack of Stack Frames
An so on ….
X = 2133Y = 2312ac = 483bd = 396(a+b)(c+d) = 1890XY = 4931496
X = 21Y = 23ac = 4bd = 3(a+b)(c+d) = 15XY = 483
X = 33Y = 12ac = 3bd = 6(a+b)(c+d) = 18XY = 396
X = 54Y = 35ac = 15bd = 20(a+b)(c+d) = 72XY = 1890
X = 2Y = 2XY=4
X = 1Y = 3XY=3
X = 3Y = 5XY=15
X = 3Y = 1XY=3
X = 3Y = 2XY=6
X = 6Y = 3XY=18
X = 5Y = 3XY=15
X = 4Y = 5XY=20
X = 9Y = 8XY=72
MULT(X,Y):
If |X| = |Y| = 1 then RETURN XY
Break X into a;b and Y into c;d
e = MULT(a,c) and f =MULT(b,d)
RETURN
e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f
Tree of Stack Frames
Solving Technique 1Guess and Verify
•Recurrence Relation:
T(1) = 1 & T(n) = 4T(n/2) + n
•Guess: G(n) = 2n2 – n
•Verify: Left Hand Side Right Hand Side
T(1) = 2(1)2 – 1
T(n)
= 2n2 – n
1
4T(n/2) + n
= 4 [2(n/2)2 – (n/2)] + n
= 2n2 – n
•Recurrence Relation:
T(1) = 1 & T(n) = 4T(n/2) + n
•Guess: G(n) = an2 + bn + c
•Verify: Left Hand Side Right Hand Side
T(1) = a+b+c
T(n)
= an2 +bn+c
1
4T(n/2) + n
= 4 [a (n/2)2 + b (n/2) +c] + n
= an2 +(2b+1)n + 4c
Solving Technique 2Guess Form and Calculate Coefficients
•Recurrence Relation:
T(1) = 1 & T(n) = 4T(n/2) + n
•Guess: G(n) = an2 + bn + 0
•Verify: Left Hand Side Right Hand Side
T(1) = a+b+c
T(n)
= an2 +bn+c
1
4T(n/2) + n
= 4 [a (n/2)2 + b (n/2) +c] + n
= an2 +(2b+1)n + 4c
Solving Technique 2Guess Form and Calculate Coefficients
c=4cc=0
•Recurrence Relation:
T(1) = 1 & T(n) = 4T(n/2) + n
•Guess: G(n) = an2 - 1n + 0
•Verify: Left Hand Side Right Hand Side
T(1) = a+b+c
T(n)
= an2 +bn+c
1
4T(n/2) + n
= 4 [a (n/2)2 + b (n/2) +c] + n
= an2 +(2b+1)n + 4c
Solving Technique 2Guess Form and Calculate Coefficients
b=2b+1b=-1
•Recurrence Relation:
T(1) = 1 & T(n) = 4T(n/2) + n
•Guess: G(n) = an2 - 1n + 0
•Verify: Left Hand Side Right Hand Side
T(1) = a+b+c
T(n)
= an2 +bn+c
1
4T(n/2) + n
= 4 [a (n/2)2 + b (n/2) +c] + n
= an2 +(2b+1)n + 4c
Solving Technique 2Guess Form and Calculate Coefficients
a=a
•Recurrence Relation:
T(1) = 1 & T(n) = 4T(n/2) + n
•Guess: G(n) = 2n2 - 1n + 0
•Verify: Left Hand Side Right Hand Side
T(1) = a+b+c
T(n)
= an2 +bn+c
1
4T(n/2) + n
= 4 [a (n/2)2 + b (n/2) +c] + n
= an2 +(2b+1)n + 4c
Solving Technique 2Guess Form and Calculate Coefficients
a+b+c=1a-1+0=1a=2
•Recurrence Relation:
T(1) = 1 & T(n) = aT(n/b) + f(n)
Solving Technique 3Approximate Form and
Calculate Exponent
which is bigger?
Guess
•Recurrence Relation:
T(1) = 1 & T(n) = aT(n/b) + f(n)
•Guess: aT(n/b) << f(n)
•Simplify: T(n) f(n)
Solving Technique 3Calculate Exponent
In this case, the answer is easy. T(n) = (f(n))
•Recurrence Relation:
T(1) = 1 & T(n) = aT(n/b) + f(n)
•Guess: aT(n/b) >> f(n)
•Simplify: T(n) aT(n/b)
Solving Technique 3Calculate Exponent
In this case, the answer is harder.
•Recurrence Relation:
T(1) = 1 & T(n) = aT(n/b)
•Guess: G(n) = cn
•Verify: Left Hand Side Right Hand Side
T(n)
= cn
aT(n/b)
= a [c (n/b) ]
= c a b n
Solving Technique 3Calculate Exponent
(log a/log b)= cn
1 = a b-
b = alog b = log a = log a/log b
•Recurrence Relation:
T(1) = 1 & T(n) = 4T(n/2)
•Guess: G(n) = cn
•Verify: Left Hand Side Right Hand Side
T(n)
= cn
aT(n/b) + f(n)
= c [a (n/b) ]
= c a b n
Solving Technique 3Calculate Exponent
1 = a b-
b = alog b = log a = log a/log b
(log a/log b)= cn(log 4/log 2)= cn
2= cn
•Recurrence Relation:
T(1) = 1 & T(n) = aT(n/b) + f(n)
Solving Technique 3Calculate Exponent
If bigger then
T(n) = (f(n))
If bigger then(log a/log b)T(n) = (n )
And if aT(n/b) f(n) what is T(n) then?
Technique 4: Decorate The Tree
• T(n) = a T(n/b) + f(n)
f(n)
T(n/b) T(n/b) T(n/b) T(n/b)
T(n) =
T(1)
•T(1) = 1
1=
a
f(n)
T(n/b) T(n/b) T(n/b) T(n/b)
T(n) =a
f(n)
T(n/b) T(n/b) T(n/b)
T(n) =a
f(n/b)
T(n/b2)T(n/b2)T(n/ b2)T(n/ b2)
a
f(n)T(n) =a
f(n/b)
T(n/b2)T(n/b2)T(n/ b2)T(n/ b2)
af(n/b)
T(n/b2)T(n/b2)T(n/ b2)T(n/ b2)
af(n/b)
T(n/b2)T(n/b2)T(n/ b2)T(n/ b2)
af(n/b)
T(n/b2)T(n/b2)T(n/ b2)T(n/ b2)
a
11111111111111111111111111111111 . . . . . . 111111111111111111111111111111111
f(n)T(n) =a
f(n/b)
T(n/b2)T(n/b2)T(n/ b2)T(n/ b2)
af(n/b)
T(n/b2)T(n/b2)T(n/ b2)T(n/ b2)
af(n/b)
T(n/b2)T(n/b2)T(n/ b2)T(n/ b2)
af(n/b)
T(n/b2)T(n/b2)T(n/ b2)T(n/ b2)
a
Evaluating: T(n) = aT(n/b)+f(n)
Level
0
1
2
i
h
Evaluating: T(n) = aT(n/b)+f(n)
LevelInstance
size
0
1
2
i
h
Evaluating: T(n) = aT(n/b)+f(n)
LevelInstance
size
0 n
1
2
i
h
Evaluating: T(n) = aT(n/b)+f(n)
LevelInstance
size
0 n
1 n/b
2
i
h
Evaluating: T(n) = aT(n/b)+f(n)
LevelInstance
size
0 n
1 n/b
2 n/b2
i
h
Evaluating: T(n) = aT(n/b)+f(n)
LevelInstance
size
0 n
1 n/b
2 n/b2
i n/bi
h n/bh
Evaluating: T(n) = aT(n/b)+f(n)
LevelInstance
size
0 n
1 n/b
2 n/b2
i n/bi
h n/bh
Evaluating: T(n) = aT(n/b)+f(n)
LevelInstance
size
0 n
1 n/b
2 n/b2
i n/bi
h n/bh = 1
base case
Evaluating: T(n) = aT(n/b)+f(n)
LevelInstance
size
0 n
1 n/b
2 n/b2
i n/bi
h n/bh = 1
Evaluating: T(n) = aT(n/b)+f(n)
LevelInstance
size
0 n
1 n/b
2 n/b2
i n/bi
h = log n/log b n/bh = 1bh = nh log b = log nh = log n/log b
Evaluating: T(n) = aT(n/b)+f(n)
LevelInstance
size
Work
in stack
frame
0 n
1 n/b
2 n/b2
i n/bi
h = log n/log b 1
Evaluating: T(n) = aT(n/b)+f(n)
LevelInstance
size
Work
in stack
frame
0 n f(n)
1 n/b f(n/b)
2 n/b2 f(n/b2)
i n/bi f(n/bi)
h = log n/log b 1 T(1)
Evaluating: T(n) = aT(n/b)+f(n)
LevelInstance
size
Work
in stack
frame
# stack frames
0 n f(n)
1 n/b f(n/b)
2 n/b2 f(n/b2)
i n/bi f(n/bi)
h = log n/log b n/bh T(1)
Evaluating: T(n) = aT(n/b)+f(n)
LevelInstance
size
Work
in stack
frame
# stack frames
0 n f(n) 1
1 n/b f(n/b) a
2 n/b2 f(n/b2) a2
i n/bi f(n/bi) ai
h = log n/log b n/bh T(1) ah
Evaluating: T(n) = aT(n/b)+f(n)
LevelInstance
size
Work
in stack
frame
# stack frames
0 n f(n) 1
1 n/b f(n/b) a
2 n/b2 f(n/b2) a2
i n/bi f(n/bi) ai
h = log n/log b n/bh T(1) ah
Evaluating: T(n) = aT(n/b)+f(n)
LevelInstance
size
Work
in stack
frame
# stack frames
0 n f(n) 1
1 n/b f(n/b) a
2 n/b2 f(n/b2) a2
i n/bi f(n/bi) ai
h = log n/log b n/bh T(1) ah
ah = a
= n
log n/log b
log a/log b
Evaluating: T(n) = aT(n/b)+f(n)
LevelInstance
size
Work
in stack
frame
# stack frames
Work in Level
0 n f(n) 1
1 n/b f(n/b) a
2 n/b2 f(n/b2) a2
i n/bi f(n/bi) ai
h = log n/log b n/bh T(1) nlog a/log b
Evaluating: T(n) = aT(n/b)+f(n)
LevelInstance
size
Work
in stack
frame
# stack frames
Work in Level
0 n f(n) 1 1 · f(n)
1 n/b f(n/b) a a · f(n/b)
2 n/b2 f(n/b2) a2 a2 · f(n/b2)
i n/bi f(n/bi) ai ai · f(n/bi)
h = log n/log b n/bh T(1) nlog a/log b n · T(1)
log a/log b
Total Work T(n) = ∑i=0..h aif(n/bi)
Evaluating: T(n) = aT(n/b)+f(n)
= ∑i=0..h aif(n/bi)
If a Geometric Sum∑i=0..n xi θ(max(first term, last term))
Evaluating: T(n) = aT(n/b)+f(n)
LevelInstance
size
Work
in stack
frame
# stack frames
Work in Level
0 n f(n) 1 1 · f(n)
1 n/b f(n/b) a a · f(n/b)
2 n/b2 f(n/b2) a2 a2 · f(n/b2)
i n/bi f(n/bi) ai ai · f(n/bi)
h = log n/log b n/bh T(1) nlog a/log b n · T(1)
log a/log b
Dominated by Top Level or Base Cases
Evaluating: T(n) = aT(n/b)+f(n)Is the sum Geometric?
Simplify by letting f(n) = nc logkn
T(n) = ∑i=0..h aif(n/bi)
= ∑i=0..h ai(n/bi)c logk(n/bi)
= nc ∑i=0..h 2[log a - c log b] · i logk(n/bi)
Geometrically IncreasingArithmetic SumGeometrically Decreasing
Sum = θ(last term)
= θ( )
Sum = θ(any term # terms)
= θ(f(n) log n)
Sum = θ(first term)
= θ(f(n)).
log a - c log b < 0log a - c log b = 0log a - c log b < 0
c < log a/log b
c = log a/log b & k<-1c = log a/log b & k>-1c > log a/log b
nlog a/log b
(n/..)c = ncai = 2[log a]·i(../bi)c = 2[-c log b]·i
= nc ∑i=0..h 2-d i logk(n/bi)
Evaluating: T(n) = aT(n/b)+f(n)Is the sum Geometric?
Simplify by letting f(n) = nc logkn
T(n) = ∑i=0..h aif(n/bi)
= ∑i=0..h ai(n/bi)c logk(n/bi)
= nc ∑i=0..h 2[log a - c log b] · i logk(n/bi)
Geometrically IncreasingArithmetic SumGeometrically Decreasing
Sum = θ(last term)
= θ( )
Sum = θ(any term # terms)
= θ(f(n) log n)
Sum = θ(first term)
= θ(f(n)).
log a - c log b < 0log a - c log b = 0log a - c log b < 0
c < log a/log b
c = log a/log b & k<-1c = log a/log b & k>-1c > log a/log b
nlog a/log b
k=-1 logk(n/bi) = 1/(logn - i logb)
1/i (backwards)= nc ∑i=0..h 20 i logk(n/bi)
(back wards)
de
Evaluating: T(n) = aT(n/b)+f(n)Is the sum Geometric?
Simplify by letting f(n) = nc logkn
T(n) = ∑i=0..h aif(n/bi)
= ∑i=0..h ai(n/bi)c logk(n/bi)
= nc ∑i=0..h 2[log a - c log b] · i logk(n/bi)
Geometrically IncreasingArithmetic SumGeometrically Decreasing
Sum = θ(last term)
= θ( )
Sum = θ(any term # terms)
= θ(f(n) log n)
Sum = θ(first term)
= θ(f(n)).
log a - c log b < 0log a - c log b = 0log a - c log b < 0
c < log a/log b
c = log a/log b & k<-1c = log a/log b & k>-1c > log a/log b
nlog a/log b
= nc ∑i=0..h 2+d i logk(n/bi)
Evaluating: T(n) = 4T(n/2)+ n3/log5n
Time for top level?:
Evaluating: T(n) = 4T(n/2)+ n3/log5n
Time for top level: f(n) = n3/log5n
Time for base cases?:
Evaluating: T(n) = 4T(n/2)+ n3/log5n
Time for top level: f(n) = n3 /log5n
Time for base cases:
Evaluating: T(n) = 4T(n/2)+ n3/log5n
θ(n ) =log a/log b θ(n )
log ?/log ?
Time for top level: f(n) = n3 /log5n
Time for base cases:
Evaluating: T(n) = 4T(n/2)+ n3/log5n
θ(n ) =log a/log b θ(n ) = ?
log 4/log 2
Time for top level: f(n) = n3/log5n
Time for base cases:
Dominated?: c = ?
Evaluating: T(n) = 4T(n/2)+ n3/log5n
θ(n ) =log a/log b θ(n ) = θ(n2)
log 4/log 2
log a/log b = ?
Time for top level: f(n) = n3/log5n
Time for base cases:
Dominated?: c = 3 > 2 = log a/log b
Evaluating: T(n) = 4T(n/2)+ n3/log5n
θ(n ) =log a/log b θ(n ) = θ(n2)
log 4/log 2
Hence, T(n) = ?
Time for top level: f(n) = n3/log5n
Time for base cases:
Dominated?: c = 3 > 2 = log a/log b
Evaluating: T(n) = 4T(n/2)+ n3/log5n
θ(n ) =log a/log b θ(n ) = θ(n2)
log 4/log 2
Hence, T(n) = θ(top) = θ(f(n)) = θ(n3/log5n).
Time for top level: f(n) = 2n
Time for base cases:
Dominated?: c = ? > 2 = log a/log b
Evaluating: T(n) = 4T(n/2)+ 2n
θ(n ) =log a/log b θ(n ) = θ(n2)
log 4/log 2
Hence, T(n) = θ(top) = θ(f(n)) = θ(2n).
bigger
>big
The time is even more dominated by the top level of recursion
Time for top level: f(n) = n log5n
Time for base cases:
Dominated?: c = ? 2 = log a/log b
Evaluating: T(n) = 4T(n/2)+ n log5n
θ(n ) =log a/log b θ(n ) = θ(n2)
log 4/log 2
Hence, T(n) = ?
1<
= θ(base cases) = θ(n ) = θ(n2).log a/log b
Time for top level: f(n) = log5n
Time for base cases:
Dominated?: c = ? 2 = log a/log b
Evaluating: T(n) = 4T(n/2)+ log5n
θ(n ) =log a/log b θ(n ) = θ(n2)
log 4/log 2
Hence, T(n) = ?
0<
= θ(base cases) = θ(n ) = θ(n2).log a/log b
Time for top level: f(n) = n2
Time for base cases:
Dominated?: c = ? 2 = log a/log b
Evaluating: T(n) = 4T(n/2)+ n2
θ(n ) =log a/log b θ(n ) = θ(n2)
log 4/log 2
Hence, T(n) = ?
2=
= θ(f(n) log(n) ) = θ(n2 log(n)).
Time for top level: f(n) = n2 log5n
Time for base cases:
Dominated?: c = 2 = 2 = log a/log b
Evaluating: T(n) = 4T(n/2)+ n2 log5n
θ(n ) =log a/log b θ(n ) = θ(n2)
log 4/log 2
k = ?5 ?
Hence, T(n) = ?
> -1
= θ(f(n) log(n) ) = θ(n2 log6(n)).
Time for top level: f(n) = n2/log5n
Time for base cases:
Dominated?: c = 2 = 2 = log a/log b
Evaluating: T(n) = 4T(n/2)+ n2/log5n
θ(n ) =log a/log b θ(n ) = θ(n2)
log 4/log 2
k = ?
Hence, T(n) = ?
-5 ?< -1
= θ(base cases) = θ(n ) = θ(n2).log a/log b
Time for top level: f(n) = n2/log n
Time for base cases:
Dominated?: c = 2 = 2 = log a/log b
Evaluating: T(n) = 4T(n/2)+ n2/log n
θ(n ) =log a/log b θ(n ) = θ(n2)
log 4/log 2
k = ?
Hence, T(n) = ?
-1 ?= -1
= θ(nc loglog n) = θ(n2 loglog n).Did not do before.
Sufficiently close to: T(n) = 4T(n/2)+ n3 = θ(n3). T2(n) = ?
Evaluating: T2(n) = 4 T2(n/2+n1/2)+ n3
θ(n3).
Evaluating: T(n) = aT(n-b)+f(n)
h = ? n-hb
n-ib
n-2b
n-b
T(0)
f(n-ib)
f(n-2b)
f(n-b)
f(n)n
Work in Level
# stack frames
Work
in stack
frame
Instance
size
a
ai
a2
a
1
n/b a · T(0)
ai · f(n-ib)
a2 · f(n-2b)
a · f(n-b)
1 · f(n)
n/b
|base case| = 0 = n-hb
h = n/b
h = n/b
i
2
1
0
Level
Likely dominated by base cases Exponential
Evaluating: T(n) = 1T(n-b)+f(n)
h = ?
Work in Level
# stack frames
Work
in stack
frame
Instance
size
1
1
1
1
1
f(0)
f(n-ib)
f(n-2b)
f(n-b)
f(n)
n-b
n
n-hb
n-ib
n-2b
T(0)
f(n-ib)
f(n-2b)
f(n-b)
f(n)
h = n/b
i
2
1
0
Level
Total Work T(n) = ∑i=0..h f(b·i) = θ(f(n)) or θ(n·f(n))
Math Review: Exponents
XA XB = XA+B (not XAB !!)
XA / XB = XA-B
(XA )B = XAB
XA + XA = 2XA
2A + 2A = 2A+1
Logarithms
• Definition: XA = B if and only if logXB = A (x is the “base” of the logarithm).– Example: 102 = 100 means log10100 = 2.
• TheoremslogXAB = logXA + logXB
logXA/B = logXA – logXB
logX AB = B logXA
Logarithms…
• Theorem: logAB = logCB / logCAProof: Let X = logCB, Y = logCA, and
Z = logAB. By the definition of logarithm: CX = B, CY = A, and AZ = B.
Thus CX = B = AZ = CYZ , X = YZ, Z = X/Y.
Logarithms…
• The notation for logs can be confusing. There are two alternatives:– log2(x) = log (log x) or– log2(x) = (log x)2
• Generally, we use the 2nd definition.• Note: log2(x) is not (log x2)• Note: log is not a multiplicative entity, it is
a function.
End