LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic...
Transcript of LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic...
![Page 1: LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic Notation • Basic Data Structures . 9/15/08 S. Raskhodnikova and A. Smith. Based](https://reader030.fdocuments.us/reader030/viewer/2022041117/5f2d42f9bf802d2385671c04/html5/thumbnails/1.jpg)
8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Adam Smith
Algorithm Design and Analysis
LECTURE 3 • Asymptotic Notation • Basic Data Structures
![Page 2: LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic Notation • Basic Data Structures . 9/15/08 S. Raskhodnikova and A. Smith. Based](https://reader030.fdocuments.us/reader030/viewer/2022041117/5f2d42f9bf802d2385671c04/html5/thumbnails/2.jpg)
9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson
Asymptotic notation
We write f(n) = O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0.
O-notation (upper bounds):
EXAMPLE: 2n2 = O(n3)
functions, not values
(c = 1, n0 = 2) funny, “one-way” equality
![Page 3: LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic Notation • Basic Data Structures . 9/15/08 S. Raskhodnikova and A. Smith. Based](https://reader030.fdocuments.us/reader030/viewer/2022041117/5f2d42f9bf802d2385671c04/html5/thumbnails/3.jpg)
9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson
Set Definition
O(g(n)) = { f(n) : there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0 }
EXAMPLE: 2n2 ∈ O(n3) (Logicians: λn.2n2 ∈ O(λn.n3), but it’s convenient to be sloppy, as long as we understand what’s really going on.)
![Page 4: LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic Notation • Basic Data Structures . 9/15/08 S. Raskhodnikova and A. Smith. Based](https://reader030.fdocuments.us/reader030/viewer/2022041117/5f2d42f9bf802d2385671c04/html5/thumbnails/4.jpg)
Examples
• 106 n3+ 2n2 -n +10 = O(n3)
• n½ + log(n) = O(n½)
• n (log(n) + n½) = O(n3/2)
• n = O(n2)
8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
![Page 5: LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic Notation • Basic Data Structures . 9/15/08 S. Raskhodnikova and A. Smith. Based](https://reader030.fdocuments.us/reader030/viewer/2022041117/5f2d42f9bf802d2385671c04/html5/thumbnails/5.jpg)
9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson
Ω-notation (lower bounds)
Ω(g(n)) = { f(n) : there exist constants c > 0, n0 > 0 such that 0 ≤ cg(n) ≤ f(n) for all n ≥ n0 }
EXAMPLE: (c = 1, n0 = 16)
O-notation is an upper-bound notation. It makes no sense to say f(n) is at least O(n2).
![Page 6: LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic Notation • Basic Data Structures . 9/15/08 S. Raskhodnikova and A. Smith. Based](https://reader030.fdocuments.us/reader030/viewer/2022041117/5f2d42f9bf802d2385671c04/html5/thumbnails/6.jpg)
8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Ω-notation (lower bounds) • Be careful: “Any comparison-based sorting algorithm requires at least O(n log n) comparisons.”
– Meaningless! – Use Ω for lower bounds.
![Page 7: LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic Notation • Basic Data Structures . 9/15/08 S. Raskhodnikova and A. Smith. Based](https://reader030.fdocuments.us/reader030/viewer/2022041117/5f2d42f9bf802d2385671c04/html5/thumbnails/7.jpg)
9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson
Θ-notation (tight bounds)
Θ(g(n)) = O(g(n)) ∩ Ω(g(n))
EXAMPLE:
Polynomials are simple: ad nd + ad–1nd–1 + + a1n + a0 = Θ(nd)
![Page 8: LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic Notation • Basic Data Structures . 9/15/08 S. Raskhodnikova and A. Smith. Based](https://reader030.fdocuments.us/reader030/viewer/2022041117/5f2d42f9bf802d2385671c04/html5/thumbnails/8.jpg)
9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson
o-notation and ω-notation
o(g(n)) = { f(n) : for any constant c > 0, there is a constant n0 > 0 such that 0 ≤ f(n) < cg(n) for all n ≥ n0 }
EXAMPLE: (n0 = 2/c)
O-notation and Ω-notation are like ≤ and ≥. o-notation and ω-notation are like < and >.
2n2 = o(n3)
![Page 9: LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic Notation • Basic Data Structures . 9/15/08 S. Raskhodnikova and A. Smith. Based](https://reader030.fdocuments.us/reader030/viewer/2022041117/5f2d42f9bf802d2385671c04/html5/thumbnails/9.jpg)
9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson
o-notation and ω-notation
ω(g(n)) = { f(n) : for any constant c > 0, there is a constant n0 > 0 such that 0 ≤ cg(n) < f(n) for all n ≥ n0 }
EXAMPLE: (n0 = 1+1/c)
O-notation and Ω-notation are like ≤ and ≥. o-notation and ω-notation are like < and >.
![Page 10: LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic Notation • Basic Data Structures . 9/15/08 S. Raskhodnikova and A. Smith. Based](https://reader030.fdocuments.us/reader030/viewer/2022041117/5f2d42f9bf802d2385671c04/html5/thumbnails/10.jpg)
8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Common Functions: Asymptotic Bounds
• Polynomials. a0 + a1n + … + adnd is Θ(nd) if ad > 0. • Polynomial time. Running time is O(nd) for some constant d independent of the input size n. • Logarithms. log a n = Θ(log b n) for all constants a, b > 0.
For every x > 0, log n = O(nx).
• Exponentials. For all r >1 and all d > 0, nd = O(rn). • Factorial.
grows faster than every exponential
can avoid specifying the base log grows slower than every polynomial
Every exponential grows faster than every polynomial
n! = (!
2!n)!n
e
"n(1 + o(1)) = 2!(n log n)
![Page 11: LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic Notation • Basic Data Structures . 9/15/08 S. Raskhodnikova and A. Smith. Based](https://reader030.fdocuments.us/reader030/viewer/2022041117/5f2d42f9bf802d2385671c04/html5/thumbnails/11.jpg)
8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Sort by asymptotic order of growth
a) n log(n) b) c) log(n) d) n2 e) 2n f) n g) n! h) n1,000,000
i) n1/log(n) j) log(n!)
1. n1/log(n) 2. log(n) 3. 4. n 5. n log(n) = Θ(log(n!)) 6. 7. n2 8. n1,000,000
9. 2n 10. n!
![Page 12: LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic Notation • Basic Data Structures . 9/15/08 S. Raskhodnikova and A. Smith. Based](https://reader030.fdocuments.us/reader030/viewer/2022041117/5f2d42f9bf802d2385671c04/html5/thumbnails/12.jpg)
9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson
Conventions for formulas
Convention: A set in a formula represents an anonymous function in the set.
f(n) = n3 + O(n2) means f(n) = n3 + h(n) for some h(n) ∈ O(n2) .
EXAMPLE: (right-hand side)
![Page 13: LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic Notation • Basic Data Structures . 9/15/08 S. Raskhodnikova and A. Smith. Based](https://reader030.fdocuments.us/reader030/viewer/2022041117/5f2d42f9bf802d2385671c04/html5/thumbnails/13.jpg)
9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson
Convention for formulas
Convention: A set in a formula represents an anonymous function in the set.
n2 + O(n) = O(n2) means for any f(n) ∈ O(n): n2 + f(n) = h(n) for some h(n) ∈ O(n2) .
EXAMPLE: (left-hand side)
![Page 14: LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic Notation • Basic Data Structures . 9/15/08 S. Raskhodnikova and A. Smith. Based](https://reader030.fdocuments.us/reader030/viewer/2022041117/5f2d42f9bf802d2385671c04/html5/thumbnails/14.jpg)
8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Properties
• Transitivity. – If f = O(g) and g = O(h) then f = O(h). – If f = Ω(g) and g = Ω(h) then f = Ω(h). – If f = Θ(g) and g = Θ(h) then f = Θ(h).
• Additivity. – If f = O(h) and g = O(h) then f + g = O(h). – If f = Ω(h) and g = Ω(h) then f + g = Ω(h). – If f = Θ(h) and g = O(h) then f + g = Θ(h).
![Page 15: LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic Notation • Basic Data Structures . 9/15/08 S. Raskhodnikova and A. Smith. Based](https://reader030.fdocuments.us/reader030/viewer/2022041117/5f2d42f9bf802d2385671c04/html5/thumbnails/15.jpg)
Exercise: Show that log(n!) = Θ(n log n)
• Stirling’s formula:
8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
n! = (!
2!n)!n
e
"n(1 + o(1))
log(n!) = log(!
2!n) + n log(n)" n log(e) + log(1 + o(1))! "# $=log(1)+o(1)
since log is continuous
= n%
log(n)" log(e) +log(2!n)
n! "# $constant+o(1)
&+ o(1)
= n(log(n)"O(1)) + o(1)= n log n(1"O( 1
log n )) + o(1)
= n log n(1± o(1)) = !(n log n)
![Page 16: LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic Notation • Basic Data Structures . 9/15/08 S. Raskhodnikova and A. Smith. Based](https://reader030.fdocuments.us/reader030/viewer/2022041117/5f2d42f9bf802d2385671c04/html5/thumbnails/16.jpg)
8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Overview Notation … means … Think… E.g. Lim f(n)/g(n)
f(n)=O(n) 9 c>0, n0>0, 8 n > n0 : 0 · f(n) < cg(n)
Upper bound
100n2 = O(n3)
If it exists, it is < 1
f(n)=Ω(g(n)) 9 c>0, n0>0, 8 n > n0 : 0 · cg(n) < f(n)
Lower bound
n100 = Ω(2n)
If it exists, it is > 0
f(n)=Θ(g(n)) both of the above: f=Ω(g) and f = O(g)
Tight bound log(n!) = Θ(n log n)
If it exists, it is > 0 and < 1
f(n)=o(g(n)) 8 c>0, n0>0, 8 n > n0 : 0 · f(n) < cg(n)
Strict upper bound
n2 = o(2n) Limit exists, =0
f(n)=ω(g(n)) 8 c>0, n0>0, 8 n > n0 : 0 · cg(n) < f(n)
Strict lower bound
n2 = ω(log n)
Limit exists, =1