Complexity 5-1 Complexity Andrei Bulatov Complexity of Problems.
Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.
-
Upload
markus-keane -
Category
Documents
-
view
218 -
download
1
Transcript of Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.
![Page 1: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/1.jpg)
Copyright © Zeph Grunschlag, 2001-2002.
Algorithms and Complexity
Zeph Grunschlag
![Page 2: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/2.jpg)
L8 2
AgendaSection 2.1: Algorithms Pseudocode Recursive Algorithms (Section 3.4)
Section 2.2: Complexity of AlgorithmsSection 1.8: Growth of Functions Big-O Big- (Omega) Big- (Theta)
![Page 3: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/3.jpg)
L8 3
Section 2.1Algorithms and
Pseudocode
DEF: An algorithm is a finite set of precise instructions for performing a computation or solving a problem.
Synonyms for a algorithm are: program, recipe, procedure, and many others.
![Page 4: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/4.jpg)
L8 4
Pseudo-JavaPossible alternative to text’s pseudo-Java
Start with “real” Java and simplify:int f(int[] a){int x = a[0];for(int i=1; i<a.length; i++){
if(x > a[i])x = a[i];
}return x;
}
![Page 5: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/5.jpg)
L8 5
Pseudo-JavaVersion 1
integer f(integer_array (a1, a2, …, an) ){
x = a1
for(i =2 to n){if(x > ai)
x = ai
}return x
}
![Page 6: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/6.jpg)
L8 6
Pseudo-Javaversion 2
INPUT: integer_array V = (a1, a2, …, an)
beginx = a1
for(y V)if(x > y)
x = yendOUTPUT: x
![Page 7: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/7.jpg)
L8 7
Algorithm for Surjectivityboolean isOnto( function f: (1, 2,…, n) (1, 2,…, m) ){
if( m > n ) return false // can’t be ontosoFarIsOnto = truefor( j = 1 to m ){
soFarIsOnto = falsefor(i = 1 to n ){
if ( f(i ) == j )soFarIsOnto = true
if( !soFarIsOnto ) return false;}
}return true;
}
![Page 8: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/8.jpg)
L8 8
Improved Algorithm for Surjectivity
boolean isOntoB( function f: (1, 2,…, n) (1, 2,…, m) ){if( m > n ) return false // can’t be ontofor( j = 1 to m )
beenHit[ j ] = false; // does f ever output j ? for(i = 1 to n )
beenHit[ f(i ) ] = true;for(j = 1 to m )
if( !beenHit[ j ] ) return false;
return true;}
![Page 9: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/9.jpg)
L8 9
Recursive Algorithms(Section 3.4)
“Real” Java:
long factorial(int n){
if (n<=0) return 1;
return n*factorial(n-1);
}
![Page 10: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/10.jpg)
L8 10
Recursive Algorithmslong factorial(int n){
if (n<=0) return 1;
return n*factorial(n-1);
}
Compute 5!
![Page 11: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/11.jpg)
L8 11
Recursive Algorithmslong factorial(int n){
if (n<=0) return 1;
return n*factorial(n-1);
}
f(5)=5·f(4)
![Page 12: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/12.jpg)
L8 12
Recursive Algorithmslong factorial(int n){
if (n<=0) return 1;
return n*factorial(n-1);
}
f(4)=4·f(3)
f(5)=5·f(4)
![Page 13: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/13.jpg)
L8 13
Recursive Algorithmslong factorial(int n){
if (n<=0) return 1;
return n*factorial(n-1);
}
f(3)=3·f(2)
f(4)=4·f(3)
f(5)=5·f(4)
![Page 14: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/14.jpg)
L8 14
Recursive Algorithmslong factorial(int n){
if (n<=0) return 1;
return n*factorial(n-1);
}
f(2)=2·f(1)
f(3)=3·f(2)
f(4)=4·f(3)
f(5)=5·f(4)
![Page 15: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/15.jpg)
L8 15
Recursive Algorithmslong factorial(int n){
if (n<=0) return 1;
return n*factorial(n-1);
}
f(1)=1·f(0)
f(2)=2·f(1)
f(3)=3·f(2)
f(4)=4·f(3)
f(5)=5·f(4)
![Page 16: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/16.jpg)
L8 16
Recursive Algorithmslong factorial(int n){
if (n<=0) return 1;
return n*factorial(n-1);
}
f(0)=1
f(1)=1·f(0)
f(2)=2·f(1)
f(3)=3·f(2)
f(4)=4·f(3)
f(5)=5·f(4)
![Page 17: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/17.jpg)
L8 17
Recursive Algorithmslong factorial(int n){
if (n<=0) return 1;
return n*factorial(n-1);
}
1·1=1
f(2)=2·f(1)
f(3)=3·f(2)
f(4)=4·f(3)
f(5)=5·f(4)
![Page 18: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/18.jpg)
L8 18
Recursive Algorithmslong factorial(int n){
if (n<=0) return 1;
return n*factorial(n-1);
}
2·1=2
f(3)=3·f(2)
f(4)=4·f(3)
f(5)=5·f(4)
![Page 19: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/19.jpg)
L8 19
Recursive Algorithmslong factorial(int n){
if (n<=0) return 1;
return n*factorial(n-1);
}
3·2=6
f(4)=4·f(3)
f(5)=5·f(4)
![Page 20: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/20.jpg)
L8 20
Recursive Algorithmslong factorial(int n){
if (n<=0) return 1;
return n*factorial(n-1);
}
4·6=24
f(5)=5·f(4)
![Page 21: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/21.jpg)
L8 21
Recursive Algorithmslong factorial(int n){
if (n<=0) return 1;
return n*factorial(n-1);
}
5·24=
120
![Page 22: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/22.jpg)
L8 22
Recursive Algorithmslong factorial(int n){
if (n<=0) return 1;
return n*factorial(n-1);
}
Return 5! = 120
![Page 23: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/23.jpg)
L8 23
Section 2.2Algorithmic Complexity
Compare the running time of 2 previous algorithms for testing surjectivity.
Measure running time by counting the number of “basic operations”.
![Page 24: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/24.jpg)
L8 24
Running TimeBasic steps—
Assignment IncrementComparison NegationReturn Random array accessFunction output access etc.
In a particular problem, may tell you to consider other operations (e.g. multiplication) and ignore all others
![Page 25: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/25.jpg)
L8 25
Running time of 1st algorithm
boolean isOnto( function f: (1, 2,…, n) (1, 2,…, m) ){if( m > n ) return false soFarIsOnto = truefor( j = 1 to m ){ soFarIsOnto = false for(i = 1 to n ){ if ( f(i ) == j ) soFarIsOnto = true if( !soFarIsOnto ) return false }}return true;
}
1 step OR:1 step (assigment)m loops: 1 increment plus 1 step (assignment) n loops: 1 increment plus 1 step possibly leads to:
1 step (assignment) 1 step possibly leads to:
1 step (return)
possibly 1 step
![Page 26: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/26.jpg)
L8 26
Running time of 1st algorithm
1 step (m>n) OR:1 step (assigment)m loops: 1 increment plus 1 step (assignment) n loops: 1 increment plus 1 step possibly leads to:
1 step (assignment) 1 step possibly leads to:
1 step (return) possibly 1 step
WORST-CASE running time:
Number of steps = 1 OR 1+1 +m ·(1+ 1 + n ·
(1+1 + 1
+ 1 + 1)
+ 1)
= 1 (if m>n) OR 5mn+3m+2
![Page 27: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/27.jpg)
L8 27
Running time of 2nd algorithm
boolean isOntoB( function f: (1, 2,…, n) (1, 2,…, m) ){ if( m > n ) return false for( j = 1 to m )
beenHit[ j ] = falsefor(i = 1 to n )
beenHit[ f(i ) ] = truefor(j = 1 to m )
if( !beenHit[ j ] ) return false
return true}
1 step OR:m loops: 1 increment plus
1 step (assignment)n loops: 1 increment plus
1 step (assignment)m loops: 1 increment plus
1 step possibly leads to:
1 step possibly 1 step
.
![Page 28: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/28.jpg)
L8 28
Running time of 2nd algorithm
1 step (m>n) OR:
m loops: 1 increment plus 1 step (assignment)
n loops: 1 increment plus 1 step (assignment)
m loops: 1 increment plus 1 step possibly leads to:
1 step possibly 1 step
.
WORST-CASE running time:
Number of steps = 1 OR 1+m · (1+ 1)
+ n · (1+ 1 )
+ m · (1+ 1 + 1)
+ 1= 1 (if m>n) OR 5m + 2n +
2
![Page 29: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/29.jpg)
L8 29
Comparing Running Times1. At most 5mn+3m+2 for first algorithm2. At most 5m+2n+2 for second algorithmWorst case when m n so replace m by n:
5n 2+3n+2 vs. 8n+2To tell which is better, look at dominant
term:
5n 2+3n+2 vs. 8n+2
So second algorithm is better.
![Page 30: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/30.jpg)
L8 30
Comparing Running Times.Issues
1. 5n 2+3n+2 , 8n+2 are more than just their biggest term. Consider n = 1.
2. Number of “basic steps” doesn’t give accurate running time.
3. Actual running time depends on platform.
4. Overestimated number of steps: under some conditions, portions of code will not be seen.
![Page 31: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/31.jpg)
L8 31
Running Times IssuesBig-O Response
Asymptotic notation (Big-O, Big- , Big-) gives partial resolution to problems:
1. For large n the largest term dominates so 5n 2+3n+2 is modeled by just n 2.
![Page 32: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/32.jpg)
L8 32
Running Times IssuesBig-O Response
Asymptotic notation (Big-O, Big- , Big-) gives partial resolution to problems:
2. Different lengths of basic steps, just change 5n 2 to Cn 2 for some constant, so doesn’t change largest term
![Page 33: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/33.jpg)
L8 33
Running Times IssuesBig-O Response
Asymptotic notation (Big-O, Big- , Big-) gives partial resolution to problems:
3. Basic operations on different (but well-designed) platforms will differ by a constant factor. Again, changes 5n 2 to Cn 2 for some constant.
![Page 34: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/34.jpg)
L8 34
Running Times IssuesBig-O Response
Asymptotic notation (Big-O, Big- , Big-) gives partial resolution to problems:
4. Even if overestimated by assuming iterations of while-loops that never occurred, may still be able to show that overestimate only represents different constant multiple of largest term.
![Page 35: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/35.jpg)
L8 35
Worst Case vs. Average Case
Worst case complexity: provides absolute guarantees for time a program will run. The worst case complexity as a function of n is longest possible time for any input of size n.
Average case complexity: suitable if small function is repeated often or okay to take a long time –very rarely. The average case as a function of n is the avg. complexity over all possible inputs of that length.
Avg. case complexity analysis usually requires probability theory. (Delayed till later)
![Page 36: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/36.jpg)
L8 36
Section 1.8Big-O, Big-, Big-
Useful for computing algorithmic complexity, i.e. the amount of time that it takes for computer program to run.
![Page 37: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/37.jpg)
L8 37
Notational IssuesBig-O notation is a way of comparing
functions. Notation unconventional:EG: 3x 3 + 5x 2 – 9 = O (x 3)Doesn’t mean “3x 3 + 5x 2 – 9 equals the function O (x
3)” Which actually means
“3x 3+5x 2 –9 is dominated by x 3”Read as: “3x 3+5x 2 –9 is big-Oh of x 3”
![Page 38: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/38.jpg)
L8 38
Intuitive Notion of Big-O
Asymptotic notation captures behavior of functions for large values of x.
EG: Dominant term of 3x 3+5x 2 –9 is x 3. As x becomes larger and larger, other terms become insignificant and only x 3 remains in the picture:
![Page 39: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/39.jpg)
L8 39
Intuitive Notion of Big-Odomain – [0,2]
y = 3x 3+5x 2 –9
y = x 3
y = x
y = x 2
![Page 40: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/40.jpg)
L8 40
Intuitive Notion of Big-Odomain – [0,5]
y = 3x 3+5x 2 –9
y = x 3
y = x
y = x 2
![Page 41: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/41.jpg)
L8 41
Intuitive Notion of Big-Odomain – [0,10]
y = 3x 3+5x 2 –9
y = x 3
y = xy = x 2
![Page 42: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/42.jpg)
L8 42
Intuitive Notion of Big-Odomain – [0,100]
y = 3x 3+5x 2 –9
y = x 3
y = xy = x 2
![Page 43: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/43.jpg)
L8 43
Intuitive Notion of Big-OIn fact, 3x 3+5x 2 –9 is smaller than
5x 3 for large enough values of x:
y = 3x 3+5x 2 –9
y = 5x 3
y = xy = x 2
![Page 44: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/44.jpg)
L8 44
Big-O. Formal Definitionf (x ) is asymptotically dominated by g (x )
if there’s a constant multiple of g (x ) bigger than f (x ) as x goes to infinity:
DEF: Let f , g be functions with domain R0 or N and codomain R. If there are constants C and k such
x > k, |f (x )| C |g (x )|then we write:
f (x ) = O ( g (x ) )
![Page 45: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/45.jpg)
L8 45
Common Misunderstanding
It’s true that 3x 3 + 5x 2 – 9 = O (x 3) as we’ll prove shortly. However, also true are: 3x 3 + 5x 2 – 9 = O (x 4) x 3 = O (3x 3 + 5x 2 – 9) sin(x) = O (x 4)
NOTE: C.S. usage of big-O typically involves mentioning only the most dominant term.
“The running time is O (x 2.5)”Mathematically big-O is more subtle.
![Page 46: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/46.jpg)
L8 46
Big-O. Example
EG: Show that 3x 3 + 5x 2 – 9 = O (x 3).
Previous graphs show C = 5 good guess.
Find k so that 3x 3 + 5x 2 – 9 5x 3
for x > k
![Page 47: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/47.jpg)
L8 47
EG: Show that3x 3 + 5x 2 – 9 = O (x 3).
Find k so that 3x 3 + 5x 2 – 9 5x 3
for x > k 1. Collect terms: 5x 2 ≤ 2x 3 + 9
![Page 48: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/48.jpg)
L8 48
EG: Show that3x 3 + 5x 2 – 9 = O (x 3).
Find k so that 3x 3 + 5x 2 – 9 5x 3
for x > k 1. Collect terms: 5x 2 ≤ 2x 3 + 92. What k will make 5x 2 ≤ x 3 for x
> k ?
![Page 49: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/49.jpg)
L8 49
EG: Show that3x 3 + 5x 2 – 9 = O (x 3).
Find k so that 3x 3 + 5x 2 – 9 5x 3
for x > k 1. Collect terms: 5x 2 ≤ 2x 3 + 92. What k will make 5x 2 ≤ x 3 for x
> k ?3. k = 5 !
![Page 50: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/50.jpg)
L8 50
EG: Show that3x 3 + 5x 2 – 9 = O (x 3).
Find k so that 3x 3 + 5x 2 – 9 5x 3
for x > k 1. Collect terms: 5x 2 ≤ 2x 3 + 92. What k will make 5x 2 ≤ x 3 for x
> k ?3. k = 5 !4. So for x > 5, 5x 2 ≤ x 3 ≤ 2x 3 + 9
![Page 51: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/51.jpg)
L8 51
EG: Show that3x 3 + 5x 2 – 9 = O (x 3).Find k so that
3x 3 + 5x 2 – 9 5x 3
for x > k 1. Collect terms: 5x 2 ≤ 2x 3 + 92. What k will make 5x 2 ≤ x 3 for x >
k ?3. k = 5 !4. So for x > 5, 5x 2 ≤ x 3 ≤ 2x 3 + 95. Solution: C = 5, k = 5 (not unique!)
![Page 52: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/52.jpg)
L8 52
EG: Show that3x 3 + 5x 2 – 9 = O (x 3).Find k so that
3x 3 + 5x 2 – 9 5x 3
for x > k 1. Collect terms: 5x 2 ≤ 2x 3 + 92. What k will make 5x 2 ≤ x 3 for x >
k ?3. k = 5 !4. So for x > 5, 5x 2 ≤ x 3 ≤ 2x 3 + 95. Solution: C = 5, k = 5 (not unique!)
![Page 53: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/53.jpg)
L8 53
Big-O. Negative Examplex 4 O (3x 3 + 5x 2 – 9) :No pair C, k exist for which x > k
implies C (3x 3 + 5x 2 – 9) x 4 Argue using limits:
x 4 always catches up regardless of C. �
)/9/53(lim
)953(lim
323
4
xxC
x
xxC
xxx
x
CC
xxxlim
3
1
)003(lim
![Page 54: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/54.jpg)
L8 54
Big-O and limitsLEMMA: If the limit as x of the
quotient |f (x) / g (x)| exists then f (x ) = O ( g (x ) ).
EG: 3x 3 + 5x 2 – 9 = O (x 3 ). Compute:
…so big-O relationship proved.
31
/9/53lim
953lim
3
3
23
xx
x
xxxx
![Page 55: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/55.jpg)
L8 55
Little-o and limitsDEF: If the limit as x of the
quotient |f (x) / g (x)| = 0 then f (x ) = o (g (x ) ).
EG: 3x 3 + 5x 2 – 9 = o (x 3.1 ). Compute:
01
/9/5/3lim
953lim
1.31.11.0
1.3
23
xxx
x
xxxx
![Page 56: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/56.jpg)
L8 56
Big- and Big-Big-: reverse of big-O. I.e.
f (x ) = (g (x )) g (x ) = O (f (x ))so f (x ) asymptotically dominates g (x ).Big-: domination in both directions. I.e.
f (x ) = (g (x ))
f (x ) = O (g (x )) f (x ) = (g (x ))Synonym for f = (g): “f is of order g ”
![Page 57: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/57.jpg)
L8 57
Useful facts
Any polynomial is big- of its largest term EG: x 4/100000 + 3x 3 + 5x 2 – 9 = (x
4)
The sum of two functions is big-O of the biggest EG: x 4 ln(x ) + x 5 = O (x 5)
Non-zero constants are irrelevant: EG: 17x 4 ln(x ) = O (x 4 ln(x ))
![Page 58: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/58.jpg)
L8 58
Big-O, Big-, Big-. Examples
Q: Order the following from smallest to largest asymptotically. Group together all functions which are big- of each other:
xex xxexxx
xxxxx ,,,13,1
13,1,,ln,sin
xxxxxxxx 2220 lg,)(ln,ln),102)(sin(
![Page 59: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/59.jpg)
L8 59
Big-O, Big-, Big-. ExamplesA:
1.2.3. , (change of base formula)4. 5.6.7.8.9.10.
xe)102)(sin( 20 xxx
x1
xlnx113x2lg
xxxxx 13,,sin
ex
xx ln2)(ln xx
xx
![Page 60: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/60.jpg)
L8 60
Incomparable Functions
Given two functions f (x ) and g (x ) it is not always the case that one dominates the other so that f and g are asymptotically incomparable.
E.G:f (x) = |x 2 sin(x)| vs. g (x) = 5x 1.5
![Page 61: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/61.jpg)
L8 61
Incomparable Functions
0 5 10 15 20 25 30 35 40 45 500
500
1000
1500
2000
2500
y = |x 2 sin(x)|
y = x 2
y = 5x 1.5
![Page 62: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/62.jpg)
L8 62
Incomparable Functions
0 20 40 60 80 100 120 140 160 180 2000
0.5
1
1.5
2
2.5
3
3.5
4x 10
4
y = |x 2 sin(x)|
y = x 2
y = 5x 1.5
![Page 63: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/63.jpg)
L8 63
Big-OA Grain of Salt
Big-O notation gives a good first guess for deciding which algorithms are faster. In practice, the guess isn’t always correct.
Consider time functions n 6 vs. 1000n 5.9. Asymptotically, the second is better. Often catch such examples of purported advances in theoretical computer science publications. The following graph shows the relative performance of the two algorithms:
![Page 64: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/64.jpg)
L8 64
Big-OA Grain of SaltRunning-time
In days
Input size n
T(n) = n 6
T(n) = 1000n 5.9
Assuming each operationtakes a nano-second, socomputer runs at 1 GHz
![Page 65: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/65.jpg)
L8 65
Big-OA Grain of Salt
In fact, 1000n 5.9 only catches up to n 6 when 1000n 5.9 = n 6, i.e.:
1000= n 0.1, i.e.:n = 100010 = 1030 operations
= 1030/109 = 1021 seconds 1021/(3x107) 3x1013 years 3x1013/(2x1010)
1500 universe lifetimes!
![Page 66: Copyright © Zeph Grunschlag, 2001-2002. Algorithms and Complexity Zeph Grunschlag.](https://reader036.fdocuments.us/reader036/viewer/2022062318/551c4d14550346a66a8b4a35/html5/thumbnails/66.jpg)
L8 66
Example for Section 1.8
Link to example proving big-Omega of a sum.