David Luebke 1 1/6/2016 CS 332: Algorithms Asymptotic Performance.
Asymptotic Analysis of Algorithms (how to evaluate your programming power tools)
description
Transcript of Asymptotic Analysis of Algorithms (how to evaluate your programming power tools)
CS 307 Fundamentals of Computer Science
1
Asymptotic Analysis of Algorithms(how to evaluate your programming power tools)
based on presentation material by
Mike Scott
Honors CS 102
Sept. 4, 2007
CS 307 Fundamentals of Computer Science
2
Algorithmic Analysis A technique used to characterize the
execution behavior of algorithms in a manner independent of a particular platform, compiler, or language.
Abstract away the minor variations and describe the performance of algorithms in a more theoretical, processor independent fashion.
A method to compare speed of algorithms against one another.
CS 307 Fundamentals of Computer Science
3
Different Algorithms Why are the words in a dictionary in
alphabetical order? A brute force approach
– linear search– worst case is (d x N)
Another way– divide and conquer– worst case is (c x log N)
Constants are unknown and largely irrelevant.
CS 307 Fundamentals of Computer Science
4
Big O The most common method and notation for
discussing the execution time of algorithms is "Big O”.
For the alphabetized dictionary the algorithm requires O(log N) steps.
For the unsorted list the algorithm requires O(N) steps.
Big O is the asymptotic execution time of the algorithm.
CS 307 Fundamentals of Computer Science
5
Formal Definition of Big O T(N) is O( F(N) ) if there are positive constants c
and N0 such that T(N) < cF(N) when N > N0
– There is a point N0 such that for all values of N that are past this point, T(N) is bounded by some multiple of F(N).
– Thus if T(N) of the algorithm is O( N2 ) then, ignoring constants, at some point we can bound the running time by a quadratic function of the input size.
– Given a linear algorithm, it is technically correct to say the running time is O(N2). O(N) is a more precise answer as to the Big O bound of a linear algorithm.
CS 307 Fundamentals of Computer Science
6
Big O Examples
3n3 = O(n3) 3n3 + 8 = O(n3) 8n2 + 10n * log(n) + 100n + 1020 = O(n2) 3log(n) + 2n1/2 = O(n1/2) 2100 = O(1) TlinearSearch(n) = O(n)
TbinarySearch(n) = O(log(n))
CS 307 Fundamentals of Computer Science
7
Other Algorithmic Analysis Tools Big Omega T(N) is ( F(N) ) if there are
positive constants c and N0 such that T(N) > cF( N )) when N > N0
– Big O is similar to less than or equal, an upper bound.
– Big Omega is similar to greater than or equal, a lower bound.
Big Theta T(N) is ( F(N) ) if and only if T(N) is O( F(N) )and T( N ) is ( F(N) ).– Big Theta is similar to equals.
CS 307 Fundamentals of Computer Science
8
Relative Rates of GrowthAnalysis
TypeMathematicalExpression
Relative Rates of Growth
Big O T(N) = O( F(N) ) T(N) < F(N)
Big T(N) = ( F(N) ) T(N) > F(N)
Big T(N) = ( F(N) ) T(N) = F(N)
"In spite of the additional precision offered by Big Theta,Big O is more commonly used, except by researchersin the algorithms analysis field" - Mark Weiss
CS 307 Fundamentals of Computer Science
9
What it All Means T(N) is the actual growth rate of the
algorithm. F(N) is the function that bounds the growth
rate.– may be upper or lower bound
T(N) may not equal F(N).– constants and lesser terms ignored because it is
a bounding function
CS 307 Fundamentals of Computer Science
10
Assumptions in Big O Analysis Once found accessing the value of a
primitive is constant timex = y;
mathematical operations are constant timex = y * 5 + z % 3;
if statement: constant time if test and maximum time for each alternative are constantsif( iMySuit ==DIAMONDS || iMySuit == HEARTS)return RED;
elsereturn BLACK;
CS 307 Fundamentals of Computer Science
11
Fixed-Size Loops
Loops that perform a constant number of iteration are considered to execute in constant time. They don't depend on the size of some data set
for(int suit = Card.CLUBS; suit <= Card.SPADES; suit++) { for(int value = Card.TWO; value <= Card.ACE; value++)
{ myCards[cardNum] = new Card(value, suit); cardNum++; }
}
CS 307 Fundamentals of Computer Science
12
Loops That Work on a Data Set Loops like on the previous slide are fairly rare. Normally a loop operates on a data set which can
vary is size. public double minimum(double[] values)
{ int n = values.length; double minValue = values[0]; for(int i = 1; i < n; i++) if(values[i] < minValue) minValue = values[i]; return minValue;}
The number of executions of the loop depends on the length of the array, values. The actual number of executions is (length - 1).
The run time is O(N).
CS 307 Fundamentals of Computer Science
13
Nested Loops
Number of executions?
public void bubbleSort(double[] data){ int n = data.length; for(int i = n - 1; i > 0; i--) for(int j = 0; j < i; j++)
if(data[j] > data[j+1]) { double temp = data[j];
data[j] = data[j + 1]; data[j + 1] = temp;
}}
CS 307 Fundamentals of Computer Science
14
Summing Execution Times If an algorithm’s execution time is N2 + N
then it is said to have O(N2) execution time, not O(N2 + N).
When adding algorithmic complexities the larger value dominates.
Formally, a function f(N) dominates a function g(N) if there exists a constant value n0 such that for all values N > N0 it is the case that g(N) < f(N).
CS 307 Fundamentals of Computer Science
15
Example of Dominance Suppose we go for precision and determine
how fast an algorithm executes based on the number of items in the data set.
x2/10000 + 2x log10 x + 100000
Is it plausible to say the x2 term dominates even though it is divided by 10000?
What if we separate the equation into (x2/10000 ) and (2x log x + 100000)?
CS 307 Fundamentals of Computer Science
16
Summing Execution Times
For large values the x2 term dominates so the algorithm is O(N2).
CS 307 Fundamentals of Computer Science
17
Function Common Name
N! factorial
2N Exponential
Nd, d > 3 Polynomial
N3 Cubic
N2 Quadratic
N N
N log N
N Linear
N Root - n
log N Logarithmic
1 Constant
Ranking of Algorithmic Behaviors
CS 307 Fundamentals of Computer Science
18
Running Times Assume N = 100,000 and processor speed is
1,000,000 operations per second
Function Running Time
2N over 100 years
N3 31.7 years
N2 2.8 hours
N N 31.6 seconds
N log N 1.2 seconds
N 0.1 seconds
N 3.2 x 10-4 seconds
log N 1.2 x 10-5 seconds
CS 307 Fundamentals of Computer Science
19
Graphical Results
CS 307 Fundamentals of Computer Science
20
Determining Big O A DNA problem Is a number prime? Hand actions
– inserting card– determining if card is in Hand– combining Hands– copying Hand– retrieving Card
CS 307 Fundamentals of Computer Science
21
Putting it in Context Why worst-case analysis? Wouldn’t
average-case analysis be more useful? A1: No. E.g., doesn’t matter if a cashier’s
terminal handles 2 transactions per second or 3, as long as it never takes more than 20.
A2: Well, OK, sometimes, yes. However, this often requires much more sophisticated mathematics. In fact, for some common algorithms, nobody has been able to do an average case analysis.