Intro to Analysis of Algorithms. Algorithm “A sequence of unambiguous instructions for solving a...
-
Upload
reilly-hewes -
Category
Documents
-
view
213 -
download
0
Transcript of Intro to Analysis of Algorithms. Algorithm “A sequence of unambiguous instructions for solving a...
Algorithm
• “A sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.”
• Named for Al Khwarizmi, who laid out basic procedures for arithmetic functions. (Read about him!)
Analysis of Algorithms
• Correctness• Generality• Optimality• Simplicity• Time Efficiency• Space Efficiency
Measuring Efficiency
• What is basic unit to measure input size? (n)• What is basic unit of resource?
– Time: basic unit operation– Space: memory units
• Best, worst, or average case?• Find its efficiency class
• This definition can be translated directly into code – a recursive method.
• How many additions does it take to compute Fn?
Which is better?
function fib2(n)create an array f[0...n]f[0] = 0, f[1] = 1for i = 2...n:
f[i] = f[i-1] + f[i-2]return f[n]
function fib1(n)if n = 0: return 0if n = 1: return 1return fib1(n-1) + fib1(n-2)
• Consider calculating F200. The fib1 method takes over 2138 steps.
• Computers can do several billion instructions per second.
• Suppose we have a supercomputer that does 40 trillion instructions per second.
• Consider calculating F200. The fib1 method takes over 2138 steps.
• Computers can do several billion instructions per second.
• Suppose we have a supercomputer that does 40 trillion instructions per second.
• Even on this machine, fib1(200) takes at least 292 seconds, or 1018 centuries, long after the expected end of our sun!!!
• Consider calculating F200. The fib1 method takes over 2138 steps.
• Computers can do several billion instructions per second.
• Suppose we have a supercomputer that does 40 trillion instructions per second.
• Even on this machine, fib1(200) takes at least 292 seconds, or 1018 centuries, long after the expected end of our sun!!!
• But, fib2(200) would take less than a billionth of a second to compute!!!
106 instructions/sec, runtimes
N O(log N) O(N) O(N log N)
O(N2)
10 0.000003 0.00001 0.000033 0.0001
100 0.000007 0.00010 0.000664 0.1000
1,000 0.000010 0.00100 0.010000 1.0
10,000 0.000013 0.01000 0.132900 1.7 min
100,000 0.000017 0.10000 1.661000 2.78 hr
1,000,000 0.000020 1.0 19.9 11.6 day
1,000,000,000
0.000030 16.7 min 18.3 hr 318 centuries
Some helpful mathematics
• 1 + 2 + 3 + 4 + … + N– N(N+1)/2 = N2/2 + N/2 is O(N2)
• N + N + N + …. + N (total of N times)– N*N = N2 which is O(N2)
• 1 + 2 + 4 + … + 2N – 2N+1 – 1 = 2 x 2N – 1 which is O(2N )
Basics of Efficiency
• Big-oh – upper bound on the efficiency class• Efficiency classes don’t worry with constants• A cubic is worse than a quadratic, quartic
worse than cubic…• Getting big-oh analysis of non-recursive code
is pretty easy
• Maximum(A[1..n]) max <- A[0] for i<- 1 to n
if A[i] > max max <- A[i]
return max
1. What is input size?2. What is unit of time?3. What is big-oh analysis?
• Maximum(A[1..n]) max <- A[0] for i<- 1 to n
if A[i] > max max <- A[i]
return max
1 assignmentn times:
1 comparemaybe 1 assignment1 addition to i
1 return______________________1 + n(3) + 1 = O(3n+2)
• The algorithm is O(3n+2), which is O(n).• We only care about the efficiency class. Why?
• At some point, every parabola (n2) overtakes any line (n). We only really care about large input.
Efficiency classes
• So we really just care about the leading term, which determines the shape of the graph.
• This means for non-recursive algorithms, what’s important is the loops.
Analyze this…
AllUnique(A[1..n])for i<-1 to n
for j<- 1 to nif A[i] = A[j] and i ≠ j return false
return true
Best case?Worst case?Average case?
Analyze this…
AllUnique(A[1..n])for i<-1 to n
for j<- 1 to nif A[i] = A[j] and i ≠ j return false
return trueAverage case?Quit halfway through, O(n*n/2)Still O(n2)Often, Average case = Worst case.
AllUnique(A[1..n])for i<-1 to n
for j<- i to nif A[i] = A[j] return false
return true
n + (n-1) + (n-2) + … + 3 + 2 + 1 = n(n+1)/2
MatrixMultiply(A[nxn], B[nxn])Initialize empty C[nxn]for i<- 1 to n
for j <- 1 to nC[i,j] <- 0for k<- 1 to n
C[i,j] <- C[i,j] + A[i,k]*B[k,j]return C
• MethodA(n)– answer <- 1– for i<- 1 to n/2:
• answer <- answer + i• for i<- 1 to n
– answer <- answer + 1
– return answer• MethodB(n)
– answer <- 1– for i <- 1 to lg n:
• answer = answer + MethodA(n)
return answer
Big-oh Definition
• Let f(n) and g(n) be functions from positive integers to positive reals. We say f = O(g) if there exists some constant c > 0 such that
f(n) ≤ cg(n) for all n.
Big-oh Definition
• Let f(n) and g(n) be functions from positive integers to positive reals. We say f = O(g) if there exists some constant c > 0 such that
f(n) ≤ cg(n) for all n.
Say what?
Big-O• f(n) = O(g(n), think: f(n) ≤ g(n)n = O(n2) because I can find a big number to multiply the parabola by that makes it lie completely above the line. I can’t do that in reverse.
Theta Notation
• Theta = tight bound
• Multiply constants by a function, results are always (asymptotically) above and below.
• What it means: If you have a parabola, you can always find parabolas that stay above and below the given parabola. Same for lines, and other curves. Just tells us the efficiency category.
Practice – True or False?• ½n(n-1) = Ө(n)• if f(n) = Ө g(n), then g(n) =
Ө(f(n))• -n – 100000 = Ө(n)
O - notation• f = O(g) if f grows slower than or the
same as g• That is, asymptotically, g is not below f• g is an upper bound• Think: O is “≤”
Practice – True or False?
• n = O(n2)• n3 = O(n2)• .00000001n3 = O(n2)• 100n + 5 = O(n2)• ½ n(n-1) = O(n2)• n4 +n+1= O(n2)
-Notation
• Opposite of big-oh• f = (g) means f is always above or
equal to g.• It gives an asymptotic lower bound.• Think: is “≥”
o and ω
• f = O(g) means g is above or equal to f
• f = o(g) means g is strictly above f• Also means – there is a better,
tighter bound • ω is analogous for - there is a
tighter bound available
Function Growth Rates
Say that f is Mean that f is Write
“big oh of g” No faster than g, ≤
f = O(g)
“theta g” About as fast as g, =
f = Θ(g)
“omega g” No slower than g, ≥
f = Ω(g)
Polynomials are Easy – What about other functions?
• constant (1): Very few examples of algorithms that don’t grow with input size
• logarithmic (lg n): Usually result of cutting input size by a constant factor each time through the loop
• linear (n): Look at each input element a constant number of times
• nlgn: Divide and conquer• quadratic (n2): Two embedded loops• cubic (n3): Three embedded loops• exponential (2n): Generate all subsets of the input
elements• factorial (n!): Generate all permutations of the
input• nn : Generate all permutations of input, allowing
repetitions
Other standard functions
• Polylog – log3n = (logn)3
• Any log grows slower than any polynomial:logan = o(nb), for any a,b > 0
• Any polynomial grows slower than any exponential with base c > 1.nb = o(cn) for any c > 1
• n! = o(nn)• n! = ω(2n)• lg(n!) = Ө(nlgn)