Algorithm Design and Analysis (ADA)
description
Transcript of Algorithm Design and Analysis (ADA)
![Page 1: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/1.jpg)
242-535 ADA: 2. Running Time
1
• Objectiveo to introduce the Big-Oh notation for estimating
the worst case running time of programs
Algorithm Design and Analysis (ADA)
242-535, Semester 1 2017-2018
2. Running Time of
Programs
![Page 2: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/2.jpg)
242-535 ADA: 2. Running Time
2
1. Running Time: T()2. Big-Oh and Approximate Running Time3. Calculating Big-Oh Directly4. Analyzing Function Calls5. Analyzing Recursive Functions6. Towers of Hanoi7. Other Asymptotic Notations
Overview
![Page 3: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/3.jpg)
242-535 ADA: 2. Running Time
3
• What is the running time of this program?
void main() int i, n; scanf("%d", &n); for(i=0; i<n; i++) printf("%d"\n", i);
1. Running Time: T()
continued
![Page 4: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/4.jpg)
242-535 ADA: 2. Running Time
4
• Assume 1 instruction takes 1 ms
Counting Instructions
n value no. of loops Time (ms)1 1 31,000 1,000 3,0001,000,000 1,000,000 3,000,000
![Page 5: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/5.jpg)
242-535 ADA: 2. Running Time
5
• There is no single answer!o the running time depends on the size of the n value
• Instead of a time answer in seconds, we want a time answer which is related to the size of the input.
continued
![Page 6: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/6.jpg)
242-535 ADA: 2. Running Time
6
• For example:o running time T(n) = constant * no this means that as n gets bigger, so does the
program timeo running time is linearly related to the input size
size of n
runningtime
constant * n
![Page 7: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/7.jpg)
242-535 ADA: 2. Running Time
7
• A simple way of writing the running time is:T(n) = 3n
n value no. of loops Time (ms)1 1 31,000 1,000 3,0001,000,000 1,000,000 3,000,000
![Page 8: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/8.jpg)
242-535 ADA: 2. Running Time
8
• A program/algorithm has a running time T(n)o n is some measure of the input size
• T(n) is the largest amount of time the program takes on any input of size n
• T(n) is the worst running timeo not always accurate, but easy to calculate
• Time units are left unspecified.
Running Time Theory
![Page 9: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/9.jpg)
242-535 ADA: 2. Running Time
9
Worst-case: (we use this usually)• T(n) = maximum time of algorithm on any input of size
n.- one possible value
Average-case: (we sometimes use this)• T(n) = expected time of algorithm over all inputs of size
n.- this approach needs info about the statistical distribution (probability) of the inputs.- e.g. uniform spread of data (i.e. all data is equally likely)
Best-case: (don't use this, it's misleading)• e.g. write a slow algorithm that works fast on specially
selected input.
1.1. Kinds of Running Time
![Page 10: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/10.jpg)
242-535 ADA: 2. Running Time
10
• Loop fragment for finding the product of all the positive numbers in the A[] array of size n:
(2) int prod = 1;(3) for(j = 0; j < n; j++)(4) if (A[j] > 0)(5) prod = prod * A[j];
• Count each assignment and test as 1 “time unit”.
1.2. T(n) Example
![Page 11: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/11.jpg)
242-535 ADA: 2. Running Time
11
• The while-loop is easier to count (and equivalent to the for-loop):
int prod = 1; // 1int j = 0; // 1while (j < n) // 1 for the test if (A[j] > 0) // 1 prod = prod*A[j]; // 1 + 1 j++; // 1
Convert 'for' to 'while'
We assume that 1 instructiontakes 1 "time unit"
What aboutcounting theloop?
![Page 12: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/12.jpg)
242-535 ADA: 2. Running Time
12
• The for loop executes n timeso each loop carries out (in the worse case) 5 ops
• test of j < n, if test, multiply, assign, j incremento total loop time = 5no plus 3 ops at start and end
• small assign (line 2), init of j (line 3), final j < n test
• Total time T(n) = 5n + 3o running time is linear with the size of the array
Calculation
![Page 13: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/13.jpg)
242-535 ADA: 2. Running Time
13
• If input size < 50, program B takes less time.• But for large n’s, which are more common in
real code, program B is worse and worse (slower).
1.3. Comparing Different T()’s
T(n)
input size n
TA(n) = 100n
TB(n) = 2n2
![Page 14: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/14.jpg)
242-535 ADA: 2. Running Time
14
• Formula (n = input size) Namen linearn2 quadraticn3 cubicnm polynomial, e.g. n10
mn ( m >= 2) exponential, e.g. 5n
n! factorial1 constantlog n logarithmicn log nlog log n
1.4. Common Growth Formulae &
Names
![Page 15: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/15.jpg)
242-535 ADA: 2. Running Time
15
Growth Examples
0
1000
1 3 5 7 9 11 13 15 17 19
f(n) = n
f(n) = log(n)
f(n) = n log(n)f(n) = n 2
f(n) = n 3
f(n) = 2 n
n3
2n
n2
n log nnlog n
![Page 16: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/16.jpg)
242-535 ADA: 2. Running Time
16
• 3 9 50 100 1000 106
n 3 9 50 100 1ms 1secn2 9 81 2.5ms 10ms 1sec 12 daysn3 27 729 125ms 1sec 16.7 min 31,710yr2n 8 512 36yr 4*1016 yr 3*10287 yr3*10301016yrlog n 2 3 6 7 10 20
1.5. Execution Times
n (no. of instructions)
grow
th fo
rmul
a T(
)
if n is 50, you willwait 36 years for an answer!
Assume 1 instruction takes 1 microsec (10-6 secs) to execute.How long will n instructions take?
![Page 17: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/17.jpg)
242-535 ADA: 2. Running Time
17
• Logarithmic running times are best.
• Polynomial running times are acceptable, if the power isn’t too bigo e.g. n2 is ok, n100 is terrible
• Exponential times mean sloooooooow code.o some size problems may take longer to finish than
the lifetime of the universe!
Notes
![Page 18: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/18.jpg)
242-535 ADA: 2. Running Time
18
• T() can guide our choice of which algorithm to implement, or program to useo e.g. selection sort or merge sort?
• T() helps us look for better algorithms in our own code, without expensive implementation, testing, and measurement.
1.6. Why use T(n)?
![Page 19: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/19.jpg)
242-535 ADA: 2. Running Time
19
• Algorithms often perform much better on average than the worst case used in T()o quicksort is n log n on a “random” array, but n2 in the
worse case
o but for most algorithms, the worst case is a good predictor of its running time
o average case analyses can be done, but they are harder mathematically
(Wrong) Arguments against T(n)
continued
![Page 20: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/20.jpg)
242-535 ADA: 2. Running Time
20
• Some people say:o “Who cares about running time? In a few years,
machines will be so fast that even bad algorithms will be fast.”
o History shows this argument to be wrong. As machines get faster, problem sizes get bigger.
o Most interesting problems (e.g. computer vision, natural language processing) always require more resources
• fast algorithms will always be needed
continued
![Page 21: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/21.jpg)
242-535 ADA: 2. Running Time
21
• Some people say:o "Benchmarking (running programs on a standard set
of test cases) is easier."
o This is sometimes true, but the benchmarks only give numbers for that particular machine, OS, compiler, computer language.
![Page 22: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/22.jpg)
242-535 ADA: 2. Running Time
22
• We want T() to be the same for an algorithm independent of the machine where it is running.
• This is not true since different machines (and OSes) execute instructions at different speeds.
• Consider the loop example (slide 11)o on machine A, every instruction takes 1 "time unit"o the result is TA(n) = 5n + 3
T() is too Machine Dependent
![Page 23: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/23.jpg)
242-535 ADA: 2. Running Time
23
• On machine B, every instruction takes 1 "time unit" except for multiplication, which takes 5 "time units".
• The for loop executes n timeso each loop carries out (in the worse case) 5 ops
• test of j < n, if test, multiply, assign, j incremento total loop time = 9no plus 3 ops at start and end
• small assign (line 2), init of j (line 3), final j < n test
• Total time TB(n) = 9n + 3o running time is linear with the size of the array
![Page 24: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/24.jpg)
242-535 ADA: 2. Running Time
24
• TA() = 5n + 3 and TB() = 9n +3
• These are both linear equations (which is good), but the constants are different (which is bad)
• We want a T() notation that is independent of machines.
![Page 25: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/25.jpg)
242-535 ADA: 2. Running Time
25
• Big-Oh notation for T(n) ignores constant factors which depend on compiler/machine behaviouro that's good
• Big-Oh simplifies the process of estimating the running time of programso we don't need to count every code lineo that's also good
2. Big-Oh and Running Time
![Page 26: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/26.jpg)
242-535 ADA: 2. Running Time
26
• The Big-Oh value specifies running time independent of:o machine architecture
• e.g. don’t consider the running speed of individual machine operations
o machine load (usage)• e.g. time delays due to other users
o compiler design effects • e.g. gcc versus Visual C
![Page 27: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/27.jpg)
242-535 ADA: 2. Running Time
27
• When we counted instructions for the loop example, we found:o TA() = 5n + 3 o TB() = 9n + 3
• The Big-Oh equation, O(), is based on the T(n) equation but ignores constants (which vary from machine to machine). This means for both machine A and B:
T(n) is O(n)we say "T(n) is order n"
Example
![Page 28: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/28.jpg)
242-535 ADA: 2. Running Time
28
• T(n) value Big Oh : O()o10n2+ 50n+100 O(n2)o (n+1)2 O(n2)o n10 O(2n)o5n3 + 1 O(n3)
• These simplifications have a mathematical reason, which is explained in section 2.2.
More Examples
hard tounderstand
![Page 29: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/29.jpg)
242-535 ADA: 2. Running Time
29
• O() ignores constant factors, which means it is a more general measure of running time for algorithms across different platforms/compilers.
• It can be compared with Big-Oh values for other algorithms.o i.e. linear is better than polynomial and exponential, but
worse than logarithmico i.e. O(log n) < O(n) < O(n2) < O(2n)
2.1. Is Big-Oh Useful?
![Page 30: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/30.jpg)
242-535 ADA: 2. Running Time
30
• The connection between T() and O() is:o the T() = f(n) can be written as T(n) is O( g(n) )o this means that g(n) is the most important thing
in T()'s f(n) function when n is large• Example 1:
o T(n) = 5n + 3 // the f() function is 5n + 3o write as T(n) is O(n) // the g() function is n
• Example 2:o T(n) = 9n + 3 // the f() function is 9n + 3o write as T(n) is O(n) // the g() function is n
2.2. Definition of Big-Oh
continued
![Page 31: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/31.jpg)
242-535 ADA: 2. Running Time
31
• n0 and c are called witnesses to the relationship:T(n) = f(n) and T(n) is O(g(n) )
• In some textbooks this is written as:o f(n) is O(g(n)) // leave out the
T(n)
More Formally
We write T(n) = f(n) as T(n) is O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 f(n) c*g(n) for all n n0.
![Page 32: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/32.jpg)
242-535 ADA: 2. Running Time
32
• g(n) is a simple function of n without constants and additional smaller terms
• e.g. n, n2, 2n, n log n
• Not 5n2, 2n + 4, n3 + n2
• Use n2, 2n , n3
What is g(n)?
![Page 33: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/33.jpg)
242-535 ADA: 2. Running Time
33
• O-notation gives an upper bound for a function to within a constant factor. We write T(n)=f(n) as T(n) is O(g(n)) if there are positive constants n0 and c such that at and to the right of n0, the value of f(n) always lies on or below c*g(n).
O-notation as a Graph
is
abovesimple to calculateand analyse andbigger
complex
![Page 34: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/34.jpg)
242-535 ADA: 2. Running Time
34
• The fancy name for calculating big-Oh (and the other bounds mentioned later) is asymptotic analysis.
• Asymptotic means "a curve whose distance to another curve tends to zero as the curves travel off to infinity"o this is seen by the limit of f(n)/c*g(n) curve, as n ∞:o 0 ≤ f(n) ≤ c*g(n) for all n ≥ n0o so 0 ≤ f(n)/c*g(n) ≤ 1, such that f(n)/c*g(n) a constant
Asymptotic Analysis
![Page 35: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/35.jpg)
242-535 ADA: 2. Running Time
35
• = 1 if d small relative to ho so c g(n) is close to f(n),
or 'tight'; we want this
o or 0 if d is big relative to h; no good, we need a closer g(n)
o or a constant between 0 and 1; this is okay (closer to 1 is better)
d+h
h
d
Limit of f(n)/c g(n)alwaysbigger
![Page 36: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/36.jpg)
242-535 ADA: 2. Running Time
36
• T(n) = 10n2 + 50n + 100o can be written as T(n) is O(n2)
• Why?o f(n) = 10n2 + 50n + 100; g(n) = n2
o Witnesses: n0 = 1, c = 160o then f(n) <= c*g(n), n >= 1
so 10n2 + 50n + 100 <= 160 n2
since10n2 + 50n + 100 <=10n2 + 50n2 + 100n2 <= 160 n2
Example 1Informally, the n2 part is the most
important thing inthe T() function
![Page 37: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/37.jpg)
242-535 ADA: 2. Running Time
37
T() and O() Graphed
T(n) = 10n2 + 50n + 10( f(n) == 10n2 + 50n + 10)
T(n) is O(n2)(c g(n) == 160n2)
n0 == 1
T(n)
n
above
http://dlippman.imathas.com/ graphcalc/graphcalc.html
![Page 38: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/38.jpg)
242-535 ADA: 2. Running Time
38
f(n)/c g(n) constant
(10n2 + 50n + 10) / (160n2)
ny = 1/16 = 0.0625
as n ∞ the equationwill become a constant
• =
![Page 39: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/39.jpg)
242-535 ADA: 2. Running Time
39
• T(n) = (n+1)2
o can be wriiten as T(n) is O(n2)
• Why?o f(n) = (n+1)2; g(n) = n2
o Witnesses: n0 = 1, c = 4o then f(n) <= c*g(n), n >= 1
so (n+1)2 <= 4n2
sincen2 + 2n + 1 <=n2 + 2n2 + n2 <= 4n2
Example 2
![Page 40: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/40.jpg)
242-535 ADA: 2. Running Time
40
T() and O() Graphed
T(n) = (n+1)2
(f(n) == (n+1)2
T(n) is O(n2)(c g(n) == 4n2)
n0 == 1
T(n)
n
above
• =
![Page 41: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/41.jpg)
242-535 ADA: 2. Running Time
41
• T(n) = n10
o can be written as T(n) is O(2n)• Why?
o f(n) = n10 ; g(n) = 2n
o Witnesses: n0 = 64, c = 1o then f(n) <= c*g(n), n >= 64
so n10 <= 2n
since 10*log2 n <= n (by taking logs of both sides) which is true when n >= 64
(10*log2 64 == 10*6; 60 <= 64)
Example 3
![Page 42: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/42.jpg)
242-535 ADA: 2. Running Time
42
n10 and 2n Graphed
(58.770, 4.915E17)
T(n) = n10
f(n) == n10
T(n) is O(2n)(c g(n) == 2n
T(n)
n
above
![Page 43: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/43.jpg)
242-535 ADA: 2. Running Time
43
• Is a running tgime of n10 slow?o not sure
• But the maths shows that T(n) = n10 is about the same as 2n
• We know 2n is very slow, and that means that n10 is very slow also
Is n10 slow?
![Page 44: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/44.jpg)
242-535 ADA: 2. Running Time
44
• When choosing an O() approximation to T(), remember that:o constant factors do not matter
• e.g. T(n) = (n+1)2 is O(n2)
o low-order terms do not matter• e.g. T(n) = 10n2 + 50n + 100 is O(n2)
o there are many possible witnesses because there are usually many O() graphs that are above the T() equation
2.4. Some Observations about O()
![Page 45: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/45.jpg)
242-535 ADA: 2. Running Time
45
• Inside an O() expressions, always drop constant factors and low-order terms.
• For example:o T(n) = 3n5+ 10n4 + no T(n) is O(3n5)o but, T(n) is O(n5) is simpler and tighter
• this means that the O() is closer to the T() curve
2.5. Simplifying O() Expressions
![Page 46: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/46.jpg)
242-535 ADA: 2. Running Time
46
• Up to now, I have calculated T(n) = f(n) by counting instructions (e.g. see the loop example), and then simplified T(n) to becomes T(n) is O(g(n))
• We can calculate the big-oh function, g(), directly, without counting instructionso easier and faster
3. Calculating Big-Oh Directly
![Page 47: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/47.jpg)
242-535 ADA: 2. Running Time
47
• First decide on a size measure for the data in the program. This will become the n.
• Data Type Possible Size Measureinteger its valuestring its lengtharray its length
3. Big-Oh for Programs
![Page 48: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/48.jpg)
242-535 ADA: 2. Running Time
48
• The Big-Oh value for a program is built up in stages by:o 1) Calculate the Big-Oh’s for all the simple statements in
the program• e.g. assignment, arithmetic
o 2) Then use those value to obtain the Big-Oh’s for the complex statements
• e.g. blocks, for loops, if-statements
3.1. Building a Big-Oh Result
![Page 49: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/49.jpg)
242-535 ADA: 2. Running Time
49
• We assume that simple statements always take a constant amount of time to executeo written as O(1)o this is not a time unit (not 1 ms, not 1 microsec)o O(1) means a running time independent of the input size
n
• Kinds of simple statements:o assignment, break, continue, return, all library functions
(e.g. putchar(),scanf()), arithmetic, boolean tests, array indexing
Simple Statements (in C)
![Page 50: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/50.jpg)
242-535 ADA: 2. Running Time
50
• The Big-Oh value for a complex statement is a combination of the Big-Oh values of its component simple statements.
• Kinds of complex statements:o blocks ... o conditionals: if-then-else, switcho loops: for, while, do-while
Complex Statements
continued
![Page 51: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/51.jpg)
242-535 ADA: 2. Running Time
51
• The easiest way to see how complex statement timings are based on simple statements (and other complex statements) is by drawing a structure tree for the program.
3.2. Structure Trees
![Page 52: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/52.jpg)
242-535 ADA: 2. Running Time
52
void main() int i;
(1) scanf(“%d”, &i);(2) while (i > 0) (3) putchar(‘0’ + i%2);(4) i = i/2;
(5) putchar(‘\n’);
Example: binary conversion
![Page 53: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/53.jpg)
242-535 ADA: 2. Running Time
53
Structure Tree for Example
block1-5
1 5while2-4
block3-4
3 4
the time for thisis the time for
(3) + (4)
![Page 54: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/54.jpg)
242-535 ADA: 2. Running Time
54
• Blocks: Running time bound = summation of the bounds of its parts.
• The summation rule means that only the largest Big-Oh value is considered.
3.3. Details for Complex Statements
"summation" means 'add'
![Page 55: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/55.jpg)
242-535 ADA: 2. Running Time
55
Block Calculation Graphically
O( f1(n) )
O( f2(n) )
O( fk(n) )
O( f1(n) + f2(n) + ... + fk(n))
In other words:
O( largest fi(n) )
summation rule
The cost of a block is the cost of the biggest statement.
![Page 56: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/56.jpg)
242-535 ADA: 2. Running Time
56
• First block's time T1(n) = O(n2)• Second block's time T2(n) = O(n)
• Total running time = O(n2 + n)= O(n2)the largest part
Block Summation Rule Example
![Page 57: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/57.jpg)
242-535 ADA: 2. Running Time
57
• Conditionals: Running time bound = the cost of the if-test + larger of the bounds for the
if- and else- parts
• When the if-test is a simple statement (a boolean test), it is O(1).
Conditionalse.g. if statements, switches
![Page 58: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/58.jpg)
242-535 ADA: 2. Running Time
58
Conditional Graphically
Test
ElsePart
IfPart
O(1)
O( max( f1(n), f2(n)) +1 )which is the same asO( max( f1(n), f2(n)) )
O( f1(n) ) O( f2(n) )
![Page 59: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/59.jpg)
242-535 ADA: 2. Running Time
59
• Code fragment:if (x < y) // O(1) foo(x); // O(n)else bar(y); // O(n2)
• Total running time = O( max(n,n2) + 1)= O(n2 + 1)= O(n2)
If Example
Usually the cost ofan if-statement is the cost of the biggestbranch.
![Page 60: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/60.jpg)
242-535 ADA: 2. Running Time
60
• Loops: Running time bound is usually =the max. number of times round the loop *the time to execute the loop body once
• But we must include O(1) for the increment and test each time around the loop.
• Must also include the initialization and final test costs (both O(1)).
Loops
![Page 61: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/61.jpg)
242-535 ADA: 2. Running Time
61
While Graphically
Test
Body
O(1)
O( f(n) )
At mostlp(n) times
aroundO( lp(n)*f(n) )
Altogether this is:O( lp(n)*(f(n)+1) + 1 )
which can be simplified to:O( lp(n)*f(n) )
![Page 62: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/62.jpg)
242-535 ADA: 2. Running Time
62
• Code fragment:x = 0;while (x < n) // O(1) for test foo(x, n); // O(n2) x++; // O(1)
• Total running time of loop:= O( n*( 1 + n2 + 1) + 1 )= O(n3 + 2n + 1) = O(n3)
While Loop Example
Usually the cost ofa loop is the cost of the body * the number of loops.lp(n) = n f(n) = 1 + n2 + 1
![Page 63: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/63.jpg)
242-535 ADA: 2. Running Time
63
Do-While Graphically
Test
Body
O(1)
O( f(n) )
At mostlp(n) times
around
O( lp(n)*f(n) )
Altogether this is:O( lp(n)*(f(n)+1) + 1 )
which can be simplified to:O( g(n)*f(n) )
Usually the cost ofa loop is the cost of the body * the number of loops.
![Page 64: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/64.jpg)
242-535 ADA: 2. Running Time
64
For-loop Graphically
Test
Body
Increment
Initialize
O(1)
O( f(n) )At most
lp(n) timesaround
O( lp(n)*(f(n)+1+1) + 1)which can be simplified to:
O( lp(n)*f(n) )
O(1)
O(1)Usually the cost ofa loop is the cost of the body * the number of loops.
![Page 65: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/65.jpg)
242-535 ADA: 2. Running Time
65
• Code Fragment:for (i=0; i < n; i++) foo(i, n); // O(n2)
• It helps to rewrite this as a while loop:
i=0; // O(1)while (i < n) // O(1) for test foo(i, n); // O(n2) i++; // O(1)
For Loop Example
continued
![Page 66: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/66.jpg)
242-535 ADA: 2. Running Time
66
• Running time for the for loop:= O( 1 + n*( 1 + n2 + 1) + 1 )= O( 2 + n3 + 2n )= O(n3)
Usually the cost ofa loop is the cost of the body * the number of loops.lp(n) = n f(n) = 1 + n2 + 1
![Page 67: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/67.jpg)
242-535 ADA: 2. Running Time
67
(1) for(i=0; i < n; i++)_ (2) for (j = 0; j < n; j++)(3) A[i][j] = 0;
• line (3) is a simple op - takes O(1)• line (2) is a loop carried out n times
o takes O(n *1) = O(n)• line (1) is a loop carried out n times
o takes O(n * n) = O(n2)
3.4.1. Example: nested loops
![Page 68: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/68.jpg)
242-535 ADA: 2. Running Time
68
(1) if (A[0][0] == 0) (2) for(i=0; i < n; i++)_ (3) for (j = 0; j < n; j++)(4) A[i][j] = 0;
(5) else (6) for (i=0; i < n; i++)(7) A[i][i] = 1;
3.4.2. Example: if statement
continued
![Page 69: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/69.jpg)
242-535 ADA: 2. Running Time
69
• The if-test takes O(1);the if block takes O(n2);the else block takes O(n).
• Total running time:= O(1) + O( max(n2, n) )
= O(1) + O(n2)
= O(n2) // using the summation rule
![Page 70: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/70.jpg)
242-535 ADA: 2. Running Time
70
void main() int i;
(1) scanf(“%d”, &i);(2) while (i > 0) (3) putchar(‘0’ + i%2);(4) i = i/2;
(5) putchar(‘\n’);
3.4.3. Time for a Binary Conversion
continued
![Page 71: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/71.jpg)
242-535 ADA: 2. Running Time
71
• Lines 1, 2, 3, 4, 5: each O(1)• Block of 3-4 is O(1) + O(1) = O(1)• While of 2-4 loops at most (log2 i)+1 times
o total running time = O(1 * log2 i+1) = O(log2 i)• Block of 1-5:
= O(1) + O(log2 i) + O(1)= O(log2 i)
why?
![Page 72: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/72.jpg)
242-535 ADA: 2. Running Time
72
• Assume i = 2k
• Start 1st iteration, i = 2k
Start 2nd iteration, i = 2k-1
Start 3rd iteration, i = 2k-2
Start kth iteration, i = 2k-(k-1) = 21 = 2Start k+1th iteration, i = 2k-k = 20 = 1o the while will terminate after this iteration
• Since 2k = i, so k = log2 i• So k+1, the no. of iterations, = (log2 i)+1
Why (log2 i)+1 ?
![Page 73: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/73.jpg)
242-535 ADA: 2. Running Time
73
Product rule
Quotient rule
Power rule
Change of base
Cancelation
Identities
![Page 74: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/74.jpg)
242-535 ADA: 2. Running Time
74
Using a Structure Treeblock1-5
1 5while2-4
block3-4
3 4
O(1)
O(1)O(1)
O(1)
O(1)
O(log2 i)
O(log2 i)
![Page 75: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/75.jpg)
242-535 ADA: 2. Running Time
75
void selectionSort(int A[], int n) int i, j, small, temp;(1) for (i=0; i < n-1; i++) //outer loop(2) current = i;(3) for(j=i+1; j < n; j++) //inner loop(4) if (A[j] < A[small]) (5) current= j;(6) temp = A[current]; // exchange(7) A[current] = A[i];(8) A[i] = temp;
3.4.4. Selection Sort Code
![Page 76: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/76.jpg)
242-535 ADA: 2. Running Time
76
Execution Example
OuterLoop #1
OuterLoop #2
OuterLoop #3
OuterLoop #4
inner loops
when n-1 elemsare sortedwe can stop!
sorted
![Page 77: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/77.jpg)
242-535 ADA: 2. Running Time
77
Selection Sort Structure Tree
for1-8
6 7
block2-8
for3-52
5
if4-5
8
![Page 78: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/78.jpg)
242-535 ADA: 2. Running Time
78
• Lines 2, 5, 6, 7, 8: each is O(1)• If of 4-5 is O(max(1,0)+1) = O(1)• For of 3-5 is O( (n-(i+1))*1) = O(n-i-1)
= O(n), simplified• Block of 2-8
= O(1) + O(n) + O(1) + O(1) + O(1) = O(n)• For of 1-8 is:
= O( (n-1) * n) = O(n2 - n)= O(n2), simplified
if part else part
![Page 79: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/79.jpg)
242-535 ADA: 2. Running Time
79
• In this section, we assume that the functions are not recursiveo we add recursion in section (5)
• Size measures for all the functions must be similar, so they can be combined to give the program’s Big-Oh value.
4. Analyzing Function calls
![Page 80: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/80.jpg)
242-535 ADA: 2. Running Time
80
#include <stdio.h>
int bar(int x, int n); int foo(int x, int n):
void main() int a, n;(1) scanf(“%d”, &n);(2) a = foo(0, n);(3) printf(“%d\n”, bar(a,n));
Example Program
continued
![Page 81: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/81.jpg)
int bar(int x, int n) int i;(4) for(i = 1; i <= n; i++)(5) x += i;(6) return x;
int foo(int x, int n) int i;(7) for(i = 1; i <= n; i++)(8) x += bar(i, n);(9) return x;
![Page 82: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/82.jpg)
242-535 ADA: 2. Running Time
82
Calling Graph
main
foo
bar
![Page 83: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/83.jpg)
242-535 ADA: 2. Running Time
83
• 1. Calculate times for Group 0 functionso those that call no other user functions
• 2. Calculate times for Group 1 functionso those that call Group 0 functions only
• 3. Calculate times for Group 2 functionso those that call Group 0 and Group 1 functions only
• 4. Continue until the time for main() is obtained.
Calculating Times with a Calling Graph
![Page 84: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/84.jpg)
242-535 ADA: 2. Running Time
84
• Group 0: bar() is O(n)
• Group 1: foo() is O( n * n) = O(n2)
• Group 2: main() is= O(1) + O(n2) + O(1) + O(n)
= O(n2)
Example Program Analysis
bar() in body
![Page 85: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/85.jpg)
242-535 ADA: 2. Running Time
85
• Recursive functions call themselves with a “smaller size” argument, and terminate by calling a base case.
int factorial(int n) if (n <= 1) return 1; else return n*factorial(n-1);
5. Analyzing Recursive Functions
![Page 86: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/86.jpg)
242-535 ADA: 2. Running Time
86
• 1. Develop basis and inductive statements for the running time.
• 2. Solve the corresponding recurrence relation.o this usually requires the Big-Oh notation to be
rewritten as constants and multiples of no e.g. O(1) becomes a, O(n) becomes b*n,
O(n2) becomes c*n2, etc.
Running Time for a Recursive Function
continued
this is sometimes called the Iteration Method
![Page 87: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/87.jpg)
242-535 ADA: 2. Running Time
87
• 3. Translate the solved relation back into Big-Oh notationo rewrite the remaining constants back into Big-Oh
formo e.g. a becomes O(1), b*n becomes O(n)
![Page 88: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/88.jpg)
242-535 ADA: 2. Running Time
88
• Step 1.o Basis: T(1) = O(1)o Induction: T(n) = O(1) + T(n-1), for n > 1
• Step 2.o Simplify the relation by replacing the O() notation
with constants.o Basis: T(1) = ao Induction: T(n) = b + T(n-1), for n > 1
5.1. Factorial Running Time
![Page 89: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/89.jpg)
242-535 ADA: 2. Running Time
89
• The simplest way to solve T(n) is to calculate it for some values of n, and then guess the general expression.
T(1) = aT(2) = b + T(1) = b + aT(3) = b + T(2) = 2b + aT(4) = b + T(3) = 3b + a
• “Obviously”, the general form is:T(n) = ((n-1)*b) + a
= bn + (a-b)
continued
![Page 90: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/90.jpg)
242-535 ADA: 2. Running Time
90
• Step 3. Translate back:T(n) = bn + (a-b)
• Replace constants by Big-Oh notation:T(n) = O(n) + O(1)
= O(n)
• The running time for recursive factorial is O(n). That is fast.
![Page 91: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/91.jpg)
242-535 ADA: 2. Running Time
91
void rSSort(int A[], int n) int imax, i; if (n == 1) return; else imax = 0; /* A[0] is biggest */ for (i=1; i < n; i++) if (A[i] > A[imax]) imax = i; swap(A, n-1, imax); rSSort(A, n-1);
5.2. Recursive Selection Sort
![Page 92: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/92.jpg)
242-535 ADA: 2. Running Time
92
• Step 1.o Basis: T(1) = O(1)o Induction: T(n) = O(n-1) + T(n-1), for n > 1
• Step 2.o Basis: T(1) = ao Induction: T(n) = b(n-1) + T(n-1), for n > 1
Running Time
continued
multiple of n-1
theloop call to
rSSort()
Assume swap() is O(1), so ignoren == the size of the array
![Page 93: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/93.jpg)
242-535 ADA: 2. Running Time
93
• Solve the relation:o T(1) = a
T(2) = b + T(1) = b + aT(3) = 2b + T(2) = 2b + b + aT(4) = 3b + T(3) = 3b + 2b + b + a
• General Form:o T(n) = (n-1)b + ... + b + a
= a + b(n-1)n/2
1
0
)(n
i
ibanT
continued
![Page 94: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/94.jpg)
242-535 ADA: 2. Running Time
94
• Step 3. Translate back:T(n) = a + b(n-1)n/2
• Replace constants by Big-Oh notation:T(n) = O(1) + O(n2) + O(n)
= O(n2)
• The running time for recursive selection sort is O(n2). That is slow for large arrays.
![Page 95: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/95.jpg)
242-535 ADA: 2. Running Time
95
int binSrch(char A[], int i,int j, char key) int k; if (i > j) /* key not found */ return -1; k = (i+j)/2; if (key == A[k]) /* key found */ return k; if (key < A[k]) j = k-1; /* search left half */ else i = k+1; /* search right half */ return binSrch(A, i, j, key);
5.3. Binary Search
![Page 96: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/96.jpg)
242-535 ADA: 2. Running Time
96
/* find 'h' in A[] */binSrch(A, 0, 7, ‘h’);
Execution Example0 1 2 3 4 5 6 7
A a d f g h w x y
you use binary searchto search for a numberin a phone book
![Page 97: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/97.jpg)
242-535 ADA: 2. Running Time
97
• // find '6' in B[]binSrch(B, 0, 6, '6')
Another Example0 1 2 3 4 5 6
B
![Page 98: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/98.jpg)
242-535 ADA: 2. Running Time
98
• Step 1.o Basis: T(1) = O(1)o Induction: T(n) = O(1) + T(< n/2 >), for n > 1
• Step 2.o Basis: T(1) = ao Induction: T(n) = b + T(< n/2 >), for n > 1
Running Timen == the range of the array
being looked at
![Page 99: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/99.jpg)
242-535 ADA: 2. Running Time
99
• This time the relation is harder to solve since the precise value of < n/2 > depends on n.
• This affects how many calls it takes to get to T(1).
• Assume the simple case:o n starts as a power of 2: 2k
Solve the Relation
continued
![Page 100: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/100.jpg)
242-535 ADA: 2. Running Time
100
• Evaluate T(n), where n is 2k:T(2k) = b + T(2k-1)
= b + b + T(2k-2)= b + b + b + T(2k-3)= b + b + b + .... + b + T(1)
• General Form:T(n) = bk +a
continued
k of these
e.g. T(32) = b + b + b + b + b + 1
Notice we are notstarting at T(1) in thisexample; that's ok.
![Page 101: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/101.jpg)
242-535 ADA: 2. Running Time
101
• We assumed that 2k = n, so k = log2 n• This means that:
T(n) = b*log2 n + a
• Step 3. Replace constants by Big-Oh notation:T(n) = O(log2 n) + O(1)
= O(log2 n)
• Running time for binary search is O(log2 n).
![Page 102: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/102.jpg)
242-535 ADA: 2. Running Time
102
6. Towers of HanoiPole A
Pole BPole C
disks
How long to move 50 disksfrom A to B? 36 years
![Page 103: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/103.jpg)
242-535 ADA: 2. Running Time
103
• Problem: transfer all the disks from pole A to pole B.o 1. In each step, only 1 disk can be moved from one
pole to another.
o 2. A disk may never be placed on top of a smaller disk.
o 3. Pole C may be used.
![Page 104: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/104.jpg)
242-535 ADA: 2. Running Time
104
• In the following, X --> Y means “move the top disk from Pole X to Pole Y”.
• Problem: move 3 disks from A to B
• Solution:A --> B, A --> C, B --> C,
A --> B,C --> A, C --> B, A --> B
Solution for 3 Disks
![Page 105: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/105.jpg)
242-535 ADA: 2. Running Time
105
void move(int n, Pole x, Pole y, Pole z)/* move n disks from x to y, use z */ if (n == 1) print “x --> y”; else move( n-1, x, z, y); // x to z, use y print “x --> y”; move( n-1, z, y, x); // z to y, use x
Pseudocode
![Page 106: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/106.jpg)
242-535 ADA: 2. Running Time
106
• move( n-1, x, z, y);
print “x --> y”;
move( n-1, z, y, x);
Recursive Part Graphically
x y zmove(n, x, y, z)
x y z
x y z
x y z
![Page 107: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/107.jpg)
242-535 ADA: 2. Running Time
107
Execution for 3 Disksmove 3 from A to B
using C
move 2 from A to Cusing B
move 2 from C to Busing A
move 1 from A to Busing C
move 1 from B to Cusing A
A --> B
A --> C
A --> B B --> C
start stop
. . . .
continued
nextslide
![Page 108: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/108.jpg)
move 2 from C to Busing A
move 1 from A to Busing C
move 1 from C to Ausing B C --> B
C --> A A --> B
. . . .
. . . .
![Page 109: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/109.jpg)
242-535 ADA: 2. Running Time
109
• We will calculate the running time by using the number of disk moves as the size measure.
• So T(n) = number of disk moves to solve the n-disk problem.
Size for the Running Time
![Page 110: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/110.jpg)
242-535 ADA: 2. Running Time
110
• Step 1.o Basis: T(1) = O(1)o Induction: T(n) = 2*T(n-1) + O(1), for n > 1
• Step 2.o Basis: T(1) = ao Induction: T(n) = 2*T(n-1) + b, for n > 1
Running Time
continued
![Page 111: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/111.jpg)
242-535 ADA: 2. Running Time
111
• Solve the relation:T(n) = 2T(n-1) + b
= 2( 2T(n-2) + b ) + b= 4T(n-2) + 2b + b= 4( 2T(n-3) + b ) + 2b + b= 8T(n-3) + 4b + 2b + b
:= 2n-1T(1) + (2n-2 + 2n-3 + ... + 2 + 1)b= 2n-1a + (2n-2 + 2n-3 + ... + 2 + 1)b~= (2n-1 + 2n-2 + 2n-3 + ... + 2 + 1)b= (2n - 1)b
continued
![Page 112: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/112.jpg)
242-535 ADA: 2. Running Time
112
• Step 3. Translate back:T(n) = (2n - 1)b
• Replace constants by Big-Oh notation:T(n) = O(2n)
• The running time for “Towers of Hanoi” is O(2n)o exponential growth!
continued
![Page 113: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/113.jpg)
242-535 ADA: 2. Running Time
113
• This exponential growth means that a practical limit is quickly reached to the number of disks that can be moved.
• Look at slide 16 to see approximately how long it would take to move 1000 disks if a single disk move takes 1 microsec.
![Page 114: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/114.jpg)
242-535 ADA: 2. Running Time
114
• “Towers of Hanoi” written in every language (C, C++, Java, JavaScript, Fortran, Ada, ML, etc):http://www.pangea.ca/kolar/javascript/Hanoi/HTonWebE.html
• Some history about the problem, animations, a Windows game:http://www.lhs.berkeley.edu/Java/Tower/
More Information
![Page 115: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/115.jpg)
242-535 ADA: 2. Running Time
115
• Most of the time we want to calculate the worst case running time, which means big-Oh (O()).
• There are several other asymptotic notations, which are quite useful, especially big-theta.
7. Other Asymptotic Notations
![Page 116: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/116.jpg)
242-535 ADA: 2. Running Time
116
Big-Oh Definition Again
continued
• n0 and c are called witnesses to the relationship:T(n) = f(n) and T(n) is O(g(n) )
• We can write this as:o f(n) is O(g(n)) // leave out the
T(n)
We write T(n) = f(n) as T(n) is O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 f(n) c*g(n) for all n n0.
![Page 117: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/117.jpg)
242-535 ADA: 2. Running Time
117
• O-notation gives an upper bound for a function to within a constant factor. We write f(n) is O(g(n)) if there are positive constants n0 and c such that at and to the right of n0, the value of f(n) always lies on or below c*g(n).
O-notation as a Graph
is
above
![Page 118: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/118.jpg)
242-535 ADA: 2. Running Time
118
• Called big-omega.
Ω Notation
Ω(g(n)) = f(n) : there exist constants c > 0, n0 >0 such that 0 c*g(n) f(n) for all n n0
• f(n) is in a set of functions that are greater than or equal to g(n)• f(n) is bounded below by g(n)
EXAMPLE: √n is Ω(log n)√n is greater than or equal to log n
![Page 119: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/119.jpg)
242-535 ADA: 2. Running Time
119
• Ω-notation gives a lower bound for a function to within a constant factor. We write f(n) is Ω(g(n)) if there are positive constants n0 and c such that at and to the right of n0, the value of f(n) always lies on or above c*g(n).
Ω-notation as a Graph
is
below
![Page 120: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/120.jpg)
242-535 ADA: 2. Running Time
120
T() and omega() Graphed
T(n) = n0.5
Ω(n) = log n
T(n)
n
below
√n is Ω(log n)
![Page 121: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/121.jpg)
242-535 ADA: 2. Running Time
121
-notation (tight bounds)
(g(n)) = O(g(n)) (g(n))
EXAMPLE: )( is 2 2221 nnn
Less than and greater than or equal to.This make polynomials easy to simplify: ad nd + ad–1nd–1 + + a1n + a0 = (nd)
big-theta
![Page 122: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/122.jpg)
242-535 ADA: 2. Running Time
122
• -notation bounds a function to within constant factors. We write f(n) is (g(n)) if there exist positive constants n0, c1, and c2 such that at and to the right of n0, the value of f(n) always lies between c1*g(n) and c2*g(n) inclusive.
-Notation as a Graph
is
above
below
![Page 123: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/123.jpg)
242-535 ADA: 2. Running Time
123
T() and () GraphedT(n) = 0.5 n2 – 2n
Ω(n) = 0.2 n2
T(n)
n
below
above)( is 2 22
21 nnn O(n) = n2
C1 = 0.2and C2 = 0.5
n0
The O() and Ω(n) curvescould easily be tighter(i.e. closer to T(n)).How?
![Page 124: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/124.jpg)
242-535 ADA: 2. Running Time
124
• The -notation means that:o c1*n2 ≤ 0.5*n2 – 2*n ≤ c2*n2 , for all n >=
n0
• Dividing by n2 gives:o c1 ≤ 0.5 - 2/n ≤ c2
Consider c2• When n ≥ 0, then the r.h.s of the inequality is
o 0.5 - 2/n (or smaller subtraction) ≤ c2• When n == ∞
o 0.5 ≤ c2
Calculating c1 and c2
![Page 125: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/125.jpg)
242-535 ADA: 2. Running Time
125
Consider c1• When n ≥ 5, then the l.h.s of the inequality is
o c1 ≤ 0.5 - 2/n (or a smaller subtraction)• When n == 5
o c1 ≤ 1/2 - 2/5 = 5/10 - 4/10o c1 ≤ 1/10
• Thus we can choose c1 = 1/10 c2 = 1/2, n0 = 5
![Page 126: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/126.jpg)
242-535 ADA: 2. Running Time
126
T() and () Graphed Again
T(n) = 0.5 n2 – 2n
Ω(n) = 0.1* n2
T(n)
n
below
above
)( is 2 2221 nnn O(n) = 0.5*n2
C1 = 0.1and C2 = 0.5
n0 == 5
![Page 127: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/127.jpg)
242-535 ADA: 2. Running Time
127
o-notation and -notation
o(g(n)) = f(n) : for any constant c > 0, there is a constant n0 > 0 such that 0 f(n) < c*g(n) for all n n0
EXAMPLE: (n0 = 2/c)
O-notation and -notation are like and .o-notation and -notation are like < and >.
2n2 is o(n3)
little-oh little-omega
![Page 128: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/128.jpg)
242-535 ADA: 2. Running Time
128
)(log is nn
o-notation and -notation
(g(n)) = f(n) : for any constant c > 0, there is a constant n0 > 0 such that 0 c*g(n) < f(n) for all n n0
EXAMPLE: (n0 = 1+1/c)
O-notation and -notation are like and .o-notation and -notation are like < and >.
little-oh little-omega
![Page 129: Algorithm Design and Analysis (ADA)](https://reader035.fdocuments.us/reader035/viewer/2022062521/56816391550346895dd488f9/html5/thumbnails/129.jpg)
242-535 ADA: 2. Running Time
129
• Intuitively,
Asymptotic Summary
o() is like < O() is like
() is like > () is like
() is like =
the two important ones