Intro Algo
-
Upload
rohit-chaudhary -
Category
Documents
-
view
233 -
download
0
Transcript of Intro Algo
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 1/35
Analysis of Algorithms
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 2/35
I ntroductionWhat is Algorithm?
a clearly specified set of simple instructions to be followed tosolve a problem
Takes a set of values, as input andproduces a value, or set of values, as output
May be specifiedIn English
As a computer program As a pseudo-code
Data structuresMethods of organizing data
Program = algorithms + data structures
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 3/35
I ntroduction
Why need algorithm analysis ?writing a working program is not good enoughThe program may be inefficient!If the program is run on a large data set , then therunning time becomes an issue
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 4/35
Ex ample: Selection ProblemG iven a list of N numbers, determine the k thlargest , where k e N.
Algorithm 1:(1) Read N numbers into an array(2) Sort the array in decreasing order by some
simple algorithm(3) Return the element in position k
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 5/35
Ex ample: Selection Problem«
Algorithm 2:(1) Read the first k elements into an array and sort
them in decreasing order (2) Each remaining element is read one by one
If smaller than the kth element, then it is ignoredOtherwise, it is placed in its correct spot in the array,bumping one element out of the array.
(3) The element in the kth position is returned asthe answer.
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 6/35
Ex ample: Selection Problem«
Which algorithm is better whenN =100 and k = 100?N =100 and k = 1?
What happens when N = 1,000,000 and k =500,000?There exist better algorithms
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 7/35
Algorithm AnalysisWe only analyze correct algorithms
An algorithm is correctIf, for every input instance, it halts with the correct output
Incorrect algorithmsMight not halt at all on some input instancesMight halt with other than the desired answer
Analyzing an algorithm
Predicting the resources that the algorithm requiresResources include
MemoryCommunication bandwidthComputational time (usually most important)
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 8/35
Algorithm Analysis«F actors affecting the running time
computer compiler algorithm usedinput to the algorithm
The content of the input affects the running timetypically, the inpu t s iz e (number of items in the input) is the mainconsideration
E.g. sorting problem the number of items to be sorted
E.g. multiply two matrices together the total number of elements in the two matrices
Machine model assumedInstructions are executed one after another, with noconcurrent operations Not parallel computers
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 9/35
Ex ampleCalculate
Lines 1 and 4 count for one unit eachLine 3: executed N times, each time four unitsLine 2: (1 for initialization, N+1 for all the tests, N for all the increments) total 2N + 2total cost: 6N + 4 O(N)
§!
N
i
i1
3
1
23
4
1
2N+24N
1
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 10/35
Worst- / average- / best-caseWorst-case running time of an algorithm
The longest running time for any input of size n An upper bound on the running time for any input
guarantee that the algorithm will never take longer Example: Sort a set of numbers in increasing order; and thedata is in decreasing order The worst case can occur fairly often
E.g. in searching a database for a particular piece of information
Best-case running timesort a set of numbers in increasing order; and the data isalready in increasing order
Average-case running timeMay be difficult to define what ³average´ means
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 11/35
R unning-time of algorithms
Bounds are for the algorithms , rather thanprograms
programs are just implementations of an algorithm,and almost always the details of the program donot affect the bounds
Bounds are for algorithms, rather thanproblems
A problem can be solved with several algorithms,some are more efficient than others
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 12/35
Growth R ate
The idea is to establish a relative order among functionsfor large n
c , n 0 > 0 such that f(N) e c g(N) when N u n0
f(N) grows no faster than g(N) for ³large´ N
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 13/35
Asymptotic notation: Big-Oh
f(N) = O(g(N))There are positive constants c and n 0 suchthat
f(N) e c g(N) when N u n0
The growth rate of f(N) is l ess tha n or eq ua l to
the growth rate of g(N)g(N) is an upper bound on f(N)
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 14/35
Big-Oh: e x ample
Let f(N) = 2N 2. Thenf(N) = O(N 4)f(N) = O(N 3)f(N) = O(N 2) (best answer, asymptotically tight)
O(N 2): reads ³order N-squared´ or ³Big-Oh N-squared´
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 15/35
Big Oh: more e x amplesN2 / 2 ± 3N = O(N 2)1 + 4N = O(N)7N 2 + 10N + 3 = O(N 2) = O(N 3)
log 10 N = log 2 N / log 2 10 = O(log 2 N) = O(log N)sin N = O(1); 10 = O(1), 10 10 = O(1)
log N + N = O(N)log k N = O(N) for any constant kN = O(2 N), but 2 N is not O(N)210N is not O(2 N)
)( 32
1
2 N O N N i N
i
!e
§ !
)( 2
1 N O N N i
N
i!e§ !
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 16/35
Math R eview: logarithmic functions
xdx
xd
aaana
aba
abb
baab
abiff b x
e
bbb
an
b
m
ma
xa
1log
log)(loglog
loglog
logloglog
logloglog
log
loglog
!
{!
!
!
!
!
!!
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 17/35
Some rules
When considering the growth rate of a function usingBig-OhIgnore the lower order terms and the coefficients of
the highest-order termNo need to specify the base of logarithm
Changing the base from one constant to another changes thevalue of the logarithm by only a constant factor
If T1(N) = O(f(N) and T 2(N) = O(g(N)), thenT1(N) + T 2(N) = max(O(f(N)), O(g(N))),T1(N) * T 2(N) = O(f(N) * g(N))
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 18/35
Big-Omega
c , n 0 > 0 such that f(N) u c g(N) when N u n0
f(N) grows no slower than g(N) for ³large´ N
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 19/35
Big-Omega
f(N) = ; (g(N))
There are positive constants c and n 0 suchthatf(N) u c g(N) when N u n0
The growth rate of f(N) is g reater tha n or eq ua l to the growth rate of g(N).
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 20/35
Big-Omega: e x amples
Let f(N) = 2N 2. Thenf(N) = ; (N)f(N) = ; (N2) (best answer)
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 21/35
f N = 5 g N
the growth rate of f(N) i s the same as the growth rateof g(N)
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 22/35
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 23/35
Some rules
If T(N) is a polynomial of degree k, thenT(N) = 5 (Nk).
F or logarithmic functions,T(log m N) = 5 (log N).
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 24/35
Typical Growth R ates
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 25/35
Growth rates «Doubling the input size
f(N) = c f(2N) = f(N) = cf(N) = log N f(2N) = f(N) + log 2
f(N) = N f(2N) = 2 f(N)f(N) = N 2 f(2N) = 4 f(N)f(N) = N 3 f(2N) = 8 f(N)f(N) = 2 N f(2N) = f 2(N)
Advantages of algorithm analysisTo eliminate bad algorithms earlypinpoints the bottlenecks, which are worth codingcarefully
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 26/35
Using L' Hopital's ruleL' Hopital's rule
If and
then =
Determine the relative growth rates (using L' Hopital's rule if necessary)
compute
if 0: f(N) = O(g(N)) and f(N) is not 5 (g(N))
if constant { 0: f(N) = 5 (g(N))if g : f(N) = ; (f(N)) and f(N) is not 5 (g(N))
limit oscillates: no relation
)(
)(lim
N g
N f n gp )(
)(li
N g
N f n d
d
p
! p
)(lim N f n
g!gp
)( N g n
)()(
lim N g N f
n gp
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 27/35
General R ulesF or loops
at most the running time of the statements insidethe for-loop (including tests) times the number of iterations.
Nested for loops
the running time of the statement multiplied by theproduct of the sizes of all the for-loops.O(N 2)
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 28/35
General rules (cont¶d )Consecutive statements
These just add
O(N) + O(N 2) = O(N 2)If S1
Else S2
never more than the running time of the test plus the larger of the running times of S1 and S2.
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 29/35
Another Ex ample
Maximum Subsequence Sum ProblemG iven (possibly negative) integers A 1, A2, ....,
An, find the maximum value of
F or convenience, the maximum subsequence sumis 0 if all the integers are negative
E.g. for input ±2, 11, -4, 13, -5, -2 Answer: 20 (A 2 through A 4)
§!
j
ik k A
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 30/35
Algorithm : SimpleExhaustively tries all possibilities (brute force)
O(N 3)
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 31/35
Algorithm 2: Divide-and-conquer Divide-and-conquer
split the problem into two roughly equal subproblems, whichare then solved recursivel y
patch together the two solutions of the subproblems to arriveat a solution for the whole problem
The ma x imum subsequence sum can beE ntirely in the left half of the inputE ntirely in the right half of the input
I t crosses the middle and is in both halves
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 32/35
Algorithm 2 (cont¶d )
The first two cases can be solved recursivelyF or the last case:
find the largest sum in the first half that includes the lastelement in the first half the largest sum in the second half that includes the firstelement in the second half add these two sums together
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 33/35
Algorithm 2 «
O(1)
T(m/ 2)
O(m )
O(1)
T(m/ 2)
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 34/35
Algorithm 2 (cont¶d )
Recurrence equation
2 T(N/2): two subproblems, each of size N/2
N: for ³patching´ two solutions to find solution towhole problem
N N
T N T
T
!
!
)2
(2)(
1)1(
8/8/2019 Intro Algo
http://slidepdf.com/reader/full/intro-algo 35/35
Algorithm 2 (cont¶d )Solving the recurrence:
With k=log N (i.e. 2k
= N), we have
Thus, the running time is O(N log N)faster than Algorithm 1 for large data sets
N N N
N N T N N T
!
!
log
log)1()(
kN N
T
N N
T
N N
T
N N
T N T
k
k !
!
!
!
!
)(
3)8(8
)4
(4
)()(
.