1. Complexity Analysis

7/29/2019 1. Complexity Analysis

1/44

Complexity Analysis(newer version)


2/44

What is an algorithm?

The ideas behind computer programs Stays the same no matter

Which kind of hardware it is running on

Which programming language it is writtenin

Solves a general, well-specified problem

Is specified by Describing the set of instances (input) it

must work on

Describing the desired properties of theoutput


3/44

Important Properties of Algorithms

Correct

always returns the desired output for all

legal instances of the problem. Efficient

Can be measured in terms of

Time

Space

Time tends to be more important


4/44

Expressing Algorithms

English description

Pseudocode

High-levelprogramminglanguage

Moreprecise

More easilyexpressed


5/44

Pseudocode

A shorthand for specifying algorithms Leaves out the implementation details

Leaves in the essence of the algorithm

Algorithm ArrayMax(A,n)

Input: An array A storing n1 integers

Output: The maximum element in A.

currentMax A[0]

for i 1 to n-1 do

ifcurrentMax < A[i] then

currentMax A[i]

return currentMax


6/44

Analysis of Algorithms

Why analyze algorithms? Evaluate algorithm performance

Compare different algorithms

Analyze what about them? Running time, memory usage, solution

quality

Worst-case and typical case Analysis of algorithms compare

algorithms, not programs.


7/44

If each line takes constant time the whole algorithm (anyalgorithm) will take constant time, right?

Wrong!

Although some algorithms may take constant time, the

majority of algorithms varies its number of steps based onthe size of instance we're trying to solve

Therefore the efficiency of an algorithm is always normallystated as a function of the problem size

We generally use the variable n to represent the problem size

On the implementation, we could find out thatSuperDuper Sort takes 0.6n2 + 0.3n + 0.45 seconds on aPentium 3.

Plug a value for n and you have how long it takes

Analysis of Algorithms (Cont.)


8/44

Algorithm Complexity

Worst Case Complexity: The function defined by the maximum

number of steps taken on any instance ofsize n

Best Case Complexity:

The function defined by the minimumnumber of steps taken on any instance of

size n

Average Case Complexity:

The function defined by the average number

of steps taken on any instance of size n


9/44

Best, Worst, and Average CaseComplexity

Worst CaseComplexity

Average CaseComplexity

Best CaseComplexity

Number ofsteps

n(input size)


10/44

Doing the Analysis

Its hard to estimate the running time exactly

Best case depends on the input

Average case is difficult to compute

So we usually focus on worst case analysis

Easier to compute

Usually close to the actual running time

Strategy: try to find upperand lower bounds of

the worst case function.

Upper bound

Lower bound

Actual function


11/44

Running Time Analysis


12/44

Running Time Analysis


13/44

Comparing Algorithms

Establish a relative order amongdifferent algorithms in terms of

their relative rates of growth.The rates of growth are expressed

as functions, which are generally

in terms of the number of inputs n.


14/44

Asymptotic Analysis

Asymptotic analysis of an algorithm describesthe relative efficiency of an algorithm as n getvery large.

When you're dealing with small input size,

most of algorithms will do When the input size is very large things change

In the example it is easy to see that for verylarge n, g(n) grows faster than f(n) Take for instance the value n=20000000

Remember that the goal here is to comparealgorithms. In practice, if you're writing smallprograms, asymptotic analysis may not be thatimportant


15/44

A simple comparison

Let's assume that you have 3 algorithms to sort a list

f(n) = n log2n g(n) = n2

h(n) = n3

Let's also assume that each step takes 1 microsecond(10-6)

Most of the algorithms discussed here will be givenin terms of common functions: polynomials,logarithms, exponentials and product of thesefunctions

n n log n n^2 n^3

10 33.2 100 1000100 664 10000 1seg

1000 9966 1seg 16min

100000 1.7s 2.8 hours 31.7 years


16/44

Big-Oh Notation

T(n) = O(f(n)) if there are positive constants cand n0 such that T(n) cf(n) for n n0

This says that function T(n) grows at a rate no

faster thanf(n). Thus,f(n) is an upper

bound on T(n).

T(n) = O(f(n))

cf(n)

T(n)


17/44

n0

T(n) = O(f(n))

Big-Oh Upper Bound


18/44

Example

Prove that 7n3+2n2 = O(n3)

Since 7n3+2n2 < 7n3 +2n3 = 9n3 (for n 1)

Then 7n3+2n2 = O(n3) with c = 9 and n0 = 1

Similarly, we can prove that 7n3

+2n2

=O

(n4

). The first bound is tighter.


19/44

Tighter Upper Bound

n0


20/44

Big-Omega Notation

T(n) = (f(n)) if there are positive constants c and n0such that T(n) cf(n) for n n0

This says that function T(n) grows at a rate no slower

thanf(n).

Thus,f(n) is a lower bound

on T(n)

T(n) = ( (n))

cf(n)

T(n)


21/44

T(n) =

(f(n))

n0

Big-Omega Lower Bound


22/44

Example

Prove that 2n+5n2 = (n2)

Since 2n+5n2 > 5n2 > 1 n2 (for n 1)

Then 2n+5n2 = (n2) with c = 1 and n0 = 1 Similarly, we can prove that 2n+5n2 = (n).

The first bound is tighter.


23/44

Tighter Lower Bound

n0

h


24/44

Big-Theta Notation

T(n) = (f(n)) if and only ifT(n) = O(f(n)) andT(n) = (f(n))

This says that function T(n) grows at the same rate asf(n).

Put it another way:

T(n) = (f(n)) if there are

positive constants c1

, c2

,

and n0 such that

c1f(n) T(n) c2f(n) for n n0

T(n) = ( (n))

T(n)

c2f(n)

c1f(n)


25/44

Li l h


26/44

Little-oh

T(n) = o(f(n)) if and only if

T(n) = O(f(n)) and T(n) (f(n)) if

This says that function T(n) grows at a ratestrictly less thanf(n)

Hi h f G h R


27/44

Hierarchy of Growth Rates

c < log n < log2 n < logk n < n


28/44


29/44


30/44


31/44


32/44


33/44


34/44


35/44


36/44

36


37/44


38/44


39/44


40/44


41/44


42/44


43/44


44/44

44

1. Complexity Analysis

Documents

Transcript of 1. Complexity Analysis