Growth of functions

Growth of Functions 2

Function Growth

• The running time of an algorithm as input size approaches infinity is called the asymptotic running time

• We study different notations for asymptotic efficiency.• In particular, we study tight bounds, upper bounds and lower bounds.


The “sets” and their use – big Oh• Big “oh” - asymptotic upper bound on the growth of

an algorithm• When do we use Big Oh?1. Theory of NP-completeness2. To provide information on the maximum number of

operations that an algorithm performs• Insertion sort is O(n2) in the worst case

• This means that in the worst case it performs at most cn2 operations

• Insertion sort is also O(n6) in the worst case since it also performs at most dn6 operations


The “sets” and their use – OmegaOmega - asymptotic lower bound on the growth of an

algorithm or a problem*

When do we use Omega?1. To provide information on the minimum number of

operations that an algorithm performs• Insertion sort is (n) in the best case

• This means that in the best case its instruction count is at least cn, • It is (n2) in the worst case

• This means that in the worst case its instruction count is at least cn2


The “sets” and their use – Omega cont.2. To provide information on a class of algorithms that solve a problem

• Sort algorithms based on comparison of keys are (nlgn) in the worst case• This means that all sort algorithms based only on comparison of keys have to do at least

cnlgn operations• Any algorithm based only on comparison of keys to find the maximum of n

elements is (n) in every case• This means that all algorithms based only on comparison of keys to find maximum have

to do at least cn operations


The “sets” and their use - Theta• Theta - asymptotic tight bound on the growth rate of an algorithm

• Insertion sort is (n2) in the worst and average cases• The means that in the worst case and average cases insertion

sort performs cn2 operations• Binary search is (lg n) in the worst and average cases

• The means that in the worst case and average cases binary search performs clgn operations

• Note: We want to classify an algorithm using Theta. • In Data Structures used Oh

• Little “oh” - used to denote an upper bound that is not asymptotically tight. n is in o(n3). n is not in o(n)


The functions

• Let f(n) and g(n) be asymptotically nonnegative functions whose domains are the set of natural numbers N={0,1,2,…}.

• A function g(n) is asymptotically nonnegative, if g(n)³0 for all n³n0 where n0ÎN


Asymptotic Upper Bound: big O

f (n)

c g (n)

f (n) = O ( g ( n ))N

Why only for n N ?What is the purpose of multiplying by c > 0?

Graph shows thatfor all n N,f(n) c*g(n)


Asymptotic Upper Bound: O

Definition: Let f (n) and g(n) be asymptotically non-negative functions. We say f (n) is in O ( g ( n )) if there is a real positive constant c and a positive Integer N such that for every n ³ N 0 £ f (n) £ c g (n ).

Or using more mathematical notationO ( g (n) ) =

{ f (n )| there exist positive constant c and a positive integer N such that

0 £ f( n) £ c g (n ) for all n ³ N }


n2 + 10 n O(n2) Why?

0200400600800

100012001400

0 10 20 30

n2 + 10n

2 n2

take c = 2N = 10

2n2 > n2 + 10 n for all n>=10


Does 5n+2 ÎO(n)?Proof: From the definition of Big Oh, there must exist c>0

and integer N>0 such that 0 £ 5n+2£cn for all n³N.Dividing both sides of the inequality by n>0 we get:

0 £ 5+2/n£c.2/n £ 2, 2/n>0 becomes smaller when n increasesThere are many choices here for c and N. If we choose N=1 then c ³ 5+2/1= 7.If we choose c=6, then 0 £ 5+2/n£6. So N ³ 2.In either case (we only need one!) we have a c>o and N>0

such that 0 £ 5n+2£cn for all n ³ N. So the definition is satisfied and 5n+2 ÎO(n)


Does n2Î O(n)? No.

We will prove by contradiction that the definition cannot besatisfied. Assume that n2Î O(n).From the definition of Big Oh, there must exist c>0 and

integer N>0 such that 0 £ n2£cn for all n³N.

Dividing the inequality by n>0 we get 0 £ n £ c for all n³N.

n £ c cannot be true for any n >max{c,N }, contradicting our assumption

So there is no constant c>0 such that n£c is satisfied for all n³N, and n2 O(n)


O ( g (n) ) = { f (n )| there exist positive constant c

and positive integer N such that 0 £ f( n) £ c g (n ) for all n ³ N }

• 1,000,000 n2 O(n2) why/why not?

• (n - 1)n / 2 O(n2) why /why not?

• n / 2 O(n2) why /why not?

• lg (n2) O( lg n ) why /why not?

• n2 O(n) why /why not?


Asymptotic Lower Bound, Omega: Wf (n)

c * g (n)

f(n) = ( g ( n ))N


Asymptotic Lower Bound: W

Definition: Let f (n) and g(n) be asymptotically non-negative functions. We say f (n) is W ( g ( n )) if there is a positive real constant c and a positive integer N such that for every n ³ N 0 £ c * g (n ) £ f ( n).

Or using more mathematical notationW ( g ( n )) =

{ f (n) | there exist positive constant c and a positive integer N such that

0 £ c * g (n ) £ f ( n) for all n ³ N }


Is 5n-20Î W (n)?Proof: From the definition of Omega, there must exist c>0 and integer N>0

such that 0 £ cn £ 5n-20 for all n³N

Dividing the inequality by n>0 we get: 0 £ c £ 5-20/n for all n³N.

20/n £ 20, and 20/n becomes smaller as n grows.

There are many choices here for c and N.

Since c > 0, 5 – 20/n >0 and N >4For example, if we choose c=4, then 5 – 20/n ³ 4 and N ³ 20

In this case we have a c>o and N>0 such that 0 £ cn £ 5n-20 for all n ³ N. So the definition is satisfied and 5n-20 Î W (n)


W ( g ( n )) = { f (n) | there exist positive constant c and

a positive integer N such that 0 £ c * g (n ) £ f ( n) for all n ³ N }

• 1,000,000 n2 W (n2) why /why not?

• (n - 1)n / 2 W (n2) why /why not?

• n / 2 W (n2) why /why not?

• lg (n2) W ( lg n ) why /why not?

• n2 W (n) why /why not?


Asymptotic Tight Bound: Q

f (n)

d g (n)

f (n) = Q ( g ( n ))

N

c g (n)


Asymptotic Bound Theta: Q

Definition: Let f (n) and g(n) be asymptotically non-negative functions. We say f (n) is Q( g ( n )) if there are positive constants c, d and a positive integer N such that for every n ³ N 0 £ c g (n ) £ f ( n) £ d g ( n ).

Or using more mathematical notationQ ( g ( n )) =

{ f (n) | there exist positive constants c, d and a positive integer N such that

0 £ c g (n ) £ f ( n) £ d g ( n ). for all n ³ N }


More on Q• We will use this definition:

Q (g (n)) = O( g (n) ) Ç W ( g (n) )


• We show:

1.

2.

?)(32

1 Does 22 nnn

)(32

1 22 nOnn

)(32

1 22 nnn


?)(32

1 Does 22 nOnn

12 and ,

. all for

:get weby inequality the Dividing

. all for

that such and, exist must there definition the From

Choose

1/2. Since

Nn

Nncn

n

Nncnnn

Nc

c

cnn

4

1321

3210

02

232210

00

So

.4/1

, finitefor 0/3


?)(32

1 Does 22 nnn

. and So

. all for

1/4.c Choose finite for 0 Since

6. Since

.

that such and exist must There

1241

12321

41

.2/1 ,3

and 32100

3210

get we02by Dividing

allfor 322120

00

N/c

nn

cn/n

N, c

nc

n

Nnnncn

Nc

N


More Q• 1,000,000 n2 Q(n2) why /why not?

• (n - 1)n / 2 Q(n2) why /why not?

• n / 2 Q(n2) why /why not?

• lg (n2) Q ( lg n ) why /why not?

• n2 Q(n) why /why not?

Qualified Notation

Ω Omega Notation • best case Ω - describes a lower bound for all input (it can't get any

better than this). Example: the array is already correctly sorted.

• worst case Ω - describes a lower bound for worst case input, possibly greater than best case. Example: the array is sorted in reverse order.

• just Ω - same as best case Ω

Qualified Notation

O Big-O Notation • best case O - describes an upper bound for best case input, possibly

lower than worst case. Example: the array is already correctly sorted.

• worst case O - describes an upper bound for all input (it can't get any worse than this). Example: the array is sorted in reverse order.

• just O - same as worst case O

Qualified Notation

Θ Theta Notation • best case Θ - not used

• worst case Θ - describes asymptotic bounds for worst case input

• just Θ - same as worst case Θ

Comparing functions

Similarly forReflexivitySymmetryTranspose symmetry

Growth of functions

Documents

Transcript of Growth of functions