Design and Analysis of Algorithms Chapter 2.21

Asymptotic Notations*

Dr. Ying Luylu@cse.unl.edu

August 30, 2012

http://www.cse.unl.edu/~ylu/raik283

RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms

*slides refrred to http://www.aw-bc.com/info/levitinhttp://www.aw-bc.com/info/levitin

Design and Analysis of Algorithms Chapter 2.22

Review: algorithm efficiency Review: algorithm efficiency indicatorindicator

order of growth of

an algorithm’s basic operation count

the algorithm’s time efficiency

Design and Analysis of Algorithms Chapter 2.23

Asymptotic growth rateAsymptotic growth rate

A way of comparing functions that ignores constant factors A way of comparing functions that ignores constant factors and small input sizesand small input sizes

O(O(gg((nn)): class of functions )): class of functions t(n)t(n) that grow that grow no fasterno faster than than gg((nn))

Θ Θ ((gg((nn)): class of functions )): class of functions t(n)t(n) that grow that grow at same rateat same rate as as gg((nn))

ΩΩ((gg((nn)): class of functions )): class of functions t(n)t(n) that grow that grow at least as fastat least as fast as as gg((nn))

Design and Analysis of Algorithms Chapter 2.24

Big-ohBig-oh

c > 0, n0 0 , n n0, t(n) cg(n)

t(n) O(g(n))

Design and Analysis of Algorithms Chapter 2.25

Small-ohSmall-oh

c > 0, n0 0 , n n0, t(n) < cg(n)

t(n) o(g(n))

Design and Analysis of Algorithms Chapter 2.26

Big-omegaBig-omega

t(n) (g(n))

Design and Analysis of Algorithms Chapter 2.27

Big-omegaBig-omega

c > 0, n0 0 , n n0, t(n) cg(n)

t(n) (g(n))

Design and Analysis of Algorithms Chapter 2.28

Small-omegaSmall-omega

c > 0, n0 0 , n n0, t(n) > cg(n)

t(n) (g(n))

Design and Analysis of Algorithms Chapter 2.29

Big-thetaBig-theta

t(n) (g(n))

c1>c2>0, n00, n n0, c2g(n) t(n) c1g(n)

Design and Analysis of Algorithms Chapter 2.210

Big thetaBig theta

The reverse statement of

t(n) (g(n))

c1>c2>0, n00, n n0, c2g(n) t(n) c1g(n)

Design and Analysis of Algorithms Chapter 2.211

Big thetaBig theta

t(n) (g(n))

c1>c2>0, n00, n n0, t(n) < c2g(n) or t(n) > c1g(n)

Design and Analysis of Algorithms Chapter 2.212

Establishing rate of growth: Method 1 – using Establishing rate of growth: Method 1 – using definitiondefinition

t(n)t(n) is O( is O(gg((nn)) if order of growth of )) if order of growth of t(n)t(n) ≤ order of growth ≤ order of growth of of gg((nn) (within constant multiple)) (within constant multiple)

There exist positive constant There exist positive constant cc and non-negative integer and non-negative integer nn00

such thatsuch that

t(n)t(n) ≤ ≤ c gc g((nn) for every ) for every nn ≥ ≥ nn0 0

Examples:Examples: 1010nn O(2 O(2nn22))

55nn+20 +20 O(10 O(10nn))

Design and Analysis of Algorithms Chapter 2.213

A B O Ω Θ

1 ln2n n YesYes NoNo NoNo

2 nk cn YesYes NoNo NoNo

3 nsinn NoNo NoNo NoNo

4 2n 2n/2 NoNo YesYes NoNo

5 nlgc clgn YesYes YesYes YesYes

6 lg(n!) lg(nn) YesYes YesYes YesYes

n

2n (2n/2) 2n (2n/2)

Establishing rate of growth: Method 1 – using Establishing rate of growth: Method 1 – using definitiondefinition

Examples:Examples:

Design and Analysis of Algorithms Chapter 2.214

A B O Ω Θ

1 ln2n n YesYes NoNo NoNo

2 nk cn YesYes NoNo NoNo

3 nsinn NoNo NoNo NoNo

4 2n 2n/2 NoNo YesYes NoNo

5 nlgc clgn YesYes YesYes YesYes

6 lg(n!) lg(nn) YesYes YesYes YesYes

n

O(nsinn) (nsinn) n n

Establishing rate of growth: Method 1 – using Establishing rate of growth: Method 1 – using definitiondefinition

Examples:Examples:

Design and Analysis of Algorithms Chapter 2.215

Establishing rate of growth: Method 2 – Establishing rate of growth: Method 2 – using limitsusing limits

limlimn→∞ n→∞ tt((nn)/)/gg((nn))

0 order of growth of tt((n)n) < order of growth of gg((nn))t(n) t(n) o(g(n)), o(g(n)), t(n) t(n) O(g(n)) O(g(n))

c>0 order of growth of tt((n)n) = order of growth of gg((nn))t(n) t(n) (g(n)),(g(n)), t(n) t(n) O(g(n)), t(n) O(g(n)), t(n) (g(n))(g(n))

∞ order of growth of tt((n)n) > order of growth of gg((nn))

t(n) t(n) (g(n)),(g(n)), t(n) t(n) (g(n))(g(n))

==

Design and Analysis of Algorithms Chapter 2.216

Establishing rate of growth: Method 2 – Establishing rate of growth: Method 2 – using limitsusing limits

Examples:Examples:• loglogbb n vs. log n vs. logc c nn

loglogbbn = logn = logbbc logc logccn n

limlimn→∞n→∞( log( logbbn / logn / logccn) = limn) = limn→∞n→∞ (log(logbbc) = logc) = logbbcc

loglogbbn n (log(logccn)n)

Design and Analysis of Algorithms Chapter 2.217

Exercises: establishing rate of growth – Exercises: establishing rate of growth – using limitsusing limits

lnln22nn vs. ln vs. lnnn22

22nn vs. 2 vs. 2n/2n/2

22n-1 n-1 vs. 2vs. 2nn

loglog22n n vs. vs. nn

Design and Analysis of Algorithms Chapter 2.218

L’Hôpital’s ruleL’Hôpital’s rule

IfIf limlimn→∞ n→∞ t(n)t(n) = = limlimn→∞ n→∞ gg((nn) = ∞) = ∞

The derivatives The derivatives ff´, ´, gg´ exist,´ exist,

ThenThen

t(n)t(n)gg((nn))

limlimnn→∞→∞

= t t ´(´(nn))g g ´(´(nn))

limlimnn→∞→∞

• Example: logExample: log22nn vs. vs. nn

Design and Analysis of Algorithms Chapter 2.219

A B O Ω Θ

1 ln2n n YesYes NoNo NoNo

2 nk cn YesYes NoNo NoNo

3 nsinn NoNo NoNo NoNo

4 2n 2n/2 NoNo YesYes NoNo

5 nlgc clgn YesYes YesYes YesYes

6 lg(n!) lg(nn) YesYes YesYes YesYes

n

Establishing rate of growthEstablishing rate of growth

Examples:Examples:

Design and Analysis of Algorithms Chapter 2.220

Stirling’s formulaStirling’s formula

n

en

nn )(2!

Design and Analysis of Algorithms Chapter 2.221

n! v.s. nn! v.s. nnn

lg(n!) v.s. lg(nlg(n!) v.s. lg(nnn))

Examples using stirling’s Examples using stirling’s formulaformula

OO

Design and Analysis of Algorithms Chapter 2.222

n! n! o(n o(nnn))

lg(n!) v.s. lg(nlg(n!) v.s. lg(nnn))

??? lg(n!) ??? lg(n!) o(lg(no(lg(nnn))))

Examples using stirling’s Examples using stirling’s formulaformula

OO

Design and Analysis of Algorithms Chapter 2.223

n! n! o(n o(nnn))

lg(n!) v.s. lg(nlg(n!) v.s. lg(nnn))

lg(n!) lg(n!) (lg(n(lg(nnn))))

Examples using stirling’s Examples using stirling’s formulaformula

OO

Design and Analysis of Algorithms Chapter 2.224

Special attentionSpecial attention

n! n! o(n o(nnn))

However,However,

lg(n!) lg(n!) o(lg(n o(lg(nnn)))) lg(n!) lg(n!) (lg(n(lg(nnn))))

sinn sinn (1/2) (1/2) sinn sinn O(1/2) O(1/2) sinn sinn (1/2) (1/2)

However,However,

nn1/2 1/2 (n(nsinnsinn)) nn1/2 1/2 O(n O(nsinnsinn)) nn1/2 1/2 (n(nsinnsinn))

Design and Analysis of Algorithms Chapter 2.225

Asymptotic notation propertiesAsymptotic notation properties

Transitivity:Transitivity:

f(n) = f(n) = (g(n)) && g(n) = (g(n)) && g(n) = (h(n)) (h(n)) f(n) = f(n) = (h(n)) (h(n))

f(n) = O(g(n)) && g(n) = O(h(n)) f(n) = O(g(n)) && g(n) = O(h(n)) f(n) = O(h(n)) f(n) = O(h(n))

f(n) = Ω(g(n)) && g(n) = Ω(h(n)) f(n) = Ω(g(n)) && g(n) = Ω(h(n)) f(n) = Ω(h(n)) f(n) = Ω(h(n))

If tIf t11(n) (n) O(g O(g11(n)) and t(n)) and t22(n) (n) O(g O(g22(n)), then (n)), then

tt11(n) + t(n) + t22(n) (n)

Design and Analysis of Algorithms Chapter 2.226

Asymptotic notation propertiesAsymptotic notation properties

Transitivity:Transitivity:

f(n) = f(n) = (g(n)) && g(n) = (g(n)) && g(n) = (h(n)) (h(n)) f(n) = f(n) = (h(n)) (h(n))

f(n) = O(g(n)) && g(n) = O(h(n)) f(n) = O(g(n)) && g(n) = O(h(n)) f(n) = O(h(n)) f(n) = O(h(n))

f(n) = Ω(g(n)) && g(n) = Ω(h(n)) f(n) = Ω(g(n)) && g(n) = Ω(h(n)) f(n) = Ω(h(n)) f(n) = Ω(h(n))

If tIf t11(n) (n) O(g O(g11(n)) and t(n)) and t22(n) (n) O(g O(g22(n)), then (n)), then

tt11(n) + t(n) + t22(n) (n) O(maxg O(maxg11(n), g(n), g22(n)) (n))

In-Class ExercisesIn-Class Exercises

Exercises 2.2: Problem 1, 2, 3 & 12Exercises 2.2: Problem 1, 2, 3 & 12 Problem 1: Use the most appropriate notation among O, Problem 1: Use the most appropriate notation among O, , ,

and and to indicate the time efficiency class of sequential to indicate the time efficiency class of sequential search: search: • a. in the worst casea. in the worst case

• b. in the best caseb. in the best case

• c. in the average case ( Hint: C(n) = p*(n+1)/2 + (1-p)*n )c. in the average case ( Hint: C(n) = p*(n+1)/2 + (1-p)*n )

Problem 2: Use the informal definitions of O, Problem 2: Use the informal definitions of O, , and , and to to determine whether the following assertions are true or determine whether the following assertions are true or false. false. • a. n(n+1)/2 a. n(n+1)/2 O(n O(n33)) b. n(n+1)/2 b. n(n+1)/2 O(n O(n22))

• c. n(n+1)/2 c. n(n+1)/2 (n(n33)) d. n(n+1)/2 d. n(n+1)/2 (n)(n)

Design and Analysis of Algorithms Chapter 2.227

AnnouncementAnnouncement

40-minute quiz next Tuesday40-minute quiz next Tuesday• pproblems based on materials covered in roblems based on materials covered in Chapter 2.2 (Asymptotic Chapter 2.2 (Asymptotic

Notations and Basic Efficiency Classes)Notations and Basic Efficiency Classes)

Design and Analysis of Algorithms Chapter 2.228

In-Class ExercisesIn-Class Exercises

Establish the asymptotic rate of growth (O, Establish the asymptotic rate of growth (O, , and , and ) of ) of the following pair of functions. Prove your assertions. the following pair of functions. Prove your assertions. • a. 2a. 2nn vs. 3 vs. 3nn b. ln(n+1) vs. ln(n) b. ln(n+1) vs. ln(n)

Problem 3: For each of the following functions, indicate the Problem 3: For each of the following functions, indicate the class class (g(n)) the function belongs to. (Use the simplest g(n) (g(n)) the function belongs to. (Use the simplest g(n) possible in your answers.) Prove your assertions. possible in your answers.) Prove your assertions. • a. (na. (n22 + 1) + 1)10 10 b. b.

• c. 2nlg(n+2)c. 2nlg(n+2)22 + (n+2) + (n+2)22lg(n/2)lg(n/2)

• d. 2d. 2n+1n+1 + 3 + 3n-1 n-1 e. e. loglog22nn

Design and Analysis of Algorithms Chapter 2.229

3710 2 nn