Big Oh Notation

5/1/2006 Algorithm Analysis & Design CS 007 BE CS 5th Semester 2

Introduction

In most of the algorithms it is difficult to

analyze the exact number of operations

To simplify the analysis:

Identify the fastest growing term

Neglect the slow growing terms

Neglect the constant factor in the

fastest growing term

5/1/2006 Algorithm Analysis & Design CS 007 BE CS 5th Semester 3

Growth rate

resultant of such a simplification is called the

algorithm’s time complexity.

It focuses on the growth rate of the algorithm

with respect to the problem size

it makes sense (both mathematically and

logically) to ignore the slow growing terms

and also the co-efficient of the fastest growing

term.

5/1/2006 Algorithm Analysis & Design CS 007 BE CS 5th Semester 4

Big- Oh Notation

An algorithm is said to have a worst-case running time of O(n2) (read as Order of n2) if, its worst-case running time is not more than proportional to n2 where n is a measure of problem size. An analogous definition holds true for the average-case running time. More formally, we can define the following:

T(n) = O(f(n)) if there are positive constants c and n0 such that T(n) <= cf(n) for all n >= n0

This notation is known as Big-Oh notation

5/1/2006 Algorithm Analysis & Design CS 007 BE CS 5th Semester 5

Big- Oh Notation

The Big-Oh notation is more like an upper bound. Since it is tough to find the exact number of operations performed by an algorithm in terms of its worst case complexity, we are looking at proportional growth rates. The Big-Oh notation can be viewed as an upper bound for the worst case complexity.

Here T(n) would indicate worst case, average case, or best case running times of an algorithm, for an input size of n

5/1/2006 Algorithm Analysis & Design CS 007 BE CS 5th Semester 6

Upper Bound

Suppose that we have a set of numbers

ranging from a to b. Then any number

greater than or equal to b (it could be a

or a+1, or a+2, …) is an upper bound

for this set. As the number increases

from a to b it moves closer to this upper

bound but at no point of time it is

greater than this upper bound.

5/1/2006 Algorithm Analysis & Design CS 007 BE CS 5th Semester 7

Big- Omega Notation

T(n) = W(g(n)) if there are positive

constants c and n0 such that T(n) >=

cg(n) for all n >= n0. This notation is

known as Big-Omega notation

5/1/2006 Algorithm Analysis & Design CS 007 BE CS 5th Semester 8

Big- Omega Notation

The Big-Omega notation can be

considered as a lower bound for the

T(n) which is the actual running time of

an algorithm. Informally W(g(n)) denotes

the set of all functions with a larger or

same order of growth as g(n). For

example, n2 belongs

W(n).

5/1/2006 Algorithm Analysis & Design CS 007 BE CS 5th Semester 9

Big-Theta Notation

T(n) = Q(g(n)) if there are positive

constants c1, c2 and n0 such that

c2 <= T(n) <= cg(n), for all n >= n0. This

notation is known as Big-Theta notation

5/1/2006 Algorithm Analysis & Design CS 007 BE CS 5th Semester 10

General rulesIn expressing running times, each elementary step

such as an assignment, an addition or an

initialization is counted as one unit of time

Leading constants and lower order terms are

ignored

The running time of a for loop, is at most the

running time of the statements inside the for loop

(including tests) multiplied by the number of

iterations

5/1/2006 Algorithm Analysis & Design CS 007 BE CS 5th Semester 11

General rules

Nested loops should be analyzed inside out. The total running time for a statement inside innermost loop is given by its running time multiplied by the product of the sizes of all for loops

The running time of an if/else statement is not more than the running time of the test, plus the larger of the running times of statements contained inside if and else conditions

5/1/2006 Algorithm Analysis & Design CS 007 BE CS 5th Semester 12

Big-Oh Example

Example: the function n2 is not O(n)

n2 cn

n c

The above inequality cannot be satisfied since c must be a

constant

1

10

100

1,000

10,000

100,000

1,000,000

1 10 100 1,000n

n^2

100n

10n

n

5/1/2006 Algorithm Analysis & Design CS 007 BE CS 5th Semester 13

More Big-Oh Examples

7n-27n-2 is O(n)

need c > 0 and n0 1 such that 7n-2 c•n for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 53n3 + 20n2 + 5 is O(n3)

need c > 0 and n0 1 such that 3n3 + 20n2 + 5 c•n3 for n n0

this is true for c = 4 and n0 = 21

3 log n + log log n

3 log n + log log n is O(log n)

need c > 0 and n0 1 such that 3 log n + log log n c•log n for n n0

this is true for c = 4 and n0 = 2