SORTING TECHNIQUES

7/18/2019 SORTING TECHNIQUES

1/52

Data Structure and Algorithm Ana

Slide Set 8

13 CS- I & IIEngr. Maria Shaikh

[email protected]


2/52

Merge Sort

Divide-and-conquertechnique for algorithm design. E

the merge sort.

Writing and solving recurrences

Engr. Maria Shaikh


3/52

Divide and Conquer

Very important strategy in computer science:

Divide problem into smaller parts

Independently solve the parts

Combine these solutions to get overall solution

Idea 1: Divide array into two halves, recursively sort

right halves, then merge two halvesMergesort Idea 2 : Partition array into items that are small an

that are large, then recursively sort the two sets

Engr. Maria Shaikh


4/52

Divide and Conquer

Divide-and-conquermethod for algorithm desig

Divide: If the input size is too large to dealstraightforward manner, divide the problem into twdisjoint subproblems

Conquer: Use divide and conquer recursively to

subproblemsCombine: Take the solutions to the subprob

merge these solutions into a solution for thproblem

Engr. Maria Shaikh


5/52

Merge Sort

Divide it in two at the midpoint

Conquer each side in turn (by recursively sorting

Merge two halves together

8 2 9 4 5 3 1 6

Engr. Maria Shaikh


6/52

Merge Sort Example

8 2 9 4 5 3 1 6

8 2 1 69 4 5 3

8 2 9 4 5 3 1 6

2 8 4 9 3 5 1 6

2 4 8 9 1 3 5 6

1 2 3 4 5 6 8 9

Merge

Merge

Merge

Divide

Divide

Divide

1 element

8 2 9 4 5 3 1 6

Engr. Maria Shaikh


7/52

Auxiliary Array

The merging requires an auxiliary array.

2 4 8 9 1 3 5 6

Auxiliary arra

Engr. Maria Shaikh


8/52

Auxiliary Array


2 4 8 9 1 3 5 6

1Auxiliary arra

Engr. Maria Shaikh


9/52

Auxiliary Array


2 4 8 9 1 3 5 6

1 2 3 4 5 6 8 9Auxiliary arra

Engr. Maria Shaikh


10/52

Merge Sort (Example) - 2

Engr. Maria Shaikh


11/52


Engr. Maria Shaikh


12/52


Engr. Maria Shaikh


13/52


Engr. Maria Shaikh


14/52


Engr. Maria Shaikh


15/52


Engr. Maria Shaikh


16/52


Engr. Maria Shaikh


17/52


Engr. Maria Shaikh


18/52


Engr. Maria Shaikh


19/52


Engr. Maria Shaikh


20/52


Engr. Maria Shaikh


21/52


Engr. Maria Shaikh


22/52


Engr. Maria Shaikh


23/52


Engr. Maria Shaikh


24/52


Engr. Maria Shaikh


25/52


Engr. Maria Shaikh


26/52


Engr. Maria Shaikh


27/52


Engr. Maria Shaikh


28/52


Engr. Maria Shaikh


29/52


Engr. Maria Shaikh


30/52


Engr. Maria Shaikh


31/52


Engr. Maria Shaikh


32/52

Merge Sort Revisited

To sort n numbers

if n=1 done! recursively sort 2 lists of numbers

merge 2 sorted lists in Q(n) time

Strategy

break problem into similar (smaller)subproblems

recursively solve subproblems

combine solutions to answer

Engr. Maria Shaikh


33/52

Running Time of an Algorithm

Depends upon

Input Size Nature of Input

Generally time grows with size of input, so running timalgorithm is usually measured as function of input size.

Running time is measured in terms of number of steps/poperations performed.

Independent from machine, OS

33Engr. Maria Shaikh

Finding running time of an Algorit


34/52

Finding running time of an AlgoritAnalyzing an Algorithm

Running time is measured by number of steps/primitiv

performed.

Steps means elementary operation like

, + , * , < , = , A[i] etc

We will measure number of steps taken in term of size


Simple Example (1)


35/52

Simple Example (1)

// Input: int A[N], array of N integers

// Output: Sum of all numbers in array A

int Sum(int A[], int N)

{

int s=0;

for (int i=0; i< N; i++)

s = s + A[i];

return s;

}

How should we analyse this?


SIMPLE EXAMPLE (2)


36/52

SIMPLE EXAMPLE (2)

// Input: int A[N], array of N integers

// Output: Sum of all numbers in array A

int Sum(int A[], int N){

int s=0;

for (int i=0; i< N; i++)

s = s + A[i];

return s;}

1

2 3 4

56 7

8

1,2,8: Once

3,4,5,6,7: Once per each iterati

of for loop, N iterati

Total: 5N + 3

The complexity function of the

algorithm is :f(N) = 5N +3

l E l ( ) h f


37/52

Simple Example (3) Growth of 5

Estimated running time for different values of N:

N = 10 => 53 steps

N = 100 => 503 steps

N = 1,000 => 5003 steps

N = 1,000,000 => 5,000,003 steps

As N grows, the number of steps grow in linearproportfor this function Sum


Wh D P E


38/52

What Dominates in Previous Exa

What about the +3 and 5 in 5N+3? As N gets large, the +3 becomes insignificant

5 is inaccurate, as different operations require varying amounts of time and also doesignificant importance

What is fundamental is that the time is linear in N.

Asymptotic Complexity: As N gets large, concentrate onorder term: Drop lower order terms such as +3

Drop the constant coefficient of the highest order term i.e.


A i C l i


39/52

Asymptotic Complexity

The 5N+3 time bound is said to "grow asymptotically"

This gives us an approximation of the complexity of th

Ignores lots of (machine dependent) details, concentra

bigger picture


COMPARING FUNCTIONS: ASYMPTOT


40/52

NOTATION

Big Oh Notation: Upper bound

Omega Notation: Lower bound

Theta Notation: Tighter bound


Bi Oh N t ti [1]


41/52

Big Oh Notation [1]

If f(N) and g(N) are two complexity functions, we say

f(N) = O(g(N))

(read "f(N) is order g(N)", or "f(N) is big-O of g(N)")

if there are constants c and N0

such that for N > N0,

f(N) c * g(N)

for all sufficiently large N.


Bi Oh N t ti


42/52

Big-Oh Notation

The mathematical artifact that allows us to suppress detail whe

analyzing algorithms is called the O-notation, or "big-Oh nota Definition 1 A function g(N) is said to be O(f (N)) if there

constants co and No such that g(N) < co f (N) for all N > No.

We use the O-notation for three distinct purposes:

To bound the error that we make when we ignore small t

mathematical formulas To bound the error that we make when we ignore parts o

that contribute a small amount to the total being analyzed

To allow us to classify algorithms according to upper boutotal running times

Engr. Maria Shaikh

Big Oh Notation (Continue


43/52

Big-Oh Notation (Continue

Often, the results of a mathematical analysis are not exact, but rather are

a precise technical sense

The O-notation allows us to keep track of the leading terms while ignoring

when manipulating approximate mathematical expressions

For example, if we expand the expression:

(N + O (1)) (N + O (log N) + O(1)),

we get six terms: N2 + O (N) + O (N log N) + O (log N) + O (N) + O

but can drop all but the largest O-term, leaving the approximationN2 + O (N log N).

That is, N2 is a good approximation to this expression when N is larg

Engr. Maria Shaikh

Bi Oh N t ti [2]


44/52

Big Oh Notation [2]

O(f(n)) is a set of functions.

n = O(n2) means that function nbelongs to

the set of functions O(n2)


O ( f ( ) )


45/52

O ( f (n) )


Comparing Functions


46/52

Comparing Functions

As inputs get larger, any algorithm of a

smaller order will be more efficient thanan algorithm of a larger order

46

Time

(steps)

Input (size)

3N = O(N)

0.05 N2 = O(N2)

N = 60

Engr. Maria Shaikh

Big Omega Notation


47/52

Big Omega Notation

If we wanted to say running time is at least

use

Big Omega notation, , is used to express thelower bounds on a function.

If f(n) and g(n) are two complexity functions thewe can say:

f(n) is (g(n)) if there exist positive numbers c and n 0 such that 0=c (n) for all n>=n0

47

Big Theta Notation


48/52

Big Theta Notation

If we wish to express tight bounds we use the th

notation,

f(n) = (g(n)) means that f(n) = O(g(n)) and f(n(g(n))


What does this all mean?


49/52

What does this all mean?

If f(n) = (g(n)) we say that f(n) and g(n) grow at the

asymptotically

If f(n) = O(g(n)) and f(n) (g(n)), then we say that

asymptotically slower growing than g(n).

If f(n) = (g(n)) and f(n) O(g(n)), then we say that f

asymptotically faster growing than g(n).


Which Notation do we use?


50/52

Which Notation do we use?

To express the efficiency of our algorithms which of th

notations should we use?

As computer scientist we generally like to express our as big O since we would like to know the upper boundalgorithms.

Why?

If we know the worse case then we can aim to improveavoid it.


Performance Classification


51/52

f(n) Classification

1 Constant: run time is fixed, and does not depend upon n. Most instructions are executed once, or only a few the amount of information being processed

log n Logarithmic: when nincreases, so does run time, but much slower. Common in programs which solve large ptransforming them into smaller problems.

n Linear: run time varies directly withn. Typically, a small amount of processing is done on each element.

n log n When ndoubles, run time slightly more than doubles. Common in programs which break a problem down in

problems, solves them independently, then combines solutions

n2 Quadratic: when n doubles, runtime increases fourfold. Practical only for small problems; typically the progpairs of input (e.g. in a double nested loop).

n3 Cubic: when n doubles, runtime increases eightfold

2n Exponential: when n doubles, run time squares. This is often the result of a natural, brute force solution.


52/52

END OF SLID

SET 8

SORTING TECHNIQUES

Documents

Transcript of SORTING TECHNIQUES