HEAPS

HEAPS

Amihood Amir

Bar Ilan University2014

Sorting

Bubblesort:Until no exchanges:

For i=1 to n-1

if A[i]>A[i+1] then exchange their values

end

end

Time: O(n2)

"Bubbling Up" the Largest Element

Traverse a collection of elements– Move from the front to the end– “Bubble” the largest value to the end

using pair-wise comparisons and swapping

512354277 101

1 2 3 4 5 6

Swap42 77




512357742 101

1 2 3 4 5 6

Swap35 77




512773542 101

1 2 3 4 5 6

Swap12 77




577123542 101

1 2 3 4 5 6

No need to swap




577123542 101

1 2 3 4 5 6

Swap5 101




77123542 5

1 2 3 4 5 6

101

Largest value correctly placed

Sorting

Selection sort:For i=1 to n do

For j=i+1 to n do

If A[i]>A[j] then exchange them

end

end

Time: = O(n2)

n

i

i1

Selecting the Smallest Element

Traverse a collection of elements– Move from the front to the end– Select the smallest value using pair-wise

comparisons and swapping

512354277 101

1 2 3 4 5 6

Swap42 77




512357742 101

1 2 3 4 5 6

Swap

35 42




51277 101

1 2 3 4 5 6

Swap

35 4212 35




577 101

1 2 3 4 5 6

4212 35

No need to swap




577 101

1 2 3 4 5 6

4212 35

Swap

5 12




77 101

1 2 3 4 5 6

42 355 12

Smallest value correctly placed

Sorting

Insertion sort:

For i=1 to n do

Insert A[i+1] into appropriate (sorted)

position in A[1],…,A[i]

end

Time: = O(n2)

n

i

i1

Keeping Prefix Sorted

Traverse a collection of elements– Move from the front to the end– Keep the prefix sorted throughout

512354277 101

1 2 3 4 5 6



512357742 101

1 2 3 4 5 6



512773542 101

1 2 3 4 5 6



512774235 101

1 2 3 4 5 6



577124235 101

1 2 3 4 5 6



577421235 101

1 2 3 4 5 6



577423512 101

1 2 3 4 5 6



10177423512 5

1 2 3 4 5 6



101 5423512 77

1 2 3 4 5 6



101 42 53512 77

1 2 3 4 5 6



101 42 35 512 77

1 2 3 4 5 6



101 42 35 12 5 77

1 2 3 4 5 6

Complexity

The Time of all algorithms we saw:

O(n2)

Can we do better?

Merge Sort

Based on the Merging operation.

Given two sorted arrays: A[1],…,A[n] and B[1],…,B[m].

merge them into one sorted array: C[1],…,C[n+m]

MergingPa, Pb, Pc <- 1

While (Pa < n+1 and Pb < m+1)

If A[Pa] ≤ B[Pb] then C[Pc] <- A[Pa]

Pa <- Pa+1

Pc <- Pc+1

else C[Pc] <- B[Pb]

Pb <- Pb+1

Pc <- Pc+1

end

MergingIf Pa=n+1 and Pb < m+1

then for i=Pb to m do

C[Pc] <- B[i]Pc <- Pc +1

endIf Pb=n+1 and Pa < m+1

then for i=Pa to n doC[Pc] <- A[i]Pc <- Pc +1

endend Algorithm

Merging

3102354 152575A: B:

C:

Merging

3102354 52575

1

A: B:

C:

Merging

102354 52575

13

A: B:

C:

Merging

102354 2575

135

A: B:

C:

Merging

2354 2575

13510

A: B:

C:

Merging

54 2575

1351023

A: B:

C:

MergingMerging

54 75

135102325

A: B:

C:

Merging

75

13510232554

A: B:

C:

Merging

1351023255475

A: B:

C:

Time: O(n+m) Linear !!!

Merge SortA recursive sorting algorithm:Mergesort(A)If n=1 then Return(A) else

Split A[1],…,A[n] to two length n/2 arrays:

A[1],…,A[n/2] and A[n/2+1],…,A[n]

Mergesort(A[1],…,A[n/2])

Mergesort(A[n/2+1],…,A[n])

Merge(A[1],…,A[n/2], A[n/2+1],…,A[n], B)

A <- B

Return(A)

Merge Sort Example996861558358640


9968615 58358640


9968615 58358640

8615996 5835 8640


9968615 58358640

8615996 5835 8640

99 6 86 15 58 35 86 40


9968615 58358640

8615996 5835 8640

99 6 86 15 58 35 86 40

4 0

Merge Sort Example

99 6 86 15 58 35 86 04

4 0Merge

Merge Sort Example

1586699 5835 0486

99 6 86 15 58 35 86 04

Merge

Merge Sort Example

6158699 04355886

1586699 5835 0486

Merge


6158699 04355886

Merge

Merge Sort TimeThe recurrence:

TM(n)=2TM(n/2)+n

TM(1)=1

Closed Form of TM(n):

O(n log n)

A Data Structures AlgorithmSort array A using AVL Trees:

For i=1 to n do:

Insert A[i] into AVL tree

For i=1 to n do:

Extract smallest element from AVL tree and put in A[i]

Sorting Algorithm using AVLCan we extract the smallest element

from AVL tree?

-- Yes. Go all the way to the left.

Time: O(log n)Therefore:

Sorting Algorithm Time: O(n log n)

Priority QueueWhat property of the AVL Tree did we

use?

Only that we can find the minimum fast.

A Priority Queue is a data structure that supports the following operations:

INSERT (Q,k)

EXTRACTMIN(Q,m)

Priority Queue Implementation

Heap:A binary tree where every vertex has a

key.For every vertex v, key(v) ≤ key(w),

where w is a child of v.

Difference between heap and binary search tree:10 10

5 3040 30

Heap Binary search tree

HeapAdditional Heap Property:All levels of the heap are full, except

possibly the last level, which is left-filled. 4

6

207

811

5

9

1214

15

2516

Example:

Heap PropertiesThe keys on every path from the root to

a leaf are nondecreasing.

4

6

207

811

5

9

1214

15

2516

Example:

Heap Properties

The height of a heap with n elements is O(log n).

)1(log n

4

6

207

811

5

9

1214

15

2516

Example:

1

0 0

12222h

i

h

i

hihi n h < log n ≤ h+1

Heap Insertion

Insert 6

3

35232822

8201021

74

25 6

Heap Insertion

Insert 6

3

35232822

8201021

74

25 620

6

Heap Insertion

Insert 6

3

35232822

81021

74

25 20

6

6

7

OK

Heap Deletion

Delete Min 3

3

35232822

81021

4

25 20

6

7

20

Now fix heap

Heap Deletion

Delete Min 3

35232822

81021

4

25

6

7

20

20

4

Heap Deletion

Delete Min 3

35232822

81021

25

6

720

20

4

10

OK

Heap Operations Time

O(log n)

Conclude: Heap of n elements can be constructed in time:

O(n log n)

But: We can do a lot better!!!

Build Heap

Put all elements in a tree where all levels are full and the last level is left-filled.

Construct heaps from leaves up

Time Analysis

For trees in level h : 1For trees in level h-1 : 2

For trees in level h-i : i

For trees in level 0 : log n

Time Analysis

Number of trees in level h: n/2Number of trees in level h-1: n/22

……Number of trees in level h-i:n/2i+1

…Number of trees in level 0 : 1

Total Time

Calculate it: Note that n/2 trees do constant time work, and another quarter need to go to two levels. Only one tree needs to fix log n levels.

Formally:

Need to calculate this

n

i

n

iiin

ini

nn

nnnn log

1 1log22 22

log2

...32

22

12

Total Time

We know: for x<1.

Differentiate both sides:

Multiply both sides by x:

1 1

1

i

i

xx

12

1

)1(

1

i

i

xix

12)1(i

i

x

xix

Total Time

In our case: x = 1/2.

So total time: O(n) .

2)1( 4

1

21

12

21

21

21

n

i

ii

In our case:

Look Ma No Pointers

Our tree is almost complete. All levels are full and the last one is left-filled.

So write down all keys sequentially from top to bottom and left to right.

Getting rid of Pointers

Example:

3,4,6,21,10,7,8,22,28,23,35,25,20

3

35232822

871021

64

25 20

Calculate Locationnode i in level j = location 2j-1+i in array.

Location i in array = node in level .

1logmod ii

3

35232822

871021

64

25 20

ilog

Node j in row i is the 2i-1+j’s element.Its left son is the 2i+1-1+2(j-1)+1’s

element.

But 2i+1-1+2(j-1)+1 = 2i+1+2(j-1) = 2i+1+2j-2 = 2(2i-1+j).

Location of children

row i j

Conclude:

Location of left son of i is 2i.Location of right son of i is 2i+1.Location of father of i is: i/2, if i is even (i-1)/2, if i is odd

Location of Children

How?Given array A of n elements:Construct a heap HA in-place.For i=n downto 1 do

EXTRACTMIN(HA,v)Put v in location A[i]

Endfor

Time: O(n) for heap construction and O(nlog n) for loop.

Heap Sort

Note:Array A is now sorted in non-increasing

order.What do we do if we want to sort in

non-decreasing order?Options:• Reverse order of A in linear time in-

place.• Use MAX rather than MIN priority

Queue.

Heap Sort

HEAPS

Documents

Transcript of HEAPS