Binary heaps

99
ECE 250 Algorithms and Data Structures Douglas Wilhelm Harder, M.Math. LEL Department of Electrical and Computer Engineering University of Waterloo Waterloo, Ontario, Canada ece.uwaterloo.ca [email protected] © 20143 by Douglas Wilhelm Harder. Some rights reserved. Binary heaps

description

Binary heaps. Outline. In this topic, we will: Define a binary min-heap Look at some examples Operations on heaps: Top Pop Push An array representation of heaps Define a binary max-heap Using binary heaps as priority queues. Definition. 7.2. A non-empty binary tree is a min-heap if - PowerPoint PPT Presentation

Transcript of Binary heaps

Page 1: Binary heaps

ECE 250 Algorithms and Data Structures

Douglas Wilhelm Harder, M.Math. LELDepartment of Electrical and Computer EngineeringUniversity of WaterlooWaterloo, Ontario, Canada

[email protected]

© 20143 by Douglas Wilhelm Harder. Some rights reserved.

Binary heaps

Page 2: Binary heaps

2Binary heaps

Outline

In this topic, we will:– Define a binary min-heap– Look at some examples– Operations on heaps:

• Top• Pop• Push

– An array representation of heaps– Define a binary max-heap– Using binary heaps as priority queues

Page 3: Binary heaps

3Binary heaps

Definition

A non-empty binary tree is a min-heap if– The key associated with the root is less than or equal to the keys

associated with either of the sub-trees (if any)– Both of the sub-trees (if any) are also binary min-heaps

From this definition:– A single node is a min-heap– All keys in either sub-tree are greater than the root key

7.2

Page 4: Binary heaps

4Binary heaps

Definition

Important:THERE IS NO OTHER RELATIONSHIP BETWEEN

THE ELEMENTS IN THE TWO SUBTREES

Failing to understand this is the greatest mistake a student makes

7.2

Page 5: Binary heaps

5Binary heaps

Example

This is a binary min-heap:

7.2

Page 6: Binary heaps

6Binary heaps

Example

Adding colour, we observe – The left subtree has the smallest (7) and the largest (89) objects– No relationship between items with similar priority

7.2

Page 7: Binary heaps

7Binary heaps

Operations

We will consider three operations:– Top– Pop– Push

7.2.1

Page 8: Binary heaps

8Binary heaps

Example

We can find the top object in Q(1) time: 3

7.2.1.1

Page 9: Binary heaps

9Binary heaps

Pop

To remove the minimum object:– Promote the node of the sub-tree which has the least value– Recurs down the sub-tree from which we promoted the least value

7.2.1.2

Page 10: Binary heaps

10Binary heaps

Pop

Using our example, we remove 3:

7.2.1.2

Page 11: Binary heaps

11Binary heaps

Pop

We promote 7 (the minimum of 7 and 12) to the root:

7.2.1.2

Page 12: Binary heaps

12Binary heaps

Pop

In the left sub-tree, we promote 9:

7.2.1.2

Page 13: Binary heaps

13Binary heaps

Pop

Recursively, we promote 19:

7.2.1.2

Page 14: Binary heaps

14Binary heaps

Pop

Finally, 55 is a leaf node, so we promote it and delete the leaf

7.2.1.2

Page 15: Binary heaps

15Binary heaps

Pop

Repeating this operation again, we can remove 7:

7.2.1.2

Page 16: Binary heaps

16Binary heaps

Pop

If we remove 9, we must now promote from the right sub-tree:

7.2.1.2

Page 17: Binary heaps

17Binary heaps

Push

Inserting into a heap may be done either:– At a leaf (move it up if it is smaller than the parent)– At the root (insert the larger object into one of the subtrees)

We will use the first approach with binary heaps– Other heaps use the second

7.2.1.3

Page 18: Binary heaps

18Binary heaps

Push

Inserting 17 into the last heap– Select an arbitrary node to insert a new leaf node:

7.2.1.3

Page 19: Binary heaps

19Binary heaps

Push

The node 17 is less than the node 32, so we swap them

7.2.1.3

Page 20: Binary heaps

20Binary heaps

Push

The node 17 is less than the node 31; swap them

7.2.1.3

Page 21: Binary heaps

21Binary heaps

Push

The node 17 is less than the node 19; swap them

7.2.1.3

Page 22: Binary heaps

22Binary heaps

Push

The node 17 is greater than 12 so we are finished

7.2.1.3

Page 23: Binary heaps

23Binary heaps

Push

Observation: both the left and right subtrees of 19 were greater than 19, thus we are guaranteed that we don’t have to send the new node down

This process is called percolation, that is, the lighter (smaller) objects move up from the bottom of the min-heap

7.2.1.3

Page 24: Binary heaps

24Binary heaps

Implementations

With binary search trees, we introduced the concept of balance

From this, we looked at:– AVL Trees– B-Trees– Red-black Trees (not course material)

How can we determine where to insert so as to keep balance?

7.2.2

Page 25: Binary heaps

25Binary heaps

Implementations

There are multiple means of keeping balance with binary heaps:– Complete binary trees– Leftist heaps– Skew heaps

We will look at using complete binary trees– In has optimal memory characteristics but sub-optimal run-time

characteristics

7.2.2

Page 26: Binary heaps

26Binary heaps

Complete Trees

By using complete binary trees, we will be able to maintain, with minimal effort, the complete tree structure

We have already seen– It is easy to store a complete tree as an array

If we can store a heap of size n as an array of size Q(n), this would be great!

7.2.2

Page 27: Binary heaps

27Binary heaps

Complete Trees

For example, the previous heap may be represented as the following (non-unique!) complete tree:

7.2.2

Page 28: Binary heaps

28Binary heaps

Complete Trees: Push

If we insert into a complete tree, we need only place the new node as a leaf node in the appropriate location and percolate up

7.2.2

Page 29: Binary heaps

29Binary heaps

Complete Trees: Push

For example, push 25:

7.2.2

Page 30: Binary heaps

30Binary heaps

Complete Trees: Push

We have to percolate 25 up into its appropriate location– The resulting heap is still a complete tree

7.2.2

Page 31: Binary heaps

31Binary heaps

Complete Trees: Pop

Suppose we want to pop the top entry: 12

7.2.2

Page 32: Binary heaps

32Binary heaps

Complete Trees: Pop

Percolating up creates a hole leading to a non-complete tree

7.2.2

Page 33: Binary heaps

33Binary heaps

Complete Trees: Pop

Alternatively, copy the last entry in the heap to the root

7.2.2

Page 34: Binary heaps

34Binary heaps

Complete Trees: Pop

Now, percolate 36 down swapping it with the smallest of its children– We halt when both children are larger

7.2.2

Page 35: Binary heaps

35Binary heaps

Complete Trees: Pop

The resulting tree is now still a complete tree:

7.2.2

Page 36: Binary heaps

36Binary heaps

Complete Trees: Pop

Again, popping 15, copy up the last entry: 88

7.2.2

Page 37: Binary heaps

37Binary heaps

Complete Trees: Pop

This time, it gets percolated down to the point where it has no children

7.2.2

Page 38: Binary heaps

38Binary heaps

Complete Trees: Pop

In popping 17, 53 is moved to the top

7.2.2

Page 39: Binary heaps

39Binary heaps

Complete Trees: Pop

And percolated down, again to the deepest level

7.2.2

Page 40: Binary heaps

40Binary heaps

Complete Trees: Pop

Popping 19 copies up 39

7.2.2

Page 41: Binary heaps

41Binary heaps

Complete Trees: Pop

Which is then percolated down to the second deepest level

7.2.2

Page 42: Binary heaps

42Binary heaps

Complete Tree

Therefore, we can maintain the complete-tree shape of a heap

We may store a complete tree using an array:– A complete tree is filled in breadth-first traversal order– The array is filled using breadth-first traversal

7.2.3

Page 43: Binary heaps

43Binary heaps

Array Implementation

For the heap

a breadth-first traversal yields:

7.2.3.1

Page 44: Binary heaps

44Binary heaps

Array Implementation

Recall that If we associate an index–starting at 1–with each entry in the breadth-first traversal, we get:

Given the entry at index k, it follows that:– The parent of node is a k/2 parent = k >> 1;– the children are at 2k and 2k + 1 left_child = k << 1;

right_child = left_child | 1;

Cost (trivial): start array at position 1 instead of position 0

7.2.3.1

Page 45: Binary heaps

45Binary heaps

Array Implementation

The children of 15 are 17 and 32:

7.2.3.1

Page 46: Binary heaps

46Binary heaps

Array Implementation

The children of 17 are 25 and 19:

7.2.3.1

Page 47: Binary heaps

47Binary heaps

The children of 32 are 41 and 36:

Array Implementation7.2.3.1

Page 48: Binary heaps

48Binary heaps

Array Implementation

The children of 25 are 33 and 55:

7.2.3.1

Page 49: Binary heaps

49Binary heaps

Array Implementation

If the heap-as-array has count entries, then the next empty node in the corresponding complete tree is at location posn = count + 1

We compare the item at location posn with the item at posn/2

If they are out of order– Swap them, set posn /= 2 and repeat

7.2.3.1

Page 50: Binary heaps

50Binary heaps

Consider the following heap, both as a tree and in its array representation

Array Implementation7.2.3.2

Page 51: Binary heaps

51Binary heaps

Array Implementation: Push

Inserting 26 requires no changes

7.2.3.2.1

Page 52: Binary heaps

52Binary heaps

Array Implementation: Push

Inserting 8 requires a few percolations:– Swap 8 and 23

7.2.3.2.1

Page 53: Binary heaps

53Binary heaps

Array Implementation: Push

Swap 8 and 12

7.2.3.2.1

Page 54: Binary heaps

54Binary heaps

Array Implementation: Push

At this point, it is greater than its parent, so we are finished

7.2.3.2.1

Page 55: Binary heaps

55Binary heaps

As before, popping the top has us copy the last entry to the top

Array Implementation: Pop7.2.3.2.2

Page 56: Binary heaps

56Binary heaps

Array Implementation: Pop

Instead, consider this strategy:– Copy the last object, 23, to the root

7.2.3.2.2

Page 57: Binary heaps

57Binary heaps

Array Implementation: Pop

Now percolate downCompare Node 1 with its children: Nodes 2 and 3– Swap 23 and 6

7.2.3.2.2

Page 58: Binary heaps

58Binary heaps

Array Implementation: Pop

Compare Node 2 with its children: Nodes 4 and 5– Swap 23 and 9

7.2.3.2.2

Page 59: Binary heaps

59Binary heaps

Array Implementation: Pop

Compare Node 4 with its children: Nodes 8 and 9– Swap 23 and 10

7.2.3.2.2

Page 60: Binary heaps

60Binary heaps

Array Implementation: Pop

The children of Node 8 are beyond the end of the array:– Stop

7.2.3.2.2

Page 61: Binary heaps

61Binary heaps

Array Implementation: Pop

The result is a binary min-heap

7.2.3.2.2

Page 62: Binary heaps

62Binary heaps

Array Implementation: Pop

Dequeuing the minimum again:– Copy 26 to the root

7.2.3.2.2

Page 63: Binary heaps

63Binary heaps

Array Implementation: Pop

Compare Node 1 with its children: Nodes 2 and 3– Swap 26 and 8

7.2.3.2.2

Page 64: Binary heaps

64Binary heaps

Array Implementation: Pop

Compare Node 3 with its children: Nodes 6 and 7– Swap 26 and 12

7.2.3.2.2

Page 65: Binary heaps

65Binary heaps

Array Implementation: Pop

The children of Node 6, Nodes 12 and 13 are unoccupied– Currently, count == 11

7.2.3.2.2

Page 66: Binary heaps

66Binary heaps

Array Implementation: Pop

The result is a min-heap

7.2.3.2.2

Page 67: Binary heaps

67Binary heaps

Array Implementation: Pop

Dequeuing the minimum a third time:– Copy 15 to the root

7.2.3.2.2

Page 68: Binary heaps

68Binary heaps

Array Implementation: Pop

Compare Node 1 with its children: Nodes 2 and 3– Swap 15 and 9

7.2.3.2.2

Page 69: Binary heaps

69Binary heaps

Array Implementation: Pop

Compare Node 2 with its children: Nodes 4 and 5– Swap 15 and 10

7.2.3.2.2

Page 70: Binary heaps

70Binary heaps

Array Implementation: Pop

Compare Node 4 with its children: Nodes 8 and 9– 15 < 23 and 15 < 25 so stop

7.2.3.2.2

Page 71: Binary heaps

71Binary heaps

Array Implementation: Pop

The result is a properly formed binary min-heap

7.2.3.2.2

Page 72: Binary heaps

72Binary heaps

Array Implementation: Pop

After all our modifications, the final heap is

7.2.3.2.2

Page 73: Binary heaps

73Binary heaps

Run-time Analysis

Accessing the top object is Q(1)Popping the top object is O(ln(n))– We copy something that is already in the lowest depth—it will likely be

moved back to the lowest depth

How about push?

7.2.4

Page 74: Binary heaps

74Binary heaps

Run-time Analysis

If we are inserting an object less than the root (at the front), then the run time will be Q(ln(n))

If we insert at the back (greater than any object) then the run time will be Q(1)

How about an arbitrary insertion?– It will be O(ln(n))? Could the average be less?

7.2.4

Page 75: Binary heaps

75Binary heaps

Run-time Analysis

With each percolation, it will move an object past half of the remaining entries in the tree– Therefore after one percolation, it will probably be past half of the

entries, and therefore on average will require no more percolations

Therefore, we have an average run time of Q(1)

1

0

1 2 2( )2

1 (1)

hhk

k

hh kn n

n hn

Θ

7.2.4

Page 76: Binary heaps

76Binary heaps

Run-time Analysis

An arbitrary removal requires that all entries in the heap be checked: O(n)

A removal of the largest object in the heap still requires all leaf nodes to be checked – there are approximately n/2 leaf nodes: O(n)

7.2.4

Page 77: Binary heaps

77Binary heaps

Run-time Analysis

Thus, our grid of run times is given by:

7.2.4

Page 78: Binary heaps

78Binary heaps

Run-time Analysis

Some observations:– Continuously inserting at the front of the heap (i.e., the new object being

pushed is less than everything in the heap) causes the run-time to drop to O(ln(n))

– If the objects are coming in order of priority, use a regular queue with swapping

– Merging two binary heaps of size n is a Q(n) operation

7.2.4

Page 79: Binary heaps

79Binary heaps

Run-time Analysis

Other heaps have better run-time characteristics– Leftist, skew, binomial and Fibonacci heaps all use a node-based

implementation requiring Q(n) additional memory– For Fibonacci heaps, the run-time of all operations (including merging

two Fibonacci heaps) except pop are Q(1)

7.2.4

Page 80: Binary heaps

80Binary heaps

Binary Max Heaps

A binary max-heap is identical to a binary min-heap except that the parent is always larger than either of the children

For example, the same data as before stored as a max-heap yields

7.2.5

Page 81: Binary heaps

81Binary heaps

Example

Here we have a max-heap of presents under a red-green tree:

http://xkcd.com/835/

7.2.5

Page 82: Binary heaps

82Binary heaps

Memory allocation and pointer arithmetic

Do we really have to allocate one additional memory location for a binary tree-as-heap?

Type *heap_array = new Type[capacity() + 1];

Could we not just allocate one less memory and point to the previous location in memory

0 1 2 3 4 5 6 7 8 9

heap_array

0 1 2 3 4 5 6 7 8

heap_array

Page 83: Binary heaps

83Binary heaps

Memory allocation and pointer arithmetic

To do this, we must understand pointer arithmetic:int *ptr = new int[5] = {1, 23, 45, 67, 89};std::cout << ptr << std::endl;std::cout << *ptr << std::endl;

00d3a260 00d3a264 00d3a268 00d3a26B 00d3a27000000001 00000017 0000002D 00000043 00000059

ptr

What is the output of?std::cout << (ptr + 1) << std::endl;std::cout << *(ptr + 1) << std::endl;

Page 84: Binary heaps

84Binary heaps

Memory allocation and pointer arithmetic

Just adding one to the address would be, in almost all cases, useless– Assuming big endian, this would have a value 256– If this was little endian, it would be even more bizarre…

00d3a260 00d3a264 00d3a268 00d3a26B 00d3a27000000001 00000017 0000002D 00000043 00000059

ptr

What is the output of?std::cout << (ptr + 1) << std::endl;std::cout << *(ptr + 1) << std::endl;

Page 85: Binary heaps

85Binary heaps

Memory allocation and pointer arithmetic

Instead, C and C++ add as many bytes as the size of the object being pointed to– In the cases of int, sizeof( int ) == 4 on most 32-bit machines– The output is 23

00d3a260 00d3a264 00d3a268 00d3a26B 00d3a27000000001 00000017 0000002D 00000043 00000059

ptr

What is the output of?std::cout << (ptr + 1) << std::endl;std::cout << *(ptr + 1) << std::endl;

Page 86: Binary heaps

86Binary heaps

Memory allocation and pointer arithmetic

Essentially, these two statements are identical:std::cout << ptr[i] << std::endl;std::cout << *(ptr + i) << std::endl;

Now you can do the following:Type *tmp = new Type[capacity()];Type *heap_array = tmp - 1;

Now, heap_array[1]; and tmp[0]; both point to the same memory location

00d3a260 00d3a264 00d3a268 00d3a26B 00d3a27000000001 00000017 0000002D 00000043 00000059

tmpheap_array

Page 87: Binary heaps

87Binary heaps

Memory allocation and pointer arithmetic

Issues:– Never access or modify the contents of heap_array[0]– When you deallocate memory, you must point to the original address

returned by new:delete [] (heap_array + 1);

– Pointer arithmetic is not for the faint of heart but it is fun; for example:int array[N];int *ptr = array;

// Print all the entries of the arrayfor ( int i = 0; i < N; ++i ) { std::cout << *(ptr++) << std::endl;}

Page 88: Binary heaps

88Binary heaps

Priority Queues

Now, does using a heap ensure that that object in the heap which:– has the highest priority, and– of that highest priority, has been in the heap the longest

Consider inserting seven objects, all of the same priority (colour indicates order):

2, 2, 2, 2, 2, 2, 2

7.2.6

Page 89: Binary heaps

89Binary heaps

Priority Queues

Whatever algorithm we use for promoting must ensure that the first object remains in the root position– Thus, we must use an insertion technique where we only percolate up if

the priority is lowerThe result:

Challenge:– Come up with an algorithm which removes all seven objects in the

original order

7.2.6

Page 90: Binary heaps

90Binary heaps

Lexicographical Ordering

A better solution is to modify the priority:– Track the number of insertions with a counter k (initially 0)– For each insertion with priority n, create a hybrid priority (n, k) where:

(n1, k1) < (n2, k2) if n1 < n2 or (n1 = n2 and k1 < k2)

7.2.6

Page 91: Binary heaps

91Binary heaps

Priority Queues

Removing the objects would be in the following order:

7.2.6

Page 92: Binary heaps

92Binary heaps

Priority Queues

Popped: 2– First, (2,1) < (2, 2) and (2, 3) < (2, 4)

7.2.6

Page 93: Binary heaps

93Binary heaps

Priority Queues

Removing the objects would be in the following order:

7.2.6

Page 94: Binary heaps

94Binary heaps

Priority Queues

Removing the objects would be in the following order:

7.2.6

Page 95: Binary heaps

95Binary heaps

Priority Queues

Removing the objects would be in the following order:

7.2.6

Page 96: Binary heaps

96Binary heaps

Priority Queues

Removing the objects would be in the following order:

7.2.6

Page 97: Binary heaps

97Binary heaps

Summary

In this talk, we have:– Discussed binary heaps– Looked at an implementation using arrays– Analyzed the run time:

• Head Q(1)• Push Q(1) average• Pop O(ln(n))

– Discussed implementing priority queues using binary heaps– The use of a lexicographical ordering

7.2.6

Page 98: Binary heaps

98Binary heaps

References

[1] Donald E. Knuth, The Art of Computer Programming, Volume 3: Sorting and Searching, 2nd Ed., Addison Wesley, 1998, §7.2.3, p.144.

[2] Cormen, Leiserson, and Rivest, Introduction to Algorithms, McGraw Hill, 1990, §7.1-3, p.140-7.

[3] Weiss, Data Structures and Algorithm Analysis in C++, 3rd Ed., Addison Wesley, §6.3, p.215-25.

Page 99: Binary heaps

99Binary heaps

Usage Notes

• These slides are made publicly available on the web for anyone to use

• If you choose to use them, or a part thereof, for a course at another institution, I ask only three things:– that you inform me that you are using the slides,– that you acknowledge my work, and– that you alert me of any mistakes which I made or changes which you

make, and allow me the option of incorporating such changes (with an acknowledgment) in my set of slides

Sincerely,Douglas Wilhelm Harder, [email protected]