1

ΔΟΜΕΣ ΔΕΔΟΜΕΝΩΝ & ΑΡΧΕΙΩΝ

Priority Queues (ουρές

προτεραιότητας)

HEAPS (σωροί) – binary heaps

HeapSort (ταξινόμηση σωρού)

2

Priority Queues

• A priority queue is an ADT where: – Each element has an associated priority– Efficient extraction of the highest-priority

element is supported.

• Typical operations of a priority queue are:– Enqueue (insert an element)– Dequeue (remove the highest-priority element)– Max (return the highest-priority element)– Meld (join two priority queues into one)

• Implementation : Binary Heap

3

Priority Queues: Applications

• Scheduling Examples: – Printer scheduling.

• Jobs sent to a printer by faculty have a higher priority than jobs sent by students

– Process scheduling.• Several processes require CPU time. Priority may

be based on various factors:– Shortest jobs have higher priority– Jobs with a shorter remaining time have higher priority– Jobs that have waited the longest have higher priority– etc.

4

Priority Queues: Applications

• Discrete-event simulations– Simulations of systems modeled as collections of

entities that interact with one another through events that occur at discrete times.• The priority assigned to an event is its start

time.– Examples:

• Truck fleet model – Consider the event "Truck leaves city A" with

priority 10:00am. Its execution (dequeue) triggers events "Truck arrives at depot" with priority 6:00pm, and "Truck has unloaded" with priority 7:00pm, which are then enqueued, based on their priorities.

• Circuit validation

5

Example

• How many aircraft are there in the air at peak traffic?

• How would you represent all the aircraft?– Arrays– Linked Lists– Trees

6

Characteristics of Flights

• Created and removed dynamically– Need to add and remove aircraft in realtime

• Different aircraft have different priorities– Different weights associated with

different aircraft– Length of time aircraft has been flying– Emergencies– Special aircraft (military/dignitaries)

7

Array Based Representation

• Removing Aircraft

•Removing takes 1 step, but copying takes

n

•Sorting Aircraft

– Insertion Sort -> n2 -> 25 * 106 operations

– Merge Sort -> n log n -> 20 * 103

operations

Heaps and priority queues

A heap is a data structure used to implement an efficient priority queue. The idea is to make it efficient to extract the element with the highest priority the next item in the queue to be processed.We could use a sorted linked list, with O(1) operations to remove the highest priority node and O(N) to insert a node. Using a tree structure will involve both operations being O(log2N) which is faster.

9

Tree Based Representation

10

Perfect Binary Tree

• For binary tree with height h– All nodes at levels h–1 or less have 2 children

(full)

h = 2 h = 4h = 3

h = 1

11

Complete Binary Trees

• For binary tree with height h – All nodes at levels h–2 or less have 2 children

(full)– All leaves on level h are as far left as possible

h = 2

h = 3

h = 1

12

Complete Binary Trees

h = 4

13

Complete Trees

• Nodes can be numbered in level-by-level order:

1

2 3

4 5 6 7

8 9 10 11 12

The left child of the i-th node is the node 2*i and the right child is the node 2*i + 1

The parent of the i-th node is the node i / 2

14

Complete Trees (cont’d)

• It is convenient to store a complete binary tree in an array (or an ArrayList) in the order of nodes

Argentina

Brazil Chile

Egypt Dominica Greece Italy

Haiti France

1

2 3

4 5 6 7

8 9

15

Heaps

• A complete binary tree is a full binary tree that has all leaves at the same depth

• Is the tree below complete?

• Heaps are nearly complete binary trees

16

Heaps

A heap is defined only on “nearly complete binary tress. What is a “nearly complete binary tree”?

A complete binary tree is one where all the internal nodes have exactly 2 children, and all the leaves are on the same level. An internal node comprises all the nodes except the leaf nodes.

Leaf nodes are nodes without children.

Leaf node

Interior node

Edge

17

Heaps

Height of a node: The number of edges starting at that node, and going down to the furthest leaf.

Height of the heap: The maximum number of edges from the root to a leaf.

Root

Height of blue node = 1

Height of root = 3

18

Heaps

Nearly complete – if not complete, then the only unfilled level is filled in from left to right.

23

4 56 7

8 9 10 11 12 13

1

23

4 56 7

8 9 10 11 12

1

Parent(i)

return i/2Left(i)

return 2i

Parent(i)

return 2i + 1

19

Heaps

• Two key properties– Complete binary tree– Value at node

• Smaller than or equal to values in subtrees (min-heaps)

• Example heap– X Y– X Z

Y

X

Z

20

Heap & Non-heap Examples

Heaps Non-heaps

6

2

22

8 45 25

6

2

22

8 45 25

8

6 45

5

6 22

25

5

5 45

5

21

Heap Properties

• Key operations– Insert ( X )– getSmallest ( )

• Key applications– Heapsort– Priority queue

22

Heap Operations – Insert( X )

• Algorithm1. Add X to end of tree2. While (X < parent)

Swap X with parent // X bubbles up tree

• Complexity– # of swaps proportional to height of tree– O( log(n) )

23

Heap Insert Example

• Insert ( 20 )

10

30 25

37

10

30 25

37 20

10

20 25

37 30

1) Insert to end of tree

2) Compare to parent, swap if parent key larger

3) Insert complete

24

Heap Insert Example

• Insert ( 8 )

10

30 25

37

10

30 25

37 8

10

8 25

37 30

8

10 25

37 30

1) Insert to end of tree

2) Compare to parent, swap if parent key larger

3) Insert complete

25

Heap Operation – getSmallest()

• Algorithm1. Get (remove) smallest node at root2. Replace root with X at end of tree3. While ( X > child )

Swap X with smallest child // X drops down tree

4. Return smallest node

• Complexity– # swaps proportional to height of tree– O( log(n) )

26

Heap GetSmallest Example

• getSmallest ()

8

10 25

37

30

10 25

37

10

30 25

3730

1) Replace root with end of tree

2) Compare node to children, if larger swap

with smallest child

3) Repeat swap if needed

27

Heap GetSmallest Example

• getSmallest ()

8

10 25

30

37

10 25

30

10

37 25

3037

1) Replace root with end of tree

2) Compare node to children, if larger swap

with smallest child

3) Repeat swap if needed

10

30 25

37

28

Binary Heaps

• Binary heap = – a binary tree that is – complete

• every level except possibly the bottom one is completely filled and the leaves in the bottom level are as far left as possible

– satisfies the heap property:• the key stored in every node is greater than

or equal to the keys stored in its children– this is called a max-heap. If the key at each

node is smaller than or equal to the keys of its children, then we have a min-heap.

29

Heaps

• The heap property of a tree is a condition that must be true for the tree to be considered a heap.

• Min-heap property: for min-heaps, requiresA[parent(i)] A[i]So, the root of any subtree holds the least value in that subtree.

• Max-heap property: for max-heaps, requiresA[parent(i)] A[i]The root of any subtree holds the greatest value in the subtree.

30

Heaps - example

• Looks like a max heap, but max-heap property is violated at indices 11 and 12. Why? because those nodes’ parents have larger smaller values.

2

83

84

85

76

87

1

8

69

710

511

812

9

1

9

31

Heaps

• The algorithm for removeMin uses the reheap-down procedure

• The algorithm for add uses the reheap-up procedure

Remove the root and place the last leaf at the root. Starting at the root, swap the node with its smaller child, as many times as needed to repair the heap.

Add a leaf. Starting at the last leaf, swap the node with its parent as many times as needed to repair the heap.

• Both adding and removing an item takes time proportional to the height of the heap (e.g., 20 for 1,000,000 nodes)

32

The removeMin Algorithm

France

Brazil Chile

Egypt Dominica

Greece Italy

Haiti

1

2 3

4 5 6 7

8

Brazil Chile

Egypt

Dominica Greece Italy

Haiti France

1

2 3

4 5 6 7

8 9

Argentina

Brazil

France Chile

Egypt Dominica Greece Italy

Haiti

1

2 3

4 5 6 7

8

Brazil

Dominica Chile

Egypt France Greece Italy

Haiti

1

2 3

4 5 6 7

8

Step 1: the root is

removed

Step 2: the last leaf replaces the root

Step 3: “reheap down”: the new root value moves down, swapping places with the smaller

child

33

The add Algorithm

Step 1: the new value is added as the last leaf

Step 2: “reheap up”: the new value moves up, swapping places with its parent

Brazil

Dominica Chile

Egypt France Greece Italy

Haiti

1

2 3

4 5 6 7

8 China

9

Brazil

Dominica Chile

China France Greece Italy

Haiti

1

2 3

4 5 6 7

8 Egypt

9

Brazil

China Chile

Dominica France Greece Italy

Haiti

1

2 3

4 5 6 7

8 Egypt

9

34

Binary Tree -> Array representation

• Use Breadth-First-Search• – The root node is A( 1 )• – Node i is A( i )

35

HeapsAssume we are given a tree represented by matrix A, and such that i length[A], and the trees rooted at Left(i) and Right(i) are heaps. Then, to create a heap rooted at A[i], use procedure Max-Heapify(A,i):

Max-Heapify(A,i)1. l Left(i) ) * l=left* 2. r Right(i)3. if l heap-size[A] and A[l] > A[i]4. then largest l5. else largest i6. if r heap-size[A] and A[r] > A[largest]7. then largest r8. if largest i9. then swap( A[i], A[largest])10. Max-Heapify(A, largest)

36

Heaps

• So, what must we do to build a heap? We call Max-Heapify(A,i) for every i starting at last node and going to the root.

• Why top-down?• Because Max-Heapify() moves the larger node

upward into the root. If we start at the top and go to larger node indices, the highest a node can move is to the root of that subtree. But what if there is a violation between the subtree’s root and its parent?

• So, we must start from the bottom and go towards the root to fix the problems caused by moving larger nodes into the root of subtrees.

37

Max-Heapify(A,1)

20 9

19 18 6 7

17 16 17 16 1

10

10 9

19 18 6 7

17 16 17 16 1

20

10 9

19 18 6 7

17 16 17 16 1

20

19 9

10 18 6 7

17 16 17 16 1

20

19 9

10 18 6 7

17 16 17 16 1

20

19 9

17 18 6 7

10 16 17 16 1

20

38

Heaps• In practice, heaps are usually implemented

as arrays:16

14 10

8 7 9 3

2 4 1

16 14 10 8 7 9 3 2 4 1A = =

39

Heaps

• To represent a complete binary tree as an array: – The root node is A[1]– Node i is A[i]– The parent of node i is A[i/2] (note: integer

divide)– The left child of node i is A[2i]– The right child of node i is A[2i + 1] 16

14 10

8 7 9 3

2 4 1

16 14 10 8 7 9 3 2 4 1A = =

40

Referencing Heap Elements

• So…

Parent(i) { return i/2; }

Left(i) { return 2*i; }

right(i) { return 2*i + 1; }

41

The Heap Property

• Heaps also satisfy the heap property:A[Parent( i )] A[ i ] for all nodes i > 1– In other words, the value of a node is at most the

value of its parent– Where is the largest element in a heap stored?

42

Heap Operations: Heapify()

• Heapify(): maintain the heap property

– Given: a node i in the heap with children l and r

– Given: two subtrees rooted at l and r, assumed to

be heaps

– Problem: The subtree rooted at i may violate the

heap property

– Action: let the value of the parent node “float

down” so subtree at i satisfies the heap property

43

Heapify() Example

16

4 10

14 7 9 3

2 8 1

16 4 10 14 7 9 3 2 8 1A =

44

Heapify() Example

16

4 10

14 7 9 3

2 8 1

16 10 14 7 9 3 2 8 1A = 4

45

Heapify() Example

16

4 10

14 7 9 3

2 8 1

16 10 7 9 3 2 8 1A = 4 14

46

Heapify() Example

16

14 10

4 7 9 3

2 8 1

16 14 10 4 7 9 3 2 8 1A =

47

Heapify() Example

16

14 10

4 7 9 3

2 8 1

16 14 10 7 9 3 2 8 1A = 4

48

Heapify() Example

16

14 10

4 7 9 3

2 8 1

16 14 10 7 9 3 2 1A = 4 8

49

Heapify() Example

16

14 10

8 7 9 3

2 4 1

16 14 10 8 7 9 3 2 4 1A =

50

Heapify() Example

16

14 10

8 7 9 3

2 4 1

16 14 10 8 7 9 3 2 1A = 4

51

Heapify() Example

16

14 10

8 7 9 3

2 4 1

16 14 10 8 7 9 3 2 4 1A =

52

HeapSort

53

Heap Application – Heapsort

• Use heaps to sort values– Heap keeps track of smallest element in heap

• Algorithm1. Create heap2. Insert values in heap3. Remove values from heap (in ascending order)

• Complexity– O( nlog(n))

54

Heapsort Example

• Input– 11, 5, 13, 6, 1

• View heap during insert, removal– As tree– As array

55

Heapsort – Insert Values

56

Heapsort – Remove Values

57

Heapsort – Insert in to Array 1

• Input– 11, 5, 13, 6, 1

Index = 0 1 2 3 4

Insert 11 11

58

Heapsort – Insert in to Array 2

• Input– 11, 5, 13, 6, 1

Index = 0 1 2 3 4

Insert 5 11 5

Swap 5 11

59

Heapsort – Insert in to Array 3

• Input– 11, 5, 13, 6, 1

Index = 0 1 2 3 4

Insert 13 5 11 13

60

Heapsort – Insert in to Array 4

• Input– 11, 5, 13, 6, 1

Index = 0 1 2 3 4

Insert 6 5 11 13 6

Swap 5 6 13 11

…

61

Heapsort – Remove from Array 1

• Input– 11, 5, 13, 6, 1

Index = 0 1 2 3 4

Remove root 1 5 13 11 6

Replace 6 5 13 11

Swap w/ child 5 6 13 11

62

Heapsort – Remove from Array 2

• Input– 11, 5, 13, 6, 1

Index = 0 1 2 3 4

Remove root 5 6 13 11

Replace 11 6 13

Swap w/ child 6 11 13

63

Example of Heapsort(A)

Illustrate the operation of Heapsort(A) on the array A = <27, 17, 3, 16, 13, 10, 1, 5, 7, 12, 4, 8, 9, 0>

Note: Since there a 14 nodes, and log 8 = 3 < log 14 < log

16 = 4, there are 4 levels (0,1,2 and 3) in the tree

17 3

16 13 10 1

5 7 12 4 8 9

27

0

64

17 3

16 13 10 1

5 7 12 4 8 9

27

0

Heapsort(A)

1. Build-Max-Heap(A)

2. for i length[A] downto 2

3. exchange A[1] A[i]

4. heap-size[A] heap-size[A]-1

5. Max-Heapify(A,1)

Build-Max-Heap(A)

1. heap-size[A] length[A]

2. for i length[A]/2 downto 1

3. do Max-Heapify(A,i)

17 10

16 13 3 1

5 7 12 4 8 9

27

0

17 10

16 13 9 1

5 7 12 4 8 3

27

0

Build-Max-Heap(A) is done:

A is a max-heap.

Now onto lines 2-5 of Heapsort

65

17 10

16 13 9 1

5 7 12 4 8 3

27

0

Heapsort(A)

1. Build-Max-Heap(A)

2. for i length[A] downto 2

3. exchange A[1] A[i]

4. heap-size[A] heap-size[A]-1

5. Max-Heapify(A,1)

17 10

16 13 9 1

5 7 12 4 8 3

0

27

exchange A[1] A[i]

0 10

16 13 9 1

5 7 12 4 8 3

17

27

Max-Heapify(A,1)

16 10

0 13 9 1

5 7 12 4 8 3

17

27

16 10

7 13 9 1

5 0 12 4 8 3

17

27

66

16 10

7 13 9 1

5 0 12 4 8 17

3

27

exchange A[1] A[i] Max-Heapify(A,1)

16 10

7 13 9 1

5 0 12 4 8 17

3

27

3 10

7 13 9 1

5 0 12 4 8 17

16

27

13 10

7 3 9 1

5 0 12 4 8 17

16

27

13 10

7 12 9 1

5 0 3 4 8 17

16

27

67

exchange A[1] A[i] Max-Heapify(A,1)

13 10

7 12 9 1

5 0 3 4 16 17

8

27

13 10

7 12 9 1

5 0 3 4 16 17

8

27

8 10

7 12 9 1

5 0 3 4 16 17

13

27

12 10

7 8 9 1

5 0 3 4 16 17

13

27

12 10

7 4 9 1

5 0 3 8 16 17

13

27

etc...

Notice last 3 elements in array are the largest in A, and they are also sorted in increasing order.

68

Example of Heap-Extract-Max(A)

17 10

16 13 9 1

5 7 12 4 8 3

27

0Heap-Extract-Max(A)

1. if heap-size[A] < 1

2. then error “heap underflow”

3. max A[1]

4. A[1] A[heap-size[A] ]

5. heap-size[A] heap-size[A] - 1

6. Max-Heapify(A,1)

7. return max

17 10

16 13 9 1

5 7 12 4 8 3

0

0

0 10

16 13 9 1

5 7 12 4 8 3

17

0

16 10

0 13 9 1

5 7 12 4 8 3

17

0

16 10

7 13 9 1

5 0 12 4 8 3

17

0

Example Heap Sort

Let us look at this example: we must convert the unordered array with n = 10 elements into a max-heap

None of the leaf nodes need tobe percolated down, and the firstnon-leaf node is in position n/2

Thus we start with position 10/2 = 5

Example Heap Sort

We compare 3 with its child and swap them

Example Heap Sort

We compare 17 with its two children and swap it with the maximum child (70)

Example Heap Sort

We compare 28 with its two children, 63 and 34, and swap it with the largest child

Example Heap Sort

We compare 52 with its children, swap it with the largest– Recursing, no further swaps are needed

Example Heap Sort

Finally, we swap the root with its largest child, and recurse, swapping 46 again with 81, and then again with 70

Heap Sort Example

We have now converted the unsorted array

into a max-heap:

Heap Sort Example

Suppose we pop the maximum element of this heap

This leaves a gap at the back of the array:

Heap Sort Example

This is the last entry in the array, so why not fill it with the largest element?

Repeat this process: pop the maximum element, and then insert it at the end of the array:

Heap Sort Example

Repeat this process– Pop and append 70

– Pop and append 63

Heap Sort Example

We have the 4 largest elements in order– Pop and append 52

– Pop and append 46

Heap Sort Example

Continuing...– Pop and append 34

– Pop and append 28

Finally, we can pop 17, insert it into the 2nd location, and the resulting array is sorted

Heap Sort Example

82

Heapsort

• For a binary heap, building the heap takes O(N) time.

• In a binary heap, deleteMin gives the smallest element. – Running time of O(logN).

• Basic strategy for heap sort given N elements– Build a binary heap from N elements - O(N)– Perform N deleteMin operations O(N*logN)

• Put the result of deleteMins in a new array. – The new array is sorted.

83

Heapsort

• The problem is that: – This strategy uses a new array – This means waste of space for large N.

• A way for using only one array. – When minimum item is deleted, put it to the end of the

array. – After N deleteMins:

• The original array will have the elements in sorted order from max to min.

– If you want items in increasing order (from min to max), then use a Max-Heap.

84

Example

• Assume the input that is to be sorted: – 31, 41, 59, 26, 53, 58, 97– N = 7.

• We need first to build a binary heap (a Max-Heap)– A max-Heap is heap that has the largest item in

root.

85

Example97

53

26 41 58 31

5997 53 59 26 41 58 31

10 2 3 4 5 6 7 8 9 10

59

53

26 41 31

5859 53 58 26 41 31 97

10 2 3 4 5 6 7 8 9 10

97

deleteMax

Max-Heap after BuildHeap()

Heap after first DeleteMin()

86

Example

• After 5 more deleteMins, the array will look like the following.

26 31 41 53 58 59 97

10 2 3 4 5 6 7 8 9 10

This is a sorted array

87

template <class Comparable> void heapsort( vector<Comparable> & a ) {/* 1*/ for( int i = a.size( ) / 2; i >= 0; i-- ) /* buildHeap *//* 2*/ percDown( a, i, a.size( ) );

/* 3*/ for( int j = a.size( ) - 1; j > 0; j-- ) {/* 4*/ swap( a[ 0 ], a[ j ] ); /* deleteMax *//* 5*/ percDown( a, 0, j ); } }

j is running from N -1 to 1

i is running from N/2 to 0

Swap first and last elements

88

template <class Comparable> void percDown( vector<Comparable> & a, int i, int n ) { int child; Comparable tmp;

/* 1*/ for( tmp = a[ i ]; leftChild( i ) < n; i = child ) {/* 2*/ child = leftChild( i );/* 3*/ if( child != n - 1 && a[ child ] < a[ child + 1 ] )/* 4*/ child++;/* 5*/ if( tmp < a[ child ] )/* 6*/ a[ i ] = a[ child ]; else/* 7*/ break; }/* 8*/ a[ i ] = tmp; }

Logical size of heapPoint from where to start

percolate down

89

97 53 59 26 41 58 31

10 2 3 4 5 6 7 8 9

a.size = 7

swap a[0] with a[6]

31 53 59 26 41 58 97

10 2 3 4 5 6 7 8 9

a.size = 7

percolatedown(a, 0, 6)

j=6

heapsort()

90

31 53 59 26 41 58 97

10 2 3 4 5 6 7 8 9

i=0

59 53 26 41 58 97

10 2 3 4 5 6 7 8 9

tmp = 31, n=6

59 53 26 41 58 97

10 2 3 4 5 6 7 8 9

i=2

59 53 58 26 41 31 97

10 2 3 4 5 6 7 8 9

tmp = 31, n=6

percDown()

Go outof for loop!

91

59 53 58 26 41 31 97

10 2 3 4 5 6 7 8 9

j=5

31 53 58 26 41 59 97

10 2 3 4 5 6 7 8 9

j=5

Swap(a[0], a[5])

percolateDown(a, 0, 5)

heapsort()

92

31 53 58 26 41 59 97

10 2 3 4 5 6 7 8 9

i=0

58 53 31 26 41 59 97

10 2 3 4 5 6 7 8 9

percDown()

Applications of Priority Queues

• The Selection Problem: find k-th largest number in a list of N elements

• Straightforward solution: not efficient, O(N2)– Algorithm A: sort N numbers in decreasing order,

then return the kth element – cost O(N2)– Algorithm B: read and sort k numbers in decreasing

order, then read the rest elements one after another and compare to the kth element, if larger then replace it the correct place of the remaining k-1 elements. Finally return the kth element – cost O(kN), because k*k (read first k and sort) + k*(N-k) (the remaining N-k is larger and larger). Here k could be N/2, hence total O(N2) in worst case.

93

• Solution with priority queue:– Algorithm A: (assume find kth smallest) apply

buildHeap algorithm to the array (O(N)), then perform k deleteMin operations (k.O(log N)). The last element extracted from the heap is the answer. Total cost: O(N+k log N). Here k could be N/2, hence total O(NlogN) in worst case.

– Algorithm B: the first k elements are placed into the heap in total time O(k) with a call to buildHeap. The time to process each remaining element is O(1), to test if the element goes into S, plus O(log k), to delete Sk and insert the element if this is necessary. Thus, the total time is O(k+(n-k)log k) = O(N log k). It is also O(N log N) in worst case

94

Source code 1

Source code 2

Source code 3

Source code 4

Source code 5

Source code 6

Source code 7