9. Heap :Priority Queue

Where We Are?

Stack Binary Tree

Heap

Array Linked list

Binary Search Tree

TreeQueue

Priority Queue

Queue

Queue operation is based on the order of arrivals of elements

FIFO(First-In First-Out) or FCFS(First-Come First-Serve)

Priority Queue

Queue operation is based on specific priority values,

regardless of the order of arrivals.

For instance, priority includes grade, urgency, importance, etc.

Applications: Job scheduling (OS), Emergency treatment in Hospital, etc.

Queue

frontrearInsert Delete

Priority Queue (Cont.)

Each element of Queue has its own priority value (called Key)

Priority is an integer in general

Max Priority Queue

The element of the highest priority

(i.e., key) is deleted first.

Min Priority Queue

The element of the lowest priority (i.e., key) is deleted first

Main issue of Priority Queue comes under

how to search for the maximum or minimum value

Priority Queue

Priority Queue

frontrearInsert

rear

Priority=

3

Priority=

6

Priority=

5

Priority=

1

Delete

Max Priority Queue: Implementation

How to implement Max Priority Queue?

Unordered Array

Ordered Array

Unordered Linked List

Ordered Linked List

Binary Search Tree

Max Heap (Min Heap)

Use of Unordered Array

Keys are not Sorted

Delete operation is very inefficient; O(n)

Search from the first to the last!

Sequential search is used: O(n)

Insert operation is very efficient; O(1)

Insert a new Key into the last of array

No (data) movement takes place

list 6 1 93 4 8 7

i=0

[0] [1] [2] [3] [4] [5] [6]

By sequential search

2

[7]

i=1 i=2 i=3 Insert directly

As to Priorities, Keys are sorted in ascending order

Delete is very efficient; O(1)

The highest key is always positioned

at the last of array

Insert is very inefficient; O(n)

Insert a new key into its appropriate position

Binary search is used; O(log2 n)

After insertion, data should be moved; O(n)

Use of Ordered Array

list 1 3 64 7 8 9

[0] [1] [2] [3] [4] [5] [6]

Delete directly

2

[7]

Insert by binary search

list(mid)=6

left rightlist(mid)=2

right

Unordered Linked List

Deletion is very inefficient; O(n)

Sequential search is used

Insertion is very efficient; O(1)

Always, insert a new key at the first position of linked list

Ordered Linked List

Deletion is very efficient; O(1)

Insertion is very inefficient; O(n)

Not possible to use Binary search

Use of Linked Lists

Use of BST

Keys are Stored in Binary Search Tree

Delete operation is inefficient;

To find the highest key, it visits continuously

the right child from the root

Time complexity is the height of BST

which depends on the shape of BST

Worst: O(n), Average: O(log2 n)

Insert operation is also inefficient;

New key is always inserted into the leaf node

Also, time complexity is the height of BST which depends on the shape of BST

Worst: O(n), Average: O(log2 n)

Delete by Search in BST

Insert by Search in BST

10

7

4 9

86 11

1313

h : h

eight

Performance Comparison

Data Structures Insertion Deletion

Unsorted Array O(1) O(n)

Unsorted Linked list O(1) O(n)

Sorted Array O(n) O(1)

Sorted Linked list O(n) O(1)

Binary Search Tree O(n) O(n)

Max (Min) Heap O(log2n) O(log2n)

Q) When n = 1,000, what will become of time complexity of each method?

Worst Case Complexity

Why Heap?

Heap is efficient for searching the largest (or smallest) value.

It compromises on sorted and unsorted (data) structures

Sorted structures (array or linked-list)

Very efficient for finding largest (smallest) element (i.e., Deletion)

Inefficient in insertion

Unsorted structures (array or linked-list)

Inefficient for finding largest (smallest) element (i.e., Deletion)

Very efficient in insertion

Thus, Heap is efficient for both insertion and deletion

What Max/Min Heap?

Max Heap

Heap is a Complete Binary Tree

Key of each node is no smaller than its children’s keys.

cf) Min Heap

Key of each node is no larger than its children’s keys.

K

left subtree

All≤K

right subtree

All≤K

Full Binary Tree

Complete Binary Tree

Complete Binary Tree

(a) height = 1 (b) height = 2 (c) height = 3

A

B C

A

B

D E

C

F G

(a) (b) (c)A

B

A

B

D E

C

F

A

B

D E

C

F G

H I

A

Examples : Max Heap

Root of a max heap always has the largest value

14

12 7

10 8 6

[1]

[2][3]

[4] [5] [6]

30

25

[1]

[2]

(a)

(c)9

6 3

5

[1]

[2][3]

[4]

(b)

Examples : Not a Max Heap

14

12 5

10 8 6

[1]

[2][3]

[4] [5] [6]

(a)

(c)9

6 3

5

[1]

[2][3]

[5]

(b)

30

25 40

20 27 35

[1]

[2][3]

[5] [6][4]

Problematic node!

Examples : Min Heap

Root of a min heap always has the smallest value

2

7 4

10 8 6

[1]

[2][3]

[4] [5] [6]

11

21

[1]

[2]

(a)

(c)10

20 83

50

[1]

[2][3]

[4]

(b)

Heap vs. BST

Comparison: Max heap vs. BST

Max Heap Binary Search Tree

Complete Binary Tree Binary Tree

K

left subtree

All≤K

right subtree

All≤K

K

left subtree

All<K

right subtree

All>K

Heap Implementation: Use of Array

Why Use Array for Implementing Heap?

Not waste (memory) space (since complete binary tree)

Easy to find the positions of parent and its children

Not need to use complex linked lists

Finding a node location/position

Left-Child(i) = 2i

Right-Child(i) = 2i + 1

Parent(i) = i/ 2 (if i = 1, i is the root)

14 12 7 10 8 6 -

1 2 3 4 5 6 7

14

12 7

10 8 6

[1]

[2][3]

[4] [5] [6]

Max Heap

Max Heap Operation

Insertion

New key is inserted into Max heap

Deletion

After finding the largest key, it is deleted from Max heap

Note: Two conditions of Max heap must be satisfied.

1) Complete Binary Tree

2) Key value of each node should be no smaller than values of its children

Insertion

Inserting into Max heap

(1) Insert a new key into Max heap

To preserve 1st condition, the new node should be

added at the first empty position at the last leaf level

2nd condition could be broken

(2) Repair the structure so that 2nd condition is satisfied

Reheap-Up Not a heap!

Heap!

Insertion

Procedures

Find an empty location; This is a node that is right next of the last leaf node.

If it violates any condition of (max or min) heap, repair it (Reheap).

1

23

4 5 6 7

8 9 10

a

b c

d e f g

h i j

Last leaf node

Insertion

A very simple case;

Just insert it into the right next of the last leaf node.

If It does not violate any heap condition, its insertion is done!

1

23

4 5 6 7

8 9 10

9

8 7

6 7 2 6

5 1 4

Last leaf node

Insert (4)

Insertion

1. At first, insert 20 into the next of the last leaf node

2. Compare 20 with its parent, i.e., 7. Since 20>7, swap 20 and 7, i.e., move 20 up!

3. Compare 20 with its parent, i.e., 8. Since 20>8, swap 20 and 8, i.e., move 20 up!

4. Do this process until reaching the root or its parent is greater than 20.

1

23

4 5 6 7

8 9 10

9

8

7

7

6 2 6

5 1 20

Last leaf node

Insert (20)

Insertion

Insertion Result!

Insert (20)1

23

4 5 6 7

8 9 10

20

9

8

7

6 2 6

5 1 7

Deleting from Max heap

(1) Deletion occurs at the root (because the largest key is always at the root)

2nd condition could be broken

To preserve 1st condition, the last node should move up to the root

(2) Repair the structure so that 2nd condition is satisfied

Reheap-Down

Deletion

Not a heap!

Heap!

Deletion

Delete the root since it always contains the largest key

Put the last leaf node (i.e., 8) into the root

Compare 8 with its child with larger value, i.e., 15. Since 8<15, swap 8 and 15, i.e., move 8 down!

Do this process until reaching the leaf node or its child is smaller than 8.

1

23

4 5 6 7

8 9 10

20

15

9

Last leaf node

Delete

7

6 2 6

5 1 7 811

Deletion

Delete

Deletion Result!

1

23

4 5 6 7

8 9 10

9

8

7

6 2 6

5 1 7

15

Building a Heap

Building a Max Heap from Array

8 19 23 32 45

Heap To be Inserted

8

Array Max Heap

19 8 23 32 45

19

8

23 8 19 32 45

23

8 19

32 23 19 8 45

32

23

8

19

45 32 19 8 23

45

32

8

19

23

Complexity=?

Time Complexity: Insertion/Deletion

Insertion/deletion time is bounded by the Height of a Heap.

What is a Height of Heap?

Height of any complete binary tree is ceil(log2(n+1))since the tree is “Balanced”

Note: A “Balanced” tree is a binary tree whose height difference between left andright subtrees is at most ‘1’ for all nodes.

Thus, insertion & deletion time becomes O(log2n)

h=ceil(log10)=

4h=

ceil(log7)=3

ceil(log3)=2

How to find kth largest element from an unsorted array?

Sort and Select element at location k

O(n2) for sorting + O(1) for selection

Build Heap and delete k times

O(n log2n) for building heap + O(k log2n) for deletions

Finding kth Largest Element

6 1 93 4 8 7

[0] [1] [2] [3] [4] [5] [6]

9 8 67 4 3 1

[0] [1] [2] [3] [4] [5] [6]

4th largest

Sort

ing!

42

35 30

15 20 21 18

3 7 14

21

20 18

15 14 7 3

Heap

42 35 1530 20 21 18 3 7 14

Heap

21 20 1518 14 7 3

4th largest

By deletions

Is it Possible to Sort Elements (in ascending order) using Max Heap?

Yes! because a deletion takes out the highest key from the Heap

1. At first, a deletion is performed and the delete key is stored(Generally, it is put into the end of array)

2. Second, the Heap is repaired to satisfying two (Max) Heap conditions

3. Iterate the two procedures until the Heap is empty.

Sorting Elements using Heap

42

35 30

15 20 21 18

3 7 14

Heap

42 35 1530 20 21 18 3 7 14

Deletion

35

20 30

15 14 21 18

3 7

Heap

35 20 1530 14 21 18 3 7 42

1st largest 2nd largest

Sorting Elements using Heap

30

20 21

15 14 7 18

3

Heap

30 20 1521 14 7 18 3 35 42

21

20 18

15 14 7 3

Heap

21 20 1518 14 7 3 30 35 42

3rd largest 4th largest

20

15 18

3 14 7

Heap

20 15 318 14 7 21 30 35 42

5th largest

Sorted

3 7 1817 14 20 21 30 35 42

Deletion Deletion

Deletion

It is called “Heap Sort”Complexity=?