2IL50 Data Structures Spring 2015 Lecture 3: Heaps.

2IL50 Data Structures

Spring 2015

Lecture 3: Heaps

Solving recurrences

one more time …

Solving recurrences

Easiest: Master theoremcaveat: not always applicable

Alternatively: Guess the solution and use the substitution method to prove that your guess it is correct.

How to guess:

1. expand the recursion

2. draw a recursion tree

Example(A)

► A is an array of length n

1. n = A.length

2. if n==1

3. then return A[1]

4. else begin

5. Copy A[1… n/2 ] to auxiliary array B[1... n/2 ]

6. Copy A[1… n/2 ] to auxiliary array C[1… n/2 ]

7. b = Example(B); c = Example(C)

8. for i = 1 to n

9. do for j = 1 to i

10. do A[i] = A[j]

11. return 43

12. end

Example

Example(A)

► A is an array of length n

1. n = A.length

2. if n==1

3. then return A[1]

4. else begin

5. Copy A[1… n/2 ] to auxiliary array B[1... n/2 ]

6. Copy A[1… n/2 ] to auxiliary array C[1… n/2 ]

7. b = Example(B); c = Example(C)

8. for i = 1 to n

9. do for j = 1 to i

10. do A[i] = A[j]

11. return 43

12. end

Example

Let T(n) be the worst case running time of Example on an array of length n.

Lines 1,2,3,4,11, and 12 take Θ(1) time.Lines 5 and 6 take Θ(n) time.Line 7 takes Θ(1) + 2 T( n/2 ) time.Lines 8 until 10 take

time.

If n=1 lines 1,2,3 are executed,else lines 1,2, and 4 until 12 are executed.

➨ T(n):

➨ use master theorem …

)Θ(nΘ(i)Θ(1) 2n

1i

n

1i

i

1j

Θ(1) if n=1

2T(n/2) + Θ(n2) if n>1

Let a and b be constants, let f(n) be a function, and let T(n)be defined on the nonnegative integers by the recurrence

T(n) = aT(n/b) + f(n)

Then we have:

1. If f(n) = O(nlog a – ε) for some constant ε > 0, then T(n) = Θ(nlog a).

2. If f(n) = Θ(nlog a), then T(n) = Θ(nlog a log n)

3. If f(n) = Ω(nlog a + ε) for some constant ε > 0, and if af(n/b) ≤ cf(n) for some constant c < 1 and all sufficiently large n, then T(n) = Θ(f(n))

The master theorem

b

b

b

b

b

Quiz

Recurrence

1. T(n) = 4 T(n/2) + Θ(n3)

2. T(n) = 4 T(n/2) + Θ(n)

3. T(n) = T(n/2) + 1

4. T(n) = T(n/3) + T(2n/3) + n

5. T(n) = 9 T(n/3) + Θ(n2)

6. T(n) = √n T(√n) + n

Master theorem?

yes

yes

yes

no

yes

no

T(n) = Θ(n3)

T(n) = Θ(n2)

T(n) = Θ(log n)

T(n) = Θ(n log n)

T(n) = Θ(n2 log n)

T(n) = Θ(n log log n)

Tips

Analysis of recursive algorithms:find the recursion and solve with master theorem if possible

Analysis of loops: summations

Some standard recurrences and sums:

T(n) = 2T(n/2) + Θ(n) ➨

½ n(n+1) = Θ(n2)

Θ(n3)

T(n) = Θ(n log n)

n

1i

i

n

1i

2i

Event-driven simulation

Stores a set of events, processes first event (highest priority)

Supporting data structure: insert event find (and extract) event with highest priority change the priority of an event

Priority queue

Max-priority queueabstract data type (ADT) that stores a set S of elements, each with an associated key (integer value).

Operations

Insert(S, x): inserts element x into S, that is, S ← S ⋃ {x}

Maximum(S): returns the element of S with the largest key

Extract-Max(S): removes and returns the element of S with the largest key

Increase-Key(S, x, k): give key[x] the value k

condition: k is larger than the current value of key[x]

Min-priority queue …

Implementing a priority queue

Today

Insert Maximum Extract-Max Increase-Key

sorted list

sorted array

Θ(1) Θ(1)

Θ(1)

Θ(n)

Θ(n)

Θ(n)

Θ(n)Θ(n)

Insert Maximum Extract-Max Increase-Key

heap Θ(1)Θ(log n) Θ(log n)Θ(log n)

Max-heap

Heapnearly complete binary tree, filled on all levels except possibly the lowest.(lowest level is filled from left to right)

Max-heap property: for every node i other than the root

key[Parent(i)] ≥ key[i]

35

1930

1112830

517 2

35

1930

21830

35

1930

12830

Properties of a max-heap

LemmaThe largest element in a max-heap is stored at the root.

Proof:

Properties of a max-heap

LemmaThe largest element in a max-heap is stored at the root.

Proof: x root

y arbitrary node

z1, z2, …, zk nodes on path between x and y

max-heap property ➨ key[x] ≥ key[z1] ≥ … ≥ key[zk] ≥ key[y]

➨ the largest element is stored at x ■

x

y

z1

z2

Implementing a heap with an array

A.length = length of array A

heap-size[A] =number of elements in the heap

35

1930

1112830

217 5

35 30 19 30 8 12 11 17 2 5

1 2 3heap-size[A]

4

array A[1 … A.length]

Implementing a heap with an array

kth node on level j is stored at position

left child of node at position i = Left(i) =

right child of node at position i = Right(i) =

parent of node at position i = Parent(i) =

1 2 3heap-size[A]

4

array A[1 … A.length]

level 0

level 1

level 2, position 3

A[2j+k-1]

2i

2i + 1

i / 2

Priority queue

Max-priority queueabstract data type (ADT) that stores a set S of elements, each with an associated key (integer value).

Operations

Insert(S, x): inserts element x into S, that is, S ← S ⋃ {x}

Maximum(S): returns the element of S with the largest key

Extract-Max(S): removes and returns the element of S with the largest key

Increase-Key(S, x, k): give key[x] the value k

condition: k is larger than the current value of key[x]

Implementing a max-priority queue

Set S is stored as a heap in an array A.

Operations: Maximum, Extract-Max, Insert, Increase-Key.

35 30 19 30 8 12 11 17 2 5

1 2 3heap-size[A]

4




Maximum(A)

1. if heap-size[A] < 12. then error3. else return A[1]

running time: O(1)

35 30 19 30 8 12 11 17 2 5

1 2 3heap-size[A]

4




35 30 19 30 8 12 11 17 2 5

1 2 3heap-size[A]

4

17

3530




1930

1112830

217 5



Heap-Extract-Max(A)

1. if heap-size[A] < 1 2. then error3. max = A[1]4. A[1] = A [heap-size[A]]5. heap-size[A] = heap-size[A]–16. Max-Heapify(A,1)7. return max

355

30


1930

11128

217 5

restore heap property

Max-Heapify

Max-Heapify(A, i)

► ensures that the heap whose root is stored at position i has the max-heap property

► assumes that the binary trees rooted at Left(i) and Right(i) are max-heaps

10

1035

40

Max-Heapify

Max-Heapify(A, i)

► ensures that the heap whose root is stored at position i has the max-heap property


Max-Heapify(A,1)

exchange A[1] with largest child

Max-Heapify(A,2)

exchange A[2] with largest child

Max-Heapify(A,5)

A[5] larger than its children ➨ done.

30

10

1940

111235

217 5

Max-Heapify

Max-Heapify(A, i)

► ensures that the heap whose root is stored at position i has themax-heap property


1. if Left(i) ≤ heap-size[A] and A[Left(i)] > A[i]2. then largest = Left(i)3. else largest = i4. if Right(i) ≤ heap-size[A] and A[Right(i)] > A[largest]5. then largest = Right(i)6. if largest ≠ i7. then exchange A[i ] and A[largest]8. Max-Heapify(A, largest)

running time? O(height of the subtree rooted at i) = O(log n)




Insert (A, key)

1. heap-size[A] = heap-size[A] +12. A[heap-size[A]] = - infinity3. Increase-Key(A, heap-size[A], key)




Increase-Key(A, i, key)

1. if key < A[i]2. then error3. A[i] = key4. while i>1 and A[parent(i)] < A[i]5. do exchange A[parent(i)] and A[i]6. i = parent(i)

running time: O(log n)

30

35

1930

11128

217 5

42

Building a heap

Build-Max-Heap(A)

► Input: array A[1..n] of numbers► Output: array A[1..n] with the same numbers, but rearranged, such

that the max-heap property holds

1. heap-size = A.length2. for i = A.length downto 13. do Max-Heapify(A,i)

P(k): nodes k, k+1, …, n are each the root of a max-heap

Loop InvariantP(k+1) holds before line 3 is executed with i=k, P(k) holds afterwards

starting at A.length / 2 is sufficient

Building a heap

Build-Max-Heap(A)

1. heap-size = A.length2. for i = A.length downto 13. do Max-Heapify(A,i)

∑i O(1 + height of i)

= ∑0 ≤ h ≤ log n (# nodes of height h) ∙ O(1+h)

≤ ∑0 ≤ h ≤ log n (n / 2h+1) ∙ O(1+h)

= O(n) ∙ ∑0 ≤ h ≤ log n h / 2h+1

= O(n)

h=0

h=1

h=2

O(height of node i)

height of node i # edges longest simple downward path from i to a leaf.

The sorting problem

Input: a sequence of n numbers ‹a1, a2, …, an›

Output: a permutation of the input such that ‹ai1 ≤ … ≤ ain›

Important properties of sorting algorithms:

running time: how fast is the algorithm in the worst case

in place: only a constant number of input elements are ever stored outside the input array

Sorting with a heap: Heapsort

HeapSort(A)

1. Build-Max-Heap(A)2. for i = A.length downto 23. do exchange A[1] and A[i ]4. heap-size[A] = heap-size[A] – 15. Max-Heapify(A,1)

P(k): A[k+1 .. n] is sorted and contains the n-k largest elements, A[1..k] is a max-heap on the remaining elements

Loop InvariantP(k) holds before lines 3-5 are executed with i=k, P(k-1) holds afterwards

Sorting algorithms

worst case running time in place

InsertionSort

MergeSort

HeapSort

Θ(n2) yes

Θ(n log n) noyesΘ(n log n)

Announcements

Wednesday 3+4

Reading and writing mathematical proofs

2IL50 Data Structures Spring 2015 Lecture 3: Heaps.

Documents

Transcript of 2IL50 Data Structures Spring 2015 Lecture 3: Heaps.