Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9.
-
Upload
avice-morrison -
Category
Documents
-
view
221 -
download
2
Transcript of Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9.
Analysis of AlgorithmsCS 477/677
Instructor: Monica Nicolescu
Lecture 9
CS 477/677 - Lecture 9
A Job Scheduling Application
• Job scheduling
– The key is the priority of the jobs in the queue
– The job with the highest priority needs to be executed
next
• Operations
– Insert, remove maximum
• Data structures
– Priority queues
– Ordered array/list, unordered array/list
CS 477/677 - Lecture 9
PQ Implementations & Cost
Worst-case asymptotic costs for a PQ with N items
Insert Remove max
ordered array
ordered list
unordered array
unordered list
N
N
N
N
1
1
1
1
Can we implement both operations efficiently?
CS 477/677 - Lecture 9
Background on Trees
• Def: Binary tree = structure composed of a finite set of nodes that either:– Contains no nodes, or– Is composed of three disjoint sets of nodes: a root
node, a left subtree and a right subtree
2
14 8
1
16
4
3
9 10
root
Right subtreeLeft subtree
CS 477/677 - Lecture 9
Special Types of Trees
• Def: Full binary tree = a binary tree in which each node is either a leaf or has degree (number of children) exactly 2.
• Def: Complete binary tree = a binary tree in which all leaves have the same depth and all internal nodes have degree 2.
Full binary tree
2
14 8
1
16
7
4
3
9 10
12
Complete binary tree
2
1
16
4
3
9 10
CS 477/677 - Lecture 9
The Heap Data Structure
• Def: A heap is a nearly complete binary tree with the following two properties:– Structural property: all levels are full, except
possibly the last one, which is filled from left to right– Order (heap) property: for any node x
Parent(x) ≥ x
Heap
5
7
8
4It doesn’t matter that 4 in
level 1 is smaller than 5 in
level 22
CS 477/677 - Lecture 9
Definitions
• Height of a node = the number of edges on a longest simple path from the node down to a leaf
• Depth of a node = the length of a path from the root to the node
• Height of tree = height of root node
= lgn, for a heap of n elements
2
14 8
1
16
4
3
9 10
Height of root = 3
Height of (2)= 1 Depth of (10)= 2
CS 477/677 - Lecture 9
Array Representation of Heaps
• A heap can be stored as an
array A.– Root of tree is A[1]
– Left child of A[i] = A[2i]
– Right child of A[i] = A[2i + 1]
– Parent of A[i] = A[ i/2 ]– Heapsize[A] ≤ length[A]
• The elements in the subarray
A[(n/2+1) .. n] are leaves
• The root is the maximum
element of the heap
CS 477/677 - Lecture 9
Heap Types
• Max-heaps (largest element at root), have the
max-heap property: – for all nodes i, excluding the root:
A[PARENT(i)] ≥ A[i]
• Min-heaps (smallest element at root), have the
min-heap property:– for all nodes i, excluding the root:
A[PARENT(i)] ≤ A[i]
CS 477/677 - Lecture 9
Operations on Heaps
• Maintain the max-heap property– MAX-HEAPIFY
• Create a max-heap from an unordered array– BUILD-MAX-HEAP
• Sort an array in place– HEAPSORT
• Priority queue operations
CS 477/677 - Lecture 9
Operations on Priority Queues
• Max-priority queues support the following
operations:
– INSERT(S, x): inserts element x into set S
– EXTRACT-MAX(S): removes and returns element of
S with largest key
– MAXIMUM(S): returns element of S with largest key
– INCREASE-KEY(S, x, k): increases value of element
x’s key to k (assume k ≥ current key value at x)
CS 477/677 - Lecture 9
Building a Heap
Alg: BUILD-MAX-HEAP(A)
1. n = length[A]
2. for i ← n/2 downto 1
3. do MAX-HEAPIFY(A, i, n)
• Convert an array A[1 … n] into a max-heap (n = length[A])
• The elements in the subarray A[(n/2+1) .. n] are
leaves
• Apply MAX-HEAPIFY on elements between 1 and n/2
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
4 1 3 2 16 9 10 14 8 7A:
CS 477/677 - Lecture 9
Example: A 4 1 3 2 16 9 10 14 8 7
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
14
2 8
1
16
7
4
10
9 3
1
2 3
4 5 6 7
8 9 10
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 1014
2 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
14
2 8
16
7
1
4
10
9 3
1
2 3
4 5 6 7
8 9 108
2 4
14
7
1
16
10
9 3
1
2 3
4 5 6 7
8 9 10
i = 5 i = 4 i = 3
i = 2 i = 1
CS 477/677 - Lecture 9
Correctness of BUILD-MAX-HEAP• Loop invariant:
– At the start of each iteration of the for loop, each node i + 1, i + 2,…, n is the root of a max-heap
• Initialization:– i = n/2: Nodes n/2+1, n/2+2, …, n are leaves
they are the root of trivial max-heaps
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
CS 477/677 - Lecture 9
Correctness of BUILD-MAX-HEAP• Maintenance:
– MAX-HEAPIFY makes node i a max-heap root and preserves the property that nodes i + 1, i + 2, …, n are roots of max-heaps
– Decrementing i in the for loop reestablishes the loop invariant
• Termination:– i = 0 each node 1, 2, …, n is the root of
a max-heap (by the loop invariant)
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
CS 477/677 - Lecture 9
Running Time of BUILD MAX HEAP
It would seem that running time is O(nlgn)
• This is not an asymptotically tight upper bound
Alg: BUILD-MAX-HEAP(A)
1. n = length[A]
2. for i ← n/2 downto 1
3. do MAX-HEAPIFY(A, i, n) O(lgn)O(n)
CS 477/677 - Lecture 9
Running Time of BUILD MAX HEAP
• HEAPIFY takes O(h) the cost of HEAPIFY on a node i is proportional to the height of the node i in the tree
Height Level
h0 = 3 (lgn)
h1 = 2
h2 = 1
h3 = 0
i = 0
i = 1
i = 2
i = 3 (lgn)
No. of nodes
20
21
22
23
hi = h – i height of the heap rooted at level ini = 2i number of nodes at level i
i
h
iihnnT
0
)( ihh
i
i 0
2 )(nO
CS 477/677 - Lecture 9
Running Time of BUILD MAX HEAP
i
h
iihnnT
0
)( Cost of HEAPIFY at level i number of nodes at that level
ihh
i
i 0
2 Replace the values of ni and hi computed before
hh
iih
ih2
20
Multiply by 2h both at the nominator and denominator and
write 2i as i2
1
h
kk
h k
0 22 Change variables: k = h - i
0 2k
k
kn The sum above is smaller than the sum of all elements to
and h = lgn
)(nO The sum above is smaller than 2
Running time of BUILD-MAX-HEAP: T(n) = O(n)
CS 477/677 - Lecture 9
Binary Search Trees• Tree representation:
– A linked data structure in which each node is an object
• Node representation:– Key field– Satellite data– Left: pointer to left child– Right: pointer to right child– p: pointer to parent (p [root [T]] =
NIL)
• Satisfies the binary search tree property
Left child Right child
L Rparent
key data
CS 477/677 - Lecture 9
Binary Search Tree Example
• Binary search tree property:
– If y is in left subtree of x,
then key [y] ≤ key [x]
– If y is in right subtree of x,
then key [y] ≥ key [x]
2
3
5
5
7
9
CS 477/677 - Lecture 9
Binary Search Trees
• Support many dynamic set operations
– SEARCH, MINIMUM, MAXIMUM, PREDECESSOR,
SUCCESSOR, INSERT, DELETE
• Running time of basic operations on binary
search trees
– On average: (lgn)• The expected height of the tree is lgn
– In the worst case: (n)• The tree is a linear chain of n nodes
CS 477/677 - Lecture 9
Red-Black Trees
• “Balanced” binary trees guarantee an O(lgn) running time on the basic dynamic-set operations
• Red-black tree– Binary tree with an additional attribute for its nodes:
color which can be red or black– Constrains the way nodes can be colored on any path
from the root to a leaf• Ensures that no path is more than twice as long as another
the tree is balanced
– The nodes inherit all the other attributes from the binary-search trees: key, left, right, p
CS 477/677 - Lecture 9
Red-Black Trees Properties
1. Every node is either red or black
2. The root is black
3. Every leaf (NIL) is black
4. If a node is red, then both its children are black• No two red nodes in a row on a simple path from the
root to a leaf
5. For each node, all paths from the node to
descendant leaves contain the same number of
black nodes
CS 477/677 - Lecture 9
Example: RED-BLACK TREE
• For convenience we use a sentinel NIL[T] to represent all the NIL nodes at the leafs– NIL[T] has the same fields as an ordinary node– Color[NIL[T]] = BLACK– The other fields may be set to arbitrary values
26
17 41
30 47
38 50
NIL NIL
NIL
NIL NIL NIL NIL
NIL
CS 477/677 - Lecture 9
Black-Height of a Node
• Height of a node: the number of edges in a longest path to a leaf
• Black-height of a node x: bh(x) is the number of black nodes (including NIL) on a path from x to leaf, not counting x
26
17 41
30 47
38 50
NIL NIL
NIL
NIL NIL NIL NIL
NIL
h = 4bh = 2
h = 3bh = 2
h = 2bh = 1
h = 1bh = 1
h = 1bh = 1
h = 2bh = 1 h = 1
bh = 1
CS 477/677 - Lecture 9
Properties of Red-Black Trees• Claim
– Any node with height h has black-height ≥ h/2• Proof
– By property 4, there are at most h/2 red nodes on the path from the node to a leaf
– Hence at least h/2 are black26
17 41
30 47
38 50
Property 4: if a node is red then both its children are black
CS 477/677 - Lecture 9
Properties of Red-Black Trees
Claim
The subtree rooted at any node x contains at least
2bh(x) - 1 internal nodes
Proof: By induction on height of x
Basis: height[x] = 0
x is a leaf (NIL[T])
bh(x) = 0
# of internal nodes: 20 - 1 = 0
NIL
x
CS 477/677 - Lecture 9
Properties of Red-Black TreesInductive step: • Let height(x) = h and bh(x) = b• Any child y of x has:
– bh (y) =– bh (y) =
26
17 41
30 47
38 50
b (if the child is red), or
b - 1 (if the child is black)
CS 477/677 - Lecture 9
Properties of Red-Black Trees• Want to prove:
– The subtree rooted at any node x contains at
least 2bh(x) - 1 internal nodes
• Assume true for children of x: – Their subtrees contain at least 2bh(x) - 1 – 1
internal nodes
• The subtree rooted at x contains at least:
(2bh(x) - 1 – 1) + (2bh(x) - 1 – 1) + 1 =
2 · (2bh(x) - 1 - 1) + 1 =
2bh(x) - 1 internal nodes
x
l r
CS 477/677 - Lecture 9
Properties of Red-Black Trees
Lemma: A red-black tree with n internal nodes has height at most 2lg(n + 1).
Proof:
n
• Add 1 to all sides and then take logs:
n + 1 ≥ 2b ≥ 2h/2
lg(n + 1) ≥ h/2 h ≤ 2 lg(n + 1)
root
l r
height(root) = hbh(root) = b
number n of internal nodes
≥ 2b - 1 ≥ 2h/2 - 1
since b h/2
CS 477/677 - Lecture 9
Operations on Red-Black Trees
• The non-modifying binary-search tree operations
MINIMUM, MAXIMUM, SUCCESSOR,
PREDECESSOR, and SEARCH run in O(h) time
– They take O(lgn) time on red-black trees
• What about TREE-INSERT and TREE-DELETE?
– They will still run in O(lgn)
– We have to guarantee that the modified tree will still be
a red-black tree
CS 477/677 - Lecture 9
INSERT
INSERT: what color to make the new node?• Red? Let’s insert 35!
– Property 4: if a node is red, then both its children are black
• Black? Let’s insert 14!– Property 5: all paths from a node to its leaves contain
the same number of black nodes
26
17 41
30 47
38 50
CS 477/677 - Lecture 9
DELETE
DELETE: what color was the
node that was removed? Red?
1. Every node is either red or black
2. The root is black
3. Every leaf (NIL) is black
4. If a node is red, then both its children are black
5. For each node, all paths from the node to descendant
leaves contain the same number of black nodes
OK!
OK!
OK!
OK! Does not createtwo red nodes in a row
OK! Does not changeany black heights
26
17 41
30 47
38 50
CS 477/677 - Lecture 9
DELETE
DELETE: what color was the
node that was removed? Black?
1. Every node is either red or black
2. The root is black
3. Every leaf (NIL) is black
4. If a node is red, then both its children are black
5. For each node, all paths from the node to descendant
leaves contain the same number of black nodes
OK!
OK!
Not OK! Could createtwo red nodes in a row
Not OK! Could change theblack heights of some nodes
26
17 41
30 47
38 50
Not OK! If removing the root and the child that replaces it is red
CS 477/677 - Lecture 9
Rotations
• Operations for restructuring the tree after insert
and delete operations on red-black trees
• Rotations take a red-black tree and a node within
the tree and:– Together with some node re-coloring they help restore
the red-black tree property
– Change some of the pointer structure
– Do not change the binary-search tree property
• Two types of rotations:– Left & right rotations
CS 477/677 - Lecture 9
Left Rotations• Assumption for a left rotation on a node x:
– The right child of x (y) is not NIL
• Idea:– Pivots around the link from x to y– Makes y the new root of the subtree– x becomes y’s left child– y’s left child becomes x’s right child
CS 477/677 - Lecture 9
Readings
• Chapter 8, Chapter 6