8. Binary Search Trees · 2020. 5. 17. · Binary search tree A binarysearchtree is a binary tree...
65
8. Binary Search Trees [Ottman/Widmayer, Kap. 5.1, Cormen et al, Kap. 12.1 - 12.3] 127
Transcript of 8. Binary Search Trees · 2020. 5. 17. · Binary search tree A binarysearchtree is a binary tree...
8. Binary Search Trees
[Ottman/Widmayer, Kap. 5.1, Cormen et al, Kap. 12.1 - 12.3]
127
Trees
Trees areGeneralized lists: nodes can have more than one successorSpecial graphs: graphs consist of nodes and edges. A tree is a fullyconnected, directed, acyclic graph.
128
Trees
UseDecision trees: hierarchic representation ofdecision rulessyntax trees: parsing and traversing ofexpressions, e.g. in a compilerCode tress: representation of a code, e.g. morsealphabet, hu�man codeSearch trees: allow e�cient searching for anelement by value
129
Examples
start
E
I
S
H V
U
F U
A
R
L A
W
P I
T
N
D
B X
K
C Y
M
G
Z Q
O
Ö CH
longshort
Morsealphabet
130
Examples
3/5 + 7.0
+
/
3 5
7.0
Expression tree
131
Nomenclature
Wurzel
W
I E
K
parent
child
inner node
leaves
Order of the tree: maximum number of child nodes, here: 3Height of the tree: maximum path length root – leaf (here: 4)
132
Binary Trees
A binary tree iseither a leaf, i.e. an empty tree,or an inner leaf with two trees Tl (left subtree) and Tr (right subtree) asleft and right successor.
In each inner node v we storea key v.key andtwo nodes v.left and v.right to the roots of the left and right subtree.
a leaf is represented by the null-pointer
key
left right
133
Linked List Node in Python
1 5 6 nullListNode
key next
class ListNode:# entries key, next implicit via constructor
def __init__(self, key , next = None):"""Constructor that takes a key and, optionally, next."""self.key = keyself.next = next
}
134
Now: tree nodes in Python
class SearchNode:# implicit entries key, left, right
def __init__(self, k, l=None, r=None):# Constructor that takes a key k,# and optionally a left and right node.self.key = kself.left, self.right = l, r
5
3 8
2
None None
None None None
SearchNodekey
left right
135
Now: tree nodes in Python
class SearchNode:# implicit entries key, left, right
def __init__(self, k, l=None, r=None):# Constructor that takes a key k,# and optionally a left and right node.self.key = kself.left, self.right = l, r
5
3 8
2
None None
None None None
SearchNodekey
left right 135
Binary search treeA binary search tree is a binary tree that fulfils the search tree property:
Every node v stores a keyKeys in left subtree v.left are smaller than v.keyKeys in right subtree v.right are greater than v.key
16
7
5
2
10
9 15
18
17 30
99
136
Searching
Input: Binary search tree with root r, key kOutput: Node v with v.key = k or nullv ← rwhile v 6= null do
if k = v.key thenreturn v
else if k < v.key thenv ← v.left
elsev ← v.right
return null
8
4 13
10
9
19
Search (12)
→ null
137
Searching
Input: Binary search tree with root r, key kOutput: Node v with v.key = k or nullv ← rwhile v 6= null do
if k = v.key thenreturn v
else if k < v.key thenv ← v.left
elsev ← v.right
return null
8
4 13
10
9
19
Search (12)
→ null
137
Searching
Input: Binary search tree with root r, key kOutput: Node v with v.key = k or nullv ← rwhile v 6= null do
if k = v.key thenreturn v
else if k < v.key thenv ← v.left
elsev ← v.right
return null
8
4 13
10
9
19
Search (12)
→ null
137
Searching
Input: Binary search tree with root r, key kOutput: Node v with v.key = k or nullv ← rwhile v 6= null do
if k = v.key thenreturn v
else if k < v.key thenv ← v.left
elsev ← v.right
return null
8
4 13
10
9
19
Search (12)→ null
137
Searching in Python
def findNode(root, key):n = rootwhile n != None and n.key != key:
if key < n.key:n = n.left
else:n = n.right
return n
138
Height of a tree
The height h(T ) of a binary tree T with root r is given by
h(r) =
0 if r = null1 + max{h(r.left), h(r.right)} otherwise.
The worst case run time of the search is thus O(h(T ))
139
Insertion of a key
Insertion of the key k
Search for k
If successful search: e.g. outputerrorOf no success: insert the key at theleaf reached
8
4
5
13
10
9
19
Insert (5)
140
Insert Nodes in Python
def addNode(root, key):n = rootif n == None:
root = Node(key)while n.key != key:
if key < n.key:if n.left == None:
n.left = Node(key)n = n.left
else:if n.right == None:
n.right = Node(key)n = n.right
return root141
Tree in Python
class Tree:def __init__(self):
self.root = None
def find(self,key):return findNode(self.root, key)
def has(self,key):return self.find(key) != None
def add(self,key):self.root = addNode(self.root, key)
# ....142
Remove node
Three cases possible:Node has no childrenNode has one childNode has two children
[Leaves do not count here]
8
3
5
4
13
10
9
19
143
Remove node
Node has no childrenSimple case: replace node by leaf.
8
3
5
4
13
10
9
19
remove(4)−→
8
3
5
13
10
9
19
144
Remove node
Node has one childAlso simple: replace node by single child.
8
3
5
4
13
10
9
19
remove(3)−→
8
5
4
13
10
9
19
145
Remove node
Node v has two children
The following observation helps: the smallestkey in the right subtree v.right (the symmet-ric successor of v)
is smaller than all keys in v.rightis greater than all keys in v.leftand cannot have a left child.
Solution: replace v by its symmetric succes-sor.
8
3
5
4
13
10
9
19
146
By symmetry...
Node v has two children
Also possible: replace v by its symmetric pre-decessor.
Implementation: devil is in the detail!
8
3
5
4
13
10
9
19
147
Algorithm SymmetricSuccessor(v)
Input: Node v of a binary search tree.Output: Symmetric successor of vw ← v.rightx← w.leftwhile x 6= null do
w ← xx← x.left
return w
148
Traversal possibilities
preorder: v, then Tleft(v), then Tright(v).
8, 3, 5, 4, 13, 10, 9, 19postorder: Tleft(v), then Tright(v), then v.
4, 5, 3, 9, 10, 19, 13, 8
inorder: Tleft(v), then v, then Tright(v).
3, 4, 5, 8, 9, 10, 13, 19
8
3
5
4
13
10
9
19
149
Traversal possibilities
preorder: v, then Tleft(v), then Tright(v).8, 3, 5, 4, 13, 10, 9, 19
postorder: Tleft(v), then Tright(v), then v.
4, 5, 3, 9, 10, 19, 13, 8
inorder: Tleft(v), then v, then Tright(v).
3, 4, 5, 8, 9, 10, 13, 19
8
3
5
4
13
10
9
19
149
Traversal possibilities
preorder: v, then Tleft(v), then Tright(v).8, 3, 5, 4, 13, 10, 9, 19postorder: Tleft(v), then Tright(v), then v.
4, 5, 3, 9, 10, 19, 13, 8inorder: Tleft(v), then v, then Tright(v).
3, 4, 5, 8, 9, 10, 13, 19
8
3
5
4
13
10
9
19
149
Traversal possibilities
preorder: v, then Tleft(v), then Tright(v).8, 3, 5, 4, 13, 10, 9, 19postorder: Tleft(v), then Tright(v), then v.4, 5, 3, 9, 10, 19, 13, 8
inorder: Tleft(v), then v, then Tright(v).
3, 4, 5, 8, 9, 10, 13, 19
8
3
5
4
13
10
9
19
149
Traversal possibilities
preorder: v, then Tleft(v), then Tright(v).8, 3, 5, 4, 13, 10, 9, 19postorder: Tleft(v), then Tright(v), then v.4, 5, 3, 9, 10, 19, 13, 8inorder: Tleft(v), then v, then Tright(v).
3, 4, 5, 8, 9, 10, 13, 19
8
3
5
4
13
10
9
19
149
Traversal possibilities
preorder: v, then Tleft(v), then Tright(v).8, 3, 5, 4, 13, 10, 9, 19postorder: Tleft(v), then Tright(v), then v.4, 5, 3, 9, 10, 19, 13, 8inorder: Tleft(v), then v, then Tright(v).3, 4, 5, 8, 9, 10, 13, 19
8
3
5
4
13
10
9
19
149
Degenerated search trees
9
5
4 8
13
10 19
Insert 9,5,13,4,8,10,19ideally balanced
4
5
8
9
10
13
19
Insert 4,5,8,9,10,13,19linear list
19
13
10
9
8
5
4
Insert 19,13,10,9,8,5,4linear list
150
Probabilistically
A search tree constructed from a random sequence of numbers providesan an expected path length of O(log n).Attention: this only holds for insertions. If the tree is constructed byrandom insertions and deletions, the expected path length is O(
√n).
Balanced trees make sure (e.g. with rotations) during insertion or deletionthat the tree stays balanced and provide a O(log n) Worst-case guarantee.
151
9. Heaps
Datenstruktur optimiert zum schnellen Extrahieren von Minimum oderMaximum und Sortieren. [Ottman/Widmayer, Kap. 2.3, Cormen et al, Kap. 6]
152
[Max-]Heap*
Binary tree with the following proper-ties
1. complete up to the lowest level2. Gaps (if any) of the tree in the
last level to the right3. Heap-Condition:
Max-(Min-)Heap: key of a childsmaller (greater) that that of theparent node
root
22
20
16
3 2
12
8 11
18
15
14
17
parent
child
leaves
*Heap(data structure), not: as in “heap and stack” (memory allocation)
153
[Max-]Heap*
Binary tree with the following proper-ties
1. complete up to the lowest level
2. Gaps (if any) of the tree in thelast level to the right
3. Heap-Condition:Max-(Min-)Heap: key of a childsmaller (greater) that that of theparent node
root
22
20
16
3 2
12
8 11
18
15
14
17
parent
child
leaves
*Heap(data structure), not: as in “heap and stack” (memory allocation)
153
[Max-]Heap*
Binary tree with the following proper-ties
1. complete up to the lowest level2. Gaps (if any) of the tree in the
last level to the right
3. Heap-Condition:Max-(Min-)Heap: key of a childsmaller (greater) that that of theparent node
root
22
20
16
3 2
12
8 11
18
15
14
17
parent
child
leaves
*Heap(data structure), not: as in “heap and stack” (memory allocation)
153
[Max-]Heap*
Binary tree with the following proper-ties
1. complete up to the lowest level2. Gaps (if any) of the tree in the
last level to the right3. Heap-Condition:
Max-(Min-)Heap: key of a childsmaller (greater) that that of theparent node
root
22
20
16
3 2
12
8 11
18
15
14
17
parent
child
leaves
*Heap(data structure), not: as in “heap and stack” (memory allocation)
153
Heap as Array
Tree→ Array:children(i) = {2i, 2i + 1}parent(i) = bi/2c
221
202
183
164
125
156
177
38
29
810
1111
1412
parent
Children
22
20
16
3 2
12
8 11
18
15
14
17
[1]
[2] [3]
[4] [5] [6] [7]
[8] [9] [10] [11] [12]
Depends on the starting index4
4For array that start at 0: {2i, 2i + 1} → {2i + 1, 2i + 2}, bi/2c → b(i− 1)/2c154
Height of a Heap
What is the height H(n) of Heap with n nodes? On the i-th level of abinary tree there are at most 2i nodes. Up to the last level of a heap alllevels are filled with values.
H(n) = min{h ∈ N :h−1∑i=0
2i ≥ n}
with ∑h−1i=0 2i = 2h − 1:
H(n) = min{h ∈ N : 2h ≥ n + 1},
thusH(n) = dlog2(n + 1)e.
155
Insert
Insert new element at the first freeposition. Potentially violates the heapproperty.Reestablish heap property: climbsuccessivelyWorst case number of operations: O(log n)
22
20
16
3 2
12
8 11
18
15
14
17
156
Insert
Insert new element at the first freeposition. Potentially violates the heapproperty.
Reestablish heap property: climbsuccessivelyWorst case number of operations: O(log n)
22
20
16
3 2
12
8 11
18
15
14 21
17
156
Insert
Insert new element at the first freeposition. Potentially violates the heapproperty.Reestablish heap property: climbsuccessively
Worst case number of operations: O(log n)
22
20
16
3 2
12
8 11
18
21
14 15
17
156
Insert
Insert new element at the first freeposition. Potentially violates the heapproperty.Reestablish heap property: climbsuccessively
Worst case number of operations: O(log n)
22
20
16
3 2
12
8 11
21
18
14 15
17
156
Insert
Insert new element at the first freeposition. Potentially violates the heapproperty.Reestablish heap property: climbsuccessivelyWorst case number of operations: O(log n)
22
20
16
3 2
12
8 11
21
18
14 15
17
156
Algorithm Sift-Up(A, m)
Input: Array A with at least m elements and Max-Heap-Structure onA[1, . . . , m− 1]
Output: Array A with Max-Heap-Structure on A[1, . . . , m].v ← A[m] // valuec← m // current position (child)p← bc/2c // parent nodewhile c > 1 and v > A[p] do
A[c]← A[p] // Value parent node → current nodec← p // parent node → current nodep← bc/2c
A[c]← v // value → root of the (sub)tree
157
Remove the maximum
Replace the maximum by the lower rightelementReestablish heap property: sinksuccessively (in the direction of thegreater child)Worst case number of operations: O(log n)
21
20
16
3 2
12
8 11
18
15
14
17
158
Remove the maximum
Replace the maximum by the lower rightelement
Reestablish heap property: sinksuccessively (in the direction of thegreater child)Worst case number of operations: O(log n)
14
20
16
3 2
12
8 11
18
15 17
158
Remove the maximum
Replace the maximum by the lower rightelementReestablish heap property: sinksuccessively (in the direction of thegreater child)
Worst case number of operations: O(log n)
20
14
16
3 2
12
8 11
18
15 17
158
Remove the maximum
Replace the maximum by the lower rightelementReestablish heap property: sinksuccessively (in the direction of thegreater child)
Worst case number of operations: O(log n)
20
16
14
3 2
12
8 11
18
15 17
158
Remove the maximum
Replace the maximum by the lower rightelementReestablish heap property: sinksuccessively (in the direction of thegreater child)Worst case number of operations: O(log n)
20
16
14
3 2
12
8 11
18
15 17
158
Why this is correct: Recursive heap structure
A heap consists of two heaps:
22
20
16
3 2
12
8 11
18
15
14
17
159
Why this is correct: Recursive heap structure
A heap consists of two heaps:
22
20
16
3 2
12
8 11
18
15
14
17
159
Algorithm SiftDown(A, i, m)
Input: Array A with heap structure for the children of i. Last element m.Output: Array A with heap structure for i with last element m.while 2i ≤ m do
j ← 2i; // j left childif j < m and A[j] < A[j + 1] then
j ← j + 1; // j right child with greater key
if A[i] < A[j] thenswap(A[i], A[j])i← j; // keep sinking down
elsei← m; // sift down finished
160
Sort heap
A[1, ..., n] is a Heap.While n > 1
swap(A[1], A[n])SiftDown(A, 1, n− 1);n← n− 1
7 6 4 5 1 2
swap ⇒ 2 6 4 5 1 7
siftDown ⇒ 6 5 4 2 1 7
swap ⇒ 1 5 4 2 6 7
siftDown ⇒ 5 4 2 1 6 7
swap ⇒ 1 4 2 5 6 7
siftDown ⇒ 4 1 2 5 6 7
swap ⇒ 2 1 4 5 6 7
siftDown ⇒ 2 1 4 5 6 7
swap ⇒ 1 2 4 5 6 7
161
Sort heap
A[1, ..., n] is a Heap.While n > 1
swap(A[1], A[n])SiftDown(A, 1, n− 1);n← n− 1
7 6 4 5 1 2
swap ⇒ 2 6 4 5 1 7
siftDown ⇒ 6 5 4 2 1 7
swap ⇒ 1 5 4 2 6 7
siftDown ⇒ 5 4 2 1 6 7
swap ⇒ 1 4 2 5 6 7
siftDown ⇒ 4 1 2 5 6 7
swap ⇒ 2 1 4 5 6 7
siftDown ⇒ 2 1 4 5 6 7
swap ⇒ 1 2 4 5 6 7
161
Sort heap
A[1, ..., n] is a Heap.While n > 1
swap(A[1], A[n])SiftDown(A, 1, n− 1);n← n− 1
7 6 4 5 1 2
swap ⇒ 2 6 4 5 1 7
siftDown ⇒ 6 5 4 2 1 7
swap ⇒ 1 5 4 2 6 7
siftDown ⇒ 5 4 2 1 6 7
swap ⇒ 1 4 2 5 6 7
siftDown ⇒ 4 1 2 5 6 7
swap ⇒ 2 1 4 5 6 7
siftDown ⇒ 2 1 4 5 6 7
swap ⇒ 1 2 4 5 6 7
161
Sort heap
A[1, ..., n] is a Heap.While n > 1
swap(A[1], A[n])SiftDown(A, 1, n− 1);n← n− 1
7 6 4 5 1 2
swap ⇒ 2 6 4 5 1 7
siftDown ⇒ 6 5 4 2 1 7
swap ⇒ 1 5 4 2 6 7
siftDown ⇒ 5 4 2 1 6 7
swap ⇒ 1 4 2 5 6 7
siftDown ⇒ 4 1 2 5 6 7
swap ⇒ 2 1 4 5 6 7
siftDown ⇒ 2 1 4 5 6 7
swap ⇒ 1 2 4 5 6 7
161
Sort heap
A[1, ..., n] is a Heap.While n > 1
swap(A[1], A[n])SiftDown(A, 1, n− 1);n← n− 1
7 6 4 5 1 2
swap ⇒ 2 6 4 5 1 7
siftDown ⇒ 6 5 4 2 1 7
swap ⇒ 1 5 4 2 6 7
siftDown ⇒ 5 4 2 1 6 7
swap ⇒ 1 4 2 5 6 7
siftDown ⇒ 4 1 2 5 6 7
swap ⇒ 2 1 4 5 6 7
siftDown ⇒ 2 1 4 5 6 7
swap ⇒ 1 2 4 5 6 7
161
Heap creation
Observation: Every leaf of a heap is trivially a correct heap.
Consequence:
Induction from below!
162
Heap creation
Observation: Every leaf of a heap is trivially a correct heap.
Consequence: Induction from below!
162
Algorithm HeapSort(A, n)
Input: Array A with length n.Output: A sorted.// Build the heap.for i← n/2 downto 1 do
SiftDown(A, i, n);
// Now A is a heap.for i← n downto 2 do
swap(A[1], A[i])SiftDown(A, 1, i− 1)
// Now A is sorted.
163
Analysis: sorting a heap
SiftDown traverses at most log n nodes. For each node 2 key comparisons.⇒ sorting a heap costs in the worst case 2 log n comparisons.Number of memory movements of sorting a heap also O(n log n).
164
Analysis: creating a heapCalls to siftDown: n/2.Thus number of comparisons and movements: v(n) ∈ O(n log n).But mean length of the sift-down paths is much smaller:We use that h(n) = dlog2 n + 1e = blog2 nc+ 1 für n > 0
v(n) =blog2 nc∑
l=02l︸︷︷︸
number heaps on level l
·( blog2 nc+ 1− l︸ ︷︷ ︸height heaps on level l
−1) =blog2 nc∑
k=02blog2 nc−k · k
= 2blog2 nc ·blog2 nc∑
k=0
k
2k≤ n ·
∞∑k=0
k
2k≤ n · 2 ∈ O(n)
with s(x) :=∑∞
k=0 kxk = x(1−x)2 (0 < x < 1) and s(1
2) = 2
165
