CSC401 – Analysis of Algorithms CSC401 – Analysis of Algorithms Lecture Notes 7Lecture Notes 7

Multi-way Search Trees and Skip ListsMulti-way Search Trees and Skip Lists

Objectives:Objectives:

Introduce multi-way search trees especially Introduce multi-way search trees especially (2,4) trees, and analyze the performance of (2,4) trees, and analyze the performance of operations on multi-way search treesoperations on multi-way search treesIntroduce Splay trees and present the Introduce Splay trees and present the amortized analysis of Splayingamortized analysis of SplayingIntroduce Skip Lists and update operations and Introduce Skip Lists and update operations and present the probabilistic analysis of skip listspresent the probabilistic analysis of skip lists

22

Multi-Way Search TreeMulti-Way Search TreeA multi-way search tree is an ordered tree such that A multi-way search tree is an ordered tree such that – Each internal node has at least two children and stores Each internal node has at least two children and stores dd1 1

key-element items key-element items ((kkii, , ooii)), where , where d d is the number of children is the number of children – For a node with children For a node with children vv1 1 vv22 … … vvdd storing keys storing keys kk1 1 kk22 … … kkdd11

keys in the subtree of keys in the subtree of vv1 1 are less than are less than kk11

keys in the subtree of keys in the subtree of vvii are between are between kkii1 1 and and kkii ((ii = 2, …, = 2, …, dd1)1)

keys in the subtree of keys in the subtree of vvdd are greater than are greater than kkdd11

– The leaves store no items and serve as placeholdersThe leaves store no items and serve as placeholders11 24

2 6 8 15

30

27 32

33

Multi-Way Inorder TraversalMulti-Way Inorder TraversalWe can extend the notion of inorder traversal from We can extend the notion of inorder traversal from binary trees to multi-way search treesbinary trees to multi-way search trees

Namely, we visit item Namely, we visit item ((kkii, , ooii)) of node of node vv between the between the recursive traversals of the subtrees of recursive traversals of the subtrees of vv rooted at rooted at children children vvii and and vvii11

An inorder traversal of a multi-way search tree visits the An inorder traversal of a multi-way search tree visits the keys in increasing orderkeys in increasing order

11 24

2 6 8 15

30

27 32

1 3 5 7 9 11 13 19

15 17

2 4 6 14 18

8 12

10

16

44

Multi-Way SearchingMulti-Way SearchingSimilar to search in a binary search treeSimilar to search in a binary search tree

A each internal node with children A each internal node with children vv1 1 vv22 … … vvdd and keys and keys kk1 1 kk22 … … kkdd11

– kk kkii ( (ii = 1, …, = 1, …, dd1)1): the search terminates successfully: the search terminates successfully– kk kk11: we continue the search in child : we continue the search in child vv11

– kkii1 1 kk kkii ( (ii = 2, …, = 2, …, dd1)1): we continue the search in child : we continue the search in child vvii

– kk kkdd11: we continue the search in child : we continue the search in child vvdd

Reaching an external node terminates the search Reaching an external node terminates the search unsuccessfullyunsuccessfullyExample: search for 30Example: search for 30 11 24

2 6 8 15

30

27 32

55

(2,4) Tree(2,4) TreeA (2,4) tree (also called 2-4 tree or 2-3-4 tree) is a multi-A (2,4) tree (also called 2-4 tree or 2-3-4 tree) is a multi-way search with the following propertiesway search with the following properties– Node-Size PropertyNode-Size Property: every internal node has at most : every internal node has at most

four childrenfour children– Depth PropertyDepth Property: all the external nodes have the same : all the external nodes have the same

depthdepth

Depending on the number of children, an internal node of Depending on the number of children, an internal node of a (2,4) tree is called a 2-node, 3-node or 4-nodea (2,4) tree is called a 2-node, 3-node or 4-node

10 15 24

2 8 12 27 3218

66

Height of a (2,4) TreeHeight of a (2,4) TreeTheorem:Theorem: A (2,4) tree storing A (2,4) tree storing nn items has height items has height OO(log (log nn))Proof:Proof:– Let Let hh be the height of a (2,4) tree with be the height of a (2,4) tree with n n itemsitems– Since there are at least Since there are at least 22ii items at depth items at depth ii 0, … , 0, … , h h 1 1 and and

no items at depth no items at depth hh, we have, we have nn 1 1 2 2 4 4 … … 2 2hh1 1 22h h 11

– Thus, Thus, hh log (log (n n 1)1)

Searching in a (2,4) tree with Searching in a (2,4) tree with nn items takes items takes OO(log (log nn)) timetime

1

2

2h1

0

items

0

1

h1

h

depth

77

InsertionInsertionWe insert a new item We insert a new item ((kk, , oo)) at the parent at the parent vv of the of the leaf reached by searching for leaf reached by searching for kk– We preserve the depth property but We preserve the depth property but – We may cause an We may cause an overflowoverflow (i.e., node (i.e., node vv may become a may become a

5-node)5-node)

Example: inserting key 30 causes an overflowExample: inserting key 30 causes an overflow

27 32 35

10 15 24

2 8 12 18

v

10 15 24

2 8 12 27 30 32 3518v

88

Overflow and SplitOverflow and SplitWe handle an We handle an overflowoverflow at a 5-node at a 5-node vv with a with a split operationsplit operation::– let let vv11 … … vv55 be the children of be the children of vv and and kk11 … … kk44 be the keys of be the keys of vv

– node node vv is replaced nodes is replaced nodes vv' ' and and vv""vv'' is a 3-node with keys is a 3-node with keys kk11 kk22 and children and children vv11 vv22 vv33

vv"" is a 2-node with key is a 2-node with key kk44 and children and children vv44 vv55

– key key kk3 3 is inserted into the parent is inserted into the parent uu of of v v (a new root may be created)(a new root may be created)

The overflow may propagate to the parent node The overflow may propagate to the parent node uu

15 24

12 27 30 32 3518v

u

v1 v2 v3 v4 v5

15 24 32

12 27 3018v'

u

v1 v2 v3 v4 v5

35v"

99

Analysis of InsertionAnalysis of Insertion

AlgorithmAlgorithm insertIteminsertItem((kk, , oo))

1.1. We search for key We search for key kk to locate to locate the insertion node the insertion node vv

2.2. We add the new item (We add the new item (kk, , oo) at ) at node node vv

3. 3. whilewhile overflowoverflow((vv))

if if isRootisRoot((vv))

create a new empty root create a new empty root above above vv

v v split split((vv))

Let Let TT be a (2,4) tree be a (2,4) tree with with nn items items– Tree Tree TT has has O O(log (log nn) ) heightheight – Step 1 takes Step 1 takes OO(log (log nn)) time time

because we visit because we visit OO(log (log nn)) nodesnodes

– Step 2 takes Step 2 takes OO(1)(1) time time– Step 3 takes Step 3 takes OO(log (log nn)) time time

because each split takes because each split takes OO(1)(1) time and we perform time and we perform OO(log (log nn) ) splitssplits

Thus, an insertion in a Thus, an insertion in a (2,4) tree takes (2,4) tree takes OO(log (log nn)) timetime

1010

DeletionDeletionWe reduce deletion of an item to the case where the item We reduce deletion of an item to the case where the item is at the node with leaf childrenis at the node with leaf children

Otherwise, we replace the item with its inorder successor Otherwise, we replace the item with its inorder successor (or, equivalently, with its inorder predecessor) and delete (or, equivalently, with its inorder predecessor) and delete the latter itemthe latter item

Example: to delete key 24, we replace it with 27 (inorder Example: to delete key 24, we replace it with 27 (inorder successor)successor)

27 32 35

10 15 24

2 8 12 18

32 35

10 15 27

2 8 12 18

1111

Underflow and FusionUnderflow and FusionDeleting an item from a node Deleting an item from a node vv may cause an may cause an underflowunderflow, where node , where node vv becomes a 1-node with one becomes a 1-node with one child and no keyschild and no keysTo handle an underflow at node To handle an underflow at node v v with parent with parent uu, we , we consider two casesconsider two casesCase 1:Case 1: the adjacent siblings of the adjacent siblings of vv are 2-nodes are 2-nodes– Fusion operation:Fusion operation: we merge we merge vv with an adjacent sibling with an adjacent sibling ww and and

move an item from move an item from uu to the merged node to the merged node vv''– After a fusion, the underflow may propagate to the parent After a fusion, the underflow may propagate to the parent uu

9 14

2 5 7 10

u

v

9

10 14

u

v'w2 5 7

1212

Underflow and TransferUnderflow and TransferTo handle an underflow at node To handle an underflow at node v v with parent with parent uu, we , we consider two casesconsider two casesCase 2:Case 2: an adjacent sibling an adjacent sibling ww of of vv is a 3-node or a 4- is a 3-node or a 4-nodenode– Transfer operation:Transfer operation:

1. we move a child of 1. we move a child of ww to to vv 2. we move an item from 2. we move an item from uu to to vv3. we move an item from 3. we move an item from ww to to uu

– After a transfer, no underflow occursAfter a transfer, no underflow occurs

4 9

6 82

u

vw

4 8

62 9

u

vw

1313

Analysis of DeletionAnalysis of DeletionLet Let TT be a (2,4) tree with be a (2,4) tree with nn items items– Tree Tree TT has has O O(log (log nn) ) heightheight

In a deletion operationIn a deletion operation– We visit We visit OO(log (log nn)) nodes to locate the node from nodes to locate the node from

which to delete the itemwhich to delete the item– We handle an underflow with a series of We handle an underflow with a series of OO(log (log

nn)) fusions, followed by at most one transfer fusions, followed by at most one transfer– Each fusion and transfer takes Each fusion and transfer takes OO(1)(1) time time

Thus, deleting an item from a (2,4) tree Thus, deleting an item from a (2,4) tree takes takes OO(log (log nn)) time time

1414

From (2,4) to Red-Black TreesFrom (2,4) to Red-Black TreesA red-black tree is a representation of a (2,4) tree by A red-black tree is a representation of a (2,4) tree by means of a binary tree whose nodes are colored means of a binary tree whose nodes are colored redred or or blackblack

In comparison with its associated (2,4) tree, a red-In comparison with its associated (2,4) tree, a red-black tree hasblack tree has– same logarithmic time performancesame logarithmic time performance– simpler implementation with a single node typesimpler implementation with a single node type

2 6 73 54

4 6

2 7

5

3

3

5OR

1515

Splay Tree DefinitionSplay Tree Definitiona a splay treesplay tree is a binary search tree where is a binary search tree where a node is splayed after it is accessed (for a a node is splayed after it is accessed (for a search or update)search or update)– deepest internal node accessed is splayeddeepest internal node accessed is splayed– splaying costs O(h), where h is height of the splaying costs O(h), where h is height of the

tree – which is still O(n) worst-casetree – which is still O(n) worst-caseO(h) rotations, each of which is O(1)O(h) rotations, each of which is O(1)

Binary Search Tree Rules:Binary Search Tree Rules:– items stored only at internal nodesitems stored only at internal nodes– keys stored at nodes in the left subtree of keys stored at nodes in the left subtree of vv are less are less

than or equal to the key stored at than or equal to the key stored at vv– keys stored at nodes in the right subtree of keys stored at nodes in the right subtree of vv are greater are greater

than or equal to the key stored at than or equal to the key stored at vv

An inorder traversal will return the keys in orderAn inorder traversal will return the keys in orderSearch proceeds down the tree to the found item Search proceeds down the tree to the found item or an external node.or an external node.

1616

Splay Trees do Rotations after Splay Trees do Rotations after Every Operation (Even Search)Every Operation (Even Search)

new operation: new operation: splaysplay– splaying moves a node to the root using rotationssplaying moves a node to the root using rotations

right rotation makes the left child x of a node y into

y’s parent; y becomes the right child of x

y

x

T1 T2

T3

y

x

T1

T2T3

left rotation makes the right child y of a node x

into x’s parent; x becomes the left child of y

y

x

T1 T2

T3

y

x

T1

T2T3

(structure of tree above y is not modified)

(structure of tree above x is not modified)

a right rotation about y a left rotation about x

1717

Splaying:Splaying:

is x the root?

stop

is x a child of the root?

right-rotate about the root

left-rotate about the root

is x the left child of the

root?

is x a left-left grandchild?

is x a left-right grandchild?

is x a right-right grandchild?

is x a right-left grandchild?

right-rotate about g, right-rotate about p

left-rotate about g, left-rotate about p

left-rotate about p, right-rotate about g

right-rotate about p, left-rotate about g

start with node x

“x is a left-left grandchild” means x is a left child of its parent, which is itself a left child of its parent

p is x’s parent; g is p’s parent

no

yes

yes

yes

yes

yes

yes

no

no

yes zig-zig

zig-zag

zig-zag

zig-zig

zigzig

1818

Visualizing the Splaying CasesVisualizing the Splaying Caseszig-zag

y

x

T2 T3

T4

z

T1

y

x

T2 T3 T4

z

T1

y

x

T1 T2

T3

z

T4

zig-zig

y

z

T4T3

T2

x

T1

zig

x

w

T1 T2

T3

y

T4

y

x

T2 T3 T4

w

T1

1919

Splaying ExampleSplaying Examplelet let xx = (8,N) = (8,N)– xx is the right child of its is the right child of its

parent, which is the left parent, which is the left child of the grandparentchild of the grandparent

– left-rotate around left-rotate around pp, then , then right-rotate around right-rotate around gg

(20,Z)

(37,P)(21,O)(14,J)

(7,T)

(35,R)(10,A)

(1,C)

(1,Q)

(5,G)(2,R)

(5,H)

(6,Y)(5,I)

(8,N)

(7,P)

(36,L)

(10,U)

(40,X)

x

g

p

(10,A)

(20,Z)

(37,P)(21,O)

(35,R)

(36,L) (40,X)(7,T)

(1,C)

(1,Q)

(5,G)(2,R)

(5,H)

(6,Y)(5,I)

(14,J)(8,N)

(7,P)

(10,U)

x

g

p

(10,A)

(20,Z)

(37,P)(21,O)

(35,R)

(36,L) (40,X)

(7,T)

(1,C)

(1,Q)

(5,G)(2,R)

(5,H)

(6,Y)(5,I)

(14,J)

(8,N)

(7,P)

(10,U)

x

g

p

1.(before rotating)

2.(after first rotation)

3. (after second rotation)

x is not yet the root, so we splay again

2020

Splaying Example, ContinuedSplaying Example, Continuednownow x x is the left child of the root is the left child of the root– right-rotate around rootright-rotate around root

(10,A)

(20,Z)

(37,P)(21,O)

(35,R)

(36,L) (40,X)

(7,T)

(1,C)

(1,Q)

(5,G)(2,R)

(5,H)

(6,Y)(5,I)

(14,J)

(8,N)

(7,P)

(10,U)

x

(10,A)

(20,Z)

(37,P)(21,O)

(35,R)

(36,L) (40,X)

(7,T)

(1,C)

(1,Q)

(5,G)(2,R)

(5,H)

(6,Y)(5,I)

(14,J)

(8,N)

(7,P)

(10,U)

x

1.(before applying rotation)

2.(after rotation)

x is the root, so stop

2121

Example ResultExample Resulttree might not be more balancedtree might not be more balancede.g. splay (40,X)e.g. splay (40,X)– before, the depth of the before, the depth of the

shallowest leaf is 3 and the shallowest leaf is 3 and the deepest is 7deepest is 7

– after, the depth of shallowest after, the depth of shallowest leaf is 1 and deepest is 8leaf is 1 and deepest is 8

(20,Z)

(37,P)(21,O)(14,J)

(7,T)

(35,R)(10,A)

(1,C)

(1,Q)

(5,G)(2,R)

(5,H)

(6,Y)(5,I)

(8,N)

(7,P)

(36,L)

(10,U)

(40,X)

(20,Z)

(37,P)

(21,O)

(14,J)(7,T)

(35,R)

(10,A)

(1,C)

(1,Q)

(5,G)(2,R)

(5,H)

(6,Y)(5,I)

(8,N)

(7,P) (36,L)(10,U)

(40,X)

(20,Z)

(37,P)

(21,O)

(14,J)(7,T)

(35,R)

(10,A)

(1,C)

(1,Q)

(5,G)(2,R)

(5,H)

(6,Y)(5,I)

(8,N)

(7,P)

(36,L)

(10,U)

(40,X)

before

after first splayafter second splay

2222

Splay Trees & Ordered DictionariesSplay Trees & Ordered Dictionaries

which nodes are splayed after each which nodes are splayed after each operation?operation?

use the parent of the internal node that was use the parent of the internal node that was actually removed from the tree (the parent actually removed from the tree (the parent of the node that the removed item was of the node that the removed item was swapped with)swapped with)

removeElementremoveElement

use the new node containing the item inserteduse the new node containing the item insertedinsertElementinsertElement

if key found, use that nodeif key found, use that nodeif key not found, use parent of ending external if key not found, use parent of ending external nodenode

findElementfindElement

splay nodesplay nodemethodmethod

2323

Amortized Analysis of Splay TreesAmortized Analysis of Splay TreesRunning time of each operation is proportional to time for Running time of each operation is proportional to time for splaying.splaying.

Define rank(v) as the logn(v), where n(v) is the number of Define rank(v) as the logn(v), where n(v) is the number of nodes in subtree rooted at v.nodes in subtree rooted at v.

Costs: zig = $1, zig-zig = $2, zig-zag = $2.Costs: zig = $1, zig-zig = $2, zig-zag = $2.

Thus, cost for playing a node at depth d = $d.Thus, cost for playing a node at depth d = $d.

Imagine that we store rank(v) cyber-dollars at each node v of Imagine that we store rank(v) cyber-dollars at each node v of the splay tree (just for the sake of analysis).the splay tree (just for the sake of analysis).

Cost per Zig -- Cost per Zig -- Doing a Doing a zig at x costs at most zig at x costs at most rank’(x) - rank(x):rank’(x) - rank(x):– cost = rank’(x) + cost = rank’(x) +

rank’(y) - rank(y)rank’(y) - rank(y)

-rank(x)-rank(x)

<< rank’(x) - rank(x). rank’(x) - rank(x).

zig

x

w

T1 T2

T3

y

T4

y

x

T2 T3 T4

w

T1

2424

Cost per zig-zig and zig-zagCost per zig-zig and zig-zag

Doing a zig-zig or zig-zag at x costs at most Doing a zig-zig or zig-zag at x costs at most 3(rank’(x) - rank(x)) - 2.3(rank’(x) - rank(x)) - 2.

– Proof: See Theorem 3.9, Page 192.Proof: See Theorem 3.9, Page 192.

y

x

T1 T2

T3

z

T4

zig-zig y

z

T4T3

T2

x

T1

zig-zagy

x

T2 T3

T4

z

T1

y

x

T2 T3 T4

z

T1

2525

Performance of Splay TreesPerformance of Splay TreesCost of splaying a node Cost of splaying a node x at depth d of a tree x at depth d of a tree rooted at r is at most rooted at r is at most 3(rank(r)-rank(x))-d+23(rank(r)-rank(x))-d+2– Proof: Splaying x takes Proof: Splaying x takes

d/2 splaying substeps:d/2 splaying substeps: .2))(rank)(rank(3

2)/(2))(rank)(rank(3

2)2))(rank)(rank(3(

cost cost

0

1

2/

1

2/

1

dxr

ddxr

xx i

d

ii

i

d

i

Recall: rank of a node is logarithm of its size, thus, Recall: rank of a node is logarithm of its size, thus, amortized cost of any splay operation is amortized cost of any splay operation is O(log n).O(log n).

In fact, the analysis goes through for any reasonable In fact, the analysis goes through for any reasonable definition of rank(x).definition of rank(x).

This implies that splay trees can actually adapt to This implies that splay trees can actually adapt to perform searches on frequently-requested items perform searches on frequently-requested items much faster than O(log n) in some cases. (See much faster than O(log n) in some cases. (See Theorems 3.10 and 3.11.)Theorems 3.10 and 3.11.)

2626

What is a Skip ListWhat is a Skip ListA A skip listskip list for a set for a set SS of distinct (key, element) of distinct (key, element) items is a series of lists items is a series of lists SS00, , SS1 1 , … , , … , SShh such that such that– Each list Each list SSii contains the special keys contains the special keys and and – List List SS00 contains the keys of contains the keys of S S in nondecreasing order in nondecreasing order – Each list is a subsequence of the previous one, i.e.,Each list is a subsequence of the previous one, i.e.,

SS0 0 SS1 1 … … SShh

– List List SSh h contains only the two special keyscontains only the two special keys

We show how to use a skip list to implement the We show how to use a skip list to implement the dictionary ADTdictionary ADT

56 64 78 31 34 44 12 23 26

31

64 31 34 23

S0

S1

S2

S3

2727

SearchSearchWe search for a key We search for a key xx in a a skip list as follows: in a a skip list as follows:– We start at the first position of the top list We start at the first position of the top list – At the current position At the current position pp, we compare , we compare xx with with y y

keykey((afterafter((pp))))x x y y: we return : we return elementelement((afterafter((pp))))x x y y: we “scan forward” : we “scan forward” x x y y: we “drop down”: we “drop down”

– If we try to drop down past the bottom list, we return If we try to drop down past the bottom list, we return NO_SUCH_KEYNO_SUCH_KEY

Example: search for 78Example: search for 78

S0

S1

S2

S3

31

64 31 34 23

56 64 78 31 34 44 12 23 26

2828

Randomized AlgorithmsRandomized AlgorithmsA A randomized randomized algorithmalgorithm performs performs coin tosses (i.e., uses coin tosses (i.e., uses random bits) to random bits) to control its executioncontrol its executionIt contains statements It contains statements of the typeof the type

bb randomrandom()()if if bb 0 0

do A …do A …elseelse { { bb 1} 1}

do B … do B …

Its running time Its running time depends on the depends on the outcomes of the coin outcomes of the coin tossestosses

We analyze the expected We analyze the expected running time of a running time of a randomized algorithm randomized algorithm under the following under the following assumptionsassumptions– the coins are unbiased, and the coins are unbiased, and – the coin tosses are the coin tosses are

independentindependent

The worst-case running The worst-case running time of a randomized time of a randomized algorithm is often large but algorithm is often large but has very low probability has very low probability (e.g., it occurs when all the (e.g., it occurs when all the coin tosses give “heads”)coin tosses give “heads”)We use a randomized We use a randomized algorithm to insert items algorithm to insert items into a skip listinto a skip list

2929

To insert an item To insert an item ((xx, , oo)) into a skip list, we use a randomized into a skip list, we use a randomized algorithm:algorithm:– We repeatedly toss a coin until we get tails, and we denote We repeatedly toss a coin until we get tails, and we denote

with with i i the number of times the coin came up headsthe number of times the coin came up heads– If If i i h h, we add to the skip list new lists , we add to the skip list new lists SShh11, … , , … , SSi i 11, each , each

containing only the two special keyscontaining only the two special keys– We search for We search for x x in the skip list and find the positions in the skip list and find the positions pp00, , pp1 1 , …, , …,

ppi i of the items with largest key less than of the items with largest key less than xx in each list in each list SS00, , SS11, … , … , , SSii

– For For jj 0, …, 0, …, ii, we insert item , we insert item ((xx, , oo)) into list into list SSjj after position after position ppjj

Example: insert key Example: insert key 1515, with , with ii 2 2

InsertionInsertion

S0

S1

S2

S3

10 362315

15

2315

10 36

23

23

S0

S1

S2

p0

p1

p2

3030

DeletionDeletionTo remove an item with key To remove an item with key xx from a skip list, we from a skip list, we proceed as follows:proceed as follows:– We search for We search for x x in the skip list and find the positions in the skip list and find the positions pp00, , pp1 1 , ,

…, …, ppi i of the items with key of the items with key xx, where position , where position ppjj is in list is in list SSjj

– We remove positions We remove positions pp00, , pp1 1 , …, , …, ppii from the lists from the lists SS00, , SS11, … , , … , SSii

– We remove all but one list containing only the two special We remove all but one list containing only the two special keyskeys

Example: remove key Example: remove key 3434

4512

23

23

S0

S1

S2

S0

S1

S2

S3

4512 23 34

34

23 34p0

p1

p2

3131

ImplementationImplementationWe can implement a skip We can implement a skip list with quad-nodeslist with quad-nodes

A quad-node stores:A quad-node stores:– itemitem– link to the node beforelink to the node before– link to the node afterlink to the node after– link to the node belowlink to the node below– link to the node afterlink to the node after

Also, we define special Also, we define special keys PLUS_INF and keys PLUS_INF and MINUS_INF, and we modify MINUS_INF, and we modify the key comparator to the key comparator to handle them handle them

xquad-node

3232

Space UsageSpace UsageThe space used by a skip The space used by a skip list depends on the list depends on the random bits used by random bits used by each invocation of the each invocation of the insertion algorithminsertion algorithm

We use the following two We use the following two basic probabilistic facts:basic probabilistic facts:Fact 1:Fact 1: The probability of The probability of

getting getting ii consecutive heads consecutive heads when flipping a coin is when flipping a coin is 1122ii

Fact 2:Fact 2: If each of If each of nn items is items is present in a set with present in a set with probability probability pp, the expected , the expected size of the set is size of the set is npnp

Consider a skip list with Consider a skip list with nn items items– By Fact 1, we insert an By Fact 1, we insert an

item in list item in list SSii with with probability probability 1122ii

– By Fact 2, the expected By Fact 2, the expected size of list size of list SSii is is nn22ii

The expected number The expected number of nodes used by the of nodes used by the skip list isskip list is

nnn h

ii

h

ii

22

1

2 00

Thus, the expected Thus, the expected space usage of a skip space usage of a skip list with list with nn items is items is OO((nn))

3333

HeightHeightThe running time of the The running time of the search an insertion search an insertion algorithms is affected by algorithms is affected by the height the height hh of the skip list of the skip listWe show that with high We show that with high probability, a skip list with probability, a skip list with nn items has height items has height OO(log (log nn))We use the following We use the following additional probabilistic additional probabilistic fact:fact:Fact 3:Fact 3: If each of If each of nn events has events has

probability probability pp, the probability , the probability that at least one event that at least one event occurs is at most occurs is at most npnp

Consider a skip list with Consider a skip list with nn items items– By Fact 1, we insert an By Fact 1, we insert an

item in list item in list SSii with with probability probability 1122ii

– By Fact 3, the probability By Fact 3, the probability that list that list SSii has at least one has at least one item is at most item is at most nn22ii

By picking By picking ii 3log 3log nn, we , we have that the probability have that the probability that that SS3log 3log nn has at least one has at least one item isitem isat mostat most

nn223log 3log nn nnnn33 11nn22

Thus a skip list with Thus a skip list with nn items has height at most items has height at most 3log 3log nn with probability at with probability at least least 11 1 1nn22

3434

Search and Update TimesSearch and Update TimesThe search time in a skip The search time in a skip list is proportional tolist is proportional to– the number of drop-down the number of drop-down

steps, plussteps, plus– the number of scan-the number of scan-

forward stepsforward steps

The drop-down steps are The drop-down steps are bounded by the height of bounded by the height of the skip list and thus are the skip list and thus are OO(log (log nn) ) with high with high probabilityprobabilityTo analyze the scan-To analyze the scan-forward steps, we use forward steps, we use yet another probabilistic yet another probabilistic fact:fact:Fact 4: Fact 4: The expected number The expected number

of coin tosses required in of coin tosses required in order to get tails is 2order to get tails is 2

When we scan forward in When we scan forward in a list, the destination key a list, the destination key does not belong to a does not belong to a higher listhigher list– A scan-forward step is A scan-forward step is

associated with a former associated with a former coin toss that gave tailscoin toss that gave tails

By Fact 4, in each list the By Fact 4, in each list the expected number of expected number of scan-forward steps is 2scan-forward steps is 2Thus, the expected Thus, the expected number of scan-forward number of scan-forward steps is steps is OO(log (log nn))We conclude that a We conclude that a search in a skip list takes search in a skip list takes OO(log (log nn) ) expected timeexpected timeThe analysis of insertion The analysis of insertion and deletion gives similar and deletion gives similar resultsresults

3535

SummarySummaryA skip list is a data A skip list is a data structure for structure for dictionaries that uses dictionaries that uses a randomized a randomized insertion algorithminsertion algorithmIn a skip list with In a skip list with nn items items – The expected space The expected space

used is used is OO((nn))– The expected search, The expected search,

insertion and deletion insertion and deletion time is time is OO(log (log nn))

Using a more complex Using a more complex probabilistic analysis, probabilistic analysis, one can show that one can show that these performance these performance bounds also hold with bounds also hold with high probabilityhigh probability

Skip lists are fast and Skip lists are fast and simple to implement simple to implement in practicein practice