Cormen Algo-lec13

8/14/2019 Cormen Algo-lec13

1/42

October 31, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L13.1

Introduction to Algorithms

6.046J/18.401J

LECTURE 13Amortized Analysis

Dynamic tables

Aggregate method Accounting method

Potential method

Prof. Charles E. Leiserson


2/42


How large should a hash

table be?Goal: Make the table as small as possible, butlarge enough so that it wont overflow (orotherwise become inefficient).

Problem: What if we dont know the proper size

in advance?

IDEA: Whenever the table overflows, grow itby allocating (via malloc ornew) a new, largertable. Move all items from the old table into the

new one, and free the storage for the old table.

Solution:Dynamic tables.


3/42


Example of a dynamic table

1. INSERT 1

2. INSERT overflow


4/42October 31, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L13.4


1. INSERT 11

2. INSERT overflow




1. INSERT 11

2. INSERT 2




1. INSERT 11

2. INSERT 22

3. INSERT overflow




1. INSERT2

1

2. INSERT3. INSERT overflow


8/42



1. INSERT 1

2. INSERT 2

3. INSERT


9/42



1. INSERT

2. INSERT3. INSERT

4. INSERT4

3

2

1


10/42



1. INSERT 1

2. INSERT 233. INSERT

4. INSERT5. INSERT

4

overflow


11/42



1. INSERT

2. INSERT

4

3

2

1

3. INSERT

4. INSERT5. INSERT overflow


12/42



1. INSERT

2. INSERT

1

2

33. INSERT

4. INSERT5. INSERT

4


13/42



1. INSERT

2. INSERT3. INSERT

4. INSERT

6. INSERT 65. INSERT 5

4

3

2

1

77. INSERT


14/42


Worst-case analysis

Consider a sequence ofn insertions. The

worst-case time to execute one insertion is(n). Therefore, the worst-case time forninsertions is n (n) = (n2).

WRONG! In fact, the worst-case cost forn insertions is only (n) (n2).

Lets see why.


15/42


Tighter analysis

Let ci = the cost of the i th insertion

=i ifi 1 is an exact power of2,

1 otherwise.

i 1 2 3 4 5 6 7 8 9 10

sizei 1 2 4 4 8 8 8 8 16 16

ci 1 2 3 1 5 1 1 1 9 1


16/42


Tighter analysis

Let ci = the cost of the i th insertion

=i ifi 1 is an exact power of2,

1 otherwise.

i 1 2 3 4 5 6 7 8 9 10

sizei 1 2 4 4 8 8 8 8 16 16

1 1 1 1 1 1 1 1 1 11 2 4 8

ci


17/42


Tighter analysis (continued)

)(

3

2

)1lg(

0

1

n

n

n

c

n

j

j

n

i

i

=

+

=

=

=

Cost ofn insertions

.

Thus, the average cost of each dynamic-table

operation is (n)/n = (1).


18/42


Amortized analysis

An amortized analysis is any strategy foranalyzing a sequence of operations to

show that the average cost per operation issmall, even though a single operationwithin the sequence might be expensive.

Even though were taking averages, however,probability is not involved!

An amortized analysis guarantees theaverage performance of each operation inthe worst case.


19/42


Types of amortized analyses

Three common amortization arguments:

the aggregate method,

the accountingmethod, thepotentialmethod.

Weve just seen an aggregate analysis.

The aggregate method, though simple, lacks the

precision of the other two methods. In particular,the accounting and potential methods allow aspecific amortized costto be allocated to each

operation.


20/42


Accounting method

Charge i th operation a fictitious amortized costi, where $1pays for1 unit of work (i.e., time).

This fee is consumed to perform the operation. Any amount not immediately consumed is storedin the bankfor use by subsequent operations.

The bank balance must not go negative! Wemust ensure that

==

n

ii

n

ii cc 11

for all n. Thus, the total amortized costs provide an upperbound on the total true costs.

A ti l i f


21/42


Accounting analysis of

dynamic tables

$0$0 $0$0 $0$0 $0$0 $2$2 $2$2

Example:

$2 $2

Charge an amortized cost ofi = $3 for the i thinsertion.

$1pays for the immediate insertion. $2 is stored for later table doubling.

When the table doubles, $1pays to move arecent item, and $1pays to move an old item.

overflow

Acco nting anal sis of


22/42



dynamic tables

Example:




overflow

$0$0 $0$0 $0$0 $0$0 $0$0 $0$0 $0$0 $0$0



23/42


Accounting analysis ofdynamic tables



Example:


$0$0 $0$0 $0$0 $0$0 $0$0 $0$0 $0$0 $0$0 $2 $2 $2

Accounting analysis


24/42


Accounting analysis(continued)

Key invariant: Bank balance never drops below 0.Thus, the sum of the amortized costs provides an

upper bound on the sum of the true costs.

i

sizei

ci

i

banki

1 2 3 4 5 6 7 8 9 10

1 2 4 4 8 8 8 8 16 16

1 2 3 1 5 1 1 1 9 1

2 3 3 3 3 3 3 3 3 3

1 2 2 4 2 4 6 8 2 4

*

*Okay, so I lied. The first operation costs only $2, not $3.


25/42


Potential method

IDEA: View the bank account as the potentialenergy ( laphysics) of the dynamic set.

Framework:

Start with an initial data structureD0.

Operation i transformsDi1 toDi. The cost of operation i is ci. Define apotential function : {Di} R,

such that (D0 ) = 0 and (Di ) 0 for all i. The amortized costi with respect to is

defined to be

i = ci + (Di) (Di1).


26/42


Understanding potentials

i = ci + (Di) (Di1)

potential difference i

If i > 0, then i > ci. Operation i storeswork in the data structure for later use.

If i< 0, then

i< c

i. The data structure

delivers up stored work to help pay foroperation i.

The amortized costs bound


27/42


The amortized costs boundthe true costs

The total amortized cost ofn operations is

( ) =

=+=

n

iiii

n

ii DDcc

11

1

)()(

Summing both sides.



28/42




( )

)()(

)()(

01

11

1

DDc

DDcc

n

n

ii

n

iiii

n

ii

+=

+=

=

=

=

The series telescopes.



29/42




( )

=

=

=

=

+=

+=

n

ii

n

n

ii

n

iiii

n

ii

c

DDc

DDcc

1

01

11

1

)()(

)()(

since (Dn) 0 and(D0 ) = 0.


30/42


31/42

C l l i f i d


32/42


Calculation of amortized costs

The amortized cost of the i th insertion is

i = ci +

(Di)

(Di1)

i ifi 1 is an exact power of2,

1 otherwise;+ (2i 2lg i)(2(i 1) 2lg (i1))

=

C l l ti f ti d t


33/42


Calculation of amortized costs

The amortized cost of the i th insertion is

i = ci +

(Di)

(Di1)

i ifi 1 is an exact power of2,

1 otherwise;

=

+ (2i 2lg i)(2(i 1) 2lg (i1))

i ifi 1 is an exact power of2,1 otherwise;

=

+ 2 2lg i

+ 2lg (i1)

.


34/42

C l l ti


35/42


Calculation

Case 1: i 1 is an exact power of2.

i = i + 2 2lg i + 2lg (i1)

= i + 2 2(i 1) + (i 1)

C l l ti


36/42


Calculation


i = i + 2 2lg i + 2lg (i1)

= i + 2 2(i 1) + (i 1)= i + 2 2i + 2 + i 1

C l l ti


37/42


Calculation


i = i + 2 2lg i + 2lg (i1)

= i + 2 2(i 1) + (i 1)= i + 2 2i + 2 + i 1= 3

Calculation


38/42


Calculation


i = i + 2 2lg i + 2lg (i1)

= i + 2 2(i 1) + (i 1)= i + 2 2i + 2 + i 1= 3

Case 2: i 1 is notan exact power of2.i = 1 + 2 2

lg i + 2lg (i1)


39/42

Calculation


40/42


Calculation


i = i + 2 2lg i + 2lg (i1)

= i + 2 2(i 1) + (i 1)= i + 2 2i + 2 + i 1= 3


lg i + 2lg (i1)

= 3

Therefore, n insertions cost (n) in the worst case.

Calculation


41/42


Calculation


i = i + 2 2lg i + 2lg (i1)

= i + 2 2(i 1) + (i 1)= i + 2 2i + 2 + i 1= 3


lg i + 2lg (i1)

= 3

Therefore, n insertions cost (n) in the worst case.Exercise: Fix the bug in this analysis to show that

the amortized cost of the first insertion is only 2.

Conclusions


42/42


Conclusions

Amortized costs can provide a clean abstractionof data-structure performance.

Any of the analysis methods can be used whenan amortized analysis is called for, but eachmethod has some situations where it is arguablythe simplest or most precise.

Different schemes may work for assigning

amortized costs in the accounting method, orpotentials in the potential method, sometimesyielding radically different bounds.

Cormen Algo-lec13

Documents

Transcript of Cormen Algo-lec13