LOVELY PROFESSIONAL UNIVERSITY

Algorithm design and analysis

Topic: Amortized analysis: Aggregate method

Submitted to: Submitted by:

Mr. Vijay Kumar Garg Navdeep Singh

(CSE408) RK1R08B42

10804943

Acknowledgment

First of all I am heartily thankful to my teacher “MR. Vijay Kumar Garg” who stand a chance for me to get more knowledge about the topic “Amortized analysis: Aggregate method”

I am also thankful to my friends and my parents who supported me in completion of this term paper.

And last but not the least very much thankful to the University Providing Internet Facility inside the campus and providing excellent library facilities hence making the work easier.

Navdeep Singh

BTECH MTECH (CSE)

Introduction

Amortized analysis is a method of analyzing algorithms that considers the entire sequence of operations of the program. It allows for the establishment of a worst-case bound for the performance of an algorithm irrespective of the inputs by looking at all of the operations. At the heart of the method is the idea that while certain operations may be extremely costly in resources, they cannot occur at a high-enough frequency to weigh down the entire program because the number of less costly operations will far outnumber the costly ones in the long run, "paying back" the program over a number of iterations. It is particularly useful because it guarantees worst-case performance rather than making assumptions about the state of the program.

Amortized analysis initially emerged from a method called aggregate analysis, which is now subsumed by amortized analysis. However, the technique was first formally introduced by Robert Tarjan in his paper Amortized Computational Complexity, which addressed the need for a more useful form of analysis than the common probabilistic methods used. Amortization was initially used for very specific types of algorithms, particularly those involving binary trees and union operations. However, it is now ubiquitous and comes into play when analyzing many other algorithms as well.

In an amortized analysis, the time required to perform a sequence of data-structure operations is averaged over all the operations performed. Amortized analysis can be used to show that the average cost of an operation is small, if one averages over a sequence of operations, even though a single operation might be expensive. Amortized analysis differs from average-case analysis in that probability is not involved; an amortized analysis guarantees the average performance of each operation in the worst case.

Key ideas

●Amortized analysis is an upper bound: it's the average performance of each operation in the worst case.

●Amortized analysis is concerned with the overall cost of a sequence of operations. It does not say anything about the cost of a specific operation in that sequence. For example, it is invalid toreason, “The amortized cost of insertion into a splay tree with n items is O(log n), so when I insert '45' into this tree, the cost will be O(log n).” In fact, inserting '45' might require O(n) operations! It is only appropriate to say, “When I insert m items into a tree, the average time for each operation will be O(log n).”

● Amortized analysis is concerned with the overall cost of arbitrary sequences. An amortized

bound will hold regardless of the specific sequence; for example, if the amortized cost of insertion is O(log n), it is so regardless of whether you insert the sequence '10,' '160,' '2' or the sequence '2', '160', '10,' '399', etc.

●The two points above imply that both amortized and worstcase bounds should be understood when choosing an algorithm to use in practice. Because an amortized bound says nothing about the cost of individual operations, it may be possible that one operation in sequence requires a huge cost. Practical systems in which it is important that all operations have low and/or comparable costs may require an algorithm with a worse amortized cost but a better worst-case per operation bound.

●Amortized analysis can be understood to take advantage of the fact that some expensive operations may “pay” for future operations by somehow limiting the number or cost of expensive operations that can happen in the near future.

●If good amortized cost is a goal, an algorithm may be designed to explicitly perform this “cleanup” during expensive operations

Comparison to other analysis techniques

As mentioned above, worstcase analysis can give overly pessimistic bounds for sequences of operations, because such analysis ignores interactions among different operations on the same data structure (Tarjan 1985). Amortized analysis may lead to a more realistic worstcase bound by taking these interactions into account. Note that the bound offered by amortized analysis is, in fact, a worstcase bound on the average time per operation; a single operation in a sequence may have cost worse than this bound, but the average cost over all operations in any valid sequence will always perform within the bound.

Amortized analysis is similar to averagecase analysis, in that it is concerned with the cost averaged over a sequence of operations. However, averagecase analysis relies on probabilistic assumptions about the data structures and operations in order to compute an expected running time of an algorithm. Its applicability is therefore dependent on certain assumptions about probability distributions on algorithm inputs, which means the analysis is invalid if these assumptions do not hold (or that probabilistic analysis cannot be used at all, if input distributions cannot be described!) (Cormenet al. 2001, 92‒3). Amortized analysis needs no such assumptions. Also, it offers an upperbound on the worst case running time of a sequence of operations, and this bound will always hold. An averagecase bound, on the other hand, does not preclude the possibility that one will get “unlucky” and encounter an input that requires much more than the expected computation time, even if the assumptions on the distribution of inputs are valid. These differences between probabilistic and amortized analysis therefore have important consequences for the interpretation and relevance of the resulting bounds.

Amortized analysis is closely related to competitive analysis, which involves comparing the worstcase performance of an online algorithm to the performance of an optimal offline algorithm on the same data. Amortization is useful because competitive analysis's performance bounds must hold regardless of the particular input, which by definition is seen by the online algorithm in sequence rather than at the beginning of processing. Sleator and Tarjan (1985a) offer an example of using amortized analysis to perform competitive analysis.

Types:

Amortized analysis is of three types:

1) Aggregate analysis2) Accounting method3) Potential method

Aggregate analysis, in which we determine an upper bound T(n) on the total cost of a sequence of n operations. The amortized cost per operation is then T(n)/n.

Accounting method, in which we determine an amortized cost of each operation. When there is more than one type of operation, each type of operation may have a different amortized cost. The accounting method overcharges some operations early in the sequence, storing the overcharge as "prepaid credit" on specific objects in the data structure. The credit is used later in the sequence to pay for operations that are charged less than they actually cost.

The potential method, which is like the accounting method in that we determine the amortized cost of each operation and may overcharge operations early on to compensate for undercharges later. The potential method maintains the credit as the "potential energy" of the data structure instead of associating the credit with individual objects within the data structure.

We shall use two examples to examine these three models. One is a stack with the additional operation MULTIPOP, which pops several objects at once. The other is a binary counter that counts up from 0 by means of the single operation INCREMENT.

The aggregate analysis method

In the aggregate method of amortized analysis, we show that for all n, a sequence of n operations takes worst-case time T(n) in total. In the worst case, the average cost, or amortized cost, per operation is therefore T(n) / n. Note that this amortized cost applies to each operation, even when there are several types of operations in the sequence. The other two methods we shall study in this chapter, the accounting method and the potential method, may assign different amortized costs to different types of operations.

Stack operations

In our first example of the aggregate method, we analyze stacks that have been augmented with a new operation.

PUSH(S, x) pushes object x onto stack S.

POP(S) pops the top of stack S and returns the popped object.

Since each of these operations runs in O(1) time, let us consider the cost of each to be 1. The total cost of a sequence of n PUSH and POP operations is therefore n, and the actual running time for n operations is therefore (n).

The situation becomes more interesting if we add the stack operation MULTIPOP(S, k), which removes the k top objects of stack S, or pops the entire stack if it contains less than k objects. In the following pseudo code, the operation STACK-EMPTY returns TRUE if there are no objects currently on the stack, and FALSE otherwise.

MULTIPOP(S,k)1 while not STACK-EMPTY(S) and k 02 do POP(S)3 k k - 1

Figure 1 shows an example of MULTIPOP.

What is the running time of MULTIPOP(S, k) on a stack of s objects? The actual running time is linear in the number of POP operations actually executed, and thus it suffices to analyze MULTIPOP in terms of the abstract costs of 1 each for PUSH and POP. The number of iterations of the while loop is the number min(s, k) of objects popped off the stack. For each iteration of the loop, one call is made to POP in line 2. Thus, the total cost of MULTIPOP is min(s, k), and the actual running time is a linear function of this cost.

Figure 1 The action of MULTIPOP on a stack S, shown initially in (a). The top 4 objects are popped by MULTIPOP(S, 4), whose result is shown in (b). The next operation is MULTIPOP(S, 7), which empties the stack shown in (c) since there were fewer than 7 objects remaining.

Let us analyze a sequence of n PUSH, POP, and MULTIPOP operations on an initially empty stack. The worst-case cost of a MULTIPOP operation in the sequence is O(n), since the stack size is at most n. The worst-case time of any stack operation is therefore O(n), and hence a sequence of n operations costs O(n2), since we may have O(n) MULTIPOP operations costing O(n) each. Although this analysis is correct, the O(n2) result, obtained by considering the worst-case cost of each operation individually, is not tight.

Using the aggregate method of amortized analysis, we can obtain a better upper bound that considers the entire sequence of n operations. In fact, although a single MULTIPOP operation can be expensive, any sequence of n PUSH, POP, and MULTIPOP operations on an initially empty stack can cost at most O(n). Why? Each object can be popped at most once for each time it is pushed. Therefore, the number of times that POP can be called on a nonempty stack, including calls within MULTIPOP, is at most the number of PUSH operations, which is at most n. For any value of n, any sequence of n PUSH, POP, and MULTIPOP operations takes a total of O(n) time. The amortized cost of an operation is the average: O(n)/n = O(1).

We emphasize again that although we have just shown that the average cost, and hence running time, of a stack operation is O(1), no probabilistic reasoning was involved. We actually showed a worst-case bound of O(n) on a sequence of n operations. Dividing this total cost by n yielded the average cost per operation, or the amortized cost.

Incrementing a binary counter

As another example of the aggregate method, consider the problem of implementing a k-bit binary counter that counts upward from 0. We use an array A[0 . . k - 1] of bits, where length[A] = k, as the counter. A binary number x that is stored in the counter has its lowest-order bit in

A[0] and its highest-order bit in A[k - 1], so that . Initially, x = 0, and thus A[i] = 0 for i = 0, 1, . . . , k - 1. To add 1 (modulo 2k) to the value in the counter, we use the following procedure.

Amortized cost per operation is T(n)/nX A[4] A[3] A[2] A[1] A[0] Total Cost0 0 0 0 0 0 0

1 0 0 0 0 1 12 0 0 0 1 0 33 0 0 0 1 1 44 0 0 1 0 0 75 0 0 1 0 1 86 0 0 1 1 0 107 0 0 1 1 1 118 0 1 0 0 0 159 0 1 0 0 1 16

Figure 2 An 8-bit binary counter as its value goes from 0 to 16 by a sequence of 16 INCREMENT operations. Bits that flip to achieve the next value are shaded. The running cost for flipping bits is shown at the right. Notice that the total cost is never more than twice the total number of INCREMENT operations.

INCREMENT (A)1 i 02 while i< length[A] and A[i] = 13 do A[i] 04 i i + 15 if i < length[A]6 then A[i] 1

This algorithm is essentially the same one implemented in hardware by a ripple-carry counter Figure 2 shows what happens to a binary counter as it is incremented 16 times, starting with the initial value 0 and ending with the value 16. At the start of each iteration of the while loops in lines 2-4, we wish to add a 1 into position i. If A[i] = 1, then adding 1 flips the bit to 0 in position i and yields a carry of 1, to be added into position i + 1 on the next iteration of the loop. Otherwise, the loop ends, and then, if i < k, we know that A[i] = 0, so that adding a 1 into position i, flipping the 0 to a 1, is taken care of in line 6. The cost of each INCREMENT operation is linear in the number of bits flipped.

As with the stack example, a cursory analysis yields a bound that is correct but not tight. A single execution of INCREMENT takes time (k) in the worst case, in which array A contains all 1's. Thus, a sequence of n INCREMENT operations on an initially zero counter takes time O(nk) in the worst case.

We can tighten our analysis to yield a worst-case cost of O(n) for a sequence of n INCREMENT'S by observing that not all bits flip each time INCREMENT is called. As Figure 2 shows, A[0] does flip each time INCREMENT is called. The next-highest-order bit, A[1], flips only every other time: a sequence of n INCREMENT operations on an initially zero counter causes A[1] to flip n/2 times. Similarly, bit A[2] flips only every fourth time, or n/4 times in a sequence of n INCREMENT'S. In general, for i = 0, 1, . . . , lg n , bit A[i] flips n/2i times in a sequence of n

INCREMENT operations on an initially zero counter. For i > lg n , bit A[i] never flips at all. The total number of flips in the sequence is thus

by equation (3.4). The worst-case time for a sequence of n INCREMENT operations on an initially zero counter is therefore O(n), so the amortized cost of each operation is O(n)/n = O(1).

Conclusion

Amortized analysis is a useful tool that complements other techniques such as worstcase and averagecase analysis. It has been applied to a variety of problems, and it is also crucial to appreciating structures such as splay trees that have been designed to have good amortized bounds.

To understand the application of amortized analysis to common problems, it is essential to know the basics of both the accounting method and the potential method. The resources presented here supply many examples of both methods applied to real problems.

To perform an amortized analysis, one should choose either the accounting method or the potential method. The approaches yield equivalent results, but one might be more intuitively appropriate to the problem under consideration. There is no magic formula for arriving at a potential function or accounting credit scheme that will always work; the method used depends on the desired bounds and the desired complexity of the analysis. Some strategies that sometimes work, however, include enumerating the ways in which an algorithm might operate on a data structure, then performing an analysis for each case; computing the potential of a data structure as a sum of “local” potentials so that one can reason about the effects of local changes while ignoring irrelevant and unchanging components of the structure; designing the potential method around the desired form of the result (e.g., relating the potential to the log of the subtree size for splay trees, as the desired outcome is a logarithmic bound); and reasoning about each type of operation in a sequence individually before coming up with a bound on an arbitrary sequence of operations.

An understanding of amortized analysis is essential to success in an algorithms course, to understanding the implication of theoretical bounds on realworld performance, and to thoroughly appreciating the design and purpose of certain data structures. The reader is therefore again urged to consult any of the sources mentioned here to improve his or her understanding of amortized analysis and to explore these algorithms in greater depth.

References:-

1) http://www.eli.sdsu.edu/courses/fall95/cs660/notes/amortized/Amortized.html#RTFToC3

2) http://staff.ustc.edu.cn/~csli/graduate/algorithms/book6/chap18.htm

3) http://www.bowdoin.edu/~ltoma/teaching/cs231/fall11/Lectures/13-amortized/amortized.pdf