1

Data Structures

A program solves a problem.A program solves a problem. A solution consists of:A solution consists of:

a way to organize the dataa way to organize the data sequence of steps to solve the problem. sequence of steps to solve the problem.

In other words:In other words: programs = algorithms + data structures programs = algorithms + data structures

2

Data Structures

From A to ZFrom A to Z Problem specificationProblem specification Analysis and DesignAnalysis and Design

Break up the solution in well-defined Break up the solution in well-defined modulesmodules

Specify pre/post-conditionsSpecify pre/post-conditions Specify how modules interactSpecify how modules interactIDEA: separate the purpose of a module IDEA: separate the purpose of a module

from its implementationfrom its implementation

3

Data Structures

Decide how the data will be organizedDecide how the data will be organizedDecide what operations can be performed on Decide what operations can be performed on

datadataIDEA: separate the data organization and IDEA: separate the data organization and

operations from the implementationoperations from the implementation ImplementationImplementation Testing Testing MaintenanceMaintenance

Program must be easy to understand/modifyProgram must be easy to understand/modify

4

Data Structures Key ideas:Key ideas:

Modularity Modularity break problem into subproblemsbreak problem into subproblems

AbstractionAbstractionhide non-relevant information from the userhide non-relevant information from the user

Information hidingInformation hidinghide data not needed by a modulehide data not needed by a module

Software reuseSoftware reusedo not reinvent the wheeldo not reinvent the wheel

5

Data Structures

Abstract Data Type (ADT) = Abstract Data Type (ADT) = a collection of data, and a collection of data, and a set of operations on the data. (typical ops?)a set of operations on the data. (typical ops?)

The user does not know The user does not know the internal representationthe internal representation the implementations of the operationsthe implementations of the operations

The user does knowThe user does know a conceptual picture of the organization of the dataa conceptual picture of the organization of the data how to perform operations on ithow to perform operations on it

6

Data Structures

Goals:Goals: Program must be easy to understand and Program must be easy to understand and

modify.modify. Design a data structure that can be reused in Design a data structure that can be reused in

other programs.other programs. Design (or select) the Design (or select) the rightright data structure. data structure. The program must be correct and efficient The program must be correct and efficient

(memory/time).(memory/time).

7

Mathematics Review Basic formulas for derive and reviews basic proof Basic formulas for derive and reviews basic proof

techniquestechniques ExponentsExponents

1

2

222

2

)(

NNN

NNNN

ABBA

BAB

A

BABA

XXXX

XX

XX

X

XXX

8

Mathematics Review LogarithmsLogarithms All logarithms are to the base 2 unless specified otherwise.All logarithms are to the base 2 unless specified otherwise. DefinitionDefinition: :

0,0,log BXABifonlyandifBX XA

1665536log,101024log,12log,01log

0,log

logloglog

logloglog

0;log

loglog

xxx

BABA

BAAB

CA

BB

equalitiesUseful

c

cA

9

Mathematics Review

b=2 b=e

b=10

10

Mathematics Review Series: Geometric Series: Geometric

seriesseries

DerivationDerivation

Let Let S = 1+A+AS = 1+A+A22+…… +…… (1)(1)

where, 0<A<1where, 0<A<1

then then AS = A+AAS = A+A22+A+A33+…+…(2)(2)

Subtracting (1) and (2), Subtracting (1) and (2), we get S-AS = 1, i.e.we get S-AS = 1, i.e.A

A

thenAIf

A

AA

N

i

i

NN

i

i

1

1

,10

1

1

0

1

0

AS

1

1

11

Mathematics Review Series: Arithmetic Series: Arithmetic

seriesseriesExample: To find the Example: To find the

sum sum

2+5+8+….+ (3k-1)2+5+8+….+ (3k-1)

= 3(1+2+3+…+k) = 3(1+2+3+…+k)

- (1+1+1+….+1)- (1+1+1+….+1)

2

)1(

0

NNi

N

i

kkk

2

)1(3

12

Mathematics Review

The P wordThe P word

- - to proof a false to proof a false statementstatement::

proof by counter proof by counter exampleexample

- - to proof a correct to proof a correct statementstatement

- proof by induction- proof by induction(1) proving a base case(1) proving a base case(2) inductive hypothesis(2) inductive hypothesis

- proof by contradiction- proof by contradiction(1) assume it is false(1) assume it is false(2) show that this assumption (2) show that this assumption

is falseis false

13

Algorithms

A problem is a specification of an input-A problem is a specification of an input-output relationship.output relationship.

Algorithm = Algorithm = a a well-definedwell-defined computational computational

procedure that takes a set of values as procedure that takes a set of values as input and produces a set of values as input and produces a set of values as output. output.

a tool for solving a well-specified a tool for solving a well-specified computational problem.computational problem.

14

Algorithm characteristics

CorrectnessCorrectness algorithm halts with correct output for algorithm halts with correct output for

every legal instance of the problem.every legal instance of the problem. proof of correctnessproof of correctness

GeneralityGenerality must work for all legal data (eg. a must work for all legal data (eg. a

sorting algorithm should work on both sorting algorithm should work on both numbers and names)numbers and names)

15

Algorithm characteristics

FinitenessFiniteness won’t take forever to runwon’t take forever to run

EfficiencyEfficiency making efficient use of resources (time making efficient use of resources (time

and space)and space)

16

Algorithm characteristics

PseudocodePseudocode = an English-like language with limited = an English-like language with limited vocabulary that we use to describe algorithms in an easy vocabulary that we use to describe algorithms in an easy to understand manner.to understand manner.

Example: Linear SearchExample: Linear Search

Input : array A, key xOutput: index of array element equal to x, -1 if x is not in Afor i = 1 to length[A] if A[i]==x then return i endifendforreturn -1

17

Algorithm characteristics

Is the Linear Search algorithm...Is the Linear Search algorithm... correct?correct? efficient?efficient? general?general? finite?finite?

18

Algorithm analysis

Analyzing an algorithm = estimating the Analyzing an algorithm = estimating the resources it requiresresources it requires

TimeTime Number of steps/operations executedNumber of steps/operations executed Each operation has a costEach operation has a cost A function of problem (input) sizeA function of problem (input) size

SpaceSpace Amount of temporary storage required.Amount of temporary storage required. We usually don’t count the input.We usually don’t count the input.

19

Algorithm analysis

Best case analysisBest case analysis Given the algorithm and input of size n that Given the algorithm and input of size n that

makes it run fastest (compared to all other makes it run fastest (compared to all other possible inputs of size n), what is the running possible inputs of size n), what is the running time?time?

Worst case analysisWorst case analysis Given the algorithm and input of size n that Given the algorithm and input of size n that

makes it run slowest (compared to all other makes it run slowest (compared to all other possible inputs of size n), what is the running possible inputs of size n), what is the running time?time?

20

Algorithm analysis

Average case analysisAverage case analysis Given the algorithm and a typical, Given the algorithm and a typical,

average input of size n, what is the average input of size n, what is the running time?running time?

21

Insertion Sort One of the simplest methods to sort an array is an insertion sort. An One of the simplest methods to sort an array is an insertion sort. An

example of an insertion sort occurs in everyday life while playing cards. example of an insertion sort occurs in everyday life while playing cards. To sort the cards in your hand you extract a card, shift the remaining To sort the cards in your hand you extract a card, shift the remaining cards, and then insert the extracted card in the correct place. This cards, and then insert the extracted card in the correct place. This process is repeated until all the cards are in the correct sequence. Both process is repeated until all the cards are in the correct sequence. Both average and worst-case time is average and worst-case time is OO((nn22). ).

Assuming there are Assuming there are nn elements in the array, we must index through elements in the array, we must index through nn - 1 - 1 entries. For each entry, we may need to examine and shift up to entries. For each entry, we may need to examine and shift up to nn - 1 - 1 other entries, resulting in a other entries, resulting in a OO((nn22) algorithm. The insertion sort is an ) algorithm. The insertion sort is an in-in-placeplace sort. That is, we sort the array in-place. No extra memory is sort. That is, we sort the array in-place. No extra memory is required. The insertion sort is also a required. The insertion sort is also a stablestable sort. Stable sorts retain the sort. Stable sorts retain the original ordering of keys when identical keys are present in the input original ordering of keys when identical keys are present in the input data. data.

22

Starting near the top of the array in Figure (a), we Starting near the top of the array in Figure (a), we extract the 3. Then the above elements are shifted extract the 3. Then the above elements are shifted down until we find the correct place to insert the 3. down until we find the correct place to insert the 3. This process repeats in Figure (b) with the next This process repeats in Figure (b) with the next number. Finally, in Figure (c), we complete the sort number. Finally, in Figure (c), we complete the sort by inserting 2 in the correct place.by inserting 2 in the correct place.

23

Algorithm Analysis

InsertionSort(A)

for i=2 to length(A)

item = A[i]

j = i - 1

while (j > 0 and A[j] > item)

A[j+1] = A[j]

j = j - 1

A[j+1] = item

cost times executedc1 n

c2 n-1

c3 n-1

c4 ti

c5 (ti-1)

c6 (ti-1)

c7 n-1

24

Algorithm Analysis

Observations:Observations: the constants are “unimportant” detailsthe constants are “unimportant” details some terms “grow” faster than others as some terms “grow” faster than others as

the size of the input increases.the size of the input increases. Let’s simplify this...Let’s simplify this...

25

Algorithm Analysis

Consider:Consider: nn22 + 100n + 1000 + 100n + 1000 How does this change as n becomes larger? How does this change as n becomes larger? Is this a “good” running time?Is this a “good” running time?

Compare:Compare: nn22/3 to 3n/3 to 3n Do the constants really make a difference?Do the constants really make a difference?

26

Algorithm Analysis

Idea:Idea: comparing algorithms:comparing algorithms:

upper boundsupper boundslower boundslower boundstight bounds tight bounds

we are interested in the growth rate of an we are interested in the growth rate of an algorithm: how its running time changes algorithm: how its running time changes proportional to the size of the input.proportional to the size of the input.

27

Algorithm Analysis: Big Oh

A function A function f(n)f(n) is O( is O(g(n)g(n)) if there exist constants ) if there exist constants cc, , nn00>0>0 such that such that f(n)f(n) c c··g(n)g(n) for all for all n n n n00

What does this mean in English? What does this mean in English? cc··g(n)g(n) is an upper bound of is an upper bound of f(n)f(n) for large for large nn

Examples:Examples: f(n) = 3nf(n) = 3n22+2n+1 is O(n+2n+1 is O(n22)) f(n) = 2n is O(nf(n) = 2n is O(n22) (it is also O(n) )) (it is also O(n) ) f(n) = 1000nf(n) = 1000n33 is O(n is O(n1010))

28

Algorithm Analysis: Omega

A function A function f(n)f(n) is is ((g(n)g(n)) if there exist ) if there exist constants constants c,c, nn00>0>0 such that such that

f(n) f(n) c c··g(n)g(n) for allfor all nnnn00

What does this mean in English? What does this mean in English? cc··g(n)g(n) is a lower bound of is a lower bound of f(n)f(n) for large for large nn

Example:Example: f(n) = nf(n) = n22 is is (n)(n)

29

Algorithm Analysis: Theta

A function A function f(n)f(n) is is ((g(n)g(n)) if it is both O) if it is both O((g(n)) and g(n)) and (g(n)) (g(n)) , in other words:, in other words: There exist constants There exist constants c1,c2,n>0c1,c2,n>0 s.t. s.t.

0 0 c c1 1 g(n) g(n) f(n) f(n) c c22g(n) g(n) for allfor all nnnn00

What does this mean in English? What does this mean in English? f(n) f(n) andand g(n) g(n) have the same order of growth have the same order of growth indicates a tight bound.indicates a tight bound.

Example:Example: f(n) = 2n+1 is f(n) = 2n+1 is (n)(n)

30

Algorithm Analysis

O, O, , , are transitive and reflexive are transitive and reflexive is symmetricis symmetric f(n)f(n) = O( = O(g(n)g(n)) iff ) iff g(n)g(n) = = ((f(n)f(n))) analogy:analogy:

f(n)f(n) = O( = O(g(n)g(n)) ) a a b b f(n)f(n) = = ((g(n)g(n)) ) a a b b f(n)f(n) = = ((g(n)g(n)) ) a = b a = b Note: this doesn’t mean that all functions are Note: this doesn’t mean that all functions are

asymptotically comparableasymptotically comparable

31

Growth of functions

• A way to describe behavior of functions in the limit -- asymptotic efficiency.

• Growth of functions.

• Focus on what’s important by abstracting away low-order terms and constant factors.

• How to indicate running times of algorithms?

• A way to compare “sizes” of functions:

O ≤ ≥

=

o < ω >

32

33

Standard notations and common functions

Monotonicity:Monotonicity: f (n) f (n) is is monotonically increasingmonotonically increasing if if m m ≤ ≤ n n ⇒ ⇒ f (m) f (m) ≤ ≤ f (n)f (n).. f f ((nn) ) is is monotonically decreasingmonotonically decreasing if if m m n n ⇒ ⇒ f f ((mm) )

≥ ≥ f f ((nn)).. f f ((nn) ) is is strictly increasingstrictly increasing if if m m < < n n ⇒ ⇒ f f ((mm) < ) < f f ((nn)).. f f ((nn) ) is is strictly decreasingstrictly decreasing if if m m n n ⇒ ⇒ f f ((mm) > ) > f f ((nn))..

Floor and Ceilings: Floor and Ceilings: x – 1x – 1 < < xx xx xx < < x+1x+1 Modular arithmetic: Modular arithmetic: a mod n = a - a mod n = a - a/na/n nn

-- refer to the textbook (p36-38).-- refer to the textbook (p36-38). Polynomials:Polynomials:

Exponentials:Exponentials: Logarithms:Logarithms: Factorials:Factorials:

34

Algorithm Analysis

What if a constant is too large?What if a constant is too large? Consider Consider 10101010nn as compared to as compared to nn22

35

Algorithm analysis

Input: Collection of size nOutput: The smallest difference between any two different numbers in the collection Algorithm: List all pairs of different numbers in the collectionestimate=difference between values in first pair.for each remaining pair compute the difference between the two values if it is less than estimate set estimate to that differencereturn estimate

36

Algorithm analysis

Input: Collection of size nOutput: The smallest difference between any two different numbers in the collection Algorithm: estimate = infinityfor each value v in the collection for each value u v if |u-v| < estimate estimate = |u-v|return estimate