TextText

Chapters 1, 2Chapters 1, 2

SortingSorting

Sorting Problem:Sorting Problem: Input: A sequence of Input: A sequence of nn numbers numbers Output: A permutation (reordering) of Output: A permutation (reordering) of

the input sequence such that: the input sequence such that: naaa ''' 21

naaa ',,',' 21

naaa ,,, 21

instanceinstance

AlgorithmAlgorithm::well-defined computational procedurewell-defined computational procedure that that transforms input into output transforms input into output steps for the computer to follow to solve a problemsteps for the computer to follow to solve a problem

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Finding a place for item with value 5 in position 1:Finding a place for item with value 5 in position 1:Swap item in position 0 with item in position 1.Swap item in position 0 with item in position 1.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Positions 0 through 1 are now in non-decreasing order.Positions 0 through 1 are now in non-decreasing order.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Finding a place for item with value 1 in position 2:Finding a place for item with value 1 in position 2:Swap item in position 1 with item in position 2.Swap item in position 1 with item in position 2.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Finding a place for item with value 1:Finding a place for item with value 1:Swap item in position 0 with item in position 1.Swap item in position 0 with item in position 1.

Positions 0 through 2 are now in non-decreasing order.Positions 0 through 2 are now in non-decreasing order.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Finding a place for item with value 3 in position 3:Finding a place for item with value 3 in position 3:Swap item in position 2 with item in position 3.Swap item in position 2 with item in position 3.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Finding a place for item with value 3:Finding a place for item with value 3:Swap item in position 1 with item in position 2.Swap item in position 1 with item in position 2.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Positions 0 through 3 are now in non-decreasing order.Positions 0 through 3 are now in non-decreasing order.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Finding a place for item with value 2 in position 4:Finding a place for item with value 2 in position 4:Swap item in position 3 with item in position 4.Swap item in position 3 with item in position 4.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Finding a place for item with value 2:Finding a place for item with value 2:Swap item in position 2 with item in position 3.Swap item in position 2 with item in position 3.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Finding a place for item with value 2:Finding a place for item with value 2:Swap item in position 1 with item in position 2.Swap item in position 1 with item in position 2.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Positions 0 through 4 are now in non-decreasing order.Positions 0 through 4 are now in non-decreasing order.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Finding a place for item with value 6 in position 5:Finding a place for item with value 6 in position 5:Swap item in position 4 with item in position 5.Swap item in position 4 with item in position 5.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Positions 0 through 5 are now in non-decreasing order.Positions 0 through 5 are now in non-decreasing order.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Finding a place for item with value 4 in position 6:Finding a place for item with value 4 in position 6:Swap item in position 5 with item in position 6.Swap item in position 5 with item in position 6.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Finding a place for item with value 4:Finding a place for item with value 4:Swap item in position 4 with item in position 5.Swap item in position 4 with item in position 5.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Positions 0 through 6 are now in non-decreasing order.Positions 0 through 6 are now in non-decreasing order.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Finding a place for item with value 7 in position 7:Finding a place for item with value 7 in position 7:Swap item in position 6 with item in position 7.Swap item in position 6 with item in position 7.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Positions 0 through 7 are now in non-decreasing order.Positions 0 through 7 are now in non-decreasing order.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Positions 0 through 7 are now in non-decreasing order.Positions 0 through 7 are now in non-decreasing order.

Insertion Sort Animation Insertion Sort Animation

http://www.cs.brockport.edu/cs/java/apps/sorters/insertsortaniminp.html

Positions 0 through 7 are now in non-decreasing order.Positions 0 through 7 are now in non-decreasing order.

Asymptotic NotationAsymptotic Notation

courtesy of courtesy of Prof. CostelloProf. Costello

O(g(n)) is O(g(n)) is a a setset of of functions, functions, so we so we often say often say f(n) is f(n) is inin O(g(n)).O(g(n)).

Asymptotic Notation (cont.)Asymptotic Notation (cont.)courtesy of courtesy of Prof. CostelloProf. Costello

Asymptotic AnalysisAsymptotic Analysis

Math fact sheet (courtesy of Prof. Costello) is on our web site.Math fact sheet (courtesy of Prof. Costello) is on our web site.

Function Order of GrowthFunction Order of Growth

O( ) upper boundO( ) upper bound

( ) lower bound( ) lower bound

( ) upper & lower bound( ) upper & lower bound

nn11 n lg(n) n lg(n) n lgn lg22(n) (n) 22nnnn55 lg(n)lg(n)lglg(n) lglg(n) nn22

know how to use asymptotic complexity notationknow how to use asymptotic complexity notation

to describe time or space complexityto describe time or space complexity

know how to order functions know how to order functions asymptoticallyasymptotically

(behavior as n becomes large)(behavior as n becomes large)

Types of Algorithmic InputTypes of Algorithmic Input

Best-Case InputBest-Case Input: of all possible algorithm inputs of size n, it : of all possible algorithm inputs of size n, it generates the “best” resultgenerates the “best” result for Time Complexity: “best” is smallest running timefor Time Complexity: “best” is smallest running time

Best-Case Input Produces Best-Case Running TimeBest-Case Input Produces Best-Case Running Time provides a provides a lower bound on the algorithm’s asymptotic running timelower bound on the algorithm’s asymptotic running time (subject to any implementation assumptions)(subject to any implementation assumptions)

for Space Complexity: “best” is smallest storagefor Space Complexity: “best” is smallest storage

Average-Case InputAverage-Case InputWorst-Case InputWorst-Case Input

these are defined similarlythese are defined similarly

Best-Case Time <= Average-Case Time <= Worst-Case TimeBest-Case Time <= Average-Case Time <= Worst-Case Time

Bounding Algorithmic Time(using cases)

Bounding Algorithmic Time(using cases)

nn11 n lg(n) n lg(n) n lgn lg22(n) (n) 22nnnn55 lg(n)lg(n)lglg(n) lglg(n) nn22

T(n) = T(n) = (1)(1) T(n) = T(n) = (2(2nn))very loose bounds are not very useful!very loose bounds are not very useful!

Worst-CaseWorst-Case time of T(n) = time of T(n) = (2(2nn) tells us that worst-case inputs cause the ) tells us that worst-case inputs cause the algorithm to take algorithm to take at mostat most exponential time (i.e. exponential time is exponential time (i.e. exponential time is sufficientsufficient). ). But, can the algorithm every really take exponential time? (i.e. is exponential But, can the algorithm every really take exponential time? (i.e. is exponential timetime necessary necessary?)?)

If, If, for arbitrary nfor arbitrary n, we find a worst-case input that forces the algorithm to use , we find a worst-case input that forces the algorithm to use exponential time, then this exponential time, then this tightenstightens the the lower boundlower bound on the worst-case on the worst-case running time. If we can force the lower and upper bounds on the worst-case running time. If we can force the lower and upper bounds on the worst-case time to match, then we can say that, for the worst-case running time, time to match, then we can say that, for the worst-case running time,

T(n) = T(n) = ((22n n ) ) (i.e. we’ve found the (i.e. we’ve found the minimum upper boundminimum upper bound, so the upper , so the upper bound is bound is tight.)tight.)

Using “case” we can discuss lower and/or upper bounds on:Using “case” we can discuss lower and/or upper bounds on:best-case running timebest-case running time or or average-case running timeaverage-case running time or or worst-case running timeworst-case running time

Bounding Algorithmic Time(tightening bounds)

Bounding Algorithmic Time(tightening bounds)

nn11 n lg(n) n lg(n) n lgn lg22(n) (n) 22nnnn55 lg(n)lg(n)lglg(n) lglg(n) nn22

TTBB(n) = (n) = (1)(1) TTWW (n) = (n) = (2(2nn))

for example...for example...

1st attempt1st attempt 1st attempt1st attempt

TTBB (n) = (n) = (n)(n)

1st attempt1st attempt

2nd attempt2nd attempt

TTWW (n) = (n) = (n(n22))

Here we denote best-case time by THere we denote best-case time by TBB(n); worst-case time by T(n); worst-case time by TWW(n) (n)

TTBB(n) = (n) = (n)(n)2nd attempt2nd attempt

1st attempt1st attempt

TTWW(n) = (n) = (n(n22))

Algorithm BoundsAlgorithm Bounds

Know the Difference!Know the Difference!

(n)(n)11 22nnO(nO(n5)5)

worst-case boundsworst-case bounds on problemon problem

An inefficient algorithm for An inefficient algorithm for the problem might exist the problem might exist that takes this much time, that takes this much time, but would not help us.but would not help us.

No algorithm for the problem No algorithm for the problem exists that can solve it for exists that can solve it for worst-case inputs in less worst-case inputs in less than linear time .than linear time .

Strong BoundStrong Bound: A : A worst-case worst-case lowerlower bound bound on a problem holds for on a problem holds for everyevery algorithm that algorithm that solves the problem and solves the problem and abides by the problem’s abides by the problem’s assumptions.assumptions.

Weak BoundWeak Bound: A worst-case : A worst-case upperupper bound on a problem bound on a problem comes from just considering comes from just considering oneone algorithm. Other, less algorithm. Other, less efficient algorithms that solve efficient algorithms that solve this problem might exist, but this problem might exist, but we don’t care about them!we don’t care about them!

Both the upper and lower bounds could be Both the upper and lower bounds could be looseloose (i.e. perhaps could be (i.e. perhaps could be tightenedtightened later on). later on).