Computer Science 101

Efficiency and Complexity Analysis

Things to Desire in an Algorithm

• Correctness

• Maintainability

• Efficiency

Measuring Efficiency

• Empirical - use clock to get actual running times for different inputs

• Problems:– Different machines have different running

times– Some running times are so long that it is

impractical to check them

Measuring Efficiency

• Analytical - use pencil and paper to determine the abstract amount of work that a program does for different inputs

• Advantages:– Machine independent– Can predict running times of programs that are

impractical to run

Complexity Analysis

• Pick an instruction that will run most often in the code

• Determine the number of times this instruction will be executed as a function of the size of the input data

• Focus on abstract units of work, not actual running time

Example: Search for Largest

We focus on the comparison (>) inside the loop and ignore the other instructions.

for a list of length 1, zero comparisons for a list of length 2, one comparison.for list of length N, N - 1 comparisons

set Largest 1 set Current to 2while Current <= N do if A(Current) > A(Largest) then set Largest to Current increment Current

Big-O Notation

• Big-O notation expresses the amount of work a program does as a function of the size of the input

• O(n) stands for order of magnitude n, or order of n for short

• Search for the largest is O(n), where n is the size of the input

Common Orders of Magnitude

Constant O(k)

Logarithmic O(log2n)

Linear O(n)

Quadratic O(n2)

Exponential O(kn)

Common Orders of Magnitude

n O(log2n) O(n) O(n2) O(2n)

2 1 2 4 4 4 2 4 16 64 8 3 8 64 256 16 4 16 256 65536 32 5 32 1024 4294967296 64 6 64 4096 19 digits! 128 7 128 16384 256 8 256 65536 512 9 512 2621441024 10 1024 1048576

• Recall that search for a value required exactly 3N + 3 steps in the worst case

• As N gets very large, the difference between N and N + K becomes negligible (where K is a constant)

• As N gets very large, the difference between N and N / K or N * K also becomes negligible

• Use the highest degree term in a polynomial and drop the others (N2 – 2N + 2) N2

Approximations

Example Approximations

n O(n) O(n) + 2 O(n2) O(n2) + n 2 2 4 4 6 4 4 6 16 20 8 8 10 64 72 16 16 18 256 272 32 32 34 1024 1056 64 64 66 4096 5050 128 128 130 16384 16512 256 256 258 65536 65792 512 512 514 262144 2626561024 1024 1026 1048576 1049600

Analysis of Selection Sort

• The main loop runs approximately N times

• Thus, the search for largest algorithm runs N times

• The loop within the ith run of search for largest runs N - i times

Analysis of Selection Sort

Overall, the number of comparisons performed by the search for the largest value within selection sort is

N - 1 + N - 2 + N - 3 + . . + 1 = (N2 – N)/2 N2

Selection sort is a quadratic algorithm

Other Algorithms (Worst Cases)

SearchSearch for largest O(?)Sequential search for a target value O(?)Binary search for a target value O(?)

SortingBubble sort O(?)Quick sort O(?)