Complexity (Running Time) Introduction to Computing Science and Programming I.

Complexity Complexity (Running Time)(Running Time)

Introduction to Computing Introduction to Computing Science and Programming IScience and Programming I

ComplexityComplexity

An important part of Computer An important part of Computer Science that deals with how difficult Science that deals with how difficult (complex) different problems are and (complex) different problems are and how long it will take to solve them.how long it will take to solve them.

Complexity is the term generally Complexity is the term generally used to describe this area.used to describe this area.

For the most part we’ll be talking For the most part we’ll be talking about how to determine the running-about how to determine the running-time of different algorithms.time of different algorithms.


Essentially what we’ll be doing is Essentially what we’ll be doing is examining and comparing algorithms that examining and comparing algorithms that are used to solve the same problem.are used to solve the same problem.

If one algorithm has a shorter running-If one algorithm has a shorter running-time, it is generally the better algorithm. time, it is generally the better algorithm. (There are exceptions, but we won’t get (There are exceptions, but we won’t get into those)into those)

An example of a problem that has a wide An example of a problem that has a wide array of algorithms for solving is that of array of algorithms for solving is that of sorting data.sorting data.


Why is this important?Why is this important? If you’re trying to write a program that solves a If you’re trying to write a program that solves a

complex problem and deals with a large amount complex problem and deals with a large amount of data, using a bad algorithm can cause a of data, using a bad algorithm can cause a slowdown of many orders of magnitude slowdown of many orders of magnitude (multiples of 10)(multiples of 10)

When dealing with simple problems and small When dealing with simple problems and small amounts of data as we have for the most part in amounts of data as we have for the most part in this course, it doesn’t always really matter, no this course, it doesn’t always really matter, no matter how badly you write your program, it matter how badly you write your program, it probably won’t take long to finish.probably won’t take long to finish.

Running TimeRunning Time

We’re going to start by looking at We’re going to start by looking at two algorithms for a guessing game.two algorithms for a guessing game.

The guessing game works as follows.The guessing game works as follows. The computer tells the user to guess a The computer tells the user to guess a

number between 1 and 100number between 1 and 100 The computer tries to guess the user’s The computer tries to guess the user’s

number, repeating until it guesses correctlynumber, repeating until it guesses correctly


Here’s a simple program that will Here’s a simple program that will workwork

print "Think of a number between 1 and 100."print "Think of a number between 1 and 100."

guess = 1guess = 1

answer = ""answer = ""

while answer != "equal":while answer != "equal":

answer = raw_input("Is your number equal to or not equal to answer = raw_input("Is your number equal to or not equal to " \" \

+ str(guess) + "?")+ str(guess) + "?")

guess = guess + 1guess = guess + 1


How many guesses might the How many guesses might the computer need to make before computer need to make before getting the correct answer?getting the correct answer?

Now let’s look at another algorithm Now let’s look at another algorithm for the game. This algorithm is for the game. This algorithm is discussed in some detail in the guide.discussed in some detail in the guide.

Running TimeRunning Timeprint "Think of a number from 1 to 100."smallest = 1largest = 100answer = "“ while answer != "equal": guess = (smallest + largest) / 2 answer = raw_input( "Is your number ’more’, ’less’," \ " or ’equal’ to " + str(guess) + "? " ) if answer == "more": smallest = guess + 1 elif answer == "less": largest = guess - 1

print smallest, largest

print "I got it!"

Running TimeRunning Time How many guesses might the computer have to make with How many guesses might the computer have to make with

this algorithm?this algorithm? The range of possibilities is printed out each time the loop The range of possibilities is printed out each time the loop

repeats. Look at how the values change if the user’s guess is 2.repeats. Look at how the values change if the user’s guess is 2. 1-1001-100 100 possibilities100 possibilities 1-491-49 4949 1-241-24 2424 1-111-11 1111 1-51-5 55 1-21-2 22 2-22-2 11

The number of possibilities is (approximately) halved each time. The number of possibilities is (approximately) halved each time. 1_________________________________________________1001_________________________________________________1001_____________________491_____________________491___________241___________241______111______11… …


With the first algorithm we could With the first algorithm we could have as many as 100 guesses.have as many as 100 guesses.

With the second algorithm halving With the second algorithm halving the number of possibilities after the number of possibilities after every guess, there’ll never be more every guess, there’ll never be more than 7 guesses.than 7 guesses.

Running TimeRunning Time Let’s generalize these algorithms to a game where Let’s generalize these algorithms to a game where

the user guesses a number between 1 and n.the user guesses a number between 1 and n. The first algorithm will need up to n guesses.The first algorithm will need up to n guesses. The second algorithm will need up to The second algorithm will need up to loglog22

nn steps. steps. represent the ceiling operator, represent the ceiling operator, 1.21.2 = 2 etc. = 2 etc. loglog22 x is the inverse of 2 x is the inverse of 2x x

log log22 8 = 3 8 = 3 loglog22 1 = 0 1 = 0 loglog22 16 = 4 16 = 4 loglog22 1024 = 10 1024 = 10 loglog22 1048576 = 20 1048576 = 20

This illustrates how much more powerful the second This illustrates how much more powerful the second algorithm isalgorithm is


The running time for an algorithm The running time for an algorithm tells you approximately how many tells you approximately how many steps the algorithm will take to steps the algorithm will take to complete.complete.

We will just look at worst case We will just look at worst case running times though examining running times though examining average or best case running times average or best case running times can also be useful.can also be useful.

Running TImeRunning TIme Repeated LettersRepeated Letters Check a string for any repeated letters (not necessarily Check a string for any repeated letters (not necessarily

consecutive)consecutive)

write “Enter the word:”read wordset counter to 0for all letters letter a in the word, do this: for all letters letter b to the right of letter a, do this: if letter a is equal to letter b then set counter to counter+1

if counter > 0 then write “There are repetitions”else write “No repetitions”


The two loops compare every The two loops compare every character with all the characters that character with all the characters that come after it.come after it. If the string is n characters long this works If the string is n characters long this works

out to n(n-1)/2 = nout to n(n-1)/2 = n22/2 - n/2 steps/2 - n/2 steps We’ll simplify this to say that the We’ll simplify this to say that the

algorithm has a running time of algorithm has a running time of approximately napproximately n22 steps steps

We’ll discuss why the n/2 and the ½ coefficient We’ll discuss why the n/2 and the ½ coefficient in front of the nin front of the n22 aren’t too important later aren’t too important later


Subset SumSubset Sum Given a set of numbers and a target, determine if some Given a set of numbers and a target, determine if some

subset sums to the target valuesubset sums to the target value

for every subset in the list: set sum to the sum of this subset if sum is equal to target : answer “yes” and quitanswer “no”


How many steps will the algorithm How many steps will the algorithm take?take? The loop repeats for every possible subset.The loop repeats for every possible subset. Each element is either in, or not in, each Each element is either in, or not in, each

subset so if there are n elements there are subset so if there are n elements there are 22

nn subsets giving about 2 subsets giving about 2nn

steps. steps. We actually get about n2We actually get about n2nn

total since finding total since finding the sum can take n steps, but the exponential the sum can take n steps, but the exponential part the important part.part the important part.

Exponential algorithms are terribly slow.Exponential algorithms are terribly slow.


Assuming the loop is repeated 1000 Assuming the loop is repeated 1000 times per second here are running times times per second here are running times for the algorithm.for the algorithm.


So far we’ve just talked about “steps” in So far we’ve just talked about “steps” in general.general.

To be specific you could just count the To be specific you could just count the number of lines of code that would be number of lines of code that would be executed.executed.

However, we’re generally just concerned However, we’re generally just concerned with the parts of the program that will with the parts of the program that will repeat the highest number of times (loops) repeat the highest number of times (loops) and just with how many times they repeat, and just with how many times they repeat, not how many lines of code are within them.not how many lines of code are within them.


The innermost loop will be repeated The innermost loop will be repeated most. For nested loops multiply the most. For nested loops multiply the number of times the outer loop number of times the outer loop repeats by the number of times the repeats by the number of times the inner loop repeats.inner loop repeats.

code …code …for i from 1 to n:for i from 1 to n: for j from 1 to n:for j from 1 to n: codecode

The outer loop repeats n times, so The outer loop repeats n times, so does the inner which gives n * n = ndoes the inner which gives n * n = n22


Using algorithms with good running times Using algorithms with good running times becomes extremely important when a program becomes extremely important when a program may deal with large inputs. may deal with large inputs.

As to why we throw out smaller terms and As to why we throw out smaller terms and leading coefficients.leading coefficients. Any running time involving a more significant term Any running time involving a more significant term

(2(2nn > n > n2 2 > n) will always become larger than a > n) will always become larger than a

running time only involving smaller terms as n running time only involving smaller terms as n becomes large.becomes large.

This is true independent of coefficients for the termsThis is true independent of coefficients for the terms

Complexity (Running Time) Introduction to Computing Science and Programming I.

Documents

Transcript of Complexity (Running Time) Introduction to Computing Science and Programming I.