ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday,...

1

ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION

CS16: Introduction to Data Structures & Algorithms

Tuesday, February 3, 2015

2

Announcements 2/3/15

• Sections have started• Note that rooms this week may be different than

last week!

• If you didn’t receive your graded HW1 and a grade report via email, let us know

• Homework 2 due Thursday 11:59pm• Seamcarve due Monday 11:59pm• Thursday is Python Lab part 2

• Please go to the room you went to last week


3

Outline

1) Recurrence Review

2) Recurrence Relations

3) Plug ‘n’ Chug

4) Induction

5) Strong vs. Weak Induction


4

Recursion Review

• Recursion is way of decomposing problems into smaller, simpler sub-tasks that are similar to the original.

• Thus, each sub-task can be solved by applying a similar technique.

• The whole problem is solved by combining the solutions to the smaller problems.

• Requires a BASE CASE (A case simple enough to solve without recursion) to end recursion.


5

Recursion Example

def factorial(n): if n == 1: return 1 else: return n * factorial(n-1)

• Compute the factorial of a number, n.


6

Recursion Simulation

• Calculate 3 factorial• This is a call to factorial(3), sowe put factorial(3) on the call stack


def factorial(n): if n == 1: return 1 else: return n*factorial(n-1)

Call Stack

7


• Calculate 3 factorial• This is a call to factorial(3), sowe put factorial(3) on the call stack



Call Stack

factorial(3)

8


• n != 1, so we returnn*factorial(n-1), which includes a call to factorial(2).

• Remember, the call to factorial(3)has not returned yet, so it is stillon the call stack!



Call Stack

factorial(3)

9


• n != 1, so we returnn*factorial(n-1), which includes a call to factorial(2).

• Remember, the call to factorial(3)has not returned yet, so it is stillon the call stack!def factorial(n): if n == 1: return 1 else: return n*factorial(n-1)

Call Stack

factorial(3)

factorial(2)


10


• n still != 1, so we returnn*factorial(n-1), which includes a call to factorial(1).

• Neither factorial(2) nor factorial(3)has returned at this point!


Call Stack

factorial(3)

factorial(2)


11


• n still != 1, so we returnn*factorial(n-1), which includes a call to factorial(1).

• Neither factorial(2) nor factorial(3) has returned at this point!


Call Stack

factorial(3)

factorial(2)

factorial(1)


12


• Now n =1, so we return 1!• This is not a recursive call, so factorial(1) returns, and we takeit off of the call stack.


Call Stack

factorial(3)

factorial(2)

factorial(1)


13


• Now n =1, so we return 1!• This is not a recursive call, so factorial(1) returns, and we takeit off of the call stack.


Call Stack

factorial(3)

factorial(2)


14

Recursion Simulation• Now factorial(2) is at the top of the

call stack, so we return to where wewere in factorial(2).

• So we return 2*factorial(1), whichwe now know is 2*1, so factorial(2)returns 2 and is removed from the call stack!def factorial(n): if n == 1: return 1 else: return n*factorial(n-1)

Call Stack

factorial(3)

factorial(2)


15


call stack, so we return to where wewere in factorial(2).

• So we return 2*factorial(1), whichwe now know is 2*1, so factorial(2)returns 2 and is removed from the call stack!def factorial(n): if n == 1: return 1 else: return n*factorial(n-1)

Call Stack

factorial(3)


16


call stack, so we’re back in factorial(3).• Return 3*factorial(2), which we now know

is 3*2. • Factorial(3) returns 6 and removes itself

from the call stack.


Call Stack

factorial(3)


17


call stack, so we’re back in factorial(3).• Return 3*factorial(2), which we now know

is 3*2. • Factorial(3) returns 6 and removes itself

from the call stack.


Call Stack


18


• Now our call stack is empty,and we know that factorial(3) is 6!


Call Stack


19

Determining Running Times

• For many algorithms, one can investigate code to see how size impacts running time• For seamcarve algorithm, each pixel is only evaluated in the for loops a constant number of times, and therefore the algorithm is O(n)

• But for some algorithms, counting can be tricky!• For example, determining the running time of a recursive algorithm can be complex


20

Recurrence Relations

• Recurrence relations describe the runtime of a recursive algorithm in two parts:• base case

• how many instructions are executed in the base case of the recursive function, usually when n=0 or n=1

• general case • how many instructions are executed in the recursive

case


21

Recursive array_max# Returns the maximum value of the first n elements in the array# Example: array_max([5,1,9,2], 4) 9

def array_max(array, n): if n == 1: return array[0] else: return max(array[n-1], array_max(array, n-1))

• T(n), the number of instructions executed as a function of the input size, can be expressed as a recurrence relation• T(1) = c0

• constant number of operations to compare and return

• T(n) = c1 + T(n-1)• constant number to do compare and calculate max• plus the operations of the recursive call

• But how do we get a big-O out of this?


22

Recursive array_maxdef array_max(array, n): if n == 1: return array[0] else: return max(array[n-1], array_max(array, n-1))

array_max([5,1,9,2], 4)= max(2, array_max([5,1,9], 3))= max(2, max(9, array_max([5,1], 2)))= max(2, max(9, max(1, array_max([5], 1))))= max(2, max(9, max(1, 5)))= max(2, max(9, 5))= max(2, 9)= 9


Note: We only show the portion of the list we are working with, because we decrease n by 1 each time. Actually shrinking the list by 1 each time would require creating a copy, which is linear, and thus not optimal.

23

Plug ‘n’ Chug• Given the recurrence relation (the base case T(1) and the general case T(n)),

we can “plug ‘n’ chug” to find a recurrence solution• A recurrence solution is the “answer” to a recurrence relation: it turns the

recursive definition into a simpler, closed-form mathematical expression

• Simplifying, we get: T(n) = c1n – c1 + c0• We can see that T(n) is a linear function, which makes array_max O(n)

T(1) = c0T(2) = c1 + T(2-1) = c1 + T(1) = c1 + c0T(3) = c1 + T(3-1) = c1 + T(2) = c1 + c1 +c0 = 2c1 + c0T(4) = c1 + T(4-1) = c1 + T(3) = c1 + 2c1 + c0 = 3c1 + c0 ⋮T(n) = c1 + T(n-1) = (n-1)c1 + c0


24

How can we be sure?

• We just used the plug ‘n’ chug method to posit that the runtime of array_max was O(n)• But are we sure? Can we prove it?

• We observed a pattern in order to reach a recurrence solution, but a “pattern” isn’t a formal proof

• In order to prove that the algorithm has the runtime we think it does, we’ll need to prove that our recurrence solution is correct!


25

Induction• Induction is a method of mathematical proof used to establish that a

statement is true for all positive integers• first demonstrate the statement’s truth for a single positive integer• second prove that, given the assumption that the statement is true

for an arbitrary input, the statement is true for the next input

• In other words:• If want to prove something for all n• Prove for:

• n = 1• n = k + 1 if true for n = k

• Celebrate!

• See the handout on website for another example.


26

Induction ExampleClaim: The solution for T(1) = c0, T(n) = c1 + T(n-1) is T(n) = (n-1)c1 + c0Base Case• Prove the base case by plugging in n=1 to the recurrence solution:

• T(1) = (1-1)c1 + c0 = c0• We’re given that T(1) = c0 in the base case of the recurrence relation, so recurrence

solution works for the base case!

Inductive Assumption• Assume the solution works for k

• T(k) = (k-1)c1 + c0Inductive Step• Show that it works for k+1 given the assumption that it works for k -- ultimately, we want to

show that T(k+1) = (k)c1 + c0• T(k+1) = c1 + T(k) according to recurrence relation• T(k+1) = c1 + (k-1)c1 + c0 substituting inductive assumption• T(k+1) = (k)c1 + c0 simplifying

Conclusion• Because we’ve proven our claim for the base case n = 1 and shown

truth for n = k implies truth for n = k+1, therefore T(n) = (n-1)c1 + c0 for all positive integers n.


27

Induction Example 2• Given:

• A(n) is the number of subsets of a set of size n• A(1) = 2

• empty set and set of one element

• A(n) = A(n-1) + A(n-1) = 2A(n-1)• all the sets in n-1 case with and without nth element included

• By plug ‘n’ chug: 2, 4, 8, 16, 32, …, A(n)• Looks like the recurrence solution is A(n) = 2n• Can we prove it?

• Statement to prove:• A(n) = 2n is the recurrence solution for A(n) = 2A(n-1)


28

Induction Example 2• Given:

• A(n) is the number of subsets of a set of size n• A(1) = 2 (empty set and set of one element)• A(n) = 2A(n-1)

Claim: A(n) = 2n is the recurrence solution for A(n) = 2A(n-1)• Proof:

• base case A(1) = 21 = 2 given• inductive assumption A(k) = 2k assume for n = k• inductive step A(k+1) = 2A(k) givenA(k+1) = 2 * 2k substituting

assumption A(k+1) = 2k+1 simplifying

• conclusion: Because we’ve proved our claim for n = 1 and shown that n = k implies n = k+1, therefore A(n) = 2n for all positive integers


29

Induction Example 3• Prove P(n) is true for all positive integers, n:

• Base case P(1):

• Assume P(k) is true:

Sometimes called the “predicate”


30

Induction Example 3 (2)• Prove

“implies”

Plug in inductive assumption

Multiply by 2/2

Factor out (k+1)

Start with definition of Σ

AWESOME!


31

Strong vs. Weak Induction

• This induction example is weak induction• Logic of the inductive step (n = k +1) relies only on

the previous step (n = k)

• This differs from strong induction• Logic of the inductive step relies on a “stronger”

assumption of more than one value (n ≤ k)• Sometimes makes inductive step easier

• The strong and weak refer to the assumptions you have to make to complete the proof, not the strength of the proof

• For much of CS16, weak induction is sufficient


32

Readings

• Read the “Induction Handout” on the website! (in the Docs section)• http://cs.brown.edu/courses/cs016/docs/induction.pdf


http://cs.brown.edu/courses/cs016/docs/induction.pdf

ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday,...

Documents

Transcript of ANALYSIS, PLUG ‘N’ CHUG, & INDUCTION CS16: Introduction to Data Structures & Algorithms Tuesday,...