13-1 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii

ICS141: Discrete Mathematics for Computer Science I

Dept. Information & Computer Sci., University of Hawaii

Jan Stelovsky based on slides by Dr. Baek and Dr. Still

Originals by Dr. M. P. Frank and Dr. J.L. Gross Provided by McGraw-Hill

13-2 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii

Lecture 16

Chapter 3. The Fundamentals 3.3 Complexity of Algorithms 3.4 The Integers and Division

13-3 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii 3.3 Complexity of Algorithms n  An algorithm must always produce the correct answer,

and should be efficient. n  How can the efficiency of an algorithm be analyzed? n  The algorithmic complexity of a computation is, most

generally, a measure of how difficult it is to perform the computation.

n  That is, it measures some aspect of the cost of computation (in a general sense of “cost”). n  Amount of resources required to do a computation.

n  Some of the most common complexity measures: n  “Time” complexity: # of operations or steps required n  “Space” complexity: # of memory bits required

13-4 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii Complexity Depends on Input

n  Most algorithms have different complexities for inputs of different sizes. n  E.g. searching a long list typically takes

more time than searching a short one.

n  Therefore, complexity is usually expressed as a function of the input size. n  This function usually gives the complexity

for the worst-case input of any given length.

13-5 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii Worst-, Average- and Best-Case Complexity

n  A worst-case complexity measure estimates the time required for the most time consuming input of each size.

n  An average-case complexity measure estimates the average time required for input of each size.

n  An best-case complexity measure estimates the least time consuming input of each size.

13-6 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii Example 1: Max algorithm

n  Problem: Find the simplest form of the exact order of growth (Θ) of the worst-case time complexity of the max algorithm, assuming that each line of code takes some constant time every time it is executed (with possibly different times for different lines of code).

13-7 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii Complexity Analysis of max procedure max(a1, a2, …, an: integers) v := a1 t1 for i := 2 to n t2 if ai > v then v := ai t3 return v t4

n  First, what is an expression for the exact total worst-case time? (Not its order of growth.) n  t1: once n  t2: n – 1 + 1 times n  t3 (comparison): n – 1 times n  t4: once

13-8 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii Complexity Analysis (cont.) n  Worst-case time complexity

12)1()11(

)( 32

−=

−++−=

+=

nnn

ttnt

121)1()11(1

)( 4321

+=

+−++−+=

+++=

nnn

ttttnt

n  In terms of the number of comparisons made

n  Now, what is the simplest form of the exact (Θ) order of growth of t(n)? t(n) = Θ(n)

# of comparisons

13-9 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii Example 2: Linear Search n  In terms of the number of comparisons

procedure linear_search (x: integer, a1, a2, …, an: distinct integers)

i := 1 while (i ≤ n ∧ x ≠ ai) t11 & t12

i := i + 1 if i ≤ n then location := i t2 else location := 0 return location

13-10 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii Linear Search Analysis n  Worst case time complexity:

n  Best case:

n  Average case, if item is present:

)(221)1()( 21211

nnnntttnt

Θ=+=+++=

++=

)1(111)( 21211 Θ=++=++= tttnt

[ ] )(212/)1(2

)21(2)12(753)(

nnnnn

nnn

nnnt

Θ=+=++

=

++++=

+++++=

# of comparisons

13-11 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii Example 3: Binary Search

procedure binary_search (x:integer, a1, a2, …, an: distinct integers, sorted smallest to largest)

i := 1 j := n while i < j begin t1

m := ⎣(i + j)/2⎦ if x > am then i := m + 1 else j := m t2

end if x = ai then location := i else location := 0 t3 return location

Key question: How many loop iterations?

13-12 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii Binary Search Analysis n  Suppose that n is a power of 2, i.e., ∃k: n = 2k. n  Original range from i = 1 to j = n contains n items. n  Each iteration: Size j - i + 1 of range is cut in half.

n  Size decreases as 2k, 2k–1, 2k–2,… n  Loop terminates when size of range is 1 = 20 (i = j). n  Therefore, the number of iterations is: k = log2n

n  Even for n ≠ 2k (not an integral power of 2), time complexity is still the same.

)(log2log2221)1()(

22

321

nnkkktttnt

Θ=+=+=+++=

++=

13-13 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii Analysis of Sorting Algorithms

n  Check out Rosen 3.3 Example 5 and Example 6

for worst-case time complexity of bubble sort and insertion sort algorithms in terms of the number of comparisons made.

13-14 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii Bubble Sort Analysis

procedure bubble_sort (a1, a2, …, an: real numbers with n ≥ 2)

for i := 1 to n – 1 for j := 1 to n – i

if aj > aj+1 then interchange aj and aj+1 {a1, a2, …, an is in increasing order}

n  Worst-case complexity in terms of the number of comparisons: Θ(n2)

13-15 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii Insertion Sort procedure insertion_sort (a1, a2, …, an: real numbers; n ≥ 2)

for j := 2 to n begin i := 1 while aj > ai i := i + 1 m := aj for k := 0 to j – i – 1 aj-k := aj-k-1 ai := m end {a1, a2, …, an are sorted in increasing order}

n  Worst-case complexity in terms of the number of comparisons: Θ(n2)

13-16 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii Common Terminology for the Complexity of Algorithms

13-17 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii Computer Time Examples

#ops(n) n = 10 n = 106

log2 n 3.3 ns 19.9 ns n 10 ns 1 ms n log2 n 33 ns 19.9 ms n2 100 ns 16 m 40 s 2n 1.024 µs 10301,004.5 n! 3.63 ms Ouch!

(125 kB) (1.25 bytes)

n  Assume that time = 1 ns (10-9 second) per operation, problem size = n bits, and #ops is a function of n.

13-18 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii Review: Complexity n  Algorithmic complexity = cost of computation. n  Focus on time complexity for our course.

n  Although space & energy are also important. n  Characterize complexity as a function of input size:

n  Worst-case, best-case, or average-case. n  Use orders-of-growth notation to concisely

summarize the growth properties of complexity functions.

n  Need to know n  Names of specific orders of growth of complexity. n  How to analyze the order of growth of time

complexity for simple algorithms.

13-19 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii Tractable vs. Intractable n  A problem that is solvable using an algorithm

with at most polynomial time complexity is called tractable (or feasible). P is the set of all tractable problems.

n  A problem that cannot be solved using an algorithm with worst-case polynomial time complexity is called intractable (or infeasible).

n  Note that n1,000,000 is technically tractable, but really very hard. nlog log log n is technically intractable, but easy. Such cases are rare though.

13-20 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii P vs. NP n  NP is the set of problems for which there

exists a tractable algorithm for checking a proposed solution to tell if it is correct.

n  We know that P⊆NP, but the most famous unproven conjecture in computer science is that this inclusion is proper.

n  i.e., that P⊂NP rather than P=NP.

n  Whoever first proves this will be famous! (or disproves it!)

13-21 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii 3.4 The Integers and Division n  Of course, you already know what the integers are,

and what division is…

n  But: There are some specific notations, terminology, and theorems associated with these concepts which you may not know.

n  These form the basics of number theory.

n  Number theory is vital in many today important algorithms (hash functions, cryptography, digital signatures,…).

13-22 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii Divides, Factor, Multiple n  Let a,b ∈ Z with a ≠ 0.

n  Definition: a|b ⇔ “a divides b” ⇔ (∃c∈Z: b = ac) “There is an integer c such that c times a equals b.”

n  Example: 3|-12 (True), but 3|7 (False).

n  If a divides b, then we say a is a factor or a divisor of b, and b is a multiple of a.

n  E.g.: “b is even” ⇔ 2|b.

13-23 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii The Divides Relation n  Theorem: ∀a,b,c ∈ Z:

1. a|0 2. (a|b ∧ a|c) → a|(b + c) 3. a|b → a|bc ∀c ∈ Z 4. (a|b ∧ b|c) → a|c

n  Corollary: ∀a,b,c ∈ Z n  (a|b ∧ a|c) → a|(mb + nc), m,n ∈ Z

13-24 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii Proof of (2)

n  Show ∀a,b,c ∈ Z: (a|b ∧ a|c) → a|(b + c).

n  Let a, b, c be any integers such that a|b and a|c, and show that a|(b + c).

n  By definition of |, we know ∃s∈Z: b = as, and ∃t∈Z: c = at. Let s, t, be such integers.

n  Then b + c = as + at = a(s + t), so ∃u∈Z: b + c = au, namely u = s + t. Thus a|(b + c).

13-25 ICS 141: Discrete Mathematics I – Fall 2011

University of Hawaii University of Hawaii The Division “Algorithm” n  It’s really just a theorem, not an algorithm…

n  Only called an “algorithm” for historical reasons.

n  Theorem: For any integer dividend a and divisor d∈Z+, there are unique integers quotient q and remainder r∈N such that a = dq + r and 0 ≤ r < d. Formally, the theorem is:

∀a∈Z, d∈Z+: ∃!q,r∈Z: 0 ≤ r < d, a = dq + r

n  We can find q and r by: q = ⎣a/d⎦, r = a - dq