9/26/2019

1

Algorithms:

Design and Analysis

Week 02

2

I. The Role of Algorithms in Computing

– What are algorithms?

– Why is the study of algorithms worthwhile?

– What is the role of algorithms relative to other technologies used in computers?

• Algorithms

• Algorithms as a technology

3

Computational problems• A computational problem specifies an input-output

relationship

– What does the input look like?

– What should the output be for each input?

• Example:

– Input: an integer number N

– Output: Is the number prime?

• Example:

– Input: A list of names of people

– Output: The same list sorted alphabetically

• Example:

– Input: A picture in digital format

– Output: An English description of what the picture shows 4

Algorithms• An algorithm is an exact specification of how to solve a

computational problem• Takes a set of values, as input and

• produces a value, or set of values, as output

– May be specified

• In English

• As a computer program

• As a pseudo-code

– Data structures

– Methods of organizing data

• Program = algorithms + Coding (data structures)

5

• An algorithm must specify every step completely, so a computer can implement it without any further “understanding”

• An algorithm must work for all possible inputs of the problem.

• Algorithms must be:

– Correct: For each input produce an appropriate output

– Efficient: run as quickly as possible, and use as little memory as possible

• There can be many different algorithms for each computational problem.

6

Algorithms

9/26/2019

2

AlgorithmsAn algorithm is any well-defined computational procedurethat takes some value, or set of values, as input and produces some value, or set of values, as output.

Input OutputSequence of

computational steps

Computer

(source code-Executable)

algorithm

7

Example :The Sorting Problem

Input: A sequence of n numbers [A1, A2, … , An].

Output: A permutation or reordering [A'1, A'2, … , A'n ] of the input

sequence such that A'1 A'2 … A'n .

An instance of the Sorting Problem:

Input: A sequence of 6 number [31, 41, 59, 26, 41, 58].

Expected output for given instance:

Expected

Output: The permutation of the input [26, 31, 41, 41, 58 , 59].

8

Insertion Sort

The main idea …

9

Describing Algorithms

• Algorithms can be implemented in any programming language

• Usually we use “pseudo-code” to describe algorithms

Testing whether input N is prime:

For j = 2 .. N-1

If j|N

Output “N is composite” and halt

Output “N is prime”

10

Correct Algorithms

• An algorithm is said to be correct if, for every input instance, it halts with the correct output. We say that a correct algorithm solves the given computational problem.

• An incorrect algorithm might not halt at all on some input instances, or it might halt with an answer other than the desired one.

• Incorrect algorithms can sometimes be useful, if their error rate can be controlled.

11

What kinds of problems are solved by algorithms?

• We are given a road map on which the distance between each pair of adjacent intersections is marked, and our goal is to determine the shortest route from one intersection to another.

• We are given a sequence A1, A2, ..., An of n matrices, and we wish to determine their product A1. A2. … . An.

• We are given an equation ax ≡ b (mod n), where a, b, and n are integers, and we wish to find all the integers x, modulo n, that satisfy the equation.

12

9/26/2019

3

• Teach you techniques of algorithm design and analysis so that

– you can develop algorithms on your own,

– show that they give the correct answer, and

– understand their efficiency.

13

Designing Algorithms Designing Algorithms

• Several techniques/patterns for designing algorithms exist

– Incremental approach: builds the solution one component at a time

– Divide-and-conquer approach: breaks original problem into several smaller instances of the same problem• Results in recursive algorithms

• Easy to analyze complexity using proven techniques

14

Choosing algorithms

Ex: Fibonacci sequence is defined as follows.

F(0) = 0, F(1) = 1, and

F(n) = F(n-1) + F(n-2) for n > 1.

Write an algorithm to computer F(n).

There many algorithms, but what is the most efficient ?

15

Algorithms 1 and 2 for Fibonaccifunction fib1(n){

if n < 2 then return n;else return fib1(n-1) + fib1(n-2);

}

function fib2(n){

i = 1; j = 0;for k = 1 to n do { j = i + j; i = j - i;

}return j;

}16

Algorithm 3 for Fibonacci

function fib3(n){i = 1; j = 0; k = 0; h = 1;(while n>0) do {

if (n odd) then do { t = j*h; j = i*h + j*k +t;i = i*k +t;

}t = h^2;h = 2*k*h + t;k = k^2 + t; // pow(k,2)n = n / 2;

}return j;

} 17

How fast will your program run?• The running time of your program will depend upon:

– The algorithm

– The input

– Your implementation of the algorithm in a programming language

– The compiler you use

– The OS on your computer

– Your computer hardware

– Maybe other things: temperature outside; other programs on your computer; …

• Our Motivation: analyze the running time of an algorithm as a function of only simple parameters of the input.

18

9/26/2019

4

Analyzing algorithms

• Analyzing an algorithm: for an input size,– measure memory (space)

– measure computational time (running time).

• Input size: depends on the problem:– Sorting: number of items in the input; array size,… O(n)

– Big integer (multiplying, …): number of bits to represent the input in binary notation O(log n)

– Two number: input of a graph can be O(n,m), number of vertices and number of edges.

• Running time:– A constant amount of time is required to execute each line

– each execution of the ith line takes time c_i , where c_i is a constant.

19

Basic idea: counting operations• Each algorithm performs a sequence of basic operations:

– Arithmetic: (low + high)/2

– Comparison: if ( x > 0 ) …

– Assignment: temp = x

– Branching: while ( true ) { … }

– …

• Idea: count the number of basic operations performed on the input.

• Difficulties:

– Which operations are basic?

– Not all operations take the same amount of time.

– Operations take different times with different hardware or compilers 20

Example:Insertion sortEfficient algorithm for sorting a small number of

elements:– We start with an empty left hand and the cards face down on the table. – We then remove one card at a time from the table and insert it into the

correct position in the left hand. To find the correct position for a card, we compare it with each of the cards already in the hand, from right to left.

INSERTION-SORT(A) 1. for j ← 2 to length[A]2. do key ← A[j]3. Insert A[j] into the sorted sequence A[1.. j - 1].4. i←j-1 5. while i > 0 and A[i]>key6. do A[ i + 1 ] ← A[ i ] 7. i ← i-18. A[ i+1 ]←key

21

Example

22

Proof of the correctness of Insertion sort

• We use loop invariants to help us understand why an algorithm is correct.

• We must show three things about a loop invariant:

– Initialization: It is true prior to the first iteration of the loop.

– Maintenance: If it is true before an iteration of the loop, it remains true before the next iteration.

– Termination: When the loop terminates, the invariant gives us a useful property that helps show that the algorithm is correct.

23

n

(sorted)

MERGE

Merge Sort Strategy

• Divide stage: Split the n-element sequence into two subsequences of n/2 elements each

• Conquer stage: Recursively sort the two subsequences

• Combine stage: Merge the two sorted subsequences into one sorted sequence (the solution)

n

(unsorted)

n/2

(unsorted)

n/2

(unsorted)

MERGE SORT MERGE SORT

n/2

(sorted)

n/2

(sorted)

24

9/26/2019

5

Analyzing divide-and-conquer algorithms

• Divide: D(n) = Θ(1).

• Conquer: solve two subproblems, each of size n/2, which contributes 2T (n/2) to the running time.

• Combine: the MERGE procedure on an n-element subarray takes time Θ(n), so C(n) = Θ(n).

T(n) = θ(1) if n = 1

2 T(n/2) + θ(n) if n >1

25

Example:The divide-and-conquer approach of Designing algorithms

26

EXAMPLE - 1

Problem: Write an algorithm in pseudocode that

finds the average of two numbers

Solution:

• AverageOfTwo

• Input: Two numbers

1.Add the two numbers

2.Divide the result by 2

3.Return the result by step 2

• End 27

EXAMPLE-2Problem : Write an algorithm to change a numeric

grade to a pass/no pass grade.

• Solution: Pass/NoPassGrade

• Input: One number

1. if (the number is greater than or equal to 70)then

– Set the grade to “pass”

else

– Set the grade to “fail”

End if

2. Return the grade

• End28

EXAMPLE-3Problem: Write an algorithm to change a numeric grade to a letter grade.

Solution:

Input: One number

1. if (the number is between 90 and 100, inclusive)

then Set the grade to “AA”

End if

2. if (the number is between 80 and 89, inclusive)

then Set the grade to “BA”

End if

3. if (the number is between 70 and 79, inclusive)

then Set the grade to “CC”

End if

4. if (the number is between 60 and 69, inclusive)

then Set the grade to “DD”

End if

5. If (the number is less than 60)

then Set the grade to “FF”

End if

6. Return the grade

End29

EXAMPLE-4• Problem: Write an algorithm to find the largest of a set of numbers.

You do not know the number of numbers.

• Solution: FindLargest

Input: A list of positive integers

1. Set Largest to 0

2. while (more integers)if (the integer is greater than Largest)

thenSet largest to the value of the integerEnd if

End while

3. Return Largest

• End30

9/26/2019

6

EXAMPLE-5• Problem: Write an algorithm to find the largest of 1000 numbers.

• Solution: FindLargest

• Input: 1000 positive integers

1. Set Largest to 0

2. Set Counter to 0

3. while (Counter less than 1000)if (the integer is greater than Largest)

thenSet Largest to the value of the integerEnd if

Increment CounterEnd while

4. Return Largest

• End31

Reference

32

• "Introduction to Algorithms (2nd edition)",

– Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, Clifford Stein, (2001)

• The McGraw-Hill Companies,