Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.
-
date post
20-Dec-2015 -
Category
Documents
-
view
223 -
download
2
Transcript of Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.
![Page 1: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/1.jpg)
eleg667/2001-f/Topic-1a 1
A Brief Review of
Algorithm Design and Analysis
![Page 2: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/2.jpg)
eleg667/2001-f/Topic-1a 2
Outline
Basic concepts; Case study; Analyzing algorithms Comparing algorithms Problem complexity
![Page 3: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/3.jpg)
eleg667/2001-f/Topic-1a 3
Algorithm - Definition
Sequence of steps to carry out a particular task
Examples: Recipe; Assembly instructions; ……
![Page 4: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/4.jpg)
eleg667/2001-f/Topic-1a 4
Characteristics of Algorithms
Algorithms are precise. Each step has a clearly defined meaning;
Algorithms are effective. The task is always done as required;
Algorithms have a finite number of steps; Algorithms must terminate.
![Page 5: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/5.jpg)
eleg667/2001-f/Topic-1a 5
Any algorithm can be stated using the following logic constructs
Sequence: steps must follow each other in a logical sequence;
Decision: there may be alternative steps that may be taken subject to a particular condition;
Repetition: certain steps may be repeated while, or until, a certain condition is true.
![Page 6: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/6.jpg)
eleg667/2001-f/Topic-1a 6
Why study algorithms?
A motivating example…
![Page 7: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/7.jpg)
eleg667/2001-f/Topic-1a 7
Example: sorting playing cards
Start with one card on the left hand and
the others cards facing down on the table;
while there are cards left on the table do
{ take one card from the table;
compare it with each of the cards
already in the hand, from left to right,
inserting it into the right position;
}
Sequence
Decision
Repetition
Insertion Sort
![Page 8: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/8.jpg)
eleg667/2001-f/Topic-1a 8
Expressing computer algorithms
It should be expressed in a language more precise, less ambiguous, and more compact than a “natural language” such as English;
Algorithms are usually written in a pseudocode language and later translated to a real programming language.
![Page 9: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/9.jpg)
eleg667/2001-f/Topic-1a 9
Insertion Sort – An Intuitive Picture
j Q K5
…
Q
i
1 1
5252
key
A A1
…
![Page 10: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/10.jpg)
eleg667/2001-f/Topic-1a 10
Insertion Sort in Pseudocode
Insertion-Sort(A) ; A is an array of numbers for j = 2 to length(A) key = A[j] i = j - 1 while i > 0 and A[i] > key A[i+1] = A[i] i = i - 1 A[i+1] = key
SequenceRepetition
Decision
![Page 11: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/11.jpg)
eleg667/2001-f/Topic-1a 11
Algorithms and Machine Models
Sequential algorithms and machine
models
Parallel algorithms and machine models
![Page 12: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/12.jpg)
eleg667/2001-f/Topic-1a 12
Analyzing algorithms
Estimate the running time of algorithms;
= F(Problem Size)
= F(Input Size)
= number of primitive operations used (add, multiply, compare etc)
![Page 13: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/13.jpg)
eleg667/2001-f/Topic-1a 13
Analysis for Insertion Sort
Insertion-Sort(A) Cost Times (Iterations) 1 for j = 2 to length(A) { c1
2 key = A[j] c2
3 i = j – 1 c3 4 while i > 0 and A[i] > key c4
5 A[i+1] = A[i] c5
6 i = i – 1 c6
7 A[i+1] = key c7 }
n
n - 1
n - 1
n - 1
![Page 14: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/14.jpg)
eleg667/2001-f/Topic-1a 14
Insertion Sort Analysis (cont.)
Best Case:
Array already sorted, tj = 1 for all j
![Page 15: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/15.jpg)
eleg667/2001-f/Topic-1a 15
Insertion Sort Analysis (cont.)
Worst Case:
Array in reverse order, tj = j for all j
Note that
![Page 16: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/16.jpg)
eleg667/2001-f/Topic-1a 16
Insertion Sort Analysis (cont.)
Average Case:
Check half of array on average,
tj = j/2 for all j
T(n) = a.n2 + bn + c (quadractic in n)
We are usually interested in the worst-case running timeWe are usually interested in the worst-case running time
![Page 17: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/17.jpg)
eleg667/2001-f/Topic-1a 18
Growth rates
constant growth rate: T(n) = c linear growth rate : T(n) = c*n logarithmic growth rate : T(n) = c*log n quadratic growth rate : T(n) = c*n2
exponential growth rate : T(n) = c*2n
![Page 18: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/18.jpg)
eleg667/2001-f/Topic-1a 19
Asymptotic Analysis
Ignoring constants in T(n) Analyzing T(n) as n "gets large"
The running time grows roughly on the order of
Notationally, =
" n
T n O n
3
3
"
( ) ( )
As grows larger, is MUCH large than , , and ,so it dominates
n n n n nT n
n 23 log( )
T n n n n n n( ) log 13 42 2 43 2Example:
The big-oh (O) Notation
![Page 19: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/19.jpg)
eleg667/2001-f/Topic-1a 20
Formally…
T(n) = O(f(n)) if there are constants c and n such that T(n) < c.f(n) when n n0
c.f(n)
T(N)
n0 n
f(n) is an upper bound for T(n)
![Page 20: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/20.jpg)
eleg667/2001-f/Topic-1a 21
O ( big-oh)
Describes an upper bound for the running time of an algorithm
Upper bounds for Insertion Sort running times:
•worst case: O(n2) T(n) = c1.n2 + c2.n + c3
•best case: O(n) T(n) = c1.n + c2
•average case: O(n2) T(n) = c1.n2 + c2.n + c3
Time Complexity
![Page 21: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/21.jpg)
eleg667/2001-f/Topic-1a 22
Some properties of the O notation
Fastest growing function dominates a sum O(f(n)+g(n)) is O(max{f(n), g(n)})
Product of upper bounds is upper bound for the product If f is O(g) and h is O(r) then fh is O(gr)
f is O(g) is transitive If f is O(g) and g is O(h) then f is O(h)
Hierarchy of functions O(1), O(logn), O(n1/2), O(nlogn), O(n2), O(2n), O(n!)
![Page 22: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/22.jpg)
eleg667/2001-f/Topic-1a 23
Times on a 1-billion-steps-per-second computer
![Page 23: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/23.jpg)
eleg667/2001-f/Topic-1a 24
Simple statement sequence s1; s2; …. ; sk
O(1) as long as k is constant
Simple loops for(i=0;i<n;i++) { s; } where s is O(1) Time complexity is n O(1) or O(n)
Analyzing an Algorithm
![Page 24: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/24.jpg)
eleg667/2001-f/Topic-1a 25
Analyzing an Algorithm (cont.)
Nested loops for(i=0;i<n;i++) for(j=0;j<n;j++) { s; }
Complexity is n O(n) or O(n2)
Recursion Search(n) if middle_element = key return it
else if it’s greater search_left(n/2) else search_right(n/2) Solve recurrence equation: T(n) = T(1) + T(n/2) Complexity is O(log2n)
![Page 25: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/25.jpg)
eleg667/2001-f/Topic-1a 26
Reasonable and Unreasonable Algorithms
Polynomial Time algorithms An algorithm is said to be polynomial if it is
O( nc ), c >1 Polynomial algorithms are said to be reasonable
They solve problems in reasonable times!
Exponential Time algorithms An algorithm is said to be exponential if it is
O( rn ), r > 1 Exponential algorithms are said to be unreasonable
![Page 26: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/26.jpg)
eleg667/2001-f/Topic-1a 27
Another example: Quick Sort
Uses recursion to repetitively divide list in 1/2 and sort the two halves
QuickSort partitions the list into a left and a right sublist, where all values in the left sublist are less than all values in the right sublist. Then the QuickSort is applied recursively to the sublists until each sublist has only one element.
![Page 27: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/27.jpg)
eleg667/2001-f/Topic-1a 28
Quick Sort (cont.)
9 1 25 4 15 4 1 9 25 15
becomes
4 1
25 15 becomes
becomes 1
4 15 25
1 4 15 25
![Page 28: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/28.jpg)
eleg667/2001-f/Topic-1a 29
Quick Sort (cont.)
QuickSort (list(0,n)):
If length (list) > 1 then
partition into 2 sublists from
0..split-1 and from split + 1..n
QuickSort (list (0,split-1))
QuickSort (list (split + 1, n)
![Page 29: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/29.jpg)
eleg667/2001-f/Topic-1a 30
1. Pick an array value, pivot.2. Use two indices, i and j.
a. Begin with i = left and j =right + 1b. As long as i is less than j do 3 steps:
i. Keep increasing i until we come to an element A[i] >= pivot.ii. Keep decreasing j until we come to an element A[j] <= pivot.iii. Swap A[i] and A[j].
Quick Sort – Partition phase
![Page 30: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/30.jpg)
eleg667/2001-f/Topic-1a 31
The Operation of Quicksort
![Page 31: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/31.jpg)
eleg667/2001-f/Topic-1a 32
Quick Sort – Complexity
Assume the list size (n) is a power of 2 and that pivot is always chosen such that it divides the list into two equal pieces
At each step we are cutting in half the size to be sorted
O(nlog2n)
![Page 32: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/32.jpg)
eleg667/2001-f/Topic-1a 33
Quick Sort demo…
![Page 33: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/33.jpg)
eleg667/2001-f/Topic-1a 34
Comparing the Sort Algorithms…
![Page 34: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/34.jpg)
eleg667/2001-f/Topic-1a 35
Problem complexity
problems
sort
. . .Insertion Shellsort QuicksortAlgorithms:
![Page 35: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/35.jpg)
eleg667/2001-f/Topic-1a 36
Definitions
A problem’s upper bound refers to the best that we know how to do. It is determined by the best algorithmic solution that we have found for a given problem.
Example: For the sorting problem O(n.logn) is the upper bound.
![Page 36: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/36.jpg)
eleg667/2001-f/Topic-1a 37
Definitions (cont.)
A problem’s lower bound refers to the best algorithmic solution that is theoretically possible.
Example: It can be proved that sorting an unsorted list cannot be done in less than O(n.log n), unless we make assumptions about the particular input data. Thus O(n.log n) is the lower bound for sorting problems.
![Page 37: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/37.jpg)
eleg667/2001-f/Topic-1a 38
Definitions (cont.)
Upper bounds tell us the best we’ve been able to do so far;
Lower bounds tell us the best we can hope to do.
For a given problem, if the upper and lower bounds are the same, then we refer to that problem as a closed problem.
![Page 38: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/38.jpg)
eleg667/2001-f/Topic-1a 39
Tractable and Intractable problems
Tractable problems have both upper and lower bounds polynomial – O(logn), O(n), O(n.logn), O(nk);
Intractable problems have both upper and lower bounds exponential - O(2n), O(n!), O(nn);
![Page 39: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/39.jpg)
eleg667/2001-f/Topic-1a 40
Problems that cross the line
We have found only exponential solutions (i.e. from the standpoint of our algorithms, it appears to be intractable);
We cannot prove the necessity of an exponential solution (i.e. from the standpoint of our proofs, we cannot say that it is intractable).
![Page 40: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/40.jpg)
eleg667/2001-f/Topic-1a 41
NP-complete problems
The upper bound suggests that the problem is intractable;
No proof exists that shows these problems are intractable;
Once a solution is found, it can be checked in polynomial time;
They are all reducible to one another;
![Page 41: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/41.jpg)
eleg667/2001-f/Topic-1a 42
Example
Traveling salesman: give a weighted graph (e.g., cities with distances between them) and some distance k (e.g., the maximum distance you want to travel), is there some tour that visits all the points (e.g., cities) and returns home such that distance k?
…
![Page 42: Eleg667/2001-f/Topic-1a 1 A Brief Review of Algorithm Design and Analysis.](https://reader036.fdocuments.us/reader036/viewer/2022062304/56649d455503460f94a22094/html5/thumbnails/42.jpg)
eleg667/2001-f/Topic-1a 43
P = NP ?
Win the Turing Award
and
get a $1 million prize!http://www.claymath.org/prize_problems/p_vs_np.htm