Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of...
Transcript of Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of...
![Page 1: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/1.jpg)
Fundamentals of the Analysis of Algorithm Efficiency (2)
DR. J IRABHORN CHAIWONGSAI
ดร.จิราพร ไชยวงศสายD E P A R T M E N T O F C O M P U T E R E N G I N E E R I N G
S C H O O L O F I N F O R M AT I O N A N D C O M M U N I C AT I O N T E C H N O L O G Y
U N I V E R S I T Y O F P H AYA O
![Page 2: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/2.jpg)
• สิ่งที่ใชแทนความสัมพันธของฟงกชันในแงของอัตราการเติบโต
• ประเมินประสิทธิภาพของอัลกอริทึม
• A way of comparing functions that ignores constant factors and small input sizes
• มี 3 ตัวดังนี้ θ, O, และ Ω Big oh (O)
Big theta (θ)
Big omega (Ω)
**ใชกับฟงกชันที่ใหคาไมติดลบ
Asymptotic Notations (สัญกรเชิงเสนกํากับ)
2
![Page 3: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/3.jpg)
• O(g(n)): class of functions f(n) that grow not faster than g(n)
• Θ(g(n)): class of functions f(n) that grow at same rate as g(n)
• Ω(g(n)): class of functions f(n) that grow at least as fast as g(n)
Asymptotic Notations
3
![Page 4: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/4.jpg)
Big-oh
Definition: f(n) is in O(g(n)) if order of growth of
f(n) ≤ order of growth of g(n) (within constant multiple),
i.e., there exist positive constant c and non-negative
integer n0 such that
f(n) ≤ c g(n) for every n ≥ n0
Example1:
• 10n is in O(n2) ? 10n ϵ O(n2)
• 5n+20 is in O(n) ? 5n+20 ϵ O(n)
4
![Page 5: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/5.jpg)
Big-oh (cont.)
5
![Page 6: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/6.jpg)
Big-oh (cont.)
6
![Page 7: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/7.jpg)
Definition: f(n) is in Ω(g(n)) if order of growth of f(n) ≥ order of growth of g(n) (within constant multiple),i.e., there exist positive constant c and non-negative integer n0 such that
f(n) ≥ c g(n) for every n ≥ n0
Example2:
• n3 is in Ω(n2) ?
• Show that (n/2)log(n/2) = Ω (nlog(n))
Big-omega
7
![Page 8: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/8.jpg)
Big-omega (cont.)
8
![Page 9: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/9.jpg)
Big-omega (cont.)
9
![Page 10: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/10.jpg)
Definition: f(n) is in θ(g(n)) if order of growth of f(n) is bounded above and below order of growth of g(n) (within constant multiple),i.e., there exist positive constant c1 and c2 and some non-negative integer n0 such that
c2 g(n) ≤ f(n) ≤ c1 g(n) for every n ≥ n0
Example3: Prove that ϵ θ (n log n)
Big-theta
1
2� � − 1
10
![Page 11: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/11.jpg)
Big-theta (cont.)
11
![Page 12: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/12.jpg)
• Transitivity:
12
![Page 13: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/13.jpg)
• Reflexivity
• Symmetry
))(()(
))(()(
))(()(
nfnf
nfOnf
nfnf
))(()())(()( nfngngnf
13
![Page 14: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/14.jpg)
Properties of Asymptotic Notations
• Transpose Symmetry
))(()())(()( nfngngOnf
14
![Page 15: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/15.jpg)
Properties of Asymptotic Notations
• เราสามารถลดรูปเพื่อใหอยูในรูปแบบที่งายขึ้น ดังนี้• กําหนดให f1(n) = O(g1(n)) และ f2(n) = O(g2(n))
f1(n) + f2(n) = O(g1(n) + g2(n)) f1(n) + f2(n) = O(max(g1(n), g2(n))) f1(n) • f2(n) = O(g1(n) • g2(n)) f1(n)k = O(g1(n)k)
• a
n
k
n
k
kfOkfO1 1
)())((
15
![Page 16: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/16.jpg)
• ใชกับสวนของฟงกชันที่มีอัตราการเติบโตที่ชากวา
• ใชในกรณีที่ไมทราบรายละเอียด หรือรายละเอียดนั้นยืดยาวมาก
• ใชเขียนเปนสวนหนึ่งของความสัมพันธเวียนเกิด เพื่อแทนประสิทธิภาพของอัลกอริทึม
16
![Page 17: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/17.jpg)
f(n) = 2n3+3n+7.5/n
f(n) = 2n3 + θ(n)
42 12012
1
2
1ln
nnnnH n
n
)/1(ln nnHn
17
![Page 18: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/18.jpg)
Orders of growth of some important functions
• ใน Asymptotic Notations นั้น
• ไมวาจะเปน log ฐานอะไร ก็จะมีอัตราการเติบโตเทากัน เชน log2(n) = θ(log1000(n))
• เลขชี้กําลังภายใน log ไมมีผลตออัตราการเติบโตเชน log(n2) = θ(log n0.1)
18
![Page 19: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/19.jpg)
• All logarithmic functions loga n belong to the same class θ(log n) no matter what the logarithm’s base a > 1 is
• All polynomials of the same degree k belong to the same class:
aknk + ak-1nk-1 + … + a0 (nk)
• Exponential functions an have different orders of growth for different a’s
• order log n < order n (>0) < order an < order n! < order nn
Orders of growth of some important functions
19
![Page 20: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/20.jpg)
Basic asymptotic efficiency classes
1 constant
log n logarithmic
n linear
n log n n-log-n or linearithmic
n2 quadratic
n3 cubic
2n exponential
n! factorial
20
![Page 21: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/21.jpg)
Time efficiency of non-recursive algorithms
General Plan for Analysis
• Decide on parameter n indicating input size
• Identify algorithm’s basic operation
• Determine worst, average, and best cases for input of size n
• Set up a sum for the number of times the basic operation is executed
21
![Page 22: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/22.jpg)
Useful summation formulas and rules
• liu1 = 1+1+ ⋯ +1 = u - l + 1
In particular, liu1 = n - 1 + 1 = n (n)
• 1in i = 1+2+ ⋯ +n = n(n+1)/2 n2/2 (n2)
• 1in i2 = 12+22+ ⋯ +n2 = n(n+1)(2n+1)/6 n3/3 (n3)
• 0in ai = 1 + a + ⋯ + an = (an+1 - 1)/(a - 1) for any a 1
In particular, 0in 2i = 20 + 21 + ⋯ + 2n = 2n+1 - 1 (2n )
22
![Page 23: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/23.jpg)
Mathematical Analysis of Non-recursive algorithms
Example 1: Maximum element
Problem Input size Basic operation
23
![Page 24: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/24.jpg)
Example 2: Element uniqueness problem
Problem Input size Basic operation
24
![Page 25: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/25.jpg)
Example 3: Matrix multiplication
Problem Input size Basic operation
25
![Page 26: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/26.jpg)
Example 4: Counting binary digits
It cannot be investigated the way the previous examples are.
Problem Input size Basic operation
26
![Page 27: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/27.jpg)
Time efficiency of recursive algorithms
Plan for Analysis of Recursive Algorithms
• Decide on a parameter indicating an input’s size.
• Identify the algorithm’s basic operation.
• Check whether the number of times the basic op. is executed may vary on different inputs of the same size. (If it may, the worst, average, and best cases must be investigated separately.)
• Set up a recurrence relation with an appropriate initial condition expressing the number of times the basic op. is executed.
• Solve the recurrence (or, at the very least, establish its solution’s order of growth) by backward substitutions or another method.
27
![Page 28: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/28.jpg)
Example 1: Recursive evaluation of n!
Definition: n ! = 1 2 … (n-1) n for n ≥ 1 and 0! = 1
Recursive definition of n!: F(n) = F(n-1) n for n ≥ 1 and
F(0) = 1
Size:
Basic operation:
Recurrence relation:
28
![Page 29: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/29.jpg)
Solving the recurrence for M(n)
M(n) = M(n-1) + 1, M(0) = 0
29
![Page 30: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/30.jpg)
Example 2: The Tower of Hanoi Puzzle
1
2
3
Recurrence for number of moves:
30
![Page 31: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/31.jpg)
Solving recurrence for number of moves
M(n) = 2M(n-1) + 1, M(1) = 1
31
![Page 32: Fundamentals of the Analysis of Algorithm Efficiency (2) · Fundamentals of the Analysis of Algorithm Efficiency (2) DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ](https://reader036.fdocuments.us/reader036/viewer/2022062607/605c64bd9ca4aa0d09798bb2/html5/thumbnails/32.jpg)
Example 3: Counting #bits
32