Merge Sort Mit
-
Upload
anshubansal2000 -
Category
Documents
-
view
224 -
download
0
Transcript of Merge Sort Mit
-
8/8/2019 Merge Sort Mit
1/55
Introduction to Algorithms6.046J
Lecture 1Prof. Shafi Goldwasser
Prof. Erik Demaine
-
8/8/2019 Merge Sort Mit
2/55
L1.2
Welcome toIntroduction to
Algorithms, Spring 2004
Handouts
1. Course Information
2. Course Calendar
3. Problem Set 1
4. Akra-Bazzi Handout
http://theory.lcs.mit.edu/classes/6.046/handouts/course-information.pdfhttp://theory.lcs.mit.edu/classes/6.046/handouts/calendar.pdfhttp://theory.lcs.mit.edu/classes/6.046/handouts/calendar.pdfhttp://theory.lcs.mit.edu/classes/6.046/handouts/course-information.pdf -
8/8/2019 Merge Sort Mit
3/55
L1.3
Course information
1. Staff
2. Prerequisites
3. Lectures & Recitations4. Handouts
5. Textbook (CLRS)
6. Website
8. Extra Help
9. Registration
10.Problem sets11.Describing algorithms
12.Grading policy
13.Collaboration policy
http://theory.lcs.mit.edu/classes/6.046/staff.htmlhttp://theory.lcs.mit.edu/classes/6.046/recitation.htmlhttp://theory.lcs.mit.edu/classes/6.046/handouts.htmlhttp://www.amazon.com/exec/obidos/ASIN/0262032937/qid=999563592/sr=1-3/ref=sc_b_3/002-9233126-5488026http://theory.lcs.mit.edu/classes/6.046/collaboration.htmlhttp://theory.lcs.mit.edu/classes/6.046/collaboration.htmlhttp://www.amazon.com/exec/obidos/ASIN/0262032937/qid=999563592/sr=1-3/ref=sc_b_3/002-9233126-5488026http://theory.lcs.mit.edu/classes/6.046/handouts.htmlhttp://theory.lcs.mit.edu/classes/6.046/recitation.htmlhttp://theory.lcs.mit.edu/classes/6.046/staff.html -
8/8/2019 Merge Sort Mit
4/55
L1.4
What is course about?
The theoretical study of design and
analysis of computer algorithms
Basic goals for an algorithm: always correct always terminates This class: performance Performance often draws the line betwee
what is possible and what is impossible.
-
8/8/2019 Merge Sort Mit
5/55
L1.5
Design and Analysis of Algorithms
Analysis:predict the cost of an algorithm in
terms of resources and performance
Design: design algorithms which minimize the
cost
-
8/8/2019 Merge Sort Mit
6/55
L1.6
Our Machine Model
Generic Random Access Machine (RAM)
Executes operations sequentially
Set of primitive operations: Arithmetic. Logical, Comparisons, Function calls
Simplifying assumption: all ops cost 1 unit
Eliminates dependence on the speed of our computer,otherwise impossible to verify and to compare
-
8/8/2019 Merge Sort Mit
7/55L1.7
The problem of sorting
Input: sequence a1, a2, , an of numbers.
Example:
Input: 8 2 4 9 3 6
Output: 2 3 4 6 8 9
Output: permutation a'1, a'2, , a'n suchthat a'1 a'2 a'n.
-
8/8/2019 Merge Sort Mit
8/55L1.8
Insertion sort
INSERTION-SORT (A, n) A[1 . . n]forj 2ton
do key A[j]
i j 1
while i > 0 andA[i] > keydo A[i+1] A[i]
i i 1
A[i+1] = key
pseudocode
i j
keysorted
A:1 n
-
8/8/2019 Merge Sort Mit
9/55L1.9
Example of insertion sort
8 2 4 9 3 6
-
8/8/2019 Merge Sort Mit
10/55L1.10
Example of insertion sort
8 2 4 9 3 6
-
8/8/2019 Merge Sort Mit
11/55L1.11
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
-
8/8/2019 Merge Sort Mit
12/55L1.12
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
-
8/8/2019 Merge Sort Mit
13/55L1.13
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
-
8/8/2019 Merge Sort Mit
14/55
L1.14
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
-
8/8/2019 Merge Sort Mit
15/55
L1.15
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
-
8/8/2019 Merge Sort Mit
16/55
L1.16
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
-
8/8/2019 Merge Sort Mit
17/55
L1.17
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
-
8/8/2019 Merge Sort Mit
18/55
L1.18
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
-
8/8/2019 Merge Sort Mit
19/55
L1.19
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
2 3 4 6 8 9 done
-
8/8/2019 Merge Sort Mit
20/55
L1.20
Running time
The running time depends on the input: analready sorted sequence is easier to sort.
Major Simplifying Convention:Parameterize the running time by the size ofthe input, since short sequences are easier tosort than long ones.
TA(n) = time of A on length n inputs Generally, we seek upper bounds on the
running time, to have a guarantee of
performance.
-
8/8/2019 Merge Sort Mit
21/55
L1.21
Kinds of analyses
Worst-case: (usually) T(n) = maximum time of algorithm
on any input of size n.
Average-case: (sometimes) T(n) = expected time of algorithmover all inputs of size n.
Need assumption of statistical
distribution of inputs.Best-case: (NEVER)
Cheat with a slow algorithm thatworks fast onsome input.
-
8/8/2019 Merge Sort Mit
22/55
L1.22
Machine-independent time
What is insertion sorts worst-case time?
BIG IDEAS:
Ignore machine dependent constants,otherwise impossible to verify and to compare algorithms
Look atgrowthofT(n) as n .
Asymptotic AnalysisAsymptotic Analysis
-
8/8/2019 Merge Sort Mit
23/55
L1.23
-notation
Drop low-order terms; ignore leading constants.
Example: 3n3 + 90n2 5n + 6046 = (n3)
DEF: (g(n)) = {f(n): there exist positive constants c1, c2, and
n0 such that 0
c1g(n)
f(n)
c2g(n)for all n n0}
Basic manipulations:
-
8/8/2019 Merge Sort Mit
24/55
L1.24
Asymptotic performance
n
T(n)
n0
.
Asymptotic analysis is a
useful tool to help tostructure our thinkingtoward better algorithm
We shouldnt ignore
asymptoticallyslower algorithms,
however. Real-world design
situations often call for a
When n gets large enough, a (n2) algorithmalways beats a (n3) algorithm.
-
8/8/2019 Merge Sort Mit
25/55
L1.25
Insertion sort analysis
Worst case: Input reverse sorted.
( )=
==n
j
njnT2
2)()(
Average case: All permutations equally likely.
( )=
==n
j
njnT2
2)2/()(
Is insertion sort a fast sorting algorithm? Moderately so, for small n.Not at all, for large n.
[arithmetic series]
-
8/8/2019 Merge Sort Mit
26/55
L1.26
Example 2: Integer
Multiplication
Let X = A B and Y = C D where A,B,C
and D are n/2 bit integers
Simple Method: XY = (2n/2A+B)(2n/2C+D)
Running Time Recurrence
T(n) < 4T(n/2) + 100n
Solution T(n) = (n2)
-
8/8/2019 Merge Sort Mit
27/55
L1.27
Better Integer Multiplication
Let X = A B and Y = C D where A,B,C and D
are n/2 bit integers
Karatsuba:XY = (2n/2+2n)AC+2n/2(A-B)(C-D) + (2n/2+1) BD
Running Time Recurrence
T(n) < 3T(n/2) + 100n
Solution: (n) = O(n log 3)
-
8/8/2019 Merge Sort Mit
28/55
L1.28
Example 3:Merge sort
MERGE-SORTA[1 . . n]
1. Ifn = 1, done.
2. Recursively sortA[ 1 . . n/2 ] andA[ n/2 +1 . . n] .
3. Merge the 2 sorted lists.
Key subroutine:MERGE
-
8/8/2019 Merge Sort Mit
29/55
L1.29
Merging two sorted arrays
20
13
7
2
12
11
9
1
-
8/8/2019 Merge Sort Mit
30/55
L1.30
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
-
8/8/2019 Merge Sort Mit
31/55
L1.31
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
-
8/8/2019 Merge Sort Mit
32/55
L1.32
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
-
8/8/2019 Merge Sort Mit
33/55
L1.33
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
20
13
7
12
11
9
-
8/8/2019 Merge Sort Mit
34/55
L1.34
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
20
13
7
12
11
9
7
-
8/8/2019 Merge Sort Mit
35/55
L1.35
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
20
13
7
12
11
9
7
20
13
12
11
9
-
8/8/2019 Merge Sort Mit
36/55
L1.36
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
20
13
7
12
11
9
7
20
13
12
11
9
9
-
8/8/2019 Merge Sort Mit
37/55
L1.37
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
20
13
7
12
11
9
7
20
13
12
11
9
9
20
13
12
11
-
8/8/2019 Merge Sort Mit
38/55
L1.38
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
20
13
7
12
11
9
7
20
13
12
11
9
9
20
13
12
11
11
-
8/8/2019 Merge Sort Mit
39/55
L1.39
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
20
13
7
12
11
9
7
20
13
12
11
9
9
20
13
12
11
11
20
13
12
-
8/8/2019 Merge Sort Mit
40/55
L1.40
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
20
13
7
12
11
9
7
20
13
12
11
9
9
20
13
12
11
11
20
13
12
12
-
8/8/2019 Merge Sort Mit
41/55
L1.41
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
20
13
7
12
11
9
7
20
13
12
11
9
9
20
13
12
11
11
20
13
12
12
Time = (n) to merge a totalofn elements (linear time).
-
8/8/2019 Merge Sort Mit
42/55
L1.42
Analyzing merge sort
MERGE-SORTA[1 . . n]
1. Ifn = 1, done.
2. Recursively sortA[ 1 . . n/2 ] andA[ n/2 +1 . . n] .
3. Mergethe 2 sorted lists
T(n)
(1)
2T(n/2)
(n)
Sloppiness: Should be T( n/2 ) +T( n/2 ) , but it turns out not to matter
asymptotically.
-
8/8/2019 Merge Sort Mit
43/55
L1.43
Recurrence for merge sort
T(n) = (1) ifn = 1;
2T(n/2) + (n) ifn > 1.
We shall usually omit stating the basecase when T(n) = (1) for sufficientlysmall n, but only when it has no effect onthe asymptotic solution to the recurrence.
Lecture 2 provides several ways to find agood upper bound on T(n).
-
8/8/2019 Merge Sort Mit
44/55
L1.44
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
-
8/8/2019 Merge Sort Mit
45/55
L1.45
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
T(n)
-
8/8/2019 Merge Sort Mit
46/55
L1.46
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
T(n/2) T(n/2)
cn
-
8/8/2019 Merge Sort Mit
47/55
L1.47
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
T(n/4) T(n/4) T(n/4) T(n/4)
cn/2 cn/2
-
8/8/2019 Merge Sort Mit
48/55
L1.48
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/4 cn/4 cn/4 cn/4
cn/2 cn/2
(1)
-
8/8/2019 Merge Sort Mit
49/55
L1.49
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/4 cn/4 cn/4 cn/4
cn/2 cn/2
(1)
h = lg n
-
8/8/2019 Merge Sort Mit
50/55
L1.50
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/4 cn/4 cn/4 cn/4
cn/2 cn/2
(1)
h = lg n
cn
-
8/8/2019 Merge Sort Mit
51/55
L1.51
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/4 cn/4 cn/4 cn/4
cn/2 cn/2
(1)
h = lg n
cn
cn
-
8/8/2019 Merge Sort Mit
52/55
L1.52
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/4 cn/4 cn/4 cn/4
cn/2 cn/2
(1)
h = lg n
cn
cn
cn
-
8/8/2019 Merge Sort Mit
53/55
L1.53
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/4 cn/4 cn/4 cn/4
cn/2 cn/2
(1)
h = lg n
cn
cn
cn
#leaves = n (n)
-
8/8/2019 Merge Sort Mit
54/55
L1.54
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/4 cn/4 cn/4 cn/4
cn/2 cn/2
(1)
h = lg n
cn
cn
cn
#leaves = n (n)
Total = (n lg n)
-
8/8/2019 Merge Sort Mit
55/55
Conclusions
(n lg n) grows more slowly than (n2).
Therefore, merge sort asymptoticallybeats insertion sort in the worst case.
In practice, merge sort beats insertionsort forn > 30 or so.