Insertion Sort

CSE 331Section 2James Daly

Insertion Sort

• Basic idea: keep a sublist sorted, then add new items into the correct place to keep it sorted

1 3 4 6Sorted Part

25Next

Insertion Sort Algorithm

• InsertionSort(A):for j = 2 to len(A):

key ← A[j]i ← j – 1while i > 0 and A[i] > key:

A[i + 1] ← A[i]i ← i – 1

A[i + 1] ← key

Total running time: )1()1()1()1()1()( 72

62

52

4321

nctctctcncncncnTn

jj

n

jj

n

jj

Running Times

• Min value of T(n) [best case]• tj = 1

• Max value of T(n) [worst case]• tj = j

ban

ccccnccccc

nccncncncnTn

j

)()(

)1(1)1()1()(

743274321

72

4321

12

)1(432

22

nnnjt

n

j

n

jj

cbnan

ccccncccc

cccnccc

ncnn

cnn

cnn

cncncncnT

2

74327654

3212654

7654321

)()222

(2

)(

)1(2

)1(

2

)1()1

2

)1(()1()1()(

Worst-case and average-case analysis

• The longest running time for any input of size n = worst case• Eg. 5 3 2 1 0 for insertion sort

• The upper-bound on the running time for any input

Worst-case and average-case analysis

• The worst case occurs often• Eg. Database search: failed to find a match

• The average case is often roughly as bad as the worst case• Eg. Insertion sort: roughly half elements on

either side of key

Some caveats

• List.length()• Multiplying two matrices

Simplifications / Approximations

n 3/2 n2 3/2 n2 + 7/2 n – 4 % Difference

10 150 181 17%

50 3,750 3,921 4.4%

100 45,000 15,436 2.3%

500 375,000 376,746 0.5%

Big-Oh Notation (Asymptotic upper bound)

• f(n) = O(g(n)) iff there exists• Constant c > 0• Constant n0

• Such that f(n) ≤ c g(n) for all n ≥ n0

• Eventually, always smaller than g(n)

Examples

Examples

Big-Oh Notation

• To show that f(n) = O(g(n)), you need to provide c and n0 and show f(n) ≤ c g(n) for all n ≥ n0

Example

• Example: n2 + 7n + 5 = O(n2)• Method 1:

• f(n) = n2 + 7n + 5 ≤ n2 + 7 n2 + 5n2 = 13n2 when n ≥ 1

• Thus f(n) ≤ 13n2 when n ≥ 1

Example

• Example: n2 + 7n + 5 = O(n2)• Method 2:

• f(n) ≤ c g(n) → f(n) - c g(n) ≤ 0 when n ≥ n0

• Let c = 2 and n0 = 8

• n2 + 7n + 5 – 2n2 = 0 when n ≈ 7.65

• Be sure to check the derivative is negative• -n + 7 < 0 when n ≥ 8

Big-Omega (Asymptotic lower bound)

• f(n) = Ω(g(n)) iff there exists• Constant c > 0• Constant n0

• Such that c g(n) ≤ f(n) for all n ≥ n0

• Eventually, always bigger than g(n)• Reverse of O(g(n))

Big-Theta (Asymptotic tight bound)

• f(n) = Θ(g(n)) iff• f(n) = O(n) and• f(n) = Ω(g(n))

Examples

Examples

Tricks for proving f(n) = O(g(n))

• Observe the highest order term• Highest term in f(n) must be ≤ that of g(n)

• Try fixing c first, then find n0.

• Let a be the coefficient of the highest term in f(n) Try to let c = a, a+1, etc.

• Or let c = sum of all coefficients

Selection Sort

• Another sorting method• Find the smallest unsorted item, then

move it to the front• Then find the next smallest, and so on

SelectionSort(A)

for i = 1 to len(A):minj ← ifor j = i + 1 to len(A):

if A[j] < A[minj]:minj ← j

Swap(A, i, minj)