CS 146: Data Structures and Algorithms July 7 Class Meeting Department of Computer Science San Jose...

CS 146: Data Structures and AlgorithmsJuly 7 Class Meeting

Department of Computer ScienceSan Jose State University

Summer 2015Instructor: Ron Mak

www.cs.sjsu.edu/~mak

http://www.cs.sjsu.edu/~mak

Computer Science Dept.Summer 2015: July 7

CS 146: Data Structures and Algorithms© R. Mak

2

Midterm Solutions

1. Suppose variable p contains a reference to a node in a singly linked list. Each list node has a data field and a next field that refers to the next node in the list, and you can access both these fields. You are told that p does not refer to the last node of the list. You do not have a reference to the head node of the list. Describe in words or in Java pseudocode how you can delete the node referred to by p and yet maintain the integrity of the list.



3

Midterm Solutions, cont’d

Problem: You cannot simply delete node C because you can’t access node B in order to get it to point to node D.

Solution: Copy the data field of node D into the data field

of node C. Now point node C to node D’s successor (node E).

p

A B C D E F



4


2. Use Big-Oh notation to describe the number of times with respect to N that the variable sum is incremented in of each of the following program fragments.

sum = 0;for (n = N; n > 0; n /= 2) { for (i = 0; i < n; i++) { sum++; }}

Answer:The inner loop goes N + N/2 + N/4 + N/8 + … times.

Therefore, O(N).



5


sum = 0;for (i = 1; i < N; i *= 2) { for (j = 0; j < N; j++) { sum++; }}

Answer: The inner loop always goes N times.The outer loop goeslog2 N times.

sum = 0;for (i = 1; i < N; i *= 2) { for (j = 0; j < i; j++) { sum++; }}

Answer: The inner loop goes 1 + 2 + 4 + 8 + … + N times. Therefore, O(N)

Therefore, O(N log2 N)



6


d.

sum = 0;for (i = 0; i < N; i++) { for (j = 0; j < i*i; j++) { for (k = 0; k < j; k++) { sum++; } }}

sum = 0;for (i = 1; i < N; i++) { for (j = 1; j < i*i; j++) { if (j%i == 0) { for (k = 0; k < j; k++) { sum++; } } }}

Answer: Without the ifstatement, the answerwould be O(N5) as above.But the if statement is true only i times as j loops i2. times.

Answer: j can be as large as i2, which could be as large as N2. k can be as large as j,which can be as large as N2.

Therefore, the count is proportional to NN2N2, which is O(N5)

Therefore, O(N4).



7


3. Write in Java or pseudocode a method that, when passed the root of an arbitrary binary tree, converts the tree to its mirror and returns the root of the mirror:

Solution: Do a preorder traversal of the tree. Swap the left and right children of each node that you visit.



8


4. Draw step-by-step the insertion of the node values 37, 54, 65, 49, 39, 28, and 59 into an initially empty AVL tree. Identify any rotations along the way.

Inserting node 37:

37

Inserting node 54:

37 \ \ 54

Inserting node 65:

Single left rotation: 54

54 /\ / \ 37 65



9

Midterm Solutions, cont’dInserting node 49:

54 /\ -- -- / \ 37 65 \ \ 49

Inserting node 39:

Double right-left rotation: 39

54 /\ -- -- / \ 39 65 /\ / \ 37 49

Inserting node 28:

Single right rotation: 39

39 /\ -- -- / \ 37 54 / /\ / / \ 28 49 65

Inserting node 59:

39 /\ ------ ------ / \ 37 54 / /\ -- -- -- / / \ 28 49 65 / / 59



10


5. Consider an N-by-N matrix of numbers.a. Suppose the numbers in the matrix are in a random

order. Using Big-Oh notation in terms of N, how long would it take to search the matrix for a particular value?

b. Now suppose the numbers in the matrix are in sequential order. Describe in words an algorithm to search for a particular value, and how long would it take in terms of N?

Answer: Up to N2 numbers have to be checked, so O(N2).

Answer: Start with the upper left corner of the matrix, and go either down or to the right while checking each number. You will go down at most N rows and to the right at most N columns to check at most 2N numbers. Therefore, O(N).



11


6. You are given two unsorted arrays of numbers, each with N elements.

a. Describe in words an algorithm that successively finds the smallest N pairs of elements, with each pair containing an element from each array. The first pair contains the smallest value from each array, the second pair contains the second smallest value from each array, etc.

Answer: First convert each array into a heap. Then successively remove the first element of each heap to form the pairs.



12


b. Using Big-Oh in terms of N, how long will your algorithm take to find all N pairs?

Answer:

It takes O(N) time to build a heap from an array (method buildHeap()).

It takes O(log N) to remove a single value from a heap,therefore O(N log N) to remove N values.

Altogether: O(N log N) to find all N pairs.



13


7. Consider the following recurrence relation for n that are exact powers of 2:

a. What is T(4)?

T(16)?

{T(n) =

2 if n = 22T(n/2) + n if n = 2k, for k > 1

Answer: T(4) = 2T(2) + 4 = 4 + 4 = 8

Answer: T(8) = 2T(4) + 8 = 16 + 8 = 24 T(16) = 2T(8) + 16 = 48 + 16 = 64



14


b. Use mathematical induction to prove that the solution of the recurrence relation is T(n) = n(log2 n).

{T(n) =

2 if n = 22T(n/2) + n if n = 2k, for k > 1

Base case: n = 2 T(2) = 2(log22) = 2(1) = 2 is true



15


Induction hypothesis: Assume it’s true for n = 2k-1:

{T(n) =

2 if n = 22T(n/2) + n if n = 2k, for k > 1

On both sides:Multiply by 2 and add 2k

Therefore, if then

On the left:Apply the recurrence relation

On the right: factor

Prove thatT(n) = n(log2 n)



16

A Solution to Assignment #2

get(i), k = 100

N ArrayList LinkedList IndexedList 100 0 0 0 1000 0 5 3 10000 0 47 4 100000 0 4071 4

add(x,i), k = 100




17

A Solution to Assignment #2, cont’d

remove(i), k = 100


Mixture, k = 100




18


This is why I chose k = 100 for my other tests.

Indexed list, N = 100000

Operation k=2 k=10 k=100 k=1000 k=10000 get(i) 2 2 4 38 536 add(x,i) 1883 435 51 66 579remove(i) 2095 438 48 38 309 mixture 1390 359 48 46 363



19

public Integer get(int i){ return getNode(i).data();}

private Node getNode(int i){ xPosition = i/k; xDelta = i - k*xPosition; if (xDelta > k/2) { if (xPosition < index.size()-1) { xPosition++; xDelta -= k; // now negative to move backward } } Node node = index.get(xPosition); if (xDelta > 0) { int d = xDelta; while (d-- > 0) node = node.next(); } else { int d = xDelta; while (d++ < 0) node = node.prev(); } return node;}

Get the position in the index arrayand compute how far we are from the “indexed node” to the desired node.

If more than halfway forward from theindexed node, use the next indexed nodeand go backward instead to the desired node.

Get the indexed node and move forward or backward to the desired node.



20

A Solution to Assignment #2, cont’dpublic void add(int i, Integer data){ Node nodeAfter = getNode(i); Node newNode = new Node(data); insert(newNode, nodeAfter); // Initialize the index array if this is // the first hybrid list node. if (N == 1) { index.add(newNode); xResidual = 0; } // Update the index array by pointing backward one node // following the insertion point. else if (xDelta > 0) { updateIndexBackward(xPosition + 1); } else { updateIndexBackward(xPosition); }}

Inserted after the indexed node.

Inserted before the indexed node.



21


public Integer remove(int i){ Node node = getNode(i); delete(node); // Update the index array by pointing forward one node // following the insertion point. if (xDelta > 0) { updateIndexForward(xPosition + 1); } else { updateIndexForward(xPosition); } return node.data();}

Deleted after the indexed node.

Deleted before the indexed node.



22


private void updateIndexBackward(int pos){ for (int j = pos; j < index.size(); j++) { index.set(j, index.get(j).prev()); } // If the residual is now k, append a new index array element. if (++xResidual == k) { index.add(tail.prev()); xResidual = 0; }}

Point each index nodeto its predecessor.

Append a new index nodeif the residual reaches k.

After each insertion, update the index array by pointing each indexed node to its predecessor. May need to append a new index array element.



23


After each deletion, update the index array by pointing each indexed node to its successor. May need to remove the last index array element.

private void updateIndexForward(int pos){ for (int j = pos; j < index.size(); j++) { index.set(j, index.get(j).next()); } // If the residual was 0, delete the last index array element. if (xResidual-- == 0) { index.remove(index.get(index.size()-1)); xResidual = k-1; }}

Point each index nodeto its successor.

Remove the last index nodeif the residual reached 0.



24

Break



25

Sorting

Convert unordered data into ordered data.

Several algorithms for sorting. Some simple, some very elegant. Many opportunities for analysis.

Sorting involves comparing two values and possibly swapping (or moving) them.

Goal: The most efficient algorithm possible. Minimize the numbers of comparisons and swaps.



26

Insertion Sort

One of the simplest and intuitive algorithms. The way you would manually sort a deck of cards.

Make N–1 passes over the list of data. For pass p = 1 through N–1, the algorithm

guarantees that the data in positions 0 through p–1 are already sorted.



27

Insertion Sort

The inner for loop terminates quickly if the tmp value does not need to be inserted too far into the sorted part. The entire sort finishes quickly if the data is nearly sorted: O(N).

public static <AnyType extends Comparable<? super AnyType>> void insertionSort(AnyType[] a) { int j;

for (int p = 1; p < a.length; p++) { AnyType tmp = a[p];

for (j = p; j > 0 && tmp.compareTo(a[j-1]) < 0; j--) { a[j] = a[j-1]; }

a[j] = tmp; }}

Slide values in the sorted part of the listone to the right to make room for a newmember of the sorted part.

Does this value belong in the sorted part?



28

Analysis of Insertion Sort

Nested loops, each can take N iterations. Therefore, insertion sort is O(N2).

An inversion is an ordered pair of data (a[i], a[j]) from the list where the indices i < j but the values a[i] > a[j]. The two values are out of order.

Assume we’re sorting from lowest to highest. Example: The list 34 8 64 51 32 21

has the inversions (34,8), (34,32), (34,21), (64,51), (64, 32), (64,21), (51,32), (51,21), and (32,21).



29

Analysis of Insertion Sort, cont’d

Swapping two adjacent values that out of order removes exactly one inversion from the list. A sorted array has no inversions.

We can compute bounds on the average running time of insertion sort by computing the average number of inversions in a permutation of values in a list.



30


What is the average number of inversions?

Suppose list L contains N values.

It has N(N-1)/2 distinct ordered pairs. Each ordered pair is a potential inversion.

Assume half of the ordered pairs is an inversion.

Therefore, the average number of inversions in each list is N(N-1)/4.



31


Conclusion: Any sorting algorithm that limits comparisons to adjacent items will cost at least N(N-1)/4 = Ω(N2) in the average case.

Each swap removes only one inversion, so Ω(N2) swaps are required.



32

Insertion Sort: Some Observations

Insertion sort is fast if the array is nearly sorted. Otherwise, it must move more values in the list.

If we can swap non-adjacent values, we may be able to remove more than one inversion at a time.



33

Insertion Sort: Some Observations, cont’d

If we can get the array “nearly sorted” as soon as possible, insertion sort can finish the job quickly.

These observations led Donald Shell to invent the Shellsort algorithm in 1959.



34

Shellsort

Like insertion sort, except we compare values that are h elements apart in the list. h diminishes after completing a pass,

for example, 5, 3, and 1.

The final value of h must be 1, so the final pass is a regular insertion sort.

The previous passes get the array “nearly sorted” quickly.



35

Shellsort, cont’d

After each pass, the array is said to be hk-sorted.

Examples: 5-sorted, 3-sorted, etc.



36

Shellsort

The effect is to perform an insertion sort on hk independent subarrays during each pass.

Shellsort is also called a diminishing increment sort.

The choice of h values affects how long the sort takes.

Don Knuth suggests reversing the sequence 1, 4, 13, 40, ...(start at 1 and for each subsequent pass, multiply the previous by 3 and add 1).



37

Analysis of Shellsort

Shellsort is difficult to analyze.

Worse case running time is Θ(N2).

Average case using Knuth’s sequence is O(N1.5), which is better than insertion sort’s O(N2).



38

Shellsort

public static <AnyType extends Comparable<? super AnyType>> void shellsort(AnyType[] a) { int j;

for (int h = a.length/2; h > 0; h /= 2) { for (int i = h; i < a.length; i++) { AnyType tmp = a[i];

for (j = i; j >= h && tmp.compareTo(a[j-h]) < 0; j -= h) { a[j] = a[j-h]; }

a[j] = tmp; } }}

Use the (suboptimal) sequence for h which starts at half the list length and is halved for each subsequent pass.



39

Assignment #4

This assignment will give you a closer look at the insertion sort and Shellsort algorithms.

Write a program that will sort integer arrays of sizes 100 1000 10,000 and 100,000 using both sorting algorithms.

Both algorithms should sort the same array, so make a copy of it each time before sorting.



40

Assignment #4, cont’d

The algorithms should sort, in the lowest to highest (increasing) order:

An unsorted array of unique random numbers in random order.

An array of unique numbers that’s already sorted in increasing order. These values do not have to be random. Your algorithms should not know ahead of time that the array was already sorted.



41


An array of unique numbers that’s already sorted in reverse (decreasing) order. These values do not have to be random. Your algorithms should not know ahead of time that the array was already sorted in reverse.

An array of all zeroes. Your algorithms should not know ahead of time that all the array elements contain the same value.



42


Do Shellsort twice: First use the suboptimal sequence for h

which starts at half the length of the array and is halved for each pass.

Then use Knuth’s interval sequence 1, 4, 13, 40, ... (reversed) for values of h. The first value of h should be as large as possible but under half the array length.

Verify that your arrays are properly sorted!

Generalize your program enough to accommodate other sorting algorithms in the future.



43

Assignment #4, cont’d For each sort, your program should output:

How much time it took in milliseconds. How many comparisons it made between two values. How many moves it made of the values.

Examples: If you shift five values over one position each,

that’s five moves. Two values exchanging places (a swap) is two moves.

You can output these results in a single table.



44


You may choose a partner to work with you on this assignment. Both of you will receive the same score.

Email your answers to [email protected] Subject line:

CS 146 Assignment #4: Your Name(s) CC your partner when you email your solution.

Due Monday, July 13 at 11:59 PM.

mailto:[email protected]



45

Assignment #4 Sample Output ===== Unsorted Random Unique =====

N = 100

ALGORITHM MOVES COMPARES MILLISECONDS Insertion sort 2,723 2,724 1 Shellsort suboptimal 590 826 0 Shellsort Knuth 627 718 0

N = 1,000

ALGORITHM MOVES COMPARES MILLISECONDS Insertion sort 247,291 247,293 9 Shellsort suboptimal 11,857 15,339 1 Shellsort Knuth 13,322 14,922 1

N = 10,000

ALGORITHM MOVES COMPARES MILLISECONDS Insertion sort 24,986,430 24,986,435 74 Shellsort suboptimal 219,431 275,513 10 Shellsort Knuth 206,966 228,206 10

N = 100,000

ALGORITHM MOVES COMPARES MILLISECONDS Insertion sort 2,502,386,612 2,502,386,607 10,227 Shellsort suboptimal 3,718,361 4,394,436 55 Shellsort Knuth 3,668,296 3,928,932 48

===== Sorted Unique =====

...

CS 146: Data Structures and Algorithms July 7 Class Meeting Department of Computer Science San Jose...

Documents

Transcript of CS 146: Data Structures and Algorithms July 7 Class Meeting Department of Computer Science San Jose...