Post on 17-Dec-2015
3
Starting example
Sorting: We have a list of numbers . We want to sort them in the increasing order.
4
An algorithm: merge sort
Merge sort: Cut it into two halves with equal size.
Suppose divides for simplicity. Suppose the two halves are sorted: Merge them.
Use two pointers, one for each half, to scan them, during which course do the appropriate merge.
How to sort each of the two halves? Recursively.
5
Complexity?
Suppose this algorithm takes time for an input with numbers.
Thus each of the two halves takes time. The merging?
Scanning elements, an time operation needed for each.
Total amount of time: .
7
A general method for designing algorithm: Divide and conquer Breaking the problem into subproblems
that are themselves smaller instances of the same type of problem
Recursively solving these subproblems
Appropriately combining their answers
8
Complexity
Running time on an input of size : Break problem into subproblems, each of
the same size . In general, is not necessarily equal to .
Time to recursively solve each subproblem: Time for breaking problem (into
subproblems) and combining the answers:
9
Master theorem
, , and are all constants. Then
Proof in textbook. Not required. But you need to know how to apply it.
12
Selection
Problem: Given a list of numbers, find the -th smallest.
We can sort the list, which needs . Can we do better, say, linear time? After all, sorting gives a lot more information
than we requested. Not always a waste: consider dynamic
programming where solutions to subproblems are also produced.
13
Idea of divide and conquer
Divide the numbers into 3 parts
, , Depending on the size of each part, we know
which part the -th element lies in.
Then search in that part.
Question: Which to choose?
14
Pivot
Suppose we use a number in the given list as a pivot.
As said, we divide the list into three parts. : Those numbers smaller than : Those numbers equal to : Those numbers larger than
15
After the partition
The division is simple: just scan the list and put elements into the corresponding part. time.
To select the -th smallest value, it becomes
Complexity?
16
Divide and conquer
Note: though there are two subproblems (of sizes and ), we need to solve only one of them. Compare: in quicksort, we need to sort both
substrings! Complexity:
A new issue: and are not determined Depends on the pivot.
19
The similarity to quicksort tells us: a random pivot performs well It’s away from either end by with const. prob.
To be more precise, it’s in with probability . And in this case, the recursion becomes
21
Thus we can use the following simple strategy: Pick a random pivot, do the recursion
Each random pivot falls in w/ prob. . . It is enough to get good pivots to make the
problem size to drop to 1. Thus .
23
Matrix multiplication
Recall: the product of two matrices is another matrix.
Question: how fast can we multiply two matrices?
Recall: : the entry in the matrix . Similar for ,
24
This takes multiplications (of numbers).
For a long time, people thought this was the best possible.
Until Straussen came up with the following.
25
If we break the matrix into blocks
Then the product is just block multiplication
8 matrix multiplications of dimension Plus additions.
27
However, Straussen observed that we can actually use only 7 (instead of 8) multiplications of matrices with dimension .
30
Best in theory? .
Conjecture: !
In practice? People still use the algorithm most of the time. (For example, in matlab.) It’s simple and robust.
32
Multiplication of polynomials
Many applications need to multiply two polynomials. e.g. signal processing.
A degree- polynomial is
The degree is at most . (The degree is exactly if .) The summation of two polynomials is easy
---------------------------------------------------------------- which still has degree at most .
33
Multiplication
Multiplication of two polynomials: a different story.
) --------------------------------------------------------------- where (convolution)
Try an example: The multiplication can have degree . If we directly use this formula to multiply two
polynomials, it takes time.
35
What determines/represents a polynomial? Let’s switch to degree-() for later simplicity. We already used coefficients to represent a
polynomial. Coefficient representation
Other ways? Yes: By giving values on (distinct) points.
Point-value representation. [Fact] A degree-() polynomial is uniquely determined
by values on distinct points.
36
Reason
We have coefficients to determine
We know values on distinct points.
How to get the coefficients? Just solve the system of equations:
37
In matrix form:
The matrix is Vandermonde matrix, which is invertible. Thus it has a unique solution for the coefficients .
unknowns
38
Advantage of point-value representation? It makes the multiplication extremely easy:
Though is hard For any fixed point , is simply multiplication of two
numbers. So given
and , it’s really easy to get
. time.
Thus we have the following interesting idea for polynomial multiplication…
39
Go to an easy world and come back HK used to have many industries. Later found mainland has less expensive labor. So:
moved the companies to mainland, did the work there, and then shipped the products back to HK to sell
This is worth doing if: it’s cheaper in mainland, and the traveling/shipping is not expensive either. which turned out to be true.
41
Traveling
From coefficient representation to point-value representation: Evaluate two polynomials both on points.
--- evaluation. From point-value representation to coefficient
representation: Compute one polynomial back from point values.
--- interpolation. Both can be done in time.
43
Number of points
How many points we need to evaluate on? Since we later want to reconstruct the
product polynomial , which has degree . So (point,value) pairs are enough to
recover . We’ll evaluate and on points , and get .
points are enough. We use for convenience. Then recover from .
44
Now evaluation on one point needs time, so evaluations on points need time.
Too bad. We want . Important fact: cost(evaluations on points) can be
cheaper than cost(evaluation on point). If we choose the points carefully.
And it turns out that the chosen points also make the (later) interpolation easier.
This powerful tool is called Fourier Transform.
Complex roots of unity
1 has a complex root , satisfing . Discrete Fourier Transform (DFT):
where Try now.
Note: evaluates polynomial on .
So our evaluation task is just DFT.
46
All the essences are here…
We want to evaluate and on , , …, . Suppose for simplicity that is a power of . For , define two new polynomials:
Then Divide and Conquer.
So we can evaluate by evaluating and on .
48
So we evaluate only points (instead of ). Recursion:
: To compute for . Rewriting recursion:
. Applying master theorem: . Since , the cost of evaluating is , as claimed.
49
Interpolation
How about interpolation? i.e., to get the coefficients by point values.
Almost the same process!
.
50
DFT matrix: . What’s the inverse of the matrix ?
Pretty much the same matrixreplace with .
…and then divide the whole matrix by for normalization.
52
Or
DFT matrix is unitary. Namely
: transpose. : complex conjugate DFT is symmetric: . So taking complex conjugate () gives
inverse.