31 May 2003IOI/ACM Training1 IOI/ACM Training Session 3 Min-Yen KAN These notes available at...

31 May 2003 IOI/ACM Training 1

IOI/ACM Training Session 3

Min-Yen KAN

These notes available at http://www.comp.nus.edu.sg/~kanmy/talks/


Topics to be covered today

1. Computer algebra2. Large-number computations3. Miscellaneous sorting4. Invariants and Modular arithmetic5. Factorization and Prime numbers

6. Dijkstra’s algorithm (if time permits)


Computer Algebra – Part I

Used to manipulate mathematical formulae in symbolic way

Common operations: Add Simplify Expand

These notes mostly from http://www.math.wpi.edu/IQP/BVCalcHist/calc5.html.


Data structures

How to represent x5 + 2x – 1? Dense (array): 15 04 03 02 21 -10

Sparse (linked list): [1,5][2,1][-1,0]

What about arbitrary expressions?

Solution: use an expression tree(e.g. a+b*c)


Simplification: Introduction

But CA systems have to deal with equivalency between different forms: (x-2) (x+2) X2 – 4

So an idea is to push all equations to a canonical form. Like Maple “simplify”

Question: how do we implement simplify?


Simplify - Transforming Negatives

Why? Kill off

subtraction, negation

Addition is commutative


Simplify – Leveling Operators

Combine like binary trees to n-ary trees


Simplify – Rational Expressions

Expressions with * and / will be rewritten so that there is a single / node at the top, with only * operators below


Simplify – Collecting Like Terms


Simplify – Final Notes

How about making a + b and b + a equivalent?

Canonical Ordering First by their node type.

variables, functions, and constants Then, lexicographically.

Simplification may take more than one run through Iterate until no changes.


Large number computation – Part II

Sample problems: Sum, difference and product of big numbers Division by a small number

You can find more information on the web about this topic under “multiprecision” or “arbitrary precision” computation.

These notes cobbled together from http://numbers.computation.free.fr/Constants/constants.html


Representation

Same way as done in computer algebra:

X = x0 + x1 B + x2 B2 + … + xn Bn

where B is: usually 10 (for convenience) Or 2 (for computer’s convenience)

To think about: what about arbitrarily long real numbers?


Canonicalization and adding

Canonicalizing 12E2 + 34E0 => 1E3 + 2E2 + 3E1 +

4E0 Strip coefficient of values larger than B

and carry to next value Reorder sparse representation if

necessary Adding

Iteratively add all Xi Yi together and then do a canonicalization.


Multiplying

Given two big integers X and Y in canonical form:

the big integer Z = X*Y can be obtained thanks to the formula:

Notes: This is the obvious way we were taught in primary

school, the complexity is Θ(N2) Canonicalize after each step to avoid overflow in

coefficients To make this operation valid, B2 must fit in the

coefficient


Karatsuba Multiplication

We can split two big numbers in half:X = X0 + B X1 and Y = Y0 + B Y1

Then the product XY is given by(X0 + BX1) (Y0 + BY1)

This results in three terms:X0Y0 + B (X0Y1 + X1Y0) + B2(X1Y1)

Look at the middle term. We can get the middle term almost for free by noting that:

(X0 + X1) (Y0 + Y1) = X0Y0 + X0Y1 + X1Y0 + X1Y1 X0Y0 X0Y1 + X1Y0 X1Y1(X0 + X1) (Y0 + Y1)

= - -


Karatsuba Demonstration

12 * 34

Given: X1 = 1, X0 = 2, Y1 = 3, Y0 = 4

Calculate: X0Y0 = 8, X1Y1 = 3 (X0+X1)(Y0+Y1) = 3*7 = 21

Final middle term = 21 – 8 – 3 = 10 Solution: 8 + 10 * 101 + 3 * 102 = 408

Notes: Recursive, complexity is about Θ(n1.5) There’s a better way: FFT based multiplication

not taught here that is Θ (n log n)http://numbers.computation.free.fr/Constants/Algorithms/fft.html,


Miscellaneous Sorting – Part III

Most sorting relies on comparisons between two items. Proven to be Θ(n log n)

We’ll go over Radix sort Counting sort


What is radix?

Radix is the same as base. In decimal system, radix = 10.

For example, the following is a decimal number with 4 digits

0 3 2 8

1st

Digit3rd

Digit4th

Digit2nd

Digit


Radix Sort

Suppose we are given n d-digit decimal integers A[0..n-1], radix sort tries to do the following:

for j = d to 1 {By ignoring digit-1 up to digit-(j-1), form the sorted array of the numbers

}


Radix Sort (Example)

0123

2043

9738

1024

0008

2048

1773

1239

0

1

2

3 0123, 2043, 1773

4 1024

5

6

7

8 9738, 0008, 2048

9 1239

Original

Group using

4th digit

0123

2043

1773

1024

9738

0008

2048

1239

Ungroup

Sorted Array if we only consider digit-4



0123

2043

1773

1024

9738

0008

2048

1239

0 0008

1

2 0123, 1024

3 9738, 1239

4 2043, 2048

5

6

7 1773

8

9

Original

Group using

3rd digit

0008

0123

1024

9738

1239

2043

2048

1773

Ungroup

Sorted Array if we only consider digits-3 and 4



0008

0123

1024

9738

1239

2043

2048

1773

0 0008, 1024, 2043, 2048

1 0123

2 1239

3

4

5

6

7 9738, 1773

8

9

Original

Group using

2nd digit

0008

1024

2043

2048

0123

1239

9738

1773

Ungroup

Sorted Array if we only consider digit-2, 3, and 4



0008

1024

2043

2048

0123

1239

9738

1773

0 0008, 0123

1 1024, 1239, 1773

2 2043, 2048

3

4

5

6

7

8

9 9738

Original

Group using

1st digit

0008

0123

1024

1239

1773

2043

2048

9738

Ungroup

Done!


Counting Sort

Works by counting the occurrences of each data value.

Assumes that there are n data items in the range of 1..k

The algorithm can then determine, for each input element, the amount of elements less than it. For example if there are 9 elements less than

element x, then x belongs in the 10th data position.

These notes are from Cardiff’s Christine Mumford: http://www.cs.cf.ac.uk/user/C.L.Mumford/


Counting Sort

The first for loop initialises C[ ] to zero.

The second for loop increments the values in C[], according to their frequencies in the data.

The third for loop adds all previous values, making C[] contain a cumulative total.

The fourth for loop writes out the sorted data into array B[].

countingsort(A[], B[], k) for i = 1 to k do

C[i] = 0

for j = 1 to length(A) doC[A[j]] = C[A[j]] + 1

for 2 = 1 to k do C[i] = C[i] + C[i-1]

for j = 1 to length(A) do B[C[A[j]]] = A[j] C[A[j]] = C[A[j]] - 1


Counting Sort

Demo from Cardiff:http://www.cs.cf.ac.uk/user/C.L.Mumford/tristan/CountingSort.html

What’s the complexity?


Invariants – Part IV

Chocolate bar problemYou are given a chocolate bar, which

consists of r rows and c columns of small chocolate squares.

Your task is to break the bar into small squares. What is the smallest number of splits to completely break it?


Variants of this puzzle?

Breaking irregular shaped objects Assembling piles of objects


The Game of Squares and Circles

Start with c circles and s squares.

On each move a player selects two shapes. These two are replaced by one according to

the following rule:Identical shapes are replaced with a square.

Different shapes are replaced with a circle. At the end of the game, you win if the last

shape is a circle. Otherwise, the computer wins.


What’s the key?

Parity of circles and squares is invariant

After every move: if # circles was even, still will be even If # circles was odd, still will be odd


Introducing Modular Arithmetic

Two numbers a and b are said to be equal or congruent modulo N:

iff their difference is exactly divisible by N, i.e. abs (a-b) / N == 0


Modulus division

From the definition one can easily derive several properties. For example:

If a=b mod N and c=d mod N then (a+c)=(b+d) mod N

If a=b mod N and c=d mod N then ac=bd mod N

The criterion of divisibility by 9 follows from the fact that for every n, 10n=1 mod 9. This, in turn, is a straightforward consequence of 10 = 1 mod 9.


Divisibility tricks

Because we use base 10, there are some tricks to test for divisibility.

N is divisible by 3 iff sum of its digits is divisible by 3 (because 10n = 1 mod 3)

N is divisible by 9 iff sum of its digits is divisible by 9 (because 10n = 1 mod 9)

These tests can be applied recursively


Sam Loyd’s Fifteen Puzzle Given a configuration, decide whether it

is solvable or not:

Key: Look for an invariant over moves

8 7

1 5

12 15

2 13

6 3

4 10

14

11

9

87

1

5

12

15

2

13

6

3 4

10

14

119

Possible?


Solution to Fifteen

Look for inversion invariance Inversion: when two tiles out of order in a row inverted not inverted

For a given puzzle configuration, let N denote the sum of the total number of inversions and the row number of the empty square. Then (N mod 2) is invariant under any legal move. In other words, after a legal move an odd N remains odd whereas an even N remains even.

11 10 10 11


Inversions remain the same

Note: if you are asked for optimal path, then it’s a search problem

b c

d

a


In general

If you are asked to verify something, sometimes the search problem isn’t the most efficient way to compute the solution.

Look for an invariant to solve the problem


Prime Numbers and Factorization – Part V

Sieve of Eratosthenes Factorization Fast Exponentiation

Advanced Topics (not covered) Function Approximation

Newton’s Method


Sieve Algorithm

Initialization: Keep a boolean array [1...n] = PRIME. We

will change entries to COMPOSITE as we go on.

Algorithm: Set k=1. iterate towards √n Find the first PRIME in the list greater than

k. (e.g., 2.) Call it m. Change the numbers 2m, 3m, 4m, ... to COMPOSITE.

Set k=m and repeat.


Sieve of Eratosthenes

Demo from Univ. of Utah (Peter Alfeld):

http://www.math.utah.edu/~alfeld/Eratosthenes.html

Notes: If at n in the sieve, we know that all

numbers not crossed out under n2 are also prime.


More divisibility – casting out

Is 3659867394557 divisible by 7? “Cast out” sevens.

You can take any number you see in x and replace it by the remainder when divided by 7.

For instance, the first two digits in x are 36. You can replace these by a 1, since 36/7 = 5 remainder 1. If we continue this process, it might look something like this:

159867390357 (replace some of the big digits)152160320350 (start at the left and replace 12160320350 the first few digits in each step) 5160320350 260320350 50320000 1320301

Not particularly efficient, but works for every number. Why?


Fast Exponentiation

How do you calculate x256?

x256 = ((((((((x2)2)2)2)2)2)2) What about x255?

Store x, x2, x4, x8, x16, x32, x64, x128 on the way up.

Complexity Θ(log n)


Shortest Path – Part VI

Unweighted Shortest Path Weighted Shortest Path

A

C

D

B

EF


ShortestPath(s)

Run BFS(s) w.level: shortest distance from s w.parent: shortest path from s


Positive weighted shortest path

A

C

D

B

EF

5

51

2

3

1

31

4


Must keep track of smallest distance so far

If we found a new, shorter path, update the distance.

BFS(s) does not work

8

10

5

1

s

w

v

u


Observation 1

10

62

8

s w

v


Observation 2

≥6

6≥6

≥6

≥6

≥6

≥6

v

s


Definition

distance(v) : shortest distance so far from s to v

parent(v) : previous node on the shortest path so far from s to v

weight(u, v) : the weight of edge from u to v


Example

10

62

8

s w

vdistance(w) = 8weight(v,w) = 2parent(w) = v


Relax(v,w)

d = distance(v) + weight(v,w)if distance(w) > d then

distance(w) = dparent(w) = v

10

62

10

s w

v

10

62

8

s w

v


Dijkstra’s algorithm

A

C

D

B

EF

5

51

2

3

1

31

4



0

8

8

8

88

5

51

2

3

1

31

4



0

5

8

8

88

5

51

2

3

1

31

4



0

5

10

8

6

8

5

51

2

3

1

31

4



0

5

8

8

610

5

51

2

3

1

31

4



0

5

8

8

610

5

1

2

3

4



color all vertices light blueforeach vertex w

distance(w) = INFINITYdistance(s) = 0



while there are light blue verticesv = light blue vertex with min distance(v)color v aquaforeach neighbour w of v

relax(v,w)

3

0 5

6 7

8 9

s

v


Running time

color all vertices light blueforeach vertex w

distance(w) = INFINITYdistance(s) = 0while there are light blue vertices

v = light blue vertex with min distance(v)color v aquaforeach neighbour w of v

relax(v,w)

O(V2 + E)


Using priority queue

foreach vertex wdistance(w) = INFINITY

distance(s) = 0pq = new PriorityQueue(V)

while pq is not emptyv = pq.deleteMin()foreach neighbour w of v

relax(v,w)


Initialization O(V)

foreach vertex wdistance(w) = INFINITY

distance(s) = 0pq = new PriorityQueue(V)


Main loop


relax(v,w)


Main loop


d = distance(v) + cost(v,w)if distance(w) > d then

// distance(w) = dpq.decreaseKey(w, d)parent(w) = v

O((V+E) log V)


Useful Sites

http://www.cut-the-knot.org/blue/Modulo.shtml

http://mathforum.org/k12/mathtips/division.tips.html

31 May 2003IOI/ACM Training1 IOI/ACM Training Session 3 Min-Yen KAN These notes available at...

Documents

Transcript of 31 May 2003IOI/ACM Training1 IOI/ACM Training Session 3 Min-Yen KAN These notes available at...