Advanced Algebraic Algorithms on Integers and

Polynomials

Prepared by

John Reif, Ph.D.

Analysis of Algorithms

Integer and Polynomial Computations

a) Newton Iteration: application to divisionb) Evaluation and Interpolation via Chinese

Remaindering

Main Lecture Material: Algorithms on Integers

Advanced Lecture Material:Extension of Algorithms to Polynomials

Taylor Expansion

20

0 0 0 0

(x-x )(x) = (x ) + (x - x ) ' (x ) + " (x ) + ...

2f f f f

Taylor Expansion (cont’d)

• To find root f(x), use Newton Iteration:

• Example:– To find reciprocal of x choose

find root

i1

i

(x ) = -

'(x )i i

fx x

f

1( ) 1f y

xy

1y

x

1

( )= (2 )

'( )i

i i i ii

f yy y y y x

f y

Application of Newton Iteration to Reciprocal of an Integer• Input integer x, accuracy bound k• Initialize if x has n bits

0 2 ny

1

2

1 1

2 2 20 0

i=1 to k

(2 )

1 where 1

2 let error = 1

then = 1 = 1 (2 )

1 = ( ) = ( ) 2 since

2

k

k k

i i i

k k

k k

k k k k

k

for do

y y y x

output y y x

proof y x

y x xy y x

Application of Newton Iteration to Reciprocal of an Integer (cont’d)• Theorem Integer Reciprocal can

be computed to accuracy 2-n in O(log n) integer mults and additions.

Steven Cooks’s Improvement (His Harvard Ph.D. Thesis)

• Key Trick: Since

we need only compute yk up to 2k+1 bit accuracy of error on kth stage

• Total Time Cost

• Where M(n) is time cost to multiply two n bit integers

2

11

2kky x

log1

0

(2 ) ( ( ))n

k

k

c M O M n

Other Applications of Newton Iterations on Integers

• O(M(n)) time algorithms:• Quotient + divisor of integer

division• Square root• Sin, cosine, etc.

• Used in practice!

• Reciprocal = polynomial r(x)

• where

and (x) has degree < n-1

2 2

( ( ))( )

nxP x

P x

2 2( ) ( ) ( )nr x p x x x

Advanced Topic: Extension to Polynomial Reciprocal

Definition of Polynomial Reciprocal

Algorithm: Reciprocal (P(x))

• Input polynomial

degree n-1, n is power of 2

1

0

( )n

ii

i

P x a x

0

12

1 1 1/ 2

(3/ 2) 2 21 1

2

1[1] 1 else

[2] (x) Reciprocal ( ( )) where ( )

[3] ( ) 2 ( ) ( ) ( )

( )[4] ( )

nn i

ii n

n

n

if n then returna

r P x P x a x

R x r x x r x P x

R xreturn r x

x

Proof of the Reciprocal Algorithm• Theorem: The Algorithm Correctly

Computes Reciprocal(P(x))

• Proof by induction

00

21 2

1 2

11 ( ) ( )

let ( ) ( ) ( )

where deg ( ) 1, deg ( ) 12 2

n

basis n P x a so r xa

inductive step P x P x x P x

n nP P

Proof Algorithm Computes Reciprocal (P(x)) (cont’d)

• By induction hypothesis, if

1 1

21 1 1

1

( ) = reciprocal ( ( )) then

( ) ( ) ( )

nwhere ( ) has degree < 1

2

n

r x P x

r x P x x x

x

Proof Algorithm Computes Reciprocal (P(x)) (cont’d)

• At line [3] we compute

(3/ 2) 2 21 1

21 2

2 2 (3/ 2) 21 1 1 2

2

21 1 1 2

( ) 2 ( ) ( ) ( )

since ( ) ( ) ( )

( ) ( ) 2 ( ) ( ) 2 ( ) ( )

( ) ( ) ( ) ( )

n

n

n n

n

R x r x x r x P x

P x P x x P x

P x R x r x P x x r x P x x

r x p x x r x p x

Proof Algorithm Computes Reciprocal (P(x)) (cont’d)

21 1 1

2

3 4 21 1 2

3 4 2 4

2

2 2 2

Substituting + (x) for ( ) ( ),

we get

( ) ( ) ( ) ( ) ( )

= 0( )

( )But r( ) so

( ) ( ) ( )

n

nn

n n

n

n n

x r x p x

R x p x x x x r x p x

x x

R xx

x

r x p x x o x

Modular Arithmetic

• Assume relatively prime P0, P1, …, Pk-1

• Let given x, 0 x < p

1

0

k

ii

p p

0 -11 1 ( , ..., ) where

mod for 0, ..., -1k

i i

x x x

x x p i k

Applications to Arithmetic

• But doesn’t extend to division (overflow problems)

i i i i

i i

i i

Compute u op v by computing for i=0,...,k-1

w = u op v mod p

where u = u mod p

v = v mod p

op {+, -, }

Super Moduli Computation

• Input p0, p1, …, pk-1, where pi < 2b

• Output Super modular Tree:

Super Moduli Computation (cont’d)

• Time Cost

log

0

22

( ( ) log )

ki

ii

kM b

O M k b k

Algorithm Residue Computation

• Input

• Output x0, x1, …, xk-1, whenxi =x mod Pi i=0, and k-1

• Recursive algorithm[1] compute quotient and remainders:

1

0

, 0k

ii

x x P P

( 1) / 2

1 1 1 10

1

2 2 2 2( 1)

12

,

,

k

ii

k

ik

i

x q v r v P

x q v r v P

Algorithm Residue Computation (cont’d)

[2] recursively compute

[3] output for i = 0, 1, …, k-1

1

2

( 1)(2.1) mod for 0,...,

2( 1)

(2.2) mod for 1,..., 12

i

i

kr P i

kr P i k

1

2

( 1)mod for

2mod( 1)

mod for 2

i

i

i

kr P i

x Pk

r P i

Time Cost for Residue Computation

• Let D(n) = time cost for integer division = O(M(n))

• Total Time for input size n = k· b

( ) 2 2 ( ) ( )2

2 ( ( ))2

( ( ) log )

nT n T kD b O n

nT O M n

O M n n

Proof of Algorithm for Residue Computation

• Idea of Proof of algorithmUses fact: if x = q v + r and v mod Pi = 0,

then X mod Pi = r mod Pi

Advanced Topic: Residue Computation on Polynomials

• Input moduli P0(x), P1(x), …, Pk-1(x) and each degree d and relatively prime

• Algorithm uses similar Super modular Tree tre , but using polynomials rather than integers

Advanced Topic: Residue Computation on Polynomials (cont’d)• Output for i=0, …, k-1

Qi(x) = Q(x) mod Pi(x)Q(x) has degree kd

• Theorem The Residue Computation can be done in time O(M(n) log n) where n = k· d

• Proof Idea use same algorithm as in integer case

Advanced Topic: Multipoint Evaluation of Polynomials by Residue Computation

• Input polynomial f(x) degree n-1 and points x0, x1, …, xn-1

[1] for i=0, …, n-1 let Pi(x) = (x-xi)[2] By Residue Algorithm Computer for i=0, …, n-1

f(xi) = f(x) mod Pi(x)

[3] output f(x0), …, f(xn-1)

• Time Cost O(M(n) log n), = O(n(log n)2)

Polynomial Interpolation

• Input evaluation points x0, …, xn-1

values y0, …, yn-1

• Output P(x) where yk = P(xk) for k=0, …, n-1

Polynomial Interpolation (cont’d)

• Interpolation formula:

• Where

1

0

( ) ( )n

k k ik i k

P x y a x x

1

( )kk i

i k

ax x

Proof of Polynomial Interpolation

• Proof uses identities:

( ) mod( ) 1

( ) mod( ) 0 for j k

k i ki k

k i ji k

a x x x x

a x x x x

Using Chinese Remaindering for Integer Interpolation

• Input relatively prime P0, P1, …, Pn-1 and yi {0,…Pi-1} for i=0,…,n-1

• Problem compute y < Pi s.t. yi = y mod Pi i=0, …, n-1

• Generalized Interpolation Formula:

• Where

• proof

,

1,

and

( ) mod

k i ki k

i k i k

a s

s P P

1mod

0k i ji k

j ka P P

j k

Using Chinese Remaindering for Integer Interpolation

1

0

n

k k ik i k

y y a P

Advanced Topic: Preconditioned Interpolation

Preconditioned Case assumes coefficients {ak | k=0, …, n-1} precomputed

• Use Divide & Conquer

1

0

n

k k ik i k

y y a P

( 1) / 2 1 11

( 1)0 10 02

n n nn

k k i k k ink i k i kki i

y a P y a P

Preconditioned Interpolation (cont’d)

( 1) / 2 ( 1)1

( 1)0 1 ( 1)2 12

( 1) / 2( 1) / 21

( 1) 01 02

n nn

i k k ini i kk n

i

nnn

i k k in k i ki i

P y a P

P y a P

Time Cost for Preconditioned Interpolation

• Assuming {a0, …, an-1} precomputed

( ) 2 22 2

( ( ) log )

n nT n T M

O M n n

Precomputation of {a0, …, an-1}

1) Compute

2) Compute bk wherebkPk = P mod (Pk)2

by Residue Computation O(M(n) log n)

3) Compute ak = (bk)-1 mod Pk by Extended GCD algorithm

1

0

n

ii

P P

Proof of Precomputation of {a0, …, an-1}

• proof

2k k k

2k k k

i k k

k i k

since b P = P mod (P )

then P = d (P ) + b P

so P d P + b

so b P mod P

i k

i k

Precomputation of {a0, …, an-1} for Polynomial Interpolations• Here Pi = (x-xi) for i=0, …, n-1

reduces to multipoint evaluation of derivative of Q(x) O(M(n) log n) time!

n-1

k jj=0k

kk

k

k

Q(x)b where Q(x)= (x-x )

(x - x )

Q(x) - Q(x ) = since Q(x ) = 0

(x - x )

d Q(x) =

dx x = x

Conclusion

• Polynomial and Integer Computations use similar divide and conquer techniques to solve:

1. Multiplication2. Division3. Interpolation and evaluation

• Open Problem Reduce from time O(M(n)log n) to O(M(n))

Newton Iteration and Polynomial Computation

Prepared by

John Reif, Ph.D.

Analysis of Algorithms