Takaaki MasuiAdvisor: Kosaku Nagasaka

February 14, 2013

Performance analysis ofapproximate GCD algorithms for

univariate polynomials

GCD (Greatest Common Divisor)

Primitive computation

- Reduction of fraction

- Solution of system of equations

x2 + x− 2

x2 − 4x + 3=

(x + 2)(x− 1)

(x− 3)(x− 1)=

x + 2

x− 3

GCD

=⇒

x(x− 2)(2x− 1)(x + 1) = 0

(2x + 3)(2x− 1)(x + 1) = 0

(−x− 1)(2x− 1)(x + 1) = 0

2x4 − 3x3 − 3x2 + 2x = 0

4x3 + 8x2 + x− 3 = 0

−2x3 − 3x2 + 1 = 0 GCD

GCD - Exact computation

f(x) = x2 +5

3x− 50

9=

�x− 5

3

��x +

10

3

�

g(x) = x2 − 104

21x +

115

21=

�x− 5

3

� �x− 23

7

�

Euclidean algorithm

139

21x− 695

63= x2 +

5

3− 50

9−

�x2 − 104

21x +

115

21

�· 1

0 = x2 − 104

21x +

115

21−

�139

21x− 695

63

�·�

21

139− 69

139

�

PRS

GCD

GCD - Approximate computation

Coefficients is represented by floating-point numbers

=⇒ f(x) and g(x) are coprimef(x) = 1.00x2 + 1.66x− 5.56

g(x) = 1.00x2 − 4.96x + 5.49

Coefficients have errors

f(x)

g(x)

f(x) = 1.01x2 + 1.64x− 5.55

g(x) = 0.99x2 − 4.98x + 5.51

What do we do to have GCD...?

GCD - Approximate computation

Coefficients is represented by floating-point numbersf(x) = 1.01x2 + 1.64x− 5.55

g(x) = 0.99x2 − 4.98x + 5.51

F (x) = 0.994x2 + 1.63x− 5.54

G(x) = 1.00x2 − 4.97x + 5.52

If f(x) and g(x) are...

F (x)

G(x)

gcd(F, G) = x− 1.68

Want to have GCD

GCD - Approximate computation

Coefficients is represented by floating-point numbersf(x) = 1.01x2 + 1.64x− 5.55

g(x) = 0.99x2 − 4.98x + 5.51

F (x) = 0.994x2 + 1.63x− 5.54

G(x) = 1.00x2 − 4.97x + 5.52

If f(x) and g(x) are...

F (x) = f(x)− 0.016x2 − 0.01x + 0.01

G(x) = g(x) + 0.01x2 + 0.01x + 0.01

magnitude ofmoving coefficients

Want to have GCD

gcd(F, G) = x− 1.68

Formulation of approximate GCD

For given f(x), g(x) ∈ R[x] and ε > 0, find d(x) ∈ R[x]

f(x) + ∆f (x) = f(x) · d(x)

g(x) + ∆g(x) = g(x) · d(x)such that

perturbation cofactor approximate GCD

where

tolerance

• f(x) and g(x) are coprime,

• deg(∆f ) ≤ deg(f), deg(∆g) ≤ deg(g),

• �∆f (x)�2 < ε�f(x)�2, �∆g(x)�2 < ε�g(x)�2.

smallerhigher

degree

Note: For f(x) = fmxm + · · · + f0, �f(x)�2 =�

f2m + · · · + f2

0

Where is approximate GCD required?

Image processing

Before

Detect shake!!Offset!!

After

Separate multiple roots of algebraic equationsand so on...

Master thesis - 1

Survey of known algorithms

Based on Algorithm

Euclidean-like QuasiGCD, COPRIME

Matrix decomposition QRGCD, SVDGCD

Optimization techniqueSTLN, UVGCD, Fastgcd,

GPGCD

Compare our implementations with official implementations

Master thesis - 1

Survey of known algorithms (brief results)- v_coprime vs. QuasiGCD (SNAP)• v_coprime is faster than QuasiGCD• QuasiGCD does not follow recent progress of Maple

- v_qrgcd vs. QRGCD (SNAP)• Different degree is detected

- v_gpgcd vs. GPGCD (Terui)• v_gpgcd is faster than GPGCD

review carefully:- Original paper- SNAP

Note: “v_XX” is our implementation on Maple 16

Our implementations are faster than official implementations

Master thesis - 2

Finding of QRGCD’s defects- Misunderstandings in main theorems in original paper- SNAP is significantly different from original paper

Improve QRGCD (ExQRGCD)- Based on corrected theorems- Guarantee that output satisfies given tolerance

QRGCD is used as a benchmark for newly algorithms

�∆f (x)�2 < ε�f(x)�2, �∆g(x)�2 < ε�g(x)�2

S(f, g) =

fm fm−1 · · · f0. . . . . . · · · . . .

fm fm−1 · · · f0

gn gn−1 · · · g0. . . . . . · · · . . .

gn gn−1 · · · g0

= Q · R

What is QRGCD?

R. M. Corless, S. M. Watt and L. Zhi (2004)

Q: orthogonal

R: upper triangular

For Sylvester’s matrix S(f, g), we have

f(x) = fmxm + fm−1xm−1 + · · · + f1x + f0

g(x) = gnxn + gn−1xn−1 + · · · + g1x + g0

Algorithm - QRGCD

1. Compute S(f, g) = QR

2. Find the gap between the k-th and (k− 1)-th row of R

3. Apply 1. - 2. for the reversal polynomials of cofactors

⇐= each row corresponds

to PRSrk(x)

rk−1(x)If there is

gap...

GCD

Algorithm - QRGCD

1. Compute S(f, g) = QR

2. Find the gap between the k-th and (k− 1)-th row of R

3. Apply 1. - 2. for the reversal polynomials of cofactors

e.g., x3 + 2x2 + 3x + 4 =⇒ 4x3 + 3x2 + 2x + 1

R may not detect outside common roots

the unit circle

reversal polynomial transfers roots from

outside to inside

MainTheorem

unit circleC Misunderstanding in the proof

Detectability of outside roots

Lemma.

The roots which are detected from row of R are not always

in order of relative closeness.

f(x) = (x + 1/1000)(x− 1/1000)(x + 1000)(x− 1000)

g(x) = (x+1/1000)(x− 9/1000)(x+1000)(x− 1000)(x− 2000)

In exact computation

Exact GCD can detected by QR factoring

What will happen in approximate computation...?

3-rd: = 71213/1689590993√

1056080967(x + 1/1000)(x + 1000)

2-nd: = 0

1-st: = 0

Detectability of outside roots

Lemma.

The roots which are detected from row of R are not always

in order of relative closeness.

f(x) = (x + 0.001)(x− 0.001)(x + 1000)(x− 1000)

g(x) = (x + 0.0010000001)(x− 0.009)(x + 1000.0001)(x− 2000)

The approximate GCD of tolerance ε = 10−12

d(x) ≈ 9.99998× 10−4(x + 0.00100000)(x + 1000.00)However,

4-th: ≈ 0.00143003(x− 0.004999981)(x + 0.001)(x + 997.993)

2-nd: ≈ 0.00565685(x + 0.001)3-rd: ≈ 0.20978(x− 0.00499978)(x + 0.001)

artificial common inside roots

Detectability of outside roots

Lemma.

The roots which are detected from row of R are not always

in order of relative closeness.

As proof, we focus on order of roots

- Sylvester’s single sum formula

row of R =⇒ PRS =⇒ subresultant =⇒ single sum

We can look relation between row of R and roots.

Sresk =�

A�⊂A,#A�=k

R(x,A�)R(A\A�, B)

R(A\A�, A�)A,B: all the roots

of f(x), g(x)large? small?

detectability of roots

R(x,A) =�

a∈A

(x− a)

Performance analysis.

- Implementation: Maple 16- Execution environment:

Ubuntu 12.04 LTS Intel Core i7, 3.30GHz, RAM 64GB

Example 1. (small leading coefficients)

Bini and Boito (2010) says if small leading coefficient then

QRGCD fails to recognize the correct gcd.

f(x) = (αx3 + 2x2 − x + 5)(x4 + 7x2 − x + 1)

g(x) = (αx3 + 2x2 − x + 5)(x3 − x2 + 4x− 2)

with tolerance ε = 10−5

For 10−10 < α < 10−5, we generate 10000 pair of polynomials.

Comparison between:ExQRGCD, SNAP, SNAP(w/o f.n.t)

Reason is the algorithm “find non-zero terms”...?

Result - Experiment 1.

#(deg = 3) #(deg = 2) �∆f (x)�2 �∆g(x)�2

ExQRGCD 10000 0 3.64× 10−14 2.18× 10−14

SNAP 742 9254 2.04× 10−6 6.06× 10−7

SNAP (w/o f.n.t) 10000 0 8.15× 10−14 3.84× 10−14

Note: we computed on Maple 16 with Digits:=16.

Result- SNAP returns fail degree over 90%- ExQRGCD and SNAP (w/o f.n.t) can work correctly- QRGCD is not weak for this case (many researchers

may misunderstand it ill-conditioned case)- The original paper does not mention this algorithm

Experiments 2. and 3.

Example 2. (random polynomials)

Example 3. (random polynomials with perturbation)

Note: input polynomials aregenerated by randpoly commandnormalized and rounded with Digits:=10

We computed with tolerance ε = 10−5

We generated 100 pairs of polynomials:

deg(d) = 50%

deg(f), deg(g) = 50%g(x) = g(x) · d(x)

f(x) = f(x) · d(x)

f(x) = f(x) + ∆f (x)

g(x) = g(x) + ∆g(x)

deg(∆f ), deg(∆g) = 100%

�∆f (x)�2 = �∆g(x)�2 = 10−8

Result - Experiment 2. and 3.

Detected degree

0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0

45.0

50.0

10 20 30 40 50 60 70 80 90 1000

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0

45.0

50.0

10 20 30 40 50 60 70 80 90 100

ExQRGCD SNAP Paper

without perturbation with perturbation

Note: “failure” is counted as 0

Result - Experiment 2. and 3.

The number of failures and wrong perturbations

#(fail): the number of failures of computation

#(∆f ): the number of �f(x)− f(x)d(x)�2 ≥ ε.

#(∆g): the number of �g(x)− g(x)d(x)�2 ≥ ε.

without perturbation with perturbation

Algorithm #(fail) #(∆f ) #(∆g) #(fail) #(∆f ) #(∆g)

ExQRGCD 0 0 0 0 0 0

SNAP 11 6 6 315 6 6

Paper 19 24 27 223 31 31

Experiments 4. (artificial common roots)

We generated 100 pairs of polynomials:

Note: input polynomials are normalized and rounded with Digits:=10

We computed with tolerance ε = 10−5

inside

roots

unit circle

outside roots

commonroots(60%)

10−2

102

cofactor(40%)

How to generate:

C

Result - Experiment 4.

1.0E-02

1.0E-01

1.0E+00

1.0E+01

1.0E+02

10 20 30 40 50 60 70 80 90 100

ExQRGCD SNAP Paper

CPU time

Note: vertical axis is log scaled

Residual

1.0E-06

1.0E-05

1.0E-04

10 20 30 40 50 60 70 80 90 100

border

Note: vertical axis is log scaled

Result - Experiment 4.

Detected degree

Algorithm #(fail) #(∆f ) #(∆g)

ExQRGCD 0 0 0

SNAP 624 110 119

Paper 172 210 216

The number of failures and wrong perturbations

SNAP returns “Failure” over 60%

0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0

10 20 30 40 50 60 70 80 90 100

Note: “failure” is counted as 0

#(detected degree)= #(expected degree)

ExQRGCD SNAP Paper

Summary of today’s talk

Suggest ExQRGCD- Based on corrected theorems in the original paper- ExQRGCD always satisfies the given tolerance- ExQRGCD is not faster than SNAP • SNAP returns “failure” (but ExQRGCD does not)• The number of loops is different from QRGCD

- ExQRGCD works better than SNAP• Polynomials with perturbations• Polynomials having artificial common roots

SNAP is difference from the original paper- find non-zero terms (there are also other differences)

Further research

Against optimization techniques- Compare with Fastgcd, UVGCD and GPGCDExQRGCD be in the SNAP package- Estimate the time-complexity of QRGCD, ExQRGCDMake it available on another CAS- Open source CAS (e.g., Sage)