AD 7 A128 1418 ERROR-FREE PARAI L E HIGH-ORDER CONVERGENT ITERATIVEMATRIX INVERSION BASE..U) MARYLAND UNIV COLLEGE PARKCOMPUTER VISION LAB E V KRISHNAMURTHY NOV 82 BR 229

UNCLASSIFIED AFOSR-TR _83-0387 AFOSR-77-3271 / 21 N

IN

6"3

a-a7

2,2

L *12.0

11.25 1.4 1111.6

MICROCOPY RESOLUTION TEST CHART

A T(T .A L PLIRI AU 01 '0 ANDARD 0

V'9 R-TR- 3 U 38 7

'10

, 7 , . 3. 19 8 3

COMPUTER SCIENCETECHNICAL REPORT SERIES

0

UNIVERSITY OF MARYLANI 'TJ+ COLLEGE PARK, MARYLAN13 ELIC

20742 MAY 251

EApproved torPubi,.

UNC.ARSTZTFI.DSECURITY CLASSIFICATION OF THIS PAGE (?en oat."Ents.,d)/ ' EPO T D CUME TAT ON AGEREAD rNS'FRUCTIONS"

REPORT DOCUMENTATION PAGE BEFORE COMPLETIN.6 FORM1. REPORT NUMBER 2. GOVT ACCESSION NO. 3. RECIPIENT'S CATALOG NUMBERAFOSR-TR- 83-s 0 3887

4. TITLE (and Subtitle) S. TYPE OF REPORT & PERIOD COVERED

ERROR-FREE PARALLEL HIGH-ORDER CONVERGENT ITERA- TECHNICALTIVE MATRIX INVERSION BASED ON p-ADICAPPROXIMATION 6- PERFORMING ORG. REPORT NUMBER

TR-l=297. AUTHOR(&) 8. CONTRACT OR GRANT NUMBER(s)

E.V. Krishnamurthy AFOSR-77-3271

9. PERFORMING ORGANIZATION NAME AND ADDRESS IW. PROGRAM ELEMENT. PROJECT. TASKDepartment of Computer Science AREA & WORK UNIT NUMBERS

University of MarylandCollege Park MD 20742 PE61102F; 2304/A2

II. CONTROLLING OFFICE NAME AND ADDRESS 12. REPORT DATEMathematical & Information Sciences Directorate NOV 82Air Force Office of Scientific Research 13 NUMBER OF PAGESBolling AFB DC 20332 18

14 MONITORING AGENCY NAME & ADDRESS(f different from Controlling Office) IS. SECURITY CLASS. (of thi, report)

UNCLASSIFIED15a. OECLASSIFICATION DOWNGRADING

SCHEDULE

16. DISTRIBUTION STATEMENT (o1 this Report)Approved for public release; distribution unlimited.

17. DISTRIBUTION STATEMENT (of the abstrect entered in Block 20, If different from Report)

IS. SUPPLEMENTARY NOTES

19. KEY WORDS (Continue on revere. side if necleary and identify by block number)

Parallel computation; matrix inversion; p-adic approximation.

20. ABSTRACT (Continue on reverie side If necessary and identify by block number)The Newton-Schultz iterative scheme is reformulated in an algebraic setting tocompute the exact inverse of a matrix (or the solution of a linear system ofequations) over the ring of integers, with a high order of convergence, byusing a finite segment p-adic representation of a rational. This method isdivergence-free; it starts with the inverse of a given matrix over a finitefield (called the priming step) and then iterates successively to construct,in parallel, the p-adic approximants (Hensel Codes) of the rational elements ofthe inverse matrix. The p-adic approximant is then converted back (CONTINUED)

DD , ,AN , 1473DD173 UNCLASSIFIED

SECURITY CLASSIFICATION Or THIS PAGE (Nhen Deis Entered)

=-

UNCLASSIFIEDSECURITY CLASSIFICATION OF TIS PAGE(Whn DaX;'nr,)

ITEM #20, CONTINUED: to the equivalent rational using the extended Euclideanalgorithm.

The method involves only parallel matrix multiplications and complementationsand has a quadratic convergence rate. Extension to achieve higher order con-vergence is straightforward if parallel matrix arithmetic facilities forhigher precision operands (in a prime base system) are available.

UNCLASSTFTIDSECURITY CLASSIFICATION OF THIS PAGE(When Date Entered)

A IF. F,0 EC ":" ." 7 :, T -NTIF IC F---- " ',Ti E 'J:.. . :, L To fTC . . . "; n

This Lc . t 1w h

approvc f.. 'Di strib,'ti:.- . ' .t,<.

MATTIEVW J-. 'r-Chlef T ¢ nl :'. iforma'tion Div is ion

TR-1229AFOSR-77-3271 November 1982

ERROR-FREE PARALLEL HIGH-ORDER CONVERGENTITERATIVE MATRIX INVERSION BASED ON

p-ADIC APPROXIMATION

E. V. Krishnamurthy*

Computer Vision LaboratoryComputer Science CenterUniversity of MarylandCollege Park, MD 20742

ABSTRACT

The Newton-Schultz iterative scheme is reformulated inan algebraic setting to compute the exact inverse of a matrix(or the solution of a linear system of equations) over the ringof integers, with a high order of convergence, by using a finitesegment p-adic representation of a rational. This method isdivergence-free; it starts with the inverse of a given matrixover a finite field (called the priming step) and then iteratessuccessively to construct, in parallel, the p-adic approximants(Hensel Codes) of the rational elements of the inverse matrix.The p-adic approximant is then converted back to the equiva-lent rational using the extended Euclidean algorithm.

The method involves only parallel matrix multiplicationsand complementations and has a quadratic convergence rate.Extension to achieve higher order convergence is straightforwardif parallel matrix arithmetic facilities for higher precisionoperands (in a prime base system) are available.

*Permanent address: Indian Institute of Science, Bangalore -

560012, INDIA

The support of the U.S. Air Force Office of Scientific Researchunder Grant AFOSR-77-3271 is gratefully acknowledged, as is thehelp of Janet Salzman in preparing this paper.

83 05 2:.063.- q.-*----~~~~~w- memo--- -* ______ . 1

1. Introduction

Error-tree direct methods for the inversion of numerical

and polynomial matrices are available in the literature [11

[2]. In this paper we describe a parallel error-free high-

order convercent matrix inversion method for matrices over

integers, based on the Newton-Schultz iterative scheme [3]

[4] and the p-adic approximation [5-9]. Some of the impor-

tant aspects of this scheme are:

(i) Inversion of matrices over p-adic fields, analogously

to inverting or reciprocating the numbers, without

any convergence problem.

(ii) The exact and simultaneous determination of the

rational elements of the inverse matrix in p-adic

digit parallel fashion with a quadratic or higher rate.

(iii) Easy realization of the scheme and its variants (higher-

order convergent extensions) by parallel matrix multi-

plications.

This paper is organized in seven sections. In the second

section we outline the principle of the Newton-Schultz scheme

for reciprocating numbers. The third section describes the

reformulation of the Newton-Schultz scheme in an algebraic

setting to compute the p-adic approximant to the inverse of a

matrix over the ring of integers. In the fourth section we

describe the extended EuclideFan algorithm that converts a

given p-adic approximant over a range of rationals into an

equivalent rational. The fifth section contains an example.

imp 0

In Section 6 we briefly deal with the solution of a linear

system of equations, having a linear convergence rate.

Several remarks pertaining to possible extensions and gen-

eralizations are provided in the last section.

'I

2. The principle

Let f(x) be a real function of the real variaole x and

x=cx be a root of f(x)=O. We assume that:

(a) f(x), f'(x) and f"(x) are continuous in a neighbor-

hood la,b] of x=; (b) x=t is an isolated root in

[a,b]; (c) f'(x) and f"(x) do not vanish in [a,b].

The seirch for the root x=, entails finding the root

of the equation

f'(x) (x)

Since >' (,=0 there exists a neighborhood of x=C( such that

the sequence {x. I defined byi 0

xn = xn-l - f(x n l)/f'C nx I (n=1,2.... (i)

converges to x=(i if the first aporoximation x=x 0 lies in this

neighborhood. Applied to the function f(x)=l/x-a (1) gives

the Newton-Schultz scheme

Xn= x nC(2-ax n-) (2)-I

The sequence (2) converges to a rhe matrix inversion algori-

thm to be described in the next section is by analogy based on

the sequence of iterates defined by (2) [31 (4].Acession For

DTIC TABUnannounced El jJus t if ica t i 0 ___o_

Dist rit i on/Avallability Coes

iAvit

. . . . . . . " -" " - " , . ' . . _ . _ , , - - -e, . -

3. The Newton-Schultz method

Let A=[aij ] be a matrix over the ring of intecers Z and

r a prime such that det A mod pO0. (The reason for this will

become clear later.) The algorithm first constructs A- I mod p

and using this in tne Newton-Schultz recurrence olbtains a segment-

ed p-adic representation of the inverse matrix 15-9].

Theorem_ : There exists a matrix sequence B 2i} i> such that22ii

"rnj mod p = I for all i>O, where A is the matrix to be inverted

and I is the identity matrix; 2 i is the inverse of A(in Z) mod

p (or B 2i is the p-adic approximant of A- ).

Proof: We show the sequence {B 2ii>0 can be generated recur-

sivzly and then prove by induction that it has the property

stated, naritely, AB.i mod p = I. The first member of the sequence

i 1 is obtained in a priming step by solving

AB1 mod p = I

by 7aussian elimination or some other method. It amounts Lo

ciic~inq the inverse of A in Z mod p. Then in a powering step

'..c use the recurrence relation

B i = B i-1 (21 - AB i-I) mod p (il) (3)

to construct the successive iterates.To see that the theorem holds let AB2i mod p = I be true

.:or i = n-l(n>l);then, by (3)2n 2 n

(AB2n) mod p = AB 2n-1(21 - AB 2n-1) mod p

... .- .--.. ...... ... . .. , 1'-

n-].

Since AB2n-1 mod p = I by the induction hypothesis, we have2n-i

AB 2n- 1 = I + P E n I ,

where E is the error matrix. Thus we can write

22o n=( 2n- 2n-i 2

AB 2n n- En _ ) (I - p En1 ) mod p

Since by construction the theorem holds for n=O, it is true for

all n>0 by induction.

Our algorithm first obtains B2k by iterating k times, where

k is the minimum integer satisfying the inequality

n n 2

p - > ( . ) (4)2 i=l j=l ii

This inequality ensures that the largest element of the inverse

matrix lies within the range of the segmented p-adic representa-

tion of the corresponding rational [5] [8].

Let N denote a positive integer satisfying the inequality

N -< (5)2

We define a finite subset F N of the rational numbers Q as the set

F = {C O<Ic<N and 0<dj<N}N d' c

We call the set FN the order N Farey fractions, or simply Farey

rationals of order N.

If p and k are properly chosen to satisfy (4) then the

rationals FN which are mapped onto their segmented p-adic

representations in B2k can be uniquely recovered using an algori-

thm which is based on the extended Euclidean algorithm for finding

the greatest comnon divisor of two integers [10] [111].

AI- -- *1

Let a/b and w be the ij-th entry of A-1 and B2k respectively.

Then

ab- mod p = W (6)

since b-1 exists mod p , due to the fact that det A mod pO.

In the following section we describe how to recover a/b

given w, provided (4) is satisfied. This algorithm filters out

a very small subset of rationals among which the desired rational

belonging to FN occurs. We will call the function that computes

a/b given w,the EUCLID; thus EUCLID(w)=a/b.

Remark

The number k determined from (4) is generally larger than

desired; so to iterate k times encails much superfluous computa-

tion. A practical method of avoiding this would be to compute

EUCLID (B2k) and EUCLID(B2k+l) starting with some reasonable k

and stop as soon as they are equal. This would unambiguously

determine the inverse.

4. Computation of Farey rationals using the Euclidean algorith!,

The Euclidean algorithm [111 constructs three pairs of rm-

bers (ui,u!), (ai,bi), (ti,t!) for each i=0,1,2,...,k starting

with u0P-,U6'=0, a0 =wb 0=l and ending when t.=0, az illustrated

in Table 1; here the symbol [ ] denotes the lower integral part

Note that the qi's here correspond to the continued fraction

excpansion [5] [12] of pr/W.

It can easily be shown that the pairs (ai1bi ) in Table

satisfy the following conditions 110] [111]

Cross-product rule:

ai*bi I +a b. = pr < 2N2+1i'b i+il +ai+l 1

Monotonicity:

<ai+ll < Jail, with a.=w,ak=l (8)

bi+l - Jbil with bo0l,bk=w- mod p r (9)

where w is such that gcd(w,p r)=l and w- denotes the multipli-

rcative inverse of iw mod p

It is now necessary to show that (i) there exists a pair

(aj,bj) in Tablu 1 which satisfies the condition of a Farey

rational F 'Section 3), and (ii), such a pair is unique in the senseN '

that there exists no other pair belonging to FN -

To prove this, we use the fact that ai (starting with ao=w)

successively decreases to 1; and bi (starting with bo=l) succes-

sively increases to w when gcd(w,p r ) = 1.

Let us assume that for some j, b. has already increased from

1 to JN'J with JN'J < INJ and is close to INI, and the correspond-

ing aj has already decreased from w to IN"! where IN"I > INI and

- - . .. --- " -*---: l, z - _ . '% .'- 1 ,'4,,,l

is close to INI. Then using (7) we can prove that the succeed-

inq pair (a j+, bj+ I ) will have to be in FN or in other words

a pair of the form (aj+ I, b9+ I ) with aj+1 I < N and lbj+l ,

which skips a Farey rational belonging to FN cannot exist.

For if jail > N+l and Ibjj < N and laj+ 1 1 < N an6 lb,+ 1

N+l, we have a < N Using this in (7) we obtain

!aj'bj l> N 2+1. But we have jajI > N+l. Therefore !bj~lI <

(N2 +)/(N+l) = [NJ. Hence our assumption jbj+l1 > N is false.

We will now show that there is only one such rational be-

longing to FN. In other words, we will show that if for some j,

(a./b.) belongs to FN then (aj+i/b j+) cannot be in FN* Note that

the cross-product is maximum when

jail = N, IbjI = N - 1

laj+ 1 1 = N-l, lbj+ 1 1 = N.

In such a case

la'bj+ll + lbjaj+iI = (N-1) 2 + N 2 < 2N 2 + 1

would still be short of satisfying (7). Notice that for any

other choice of aj, bi, aj+ 1 , b the condition (7) would beJ j+l

more severely violated. Also when jajI = lbj = N, it is not

possible for laj+ 11 = N, since aj+ 1 would become zero by the

algorithm in Table 1.

Thus a p-adic approximant (Hensel Code [5]) with the weight

w corresponds to the rational aj/b belonging to F N and the

conversion is complete.

Remarks

(i) The class of rationals generated by the above algorithm

may ccntain a rational (in non-reduced form) whose reduced

iI III. ..

form is in FN; but this is an invalid choice. (Se

example.)

(ii) If gcd(w, pr );l, the factor is taken out and the result

adjusted suitably.

Example

Let p=5, r=4, and w=448. Hence N<17. We now show in

Table 2 the computations corresponding to Table 1 of the algo-

rithm. The Farey rational is 11/7 (and not 5/60).

- ,1I

5. Matrix-inversion example

Let A = 3 2

01 -1

Let p = 3:

2 2

0 20]B 0 1 21 (mod 3)

0 22

8

1 0 2

B 8 4920 4921 1640 (mod 3 =6561)

2460 5741 4100]

10 1i80 (o 31643671

B 1 6 32285040 322C5041 10761680 (mod 3 =43C46721)

16142520 37665881 26904200

* i -----

We find that

1 0 1]

EUCLID(B1 6) = 3/4 1/4 - 1/4 = EUCL1D(B 8 )=A -

3/8 1/8 - 5/8J

Note that the inverse matrix elements are simultaneously deter-

mined in p-Adic digit parallel fashion with a quadratic rate of

convergence.

0 '

b. Solution of a system of linear equations by linear convergence

We now briefly consider the problem of determining the solu-

tion to a system of linear equations iteratively.

Let Ax=b be a system of linear equations such that det A mod

p 0, p being a prime. Let A=A 1 mod p and b,=b mod p. We first

solve A1 x M=blmod p by Gaussian elimination (say) and there-

after use the iterative scheme

x ( k + l ) = (p A11 M x ( k ) + A 1 b) mod p k+(k=l,2,...)

where A=AI-p M and M is the error matrix. We can easily show by

induction that

(Ax(k) - b) mod pk = 0.

Then, our algorithm is formally:

Step 1 Solve A1 x(1)=b mod p.

Step 2 Use x(k+l)=(p A-1 M x ( k ) + A,' b) mod pk+l to obtain

the next iterate.

Step 3 If EUCLID(xk ) = EUCLID(x k + ) stop; else go to 2.

Remark

Note that this scheme for the solution of linear equations

has only a linear order convergence. However, it has the ad-

vantage of using only matrix-vector multiplications unlike the

Newton iterative scheme where matrix-matrix multiplications are

involved.

7. Concluding remarks

(i) The scheme.Qf formula (3) gives rise to quadratic

convergence. It is possible to use schemes having

higher-order convergence. The following scheme,

for example,3n

B3n = B 3n- (I + (I - B3n-1) (21 - AB3n-1)) mod p (10)

has cubic convergence.

(ii) We have assumed throughout that det A mod p ?' 0, but in

actual computation we cannot assume this a priori. We

can keep choosing one prime after another until we suc-

ceed; but this is very expensive computationally. It

would be better to use the method of rank 1 update, which

is as follows:

We apply our algorithm to A+V instead of A where

a ll

V a :2 [bl,b 2 .... b = abt is arbitrarily (11)

L.n] chosen.

Finally, we use the formula

-1 -iA-1 = (A+V)-1 + (A+V) V(A+V) (12)

l-bt (A+V)-1 a

to retrieve the actual inverse. This method always suc-

ceeds except when A- =0 over Z and A mod p=0.

(iii) It is possible to extend the scope of our algorithm for the

determination of the g-inverse of a singular matrix.

(iv) The algorithm determines all the elements of the inverse

matrix simultaneosly in p-adic digit parallel fashion with

a quadratic or higher-order convergence rate [131.

(v) In solving a system of linear equations, we note that

we have split the matrix A in a very special way, name-

ly, A=A1 - p M. We could try splitting it as in the

Jacobi, Gauss-Seidel or SOR method (31; but unfortun-

ately, the convergence in our sense is not realizable in

these cases.

(vi) We can invert polynomial matrices whose elements are in

z [2] by constructing the inverses of the matrices Ztx]

mod pi for several primes pi and then using the Chinese

Remainder Theorem to construct the actual inverse [i].

OCOEE

(u i , u'! (a ,- bi qi (till t!)11 11 1

0 (rr, 0) (w, 1) [u o/W] (u0 - aoq0 , uo, - boq O )

1 (w, 1) (to , to) [ul/al] (U 1 - -blq l )

2 0 t, t) [u2/a2] (u2 -a 2q2, u2 -b lq2 )

k (uk, u) w [uk/ak (0, (-1) k+l pr

Table 1

Euclidean Algorithm

7-

u ') '!(a bi) qi (t i t t!)1 1 1 1 1

0 (625, 0) (448, 1) 1 (177, -1)

1 (448, 1) (177, -1) 2 (94, 3)

2 (177, -1 (94, 3) 1 (83, -4)

3 (94, 3) (83, -4) 1 (11, 7)

4 (83, -4) (11, 7) 7 (6, -53)

5 (11, 7) (6, -53) 1 (5, 60)

6 (6, -53) (5, 60) 1 (1, -113)

7 (5, 60) (1, -113) 5 (0, 625)

Table 2

Example of Euclidean algorithm

PO

' ._._ _,_ - ,, - -- .1

References

1. E. V. Krishnamurthy, Exact inversion of a rational poly-nomial matrix using finite field transforms, SIAM J.Appl. Math. 35, 453-464, 1978.

2. E. V. Krishnamurthy, T. M. Rao, and K. Subramanian, Residuearithmetic algorithms for computing g-inverses of matrices,SIAM J. Num. Anal. 13, 155-171, 1976.

3. J. Stoer and R. Bulrisch, An Introduction to NumericalAnalysis, Springer-Verlag, 1981.

4. E. V. Krishnamurthy, Economical iterative and range-trans-formation schemes for division, IEEE Trans. on Computers,Vol. C-20, 470-472, 1971.

5. E. V. Krishnamurthy, T. M. Rao and K. Subramanian, Finitesegment p-adic number systems with applications to exactcomputation, Proc. Indian Acad. Sci., Vol. 81A, pp. 58-79,February 1975.

6. E. V. Krishnamurthy, T. M. Rao, and K. Subramanian, p-adicarithmetic procedures for exact matrix computation, Proc.Indian Acad. Sci., Vol. 82A, pp. 165-175, November 1975.

7. E. V. Krishnamurthy, Matrix processors using p-adic arith-metic for exact linear computations, IEEE Trans. ComputersVol. C-26, pp. 633-639, July 1977.

8. R. T. Gregory, Error-Free Computation, Robert E. KriegerPub. Co., Huntington, NY, 1980.

9. R. T. Gregory, The use of finite segment p-adic arithmeticfor exact computation, BIT, Vol. 21, pp. 282-300, September1978.

10. D. E. Knuth, The Art of Computer Proqramming 2: Semi-numeri-cal algorithms,Addison-Wesley, Reading, MA, 1980.

11. E. V. Krishnamurthy, On the conversion of Hensel codes toFarey rationals, IEEE Trans. Computers (in press).

12. A. Ya.Khinchin, Continued Fractions, The University ofChicago Press, Chicago, IL, 1964.

13. G. Rodrigue, Parallel Computations, Academic Press, NY, 1982.

oft="-

, II

RIAD I RtUCi"QONSREPORT DOCUMENTATION PAGE IEFORE CMPLETING FORM*

-'- 2 GOVT ACCESSION NO. I PCCIPIL 'S CATAtOG NUIM R

£ TIT,[ I-dA SlbItj) S TYPE OF R(PORT 6 F'RIODCC)(VkED

L'Pi()'--}'REE PARALLEL HIGH-ORDER CONVERGENT TechicalIT'RATIVE MATRIX INVERSION BASED ON p-ADICAPP ROX IMATION I. PERFORMING ORG. Ri &RT NUMBTER

TR- -9/7 AuN~'~u. S.CONTR-ACT OR GRANT NU7m-BER(S)__

L. V. Krishnamurthy AFOSR-77-3271

9 P5 R O MING OR0AIZATION NAME AND ADDRESS 10 PROGRAM ELEMENT PROJECT, TASK

Colputer Vision Laboratory AREA 6 WORK UNIT NUMBERS

Coi, ;_ter Science Center i[iniversity of Maryland,olleqe Park, MD 20742 v,

II CONTROLLING OFFICE NAME AND ADDRESS 12. REPORT DATEMath. & Info. Sciences, AFOSR/NM November 1982

tm! ling AFB 13 NUMBER OF PAGESS!aEsington, DC 20332 18

il MONITORING AGENCY NAME & ADDRESSeal dillerant from ConIrolling Oflire) IS. SECURITY CLASS. (o this report)

UNCLASSIFIED

IS DECLASSIFICATION DOWNGRADINGSCHEDULE

16 I.!S. RIBUTION ST ATEMENT (of this Report,

lpproved for public release; distribution unlimited

17 DISTRIBUTION STATEMENT (of the abstrc, entered In Block 20. iI dlierent Iron. Report)

'B SIIPPL EMENTARY NOTES

K E V i ORDS (Continue on ,ereae side it necessar) and ide"it bloc number)

Parallel computation:atrix inversionp-adic approximation

2C AE7 RAC T (Conttnue an reverse. aide if necessary and Identify b, block number)

The Newton-Schultz iterative scheme is rcformulated in analgebraic settinq to compute the exact inverse of a matrix (or thesolution of a linear system of equations) over the ring of integers,with a high order of convergence, by using a finite segment p-adicrepresentation of a rational. This method is divergence-free; itstarts with the inverse of a given matrix over a finite field (callethe priming step) and then iterates successively to construct, in

Iaral1el, the p-ac artg (Hensel Codes) of the ration 1

DD 1473 EIDITION OF NOV5I IS CEBOLETE UNCLASSIFIED

____.. s r, i r , FL E c-Tier ,c, r

-- - . . . . l