604 OPTICS LETTERS / Vol. 14, No. 12 / June 15,1989

Adaptive optical processors

Anjan Ghosh

Department of Electrical and Computer Engineering, The University of Iowa, Iowa City, Iowa 52242

Received November 7, 1988; accepted March 14, 1989

There are two different approaches for improving the accuracy of analog optical associative processors: postpro-cessing with a bimodal system and preprocessing with a preconditioner. These two approaches can be combined todevelop an adaptive optical multiprocessor that can adjust the computational steps depending on the data andproduce solutions of linear algebra problems with a specified accuracy in a given amount of time.

The accuracy of matrix computations performed onhigh-speed analog optical associative processors(OAP's) is limited by the noise and spatial errors.There exist two different approaches for alleviatingthis limitation: postprocessing with a bimodal sys-tem' and preprocessing with a preconditioner.2 ,3 Inthis Letter it is shown that these two approaches canbe combined to develop an adaptive optical multipro-cessor that can adjust the computational steps de-pending on the data and the desired accuracy andspeed. The strategy described here can also be usedfor designing hybrid electronic multiprocessors.

The bimodal optical computing approach' is basedon the idea of using a high-speed analog optical pro-cessor coupled with a digital postprocessor to obtainthe accuracy of digital computing while still retainingthe speed and power advantages of analog optics. Theanalog processor is used to obtain an approximatesolution to a problem, and the digital processor is usedto improve iteratively the accuracy of the final solu-tion.

The preprocessing approach proposed by Ghoshand Paparao2 is based on the fact that the accuracy ofa computation depends on the data, the algorithm,and the hardware used. Thus the data used can beimproved through a preprocessing step before the fi-nal solution such that the results of the computationare less susceptible to noise and spatial errors in thehardware. This preprocessing step requires a highspeed and is tolerant to computational inaccuracies toa great extent. Thus preprocessing is suitable foranalog optical implementation. Matrix precondition-ing is the preprocessing algorithm2A4 that we find mostsuitable for a large class of engineering problems in-volving the solution of linear systems of equations.

Matrix preconditioning is a transformation for re-ducing the condition number C(A) of a matrix A.4The rate of convergence of iterative algorithms de-pends on C(A); the smaller the condition number, thefaster the convergence. In addition, computationalinaccuracies in problems involving matrix inversion orthe solution of linear algebraic equations are propor-tional to the condition number C(A) and the round-offerrors in the processor hardware. The efficacy of thebimodal approach also depends on the condition num-ber of the matrices involved: the smaller the condi-

tion number, the faster the bimodal process con-verges.' Thus preconditioning of matrix data beforethe final solution helps in reducing the time of compu-tation and improving the accuracy of the solution.

Preconditioning of a linear system of equations, Ax= b, involves the computation of a preconditioningmatrix M and the multiplication of both sides of theequations by M to obtain a modified system, MAx =Mb. The nonsingular matrix M is an approximationof A-' such that MA has a small condition numberC(MA) < C(A).4 This preconditioning is a robustoperation because the computational inaccuracies inM make it slightly less effective in preconditioning theoriginal data matrix A and do not affect the solution x= A-'b obtained by the main algorithm. Thus theeffects of errors and noise in optical processors on theresult of preconditioning are not important.

There are several methods for calculating a suitablepreconditioning matrix M.5 A popular method calledpolynomial preconditioning is based on calculation ofM as a splitting matrix for linear gradient methods,such as the successive-overrelaxation algorithm.4This method is suitable for parallel processors andallows us to control the degree of preconditioning byselecting a few parameters. The polynomial precon-ditioning method involves several matrix-matrix ormatrix-vector operations and thus is a time-consum-ing process when performed on serial digital comput-ers. These matrix multiplications can be carried outin the order of N or N2 steps on an optical processor,where Nis the dimension of the matrices.1 The speedup and the robustness of the preconditioning algo-rithms make the OAP's the ideal candidate for a paral-lel realization of matrix preconditioning.

Recently we developed a new and more efficientpolynomial preconditioning procedure called thesplit-step polynomial preconditioning (SSPPC) algo-rithm. In the SSPPC algorithm the polynomial pre-conditioning steps are repeated several times in twointerconnected loops. We split the basic polynomialpreconditioning process with a large number of itera-tions into several steps with fewer iterations in eachstep, each time starting with an improved matrix. Inthe SSPPC algorithm the condition number C(MA)decreases at the rate of Q(p-m), where m and p are thenumber of iterations in the outer and inner loops,

0146-9592/89/120604-03$2.00/0 © 1989 Optical Society of America

June 15,1989 / Vol. 14, No. 12 / OPTICS LETTERS 605

Table 1. SSPPC Algorithm and Its Variants

1. SSPPC algorithmStep 1: Initialize Di = AStep 2: Start the outer loop for i = 1, 2, ...

(a) Choose xi = ki/Xmax(Di) such that 0 < ki < 2(b) Formulate Gi = I - wiDi

Step 3: Evaluate the sum Si = ZF-ol Gii through an innerloop

Step 4: Calculate Di+, = wiSiDiStep 5: Loop through steps 2, 3, and 4 until i = m, where

m is the number of steps required to arrive at thedesired condition number

2. SSPPC-based algorithm for matrix inverse calculationStep 1: Initialize B1 = I, D1 = ASteps 2 and 3 are the same as in algorithm 1Step 4: Calculate Bi+1 = wiSiBi and Di+, = Bi+lAStep 5: Loop through steps 2,3, and 4 while III-Di+1 I >

E, where e is a small positive number thatdetermines the accuracy of the inverse

3. SSPPC-based algorithm for condition-numberestimation

Step 1: Initialize B1 = I, D1 = A and calculate IAil =Cest,(A), the initial estimate of the conditionnumber

Steps 2 and 3 are same as in algorithm 1Step 4: Calculate Bi+i = wiSiBi and Di+, = Bi+1AStep 5: Calculate Cesti(A) = IlBi+1i IIhAilStep 6: Loop through steps 2,3,4, and 5 while [Cestj(A) -

Cestii(A)I/Cesti(A) > e, where E is a small positivenumber

respectively. This rate is much faster than the O(p-')rate of a standard polynomial preconditioning meth-od. Our SSPPC algorithm is designed for realizationon parallel machines, especially OAP's. The SSPPCalgorithm is also versatile. With a few additionalsteps this algorithm can be used to calculate the in-verse of a matrix or to estimate the condition numberof a matrix6 on a parallel processor. The SSPPC andthe SSPPC-based algorithms for matrix inversion andcondition estimation are described in Table 1.

Algorithm 1 (the SSPPC) has two nested iterativeloops. The outer loop consists of steps 2-5. Theinner loop in step 3 constitutes a simple polynomialpreconditioning strategy. This inner loop is stoppedor split after every Pi iterations; the matrix Gi is reas-sembled and the loop is started again. The matrixDm+i represents the final preconditioned matrix MAdescribed above. Expressing Dmi+ as a polynomial ofthe original matrix A and with the parameters kh = 1and pi = an odd number p for i = 1 to m, it has beenshown6 that the final condition number is given by

C(Dm+i) = 1/f1 - [1 -11QA)]P'}. (1)

From Eq. (1) an estimate of the total number of outerloop steps can be evaluated depending on the desiredcondition number C(Dm+1 ), the original conditionnumber C(A), and the parameter p representing thenumber of iterations in each inner loop.

In algorithms 2 and 3 the matrix Bi+l represents aniterative approximation to A-'. Thus the norm of theerror (I - D+,) - 0 and Cesti(A) - C(A). The conver-gence of algorithms 2 and 3 can also be explained

through Eq. (1).6 Each of the three algorithms inTable 1 requires only O(mp) matrix-matrix multipli-cations and additions and therefore can be realizedefficiently on OAP's and parallel systolic processors.The convergence of the three algorithms with the pa-rameters pi = p = 5 and ki = 1 on a Vax computer withround-off errors 0(10-8) is shown in Fig. 1, where A isthe ill-conditioned (5 X 5) covariance matrix of a sim-ple phased-array radar. The vertical axis in Fig. 1depicts logarithms of the values of the performancemeasures of the three algorithms: (i) the reducedcondition number C(Di) obtained from algorithm 1,(ii) the error norm III - DilI representing the deviationof the iterated inverse Bi from A-', and (iii) the esti-mated condition number Cesti(A).

Using the SSPPC algorithm we can build an adap-tive interconnected optical system for linear algebraprocessing as shown in Fig. 2. It is a pipelined systemthat can be adjusted for a specified amount of accura-cy and computational time to solve simultaneous lin-ear algebraic equations. Simple analog OAP's areused throughout the system for their O(N2) advan-tages in speed, parallelism, and interconnection capa-bilities. Since this is an adaptive system with efficientsoftware, costly high-accuracy hardware is not re-quired.

In the first step, this processor estimates the condi-tion of the input problem to determine whether somepreprocessing is required before giving the input datato the main processor. This initial decision-makingprocess involving an on-line estimation of the condi-tion number (based on algorithm 3 in Table 1) can berealized on a two-dimensional OAP in O(mpN) timesteps. 3

Then the condition-number estimate is comparedwith a threshold value, Cth, representing the type ofdata that can be processed accurately in a givenamount of time. If the estimated Cest(A) is greaterthan the threshold, the input data are preprocessedusing the SSPPC method. Otherwise the input goesdirectly to the main processor for the final solution x =A-lb. Depending on the estimated condition number

0 2 4 6 8 10 12

NO. OF OUTER LOOP ITERATIONS

Fig. 1. Convergence of the three algorithms in Table 1.

606 OPTICS LETTERS / Vol. 14, No. 12 / June 15, 1989

Fig. 2. Adaptive optical linear algebra processor with pre-processing and postprocessing.

and the accuracy and the speed required, we also de-cide the need for either a bimodal solution with itera-tive refinement or a simple one-step solution. Thethreshold Cth can be derived from the convergencecharacteristics of the algorithm implemented in themain processor. Thus the second stage of the intelli-gent processor is an OAP realization of the SSPPCalgorithm to improve the data through precondition-ing. The condition estimation and the precondition-ing computations in the first and second stages can becombined with an efficient data flow pattern. Thusboth of these operations can be performed together inO(mpN) time steps.

The final stage is the main analog OAP coupled to adigital postprocessor. If a high accuracy is desired inthe final solution the digital unit is turned on and thecomputations are performed in bimodal fashion' for

improving the final accuracy. In this stage we canutilize the SSPPC-based algorithm 2 for matrix inver-sion or any other algorithm for solving a linear systemof equations. The amount of postprocessing to beused, i.e., the number of iterations in the bimodal loop,can be estimated from the required accuracy and com-putational time and the estimated condition number.If preconditioning is performed the number of stepsrequired for producing a solution with a given accura-cy is reduced. In our experiments with the covariancematrix data mentioned above we observed that if theroot-mean-square value of the spatial errors and noisein the optical system is 0.1 and if C(A) = 3633, we need17 bimodal iterations to reduce the error in the residu-al hIA (bimodal solution) - bhl to below 10-3. On theother hand, if data are preconditioned such that D(D5 )= C(MA) = 6, we need 6 bimodal iterations to reducethe residual to the same level.

Research on the design and tuning of such an adap-tive interconnected analog optical processor is in pro-gress in our laboratory. Similar adaptive strategiesare also being developed for calculating the eigenval-ues and eigenvectors of matrices.

This research is supported by grant EET-8707863from the National Science Foundation. The help ofP. Paparao in the experiments and simulations isgratefully acknowledged.

References

1. H. Caulfield, J. Gruninger, J. Ludman, K. Steiglitz, H.Rabitz, J. Gelfand, and E. Tsoni, Appl. Opt. 25, 3128(1986).

2. A. Ghosh and P. Paparao, Appl. Opt. 26, 2734 (1987).3. A. Ghosh and P. Paparao, J. Opt. Soc. Am. A 5,39 (1988).4. G. Golub and C. VanLoan, Matrix Computations (Johns

Hopkins U. Press, Baltimore, Md., 1983).5. J. Ortega and R. Voigt, SIAM Rev. 27, 149 (1985).6. A. Ghosh and P. Paparao, Special issue on optical com-

puting, Opt. Eng. 28, 354 (1989).