398 J. Opt. Soc. Am. A/Vol. 1, No. 4/April 1984

Space-invariant deblurring given N independently blurredimages of a common object

Dennis C. Ghiglia

Sandia National Laboratories, Organization 2644, Albuquerque, New Mexico 87185

Received September 19, 1983; accepted November 28, 1983

An algorithm is developed for space-invariant deblurring of an object given several differently blurred images ofthat object. Given some control over the creation of the individual blurred images, the restoration instabilities as-sociated with the transmission zeros can be mostly eliminated. The algorithm provides improved performancewith noisy images and also provides some insensitivity to blur model errors. It reduces in form to the Wiener filteror constrained least-squares filter for the single-image case.

INTRODUCTION

An application at Sandia National Laboratories dealing withfacilities surveillance requires restoration of images of objectsblurred by uniform motion in a plane at an oblique angle tothe camera line of sight.

This paper addresses an interesting sidelight of theabove-mentioned application, that of restoring linear,space-invariant blurred images given N independently blurredimages of the same object. This problem arises when it isphysically possible to arrange for the simultaneous exposureof a moving object with N independent cameras. It is as-sumed that some control over the exposure time Ti exists foreach camera, but it is not possible, because of lighting limi-tations, to make the exposure time short enough to obtain asharp image.

The question is then: Given that one can obtain N inde-pendently blurred images of a moving object, what can be doneto ensure the best restoration of the object, and how shouldthe restoration be formulated? Before delving into thisproblem it is to be noted that there is vast literature on bothoptical and digital image-deblurring techniques. However,this paper deals strictly with digital implementation. It willbe assumed that the reader is familiar with the basic conceptsof digital image deblurring and that the various referencesshould be consulted for a more-detailed treatment. 1' 2 Onlythe basic concepts as applied to the problem at hand arebriefly presented in this paper.

ALGORITHM DEVELOPMENT

I develop the restoration based on the constrained least-squares approach in the frequency domain because, as isshown below, it leads to a restoration transfer function withan interesting similarity to the conventional frequency-domain Wiener filter. The basic linear space-invariantimaging equation can be written in the frequency domainas3

G(w) = F(w)H(w) + Z(w), (1)

where G(w), F(w), H(M), and Z(w) are the Fourier transformsof the blurred image, the unblurred image, the point-spread

function, and the assumed additive noise process, respec-tively.

The Wiener filter transfer function can be shown to be4

H.(c) = H*(w)/[IH(W)J2 + p,(C0)/pf(W)], (2)

where P, and Pf are the power spectrum of the noise andsignal, respectively, and the * denotes the complex conjugate.H,,, (w), when multiplied by G(co), the transform of the blurredimage and inverse transformed, yields the least-mean-squareestimate of the object over the ensemble of possible noisefields. In other words, the Wiener filter is mathematicallyoptimal on the average but may be nonoptimal for any givenimage. In contrast, the constrained least-squares restorationcan be made mathematically optimal for any given image. 5

For the multiple-image case of interest, let F(w), Gi (w), andHi(w) be the Fourier transforms of the unblurred, the ithblurred image, and the ith point-spread function, respectively.[It is to be noted that the ith point-spread function resultsfrom photographing a moving object with the ith camera,whose exposure time is Ti, and not from the optical charac-teristics (such as diffraction) of the camera.] It is to be un-derstood that w can be taken as a multidimensional frequencyvariable if desired, but one-dimensional representations willbe written here for simplicity.

The restoration problem can be formulated by minimizingobjective functions of the form

W[F(w)J = J'JQ[F(w)112dw

+ N+ f a E_ I Gi (w) -Hi (w)F(W)l 2dw,

Z=1(3)

where Q( ) is some linear function of its argument of theform

Q[F(w)] = A (w)F(w) + BM(). (4)

Taking the derivative of W with respect to F and setting theresult equal to zero produces two equations, each the complexconjugate of the other (because there are squared magnitudesin the objective function equation above). Selecting theequation containing the nonconjugate F's yields the transformdomain image estimate

0740-3232/84/040398-05$02.00 C 1984 Optical Society of America

Dennis C. Ghiglia

Dennis C. Ghiglia

N [NF,() = Li Hi* (w)Gi(W) E JHi('W)12

i=1 ti=i

+ a [A(co)l2 + A(w)B(cw)]b (5)a I

with a adjusted such that a desired constraint is satisfied.The single-image constrained restoration problem is discussedin Ref. 6, and a method is presented for adjusting a. However,in practice, the best value theoretically for a is usually re-placed by the value that produces the best visual restoration.This is equivalent to the common practice of replacing theratio of the noise-to-signal power spectra in the Wiener filterby a constant that can be adjusted as necessary. 7 Thus a canbe used as a tuning parameter to help optimize visualclarity.

It should be noted that the multiple-image restorationprocess reduces in form to the Wiener filter or the constrainedleast-squares filter for the single-image case.

Having more than one blurred image of a common objectprovides information that can be used to advantage in therestoration process. As is shown in the following examples,it appears that there are three distinct advantages to havingmultiple images. The first is in the area of noise reductionthrough statistical independence between image noise fields.The second is the potential elimination of zeros in the de-nominator of the restoration filter [Eq. (5)] by proper selectionof the Hi(w)'s or by requiring just one Hi (w) to have non-common zeros with all other Hi (w)'s. The third advantageappears to be a decrease in sensitivity of the restoration pro-cess to model errors [i.e., knowing only approximately the formof each Hi (w)].

IMPLEMENTATION AND EXAMPLES

I believe it is best at this point to try to substantiate theseconjectures through visual examples. The following examplesall deal with one-dimensional deblurring of simulated hori-

Vol. 1, No. 4/April 1984/J. Opt. Soc. Am. A 399

zontal motion blur of an image of some text. The original512-pixel by 512-pixel image was blurred by a horizontalone-dimensional moving-window average; the width of thewindow in pixels describes the severity of the blur. Inde-pendent uniform noise fields were then added to each blurredimage. The noise variance was chosen to produce a signal-to-noise ratio (SNR) equal to unity in the low-SNR examplesin which the SNR is defined as the ratio of peak signal to rmsnoise. No noise was added to the blurred images in thehigh-SNR examples.

It should be noted that the transform-domain restored-image estimate was obtained by replacing the constrainingterm

- [IA(W)12 + A(w)B(w)]a

in the denominator of Eq. (5) with a single constant K, whichwas then adjusted to provide the best visual restoration.

The restoration algorithm was implemented as follows:

(1) Compute the Fourier transform of the ith blurredimage for i = 1-N. This produces Gi (w).

(2) Compute the Fourier transform of the ith estimatedpoint spread function for i = 1 through i = N. This producesHi (w).

(3) Select a tuning parameter K.(4) Compute the transform-domain image estimate:

N I NFe(CO) = X Hi*(co)Gi(w) IEH-i(w)12

+ KJ.

(5) Compute the inverse Fourier transform of Fe(w).This produces the restored image.

(6) Evaluate the quality of restoration and go to step (3)if necessary.

Figure 1 shows the original unblurred image. The signalvariance was computed to be approximately 65.3. Sevenblurred images were produced with blurs of 23, 29, 35, 43, 47,53, and 59 pixels, respectively. Figure 2 shows the least se-

Fig. 2. Least-severely blurred (23-pixel blur) noiseless image.Fig. 1. Original unblurred noiseless image.

400 J. Opt. Soc. Am. A/Vol. 1, No. 4/April 1984

nator of Eq. (5). In fact, the restoration is essentially perfect,as can be seen by comparison with Fig. 1.

Figure 6 depicts the least severely blurred (23-pixel blur)noisy image of the seven images available. The restorationof this single image with K = 0 is shown in Fig. 7. As wouldbe expected, the restoration is totally unstable. The bestWiener restoration of Fig. 6 is obtained with K = 0.1 and isshown in Fig. 8. The restoration is quite good, consideringthe quality of the input. The text, for the most part, appears

Fig. 3. Most-severely blurred (59-pixel blur) noiseless image.

Fig. 5. Restoration obtained using all seven noiseless images withK = 0.

Fig. 4. Restoration of the 23-pixel blur noiseless image using theinverse filter (K = 0).

verely blurred noiseless image (23-pixel blur), and Fig. 3 showsthe most severely blurred (59 pixels) noiseless image. Theadvantage gained by having multiple blurred images withnoncommon transmission zeros is illustrated by the next twofigures. Figure 4 shows the restoration performed on Fig. 2with K = 0 (single-image inverse filter). The obvious ringingis due to the instabilities associated with the inversion of smallspectral components, as is well known. Figure 5 shows therestoration obtained by using all seven images with K = 0. Noringing is evident because there are no zeros in the denomi-

Fig. 6. Least-sevefely blurred noisy image (23-pixel blur; SNR =1.0).

D~ennis C. Ghiglia

Vol. 1, No. 4/April 1984/J. Opt. Soc. Am. A 401

Fig. 7. Restoration of the 23-pixel blur noisy image using the inversefilter (K = 0).

all seven images are used in the restoration, a noticeable im-provement can be seen, as is evident in Fig. 10. Compare, forexample, the sharpness of the word differences in the thirdline of text in Figs. 8-10.

It is also important to note that the multiple-image resto-ration algorithm appears to provide some insensitivity tomodel errors. In other words, precise knowledge of the in-dividual blurs may not be necessary to produce an adequaterestoration. This is to be contrasted with the ringing and lossof sharpness that usually results from single-image restorationmethods.

Fig. 9. Restoration using the 23- and 29-pixel blur noisy images withK = 0.1.

Fig. 8. Restoration of the 23-pixel blur noisy image using the Weinerfilter (K = 0.1).

readable but grainy and not exceptionally sharp, as might beexpected from a low-SNR input and lost spectral componentsthat are due to the blur transmission zeros. Figure 9 showsthe restoration obtained using two images with blurs of 23 and29 pixels and K = 0.1. This two-image restoration offers someimprovement, as can be seen by comparing selected letters andwords with those of Fig. 8. Not a great deal was gained withthe addition of another image, probably because of the closesimilarity between the blurs of both pictures. However, when Fig. 10. Restoration using all seven noisy images with K = 0.1.

Dennis C. Ghiglia

402 J. Opt. Soc. Am. A/Vol. 1, No. 4/April 1984

t~~~ r

Fig. 11. Restoration using all seven noisy images withK = 0.1anddeliberate model errors introduced.

Fig. 12. Restoration using all seven noiseless images with K = 0 anddeliberate model errors introduced.

Figure 11 depicts the restoration obtained when deliberateerrors in the blur widths are introduced before the restorationusing all seven noisy images. The actual blurs were thosestated earlier, but the blur models used were 21, 31, 33, 45, 49,51, and 55 pixels. The quality of the restoration is not no-ticeably affected. In fact, the effect of the these particularmodel errors is best seen by performing the restoration on thehigh SNR imagery. Figure 12 shows that the model errorsintroduce some ringing toward the edge of the image, but theoverall quality is not seriously degraded. Several other res-torations were performed with different combinations ofmodel errors with similar results.

CONCLUSION

Having multiple blurred images of a common object appearsto provide information that can be used to advantage in imagerestoration. The constrained least-squares formulation inthe frequency domain provides a means for estimating theobject from the multiple images in a way that allows for effi-cient human intervention in addition to the ability to elimi-nate or greatly reduce the problems associated with zeros inthe blur transfer functions. The restoration equation [Eq.(5)] reduces in form to the commonly used Wiener filter forthe single-image case, thereby providing a well-known baselineby which the improvement in restoration by incorporatingadditional images can be judged. It was demonstrated thatthe multiple-image restoration algorithm seems to providesome insensitivity to model errors that will always be presentin any practical application. Of course, additional work isnecessary to determine what gains might be expected givena certain distribution of blurs, SNR's, registration errors, etc.before this method can be established as a practical tool.

ACKNOWLEDGMENTS

I wish to express my thanks to Charles V. Jakovwatz, Jr.,Eugene A. Aronson, and Gary A. Mastin for their interest andcontinuing technical contributions. This research was sup-ported by the U.S. Department of Energy.

REFERENCES

1. H. C. Andrews and B. R. Hunt, Digital Image Restoration(Prentice-Hall, New Jersey, 1977).

2. R. C. Gonzalez and P. Wintz, Digital Image Processing (Addi-son-Wesley, Reading, Mass., 1977).

3. Ref. 1, p. 81.4. Ref. 1, pp. 137 and 151.5. Ref. 2, p. 209.6. B. R. Hunt, "The application of constrained least squares esti-

mation to image restoration by digital computer," IEEE Trans.Comput. C-22, 805-812 (1973).

7. Ref. 2, p. 211.

El, "", �!

U

� , I II

,i�,

� "� � "', I �, � " ,

Dennis C. Ghiglia