Spatially efficient reference phase-encryptedjoint transform correlator

AbdulRahman AlsammanDepartment of Electrical Engineering, University of New Orleans,

Lakefront Campus, New Orleans, Louisiana 70148, USA(aalsamma@uno.edu)

Received 6 October 2009; revised 1 February 2010; accepted 9 March 2010;posted 9 March 2010 (Doc. ID 118074); published 31 March 2010

A novel joint transform correlator (JTC) system is presented in which the stored reference image is phaseencrypted prior to applying the JTC. The encryption disperses all the trivial correlation peaks in thecorrelator output. The reference image is encrypted electronically, which simplifies the need for a com-plex optical setup. The encryption removes the JTC requirement for spatial separation between the re-ference and the target images in the joint input plane. This efficient use of spatial light modulator spacecan be used to phasemultiplexmore reference images so that correlations withmultiple reference imagescan be performed in a single JTC cycle. © 2010 Optical Society of America

OCIS codes: 070.4550, 070.5010.

1. Introduction

In the classical joint transform correlator (JTC) [1],correlation is performed on spatially multiplexedreal-valued images. The JTC does not require thecomputation and display of complex-valued images.Furthermore, it does not impose the alignment re-strictions of the VanderLugt correlator [2]. However,the JTC output contains a large zero-order term andis cluttered with overlapping correlation peaks.These effects can be reduced by increasing thedisplacement between the images, but the spatiallimitation of the joint input scene makes this imprac-tical. A variety of solutions have been proposed toreduce clutter and eliminate the zero-order termsuch as Fourier plane image subtraction [3], chirp-encoded JTC [4], and phase-encoded JTC techniques[5,6]. However, none of the proposed techniquesresulted in a more efficient use of spatial light mod-ulator (SLM) space. More recently, the JTC with spa-tial code division multiplexing [7] was proposed inwhich the input and test image are interlaced inthe input plane, thus removing the need for displace-ment between them. While this does yield a more ef-

ficient use of the SLM space, it requires a structureduse of the SLM and CCD space that is restrictive,especially when additional reference images areused.

This paper presents a novel phase-encrypted JTCthat discards all extraneous peaks and removes theneed for spatial multiplexing. The reference is phaseencrypted electronically offline and stored digitallyto remove the need for a complex optical setup.The phase encryption has been demonstrated bythe use of a phase-modulating SLM such as liquid-crystal TVs [8–11]. The phase encryption scattersthe zero-order term and the cross-correlation conju-gate in the correlation output plane, while preser-ving the cross-correlation term. Accordingly, theneed for spatial multiplexing is eliminated, resultingin efficient use of the SLM. Furthermore, the systemsupports the correlation of a single input image withmultiple reference images simultaneously in a singleJTC cycle, as long as each reference image is en-crypted with a unique phase sequence. We show thatthe number of simultaneous correlations is propor-tional to the SLM size and limited by the correlationenergy in the correlation output.

In Section 2 a review of the classical JTC theory isshown. The spatially efficient reference phase-encrypted JTC is presented in Section 3. In Section 4

0003-6935/10/10B104-07$15.00/0© 2010 Optical Society of America

B104 APPLIED OPTICS / Vol. 49, No. 10 / 1 April 2010

the proposed opto-electronic architecture of the sys-tem is described. The results and analysis of the out-put of the proposed system are shown in Section 5.Finally, conclusion statements are given in Section 6.

2. Classical Joint Transform Correlator

In a single-reference JTC, a single-reference (stored)image is correlated with a target (input) image. Thetarget and reference images are placed side by side(spatially multiplexed) in the same input plane as

f ðx; yÞ ¼ tðx; yþ y0Þ þ rðx; y − y0Þ; ð1Þ

where f ðx; yÞ is the input joint image. The referenceimage, rðx; yÞ, and the target image, tðx; yÞ, are sepa-rated by displacement 2y0. The displacement can betwo dimensional but, for simplicity, Eq. (1) reflects adisplacement only in the y axis. The Fourier trans-form of the input joint image can be expressed as

Fðu; vÞ ¼ Tðu; vÞ exp½−jy0v� þ Rðu; vÞ exp½jy0v�; ð2Þ

where the terms Tðu; vÞ and Rðu; vÞ are the Fouriertransforms of rðx; yÞ and tðx; yÞ, respectively. The spa-tial displacement in Eq. (1) translates to the phaseterms exp½−jy0v� and exp½jy0v�. The intensity of theFourier transforms, known as the joint power spec-trum (JPS), is expressed as

Pðu; vÞ ¼ jRðu; vÞj2 þ jTðu; vÞj2þ Rðu; vÞT�ðu; vÞ expðj2vy0Þþ R�ðu; vÞTðu; vÞ expð−j2vy0Þ; ð3Þ

where Pðu; vÞ is the JPS and � denotes the complexconjugate operation. The inverse Fourier transformis applied to Eq. (3) to produce the correlation outputgiven by

cðx; yÞ ¼ rðx; yÞ ⊗ r�ðx; yÞ þ tðx; yÞ ⊗ t�ðx; yÞþ rðx; y − 2y0Þ ⊗ t�ðx; y − 2y0Þþ r�ðx; yþ 2y0Þ ⊗ tðx; yþ 2y0Þ; ð4Þ

where cðx; yÞ is the correlation output and ⊗ repre-sents the correlation operation. The first two terms ofthe correlation output represent the autocorrelationof the reference image and the test image; the lasttwo terms represent the cross-correlation conjugateterms of interest. It is clear from Eq. (4) that displa-cement of 2y0 in the input scene will result in a dis-placement of 4y0 in the corresponding correlationoutput. This means that the minimum SLM sizeneeded to correlate two N ×N pixel images is2N × 2N.In the case of a match, i.e., rðx; yÞ ¼ tðx; yÞ, the

cross-correlation term captured by the square-lawdevice can expressed as

jcð0; 2y0Þj2 ¼ j R Rrðx; yÞ2dxdyj2

r2max: ð5Þ

The massive parallelism of photonic systems can beexploited to increase the processing power of theJTC. Placing multiple reference images in the inputjoint image plane allows a single target image to becorrelated with multiple reference images in a singlecorrelation step. This is extremely useful in classifi-cation problems such as biometric identification ortarget detection in which an input image is to becompared with a large database of images.

If multiple reference images are used, each refer-ence image riðx; yÞ is displayed side by side with thetest image tðx; yÞ in the input plane of a JTC in theform of an input joint image as follows:

f ðx; yÞ ¼ tðx; yþ y0Þ þXNi¼1

riðx; y − yiÞ: ð6Þ

As in Eq. (1), displacement along the y axis is consid-ered only to simplify the derivation. The JPS ofEq. (6) can be expressed as

Pðu; vÞ ¼ jTðu; vÞj2 þXni¼1

jRiðu; vÞj2

þXni¼1

Riðu; vÞT�ðu; vÞ expðj2vðy0 − yiÞÞ

þXni¼1

R�i ðu; vÞTðu; vÞ expð−j2vðy0 − yiÞÞ

þXni¼1

Xi≠knk¼1

R�i ðu; vÞRkðu; vÞ expðj2vðyi − ykÞÞ

þXni¼1

Xi≠knk¼1

Riðu; vÞR�kðu; vÞ

× expð−j2vðyi − ykÞÞ: ð7ÞThe first line of Eq. (7) describes the zero-order (auto-correlation) terms. The cross-correlation term of in-terest between the reference and the input is onthe third line; its mirror (conjugate) is on the thirdline. The last two lines of the equation describe thecross correlation between all the reference images(clutter). In its present form the JPS is cluttered withcross-correlation and autocorrelation peaks thatcannot be distinguished from one another easily.Furthermore, the high zero-order terms will threa-ten to overshadow the cross-correlation terms.

Removal of the autocorrelation and the crosscorrelation among all the reference images can beaccomplished using a number of techniques [3–6].However, these require online processing of the inputimage and/or considerable digital processing. None ofthe proposed techniques is designed to improve thespatial use of the SLM.

It is seen from the above derivation that the JTCsystem uses real-valued spatial domain inputs and

1 April 2010 / Vol. 49, No. 10 / APPLIED OPTICS B105

does not require complex-valued frequency domainfilter fabrication. Thus it relaxes the requirementsfor meticulous alignment needed in matched filterbased correlators. However, the autocorrelationterms seen in Eq. (7) produce a high zero-order peakin the correlation output. This can overshadow thedesired cross-correlation peaks, especially in the pre-sence of noise or distortion. Furthermore, as the dis-placement between the two images decreases, thezero-order term of the correlation output starts todominate the output plane and overshadow thecross-correlation peaks. Ideally, a large displacementbetween the two images is desired. However, this isexpensive because larger SLMs are costly.

3. Spatially Efficient Reference Phase-Encrypted JointTransform Correlator

Consider a random phase mask defined as

Φðu; vÞ ¼ exp½jψðu; vÞ�; ð8Þwhere ψðu; vÞ is a random phase function in the Four-ier domain, normally distributed between −π and π.The process of phase encrypting the reference imageis achieved bymultiplying the reference image by thephase mask in the Fourier domain. Hence, theexpression for the input joint image in the spatialdomain becomes

f ðx; yÞ ¼ tðx; yÞ þ rðx; yÞ ⊗ φðx; yÞ; ð9Þwhere φðx; yÞ is the spatial domain transformation ofthe phase mask. There are three things to note here.First, phase encrypting is applied to the stored refer-ence image and not the input target image, which re-sults in a simpler opto-electronic architecture as willbe seen later. Additionally, the phase encryption pro-cess, i.e., rðx; yÞ ⊗ φðx; yÞ, can be computed a prioriwhile the system is offline and as such will not delaythe processing. Second, the target and referenceimages are not spatially multiplexed, i.e., the y0 dis-placement seen in Eq. (1) is no longer required. Sincethe phase encryption will eliminate the zero orderand the cross-correlation conjugate, there is no needfor the displacement as will be seen below. Finally,the phase encryption by a normally distributed ran-dom phase mask scatters the reference image into anoiselike function.The Fourier transform of Eq. (9) can be expressed

as

Fðu; vÞ ¼ Tðu; vÞ þ Rðu; vÞΦðu; vÞ: ð10ÞThe corresponding JPS is

Pðu;vÞ¼ jTðu;vÞj2þjRðu;vÞj2þTðu;vÞR�ðu;vÞΦ�ðu;vÞþT�ðu;vÞRðu;vÞΦðu;vÞ: ð11Þ

The JPS in Eq. (11) is then multiplied by the phasemask yielding

PΦ ¼ jTj2Φþ jRj2Φþ TR� þ T�RΦ2: ð12Þ

The ðu; vÞ coordinates in Eq. (12) are omitted for con-ciseness. The inverse Fourier transform is applied toEq. (12) to produce the correlation output given by

c ¼ r ⊗ r� ⊗ φþ t ⊗ t� ⊗ φþ t ⊗ r�

þ t� ⊗ r ⊗ φ ⊗ φ; ð13Þwhere the ðx; yÞ coordinates are omitted for concise-ness. In Eq. (13) All the correlation terms are affectedby the phase mask except for the cross-correlationterm, t ⊗ r�. Thus, the cross correlation betweenthe reference image and the test images is pre-served, whereas the remaining zero-order and cross-correlation terms are scattered by the phase maskinto system noise.

For multiple references, separate phase functionscan be used and Eq. (9) can be modified to

f ¼ tþXni¼1

ri ⊗ φi: ð14Þ

The corresponding JPS is

P ¼ jTj2 þXni¼1

jRij2 þXni¼1

RiT�Φi þXni¼1

R�i T�

i

þXni¼1

Xi≠knk¼1

R�i Rk�

iΦk þXni¼1

Xi≠knk¼1

RiR�kΦiΦ�

k: ð15Þ

The random phase mask scatters all the peaks, andthe corresponding correlation plane appears noise-like. To obtain correlation between a pth referenceimage, rpðx; yÞ, and target image tðx; yÞ, the JPS ismultiplied by the corresponding phase maskΦpðu; vÞ:

PΦp ¼ jTj2Φp þXni¼1

jRij2Φp þ R�pT þ

Xni¼1

RiT�ΦiΦp

þXi≠pni¼1

R�i T�

iΦp þXni¼1

Xi≠knk¼1

R�i Rk�

iΦkΦp

þXni¼1

Xi≠knk¼1

RiR�kΦiΦ�

kΦp: ð16Þ

The corresponding correlation output is

c ¼ t ⊗ t� ⊗ φp þXni¼1

ri ⊗ r�i ⊗ φp þ t ⊗ r�p

þXni¼1

ri ⊗ t� ⊗ φp ⊗ φp þXi≠pni¼1

r�i ⊗ t� ⊗ φ�i ⊗ φp

þXni¼1

Xi≠knk¼1

r�i ⊗ rk ⊗ φ�i ⊗ φk ⊗ φp

þXni¼1

Xi≠knk¼1

ri ⊗ r�k ⊗ φi ⊗ φ�k ⊗ φp: ð17Þ

B106 APPLIED OPTICS / Vol. 49, No. 10 / 1 April 2010

Thus, the cross correlation between the referenceimage p and the test images will be preserved,t ⊗ r�p, whereas the remaining zero-order and cross-correlation terms are scattered by the phase mask.There are advantages to this approach. First, itcorrelates a single input image with many multiplereference images in one opto-electronic process,which makes it powerful for classification problems.Second, the phase masks are computed and appliedto the reference images a priori and stored offline,which gives it a simpler architecture and makes itadvantageous for real-time processing.

4. Optical Implementation

An implementation of the reference phase-encryptedJTC is shown in Fig. 1. The light from source LS iscollimated by lens L1. The phase-encrypted refer-ence images are added to the test image to formthe joint input image. The joint input image is dis-played on SLM1. SLM2 is a phase-only JTC usedto depict the phase part of the reference image. LensL2 produces the Fourier transform and the JPS isthen captured by the square-law CCD device. TheJPS is routed back through the digital interface toSLM1. The phase mask is then displayed on SLM2to perform the operation described in Eq. (12). L2is then used to produce the correlation output thatis then captured by the CCD. The optical implemen-tation shown in Fig. 1 is advantageous over otherphase-encoded JTC implementations. By phase en-crypting the reference image, the number of opticalelements is kept to a minimum, which is more desir-able to limit distortions caused by misalignment andsystem noise.

5. Experimental Results

In the first test, the classical JTC is comparedwith theproposed reference phase-encrypted JTC. Using aclassical JTC system, an image containing both afighter plane anda commercial plane is usedan input;the fighter plane alone is used as a reference image.The two images are placed side by side as seen inFig. 2(a). The JTC output seen in Fig. 2(b) containsa dominant DC component (in the center) thatovershadows the cross-correlation peaks. A three-dimensional depiction of the output shown in Fig. 2(c)indicates that the DC term is more than five timesmore powerful than the cross-correlation peaks.Because of the physical displacement between theinput and the reference, the correlation plane is morethan twice as large as the images used.The same experiment is applied to the phase-

encrypted JTC. The input image containing thetwo planes is seen in Fig. 3(a). The reference imageof the fighter plane is phase encrypted according toEq. (9). The phase encryption scatters the referenceimage as shown in Fig. 3(b). When the phase functionis applied to the correlation output, cross-correlationterms are preserved and the DC term is scattered, ascan be clearly seen in Figs. 3(c) and 3(d). As an added

bonus, the correlation plane is the same size as theinput images.

Next, the phase-encrypted JTC is tested using aninput image that contains multiple signals. Theletters AEFT are used as the input image and theletter E is used as the reference image [Figs. 4(a)and 4(b)]. The reference image is phase encrypted

Fig. 1. (Color online) Multiple phase-encoded reference JTC.

Fig. 2. (Color online) Classical JTCwith a single reference image:(a) joint input image contains input on the left and a referenceimage on the right, (b) correlation output, (c) three-dimensionaldepiction of the correlation output.

1 April 2010 / Vol. 49, No. 10 / APPLIED OPTICS B107

and added to the test image to form the joint inputimage. The absolute value of the joint input imageis seen in Fig. 4(c). The correlation intensity pro-duced by the system is shown in Fig. 4(d), and athree-dimensional plot of the correlation intensityis shown in Fig. 4(e). As seen in Fig. 4(d), the extra-neous peaks are scattered uniformly over the corre-lation plane resembling noise. In effect, the totalenergy of the system is not changed, which meansthat, for larger SLM sizes, the noise would spreadover a larger area causing a high signal-to-noise ratio(SNR). For an M ×N SLM, the resulting SNR in thecase of a match, i.e., rðx; yÞ ¼ tðx; yÞ, is

SNR ¼

���R R r2dxdy

���2r2max

2MN

����R R r2dxdy

���2r2max

� : ð18Þ

Fig. 3. (Color online) Phase-encrypted JTC with a single refer-ence image: (a) test input image, (b) fighter plane phase-encryptedreference image, (c) correlation output after applying the phasefunction to the JPS, (d) three-dimensional depiction of the correla-tion output.

Fig. 5. (Color online) Normalized correlation intensity for inputtest AEFT and reference image E using various SLM sizes M: (a)N ¼ 128; (b) N ¼ 256; (c) N ¼ 512.

Fig. 4. (Color online) Phase-encoded reference JTC: (a) referenceE image, (b) input test image of letters AEFT, (c) absolute value ofthe joint image containing the test image and the reference phase-encoded image, (d) correlation intensity, (e) three-dimensionalcorrelation intensity.

B108 APPLIED OPTICS / Vol. 49, No. 10 / 1 April 2010

Thus, a larger SLM improves the quality of thecorrelation output.Next, the effect of the SLM size on the quality of

the correlation results is investigated. When theSLM sizes are increased from 128 × 128 to 256 ×256 and 512 × 512, the quality of the correlation out-put is notably improved, as shown in Figs. 5(a)–5(c).The correlation plots are limited to a 128 × 128 cor-relation area so that a qualitative comparison can bemade among all three correlation outputs. The in-crease in SLM size allows the extraneous peaks toscatter over a larger area, thus reducing the effectof the noiselike scattering. The output producedusing a 512 × 512 SLM, as shown in Fig. 5(c), isnearly perfect.Additionally, a number of statistics were collected

for a variety of SLM sizes and are listed in Table 1.As expected, the correlation peak intensity (CPI)undergoes minor changes as the SLM size is varied.The peak-to-sidelobe ratio (PSR) is evaluated in a64 × 64 window surrounding the maximum peak.The PSR shows improvements for larger SLM sizes.The anomalous PSR value at 448 × 448 is due to theless than ideal statistical properties of the phasemask. The peak-to-correlation energy (PCE) is col-lected from the 128 × 128 window in the center ofthe correlation output and shows little change asthe size of the SLM changes. The SNR is the ratioof the maximum correlation peak intensity to themaximum peak intensity of the noise in the correla-tion plane. This is the most relevant measure as it

Fig. 6. (a)–(d) Reference images, (e) test input image AEFT, (f) ab-solute value of the joint image containing the input test image andthe four phase-encoded reference images.

Table 1. Effect of SLM Size on Correlation Output

SLM Size

128 × 128 192 × 192 256 × 256 320 × 320 384 × 384 448 × 448 512 × 512

CPI 4.2E4 3.8E4 4.1E4 3.5E4 3.9E4 4.5E4 3.8E4PSR 12.4 16.9 46.1 49.6 57.4 61.0 58.5PCE 8:7E − 4 7:5E − 4 8:7E − 4 8:5E − 4 8:2E − 4 9:1E − 4 7:8E − 4SNR 2.5 4.1 8.8 10.7 12.4 18.2 19.6hPNi 2.7E3 1.2E3 6.6E2 3.9E2 2.9E2 2.2E2 1.7E2maxfPNg 1.7E4 9.3E3 4.6E3 3.3E3 3.2E3 2.5E3 1.9E3

Fig. 7. (Color online) Normalized correlation intensity whenmul-tiple references are phase encoded. Correlation of reference image(a) A with AEFT, (b) E with AEFT, (c) F with AEFT, (d) T withAEFT.

1 April 2010 / Vol. 49, No. 10 / APPLIED OPTICS B109

reflects the discernablility of the correlation outputfrom the scattered peaks. A SNR ≥ 2 is consideredto be acceptable. Higher SNR values suggest thatcorrelations with multiple reference images can beimplemented simultaneously.Finally, the effect of using multiple phase-

encrypted reference images was studied. Four refer-ence images containing the letters A, E, F, and T[Figs. 6(a)–6(d)] were correlated with a one testimage containing AEFT [Fig. 6(e)]. Each referenceimage is phase encrypted with a different randomphase mask as formulated in Eq. (14). Again, thisprocess is done offline and does not affect the speedof the system. The phase-encrypted references arethen added to the input test image online. The abso-lute value of the joint image is shown in Fig. 6(f). Theoutput contains all four correlation outputs multi-plexed in phase. To obtain any one of the four corre-lation results, the phase function corresponding tothat correlation must be used as described byEq. (16). By multiplying the phase function corre-sponding to the reference image of letter A withthe JPS, the correlation output in Fig. 7(a) is pro-duced. If the phase function corresponding to the let-ter E is used, the correlation intensity in Fig. 7(b) isproduced.

6. Conclusion

This paper presents a novel JTC system in which thestored reference is phase encrypted. The encryptiondisperses all but the desired cross-correlation termand as such the spatial displacement required inthe variant JTC systems is no longer necessary. Thisresults in an efficient use of the SLM space. It hasbeen shown that the spatial efficiency can be usedto perform multiple correlations simultaneouslyusing different reference images multiplexed inphase. The SNR of the system is inversely propor-tional to the SLM area and the energy of the non-cross-correlation terms. A simple and inexpensive

system setup similar in complexity to a JTC systemhas been presented. The power and simplicity of thisJTC system makes it ideal for real-time pattern re-cognition and tracking systems. The ability to per-form multiple correlations simultaneously makesan all-electronic implementation of the system suita-ble for real-time applications, which opens a new andexciting area in the field of correlators.

References1. C. Weaver and J. Goodman, “Technique for optically con-

volving two functions,” Appl. Opt. 5, 1248–1249 (1966.2. A. A. S. Awwal, M. A. Karim, and S. R. Jahan, “Improved

correlation discrimination using an amplitude modulatedphase-only filter,” Appl. Opt. 29, 2107 (1990).

3. O. Perez and M. A. Karim, “Optical enhancements of jointFourier transform correlator by image subtraction,” Proc.SPIE 1471, 255–264 (1991).

4. Q. Tang and B. Javidi, “Technique for reducing the redundantand self-correlation terms in joint transform correlators,”Appl. Opt. 32, 1911–1918 (1993).

5. G. Lu, Z. Zhang, S. Wu, and F. T. S. Yu, “Implementation of anon-zero-order joint-transform correlator by use of phaseshifting techniques,” Appl. Opt. 36, 470–483 (1997).

6. T. Nomura, “Phase-encoded joint transform correlator toreduce the influence of extraneous signals,” Appl. Opt. 37,3651–3655 (1998).

7. Z. Zalevsky, A. Rubner, J. García, P. Garcia-Martinez, C.Ferreira, and Emanuel Marom, “Joint transform correlatorwith spatial code division multiplexing,” Appl. Opt. 45,7325–7333 (2006).

8. C. F. Hester and M. G. Temmen, “Phase–phase implementa-tion of optical correlator,” Proc. SPIE 1297, 207–219 (1990).

9. R. R. Kallman and D. H. Goldstein, “Phase-encoding inputimage for optical pattern recognition,” Opt. Eng. 33, 1806–1812 (1994).

10. G. Lu, Z. Zhang, and F. T. S. Yu, “Phase-encoded input jointtransform correlator with improved pattern discriminability,”Opt. Lett. 20, 1307–1309 (1995).

11. G. Lu and F. T. S. Yu, “Performance of a phase-transformedinput joint transform correlator,” Appl. Opt. 35, 304–313(1996).

B110 APPLIED OPTICS / Vol. 49, No. 10 / 1 April 2010