AO-AL16 986 NAVAL UNDERWATER SYSTEMS CENTER NEW LONDON CT NEW LO--ETC F/G 20/1A TECHNIQUE FOR IMPROVING DETECTION AND ESTIMATION OF SIGNALS C--ETC(U)

JUL 82 R F DWYER

UNCLAIDNSSIFT EDN CTD171 NL

jEND

NUSC Technical Document 6717

1 July 1982

SA Technique for Improving Detection andEstimation of Signals Contaminated byUnder Ice Noise

Roger F. DwyerSurface Ship Sonar Department

DTICE t- E %.0T Ei 1 JUL 1 9 1982

F

3 Naval Underwater Systems CenterNewport, Rhode Island / New London, ConnecticutIuzJ

.-J

Approv" fot publMc rleme, dlutlbutlon unlimited.

82 07 19 058

n " ' I dII I

I.

Preface

This technical document was prepared under NUSC Project No. A75030, DataAdaptive Detection and Estimation, Principal Investigator R. F. Dwyer (Code333 1), Program Manager Dr. E. Wegman, ONR (SP41 1).

This document is based on an oral presentation given at the 102nd meeting of theAcoustical Society of America, 29 April 1982, Chicago, IL.

Reviewed and Aproved: I July 1982

Surface Ship Sonar Department

The author of this document is located at theNaval Underwater Systems Center, New London Laboratory

New London, Connecticut 06320

REPORT DOCUMENTATION PAGE BREAD COTPLETMFORM

.REPORT aaiuE Z. OVT ACCESSION NO. & x. RcWIrs wuaAO hmmNS

TD 6717 :2D i-/6' c4. "T1.4 wed Sn.dd . TYPE OF NWOT & P64OW COVER

A TECHNIQUE FOR IMPROVING DETECTION AND ESTIMATIONOF SIGNALS CONTAMINATED BY UNDER ICE NOISE

. POWOMR G oiG. P640A~ NUU

7. AUTh A L CONTRACT OR 0RANT ftMIaIf

Roger F. Dwyer

9. PWOMM ORGANIZATHON NAM ANo £0016 10. PR0GRAM EL,, NM6T. PROjECT. TAM

Naval Underwater Systems Center ,RA ow UNmumsNew London Laboratory A75030New London, Connecticut 06320

1. CONTmuOL, oPiP M NAMI Au 12. IINT DATE

Office of Naval Research (SP 411) 1 11Y 1982

Arlington, VA. 22217

14. VONrTOWAG AGINaUWNAMOIADRESSOlf Mw mG OI I L SIAMI V CLASS. W r"aq "llol

UNCLASSIFIEDISO. 0EGLASSOPCATIOWMOM~l

-w,

Ii. OSTWIMON STATIUT t l . Rqwm~

Aoproved for public release; distribution unlimited.

I?. OISTMIUT"ON rAT"MI T M s .hm, ew o in .& k A idffwJW /0 Ropol

MS SIWPLEMSNTARY NOTIS

It. KEY weRaa .. iu.sd fW~ .n diirb Ib~ a~mhw,

Arctic ambient noiseNon-Gaussian noisePartition processingFRAM II data

\.2L AdqRACT XC.ilsimioal -wf&W . Wsa 14poofr 6, bWm* a~

0 Recent analyses of FRAM II arctic data have shown that under ice ambientnoise can be at times highly impulsive and non-Gaussian. The analysesincluded time domain statistical measurements which were consistent withpreviously reported results of experiments made within the Canadian ArcticArchipelago. New findings of frequency domain estimates of complex skew andkurtosis and cumulative distribution functions, measured in 2, 6, and 10 Hzresolution cells at the output of a discrete Fourier transform, also indica

(over)

DD,', 1473

anF

I

t 20. l(Cont'd)

-the existence of strong non-Gaussian noise. It is known that the ability todetect and estimate signals contaminated with non-Gaussian noise usingconventional processing is degraded compared with optimum techniques whichutilize knowledge of the noise statistics. Results comparing the performanceof conventional and nearly optimum signal processing methods are presentedusing the FRAM II data.

Accession For

NTIS GRA&IDTIC TAB F]

Jantifloatio

Distribution/_ -

AvailabilitY Codes

Avail and/orDitt Special

i [ .. ... . . IIII III .

copy

1. TO 6717

TABLE OF CONTENTS

Page

INTRODUCTION ....... ... .. .............................. 1

MATHEMATICAL PRELIMINARIES ....... .. ....................... 1

FRAM II DATA RESULTS ........ .......................... . .. 2

FRAM II PROCESSING RESULTS ....... .. ....................... 3

SUMMARY ....... .... .. ................................. 5

REFERENCES ....... ... .. ............................... 5

LIST OF ILLUSTRATIONS

Figure Page

1 Time Domain Statistical Moments for 2500 Hz Band ........... 7

2 Frequency Domain Statistical Moments ................. 8

3 Frequency Domain Amplitude Distribution ... ............ 9

4 Time Domain Statistical Moments for 300-400 Hz Band ....... 10

5 Frequency Domain Statistical Moments for 2 Hz Resolution. .. 11

6 Frequency Domain Data and Partitioned Data .............. 12

7 Frequency Domain Data and Partitioned Data with Gain ..... .. 13

8 Frequency Domain Envelope Distribution ............... ... 14

Reverse Blank

4

TO 6717

A TECHNIQUE FOR IMPROVING DETECTION AND ESTIMATION OF SIGNALSCONTAMINATED BY UNDER ICE NOISE

INTRODUCTION

The recent analyses of FRAM II data have shown that strong non-Gaussiannoise exists i n the frequency domain. This technical document presents theresults of processing FRAM 11 data in the frequency domain using datapartitioning. We also present the frequency domain envelope distribution atthree frequencies. These results suggest that data modeling should also beconsidered as a signal processing technique. However, here we shallconcentrate on the data partitioning results and leave to a later time thediscussion of the results of data modeling.

MATHE4ATICAL PRELIMINARIES

Let the received data x(i,q) - x[(i*(q-l)M)h], i=O,1,2,...,M-1,q=i,2,...,n where h is the interval between successive samples, be composed ofan additive mixture of signal, interference, and Gaussian noise of the form

x(i,q) = m(i,q)s(i,q,a1 ) + c(i,q)I(i,q,a2 ) + N(i,q), (1)

where

s(i,q,al) is the transmitted or radiated signal,I(i,q,a'2) is a narrowband interference (ice noise),N(i,q) is independent Gaussian noise,m(i,q) represents fading or multipath effects of the signal,c(i,q) represents fading or multipath effects of the interference, andat is a random variable taking on two values, at-O, atmi with

probabilities P(atuO)=l-Lt, P(at=l)=Lt respectively, for t=1,2.

The random variables al, a2 model transient or frequency modulationcomponents of the signal and interference, respectively, in the sense that Liand L2 represent the probability of the signal or interference being in aparticular frequency location measured over the observation interval. Theparameters al, a2, m, and c render the data non-Gaussian in the frequencydomain. In general these parameters are also functions of frequency; however,for the sake of notational simplicity, we shall not explicitly show thisdependence. The medium can also cause the received signal to be frequencymodulated; we will assume that the frequency modulation caused by the medium isincorporated in the parameters al and a2.

The discrete fourier transform (OFT) is defined asM-1

Xlq, Fp 1: xli,q)expl-JFpl)

1-0

.xr(q,Fp )-j X'(q,F p), (2)

p p

TD 6717

where j=tFf, Fp=21fph, and fp=p/Mh Hz. The components Xr(q,Fp)and XI(q,Fp) signify real and imaginary parts, respectively. Temporalweighting may also be included but is not treated here.

If we assume that the components of equation (2) are mutually independent,the corresponding spectrum estimate is

P(Fp) = L1E(m2 (q)) Ss)(Fp + L2 E(c2 (q)) SI(F p ) + RN, (3)

where we have assumed slow fading with respect to the length of the DFT sothat m(i,q)=m(q) and c(i,q)=c(q), and Ss(Fp) is the spectrum estimate atthe p-th frequency of the signal, SI(Fp) corresponds to the p-th frequencyspectrum estimate of the interference, and RN represents the independentGaussian noise spectrum estimate. The components L1 and L2 were assumedto be multiplicative in the frequency domain.

The problem to be addressed here concerns processing the data in thefrequency domain (as given by equation (2)) in the presence of strongnon-Gaussian interference. This problem arose as a consequence of the FRAM IIdata analysis study, although it was addressed on a theoretical level inreferences 1 and 2.

In this paper we will present under ice ambient noise data which aredominated by non-Gaussian interference in both the time and frequencydomains. Then we will show that by processing the data using partitioningtechniques, significant performance improvements in terms of signal-to-noiseratio (SNR) are possible in the under ice envirornment, assuming that thesignal level is much smaller than the interference. This assumption is notalways met in practice. Reference 3 discusses a frequency domain processingmethod that treats the interference-free case.

The theory of data partitioning methods was introduced by Ching and Kurz.Here we will present only the FRAM II data partitioning results. Anyoneinterested in the theoretical aspects of partitioning should consultreferences 4 and 5.

Another approach to optimum processing is data modeling. The frequencydomain envelope distribution of FRAM II data is also presented for threefrequencies at three different resolutions.

FRAM II DATA RESULTS

Since the FRAM II data results of under ice ambient noise have beenreported in detail in reference 6, we will only summarize the results neededin this technical document.

The data analyses are composed of time and frequency domain statisticalmeasurements. The time domain data were filtered, sampled, and grouped intorecords of 1024 samples each. The mean, variance, skew, and kurtosis werethen estimated for each record. Over time intervals consisting of hundreds ofrecords, the cumulative distribution function (CDF) of the energy (square ofthe data samples) was estimated and as shown, for the most part, to benon-Gaussian but with nonstationary behavior over successive intervals. Thetime domain data were then transformed into the frequency domain via a 1024point fast Fourier transform (FFT). Frequency domain statistics were thencompiled for each frequency cell for both the real and imaginary parts.

2

IL

TD 6717

Figure 1* shows the statistical moments for the time domain data, whichwere filtered through a 2500 Hz lowpass filter and then sampled at a 1000 Hzrate. Therefore each record represents a time interval of about 0.1 second,giving an overall data length of 10 minutes. Some important observationsabout these data are the variability of the variance over time and thesinificant deviation from the Gaussian assumption based on the skew andkurtosis estimates. We found, by processing the data in bands, that thevariability in the variance was due to higher frequency (greater than 750 Hz)components. The kurtosis is especially important because it indicatesdeviations from the Gaussian distribution by values greater or less than 3.The values greater than 3 pertain (in many cases) to distributions that aremore peaked than the Gaussian distributions whereas values less than 3correspond (in many cases) to distributions that are less peaked.+ Forexample, a purely sinusoidal signal with uniformly distributed phase has akurtosis.of 1.5. None of the records in figure 1 has a kurtosis value of 1.5although some are near 2. The additive Gaussian noise fs most likelycontributing to this result since it is known that kurtosis is SNR dependent(reference 3).

Figure 2 compares the power spectral density (PSD) (top curve), real skew(middle curve), and the real kurtosis for the under ice data which have beenprocessed with a 10 $z resolution and averaged over 1000 consecutive FFT's.This gives a total time interval of 1.7 minutes. The data clearly indicatenon-Gaussian noise based on the frequency domain skew and kurtosis. Over arelatively flat portion of the band as seen in the PSD we estimated theamplitude CDF using the 1000 consecutive FFT's for both the real and imaginaryparts. The results (figure 3) show significant deviation from a Gaussiandistribution (dashed curve) for both the real and imaginary parts. Later inour discussion we will compare these results with the original data after ithas been processed by partitioning.

Another data set of under ice noise is given in figures 4 and 5. The datawere first filtered y a 100 Hz bandpass filter centered at 350 Hz, weresampled at 2000 Hz, and were processed in records of 1024 samples, each givingan interval of approximately 0.5 second. The important observations in figure4 are that the variability in the variance is greatly reduced in this band andthat many records deviate from a Gaussian distribution based on the kurtosisestimate. The frequency domain results (figure 5) show that many frequencylocations also significantly deviate from a Gaussian distribution based on thefrequency domain kurtosis estimate. In addition, a 60 Hz tonal and some ofits harmonics are present in the PSO estimate in figure 5. The correspondingkurtosis estimate shows values significantly less than 3. An in-depththeoretical discussion explaining the significance of these results for asignal propagating in a medium with fading or multipath effects is given inreference 3.

FRAM II PRUCESSING RESULTS

We shall discuss the results of processing signals in the frequency domainhere. uptimum techniques for processing time domain signals in non-Gaussiannoise can be found in references 7 - 10. These frequency domain processing

* All figures are located at the end of the report.+ There may be pathological distributions where this interpretation is not

justified (12). However, deviations from 3 Dertain to non-Gaussiandistributions.

3

rfTD 6717

techniques, which we shall discuss, represent a new methodology for extractingsignals embedded in narrowband non-Gaussian noise. References 1 and 2develop, on a theoretical level, optimum partitioning techniques in sufficientdetail that we can concentrate here on the results of processing FRAM IIdata. First, the results of processing the 10 Hz resolution frequency domaindata by partitioning the real and imaginary parts separately will be given.Then we shall present the frequency domain envelope distribution results in 2,6, and 10 Hz resolution cells. These results suggest that noise modelingshould also be considered for optimum signal processing in the frequencydomain.

Figure 6 compares the amplitude CDF*of the partitioned data with that ofthe original data for both the real and imaginary parts. The circlescorrespond to the partitioned results. The interesting feature is thatpartitioning for this data set produces nearly a Gaussian distribution. Thisresult can be explained by noting that partitioning is equivalent to anonlinear transformation that significantly reduces the high amplitudeexcursions and leaves the smaller amplitudes unchanged. It should also bepointed out that this result of a Gaussian output was predicted in reference11. The performance improvement can be significant, as shown by the followingsimple example. Suppose it is desired to set the threshold so that the falsealarm rate per FFT is .0001. Then, assuming a small SNR, the partitioneddetector would have approximately a 15 dB processing advantage over aconventional (linear) detector, as shown in figure 6.

Another way of comparing performance is based on the asymptotic relativeefficiency, which is essentially a ratio of the output SNR of two detectors asthe signal approaches zero and the integration time approaches infinity. Asshown in reference 9, this turns cut to be a ratio of output variances for thetwo detectors under noise-only conditions. Figure 7 compares, for the 10 Hzresolution case, the COF of the partitioned and original data for the realpart only. The circles represent the partitioned data. Again note the closeapproximation to a Gaussian distibution after partitioning. We calculated theoutput variance for both data sets and found that partitioning improvesperformance in this case by about 5 dB in the sense that its variance isdecreased by 5 dB over the variance of the original data.

Another approach to optimum processing of the frequency domain FRAM IIdata is to model the data. Figure 8 shows a plot of the envelope distributionat the output of an FFT at three frequencies and for three differentresolutions. We considered FFT's with 10, 6, and 2 Hz resolutions withtime-resolution products (TRP) of 1000, 1000, and 750, respectively. Thevertical scale represents the envelope (normalized by its standard deviation)in dB, and the horizontal axis is the exceedance probability. In order tobetter visualize the tail behavior of the envelope distribution, we included asmall number of adjacent bins in the estimate. These estimates followedapproximately the curves shown in figure 8. In this way we were able toextend the tail region and observe its trend. The solid line in the figurerepresents a Rayleigh distribution. As can be seen, the data deviate from the 4Rayleigh distribution for all three cases considered. This suggests thepossibility of modeling the envelope distribution of FRAN II data in thefrequency domain.

4

TD 6717

SUMMARY

We have presented the FRAM II data analysis results and have shown thatthe ambient noise was highly non-Gaussian in the time and frequency domains.The frequency domain results are new, but we have confirmed them by atheoretical analysis and by comparison with other data. For the small signalcase, two tecnniques were considered for improving detection and estimation ofsignals contaminated with under ice noise.

In the first technique, we employed an adaptive partitioning method whichapproximates the optimum nonlinearity, as derived by an expansion of theloglikelihood ratio, for the particular noise distribution prevailing at thetime a decision is to be made. Partitioning transformed the CDF of the FRAM11 data into a Gaussian distribution and thereby decreased the thresholdneeded to maintain a prescribed false alarm rate. A particular example showedthat, for a false alarm rate of .0001, a 15 dB improvement in performancecould be achieved. We also calculated the variance of the original andpartitioned data and found that a gain of 5 dd in performance could beachieved based solely on reduction of the variance. This improvement appliesto both detection and estimation of signals in the under ice noise environment.

Another approach to optimum processing was considered. We plotted theenvelope distribution, at the output of the FFT for three frequencies forthree different resolutions. All three cases deviated from the Rayleighdistribution which suggests that data modeling should be considered as asignal processing approach in the frequency domain.

REFERENCES

1. R. Dwyer, "Robust Sequential Dectection of Narrowband Acoustic Signals inNoise," Proceedings of IEEE International Conference on Acoustics, Speechand Signal Processing, Washington, D.C., 2-4 April 19/9.

2. R. Dwyer, "A Robust Post-Processing Technique for Detection of NarrowbandSignals," NUSC Technical Memorandum 791090, to be published.

3. R. Dwyer, "A Statistical Frequency Domain Signal Processing Method,"Proceedings of the 16th Annual Conference on Information Sciences andSystems, held at Princeton University, 17-19 March 1982.

4. Y. Ching and L. Kurz, "Nonparametric Detectors Based On M-IntervalPartitioning," IEEE Transitions on Information Theory IT-18, pp. 241-250,1972.

5. L. Kurz, "Nonparametric Detectors Based on Partition Tests," inNonparametric Methods in Communications: Selected Topics, edited byrapanton1-Kazakos ana u. Kazakos, arcel-uekker, New York, 1977.

6. R. Dwyer, FRAM II Single Channel Ambient Noise Statistics, NUSC TechnicalDocument bbdj, Zb November 198l.

7. R. Dwyer and L. Kurz, "Sequential Partition Detectors," Journal ofCybernetics, vol. 8, pp. 133-157 1978.

5

T. 6717

8. R. Dwyer and L. Kurz, "Sequential Partition Detectors with DependentSampling," Journal of Cybernetics, vol. 10, pp. 211-232, 1980.

9. R. Jwyer, "Robust Sequential Uetection of Weak Signals in Undefined NoiseUsing Acoustical Arrays,'. Journal of Acoustic Society America, vol 67,4arch 1980.

1i. R. Dwyer, "Dectection of Partitioned Signals by Discrete Cross-SpectrumAnalysis, " IEEE Conference Proceedings of ICASSP, 1980.

11. L. Kurz, "A Method of Digital Signalling in the Presence of AdditiveGaussian and Impulsive Noise," IRE International Convention Record, Part4, pp. 161-173, 1962.

12. A. Nuttall, Personal Communication

t L _ m ... . .

TD 6717

0 0

Iii

* I

7

P TD 6717

I'

o' N u)

---------------------- -: 7 -- - - - -- - -- --7 -

0, ,- - - - - - - - - - - - - - - - - - - - - - - ----

a) CI 0

cr c) *

. .. >

- - - -- - - - - - - - - - - - -I - - - - - - - - -- -- -

-- ---- - -*-- - -- - - - - - -- - -----

:* --

---- ---- ---- ....... I ------------- .. ... .--a *O o . n ' " a -

7H/ 'OS I. 7A 1//8(2 M3nS W 36 s|SOI In "v] l

~8

r

TO 6717

iFN

CucoC)0

C.

CMC

U-(

00

e~4J

CCM

w 08

<~U

200 C4

0co

0 LL.

00x

0crcLL

7U

Ux~d-d 60

a C9

,F TD 6717

I I

coSN -

_0cv. -

- 0-i

I I' $0 8'2#4E

6mmE.I SISOLcml

4J

* i

* !

101

! I !" ! ! II!! t ! ! ! !

* lE IIE E I!33 E I I El

11

TD 6717

.EC .o

:o 2N N

0=

C C .. .. .. . . - .: . .:---- . ..--- . ..---- / 0..i - . . . . .

- -- - - -- -- -- -- ----

La.

C)C

LL JIrif04 74'

- -- ---- ---- -- - - ------- --

. . . . . . . ... ,,

e.r

. 4

--- ---- -- ---- ---- -- -- -

o o c 0 o o . .. a o

ZH/'OS 110A 180 m])4S 7IYU SISOLUMI] lT]d

11

... .- . . ..*-

TC TD 6717

0 0U0 0 o -

m cm p

I- ca w L

x '0 0

Co _ 0 a

U.i

I-. 0

n~~Z -c C,

'~ 0 0 0L- 41- -

((ALd LIDo

0 0 0*RX~d 11a0

12-

I, 1TD 6717

04)

4J

cc)

< 0)

LEU

(1))

0 z 0

0L

o zcr

LL.

d coII 4-'

z VI At

C1

I' TD 6717

Nt

m~ = NZ Of-- (j =

(0 0

v cQ LO

Z cm

LL1

Z.S.i'"L 4W4

14L

INITIAL DISTRIBUTION LIST

Addressee Copies

NAVSEASYSCOAi (CUR R. Welsh, Code 63D; Dr. D. Porter, Code 63R-11) 2

OTIC 12

UWR (J. Smitn, Dr. E. Wegman, UNR-411(SP), 2

L. KurzPolytechnical Institute of New York, 333 Jay Street, New York 11201 1

D. iiddleton, 127 E 91st Street, New York, NY 10026

J. Thomas and S. Schwartz, Dept. of Electrical Engineering and 2Computer Science, Engineering Quadrangle, Princeton University,Princeton, NJ 08544

University of Rhode Island (Dr. D. Tufts), Kingston, RI I

IL