Acknowledgements : Jun Li (CIMMS/SSEC, UW-Madison) JCSDA NOAA NESDIS/GOES-R
Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of...
-
Upload
hannah-byus -
Category
Documents
-
view
217 -
download
0
Transcript of Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of...
![Page 1: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/1.jpg)
Radial-by-RadialNoise Power
Estimation
Igor Ivić and Christopher CurtisCIMMS/University of Oklahoma
and NSSL/National Oceanic and Atmospheric Administration
NEXRAD TACNorman, OKAugust 29, 2012
![Page 2: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/2.jpg)
2
• Incorrect noise power measurements can lead to:– Reduction of coverage when noise power is overestimated
• Case in most radar sites
• Radar product images cluttered by noise speckles if the noise power is underestimated
– Usually occurs in cases of strong interference• Biased meteorological variables at low to moderate SNR
• Blue-sky noise used to produce system noise power– Adjusted for lower elevations using correction factors– For each elevation, the same value is used at all azimuths
• However, noise drifts with time and varies with antenna position in azimuth and elevation
Motivation
+ the system gain can change within minutes …
![Page 3: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/3.jpg)
3
Solution• Estimate receiver noise power at each
antenna position– Noise power needs to be computed in real-
time (i.e., from data containing mixed signals and noise)
• BUT HOW DO YOU DO THAT?
• Radial-by-radial noise power estimation technique provides the solution
![Page 4: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/4.jpg)
4
Radial-by-Radial Noise Estimation
• History– First Version presented at ERAD in September 2010
• required rough initial noise value– Second version presented at AMS in January 2011
• no rough initial noise value required– Algorithm description of the latest version
delivered to the ROC in May 2012• includes high gradient signal removal• simplified so it operates only on the measured powers
• The technique has been in use on the NWRT since June 2011
![Page 5: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/5.jpg)
5
Technique Description• Expert noise power Ne determined
visually
• Used to assess the technique accuracy
200 400 600 800 1000 1200 1400 1600 1800
0
10
20
30
40
50
60
70
801
0 lo
g10
( P/ Ne)
(d
B)
Samples no.
M = 17
Expert noise Ne (gates 1300 to 1836)
![Page 6: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/6.jpg)
6
Power Profile of Test Data
Received power as a function of range at the elevation angle of 0.5 deg. The expert noise power (Ne) is indicated with a grey line.
200 400 600 800 1000 1200 1400 1600 1800
0
10
20
30
40
50
60
70
80
10
log
10( P
/ Ne)
(d
B)
Samples no.
M = 15
Expert noise Ne (gates 1440 to 1490)
200 400 600 800 1000 1200 1400 1600 1800
0
10
20
30
40
50
60
70
80
10
log
10( P
/ Ne)
(d
B)
Samples no.
M = 28
Expert noise Ne (gates 1600 to 1700)
![Page 7: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/7.jpg)
7
Step 1: Strong Point Clutter Rejection
• Gate at location k is considered to contain point clutter if its power is much larger than the power at surrounding gates
200 400 600 800 1000 1200 1400 1600 1800
0
10
20
30
40
50
60
70
80
10
log
10( P
/ Ne)
(d
B)
Samples no.
M = 28
200 400 600 800 1000 1200 1400 1600
0
10
20
30
40
50
60
70
80
10
log
10( P
/ Ne)
(d
B)
Samples no.
M = 28
200 400 600 800 1000 1200 1400 1600 1800
0
10
20
30
40
50
60
70
80
10
log
10( P
/ Ne)
(d
B)
Samples no.
M = 15
200 400 600 800 1000 1200 1400 1600
0
10
20
30
40
50
60
70
80
10
log
10( P
/ Ne)
(d
B)
Samples no.
M = 15
BEFORE POINT CLUTTER REJECTIONAFTER POINT CLUTTER REJECTION
![Page 8: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/8.jpg)
8
Step 2: Detect Flat Sections• The flat sections of the power profile are
identified as this is an indication of the potential signal-free regions
– This is done by estimating local variance along range
– (local range variance at k < threshold) => range gate at k considered potentially signal-free
200 400 600 800 1000 1200 1400 1600
0
10
20
30
40
50
60
70
80
10
log
10( P
/ Ne)
(d
B)
Samples no.
M = 28
Local range variance
Notice that the local range variance is smaller in noise regions
![Page 9: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/9.jpg)
9
Step 2: Flat Sections Detected
• The mean power is computed for each group of contiguous range gates classified as signal-free
• Out of those, the smallest one is taken to be the intermediate noise power estimate (Nint)
200 400 600 800 1000 1200 1400 1600
0
10
20
30
40
50
60
70
80
10
log
10( P
/ Ne)
(d
B)
Samples no.
M = 28
200 400 600 800 1000 1200 1400 1600
0
10
20
30
40
50
60
70
80
10
log
10( P
/ Ne)
(d
B)
Samples no.
M = 28
10 log10
(Nint
/Ne) = -0.04 dB
200 400 600 800 1000 1200 1400 1600
0
10
20
30
40
50
60
70
80
10
log
10( P
/ Ne)
(d
B)
Samples no.
M = 15
200 400 600 800 1000 1200 1400 1600
0
10
20
30
40
50
60
70
80
10
log
10( P
/ Ne)
(d
B)
Samples no.
M = 15
10 log10
(Nint
/Ne) = -0.4508 dB
![Page 10: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/10.jpg)
10
Step 3: SNR Censoring Using Nint
• Discard all samples at range locations for which the power estimate is larger than the threshold (THR(M)xNint)
200 400 600 800 1000 1200 1400 1600
0
10
20
30
40
50
60
70
80
10
log
10( P
/ Ne)
(d
B)
Samples no.
M = 15
SNR thresholdNint
200 400 600 800 1000 1200 1400 1600
0
10
20
30
40
50
60
70
80
10
log
10( P
/ Ne)
(dB
)
Samples no.
M = 28
SNR thresholdNint
![Page 11: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/11.jpg)
11
100 200 300 400 500 600 700
-3
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
10
log
10( P
/ Ne)
(d
B)
Samples no.
M = 28
200 400 600 800 1000 1200 1400
-4
-3
-2
-1
0
1
2
10
log
10( P
/ Ne)
(d
B)
Samples no.
M = 15
Step 3: Output after Nint SNR Censoring
SNR profile of data after discarding samples at range gates where power estimate is larger than the threshold.
Mean power/expert noise = -0.216 dB
Mean power/expert noise = 0.243 dB
Potential signal regions
![Page 12: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/12.jpg)
12
Step 4: Apply "range persistence" Filter
• Detects larger sample powers that exhibit continuity in range– Finds 10 or more consecutive power values larger than
the median power
100 200 300 400 500 600 700
-3
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
10
log
10( P
/ Ne)
(d
B)
Samples no.
M = 28
Gates labeled by the "persistence filter"Median power
200 400 600 800 1000 1200 1400
-4
-3
-2
-1
0
1
2
10
log
10( P
/ Ne)
(d
B)
Samples no.
M = 15
Gates labeled by the "persistence filter"Median power
![Page 13: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/13.jpg)
13
Steps 5&6: Update Active Range Gates and Censor using SNR Threshold
Mean power/expert noise = -0.2376 dB
Mean power/expert noise = 0.1586 dB
• Discard the samples marked by the range persistence filter
• Compute the mean power of the remaining samples
• Perform SNR censoring
100 200 300 400 500 600 700
-3
-2
-1
0
1
2
10
log
10( P
/ Ne)
(dB
)
Samples no.
M = 28
Mean powerThreshold
200 400 600 800 1000 1200 1400
-4
-3
-2
-1
0
1
2
3
10
log
10( P
/ Ne)
(dB
)
Samples no.
M = 15
Mean powerThreshold
![Page 14: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/14.jpg)
14
Step 7: Apply Running Sum Filter
• A running sum is performed over the array of remaining powers
– makes regions with weak signals visible
After 1 iterationmean power/expert noise = -
0.0043 dB
After 1 iterationmean power/expert noise = -0.3 dB
200 400 600 800 1000 1200 1400
28
30
32
34
36
38
Samples no.
Ru
nn
ing
su
m
M = 15
Sum of 33 samplesMean powerThreshold
100 200 300 400 500 600 700
17
18
19
20
21
22
23
Samples no.
Ru
nn
ing
su
m
M = 28
Sum of 18 samplesMean powerThreshold
![Page 15: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/15.jpg)
15
Accuracy Assessment
Algorithm failed to produce noise estimate 43 times out of 155164 (0.03%)
BIAS = -4×10-3 dB or -0.086% of the true noiseSTD = 5.3×10-2 dB or 1.22% around the mean
• To assess the accuracy‒ estimated noise is subtracted from all powers‒ simulated noise is artificially added to the real data
-0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.40
1
2
3
4
5
6
7
8
9
10
Estimated noise/True noise (dB)
%
![Page 16: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/16.jpg)
16
SUMMARY
Radial-by-radial noise power estimation technique accurately produces noise power measurements in real-time! GREAT
SUCCESS!
![Page 17: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/17.jpg)
1750 100 150 200 250 300 350
-80.5
-80
-79.5
-79
-78.5
-78
-77.5
-77
-76.5
Az (deg)
10
log
10(N
) (d
Bm
)
Measured and legacy noise (El = 0.87 deg)
Measured HLegacy HMeasured VLegacy V
Benefits of Radial-by-Radial Noise Estimation
• Comparison between the measured noise power and legacy noise power
– Can see effects from man-made (KVNX) and mountain (KPDT) clutter
– Will use data from these cases to illustrate benefits
KVNX: Vance AFB, OK KPDT: Portland, OR
50 100 150 200 250 300 350-81.4
-81.2
-81
-80.8
-80.6
-80.4
-80.2
-80
-79.8
X: 284.4Y: -80.6
Az (deg)
10
log
10(N
) (d
Bm
)
Measured and legacy noise (El = 0.53 deg)
Measured HLegacy HMeasured VLegacy V
0.9 dB
1 dB
4 dB
1 dB
![Page 18: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/18.jpg)
18
Reflectivity CoverageKPDT Data with Legacy Noise (0.9°, VCP 12, Cut 3)
![Page 19: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/19.jpg)
19
Coverage increased by 18%
Reflectivity CoverageKPDT Data with Measured Noise (0.9°, VCP 12, Cut
3)
![Page 20: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/20.jpg)
20
Invalid Spectrum Width Values
• 69.6% of v estimates with legacy noise are valid (i.e., v > 0)
• 89.0% of v estimates with measured noise are valid
1
2ˆ2ˆ ln
ˆ2 1a h h
v
h
v P N
R
KPDT Data with Legacy vs. Measured Noise (0.9°, VCP 12, Cuts 1&2)
49.4% IMPROVEMENT IN THE NUMBER OF VALID ESTIMATES
![Page 21: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/21.jpg)
21
• 78% of |hv(0)| estimates with legacy noise are less than one• 84.48% of hv(0) estimates with measured noise are less than one
14.9% IMPROVEMENT IN THE NUMBER OF VALID ESTIMATES
ˆ 0
ˆ 0ˆ ˆ
hv
hv
h h v v
R
P N P N
Invalid Correlation Coefficient Values
KVNX Data with Legacy vs. Measured Noise (0.5°, VCP 11, Cut 1)
![Page 22: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/22.jpg)
22
Effects on ZDR
• Differential reflectivity is estimated as
where Nh and Nv are errors in noise floors
• Using perturbation analysis, we get
– If errors in H and V are of different sign they add up, otherwise they may cancel
10
ˆˆ 10log
ˆh h h
DR
v v v
P N NZ
P N N
10 1 1_
ˆ ˆln10DR v h
v v h h
BIAS Z NOISE N NP N P N
![Page 23: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/23.jpg)
23
Contribution of Noise Error to ZDR Bias
KPDTNh = -80.6 dBm & Nv = -80.3 dBm
Nh = -4 dB & Nv = -1 dB
KVNXNh = -80.7 dBm & Nv = -81.2
dBmNh = -0.9 dB & Nv = -1 dB
• In the current system, noise errors are significant contributors to the ZDR bias for low to moderate SNR
• Real-time noise estimates eliminate this contribution because of accurate measurements (i.e., Nh 0, Nv 0)
5 10 15 20-0.5
-0.45
-0.4
-0.35
-0.3
-0.25
-0.2
-0.15
-0.1
-0.05
SNR (dB)
BIA
S_Z
DR
(NO
ISE
) (d
B)
M = 17, v = 2 m s-1, Z
DR = 4 dB, |
hv(0)| = 0.99
SimulationTheoretical calculation
5 10 15 20
0.1
0.2
0.3
0.4
0.5
0.6
SNR (dB)
BIA
S_Z
DR
(NO
ISE
) (d
B)
M = 15, v = 2 m s-1, Z
DR = 0 dB, |
hv(0)| = 0.99
SimulationTheoretical calculation
![Page 24: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/24.jpg)
24
Summary• Improved data quality with measured noise:– Increased coverage for all moments and dual-
polarization variables– Reflectivity – decreased bias– Spectrum Width (v) – decreased bias and
increased number of valid estimates– Correlation coefficient (|hv(0)|) – decreased bias
and increased number of valid estimates– Differential reflectivity (ZDR) – decreased bias
• Accurate noise measurement is crucial for keeping measurement biases within acceptable levels
![Page 25: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/25.jpg)
25
Conclusions• Accurate noise estimation accounts for noise
variations due to:– Changes in system gain (Melnikov et al.)– Radiation caused by clutter, storms and man made objects– These noise variations can lead to:
• Reduction of coverage when noise power is overestimated• Radar product images cluttered by noise speckles if the noise
power is underestimated• Biased meteorological variables at low to moderate SNR (e.g.,
differential reflectivity and correlation coefficient)
• The current system calibration does NOT address these issues!
• Radial-by-Radial Noise Estimation eliminates or greatly mitigates all of these issues stemming from incorrect noise power measurements.
![Page 26: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/26.jpg)
26
Backup Slides
![Page 27: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/27.jpg)
27
KVNX Data with Legacy Noise (0.5°, VCP 11, Cut 1)
Reflectivity Coverage
![Page 28: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/28.jpg)
28
Coverage increased by 6.18%
Reflectivity CoverageKVNX Data with Measured Noise (0.5°, VCP 11, Cut
1)
![Page 29: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/29.jpg)
29
KPDT Data with Legacy Noise (0.5°, VCP 12, Cuts 1&2)
Invalid Spectrum Width Values
![Page 30: Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration.](https://reader038.fdocuments.us/reader038/viewer/2022110303/5517e5f0550346d0568b4700/html5/thumbnails/30.jpg)
30
Invalid Spectrum Width Values
KPDT Data with Measured Noise (0.9°, VCP 12, Cuts 1&2)