21-23/04/2015PM27 J-L Vergely, J. Boutin, N. Kolodziejczyk, N. Martin, S. Marchand SMOS RFI/Outlier filtering Historical reminder Filtering strategy at TB level before SSS retrieval. Comparison between SMOS TB and modelled TB at : -individual TB level (5 sigmas filtering) -snapshot level if a snapshot is considered as contaminated, remove it. Test on the std and presence of outliers. -dwell line level Very difficult because RFI contamination or other contaminations can be very weak. It is not possible to detect/remove all the contaminated TBs. Last year conclusion : paradox TB does not need to be removed before retrieval. It is easier to detect biased SSS from chi2P index. Historical reminder V5 TB filtering + chi2P selection V6 no TB filtering; chi2P selection more efficient The lessons we have learned : 1/ It is more efficient to remove contaminated SSS than to remove contaminated TB. 2/ The TB contamination is, most of the time, not discontinuous. Contamination amplitude from very weak level to very high level. TB contamination with a weak amplitude is not detectable. 3/ Observation : if a dwell line is contaminated with TBs affected by high bias, it is contaminated by TBs affected by low bias. 4/ Land sea and RFI contamination : difficult to disentangle V6 V5 Example of filtering SMOS Ascending dcoast>800km Better RFI/LS sorting Example of filtering : sun contamination CHI2P checked CHI2P not checked V6 No TB outlier filtering V5 TB outlier filtering Objectives Current objectives : to provide the users unbiased SSS using only OTT correction. to be able to remove biased SSS from indicators like chi2P. New objective ? To correct biases. Correction at TB level (Land Sea correction) or/and at SSS level (L2/L3 correction). Objectives If no chi2P filtering but coastal bias, RFI etc corrected Finality : to produce SMOS SSS everywhere. SSS filtering from chi2P (and other flags) Finality : to produce unbiased SSS without correction (except OTT ). THE QUESTIONS The change of paradigm requires a change in the strategy To reconsider the filtering at TB level ? Warning : if TBs are too filtered, no more TBs left for the SSS retrieval. Trade off has to be found. 4 cases : - to do nothing, - to filter, - to correct, - to filter + to correct At TB or SSS level ? Find the good strategy according to the grid point position and the time Objectives Try to map the kind of expected SSS biases and associated strategy. Example of the SM : to map the contaminated TBs/GP. SM dynamic map (monthly updated map). Similar strategy for SSS ? From CESBIO Objectives Correction condition : bias must be stable in time. verified condition most of the time in full ocean Proposition/question : if bias is stable in time : no TB filtering. if chi2P OK without TB filtering : OTT correction is sufficient ? If SSS still biased, ensure bias stability ? to be checked from the reprocessing data. Is it possible to filter unstable biased TBs and keep stable biased TB ? SSS v5 (TB filtering) SSS v6 (no TB filtering) Differences close to the coast, RFI regions, ice. SSS CORRECTION DEPENDS ON TB FILTERING Introduction Diagnostic phase: Bias/outliers at visibilities, TBs or SSS level. Biases could have different origins : difficult to separate the different kind of biases. For example, coast bias and RFI bias are often mixed. Biases have different amplitudes : TB biases not necessary well marked difficulty to find universal thresholds. Biases are stable or variable : intermittent biases. Spatial variation of the bias : latitudinal, coast etc. Introduction : biases at TB level Bias classification at TB level. Origin of the contaminations: Sun Approximative modelisation (dielectric constant, roughness model at high/low WS) RFI source detection/mitigation Coast reconstruction problem, polarization mixing, OTT biases. AUX data (GN, WS, SST, etc) Instrument (calibration, ) Always difficult to find a bias at TB level because radiometric noise of the order of 2K. A bias of 0.1K generates 0.2 psu bias. To find a 0.1K bias, we need 400 TBs !! Introduction : biases at TB level (OTT/LS correction) How to filter TB ? Current filter implementation could be used. The correction Delta TB = mean(Tb_smos Tb_ref) Expected error on Delta TB for LS contamination : Radiometric Accuracy / sqrt(50) ~ 0.3 K Indicator of the LS stability correction std(Tb_smos Tb_ref). If the bias is stable std(Tb_smos Tb_ref) = Radiometric Accuracy. -> if std(Tb_smos Tb_ref) > threshold : problem in the correction ? Test to be used to filter the TB before the retrieval. -> propagate the error on Delta TB toward the TB covariance matrix before the retrieval. Not needed if std(Tb_smos Tb_ref)/ra add a TB filtering stage and do the SSS retrieval again. Build a dynamic map for the outlier detection. For validation : identify specific area where systematic investigations could be done. North Atlantic, Reunion island, Amazon plume, a region in full ocean (without contamination issue). Process time series for analysis. Extra slides 21/ PM 27 Max spatial std SSS 21/ PM 27 SSS self consistency std biais dwell Max -min biais dwell SSS self consistency mean biais dwell Intermittent RFI:determination 21/ PM 27 Method : Diagnostic using time series behaviour. If the fluctuations of the spatial SSS encrease statistically in some place : unexpected behaviour. Relative increase : some time series present strong level of fluctuations. Normalisation using the median level of fluctuations. Intermittent RFI :low 21/ PM 27 SSS spatial std SSS time std Intermittent RFI 21/ PM 27 Intermittent RFI 21/ PM 27 SSS spatial std SSS time std North Atlantic 21/ PM 27