WP 2000 Improved Identification of Clouds 

Jane Hurley, Anu Dudhia, Don Grainger

University of Oxford

Current Cloud Detection

Colour Index (CI) Method is now used to detect cloud …

A couple of caveats … however …

• microwindows have not been optimized – would be useful if CI ~ EF

• fails to detect cloud with cloud fraction < 30%

Cloud Identification: SVD Analysis of Clouds

Objective:

To create and analyze RFM-simulated cloudy spectrum of varying cloud effective fraction EF to formulate a new cloud detection method using Singular Vector Decomposition SVD.

Singular Vector Decomposition SVD

• is statistical technique used for finding patterns in high dimensional data;

• transforms a number of potentially correlated variables into a smaller number of uncorrelated variables (SINGULAR VECTORS)

• first SV captures the most variance … and each successive SV captures increasingly less variance

Idea is to find singular vectors that describe clear and cloudy atmospheres and use them in cloud detection

Use 2nd half of A band because more sensitive to cloud presence

RFM-simulated spectrum with EF = 0 and 9.0km tangent height and the corresponding first 8 Clear Singular Vectors SVclear

Clear Singular Vectors

Need only first few SVs to well represent signal:

First 3 SVs capture ~90% of total variance

Cloudy Singular Vectors•Subtract off mean spectral radiance from Original signal;

•Use SVclear to do a Least Squares Fit (LSF) on Original-Mean signal;

•Subtract LSF from Original signal to get Cloud-Only signal.

Compare with Aerosol signature with same EF in the FOV:

• SVD-calculated Cloud-only signal

• Aerosol signal

Do SVD on Cloud-only signal to get Cloudy Singular Vectors SVcloud

Use SVclear and SVcloud for given tangent height to do a LSF to mean-subtracted Original signal

15 km

12 km

9 km

6 km

EF ≠ 0 → Non-zero fit coefficient to SVcloud!

Application to MIPAS data

Cloud DetectionMethod 1: χ2 Ratio

Use SVclear and SVcloud to do LSF of arbitrary spectrum.

Use χ2 error to measure goodness of fit.

Method 2: Ratio of Integrated Reconstructed Radiances

Use SVclear and SVcloud to do LSF of arbitrary spectrum.

Reconstruct cloudy and total radiance using LSF.

Comparison of Methods4 Methods of Cloud Detection:

• Radiance Thresholding

• Colour Index

• SVD χ2 Ratio

• SVD Ratio of Integrated Reconstructed Radiances

Methods applied to RFM Data: Percent that method gets prognosis right

Methods applied to MIPAS 2003 data: Percent agreement between methods

Future Work

Finish selecting optimal microwindows for use with Colour Index Method – those that best correlate with EF.

Finalize choice of thresholds for SVD Methods.

Compare methods of cloud detection on large MIPAS dataset against known databases (ISCCP) etc to see what difference this makes.

Do SVD analysis of ice clouds and implement this into an identification scheme. Hopefully will then have a cloud type identification scheme.

For RFM-simulated spectra …

For MIPAS data, not so sharp a distribution, obviously … but clearly a bimodal distribution

Can fit a Gaussian to ‘clear’ peak and set a threshold:

thr = peak + 3st.dev.

Should pick up 99.5% of cloud, if truly Gaussian

Cloud Detection

Method 1: χ2 Ratio

Use SVclear and SVcloud to do LSF of arbitrary spectrum.

Use χ2 error to measure goodness of fit.

Fit given spectrum with SVclear → χ2clear

Fit given spectrum with SVclear and SVcloud → χ2clear+cloud

Consider ratio of χ2clear / χ2

clear+cloud:

• χ2clear / χ2

clear+cloud > 1 for cloudy spectra

• χ2clear / χ2

clear+cloud ≈ 1 for clear spectra

Application to RFM-simulated spectra …

Method 2: Ratio of Integrated Reconstructed Radiances

Use SVclear and SVcloud to do LSF of arbitrary spectrum.

Radiance of cloud LSVcloud = mean(Σi (fit coeff)i SVcloud i), where i ranges over the cloudy SVs only and average over spectral points.

LSVcloud = 0 for clear spectra

LSVcloud > 0 for cloudy spectra

Total radiance LSVall = mean(Σi (fit coeff)i SVi), where i ranges over all SVs and average over spectral points.

Consider ratio LSVcloud / LSVall:

LSVcloud / LSVall = 0 for clear spectra

LSVcloud / LSVall > 0 for cloudy spectra