AMSR-E Team Leader Science Computing Facility

2 June 2010 Dawn Conway, AMSR-E TLSCF Lead Software Engineer

AMSR-E Team Leader Science Computing Facility

Science Software Status


• L2A Brightness Temperatures V10– April 2009– Compute a checksum from Latitudes, Longitudes, and

Tb's, attach it to the L2A file as a global attribute, and report it in the QA file. This aids in corrupted file recognition.

– RSS added the glint angles of the DirecTV 10 and 11 satellites to the Geostationary_Satellite_Glint_Angles field.

• L2B Ocean V06– October 2007– Last algorithm updates version V05, April 2007

AMSR-E Science Software Status



• L2B Land V09– December 2008– Promote the product maturity status to Validated– Modify TLSCF software to accommodate an

incompatible shell command on the Science Team's hardware

– Last algorithm update version B05, March 2006

• L2B Rain V10– October 2007– First Linux version– No algorithm changes– Significant algorithm update expected soon



• L3 Ocean V04– May 2010– Add RFI filtering for DirecTV; this affects the United

States west coast and the Caribbean Islands – Add RFI filtering for Atlantic Bird 4 in the Mediterranean

Sea – Add RFI filtering for the Ascension Islands and in the

Gulf of Aden – Update RFI filter for Hotbird/Astra – Update Aral Sea filters – Include the RFI angle grid in the product



• L3 Land V06– December 2008– Promote the product maturity status to Validated– Modify TLSCF software to accommodate an

incompatible shell command on the Science Team's hardware

– Last algorithm update was version B04, May 2007



• L3 Snow V09– December 2008– Promote the product maturity status to Validated – Modify ancillary file format – Last algorithm update version B07, April 2006

• L3 Rain V06– April 2008– Promote the product maturity status to Validated– Skip duplicate (overlapping) scans in consecutive L2A

files



• L3 Rain V06 continued– Improve L2A Brightness Temperature binning for

monthly ocean grids– Update the leap second ancillary file to include the leap

second which occurred at the end of 2005– Adapt the software package to run under Linux– Add a shell script which is used during installation. – Use larger system stacksize parameter– Update expected soon



• L3 Sea Ice V12– March 2010– Update a result of anomalies observed by the science

team during reprocessing– Significant changes to TLSCF software to manage

intermediate files and ensure correct reprocessing – Production rules updated– Multi-year sea ice mask and snow melt mask now stored

within the 12.5 km product file – Known bug in code that calculates the sea ice

concentration difference arrays



• L3 sea ice multi-year mask used in snow depth over sea ice processing in version T08 – V11

• Initiated on October 1st of each year by the science code

• The algorithm then generated a dynamic multi-year ice mask each day (intermediate file) to be used in the next day’s processing

• Used default file when an intermediate file was not available

• Used default at start of any reprocessing stream



• Multi-year sea ice mask issues– The science team noticed anomalies in products whose

reprocessing streams started on dates other than October 1st

• Due to the dynamic nature of the mask • Reprocessed data whose processing streams began in the

middle of a sea ice season (January, for instance) were compromised due to the use of the default mask file

– The SCF discovered that incorrect intermediate files were being used at the SIPS to generate the final products

• Result of preliminary and final products processed in same location

• The intermediate mask files are overwritten each time the code is executed, causing inappropriate intermediate files to be used when a final product was made



• Multi-year sea ice mask software resolutions– Use default mask to initiate the data processing ONLY

for beginning of mission (June 2002) – Store the mask in the daily 12.5 km product– Write the data date to the mask file when it is created

and check the date before using in processing– Require that the mask date precede the data date– Add a command line argument to accommodate large

data gaps• Command line argument is a 12.5 km product file from

which the mask file is extracted• This is an override condition; the default condition is to use

the intermediate mask file



• Multi-year sea ice mask software resolutions (continued)– Under default conditions, if the intermediate mask file is

more than 7 days old, processing stops without making a product

– Under override conditions, i.e., if a previous 12.5 km product is given on the command line, the mask within the product will be extracted, processing will continue, and the metadata science quality flag will be set to Suspect if the masks are more than seven days old.



• Multi-year sea ice mask production rules• Sea ice shall be reprocessed from the beginning of

mission forward• If there is a data gap, then the mask file immediately

preceding the gap is to be used • Process preliminary and final products in separate

locations



• Multi-year sea ice mask – Lots of exception code

• October 1st, missing data on October 1st, data gaps, intermediate files, date checking, …

– Additional QA code– Numerous scenario testing – Close coordination with the SIPS concerning processing

rules– Similar issues and resolutions for the snow melt mask



Data Product Version Delivery Date to SIPS

L2A TB’s V10 April 2009

L2 Ocean V06 October 2007

L2 Land V09 December 2008

L2 Rain V10 October 2007

L3 Sea Ice V12 V13

March 2010 TBD

L3 Ocean V04 April 2010

L3 Land V06 December 2008

L3 Snow V09 December 2008

L3 Rain V06 April 2008


New AMSR-E Product

• L3 Sea Ice Drift– B01– TLSCF code

• Metadata• QA

– Integration and test at TLSCF– DAP delivery to the SIPS– Integration and test at SIPS – ESDT testing between SIPS and NSIDC


New AMSR-E Product

• L3 Sea Ice Drift– Daily sea ice motion in the Arctic derived from AMSR-E

89 GHz 6.25 km resolution Tb’s– Uses the 6.25 km and 12.5 km AMSR-E sea ice products

as input– Uses a 5 day window, so that for each output day it

requires the current day, the two previous days and the two following days

– Initially for northern hemisphere; southern hemisphere will added later


New AMSR-E Product

• L3 Sea Ice Drift File Sample (header)

Arrow number= 292 (Number of vectors in the file)Tracking period (day)= 4Tracking method= GridGrid Size (pix)= 16 (There is a vector every 16 pixels)Contour threshold= 0.040 (Maximum range in pixels that the ice is tracked)Contour Size limit (pix)= 100Contour X-axis limit (pix)= 25Contour Y-axis limit (pix)= 25Tracking range (pix)= 16Moving limit (pix)= 8.000000Frequency (GHz)= 89Resolution (km)= 6.25Row= 1024 (Output grid is a subset of the 6.25 km grid, 1024 x

1024)Column= 1024#


New AMSR-E Product

• L3 Sea Ice Drift File Sample (data)

212 580 218.000 588.000 10.000228 564 228.833 566.833 2.953228 580 232.500 585.250 6.915228 628 232.333 632.333 6.128228 660 232.000 666.000 7.211

The first two numbers are the x,y coordinates of the starting grid point, the next two are the x,y coordinates of the ending grid point, the last is the “displacement of the ice template in pixels” (the distance from the first to second coordinates).


Algorithm Updates - LANCE

• AMSR-E Real-time processing support – Update metadata generation routines for all science

software to recognize the product maturity code “R” for real-time processing

– Update the metadata input pointers to accurately reflect the product maturity code

– Update the L3 science software to recognize file names with a product maturity code of “R”

– New delivery of all DAPs to SIPS


Algorithm Updates

• L2 Rain – significant changes• L3 Rain – significant changes• L3 Sea ice – bug fix that affects the difference

arrays• L3 Snow • L2 Land ?• L3 Land ?• New L3 sea ice drift


AMSR-E L2A Quality Flags

• L2A Flags– Scan_Quality_Flag

• 4-byte• Per scan• Pertain to all observations of a scan including all Level-1A

and resampled channels. • Identical flag repeated in each swath

– Channel_Quality_Flag• 2-byte• Per scan• 12 channels for the Low Resolution Swath• 2 channels for each of the High Resolution Swaths

– The summary bit 0 of the Channel_Quality_Flag is automatically set whenever any of the bits in the Scan_Quality_Flag are set.


AMSR-E L2A Scan Quality FlagBit Flag Description (nominal value=“0”)

0 Summary flag Set if any of the other bits are set

1 Antenna Spin Rate Set if spin rate is out of range

2 Navigation Set if the position or velocity of the navigation data for that scan is out of bounds

3 RPY Variability(only AMSR-A)

Set if the roll, pitch, or yaw variability from scan to scan is out of bounds

4 Roll, Pitch, Yaw Set whenever the roll, pitch, or yaw exceeds 2.0 degrees

5 Earth Intersection Set if any of the observation locations fail to fall on the Earth

6 Hot Load Thermistors

Set whenever the thermistors on the AMSR hot load are out of bounds

7-31

Not used


AMSR-E L2A Channel Quality Flag

• Bit 0 is set if any of the bits 2 through 15 are set, also set whenever any of the bits in the Scan_Quality_Flag are set.

• Bit 1: set if no TB was computed for this channel• Bit 2: set for first and last scans • Bit 3: set if there are serious calibration problems• Bits 4,5, 7-10: quality of thermistors, hot / cold counts• Bit 11: set if there is a geolocation error as reported

by the Scan_Quality_Flag• Bit 6: indicates if Teff is dynamic or static• Bit 12: set if Teff is not available


AMSR-E L2A Quality Flags

• L2A Quality Flags– Extremely useful to end users– Easy to incorporate in algorithms

• Duplicate per scan (copy from L2A)• Read, interpret, and include in your quality flags

– Updates to include these flags are welcome at any time

AMSR-E Team Leader Science Computing Facility

Documents

Transcript of AMSR-E Team Leader Science Computing Facility