ArcGIS and NASA EOS Data

NASA GES DISC Team:Aijun Chen, Wenli Yang, Peisheng Zhao, Long Pham, Chris Lynnes, Gail Wade

NASA ESDIS:Ross Bagwell, Francis Lindsay

ESIP Winter Meeting, January 4, 2012 - Washington DC

Introduction• Most NASA EOS data products are archived in HDF-EOS

format.• Some data centers provides additional format support,

especially netCDF and GeoTiff (e.g., GES DISC’s OPeNDAP and WCS services).

• ArcGIS is one of the most frequently used GIS software packaged in the GIS community.

• Enabling direct data access from ArcGIS to NASA EOS data is greatly desirable.

• We’ll briefly introduce the status ArcGIS’ support for NASA EOS data and demonstrate how NASA GES DISC’s atmospheric data products, archived in HDF-EOS and available as times-series in netCDF via WCS service, can be visualized in ArcGIS.

ArcGIS Support for HDF4-Based HDF-EOS Data

• Level 3 HDF4- based HDF-EOS are directly supported.

• Some minor issues:• The ArcMap interface only allows to add a maximum of 29

fields, which may not adequate for some of our products, such as the AIRS standard retrievals).

• The interface does not display the entire field name when a field name is long (about 55 characters including array size, data type info). This could be a problem for a field with a very long name.

• Once added into ArcMap, the field name is not carried into the layer name (but with a field number), which makes it difficult to know which layer is which field. But this could be manually fixed, although tedious, because the field name can be viewed from the layer properties.

ArcGIS Support for Non-Grid EOS4 and Native HDF

• For other levels of HDF-EOS4 and native, data are treated as arrays without geolocation information (some may cause software crash, e.g., MLS-Aura_L2).

• Difficult to define axes for spatial dimensions, e.g., AIRS L1B radiance array stored in Y/X/Channel array (i.e., BIP in 135x90x2378) is imported in ArcMap as having X-rows and Channel columns (i.e., BSQ).

180x360

90x2378

ArcGIS Support for HDF5-Based HDF-EOS

• For HDF5-based HDF-EOS, the “he5” file extension not included default.

• They can be added by changing the settings (in ArcCatalog, add extension name and description in ArcCatalog->Customize->ArcCatalogOptions->FileTypes->New Type; in ArcMap, Customize->ArcMapOptions->File Formats and check the extension

name is associated to hdf5).• However, the geolocation information is not correctly read and the

data is essentially treated as generic rasters.

Level 2 HDF Files

• Level 2 HDF-EOS, either 4- or 5-based, treated as native raster.• AIRS L2 in HDF-EOS4:

AIRS.2011.10.17.098.L2.RetStd.v5.2.2.0.G11290152649.hdf

• OMI L2G in HDF-EOS5: OMI-Aura_L2G-OMTO3G_2011m0815_v003-2011m0816t062855.he5

CF-netCDF Data Format• nc extension not included in ArcCatalog and can be added in the

same way as the hdf. • In ArcMap, netCDF data must be imported using ArcToolBox-

>MultiDimensionTools.• The nc extension not viewable in ArcMap even “All-supported

format” is selected in Customize->ArcMapOptions->Raster ->FileFormats.

• CF-netCDF file can be imported as • Raster• Feature• Table.

CF-netCDF Data Format• When a CF-netCDF file is made to a raster, only one variable can be

selected in the interface but all fields are actually added.• Use the netCDF menu in the

Properties to select the layerto be shown.

• Don’t need to put in “Band” information otherwise nothing will be added. This is perhaps due to not allowing variable selection,i.e., “bands” vary for different variables.

• For OPeNDAP output, renaming the _FillValue attribute causes the value notbeing recognized.

CF-netCDF as Raster: OMI Examples• OMDOAO3e: Cloud Fraction (top), ColumnAmountO3 (bottom)

• Selected in the Properties.

• Window to small to show variable names.

• The L3 AIRXRET file in “nc” format obtained from OPeNDAP (AIRS.2011.01.01.L3.RetStd001.v5.2.2.0.G11003131820.hdf.nc)

• The file contains more than 230 fields.• Selecting field to display in the “Layer Properties” window.

CF-netCDF as Raster: AIRS L3 Example

CF-netCDF as Raster: AIRS L3 Example

TRMM L3 Precipitation Products• Native HDF doesn’t work well, especially with x-dim before y-

dim. • 3B42, 3B43

• 3A12, 3B31 SurfacePrecipitation and RainWater

• 3A11

• 3A25

• OPeNDAP (http://disc2.nascom.nasa.gov/opendap/TRMM_L3/) produces CF-netCDF for 3B42 and 3B43 (but not for many others, e.g., 3B31, 3A11, 3A12, 3A25, 3A26).

CF-netCDF L2G AIRS Examples• A L2G file obtained from OPeNDAP:

AIRS.2011.01.01.001.L2.RetStd.v5.2.2.0.G11003114443.hdf.nc• Data cannot be not added using the netCDF to raster tool (due to

variable spacing in coordinate variables), but can be added as a feature layer.

CF-netCDF as Feature

• Multiple variables in a CF-netCDF file can be imported as multi-attribute feature.

• When treated as feature, data do not need to be georectified and thus the unequally-distanced values can be imported, such as the previously mentioned OMI SO2 and L1/L2 files.

• The feature table could be very large, even though the “_FillValue” cells are excluded.

• Feature points can be interpolated to a raster array.

Interpolation• The Natural Neighbor (left) and IDW (right)

interpolations, from the non-FillValues, for the totO3std variable in the previous L2G AIRS data (totO3).

Interpolation• Showing both original point values and the interpolated rasters.

The original values are shown with proportional symbology.

CF-netCDF Summary• Level 3 data works fine with the ArcToolBox. • The minor issues when importing as a raster include not allowing

selecting variables (and bands) during importing and not displaying long variable names.

• Some OPenDAP converted files (OMI SO2 and OMAEROe) cannot be added as a raster layer, with an error message showing unequal spacing dimension variables.

• A possible reason might be due to the rounding errors in dimension variables, such as longitude values being (179.87502, 179.62498, 179.37502,…). However, the OMTO3e product has the same longitude value and works fine.

• Multiple variables can be added as feature layers. Each pixel is treated as a point feature.

• For Level 2, or any non-rectified models, use feature model instead of raster model.

Air Quality for US Metropolitan Areas • US Metropolitan areas and their 200km and 400km buffer zones

• The next steps would be to calculate statistics (min/max/mean/std/etc) for each zone and its buffer areas, and create tables/chart for time series of NO2, CO, etc.

• However, zonal analysis often hangs or errors for both AIRS and OMI data. AIRS data is in HDF4-based HDF-EOS and OMI is in netCDF (error message was not informative, e.g., background geoprocessing encountered a serious error. Need more tests).

Global O3: 2011/07/01 to 2011/12/31• Acquired from the GES DISC WCS server and imported into ArcGIS. Movie created using the

ArcGIS Time Slider tool.• http://acdisc.sci.gsfc.nasa.gov/daac-bin/wcsL3?

service=WCS&version=1.0.0&request=GetCoverage&Coverage=OMTO3e:ColumnAmountO3&CRS=CRS:84&BBOX=-179.5,-89.5,179.5,89.5&resx=1.0&resy=1.0&Time=2011-07-01/2011-12-31&format=netcdf

Global O3: 2011/07/01 to 2011/12/31• Acquired from the GES DISC WCS server and imported into ArcGIS. Movie created using the

ArcGIS Time Slider tool.• http://acdisc.sci.gsfc.nasa.gov/daac-bin/wcsL3?

service=WCS&version=1.0.0&request=GetCoverage&Coverage=OMTO3e:ColumnAmountO3&CRS=CRS:84&BBOX=-179.5,-89.5,179.5,89.5&resx=1.0&resy=1.0&Time=2011-07-01/2011-12-31&format=netcdf

Questions?