Multidimensional Data and GIS
26
Multidimensional Data Multidimensional Data and GIS and GIS Steve Kopp Steve Kopp Nawajish Noman Nawajish Noman ESRI ESRI
description
Multidimensional Data and GIS. Steve Kopp Nawajish Noman ESRI. Multidimensional Data. Data cube (3D) or hypercube (4D,5D…) Temperature varying with time Temperature varying with time and altitude. T. Y. X. Multidimensional Data. Time = 3. Time = 2. Time = 1. Multidimensional Data. - PowerPoint PPT Presentation
Transcript of Multidimensional Data and GIS
Multidimensional Data Multidimensional Data and GISand GIS
Steve KoppSteve KoppNawajish NomanNawajish Noman
ESRIESRI
Multidimensional DataMultidimensional Data
Data cube (3D) or hypercube (4D,5D…)Data cube (3D) or hypercube (4D,5D…)
• Temperature varying with timeTemperature varying with time
• Temperature varying with time and altitudeTemperature varying with time and altitude
XY
TZ
XY
T
Multidimensional DataMultidimensional Data
Time = 1
Time = 2
Time = 3
Multidimensional DataMultidimensional Data
141 241 341
131 231 331
121 221 321
111 211 311
441
431
421
411
142 242 342
132 232 332
122 222 322
112 212 312
442
432
422
412
143 243 343
133 233 333
123 223 323
113 213 313
443
433
423
413
Y
X
TimeTime = 1
Time = 2
Time = 3
Data Cube
Time Slices
Multidimensional DataMultidimensional Data141 241 341
131 231 331
121 221 321
111 211 311
441
431
421
411Y
X
Time142 242 342
132 232 332
122 222 322
112 212 312
442
432
422
412
143 243 343
133 233 333
123 223 323
113 213 313
443
433
423
413
Time = 1
Time = 2
Time = 3
Multidimensional DataMultidimensional Data
141 241 341
131 231 331
121 221 321
111 211 311
441
431
421
411
142 242 342
132 232 332
122 222 322
112 212 312
442
432
422
412
143 243 343
133 233 333
123 223 323
113 213 313
443
433
423
413
Y
X
Time
Altitude
Includes variation in (x,y,z,t)
What is NetCDF?What is NetCDF?NetCDFNetCDF ((netnetwork work CCommon ommon DData ata FForm) orm)
A platform independent format for representing multi-A platform independent format for representing multi-dimensional array-orientated scientific data. dimensional array-orientated scientific data.
Self DescribingSelf Describing -- a netCDF file includes information about the a netCDF file includes information about the data it contains.data it contains.
Direct Access -Direct Access - a small subset of a large dataset may be a small subset of a large dataset may be accessed efficiently, without first reading through all the accessed efficiently, without first reading through all the preceding data.preceding data.
Sharable -Sharable - one writer and multiple readers may simultaneously one writer and multiple readers may simultaneously access the same netCDF file.access the same netCDF file.
NetCDF is new to the GIS community but widely used by NetCDF is new to the GIS community but widely used by scientific communities for around many yearsscientific communities for around many years
Why NetCDF?Why NetCDF?Most commonly used format in the oceanographic and Most commonly used format in the oceanographic and atmospheric science for observational data and numerical atmospheric science for observational data and numerical modelingmodeling
• The National Center for Atmospheric Research (NCAR)The National Center for Atmospheric Research (NCAR)• University Corporation for Atmospheric Research (UCAR) University Corporation for Atmospheric Research (UCAR)
NOAA's Climate Diagnostics Center (CDC) NOAA's Climate Diagnostics Center (CDC) • Los Alamos National Laboratory (LANL)Los Alamos National Laboratory (LANL)• The National Center for Supercomputing Applications The National Center for Supercomputing Applications • US Air Force and NavyUS Air Force and Navy• Atmospheric Research in AustraliaAtmospheric Research in Australia• Australia Defense Australia Defense • UK Hydrographic OfficeUK Hydrographic Office• NATONATO• ......
What is a NetCDF file?What is a NetCDF file?NetCDF is a binary fileNetCDF is a binary file
A NetCDF file consists of:A NetCDF file consists of:Global Attributes:Global Attributes: Describe the contents of the fileDescribe the contents of the fileDimensions:Dimensions: Define the structure of the data Define the structure of the data
(e.g Time, Depth, Latitude, Longitude)(e.g Time, Depth, Latitude, Longitude)Variables:Variables: Holds the data in arrays shaped Holds the data in arrays shaped
by Dimensionsby DimensionsVariable AttributesVariable Attributes:: Describes the contents of Describes the contents of
each variableeach variableCDL (network CDL (network CCommon ommon DData form ata form LLanguage) description takes the anguage) description takes the
following formfollowing formnetCDF name { netCDF name {
dimensions: ... dimensions: ... variables: ... variables: ... data: ... data: ...
} }
Storing Data in a netCDF FileStoring Data in a netCDF File
141 241 341
131 231 331
121 221 321
111 211 311
441
431
421
411
142 242 342
132 232 332
122 222 322
112 212 312
442
432
422
412
143 243 343
133 233 333
123 223 323
113 213 313
443
433
423
413
Y
X
Time
netcdf mynetcdf{dimensions:
X=4;Y=4;Time=UNLIMITED ;
variables:float X(X);float Y(Y) ;int Time(Time);float Temperature(Time, Y, X);
data:X = 10, 20, 30, 40;Y = 110, 120, 130, 140;Time = 31, 59, 90;
}
Storing Data in a netCDF FileStoring Data in a netCDF File
141 241 341
131 231 331
121 221 321
441
431
421
142 242 342
132 232 332
122 222 322
112 212 312
442
432
422
412
143 243 343
133 233 333
123 223 323
113 213 313
443
433
423
413
Y
X
Time
Time = 1
Y = 1
X = 1 to 4
netcdf mynetcdf{dimensions:
X=4;Y=5;Time=UNLIMITED ;
variables:float X(X);float Y(Y) ;int Time(Time);float Temperature(Time, Y, X);
data:X = 10, 20, 30, 40;Y = 110, 120, 130, 140;Time = 31, 59, 90;
Temperature =111,211,311,411;
}
Storing Data in a netCDF FileStoring Data in a netCDF File
142 242 342
132 232 332
122 222 322
112 212 312
442
432
422
412
143 243 343
133 233 333
123 223 323
113 213 313
443
433
423
413
Y
X
Time
Time = 1
Y = 1 to 4
X = 1 to 4
netcdf mynetcdf{dimensions:
X=4;Y=5;Time=UNLIMITED ;
variables:float X(X);float Y(Y) ;int Time(Time);float Temperature(Time, Y, X) ;
data:X = 10, 20, 30, 40;Y = 110, 120, 130, 140;Time = 31, 59, 90;
Temperature =111,211,311,411,121,221,321,421,131,231,331,431,141,241,341,441;
}
Storing Data in a netCDF FileStoring Data in a netCDF File
Y
X
Time
Time = 1 to 3
Y = 1 to 4
X = 1 to 4
netcdf mynetcdf{dimensions:
X=4;Y=5;Time=UNLIMITED ;
variables:float X(X);float Y(Y) ;int Time(Time);float Temperature(Time, Y, X);
data:X = 10, 20, 30, 40;Y = 110, 120, 130, 140;Time = 31, 59, 90;
Temperature =111,211,311,411,121,221,321,421,131,231,331,431,141,241,341,441,112,212,312,412,122,222,322,422,132,232,332,432,142,242,342,442,113,213,313,413,123,223,323,423,133,233,333,433,143,243,343,443;
}
NetCDF Data SourcesNetCDF Data Sources
• Community Climate Systems Model (CCSM) Community Climate Systems Model (CCSM) http://www.ccsm.ucar.edu, https://www.earthsystemgrid.org/http://www.ccsm.ucar.edu, https://www.earthsystemgrid.org/
• The CCSM is fully-coupled, global climate model that provides The CCSM is fully-coupled, global climate model that provides state-of-the-art computer simulations of the Earth's past, state-of-the-art computer simulations of the Earth's past, present, and future climate states. present, and future climate states.
• 100 yrs of climate change forecast data (2000-2099)100 yrs of climate change forecast data (2000-2099)• Control runs (1870-1999) and scenario runsControl runs (1870-1999) and scenario runs• Temperature, precipitation flux, surface snow thickness, snowfall Temperature, precipitation flux, surface snow thickness, snowfall
flux, cloud water content, etc.flux, cloud water content, etc.
• Program for Climate Model Diagnosis and Program for Climate Model Diagnosis and Intercomparison (PCMDI) Intercomparison (PCMDI) http://www-pcmdi.llnl.gov/http://www-pcmdi.llnl.gov/
NetCDF Data SourcesNetCDF Data Sources
• Vegetation and Ecosystem Modeling and Analysis Vegetation and Ecosystem Modeling and Analysis Project (VEMAP) Project (VEMAP)
http://dataportal.ucar.edu/vemap/main.htmlhttp://dataportal.ucar.edu/vemap/main.html• VEMAP was a large, collaborative, multi-agency program to VEMAP was a large, collaborative, multi-agency program to
simulate and understand ecosystem dynamics for the simulate and understand ecosystem dynamics for the continental United States. continental United States.
• The VEMAP Data Portal is a central collection of files maintained The VEMAP Data Portal is a central collection of files maintained and serviced by the NCAR Data Groupand serviced by the NCAR Data Group
• Climate data interval: Annual, monthly, and daily.Climate data interval: Annual, monthly, and daily.• Data type: Historical and model resultsData type: Historical and model results• Data: Temperature, irradiance, precipitation, humidity, incident Data: Temperature, irradiance, precipitation, humidity, incident
solar radiation, vapor pressure, elevation, land area, vegetation, solar radiation, vapor pressure, elevation, land area, vegetation, water holding capacity of soil, etc. water holding capacity of soil, etc.
NetCDF Data SourcesNetCDF Data Sources
• British Atmospheric Data Center (BADC) British Atmospheric Data Center (BADC)
http://badc.nerc.ac.uk/data/http://badc.nerc.ac.uk/data/• The role of the BADC is to assist UK atmospheric researchers to The role of the BADC is to assist UK atmospheric researchers to
locate, access and interpret atmospheric data.locate, access and interpret atmospheric data.• Many datasets are publicly available but datasets marked with Many datasets are publicly available but datasets marked with
key symbol have restricted access.key symbol have restricted access.• Datasets are organized by projects or organizations.Datasets are organized by projects or organizations.• Climatology Interdisciplinary Data Collection (CIDC) has monthly Climatology Interdisciplinary Data Collection (CIDC) has monthly
means of over 70 Climate Parameters.means of over 70 Climate Parameters.• Met Office - Historical Central England Temperature Data has Met Office - Historical Central England Temperature Data has
the monthly series, which begins in 1659, is the longest available the monthly series, which begins in 1659, is the longest available instrumental record of temperature in the world. The daily series instrumental record of temperature in the world. The daily series begins in 1772. begins in 1772.
NetCDF Data SourcesNetCDF Data Sources
• National Oceanic & Atmospheric Administration (NOAA)National Oceanic & Atmospheric Administration (NOAA)• National Digital Forecast Database (NDFD) National Digital Forecast Database (NDFD)
http://www.nws.noaa.gov/ndfd/http://www.nws.noaa.gov/ndfd/• Radar Integrated Display with Geospatial Element (RIDGE) Radar Integrated Display with Geospatial Element (RIDGE)
http://www.srh.weather.gov/ridge/http://www.srh.weather.gov/ridge/• Precipitation Analysis Precipitation Analysis
http://www.srh.noaa.gov/rfcshare/precip_download.phphttp://www.srh.noaa.gov/rfcshare/precip_download.php• Climate Diagnostics Center Climate Diagnostics Center http://http://www.cdc.noaa.govwww.cdc.noaa.gov//
NetCDF in ArcGISNetCDF in ArcGIS• NetCDF data is accessed asNetCDF data is accessed as
• RasterRaster• FeatureFeature• TableTable
• Direct read (no scratch file)Direct read (no scratch file)• Exports GIS data to netCDFExports GIS data to netCDF
Gridded DataGridded Data
Raster
Point Features
NetCDF ToolsNetCDF Tools
Toolbox: Multidimension ToolsToolbox: Multidimension Tools• Make NetCDF Raster LayerMake NetCDF Raster Layer• Make NetCDF Feature LayerMake NetCDF Feature Layer• Make NetCDF Table ViewMake NetCDF Table View• Raster to NetCDFRaster to NetCDF• Feature to NetCDFFeature to NetCDF• Table to NetCDFTable to NetCDF• Select by DimensionSelect by Dimension
Unidata NetCDF FilesUnidata NetCDF Filesby Cedric Davidby Cedric David
Catalog of DatasetsCatalog of Datasets http://motherlode.ucar.edu:8080/thredds/catalog.htmlhttp://motherlode.ucar.edu:8080/thredds/catalog.html
NCEP NAM 12km DataNCEP NAM 12km Data
North American ModelNorth American Model Continuous grid over the continental USContinuous grid over the continental US Model runs are made at 06Z, 12Z, 18Z and 00ZModel runs are made at 06Z, 12Z, 18Z and 00Z Forecasts every 3 hours out to 84 hoursForecasts every 3 hours out to 84 hours Accessed WithAccessed With
OPENDAPOPENDAP HTTPServerHTTPServer WCSWCS NetcdfServerNetcdfServer
Netcdf ServerNetcdf Server
Query for subset of data with Query for subset of data with Web FormWeb Form Download netCDF fileDownload netCDF file
Query Via URLQuery Via URL Format documented atFormat documented at
http://motherlode.ucar.edu:8080/thredds/docs/http://motherlode.ucar.edu:8080/thredds/docs/NetcdfServer.htmlNetcdfServer.html
Example:Example: http://motherlode.ucar.edu:8080/thredds/ncServer/http://motherlode.ucar.edu:8080/thredds/ncServer/
model/NCEP/NAM/CONUS_12km/model/NCEP/NAM/CONUS_12km/NAM_CONUS_12km_20060803_1800.grib2?NAM_CONUS_12km_20060803_1800.grib2?gridgrid==Total_precipitationTotal_precipitation&&northnorth==3535&&southsouth==3030&&westwest==-90-90&&easteast==-85-85&&time_starttime_start==0.00.0&&time_endtime_end==12.012.0
Data Type
Bounding Box
Start/End Time
NDFD Data Also AvailableNDFD Data Also Available
NWS National Digital Forecast Database NWS National Digital Forecast Database Gridded forecasts of sensible weather Gridded forecasts of sensible weather
elementselements 5km resolution over continental US5km resolution over continental US http://motherlode.ucar.edu:8080/thredds/http://motherlode.ucar.edu:8080/thredds/
catalog/model/NCEP/NDFD/catalog/model/NCEP/NDFD/CONUS_5km/catalog.htmlCONUS_5km/catalog.html
Viewing NetCDF in ArcMapViewing NetCDF in ArcMap
