AMRC Meteorological Data Distribution System Part I: Data Sources and Archive
description
Transcript of AMRC Meteorological Data Distribution System Part I: Data Sources and Archive
![Page 1: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/1.jpg)
AMRC Meteorological Data Distribution System Part I:Data Sources and Archive
K. Elena Willmot, Matthew A. Lazzara, Linda M. Keller, Nicole M. Schroeder
Antarctic Meteorological Research CenterSpace Science and Engineering Center
University of Wisconsin-Madison
![Page 2: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/2.jpg)
Introduction
o AMRC since 1992...o Part I Outline:
o Data SourcesInternal & External
o Data Formatso Data Archive
Historical & Present
o Distributiono Challenges
No more FTP?
Access and quality of data
Lack of acknowledgements
o Statistics
(Photo by Michael Forster Rothbart, UW-Madison University Communications)
![Page 3: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/3.jpg)
Data Sources: Internal
o AWS (Automatic Weather Station) datao Significant portion of internal data comes from UW’s AWS networko Real-time and archive quality data sets
o Ten minute intervalso Hourly intervalso Three-hourly intervals
o 60+ weather stations across the continent
![Page 4: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/4.jpg)
Data Sources: Internalo Antarctic Composites
o Conceived by UW’s Professor Charles R. Stearns in October 1992.o Cover South Pole region to about 40° South.o Satellites used:GOES, GMS, MTSAT, METEOSAT, FY-2, KALPANA,
NOAA, DMSP, Aqua & Terra, and MetOp
o Created every 3 hours/1 hour every dayo Currently, 4 sizes:
o 512 x 512o 683 x 683o 1024 x 1024 (standard)o 2048 x 2048
o 6 types:o Infraredo Water Vaporo Experimental Visibleo Long-wave (Coming soon!)o Short-wave (Coming soon!)o Pseudo Color
![Page 5: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/5.jpg)
Making Antarctic Composite Satellite
Imagery:
![Page 6: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/6.jpg)
Data Sources: Internalo University of Wisconsin’s Space Science and Engineering Center
(SSEC) Data Centero Surface synoptic observationso METARs (Meteorological Aviation Reports)o RAOB (Radiosonde Observations)o Ship/buoy observations
o Numerical Models from GFS
(Global Forecast System) and ECMWF
(European Centre for Medium-Range
Weather Forecasts). o Polar-orbiting LAC (Local Area Coverage) observations from NOAA
satellites
![Page 7: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/7.jpg)
Data Sources: Externalo Participation from various researchers, USAP (United States
Antarctic Program) participants, and international groups within the Antarctic meteorological community
o International AWS (real-time only)o USAP research vessel observationso Field experiment radiosonde observationso Surface observations at:
o Palmer Stationo McMurdo Stationo South Pole Stationo Various other field camps
Lawrence M. Gould. Photo by: Al Hickeyhttp://photolibrary.usap.gov/
![Page 8: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/8.jpg)
Data Flow
International AWS
USAP Research Vessels
McMurdo, Palmer & South
Pole Stations
WisconsinAWS
AMRC Office
SSEC Data Center
NOAAHRPT
DMSPRTD
Aqua/Terra DB (McMurdo only)
NOAAGAC/LAC
MTSAT
Meteosat (2)
GOES (2)
FY-2C
Kalpana-1
Aqua/Terra NASA/NOAA
Products:
![Page 9: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/9.jpg)
Data Formats
Archive:o McIDAS
o AREAo MDo GRID
o ASCII Texto GIF, JPEG
Distribution:o McIDAS
o AREAo MDo GRID
o ASCII Texto netCDF o GIF, JPEG
![Page 10: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/10.jpg)
Data Archive: Historical
o Since 1992...o Media used to archive data:
1. Magneto-optic disks2. Compact disks (CD)3. Digital video disks (DVD)4. 8 millimeter Exabyte tapes5. DDS3 tapes (4 mm tapes)6. 3590 IBM tapes7. LTO tapes
1
2
3
4
5
6
7
![Page 11: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/11.jpg)
Data Archive: Presento RAID/5 disk arrays as well as DVDs are most prominent todayo FTP site most widely used
o Updated monthly witho AWS – SPAWAR obso USAP research vessel observationso McMurdo Station obso Palmer Station obso South Pole Station obs
o Summary / notification email sent out
![Page 12: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/12.jpg)
Distributiono Data Services to be discussed in Part II:
o FTP (File Transmission Protocol)o Rsync Serviceo Antarctic-IDD (Internet Data Distribution) LDM (Local Data Manager)o ADDE (Abstract Data Distribution Environment)o E-mailo Websiteo Future:
o RAMADDA (Repository for Archiving, Managing and Accessing Diverse Data)o GIS (Geographic Information Systems) Software
![Page 13: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/13.jpg)
Challenges:1o USAP moving away from FTP site – advancing security ruleso Don’t fret! FTP will still be available here at the AMRC:
o Other options: Link with external systemo Limited resources to maintain linkso Manually providing data is expensive and not desired
o We aim to provide data in a more semi-automated, self-service fashion.
![Page 14: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/14.jpg)
Challenges:2o Accessing data externally
o Poor qualityo Corrupt fileso Missing
o Gaps in the archiveo Which data can be used / distributed and which cannot?
o Third party redistribution limitations
o Evolving toward nearly fully automated data flow system (from source to archive)o Maintenance will be a constant issue – especially at the sourceo Documentation
![Page 15: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/15.jpg)
Challenges:3 - Metadatao Ever increasing need for Metadatao Significant effort to research/document and provideo Incomplete historical records, leave some metadata information
missing…o E.g. South Pole...
![Page 16: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/16.jpg)
We have found an issue with the South Pole data:
From 1996 and earlier, the South Pole LCDs (Local Climate Data) used data that switched between using local time and GMT time when recording observation times.
South Pole is working on re-running past LCDs and we do not know how long it will take / when they will be completed.
![Page 17: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/17.jpg)
Acknowledgements
The authors appreciate the support of the Automatic Weather Station Program and/or Antarctic Meteorological Research Center for the data set, data display, information, etc. ([AMRC/AWS staff member, as appropriate], NSF grants number ANT-0636873, and/or ANT-0838834)
Sculptor Gabriel Warren used a transparency of the B-15A Iceberg in one of his pieces called “Interrogation: Dialogue” in his “Rhegmalogia” collection.
http://www.art-farm.net/homeGabe.htm
![Page 18: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/18.jpg)
AMRC/AWS Statistics
o Show Gigabytes transferred / month on Website.o Average of 70-80 Gb year-round.
o Gigabytes transferred / month on FTP site.o Generally very variable – maybe due to field season.
![Page 19: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/19.jpg)
• ANTIDD – Antarctic-IDD mailing list• ANTDISCUSSION – Antarctic meteorology
discussion group• ANTAWS - Information and updates on
AWS data• ANTOBS - Information and updates on all
other observational datasets• ANTOPS - Notices on AMRC’s real-time
data availability, distribution and status• ANTICEBERG - Notices on latest Iceberg
monitoring images posted to the web site• ANTCOMMUNITY - Infrequent notices to
the Antarctic community on events and significant developments
Please join our mailing lists!:
How to subscribe:Visit our new website and click on the “Subscribe” button
next to the mailing list you wish to join:
![Page 20: AMRC Meteorological Data Distribution System Part I: Data Sources and Archive](https://reader035.fdocuments.us/reader035/viewer/2022062500/5681592f550346895dc65ed1/html5/thumbnails/20.jpg)
THANK YOU!
Acknowledgements:
National Science Foundation Office of Polar Programs Grant #0838834South Pole Meteorology Office/Raytheon Polar ServicesSPAWAR Office of Polar ProgramsMcMurdo Station Research AssociatePalmer Station Research AssociateUSAP Research Vessels – LM Gould & NB PalmerMcMurdo Ground StationInternational partners& More
Questions?