2015 GLM Annual Science Team Meeting: Cal/Val Tools Developers Forum 9-11 September, 2015 DATA...

2015 GLM Annual Science Team Meeting:Cal/Val Tools Developers Forum

9-11 September, 2015

DATA MANAGEMENTFor GLM Cal/Val Activities

Helen ConoverInformation Technology & Systems CenterUniversity of Alabama in [email protected]

2GLM Science Team Meeting Cal/Val Developer's Forum

Overview• UAH will work with the GLM Science Team to

provide data management for GLM Cal/Val activitieso GOES-R Field Campaign Portalo GLM Level 0 Data Storage at MSFCo GLM Cal/Val Data Portal

• These tools and services willo Leverage GHRC skills and processeso Re-use GHRC developed softwareo Not be part of the GHRC core data center infrastructure

9/9-11/2015

3

Global Hydrology Resource Center• NASA Distributed Active Archive Center

(DAAC) providing data ingest, routine and custom processing, archive, distribution, user support, and science data services

• Collaboration between NASA and the University of Alabama in Huntsville to infuse advanced information technologies to a variety of science data projects

• Global lightning data from space, airborne and ground based observations from hurricane science field campaigns and Global Precipitation Mission (GPM) ground validation experiments, and satellite passive and active microwave products

9/9-11/2015GLM Science Team Meeting Cal/Val Developer's Forum

http://ghrc.nsstc.nasa.gov/




9/9-11/2015


GOES-R Field Campaign Portal

• Collaboration Portal for planning and coordination o Project coordination docs (science plan, logistics, etc.)o Access to ancillary data and quicklooks during campaign o Mission science, project and instrument status reports

• Data Management o Science data product acquisitiono Documentation and publication

9/9-11/2015

• Current instrument status, weather forecasts, plan of the day

• Corporate knowledge (meeting minutes, presentations, etc.)

• Space to upload and share flight plans, mission reports and other documents

• Password protected for field campaign team access only

Collaboration and Planning Tools


6

• Mission information (mission objectives, science team, instrument characteristics, etc.)

• Near-real-time ground-based, satellite and/or aircraft data• Forecasting resources• Archive and distribution of all data products

Field Catalog


7

• Maps with instrument locations and base layers (state/county boundaries, roads, river networks)

• View current forecasts and ground-based, satellite and/or aircraft observations

Data Visualization


8

GOES-R Field Campaign Portal:Rough Timeline• FY 2016 – Planning and preparation phase

o Acquire and configure field campaign data servero Customize portal platform for GOES-R campaigno Develop preliminary metadata and documentation

• FY 2017 – Field phaseo Test and refine portalo Set up near real time data feeds o Support reporting, data visualization, etc. during campaign

• FY 2018 – Data acquisition and documentationo Acquire quality controlled datasets from instrument teams;

translate to standard format as neededo Work with science team to develop metadata and

documentation, based on information developed for earlier datasets from the same instruments, if available

o Provide data access and visualization services to public

9/9-11/2015 9GLM Science Team Meeting Cal/Val Developer's Forum




9/9-11/2015


GLM L0 Storage at MSFC

9/9-11/2015

Ingest, store and provide access to Global Lightning Mapper Level-0 (GLM L0) data from the GOES-R satellite series

• NOAA’S GOES-R Level 0 Storage System (LZSS)o Access is strictly limited

o Data are maintained in rolling storage for 2 years only

• Planned local GLM LZSS at MSFCo UAH personnel (shared with GHRC data center) will

procure, configure and administer the storage system

o GLM lightning team members will download data from the NOAA GOES-R LZSS to the local system


GLM LZSS at MSFC - Plan

9/9-11/2015

Sun ZFS Storage ZS3-2 system, with Sun ZFS file system for advanced error detection and self-healing capabilities

• Procure initial system in FY2016o Storage capacity 50 TBo Goal to have in place for DOE#4

(Data Operations Exercise), June 2016

o Ready to support GOES-R testing and initial operations by FY2017

• Add expansion unit each yearo Additional capacity 70 TBo Sized for data from two operational

satellites (32 TB/year per GLM instrument)




9/9-11/2015


GLM Cal/Val Portal• Core functions

o Provide access to GLM Level 0, (Level 1?), Level 2 & reference data

o Provide basic tools for data discovery and accesso Host or provide pointers to GLM Cal/Val tools

• Audienceo GLM Cal/Val community, not for public data accesso Password protection for restricted data

9/9-11/2015


GLM Cal/Val Portal - Data• Data held at MSFC:

o GLM L0 on local LZSSo GLM L2 ?o TRMM LIS, ISS LIS, NA LMA and DC LMA publicly

available from GHRCo NLDN, GLD360, ENTLN, WWLLN at GHRC for lightning

team only

• What additional lightning data do we need to acquire?

• What other lightning data can we point to? • ABI to compare with GLM?

9/9-11/2015


GLM Cal/Val Portal - Tools• What discovery / access tools are needed?

o Basic catalog for data and documentso Data access services (e.g., WMS, IDL or Python API)o Visualization tools (maps, graphs)

• What cal / val tools are needed?o Basic tools catalogo Tools to host on portal server

• Web-based• For download

o Tools to point to

9/9-11/2015

Discussion

What else do you need in a Cal / Val portal?

Helen [email protected]

9/9-11/2015 17GLM Science Team Meeting Cal/Val Developer's Forum

2015 GLM Annual Science Team Meeting: Cal/Val Tools Developers Forum 9-11 September, 2015 DATA...

Documents

Transcript of 2015 GLM Annual Science Team Meeting: Cal/Val Tools Developers Forum 9-11 September, 2015 DATA...