Data Merge Examples

,

Toolsets for Airborne Data (TAD): Customized Data Merging Function

ASDC Introduction

The Atmospheric Science Data Center (ASDC) at NASA Langley Research Center is responsible for the ingest, archive, and distribution of NASA Earth Science data in the areas of radiation budget, clouds, aerosols, and tropospheric chemistry. The ASDC specializes in atmospheric data that is important to understanding the causes and processes of global climate change and the consequences of human activities on the climate. The ASDC currently supports more than 44 projects and has over 1,700 archived data sets, which increase daily. ASDC customers include scientists, researchers, federal, state, and local governments, academia, industry, and application users, the remote sensing community, and the general public.

Weighted Merge

Most aircraft data reported during a single flight is not on a consistent time base and is difficult to intercompare. The TAD merge module provides the user with the ability to merge original data measurements from multiple data providers into a specific time interval or common time base.

In 2012, the ASDC facilitated the development of the first ever strategic plan intended for fiscal year 2013 and beyond. The 2013 Strategic Plan serves as a mission-focused plan with six defined goals, each with supporting objectives and tasks for implementation that emphasize the vision and support the mission and values of the ASDC.

Amanda Benson2, Marie Christina Peeters3, Jennifer Perez1, Lindsay Parker2, Gao Chen1

1. Atmospheric Scence Data Center, NASA Langley Research Center, Hampton VA 2. Science Systems and Applications, Inc., Hampton, VA 3. Langley Aerospace Research Student Scholars (LARSS), NASA Langley Research Center, Hampton, VA

Goal #1The ASDC will strive to expand beyond its

existing customer base by increasing accessibility to a broader, worldwide

market; through the use of innovative technologies, the ASDC will enhance data access capabilities and develop plans to share data with new user communities.

The ICARTT format in which the field campaign data is stored lacks standardization. To compensate, an extensive metadata database was created to track all Principal Investigator-provided metadata and to link the measurement variables to a common naming system. This database is used by data merge module

Pursuant to this goal, the ASDC is exploring and piloting new technologies to implement for enhanced data access capabilities. The aircraft datasets collected by ASDC pose unique challenges to science data users due to the sheer volume and variety of the data and the lack of intuitive features of the order tools available to the investigator. To meet the needs of emerging users in the aircraft community, the ASDC is addressing issues in data discovery and delivery.

Strategy & Innovation

Airborne Tropospheric Chemistry Studies

NASA has conducted airborne tropospheric chemistry studies for about three decades. These field campaigns have generated a great wealth of observations, including a wide range of the trace gases and aerosol properties. The ASDC Toolset for Airborne Data (TAD) is being designed to meet the user community needs for manipulating aircraft data for scientific research on climate change and air quality relevant issues.

Timebase Data Merge

Data ID Timebase

Merge Timebasetmi

Δtmi Δtpj

tpj

, if

58000 58100 58200 58300 58400 58500 58600 58700 58800 58900 590000.06

0.07

0.08

0.09

0.1

0.11

0.12

Original Acetone data

58000 58100 58200 58300 58400 58500 58600 58700 58800 58900 590001

1.2

1.4

1.6

1.8

2

2.2

2.4

2.6

2.8

3

60 Second Merged Acetone data

58000 58100 58200 58300 58400 58500 58600 58700 58800 58900 590000

0.5

1

1.5

2

2.5

3Data ID Timebase Merged Acetone data

Data ID Timebase Merged Acetone data Data ID Original data

, if

, if

PI data valueMerged

data value

Metadata Database

Variable Common Naming System

The naming schema used by data providers for their data variables is not standardized. In order to link like variables, a variable common naming system was developed.

#:#:#(#)_name

Level OneLevel Two

Level ThreeFrequency Flag

Variable Common Name

Aerosol: Microphysical Properties 1:1:0Aerosol: Optical Properties 1:2:0Aerosol: Composition 1:3:0Trace Gases: Hydrogen and Oxygen Compounds 2:1:0Trace Gases: Nitrogen Compounds 2:2:0Trace Gases: Sulfur Compounds 2:3:0Trace Gases: Halocarbons and Halogens 2:4:0Trace Gases: Carbon and Hydrocarbon Compounds: Alkanes 2:5:1Trace Gases: Carbon and Hydrocarbon Compounds: Alkenes & Alkynes 2:5:2Trace Gases: Carbon and Hydrocarbon Compounds: Aromatics 2:5:3Trace Gases: Carbon and Hydrocarbon Compounds: Oxygenates and Acids 2:5:4Trace Gases: Other Trace Gases 2:6:0Cloud Properties 3:0:0J values 4:0:0Meterological and Nav Parameters: Navigational Parameters 5:1:0Meterological and Nav Parameters: Meterological Parameters 5:2:0Radiation Variable 6:0:0Frequency Flag 0 = Infrequently Used Variable 1 = Frequently Used Variable

Way Forward

The metadata database and common naming scheme will be used by the merge module within the TAD User Interface to allow for users to merge and download data.

Future plans include:• Standardized ICARTT format• Spatially and temporally subsetted files• Data visualization capabilities• Spatial grid data/stats

DISCOVER-AQ 2013 P3B Flight Path