Documenting UAF Data

Ted HabermannNOAA/NESDIS/National Geophysical Data Center

Metadata Types and Sharing

Discovery

Use / Mashup

Understanding

Discovery Portal

Community Metadata Collections

UserUser

DocumentationRepositoryISO 19115,

19115-2, 19119 and extensions

DocumentationRepositoryISO 19115,

19115-2, 19119 and extensions

THREDDSTHREDDS

netCDFnetCDF

DIF, FGDC,Data.Gov

DIF, FGDC,Data.Gov

SensorMLSensorML

WCS, WMS, WFS, SOS

WCS, WMS, WFS, SOS

ISOISO

KMLKML

Multiple Dialects

Designated Communities - Users

Data preservation is communicating with the future

CustomDocumentationCustomDocumentation

FGDC MandatoryIdentificationMetadata

FGDC MandatoryIdentificationMetadata

FGDC DiscoveryDistributorFGDC DiscoveryDistributor

FGDC CompleteEntity and AttributeData QualityLineageSpatial OrganizationSpatial Reference

FGDC CompleteEntity and AttributeData QualityLineageSpatial OrganizationSpatial Reference

FGDC RSEIdentifierDocumentationMissionPlatformInstrumentAggregationAlgorithmProcessing

FGDC RSEIdentifierDocumentationMissionPlatformInstrumentAggregationAlgorithmProcessing

ISO 19115Code ListsMany IdentifiersOnLine ResourcesResponsible PartiesData Quality ReportsSpatial/Temporal ExtentsMetadata Levels / Scope

ISO 19115Code ListsMany IdentifiersOnLine ResourcesResponsible PartiesData Quality ReportsSpatial/Temporal ExtentsMetadata Levels / Scope

ISO 19115-2Quality CoverageUsabilityPlatformInstrumentProcessingAlgorithm

ISO 19115-2Quality CoverageUsabilityPlatformInstrumentProcessingAlgorithm

FGDC to ISO 19115International StandardUnified Modeling Language

FGDC to ISO 19115International StandardUnified Modeling Language

Basic ContentBasic Content Extended ContentExtended Content

ISO 19119Services

ISO 19110Features

ISO 19119Services

ISO 19110Features

Standards Evolution

Spiral 1:Initial ContentSpiral 1:Initial Content

Metadata ContentIndependent of standardMetadata ContentIndependent of standard

StandardGuidance /

Implementation

StandardGuidance /

Implementation

Check BackWith UsersCheck BackWith Users

Spiral 2-N: Scientific QuestionsNew RequirementsNew Use Cases

Check BackWith Data Collectors/

Providers

Check BackWith Data Collectors/

Providers

IdentificationIdTitleAbstractResource DateTopic CategoryTheme KeywordMetadata ContactScience Contact

IdentificationIdTitleAbstractResource DateTopic CategoryTheme KeywordMetadata ContactScience Contact

ExtentGeospatial Bounding BoxTemporal Start/EndVertical Min/MaxPlace Keywords

ExtentGeospatial Bounding BoxTemporal Start/EndVertical Min/MaxPlace Keywords

DistributionDistributor ContactOnline ResourceDistribution FormatData Center KeywordsBrowse Graphic

DistributionDistributor ContactOnline ResourceDistribution FormatData Center KeywordsBrowse Graphic

Text SearchesPurposeExtent DescriptionLineage StatementProject Keywords

Text SearchesPurposeExtent DescriptionLineage StatementProject Keywords

Quality/LineageSourcesProcess StepsQuality Reports / Coverages

Quality/LineageSourcesProcess StepsQuality Reports / Coverages

Acquisition InformationInstrumentPlatformInstrument KeywordsPlatform Keywords

Acquisition InformationInstrumentPlatformInstrument KeywordsPlatform Keywords

Content InformationAttribute TypeAttribute NamesAttribute DefinitionsAttribute Units

Content InformationAttribute TypeAttribute NamesAttribute DefinitionsAttribute Units

DiscoveryDiscovery UnderstandingUnderstanding

ConnectionOnlineResource:Linkage (URL)NameDescriptionFunction

ConnectionOnlineResource:Linkage (URL)NameDescriptionFunction

Spiral Development / Training: Potential Spirals

Spiral None 0-33% 34-66% 67-99% All

Identification

Connection

Extent

Distribution

Description

Content

Lineage

AcquisitionInformation

Metadata Evaluation for Dataset XDate: 2010/10/24

Spirals and Rubrics

NcML vs. ISO

Satellite

HIOOS

CoastWatch

NESDIS SST

ISO Scores (Initial)

ISO

Rub

ric S

core

FGDC - CoRIS FGDC - NOS UAF

This attribute exists in the NcML files extracted from data files on TDS. It was not considered in the initial translation, but can be combined with publisher information to form either distribution information or a service operation connectPoint

netcdf/@location = distribution | service

ISO

Rub

ric S

core

FGDC - CoRIS FGDC - NOS UAF

netcdf/@location = distribution | service

netcdf/attribute/@name=project/@value

This attribute provides information about the project that is responsible for the dataset. Initially this information was translated into an MD_AggregationInformation object with MD_InitiativeTypeCode=project.

As an alternative this attribute could be treated as a keyword with MD_KeywordTypeCode=project (19115-2012).

ISO keywords are associated with shared vocabularies (thesauri). If the project keywords come from a shared vocabularies, like the CF_Standard_Names, this convention would be a significant step towards increased interoperability.

The NASA Global Change Master Directory has an existing shared vocabulary for dataCenters.

netcdf/attribute/@name=project/@value

ISO

Rub

ric S

core

FGDC - CoRIS FGDC - NOS UAF

netcdf/attribute/@name=publisher/@value

This attribute provides information about the publisher of the dataset. Initially this information was translated into a CI_ResponsibleParty with CI_RoleCode = publisher.

As an alternative this attribute could be treated as a keyword with MD_KeywordTypeCode = dataCenter (19115-2012).

ISO keywords are associated with shared vocabularies (thesauri). If the dataCenter keywords come from a shared vocabularies, like the CF_Standard_Names, this convention would be a significant step towards increased interoperability.

The NASA Global Change Master Directory has an existing shared vocabulary for dataCenters.

netcdf/@name=publisher/@value =dataCenter keyword

ISO

Rub

ric S

core

FGDC - CoRIS FGDC - NOS UAF

NcML vs. ISO

Satellite

HIOOS

CoastWatch

Improvement Plan

1. Improve standard (add new keyword types)2. Create and encourage adoption of shared

vocabularies for discovery3. Improve translation from NcML to ISO4. Integrate ncISO into TDS5. Create good xsl stylesheets for documentation

display6. Encourage existing and new data providers to

support data discovery conventions7. Find partners for further improvement in NcML and

ISO

NGDC Ionosonde Group is starting to write time series data for many Ionosonde Stations. The NGDC Metadata Team has been working with them to help them understand use and discovery conventions for netCDF.

Station NcML ISO

Initial 3 13

Intermediate 20 17

Final 40 21

New Data Providers

Positive deviance says that if you want to create change, you must scale it down to the lowest level of granularity and look for people within the social system who are already manifesting the desired future state. Take only the arrows that are already pointing toward the way you want to go, and ignore the others. Identify and differentiate those people who are headed in the right direction. Give them visibility and resources. Bring them together. Aggregate them. Barbara Waugh

Leadership Model: Positive Deviance