An Overview of Geospatial Standards Support within the ArcGIS Platform

Satish Sankaran

Marten Hogeweg

Agenda

• Interoperability Challenges

• Standards Organizations and Standards

• Standards – Data formats and API, Metadata and Web Services

• ArcGIS – Standards Support

• Pragmatic Interoperability (de-facto vs de-jure standards)

• Demonstrations

Interoperability Challenges

- Multiple Platforms ( Operating Systems, Databases, …)

- Multiple Architecture ( Local, Enterprise, Web, Cloud,…)

- Multiple Clients ( Desktop, Web, Mobile, …)

- Multiple Developer Environments (.NET, Java,…)

- Multiple Protocols (SOAP, REST, OGC,….)

- Multiple Encodings ( XML, Raster Formats, GML, JSON, ….)

Cloud

Enterprise

Local

• Create• Manage• Visualize• Analyze• Collaborate• Discover

Mobile

Desktop

Web

The ArcGIS Platform

- An Open & Interoperable Platform

- Supporting a thriving ecosystem of focused apps and applications.

- A Standards Compliant Platform

- Supporting many relevant standards including those from ISO TC 211 and OGC.

http://www.esri.com/software/open

Standards

ETL Open

Interoperability

Procurement personnel

System Integrator

Developer

User

Many Standards Organizations …….

ISO

OGC

ANSI/INCITS

OASIS

IHO

WS-I

OMA

FGDC

GSDI

CEN

DGIWGACSM ASPRS EPSG

WFS

WMS

SOS

GML

WPS

ISO 19139

WCS Simple Features

OWS Context

ISO 19115

GeoXACMLFGDC - CSDGM

GeopackageCityGML

Open LS

Many Standards ……

GIS Standardization – functional areas

Basic Geographic Structures (semantic interoperability)

Geometry, topology, coordinate reference systems

Content Description (semantic interoperability)

Standardized Data Models, Feature Catalog

Modeling languages, Application schemas

Metadata

Data Management (semantic and technical)

Database management

Data Formats (technical interoperability)

Structure, Syntax, language, API

Visualization (semantic and technical)

Symbology, cartography, display

GeoWeb Services (technical interoperability)

Web Catalog, Web Services, SOA

Basic Geographic StructuresISO 19107 – Spatial Schema, ISO 19111 – Coordinate Reference Systems

Content DescriptionFACC, SDSFIE, Standardized Data ModelsISO 19110 – Feature Catalog Methodology , UML, FGDC Metadata, ISO 19115 – Metadata

Data ManagementOGC Simple Features

Data FormatsVPF, OGC GML, ISO 19139, 80+ industry and national formats

VisualizationSLD, MIL-STD 2525

Web ServicesOGC WMS, WFS, WCS, OpenLSRegistry and catalog services

CAT 1.0, OAI-PMH, CS-W (ebRIM)

GIS Standardization – functional areas

Standards

Data Formats

Metadata

Web Services

Data Formats

• Simple Features

• GML

• KML

• Geopackage

• Gml JPEG 2000

• OWS Context

• WaterML, AiXM, …

• CAD formats

• IFC

• NIEM

• …..

What are simple features?

• Simple Feature specification defines:- Data access model- Database schema- 2D Geometry model for points, lines, polygons- “Well-Known” data formats for geometry and spatial reference

• ISO and OGC specifications for simple feature access

10

11

12

area1

area2

area3

yellow

green

Blue

13 area4 red

Feature

Feature AttributeGeometry

Feature Table

Feature Tables contain rows (features) sharing common properties (Feature Attributes).

Geometry is a Feature Attribute.

Geography Markup Language (GML)

• GML or Geography Markup Language is an XML based encoding standard for geographic information developed by the Open Geospatial Consortium (OGC).

• Defines an XML schema for representing geographic features (attributes, geometries, relationships, Topologies, Observations, etc).

• GML Simple Features

- Initially motivated to help WFS use of GML 3

- GML 3 has many optional elements of schema, which complicates GML parsers

- GMLSF Profile “fixes” application schemas to only allow the elements prescribed in the profile

- This allows WFS servers and clients to be simpler and more efficient; the types of data which they need to anticipate are finite and reasonably structured for faster parsing

GML Support in ArcGIS

• GML Support in ArcGIS Desktop- Simple Feature GML support is available to all ArcGIS users

- Import/export data- Use directly as a map layer- Use in a geoprocessing model- Consume GML served by any Web feature Service (WFS)

• GML Support in ArcGIS Desktop (ArcMap) using the Data Interoperability Extension

- Support for specific Application schemas available using the Data Interoperability Extension.

- Several application schemas supported- Custom schemas defined using transformation tools- Consume GML served by any Web feature Service (WFS)

• GML Support in ArcGIS Server – Web Feature Service (WFS) implementation- GML is streamed using the Web Feature Service Protocol

Kml Support in ArcGIS

KML is an XML grammar used to visualize geographic data in an earth browser.KML uses a tag-based structure with nested elements and attributes and is based on the XML standard.

• Serve– Using ArcGIS Server

• Geoprocessing Tools

– Layer To KML Tool• The Layer To KML geoprocessing tool allows individual • layers to be exported directly from ArcMap, ArcGlobe or ArcScene.

– Map To KML Tool• The Map To KML geoprocessing tool allows multiple layers to be • exported into a single KML source.

• Consume– ArcGIS Desktop (ArcMap,

PRO)– Runtime – ArcGIS Earth– ArcGIS Online / Portal

ArcGIS Data Interoperability Extension

• Format Support –Adds support for 200+ data formats–Custom Formats–Direct use in ArcGIS Desktop

• Quick Data Translation –Data Interoperability Tools Toolbox–Convert between data formats

• Spatial ETL (Extract, Transform, Load)–Semantic Data Translation–Data Restructuring

Standards

Data Formats

Metadata

Web Services

• Data about data

- Geographic area covered

- Currency

- Rules of use

- Positional accuracy

- Data dictionary

- Means of encoding

- Datum

- Map projection

• A Key interoperability technology:

- Producers explain their product

- Users learn about products

• Non standard products can be understood

International Metadata Standards

• FGDC CSDGM

• ISO 19115, 19119 Metadata

• ISO 19139 Metadata XML Schema Implementation

• …

Metadata

Leading the way to semantic interoperability

ISO Metadata ( Today Future )

Ted Haberman : hdfcgroup.org

Metadata standards today

<xsd:element name="bounding" type="boundingType"/><xsd:complexType name="boundingType">

<xsd:sequence><xsd:element ref="westbc"/><xsd:element ref="eastbc"/><xsd:element ref="northbc"/><xsd:element ref="southbc"/>

</xsd:sequence></xsd:complexType>

Content Standard for Digital Geospatial MetadataAka “FGDC Metadata” ISO 19115 Geospatial

Information - MetadataAka “ISO Metadata”

Extensions Profiles

Overview of Metadata Standards

ISO 19115 (-1) Profiles

• A metadata profile for precision agriculture based on ISO 19115 standard.

• WMO Core Profile of the ISO 19115 Metadata Standard

• Transformation of HDF-EOS metadata from the ECS model to ISO 19115-based XML

• Draft African metadata profile of ISO19115• Samoan profile• NOKIS – an ISO 19115 Based Metadata

System• The UDK and ISO 19115 Standard• Latin American Profile• Núcleo Español de Metadatos (NEM)• UK GEMINI• Biological profile• Coastal zone profile• S100 Metadata profile (IHO)

• Australia-New Zealand Profile• North American Profile of ISO

19115:2003 (NAP)• CAN/CGSB-171.100-2009• INCITS 453-2009

• INSPIRE Metadata Directive• ISO19115/ISO19119 Application Profile

for CSW 2.0• C3Grid ISO 19115 Metadata Profile• Marine Community Metadata Profile of

ISO 19115• Energy Industry Profile of ISO 19115-1• USGIN ISO 19139 Profile• DISDI Geospatial Metadata Profile

(DGMP)• National System for Geospatial

Intelligence Metadata Foundation (NMF)

ArcGIS Metadata Standards Support

FGDC CSDGM XML

ISO 19139 (NAP) XML

ISO 19139 (INSPIRE) XML

ISO 19139 XML

ArcGIS Metadata

ArcCatalog

ArcGIS Is a Platform

Available in the Cloud . . . . . . and On-Premises

SimpleIntegratedOpen

Desktop Web Device

Server Online Content and Services

Portal

Enabling GIS Everywhere

Author / CreateUse

Serve / Publish

Catalog / Discover / search

Metadata

ArcGIS Platform and Metadata

ProductsArcGIS Desktop – ArcMap, PROArcGIS for Server ArcGIS Online / Portal for ArcGISArcGIS OpenData AppGeoportal ServerApp Widgets

Metadata Formats• Item Information

• ArcGIS Metadata

• Formal Metadata – FGDC, ISO, …

• DCAT

• Encodings – esri JSON, XML, atom,

Items Carrying Metadata• Portal and Online Items

• Mxd’s, Datasets, Services

• Folders

• …

Indexing / Access / Discovery• GeoServices REST Spec

• OpenSearch

• OGC CS-W

Esri support for metadata

ArcMap

Geoportal

ArcGIS PRO

ArcGIS OpenData

ArcGIS OnlineMxd properties

Service Metadata

Item Info

WMS metadata

ArcGIS service meatadata

What’s Next for ArcGIS?

• Investigating how to incorporate support for ISO 19115-1/19115-3 into the ArcGIS platform

- 19115-3 incorporates existing ISO 19115-2 and 19110 XML schemas- Evaluating XML schemas available in new ISO repositories

• Continue to Improve metadata support in ArcGIS Pro and throughout the platform- Desktop -> Server -> Online/Portal- Planning next steps

Standards

Data Formats

Metadata

Web Services

OGC Web Services

• Web Map Service

• Web Map Tiling Specification

• Web Feature Service

• Web Coverage Service

• OGC KML

• Sensor observation Service

• Web Processing Service

• 3D Portrayal service

• ….

from … Geospatial Interoperability Reference Architecture (GIRA)

http://ise.gov

DGIWG and OGC Service Types and

Profiles

https://www.ise.gov/sites/default/files/GIRA.pdf

Geospaial Interoperability

Reference Architecture doc

Report – Spring 2015

Server

Providing

Developer Solutions(runtime, Javascript

Api)Consuming

Portals(AGOL, Portal)

Providing & Consuming

Desktop(ArcMap & PRO)

Consuming

Metadata(ISO, FGDC, …)

Data (Geopackage, geoJSON, gml-JPEG 2000…)

Services(wms,wfs,wcs,

wps,sos,cs-w, …)

Portrayal Standards

Core StandardsArcGIS Platform Components

AppsConsuming

Esri REST

API

WM

S

WM

TS

WFS

ArcGIS for InspireInspire W

MS

Inspire WFS

Inspire ATO

M

WC

S

OGC clients (HTTP SOAP)

Inspire clients(HTTP SOAP)

GeoJSO

N

JSON clients(HTTP REST)

WP

S

ArcGIS for Server

Geoportal

Inspire CSW

KM

L

http://www.esri.com/library/whitepapers/pdfs/supported-ogc-iso-standards.pdf

Standards Requirements – Across the platform ….. What we are hearing!

• Online / Portal- Register OGC services- Consume OGC services- Create hosted OGC services- Geopackage support- Allow analyses on OGC services - Support Discovery Standards – OGC CS-

W, OpenSearch, DCAT- Managing Secured OGC Services

• Metadata Support across the platform• Federated searches across portals

• Server- Improve existing OGC support (

compliancy, update to new version, new profiles)

• Desktop (ArcMap / Pro)- Consume OGC services- Geopackage support

• Runtime and Apps- Consume OGC services- Geopackage support

• Javascript API- Consume OGC services

• ArcGIS Online

- Support for formal Metadata (Version 3.8, July 2015)

- Support for secured OGC services (Version 3.10, Nov 2015)

- Major improvements in Online (version 4.2, June 2016 ) – wms, wfs

• ArcGIS for Server- Improvements to wfs 2.0 support- Achieved KML Compliancy (Server 10.4)

• ArcGIS PRO

- Improvements to wms wmts and metadata support in PRO (PRO 1.2)

- Improvements to KML support ( Pro 1.3)

• ArcGIS Earth 1.1

- Supports kml, wms and wmts

Standards Support

Support for Hosted wms, wmts and wfs support in Online (Q4 2016) and Portal ( Version 10.5)

Native wfs client support in PRO (Version 1.4)

WCS 2.0.1 support in ArcMap and PRO

Support for wms, wmts ( Q4 2016 Quartz runtime)Support for geopackage, wfs , s57 in (Q1 2017 Quartz runtime)

WCS support in Javascript API (prototype)

Web App Builder widget to consume OGC OWS context document with embedded ogcservices (prototype)

Native netCDF Support in ArcMap (10.5) and PRO (10.4)

Better metadata support across the stack

Improvements to geopackage support

Standards Support - Roadmap

Procurement personnel

http://www.opengeospatial.org/resource/products/compliant#ESRI @ 10.4 : 16 certificates

SOA Vs REST

Use the Web Luke !

…. Some GIS Standards are still playing catchup with the Web

http://geekandpoke.typepad.com/.a/6a00d8341d3df553ef012875f312f9970c-pihttp://www.programmableweb.com/news/api-anti-patterns-how-to-avoid-common-rest-

mistakes/2010/08/13

Prag

mat

ic In

tero

pera

bilit

y

Standards, Specifications, ….

• geoJSON

• geoRSS

• Atom

• Sqlite

• Vector tiles

• DCAT / geoDCAT -AP

• CSS – Styling

• RDF

• No SQL

• xlsform

Prag

mat

ic In

tero

pera

bilit

y

• Opensearch + geoExtension

Open API’s and Specifications

• Shapefile (July 1998)

• Geodatabase XML (June 2008)

• GeoServices REST API (Sept 2010)

• FGDB API (Jan 2011)

• i3S (May 2015)

• LERC (Dec 2015)

Open Data DownloadStats for April 2016

Shapefile: 3,714,978CSV: 413,037GeoJSON: 318,052KML: 43,689GeoHash: 22,033

Prag

mat

ic In

tero

pera

bilit

y

The Proof of the Pudding…@martenhogeweg

Group on Earth Observations

GEOArcGIS Online

WMO WIS

GBIF

IOOS

GCMD

SeaDataNet

EuroGEOSS

EnviroGRIDSGSFC-DAAC

EEA-SDI

SAEOSINPE

PANGAEA

Connecting GIS and Earth Observations

Geospatial Platform and Data.gov

Geoplatform.govData.gov

EPA

DHS

DOJ

GSA

FCC

DOT

DHHS

NOAA

DOI

Census

NASA…

Access Geoplatform Content in ArcGIS

• Connect tohttp://geoplatform.maps.arcgis.com

• Find National Geospatial Data Assets• Use in Your GIS Projects

OGC Testbed 12

• Viewer- ArcGIS, WMS, WCS, KML- WMTS, Koop FeatureServer

• Discovery - OpenSearch- OGC CSW 2.0.2, 3.0.0

• OGC Context Document- GeoJSON encoding

Please take our SurveyYour feedback allows us to help maintain high standards and to help presenters

Find the session you want to review

Find your event in the Esri Events App

Scroll down to the bottom of the session

Answer survey questions and submit