GeoServer, GeoNetwork and INSPIRE: where we are and what is missing

Creare SDI INSPIRE Ready con GeoNetwork e GeoServer

Ing. Andrea Aime, GeoSolutions

Ing. Simone Giannecchini, GeoSolutions Ing. Emanuele Tajariol, GeoSolutions

Workshop di PTA Interreg, La Salle (AO) 8 Maggio 2012

http://twitter.com/geosolutions_it

Outline

Who is GeoSolutions?

What GeoServer can do for you

What GeoNetwork can do for you

INSPIRE

Where we are and where we should be


GeoSolutions

Founded in Italy in late 2006

Expertise

• Image Processing, GeoSpatial Data Fusion

• Java, Java Enterprise, C++, Python

• JPEG2000, JPIP, Advanced 2D visualization

Supporting/Developing FOSS4G projects

GeoTools, GeoServer

GeoNetwork, GeoBatch

ImageIO-Ext

Focus on Consultancy

agencies, large private companies, etc…


GeoServer

GeoSpatial enterprise gateway Java Enterprise

Management and Dissemination of

raster and vector data

Standards compliant OGC WCS 1.0, 1.1.1 (RI), 2.0 in the

pipeline

OGC WFS 1.0, 1.1 (RI), 2.0

OGC WMS 1.1.1, 1.3

OGC WPS 1.0.0

Google Earth/Maps support KML, GeoSearch, etc..


Fo

rmats

an

d P

roto

co

ls

Ge

oS

erv

er

WFS 1.0, 1.1,

2.0

WMS 1.1.1 1.3.0

PostGIS Oracle H2 DB2 SQL Server MySql Spatialite GeoCouch

Shapefile

---------- ---------- --------- ----------

---------- ---------- --------- ----------

---------- ---------- --------- ----------

ArcSDE WFS

PNG, GIF JPEG TIFF, GeoTIFF SVG, PDF KML/KMZ

Shapefile GML2 GML3 GeoRSS GeoJSON CSV/XLS

Raw vector data

Servers

Styled maps

DBMS

Vector files

WCS 1.0

1.1.1 GeoTIFF WMS ArcGrid GTopo30 Img+world Mosaic MrSID JPEG 2000 ECW,Pyramid, Oracle GeoRaster, PostGis Raster

Raster files

Raw raster data

GeoTIFF ArcGrid GTopo30 Img+World

GWC (WMTS,

TMS, WMS-C)

KML superoverlays Google maps tiles OGC tiles OSGEO tiles

WFSv*

Google*

WPS 1.0.0


Architecture

GeoTools

JTS ImageIO

GDAL

WCS

JAI

WMS WFS WFSv REST Google

Vector Raster

GWC

Style

Kakadu JAITools


Administration GUI


Integrated Security

namespace.layer.mode=role1,role2,...

A name, or * to mean “any”

r: read w: write

The roles that will be authorized to access the data under this rule

Spring Security

RBAC on layers and services

Built-in version with simplified rules


Integrated Security

private.* topp.*

TRUSTED_ROLE r/w r r r

(no access) r r/w r

(no access) r r r

topp.

districts

Other

layers

LEGISLATORS

(all other users)

*.*.r=*

*.*.w=NO_ONE

private.*.r=TRUSTED_ROLE

private.*.w=TRUSTED_ROLE

topp.districts.w=LEGISLATORS

RDONLY lock down

Per-layer override


GUI: security configuration


Security Developments in 2.1

Security Management at the feature level (record in DBMS) read and write filtering

Hide sensible attributes

Discriminate visible areas on a per-user basis

Rules for services and layers

Server side framework available, open source implementation needed!



Pluggable user sources, available out of the box:

LDAP

DBMS

Pluggable authentication mechanisms, available out of the box:

BASIC/DIGEST HTTP

CAS

Possible to integrate with other mechanisms and in-house solutions


RESTful Configuration

Programmatic configuration of layers via REST calls

Workspaces

DataStores/CoverageStores

Layers and Styles

Exposing internal configuration to remote clients

Ajax-JavaScript friendly

“Core” Module since 2.1

Integrated Security

Admin credentials required for modifications


WFS

Dissemination and filtering of vector data

WFS 1.0 and 1.1 support with transactions

WFS 2.0 on trunk

Formats:

GML 2, 3.1 e 3.2

GeoRSS, GeoJSON

Shapefile (zipped)

OGR (configurable)

Extensions

CQL

WFS 1.0 with on-the-fly reprojection

Workshop di PTA Interreg, La Salle (AO)

8 Maggio 2012

WCS*

Raster data dissemination

Raw raster data useful for analysis, no maps!

Support for TIME and ELEVATION (via ImageMosaic plugin)

WCS 1.0 e 1.1.1

WCS 2.0 being discussed

Output formats

GeoTiff, ArcGrid

GDAL based formats*

Extensions

ELEVATION as band management*

Refactor Ongoing!


WMS

Dissemination of Maps

Fusing raster and vector data seamlessly

With styling!

WMS 1.1.1 and 1.3 support

INSPIRE!

SLD

Basic support for SLD 1.1 and SE 1.1

Full support for SLD 1.0

Many rendering extensions available!

Integration with GeoWebCache


KML/KMZ

TEMPORAL SERIES

KML EXTRUDE KML SUPEROVERLAY


Advanced Projection Handling

Management of dateline Change and map-wrapping

Cutting un-reprojectable geometries


Raster Symbolizer

65536 Colors

256 Colors

Raw Data


Dynamic symbolizers


Advanced Styling

Advanced Labeling


Advanced Styling

Filter Functions Geometry

Transformations

Drop Shadow Number

Formatting


Advanced Styling

Continuous Maps

CSS Styling

<PolygonSymbolizer> <Fill>  <CssParameter name="fill">#4DFF4D</CssParameter> <CssParameter name="fill-opacity">0.7</CssParameter> </Fill> </PolygonSymbolizer>

WMS - CQL

GeoServer WMS – CQL Examples

STATE_NAME = 'Montana'

MALE > FEMALE

LAND_KM > 300000


Rendering transformations

On-the-fly data transformations

Calling WPS processes from SLD docs

Optimized for performance


Rendering transformations

Point feature extraction from two band raster data (e.g. Wind(u,v))

Computation of direction and module from SLD


ImageIO-Ext Support

JAI ImageIO Extension

Plugins:

BigTiff

netCDF-CF, GriB1

MatFile 5

HDF4

JP2000 (Kakadu)

GDAL 1.8.1

GDAL integration via ImageIO-ext

JPEG2000,MrSID, ECW

BigTIFF, ERDAS Image, HDF4


GeoWebCache Integration

GeoServer

GeoWebCache

Persistent raster/KML tile cache

Direct calls to GeoServer rendering engine

Support for layers modified through WFT-T

Support for various tile protocols

GMap, Gearth

OpenLayers, VEarth, Bing

Speed-up factor 10/100

Disk quota support


Improved Raster Reprojection

Raster reprojection complex process

Idea:

try to approximate the overall transformation with a simpler one, either a single affine transformation or a piecewise composition of them (grid warp)

Ability to specify threshold for error acceptance

Iterative approach (local optimization)

Trade off between speed and precision


http://demo.geo-solutions.it/share/OptimizingRasterReprojection.pdf

http://en.wikipedia.org/wiki/Affine_transformation

http://en.wikipedia.org/wiki/Affine_transformation

SQL Parametric Views

SQL View as a GeoServer Layer

No need to actually creating a view in the database

View can be parametric

WMS and WFS allow parameter substitution

….&request=GetMap&viewparams=low:2000000;high:5000000

Default values for parameters

Validation with regular expressions


Virtual Services

Expose multiple OGC service endpoints per installation

One virtual service per workspace

Expose only layers defined in such workspace

Use cases:

Provide clients with focused/theme centered data sets

Provide each department of a single organization with its own set of services

Limitations:

Single administrator for the whole server

Service metadata (point of contacts and the like) still configured centrally


WMS* Cascading

GetMap and GetFeatureInfo Cascading

Support for reprojection on-the-fly

GUI based mass import of remote layers

TODO: Securing unsecure WMS Services


WPS*

WPS 1.0

Official Extension

Raster and Vector data support

High performance processes development (raster/vector statistics, raster/vector format conversions and more)

Integrated WPS

Automatic publishing of results

Embedding processes into SLD styles

Reuse processes as rendering transformations


WPS*

Deficiencies

No support for asynchronous requests

Missing request limits enforcements (e.g. input/output maximum dimensions)

Wish list:

Scripting (Jython, GeoScript)

Sextante, IDL, JGrass (Grass?) integration

Improved robustness

Jiffle (jai-tools) based raster algebra

Create new layers as dynamic WPS processes (computing data on the fly as requested)


8 Maggio 2012

Complex Feature*

Application/community schemas

Complex Features

Attributes as sub-features

Attributes as list of features

Tree-like structure

Heterogeneous data sources

INSPIRE!


WMS TIME and ELEVATION

TIME = 20100512T0000000Z ELEVATION = 0.0

FeatureType Editor


Raster Pyramids

Preprocessing raster data for performance

Multiple resolutions levels

Scale decide best resolution level

Multiple file (tiles) per resolutions level

File Based Pyramids Support

DBMS-based Pyramids

Postgis

DB2

Oracle (GeoRaster)

MySQL


8 Maggio 2012

GeoNetwork

(GeoSpatial) Catalog

Registry of Metadata

Repository

Enterprise Search Appliance

GeoPortal

SDI entry point

Information Broker

Metadata Editor

Standards Based

Implemented with Java Enterprise technologies


Technologies

Java Enterprise

Apache Lucene for speed light search

Support for different backends

MYSQL, Oracle, Postgresql, etc..

XSL Templates for flexible presentation

XML Schema

Metadata Validation and Editing

Support for Enterprise Security

LDAP

Shibboleth


Standards

ISO TC 211

19110:2005

19115:2003

19115:2005

19119:2005

19139:2007

23950/Z39.50

OGC

CSW 2.0.2 (ISO Profile)

INSPIRE

Discovery Service


Others

FGDC

OpenSearch Geo

RSS/GeoRSS

WebDAV

OAI-PMH

Dublin Core

Catalogue/Registry

Manage, Publish and Search collections of metadata about Data, Services and related information ISO 19115 Metadata

about Data ISO 19119 Metadata

about Services


Enterprise Unique entry point

Act as a Broker (SOA) Can work in federation Multilingualism

Repository

GeoNetwork can be used as a repository for the data it describes

Authorization on data are independent from authorization on related metadata

Easy-to-use editor controls to upload data

and set automatically the onLineResource

URL


Catalogue/Registry - Federation

Multiple Registries for Multiple Enterprise Clouds

Different data/services

Different Organizations

Federation reduces complexity

Available Paradigms

Harvesting

Synchronized Local Copy of remote Metadata

Remote Search

Federating node as NoCache proxy for queries

GeoNetwork support both


Enterprise GeoPortal

User Friendly Search/View GeoPortal

Customizable Presentation

Search Forms

Results Presentation

Based on XSL templates

Embedding in third party sites

Integrate with A&A layers

Different views for different roles

Metadata Profiling


Enterprise GeoPortal

Metadata Profiling

Authentication

Internal user management for smaller deploys

LDAP integration

Shibboleth integration (federated SSO)

Authorization

User groups (grants on metadata)

User profiles (grants on operations)

Metadata access authorization is fine grained at group level


Metadata Editor

Terms and Rules from ISO 19115 and 19119

Encoding in 19139, FGDC and Dublin Core

Intuitive Web-Based Editor with Validation

Identification Section

Distribution Section

Reference System Section

Data Quality Section

Metadata Section

Highly Customizable

Presentation

Validation


8 Maggio 2012

Metadata Editor

Additional capabilities

Extensible Templates’ set

Visual Validation

Extensive online resource management (WMS, File download, etc..)

Additional Metadata Linking (dataset/service, parent/child, feature catalog/dataset metadata)

Role assignment for presentation profiling

Multilingual Support

Batch Import/Insert from XML


INSPIRE


Infrastructure for Spatial Information in the European Community 2007/2/EC Directive, 14/03/2007 Fully in place by 2019

Implementing Rules and Technical Guidance to implement EU-wide Interoperable SDIs IR are legally binding TC are not legally binding

Main Aspects Covered Metadata Network Services Data and Service Sharing Data Specification Monitoring and Reporting

Maximum Reuse of existing standards (OGC, ISO, W3C)

INSPIRE – Network Services


Discovery Services

View Services

Download Services

Transformation Services

Invoke Spatial Data Services

Rights Management Layer

INSPIRE – Discovery Services


Discovery of spatial dataset and services

Provide operations on services

query about service capabilities

register other federated Discovery Services

Provide operations on metadata

search

update (both push and pull modes),

perform federated searches

Based on Metadata specs from ISO

TG requires CSW 2.0.2 ISO AP as starting point

INSPIRE – Download Services


Account for copies of spatial dataset, or parts of them to be downloaded and/ accessed directly

Predefined Dataset (or part) VS Direct access

Presence of Query Capabilities

Different Discovery Metadata

Based on OGC WFS 2.0 and ISO/DIS 19142, 19143 for Direct Access

May allow WFS 1.1 in the future

Support for European CRS is Mandatory (See Annex I of directive)

INSPIRE – Download Services


Support for GML 3.2.1 is Mandatory

SOAP Encoding is suggested

Multilingualism Support is key

Support for the Temporal dimension is Mandatory

Support for Vector Data is rather mature

Support for Raster Data is unclear

WCS 1.1.2? WCS 2.0?

INSPIRE – View Services


Display, Navigate, zoom in/out, pan or overlay viewable dataset and legend information

Based on OGC WMS 1.3.0 – ISO 19128 with OGC SLD and SE

INSPIRE Profile

WMTS 1.0.0 for tile services with INSPIRE extensions

WMS 1.1.1 (INSPIRE PROFILE) is also acceptable

2 Available Scenarios Scenario 1: GetCapabilities document is extended with a link to a

Catalogue

Scenario 2: Extra elements are embedded into the GetCapabilities document itself

INSPIRE – View Services


Support for European CRS is Mandatory (See Annex I of directive)

SOAP support is suggested

Support for the Temporal data dimension

Mandatory output formats for Maps are PNG and GID

Multilingualism Support is key

INSPIRE – InvokeSD Services


Invoking individual spatial services as well as combinations of them

synchronously or asynchronously

Middleware per processing

Technical Papers recommend OGC WPS as basic building block

BPEL is recommended between the others for Service Chaining and Workflow

SOAP should be mandatory

IR and TG not available yet

INSPIRE – Who does What


Discovery service GeoNetwork

View, Download, InvokeSD, Transformation Service GeoServer

INSPIRE Discovery Services GeoNetwork


GeoNetwork implements CSW 2.0.2 ISO Profile

INPIRE extended constraints in metadata should be explicitly enabled in GeoNetwork

Complies with almost all main requirements in Metadata Implementation and Discovery Services

Main compliancy issues in:

Handling federated catalogues in Discovery Service Metadata and Link Discovery Service

operations

Federated metadata retrieving no CSW, only Z39.50

Publishing in pull mode matching CSW Harvest operation in GN is not stable yet

INSPIRE View Services GeoServer


Historically well supported WMS 1.1.1

GeoServer Stable 2.1.x Supports WMS 1.3 Ordnance Survey (UK) funded the work

INSPIRE compliancy still partial Harmonized Names (available since 2.2.0 beta)

SOAP Support missing (recommended, not mandatory)

Scenario 2 not supported

Partial support for multilingualism

INSPIRE Community Extension Separate plugin

Plugs-in additional fields for Multilingualism and Service Metadata

http://docs.geoserver.org/trunk/en/user/community/inspire/index.html



Language support:

Single language declaration support

Limitations

No multilingual metadata on layers

Missing localized support for exceptions

Missing localization support for contents (e.g., GetFeatureInfo, GetMap labels)

Robust support for Mandatory CRS (ETRS89)

Support for Mandatory Encoding Formats (PNG, GIF)

Support for SLD 1.0 solid and mature

Many Additional Vendor Options



Support for SE 1.1 present

extra operations available in SE 1.1 and not in SLD 1.0 are not supported

Some of the above are available as SLD 1.0 vendor extensions (same functionality, different syntax)

INSPIRE View Services GeoWebCache


Fully support for WMTS 1.0 (tiled view services)

Harmonized names possible

INSPIRECRS84QUAD tile matrix set not built-in, but manually configurable

Missing INSPIRE extensions:

WMTS Capabilities document does not contain the required INSPIRE extensions (WMS-C do though)

No layer metadata links

Internationalization support missing

No SOAP support

INSPIRE Download Services GeoServer


GeoServer 2.2.0 beta first release to support WFS 2.0 and GML 3.2.1 IGN France Funding

SOAP supported

Missing some functionalities

Standard Capabilites Extension missing

Local and Remote Resolve

GetPropertyValue Interactions with AppSchema/Complex Features

Basic Workflow for Complex features Store original datasets in ad hoc schema database

Generating Object-Relational mappings

On-the-fly output transformations

INSPIRE Coordinate Transformation Services - GeoServer


GeoServer provides coordinate transformation tools with the gs:Reproject WPS process

Requires some changes to become compliant

Name change

List supported SRS

Use different mime types for GML

Add “test transformation” mode (does not actually transform, checks only if possible)

INSPIRE InvokeSD Services GeoServer


GeoServer Support OGC WPS 1.0.0

Interaction with external WFS and WCS

Automatic Ingestion of produced data

Basic process chaining is supported

Interaction with BPEL/BPMN engine to be tested

The End

Questions? [email protected]


mailto:[email protected]




GSIP 57 extended authorization management

GeoRepository

Role Base Access Control

Services

Operations

Layers

Attributes (alphanumeric and geospatial)

External Web Application

Web Services + UI

Rule-based

IPTables-like


http://geoserver.org/display/GEOS/GSIP+57+-+Improving+GeoServer+authorization+framework

http://geo-solutions.blogspot.com/2011/05/preview-georepository-advanced.html

http://demo.geo-solutions.it/share/securing_geoserver.pdf


GeoRepository


RESTful Configuration

GeoServer

Javascript client

Desktop app

GeoBatch

Server side scripts

Styles

Layers Publishing

Automatic Configuration of recurring flows of data

Mass publishing of Layers


WPS*: demo builder


Image Server*

Turning GeoServer into an Image Server

Serving pure Imagery

No geo-reference need/available/(would make sense!)

Special Coordinate Reference Systems defined

Interoperability with WMS clients

Respecting EPSG conventions

EPSG:404000

See here

Improved support for data with

bad/missing geo-reference!


http://geo-solutions.blogspot.com/2011/05/developers-corner-turning-geoserver.html

Aggregating data store

N layers, remote or local, sharing the same structure

Aggregating store puts them together dynamically, the client wil think there is just one layer

Parallel data fetching

Can be configured to tolerates temporarily unreachable data sources


Cross-Layer filtering

“querylayer” community module

“Find all sites within 100 meters from roads”

Single CQL filter with WMS:

DWITHIN( the_geom, collectGeometries( queryCollection('roads', 'the_geom', 'INCLUDE') ), 200, meters)


256 Colors

Improved Clustering

New parameters to allow multiple GeoServers to work against the same network shared data directory

Per server log file location

Disabling GWC meta store

Disabling GWC disk quota management

GeoWebCache Enhancements

disable the embedded GWC completely,

hide the embedded GWC to the public

keep the embedded GWC only for fast layer seeding

only one GWC at a time is allowed to write on disk


WMS Animator

Animations for WMS requests across custom GetMap params

Controlled via aparam/avalues couples

Generates multiple frames through multiple GetMap requests

http://localhost:8080/geoserver/wms/animate?LAYERS=topp%3Aworld&aparam=time&avalues=2004-01-01T00:00:00.000Z,,2004-07-01T00:00:00.000Z&format=image/gif;subtype=animated&format_options=gif_loop_continuosly:true


http://localhost:8080/geoserver/wms/animate?LAYERS=topp:world&aparam=time&avalues=2004-01-01T00:00:00.000Z,,2004-07-01T00:00:00.000Z&format=image/gif;subtype=animated&format_options=gif_loop_continuosly:true














http://demo.geo-solutions.it/share/bluemarble-animate.gif

http://demo.geo-solutions.it/share/tuna-animated.gif

GeoServer, GeoNetwork and INSPIRE: where we are and what is missing

Technology

Transcript of GeoServer, GeoNetwork and INSPIRE: where we are and what is missing