Open Source Tools for Uncertainty Enabling

the Model Web

Benjamin ProßUniversity of Münster

FOSS4G, Denver, 12-16 Sept 2011

• UncertWeb

• Data Encodings/Profiles

• Web Services

• Other Tools

Overview

UncertWeb Facts

• Uncertainty-enabled Model Web, FP7 Project (http://www.uncertweb.org/)

• Feb 2010 – Jan 2013• 8 Partners

• Aston University• Italian National Research Council• Food and Environment Research Agency• Joint Research Centre• Norwegian Institute for Air Research• Eindhoven University of Technology• University of Muenster• Wageningen University

The UncertWeb concept

• The “model web”.

• When chaining services of limited or unknown quality, uncertainty must be accounted for if rational decisions are to be made.

ECMWFensembles

GEMSensembles

Emissions data

Air quality forecastmodel

Modelparameters

Uncertaintyenabled air quality

forecast

Air qualitysensor network

Short range airquality forecasts

Meteorologicalsensor network

ECMWF model

GEMS models

Near real-time airquality nowcasts

Land use data

Albatross individualactivity simulator

Historical activitydata

Traffic model

Individualexposure

assessment

Emissions model

GISinfrastructure

data

Demographicdata

“UncertWeb develops mechanisms, standards, tools and test-beds for accountable uncertainty propagation in web service chains.”

Definitions

• Uncertainty– Model inputs

• E.g. through measurement errors of sensors • Taking outputs of other models as inputs

– Models itself• By aproximating reality, models introduce errors,

and should inform the user about (annotate its output with) the degree this happens

• Biodiversity and climate change

• Land-use response to climatic and economic change

• Short term uncertainty-enabled forecasts for local air quality

• Individual activity in the environment

Application Scenarios

Data encodings

• UncertML is a dictionary and encoding for uncertain information.– JSON and XML encoding– Provides support for distributions, statistics

and realisations.– Aim to cover a very wide range of uses: SWE,

SBML, Semantic Web

Uncertainty Model Language (UncertML)

Profiles

• Profiles:• specification or standard consisting of a set of

references to one or more base standards and/or other profiles, and the identification of any chosen conformance test classes, conforming subsets, options and parameters of those base standards, or profiles necessary to accomplish a particular function

• In other words:– Restricts a base standard to suite a certain application

purpose

Why Profiling?

• Vector Data:– GML Profile: Restriction of geometry types– ISO 19139 extension for uncertainty– O&M Profile: Spatio-temporal and result

restrictions

• Raster Data:– NetCDF-U: extension of NetCDF for

uncertainty values

UncertWeb Profiles

• Goal: provide easy-to-use and simplified profiles

• GML Profile used to encode vector-based geometries and features:

• Restrictions on Geometries• Restrictions on Metadata

• O&M Profile:• Uses GML Profile• Restrictions on result types• Common ways to encode uncertainties

UW Profiles – Vector Data

• Spatial Geometries– Point– LineString– Polygon– Grid

• Multigeometries– Collection for each of the geometries defined above (e.g.

Multipoint, MultiLineString, etc.)

• Temporal Geometries:– TimeInstant, TimePeriod

• Uncertainty Property Type

GML Profile

ISO 19139 Extension

• Extends DQ_QuantitativeResult with DQ_UncertaintyResult

• Used for resultQuality in O&M

O&M Profile - restrictions

• Spatial restrictions• to SamplingFeature as defined in O&M

• Geometries of SamplingFeatures as defined in GML Profile

• Temporal Restrictions• TimeInstant and TimePeriod according to GML

profile

• resultQuality has to be ISO 19139 data quality

• Includes extension with Uncertainty result

O&M Profile – Observation Subtypes

• Measurement:• result is double with uom info

• BooleanObservation• DiscreteNumericObservation

• Result is integer• TextObservation

• Result is String• UncertaintyObservation

• Result is UncertML AbstractUncertainty• ReferenceObservation

• Result is reference to e.g. file on a server

O&M Profile - Collections

• Collection for each observation subtype– MeasurementCollection– BooleanObservationCollection– …

• Network Common Data Format

• support the creation, access, and sharing of array-oriented scientific data

• Structure: Header + Body:– Header defines variables– Body contains binary-encoded variable values

NetCDF-U

NetCDF-U Header Example

URLs to UncertMLDictionary

• Goal: provide easy-to-use lightweight Java API for Information Models

• Is used in Web Service implementations for encoding/decoding of inputs and outputs

UncertWeb Java APIs

Web Services

Web Services and Models

• Models not developed with automation in mind.– Can require a user interface.

• Wrappers enable them to be called with code (e.g. Java) in the service interface.– Write input files, run, parse output files into usable objects.– Mechanisms for running each model may vary substantially.

Service interface overview

• Open Geospatial Consortium (OGC):– WPS for processing (i.e. models).

• Including utility services for translating between uncertainty types.

– WCS, WFS, WMS, SOS for data access.

• W3C Web Services (WS):– SOAP for information exchange.– WSDL for service description.

• UncertWeb Proxy Service (UPS)– Proxy for (not uncertainty-enabled) model-WPS– Executes Monte Carlo simulations

• Uncertainty Transformation Service (UTS)– Transforms Uncertainties– E.g. distributions to realisations

• Spatio Temporal Aggregation Service (STAS)– Transforms different data into a common scale

Use of WPS

Uncertainty Transformation Service

• Transforms Uncertainties– E.g. distributions to realisations

• Can handle:– UncertML (UncertML parser/encoder)– NetCDF-U (UncertWeb binary

parser/encoder)– O&M

• R-backend

UncertWeb Proxy Service

• MonteCarloSimulation process• Inputs:

– Identifier of the process to be executed by the model-WPS– Uncertain inputs for the process– Static inputs for the process– URL of the model-WPS– Output uncertainty type (e.g. distribution or realisations)– Number of realisations (i.e. monte carlo runs)

• Outputs:– The specified uncertainty type

UPS

Monte Carlo Service (UPS)

Monte Carlo Service (UPS)

Samples Realisations

Model Service (WPS)

Model Service (WPS)

Model Implementation

Model Implementation

1 Sample 1 Realisation

1 Sample 1 Realisation

LOOP

UPS + UTS

Monte Carlo Service (UPS)

Monte Carlo Service (UPS)

PDF PDF

Model Service (WPS)

Model Service (WPS)

Model Implementation

Model Implementation

1 Sample 1 Realisation

1 Sample 1 Realisation

LOOP

Uncertainty Transformatio

n(UTS)

Uncertainty Transformatio

n(UTS)

Realisations

PDFUncertainty Transformatio

n(UTS)

Uncertainty Transformatio

n(UTS)

PDF

Samples

Local Air Quality Model Chain

• Demo chain with UncertWeb components

Uncertainty enabled Austal

model(UPS + WPS +

UTS)

Uncertainty enabled Austal

model(UPS + WPS +

UTS)

UncertML

JSON

O&M

Web-basedVisualisation

client

Web-basedVisualisation

client

INTAMAP service (WPS)

INTAMAP service (WPS)

Air quality observations

(SOS)

Air quality observations

(SOS)

UncertML+ GML

Overlay Service (WPS +

UTS)

Overlay Service (WPS +

UTS)

UncertML

realisations

Interpolation of background concentration

Estimation of air pollutionfrom local emissions at point locations

Adding both outputs to final concentration map

USOS/UWCS

• Can handle:– UncertWeb O&M profile (USOS)– NetCDF-U (UWCS)

SOAP/WSDL

• SOAP protocol.– Use in combination with information models

(GML, O&M).– Standard fault messages.

• WSDL service description.– Available operations.– Required structure of messages, both request

and response.– Where to find the service.

SOAP/WSDL

• Advantages:– Widely adopted by the rest of the web.– Tool support for generating code.– Compatibility with workflow software

(Taverna, Kepler) and orchestration engines.

• Disadvantages:– Lack of semantics.– Not fully adopted by the OGC.

JSON

• A lightweight format for exchanging data.

• Preferred over XML for JavaScript client development.

• Used by Google, Facebook, Twitter...

Applying the technology

• Web service supporting two interfaces: SOAP/WSDL and JSON.

• Web service interface sits separately from model.• Wrappers need to be developed so the interface

can communicate with the model.

Other tools

• Composition as a Service

• Offers functionality to– Create uncertainty enabled model chains– Execute them

CaaS

Visualization tool

• Different visualizations of uncertainties in spatio-temporal data

Elicitator

• Expert elicitation of uncertainties

• E.g. parameters of distribution functions– Mean and standard deviation of a Gaussian

normal distibution

Sensitivity Analysis tool

• Performs sensitivity analysis– Identify the influence of the different inputs to

the model output

Links

• http://www.uncertweb.org/

• http://uncertml.org/

• https://svn.52north.org/svn/geostatistics/main/uncertweb

Thank you

Questions?

The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° [248488].