istSOS: an extended Sensor Observation Service implementation for Environmental Data

istSOS: an extended Sensor Observation Service implementation for Environmental Data

2015 ICGIS - INTERNATIONAL CONFERENCE ON GEOSPATIAL INFORMATION SCIENCE – SEOUL, 16.09.2015


Massimiliano Cannata, Milan P. Antonovic, Mirko Cardoso

2015 ICGIS International Conference On Geospatial Information Science


Millenium developement goals [2015 report]

2


Millenium developement goals [2015 report]

3

istSOS: an extended Sensor Observation Service implementation for Environmental Data 4 «Climate change and

environmental degradation

undermine progress achieved,

and poor people suffer the most»


altered ecosystems

weather extremes

risk to society

urgent, critical challenge for the

global community

Overexploitation of marine fish stocks

Water scarcity and floods

Poor people live in most vulnerabe areas


Solving societal challeneges

Strengthening data production and the use of better data in policymaking and monitoring are becoming increasingly recognized as fundamental means for development

6


Data key aspects for wise decisions

7

Quality

Availability

Timeless

Completness

Understand the situation and timely react


Understanding fusing multiple data: sources and types

8

morphology Real time data

Satellite obserations

Field obserations

Socioeconomic maps

Properties and population

registers

Dynamic maps

istSOS: an extended Sensor Observation Service implementation for Environmental Data 9


Analysis

Data Understanding Action

Open architecture Based on the principles of a Service Oriented Architecture (SOA) and the specification of Open Geospatial Consortium Sensor Web Enablement (OGC - SWE) initiative

10

WEB FEATURE SERVICE

WEB COVERAGE

SERVICE

SENSOR OBSERVATION

SERVICE

WEB PROCESSING

SERVICE

WEB MAPPING SERVICE & DSS

WEB NOTIFICATION

SERVICE

POLICY MAKING

INTERVENTION


MIARIA

11

MIARIA – Adaptive Hydrogeological Monitoring in support of the Alpine Iintegrated Risk plan

Dynamically identify impacts of impending scenarios


Conceptual schema

12

Sensor

Gateway / Router

SCENARIO IDENTIFICATION &

EXPOSED ELEMENTS

EVACUATION & PREPARATION


Example: dynamic and integrated prediction

13

Industries

Highway


Industries

Highway




15

Industries

Highway


ENORASIS

The project

ENORASIS is a FP7-ENV project (Grant Agreement 282.949). The project duration is three years (January 2012 – December 2014).

Objective

Develop an integrated decision support system for environmentally optimized and thus sustainable irrigation management for farmers and water management organizations based on advanced technologies and models.

16 DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation


The idea

• New EU policies in the field of water and agricultural policies introduce the principle of “the user pays" in irrigation water management

– WFD (Water Framework Directive)

– CSS (Cross Compliance Scheme) of the EU CAP (Common Agricultural Policy).

• New tools and technologies allow the usages of granular information on the present and future state of the system:

– High resolution weather prediction models

– Low cost sensors

• New devices and spread of the Internet allow easy access to information for decision-making more conscious

17 DACD / IST / ISTSOS in support of ENORASIS for optimizing irrigation


ENORASIS System Architecture

18


Sensor data Subsystem: Data Fusion

19

FIELD 1

FIELD N

FIELD 2

Sensor

Gateway / Router

Coordinator / Seller

ENORASIS SYSTEM


20

hydro-meteorological monitoring

network of Canton Ticino, Switzerland

Management of the


Lake Verbano shores are exposed

to flooding events that cause

extensive damages Picture by Julien Daulte


Large basins (6’386 km2) compared to limited lake area (213 km2)

Intense reainfall events compared to maximum discharge (2,000 m3/s)

Almost flat area highly urbanized


Demo

• But HOW did we achieve this

How to

Picture from David Oesch

Building an Early Warning and

Descision Support System

the risk

manage

0 2 0 0


EW-DSSS components and workflow

Meteorological Forecasts

Hydrological modelling

Data Validation

Data Sharing

Early Warning & DSS:

SITGAP 2.0

Hydro-meteo monitoring

network Sensor Data

GIS Data


Real time data collection

In 2010, switch from analogue modem (collection every 4 hours) to GPRS sensor data transmission (real time)

Needs for smarter system for collecting and dispatch measurements


Looking for a solution...

26

Simple

Python

Open Standard


The Sensor Observation Service standard

27

From http://www.opengeospatial.org/standards/sos “The Sensor Observation Service standard is applicable to use cases in which sensor data needs to be managed in an interoperable way. This standard defines a Web service interface which allows querying observations, sensor metadata, as well as representations of observed features. Further, this standard defines means to register new sensors and to remove existing ones. Also, it defines operations to insert new sensor observations. This standard defines this functionality in a binding independent way; two bindings are specified in this document: a KVP binding and a SOAP binding (in version 2.0).”

http://www.opengeospatial.org/standards/sos


SOS User types

28


Suported system types (up-to-now)

in-situ - fixed - point

in-situ - mobile - points

29


Supported version (up-to-now)

SOS v1.0.0

30


The istSOS software

31

mod_wsgi

isodate psycopg2 pytz


Some of the special features...

32


Easy installation and initialization and management using GUI (WebAdmin Interface)

33


Output formats from GetObservation request

34

XML – CSV - JSON


Aggregation on-the-fly with GetObservation request

35

AVG, SUM, MAX, MIN

Extra parameters: aggregateInterval = PT1H & aggregateFunction = AVG


Real Time Quality Check

Configuration of quality index on the observed property level or sensor specific.

Automatically

data quality check

assigning quality

index on each

observation based

on specific constraints

36


Raw data

Assignment of quality indices during insertion:

Quality index example

37

Raw data

Meaningful data (Leve 1)

Statistically sound (Level 1 + 2)


Time-Zone support

eventTime 2013-01-01T01:00:00+01 / 2013-01-07T01:00:00+01

eventTime 2013-01-01T02:00:00+02 / 2013-01-07T02:00:00+02

38


Virtual procedures (EVAPOTRANSPIRATION)

39

GRABOW

Real Sensor

Air Temperature

Relative humidity

Wind velocity

Solar radiation

V_GRABOW

Virtual procedure

Reference

evapotranspiration FAO56 Calculation


Status Page (New)

40

Sensor status at a glance taking in account acquisition delay and inserting errors

AKNOWLEDGED

BY

ADMINISTATOR

NO ERRORS OR DELAY

PENDING ERRORS OR DELAY


Acquisition Job Scheduler (New)

With Advanced Python Scheduler (APScheduler, embedded in istSOS) you can schedule sensor data to be inserted at times of your choosing

41

Sensor File Transfer of Raw Data Server

Scheduler Data acquisition istSOS


istSOS Web Notification Service (New)

42


Incubating in OSGeo for quality software

43

Copyright and License

Documentation

Release Procedure

Processes

Active and healthy community


Documentation – Users & Devel (sphinx)

44


Release Procedure DEB package creation

45


EWS components with open standard and open software


Mode «Dams»: access to

dam-break hazard zones

and wave arrival times

Mode «Alarm»: access to

forecasted lake levels and

exposed elements data (detect exposed elements with a

given lake level, see plots of

forecasted lake level, access

instructions to secure exposed

elements, etc.)

Mode «Evacuation»:

access to datails of

population to manage

evacuation (create evacauation event, search

and locate person, count people in

an area, assign people an

evacauatio status, etc.)

Mode «Regular»: access to

data and funtions for

resources management (search for parcel, address or

person, query and eventually edit

elements details, etc.)

47

0 2 1 4


istSOS is one enabler of the IOT?

48

The Internet of Things (IoT) is the collection of billions of end devices intelligently connected and interoperating with servers and services.

Combine “Things” like Lego blocks to create applications, services, values….

How to chose the blocks? what are they? how do they fits together? We need well known language: standards for machine – machine interaction!

SOS is an OGC standard to:

• register new sensor (THING) to the Web,

• storing new observations

• retrieving desired data


Next steps for development

• istSOS meet pgpool to add reliability and scalability

• istSOS already implements a sort of SOS-P (P=processing) trough VirtualProcedures. Now we will add several time-series processing power collaborating with freewat H2020.

49

istSOS Web Notification Service

50

http://istsos.org

istSOS: an extended Sensor Observation Service implementation for Environmental Data

Software

Transcript of istSOS: an extended Sensor Observation Service implementation for Environmental Data