Data Integration on the Semantic Sensor Web

Alasdair J G GrayInformation Management Group

University of Manchester

Seminar at Imperial College London7 April 2011

Data integration on the SemSorWeb – Alasdair J G Gray 2

Overview of the Talk• Motivation: Estuarine Flooding

• Semantic Sensor Web – SemSorWeb

– Requirements

– Architecture

• Ontology-based query access over sensor, streaming, and stored sources

• Demo7 April 2011

Data integration on the SemSorWeb – Alasdair J G Gray 3

The Solent

7 April 2011

Image: http://www.chandlery-online.com/

Image: http://www.combinedops.com/

• Strait separating the Isle of Wight from English mainland

• Complex tidal and wave patterns

Data integration on the SemSorWeb – Alasdair J G Gray 4

The Solent

7 April 2011

Images: http://www.wikipedia.com/

Data integration on the SemSorWeb – Alasdair J G Gray 5

Estuarine Flood Warning• Financial implications

– Damage– Loss of business

• Personal factors– Emotional

• Flood prediction– Early warning/forecasting– Locations– Severity

• Requires correlating– Sea-state data– Weather forecasts– Details of sea defences

• Response Planning– Evacuation routes– Personnel deployment– …

• Requires more data– Traffic reports– Shipping– …

7 April 2011

Image: http://www.metro.co.uk/

Data integration on the SemSorWeb – Alasdair J G Gray 6

Flood Risk Alert: English South Coast

• Detect conditions likely to cause a flood

“Provide details of overtopping events in the Solent region with high wind speed observations” • Need to correlate data from

multiple sources• Sensor data• Web feeds• Databases and other

sources• Present data model in terms of

the user domain: e.g. Flood risk assessment

...

...

...

Flood defences data (database)

Other sources

Meteorological forecasts

7 April 2011

Real-time sensor data

Wave,Wind,Tide

Data integration on the SemSorWeb – Alasdair J G Gray 7

1. Accurate characterisation of conditions that define an event

2. Correlation of data of differing modalities3. Integrating data from heterogeneous data

models4. Discovery of relevant data sources5. Presentation and control of information

7 April 2011

Sensor Web Requirements“Provide flood risk details of overtopping events in the Solent region with high wind speed observations”

“Provide flood risk details of overtopping events in the Solent region with high wind speed observations”

“Provide flood risk details of overtopping events in the Solent region with high wind speed observations”

“Provide flood risk details of overtopping events in the Solent region with high wind speed observations”

“Provide flood risk details of overtopping events in the Solent region with high wind speed observations”

“Provide flood risk details of overtopping events in the Solent region with high wind speed observations”

Data integration on the SemSorWeb – Alasdair J G Gray 8

Ontology Network: Reconcile Terminology

SSN

SWEET

Service

Coastal Defences

Ordnance Survey

Additional Regions

Role

DOLCE UltraLite

Schema

FOAF

Upper

External

SSG4Env infrastructure

Flood domain

7 April 2011

9

Application Tier

Middleware Tier

Data Tier

ApplicationServices

Semantic Registry

Semantic Integrator

Data Source

Connectivity Bridge

Applications

Concrete Resource

SemSorWeb Architecture

7 April 2011

Data integration on the SemSorWeb – Alasdair J G Gray 10

DATA SOURCES

7 April 2011

Data integration on the SemSorWeb – Alasdair J G Gray 11

Channel Coastal Observatory: Observing Sea-State

43 sensors deployed around UK coast• Measuring

– Tides (7)– Waves (24)– Weather conditions (12)

• On and off shore• Bespoke hardware• Fixed functionality• Fixed (low) data rate

– No bursts• Central data centre

7 April 2011

Data integration on the SemSorWeb – Alasdair J G Gray 12

Publishing CCO Data• Relational data streams

– envdata_sandownpier_tide (ts:int, Observed:float, Tz:float, Hs:float, HMax:float, Tp:float)

• Streaming data service• Semantically annotated metadata

description– Service description– Spatiotemporal data coverage– Thematic data coverage

• Tide height7 April 2011

Data integration on the SemSorWeb – Alasdair J G Gray 13

Environment Agency: Flood Defences Database (NFCDD)

• Flood defence details– Type of defence

• Wall• Sand banks• …

– Location– Height– …

7 April 2011

Data integration on the SemSorWeb – Alasdair J G Gray 14

Publishing NFCDD Data• Relational database

– locations (id:int, latitude:decimal, longitude:decimal,location:string,storm_threshold:decimal)

• Stored data service• Semantically annotated metadata

description– Service description– Spatiotemporal data coverage– Thematic data coverage

• Sea defences7 April 2011

Data integration on the SemSorWeb – Alasdair J G Gray 15

INTEGRATING DATA

7 April 2011

Data integration on the SemSorWeb – Alasdair J G Gray 16

Abstract Problem

Stored dataSensor

Network

Integrator

7 April 2011

Sensor Network

Stored data service

Streaming data service

Streaming data service

Data integration on the SemSorWeb – Alasdair J G Gray 17

Data sourceData

streamQuery

capabilitiesData

access

Types of Heterogeneity

Stored dataSensor

Network

Integrator

7 April 2011

Sensor Network

Stored data service

Streaming data service

Streaming data service

Data semantics

Data integration on the SemSorWeb – Alasdair J G Gray

Querying Approach• Use ontologies as common model

Requires:– Representation of RDF stream– Expressing continuous queries over an

RDF stream– Establishing mappings between

ontology models and data source schemas

– Accessing data sources through queries over ontology model

7 April 2011 18

Data integration on the SemSorWeb – Alasdair J G Gray 19

RDF Stream• Named graph• Continuously updating• Triples annotated with timestamp

7 April 2011

STREAM http://www.semsorgrid4env.eu/ccometeo.srdf......( <ssg4e:Obs1, rdf:type, cd:Observation>, ti ),( <ssg4e:Obs1, cd:observationResult, “34.5”>, ti ),( <ssg4e:Obs2, rdf:type, cd:Observation>, ti+1 ),( <ssg4e:Obs2, cd:observationResult,”20.3”>, ti+1 ),......

cd:Observation

xsd:double

cd:observationResult

Data integration on the SemSorWeb – Alasdair J G Gray 20

SPARQLStream

PREFIX cd: <http://www.semsorgrid4env.eu/ontologies/CoastalDefences.owl#>PREFIX sb: <http://www.w3.org/2009/SSN-XG/Ontologies/SensorBasis.owl#> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> RSTREAMSELECT ?windspeed ?windts FROM STREAM <http://www.semsorgrid4env.eu/ccometeo.srdf> [ NOW – 1 MINUTE TO NOW STEP 5 MINUTES ] WHERE { ?WindObs a cd:Observation; cd:observationResult ?windspeed; cd:observationResultTime ?windts; cd:observedProperty ?windProperty; cd:featureOfInterest ?windFeature. ?windFeature a cd:Feature; cd:locatedInRegion cd:SolentCCO. ?windProperty a cd:WindSpeed. }

7 April 2011

cd:Observation

xsd:double

cd:observationResult

cd:Feature

cd:featureOfInterest

cd:Property

cd:observedProperty

cd:Region

cd:locatedInRegion

“Every 5 minutes give me with the wind speed observations over the last minute in the Solent Region ”

Data integration on the SemSorWeb – Alasdair J G Gray 21

Semantic Integrator

Semantic Integrator

7 April 2011

Streaming Source

Stored data

Stored data

Streaming Source

Tuples

Tuples

Tuples

Tuples

Q

SPARQLStream query over ontology

Data Translator

[[Q]]

SPARQL bound variables

Query Translator

q

S2O Mappings

SNEEql query over source

schema

SNEE Query Evaluation

Engine

[[q]]

Tuples

Data integration on the SemSorWeb – Alasdair J G Gray 22

Query TranslationRSTREAMSELECT ?xFROM STREAM <http://…> [NOW–1 MIN TO NOW STEP 5 MIN] WHERE { ?y a cd:Observation; cd:observationResult ?x.}

• Conjunctive queries

• Extended with windows

• Extended with window-to-stream

7 April 2011

Query Execution

7 April 2011 Data integration on the SemSorWeb – Alasdair J G Gray 23

PREFIX cd: <http://www.semsorgrid4env.eu/ontologies/CoastalDefences.owl#>PREFIX sb: <http://www.w3.org/2009/SSN-XG/Ontologies/SensorBasis.owl#> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> SELECT ?waveheight ?wavets ?lat ?lon FROM STREAM <http://www.semsorgrid4env/ccometeo.srdf> WHERE { ?WaveObs a cd:Observation; cd:observationResult ?waveheight; cd:observationResultTime ?wavets; cd:observationResultLatitude ?lat; cd:observationResultLongitude ?lon; cd:observedProperty ?waveProperty; cd:featureOfInterest ?waveFeature. ?waveFeature a cd:Feature; cd:locatedInRegion cd:SouthEastEnglandCCO. ?waveProperty a cd:WaveHeight. }

Ontologies Streamsvvvvenvdata_rhylflats

Timestamp: longHs : floatLon: floatLat: float

envdata_hornsea

envdata_milford

envdata_chesil

envdata_westbay

S2O Mapping

Observation

WaveHeightProperty

observedPropert

y

hasObservationResult

xsd:float

locatedInRegion

Region

Feature

SPARQLStream

(SELECT Lon,timestamp,Hs,Lat FROM envdata_rhylflats) UNION (SELECT Lon,timestamp,Hs,Lat FROM envdata_hornsea) UNION (SELECT Lon,timestamp,Hs,Lat FROM envdata_milford) UNION (SELECT Lon,timestamp,Hs,Lat FROM envdata_chesil) UNION (SELECT Lon,timestamp,Hs,Lat FROM envdata_perranporth) UNION (SELECT Lon,timestamp,Hs,Lat FROM envdata_westbay) UNION (SELECT Lon,timestamp,Hs,Lat FROM envdata_pevenseybay)

SNEEql

Data Translation

7 April 2011 Data integration on the SemSorWeb – Alasdair J G Gray 24

<ns9:sparql> <ns9:head> <ns9:variable name="waveheight"/> <ns9:variable name="wavets"/> </ns9:head> <ns9:results> <ns9:result> <ns9:binding name="waveheight"> <ns9:literal datatype=xs:double>4.850</ns9:literal> </ns9:binding> <ns9:binding name="wavets"> <ns9:literal datatype=xs:long>1272588663</ns9:literal> </ns9:binding> </ns9:result> <ns9:result> <ns9:binding name="waveheight"> <ns9:literal datatype=xs:double>2.1230</ns9:literal> </ns9:binding> <ns9:binding name="wavets"> <ns9:literal datatype=xs:long>1272587400</ns9:literal> </ns9:binding> </ns9:result> </ns9:results></ns9:sparql>

waveheight wavets

4.850 1272588663

2.1230 1272587400

Data integration on the SemSorWeb – Alasdair J G Gray 25

FLOOD APPLICATION

“Smashing it all together”

7 April 2011

Data integration on the SemSorWeb – Alasdair J G Gray 26

Flood Web Application• Discover relevant sources

– Role– Task– Location

• Mashup data– Layers on a map

• Limited execution environment– Application services provide REST

access to Web services7 April 2011

Data integration on the SemSorWeb – Alasdair J G Gray 27

Relevant Data Sources

Sensor/Streaming Data

• Sea-state sensor data– CCO– Port authority– WaveNet

• Shipping– AIS

• Traffic Alerts– RSS feed

Contextual Data• Base Maps• Flood defense details• Land usage• Weather forecasts• Flooding forecasts

7 April 2011

Data integration on the SemSorWeb – Alasdair J G Gray 28

DEMO

http://www.semsorgrid4env.eu/services/dynamic-demo

7 April 2011

Data integration on the SemSorWeb – Alasdair J G Gray 33

Conclusions: SemSorWeb1. Accurate characterisation of conditions that

define an event– Declarative queries

2. Correlation of data of differing modalities– Query evaluation over streaming and stored

3. Integrating data from heterogeneous data models

– Ontology-based access to streaming data

4. Discovery of relevant data sources– Semantic registry (not covered in this talk)

5. Presentation and control of information– Application service to support, e.g. smash-ups

7 April 2011

Data integration on the SemSorWeb – Alasdair J G Gray 34

Conclusions• Query-based access to RDF streams

– SPARQLStream • Continuous query language

– S2O mappings• Relational streams to ontology• Global-as-View approach

– Query translation to SNEEql • Query substition

– Query execution by SNEE-DQP

7 April 2011

Data integration on the SemSorWeb – Alasdair J G Gray 35

Acknowledgements

• Christian Brenninkmeijer• Alvaro Fernandes• Ixent Galpin• Norman Paton

• Jean-Paul Calbimonte• Oscar Corcho

7 April 2011

For more information: http://www.semsorgrid4env.eu/

Data integration on the SemSorWeb – Alasdair J G Gray 36

References1. I. Galpin, C. Y. A. Brenninkmeijer, A. J. G. Gray, F. Jabeen,

A. A. A. Fernandes, and N. W. Paton. SNEE: A query processor for wireless sensor networks. Distributed and Parallel Databases, 29(1-2):31–85, 2011. Special issue on Query Processing in Sensor Networks.

2. A. J. G. Gray, R. García-Castro, K. Kyzirakos, M. Karpathiotakis, J.-P. Calbimonte, K. Page, J. Sadler, A. Frazer, I. Galpin, A. A. A. Fernandes, N. W. Paton, O. Corcho, M. Koubarakis, D. De Roure, K. Martinez, and A. Gómez-Pérez. A semantically enabled service architecture for mashups over streaming and stored data. In European Semantic Web Conference (ESWC 2011), pages 300–314, 2011. To appear.

3. J.-P. Calbimonte, Ó. Corcho, and A. J. G. Gray. Enabling ontology-based access to streaming data sources. In Proceedings of 9th International Semantic Web Conference (ISWC 2010), volume 6496 of LNCS, pages 96–111, Shanghai, China, November 2010. Springer. (Alphabetical authorship, equal responsibility).

7 April 2011