On a web of data streams

On a web of data streamshttp://w3id.org/wesp

Daniele Dell’Aglio, Danh Le Phuoc, Anh Lê Tuán,

Muhammad Intizar Ali and Jean-Paul Calbimonte

[email protected] http://dellaglio.org @dandellaglio

Vienna, Austria, 22.10.2017

http://w3id.org/wesp

http://dellaglio.org/

RDF Stream Processing

StreamProcessing

RDF

&

SPARQL

RDF Stream

Processing (RSP)

Real-time

processing of

highly dynamic

data

Semantic Web

technologies for

data exchange

through the Web

DeSemWeb, 22.10.2017On a web of data streams2/16

Finding agreements


Many topics

RDF streams

Stream reasoning

Complex event processing

Stream query processing

Internet/web of things

Many studies

Data models

Query models

Prototypes

Benchmarks

Datasets

W3C RSP community group (2013 – 2016)

Effort to (discuss | formalise | standardise | combine | evangelise) the

existing studies on RSP

Outcomes

Abstract model for RDF streams

Requirements document for query languages of RDF streams

More at: https://www.w3.org/community/rsp/

https://www.w3.org/community/rsp/

But...


W3C RSP sets some foundations and requirements, but:

Standard protocols and exchanging mechanisms for RDF

stream are still missing

We need generic and flexible solutions for making RDF

streams available and exchangeable on the Web.

The goal: a decentralized web of RSPs

MorphStreams

CSPARQL

TrOWL StreamRule

CQELS

On a web of data streams DeSemWeb, 22.10.20175/16

What is a suitable data exchange infrastructure

to perform stream processing on the Web?

CSPARQL

How far are we?


Documents from RSP

Abstract model of RDF Stream

Requirements for query languages for RDF Stream

Protocols to exchange data streams on the web and

internet

WebSocket, MQTT

Description of the stream

SSN

Interfaces to control RSP engines

Requirements


A framework for RDF stream exchange should

1. prioritize active paradigms for data stream exchange

2. enable the combination of streaming and stored data

3. enable the possibility to build reliable, distributed and

scalable streaming applications

4. guarantee a wide range of operations over the streams

5. support the publication of information about the stream

6. support the exchange of a wide variety of streams

7. exploit as much as possible existing protocols and

standards

WeSP


A framework to publish and exchange RDF streams on

the Web

A model to serialise RDF streams

A model to describe RDF streams

A communication protocol

A model to serialise RDF streams


An RDF stream can be represented as an (infinite) ordered sequence of time-annotated data items (RDF graphs)…

... serialized in (compliant) JSON-LD

[{ "@graph": {"@id": "http://.../G1",{ "@id": "http://.../a", "http://.../isIn": {"@id":"http://.../rRoom"}}

},{ "@id": "http://.../G1","prov:generatedAt":"2016-16-12T00:01:00"

}},{ "@graph": {

"@id": "http://.../G2",{ "@id": "http://.../b",

"http://.../isIn": {"@id":"http://.../rRoom"}}},{ "@id": "http://.../G2",

"prov:generatedAt":" 2016-16-12T00:03:00"}

},…

Compliant with W3C RSP abstract data model

G1

G2

G3

{:a :isIn :rRoom}

{:b :isIn :bRoom}

{:c :talksIn :rRoom,

:d :talksIn :bRoom}

S

3

5

1

t

A model to describe RDF streams

A description of the RDF stream should be provided

The identifier of the stream

A description of the schema of the stream items

Data item samples

The location of the stream endpoint (e.g. WebSocket URL)

This description is provided through the RDF Stream

Descriptor

Serialised in RDF

An extension of DCAT and SPARQL Service Descriptor

Published according to the linked data principles




Two interfaces

Producer

Consumer

We distinguish three types of actors (depending on the

implemented interfaces)

Producer Consumer

Stream source

Stream transformer

Stream sink



Producer

ConsumerStream Descriptor

endpoint

RDF stream

endpoint

Get stream descriptor (SD)

SDProcess

SD

Subscribe to stream

Stream item

Stream item

Stream item…

Process

stream

Protocols


The RDF Stream Descriptor is accessible through HTTP

The transmission of the stream can happen through

different protocols

HTTP chunked encoding

WebSocket

Message Queing Telemetry Transport (MQTT)

Server-Sent Events (SSE)

...

Proof of concepts


TripleWave

Stream source

Native implementation of WeSP

http://streamreasoning.github.io/TripleWave

C-SPARQL

Stream transformer

WeSP implemented as a wrapper

https://github.com/dellaglio/csparql-wesp

CQELS

Stream transformer

Native implementation of WeSP

https://github.com/cqels/CQELS-1.x/

http://streamreasoning.github.io/TripleWave

https://github.com/dellaglio/csparql-wesp

https://github.com/cqels/CQELS-1.x/

Conclusions


WeSP: framework to exchange RDF streams on the web

RDF to serialise the stream items

RDF to describe the stream

Application and communication protocols: HTTP, WebSocket,

MQTT, etc.

Interfaces to produce and consume RDF streams

What’s next?

Relation with other technologies: LDN, Activity Streams, etc.

Adoption

Federated stream processing over the Web

Thank you! Questions?

On a web of data streams


Daniele Dell’Aglio, Danh Le Phuoc, Anh Lê Tuán,

Muhammad Intizar Ali and Jean-Paul Calbimonte

[email protected]

http://dellaglio.org

@dandellaglio



mailto:[email protected]

http://dellaglio.org/

On a web of data streams

Technology

Transcript of On a web of data streams