Post on 15-Apr-2017
INTEROPERABLE & EFFICIENT
EUGENE SIOWTHANASSIS TIROPANISWENDY HALL
LINKED DATA FOR THE INTERNET OF THINGS
UNIVERSITY OF SOUTHAMPTON
DEVICES & DATA: SECURITY, PRIVACY, LOCALITY?
“The Internet of Things is currently beset by product silos.” W3C Web of Things Interest Group
CURRENT STATE OF THE
INTERNET OF THINGSPRODUCT & DATA SILOS
DEPENDENCY ON THE CLOUD
PERFORMANCE OF APPLICATIONS & ANALYTICS
DATA OWNERSHIP & PRIVACY WITH
LIGHTWEIGHT COMPUTERSA Smart Home Scenario implementing a Personal IoT Repository
Smart Home Dashboard
Personal IoT Repository
Environmental Sensors
Energy Meters
Data Stream
Energy SavingAnalytics
Stream & Historical Queries
Motion Sensors
Data ownershipOwn and store your data at home
Less Cloud ENCRYPTIONBETTER PERFORMANCE
SPECIFIC POLICIES/CONTROLONLINE/OFFLINE, TRUST, ACCESS CONTROL
DATA LOCALITY WITH
LIGHTWEIGHT COMPUTERSA Distributed Meteorological Scenario, minimising cloud dependency for Storage and Processing
Irrigation Application
Soil Moisture Analytics
Environmental Sensors
Lightweight Computer Hub
Data Stream
WeatherData
State InclementWeather PlanningApplication
National DisasterMonitoring Application
Cloud
INTRODUCING
LINKED DATAFOR INTEROPERABILITY
URI and ontologiesEstablish common data structures & References
http://thing.io/1is a
http://ont/weather_sensorCLASS
produceshttp://thing.io/obs/1
http://ont/temp_observation
is a
13.0
has value
CLASS
℃
unit
ENABLES RICH METADATAwhat, where, WHEN, HOW of DATA
located athttp://thing.io/loc/1
latitude longitude
-1.4150.9
PERFORMANCE CHALLENGESSTORES DON’T SCALE & PERFORM WELL ON WEB YETBuil-Aranda, C., Hogan, A.: SPARQL Web-Querying Infrastructure: Ready for Action? ISWC 2013
THE SHAPE OF IOT TIME-SERIES DATA
{timestamp : 1467673132,temperature : {
max: 22.0,min: 15.0,current: 17.0,error: {
percentage: 5.0}
}}
FLAT{
timestamp : 1467673132,temperature : 32.0,wind_speed : 10.5,pressure : 1016
}
COMPLEX{
timestamp : 1467673132,temperature : 32.0,wind_speed : 10.5,pressure : 1016,precipitation: 0,humidity: 93.0,
}
12345
WIDTH
THE SHAPE OF IOT TIME-SERIES DATA
20kUNIQUE DEVICES
dweet.io
18.5kNON-EMPTYSCHEMATA
92.3%
18k99.5%
FLAT SCHEMATA
920.5%
COMPLEX SCHEMATA
1
2,3
4
5
6+
Width
OPTIMISING FOR
TIME-SERIESDATA
THING
TEMPERATURE OBS
HUMIDITY OBS
WIND SPEED OBS
13.0
2016-01-01 06:00:00
CELCIUS
93.0
2016-01-01 06:00:00
PERCENT
10.5
2016-01-01 06:00:00
MPH
LOCATION
produces
produces
located
produces
has value
unit
time
RDF GRAPH
THING
TEMPERATURE OBS
HUMIDITY OBS
WIND SPEED OBS
13.0
LOCATION
produces
produces
located
produces
has value
THING
THING
THING
TEMPERATURE OBS
timeTEMPERATURE OBS 2016-01-01 06:00:00
unitTEMPERATURE OBS celcius
93.0has valueHUMIDITY OBS
timeHUMIDITY OBS 2016-01-01 06:00:00
unitHUMIDITY OBS PERCENT
10.5has valueWIND SPEED OBS
timeWIND SPEED OBS 2016-01-01 06:00:00
unitWIND SPEED OBS MPH
OPTIMISING FOR
TIME-SERIESDATA
RDF TRIPLES
SHARE COLUMN HEADERS
NO JOINS WITHIN ROWS
‘JUST IN TIME’ METADATA
OURAPPROACH
OPTIMISING FOR
TIME-SERIESDATA
THING
TEMPERATURE OBS WIND SPEED OBS
CELCIUS PERCENT MPH
LOCATION
produces
located
HUMIDITY OBS
unit
TEMPERATURE HUMIDITY WIND SPEED
13.0 93.0 10.5
TIME
2016-01-01 06:00:00
DESIGNING OUR ENGINE
THING
TEMPERATURE OBS WIND SPEED OBS
CELCIUS PERCENT MPH
LOCATION
produces
located
HUMIDITY OBS
unit
TEMPERATURE HUMIDITY WINDSPEED
13.0 93.0 10.5
TIME
2016-01-01 06:00:00
Table1
TABLE1.TEMPERATURE
has value has value
TABLE1.HUMIDITY
has value
TABLE1.WINDSPEED
DESIGNING OUR ENGINE
THING
TEMPERATURE OBS WIND SPEED OBS
CELCIUS PERCENT MPH
LOCATION
produces
located
HUMIDITY OBS
unit
TEMPERATURE HUMIDITY WINDSPEED
13.0 93.0 10.5
TIME
2016-01-01 06:00:00
Table1
TABLE1.TEMPERATURE
has value has value
TABLE1.HUMIDITY
has value
TABLE1.WINDSPEED
DESIGNING OUR ENGINE
THING
TEMPERATURE OBS WIND SPEED OBS
CELCIUS PERCENT MPH
LOCATION
produces
located
HUMIDITY OBS
unit
TEMPERATURE HUMIDITY WINDSPEED
13.0 93.0 10.5
TIME
2016-01-01 06:00:00
Table1
TABLE1.TEMPERATURE
has value has value
TABLE1.HUMIDITY
has value
TABLE1.WINDSPEED
MAX( )?TEMPERATURESELECT
?OBS TEMPERATURE OBSa
has value?OBS ?TEMPERATURE
has unit?OBS ?uom
{
}
SELECT
MAX( )?TEMPERATURE
?OBS TEMPERATURE OBSa
has value?OBS ?TEMPERATURE
has unit?OBS ?uom
DESIGNING OUR ENGINE
TEMPERATURE OBS
CELCIUS
TEMPERATURE
13.0 10.5
TABLE1.TEMPERATURE
has value
MAX( )?TEMPERATURESELECT
?OBS TEMPERATURE OBSa
has value?OBS ?TEMPERATURE
has unit?OBS ?uom
{
}
SELECT
MAX( )?TEMPERATURE
?OBS TEMPERATURE OBSa
has value?OBS ?TEMPERATURE
has unit?OBS ?uom
SPARQL
SQL
DESIGNING OUR ENGINE
MAX( )?TEMPERATURESELECT
?OBS TEMPERATURE OBSa
has value?OBS ?TEMPERATURE
has unit?OBS ?uom
{
}
SELECT
MAX( )?TEMPERATURE
?OBS TEMPERATURE OBSa
has value?OBS ?TEMPERATURE
has unit?OBS ?uom
SELECT
MAX( )?TEMPERATURE
?OBS ?TEMPERATURE ?uom
TABLE1.TEMPERATURE CELCIUSNODE_TEMP
SELECT MAX( )TEMPERATURE FROM TABLE1
BENCHMARKS & IOT Scenarios
17
Meteorological SYSTEM
~20,000 Stations
100 – 300k triples
Wind, Rainfall, etc.
10 SRBench Queries
ANALYTICS HUB
STATION HUBSTATION HUB
Weather SENSORS
Weather SENSORS
Weather SENSORS
3 months, 1 home
~30k triples
Motion, energy, env
4 Analytics Queries
PERSONAL STORE
Weather SENSORS
Weather SENSORS
DEVICES W/SENSORS
SMART HOMEANALYTICS
LIGHTWEIGHT COMPUTER
COMPUTER/SERVER
CLUSTER
DEVICE
SENSORCo
mpu
te &
Sto
rage
Level of Distribution
github.com/eugenesiow/sparql2sql
STORAGE SIZE
3ookHurricane Ike
1ookNEVADA BLIZZARD
3okSMART HOME
OUR APPROACH (s2S)
NATIVE STORE (TDB)
x15
x68
x112
Get the rainfall observed in a particular hour from all stations01
02
SRBENCH QUERY RESULTS
Q01 with an optional clause on unit of measure
OUR APPROACH (S2S)
NATIVE STORE (TDB)
x4.6
x4
03
04
05
Detect if a hurricane has been observedX3.4
Get the average wind speed at the stations where the air temperature is >32
x88
Join between wind observation and temperature observation subtrees time-consuming in low resource
environment (Raspberry Pi)
X2.7Detect if a station is observing a blizzard
06
07
08
Get the stations with extremely low visibilityX6
Detect stations that are recently brokenx14
X5.6
Get the daily minimal and maximal air temperature observed by the sensor at a given location
09
10
Get the daily average wind force and direction observed by the sensor at a given location
Get the locations where a heavy snowfall has been observed
x305
X7
Our Approach (s2s) is shown to be faster on all queries in the Distributed Meteorological System
Join between wind force and wind direction observation subtrees is time-consuming in low resource
environment (Raspberry Pi)
Temperature aggregated by hour on a specified day01
02
SMART HOME QUERY RESULTS
Minimum and maximum temperature each day for a particular month
OUR APPROACH (S2S)
NATIVE STORE (TDB)
x29
x9
03
04
Energy Usage Per Room By Day
Diagnose unattended appliances consuming energy with no motion in room
x69
Our Approach (s2s) is shown, once again, to be faster on all queries for Smart Home Analytics
x3.6
Involves motion and meter data (much larger set), with space-time aggregations and joins between motion and
meter tables/subgraphs.
Involves meter data (larger set), with space-time aggregations.
WHY IS
OUR APPROACHFASTER THAN NATIVE RDF?
FASTER AGGREGATIONS ON LESS RESOURCESCAN SPECIFCALLY BUILD INDEXES FOR FAST RANGE QUERIES
EFFICIENT SQL QUERIES
OPTIMISE FLAT & WIDE DATA ACCESSREDUCE JOINS BETWEEN SUBGRAPHS ON THE SAME ROW
COLLAPSE INTERMEDIATE NODESREDUCE JOINS W/ BLANK OR FAUX NODES IN MAPPINGS
RELATED WORK
Rodriguez-Muro, M., Rezk, M. (2014) Efficient SPARQL-to-SQL with R2RML mappings. Web Semantics: Science, Services and Agents on the World Wide Web 33, pp. 141–169
-ontop-
morph
sparql2stream
Priyatna, F., Corcho, O., Sequeda, J. (2014) Formalisation and Experiences of R2RMLbased SPARQL to SQL Query Translation using Morph. Proceedings of the 23rd International Conference on World Wide Web pp. 479–489
GENERAL ONTOLOGY BASED DATA ACCESS ENGINES
sparql2sql
Siow, Eugene, Tiropanis, Thanassis and Hall, Wendy (2016) SPARQL-to-SQL on internet of things databases and streams. Proceedings of the 15th International Semantic Web Conference (accepted, to be published)
github.com/eugenesiow/sparql2sql
github.com/eugenesiow/piotreSiow, Eugene, Tiropanis, Thanassis and Hall, Wendy (2016) PIOTRe: Personal IoT Repository. Proceedings of the 15th International Semantic Web Conference P&D (accepted, to be published)
“Until they become conscious they will never rebel and until after they have rebelled they cannot become conscious.”
DATA OWNERSHIP & DATA LOCALITYDISTRIBUTED LIGHTWEIGHT COMPUTERS FOR STORAGE AND PROCESSING IN THE IOT
1984 by George Orwell
LINKED DATA FOR INTEROPERABILITYA rich model to describe things and integrate connected thing’s data
NOVEL TIERED LINKED DATA STOREFROM 3 to 3 orders of magnitude performance improvement
@eugene_siow