On-the-fly Integration of Static and Dynamic Linked Data
-
Upload
aharth -
Category
Technology
-
view
213 -
download
0
description
Transcript of On-the-fly Integration of Static and Dynamic Linked Data
KIT – University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association
INSTITUTE OF APPLIED INFORMATICS AND FORMAL DESCRIPTION METHODS (AIFB)
www.kit.edu
On-the-fly Integration of Static and Dynamic Linked Data
Andreas Harth (KIT), Craig Knoblock (USC), Steffen Stadtmüller (KIT), Rudi Studer (KIT), Pedro Szekely (USC)
2 On-the-fly Integration of Static and Dynamic Linked Data
Outline
Motivation
Scenario and Overview
Modelling Sources: Karma
Accessing and Integrating Sources: Data-Fu
Demo
Conclusion
3 On-the-fly Integration of Static and Dynamic Linked Data
Motivation
The relevance of many types of data perishes or degrades over time (e.g., weather information, moving objects)
Timely decision making requires access to live data and
inclusion of new sources in a flexible manner.
Our goals(Near) real-time access to a variety of data sources in a range of data formats and access modalities
Rapidly integrate sources via modeling and to generate a Linked Data interface to live sources
4 On-the-fly Integration of Static and Dynamic Linked Data
Static vs. Dynamic Sources
Various sources have different update intervals (from minutes to weeks)
We treat the access to all sources in the same way via polling (HTTP GETs)
Thus, the only distinction between „static“ and „dynamic“ sources is how fast we refresh the query results for each source
5
Scenario
On-the-fly Integration of Static and Dynamic Linked Data
Marine Vessels (AIS)
Vehicles(Campus Cruisers)
POIs(Crunchbase,
OSM, Wikimapia)
Venues/Events(Eventful, LastFM)
Buses/Stops(LA Metro)
6
Architecture
On-the-fly Integration of Static and Dynamic Linked Data
7 On-the-fly Integration of Static and Dynamic Linked Data
Karma
Interactive tool for rapidly extracting, cleaning, transforming, integrating, and publishing data
Hierarchical Sources
ServicesModel
KarmaTabularSources
Database
…
See http://isi.edu/integration/karma/ for more info and download
8
Modelling Sources with Karma
Karma is a data integration toolMap data onto an ontology to generate Linked Data
Karma extension to enable the on-the-fly lifting of API I/O data according to a pre-defined mapping model
On-the-fly Integration of Static and Dynamic Linked Data
Linked API
Web API
Vehicles(Campus Cruisers)
9 On-the-fly Integration of Static and Dynamic Linked Data
Linked Data Access to Event APIs
LastFM API
Given a lat/lon of a location, return a list of event identifiers
http://km.aifb.kit.edu/services/lastfmwrap/geo.getevents?lat={?lat}&long={?lon}Given an event identifier, return details about the event
http://lastfm.rdfize.com/events/{event-id}
Eventful API
List events given a keyword search term and a date range
http://km.aifb.kit.edu/services/eventfulwrap/search?location={?loc}&date={?date}
Venues/Events(Eventful, LastFM)
10 On-the-fly Integration of Static and Dynamic Linked Data
LastFM Data-Fu Program (I)
Program at http://km.aifb.kit.edu/services/data-fu/lastfm
with input lat/lon (in RDF via HTTP POST)
Rule to search for events at given location:
{ ?p geo:long ?lon . ?p geo:lat ?lat . } => { [] http:mthd http:GET ; http:requestURI <http://km.aifb.kit.edu/services/lastfmwrap/geo.getevents?lat={?lat}&long={?lon}> . } .
“For the input point with lat/long
perform an HTTP GET
at the KIT LastFM Wrapper URIconstructed with the lat/long”
11 On-the-fly Integration of Static and Dynamic Linked Data
LastFM Data-Fu Program (II)
Rule for retrieving information about the found events, including geolocation of event:
Query to return a table with lat/lon and label to transform to KML/Google Earth:
{ ?e rdf:type lode:Event. } => { [] http:mthd http:GET ; http:requestURI ?e . } .
“For every resource of type event
perform an HTTP GETat the resource URI”
:q1 qrl:select ( ?event ?place ?label ?lat ?lon ) ;qrl:where { ?event <http://purl.org/NET/c4dm/event.owl#place> ?place . ?event rdfs:label ?label . ?place geo:lat ?lat . ?place geo:long ?lon .} .
“Output is every entity with latitude, longitude and associated label”
12 On-the-fly Integration of Static and Dynamic Linked Data
Data Source Characteristics
13 On-the-fly Integration of Static and Dynamic Linked Data
Demo
Load http://people.aifb.kit.edu/aha/2013/d3/index.kml into Google Earth
Location of buses and ships are updated
14 On-the-fly Integration of Static and Dynamic Linked Data
Conclusion
System interoperation in distributed environments with Linked Data as interface
Rapid integration of new sources (via Karma models and Data-Fu scripts)
Realtime access to networked data via Data-Fu scripts/programshttp://code.google.com/p/data-fu/
Ability to rapidy integrate new sources via Karma modelshttp://www.isi.edu/integration/karma/
Future workModular organisation of programs
Manipulating resource state (Read-Write Linked Data)
Optimisations for limited bandwidth environments
15 On-the-fly Integration of Static and Dynamic Linked Data
Challenges
Data is provided at different places, by different owners, often over the web (decentralised data publishing)
Data and links are provided in a many different formats/protocolsDevelopers have to gain a deep understanding of every API by reading textual descriptions
Applications (user agents) are supposed to follow links as found during runtime of the application
Developers have to define their desired interaction at design time
Developers have to write individually tailored code to consume services in applications