Benchmarking Faceted Browsing Capabilities of Triple Stores
-
Upload
holistic-benchmarking-of-big-linked-data -
Category
Technology
-
view
139 -
download
3
Transcript of Benchmarking Faceted Browsing Capabilities of Triple Stores
Benchmarking Faceted Browsing Capabilities of Triple Stores
Horizon 2020GA No 688227
01/12/2015 – 30/11/2018
Henning Petzka, Claus Stadler, Georgios Katsimpras, Bastian Haarmann, Jens Lehmann
13.09.2017
SEMANTiCS Amsterdam 2017
HOllistic Benchmarking of Big lInked daTa
Rationale:A community-driven unified benchmarking platform for the community
• Focus on Big Linked Data• Provide benchmarks and baselines• Provide reference implementation of KPIs• Extensible and referenceable• Result analysis• Open Source
http://project-hobbit.eu
• Benchmarks I: Generation & Acquisitionmeasures performance of SPARQL query processing systems when faced with streams ofdata in terms of efficiency and completeness
• Benchmarks II: Analysis & Processingtest performance on instance matching tools for Linked Data and performance on machinelearning methods for data analytics
• Benchmarks III: Storage & Curationhas its focus on storage components and versioning systems to efficiently manage evolvinglinked datasets
• Benchmarks IV: Visualization & Serviceshas its focus on benchmarks regarding question answering and faceted browsing.
Faceted Browsingstands for a session-based and state-dependent
interactive method for query formulation over a multi-dimensional information space.
A browsing scenario consists of applying (or removing) filter restrictions defined by object-valued properties or of changing the range of a property value of various data types.
Choke Points
! In a browsing scenario it is the efficient transitionfrom one state to next one that determines the user
experience !
Three basic types of transition
1. Class-based transition2. Property- or property path-based transition3. Entity type switch
Scenarios• make sense in a real-world browing scenario and• cover all types of transitions as specified by the choke points
Key Performance Indicators
• Instance retrieval:• Query-per-second score• Precision• Recall• F1-Score
• Facet counts:• Query-per-second score• Several metrics for accuracy
Over all queries and for each choke pointindividually
MOCHA Challenge at ESWC 2017
Benchmark on Faceted Browsing was part of theMighty Storage Challenge at the ESWC 2017
Two participants vs. baseline system• QUAD by Ontos• Virtuoso 8.0 Commercial Edition (beta release)
vs. Virtuoso 7.2 Open-Source Edition
No results for QUAD due to time out.
Preliminary results
Georgala, Spasic, Jovanovik, Petzka, Röder, Ngonga Ngomo. MOCHA2017: The Mighty Storage Challenge at ESWC 2017, ESWC challenge proceedings (Springer)