GameQube UCF Senior Design Group 33 Stephen Monn (CpE) Omar Alami (CpE) Matthew Dworkin (CpE)
Towards A Semantic Web Application for NVD-CPE
description
Transcript of Towards A Semantic Web Application for NVD-CPE
![Page 1: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/1.jpg)
Vaibhav Khadilkar Jyothsna RachapalliDr. Bhavani ThuraisinghamThe University of Texas at Dallas
![Page 2: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/2.jpg)
Humans are capable of using the Web to carry out tasks such as finding the ◦ Finnish word for "monkey", ◦ reserving a library book,◦ searching for a low price for a DVD.
However, a Computer cannot accomplish the same tasks without human direction because web pages are designed to be read by people, not machines.
The semantic web is a vision of information that is understandable by computers, so that they can perform more of the tedious work involved in finding, sharing, and combining information on the web.
![Page 3: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/3.jpg)
CPE is a structured naming scheme for IT systems, platforms, and packages.
A CPE Name is represented by a URI. Each name consists of the prefix "cpe:" and is followed by
up to seven different components. These components are used to help build consistent and
unique names. The components relate to ◦ platform part, ◦ vendor, ◦ product name,◦ version, ◦ update level,◦ Edition◦ language.
![Page 4: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/4.jpg)
Motivation to opt for semantic web technology
Architecture of a semantic web application
Semantic web technologies overview
Strategy for creation of semantic web application
Performance metrics
![Page 5: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/5.jpg)
National Vulnerability Database (NVD) ◦Contains product and vulnerability management data
◦Based on a relational model
Goal is to enable automation of
◦Vulnerability management
◦Security measurement and compliance
Relational model imposes limitations
◦Product composition difficult to achieve. Find all products containing a TCP/IP device?
Find all products within common codebase?
Advantage of semantic model - Reasoning!
![Page 6: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/6.jpg)
An ontology provides a precise vocabulary with which knowledge can be represented”
“This vocabulary allows us to specify which entities will be represented, how they can be grouped, and what relationship connect them together”
![Page 7: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/7.jpg)
RDF is a language for representing information about resources in the World Wide Web.
RDF is intended for situations in which this information needs to be processed by applications, rather than being only displayed to people.
RDF is intended to provide a simple way to make statement◦ the part that identifies the thing the statement is about is called
the subject.◦ The part that identifies the property of the subject is called the
predicate◦ and the part that identifies the value of that property is called the
object.
![Page 8: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/8.jpg)
Creation of products ontology for NVD-CPE
Creation of a corresponding view in relational DB
Migrate data from relational to semantic model
Create a web application using the new model
This application should enable user to◦Navigate ◦Search ◦Query the data
![Page 9: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/9.jpg)
Converter
◦Converts data form various sources(e.g.,tables, spreadsheets, webpages) into RDF
RDF Parser and Serializer
◦Facilitates reading and writing RDF in one of several file formats (e.g., N3, N-TRIPLE, RDF/XML)
RDF Store (or triple store) ◦ Is a database that is optimized for the storage and
retrieval of many short statements called triples
![Page 10: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/10.jpg)
Reasoner
◦A program that performs inferences according to specified inference rules
SPARQL
◦The W3C standard query language for RDF
Application interface
◦Uses the content of an RDF store in an interaction with some user
![Page 11: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/11.jpg)
Converters D2RQ used during first approach Jena API to read relational data into a Jena model
Parser/Serializer Jena API to read and write the triples into any serialization
format RDF Store
RDB, SDB and Allegrograph Inferencing
Pellet Reasoner SPARQL
ARQ is a query engine for Jena that supports SPARQL
![Page 12: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/12.jpg)
The Jena Framework provides
◦ A RDF API
◦ Reading and writing RDF in RDF/XML, N3 and N-Triples
◦ An OWL API
◦ In-memory and persistent storage
◦ SPARQL query engine
◦ Built in Reasoners
◦ Plug-in for external reasoners
![Page 13: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/13.jpg)
SPARQL SPARQL
Ontology API
Core RDF Model API
Inference API (Reasoners)
Ontology API
Core RDF Model API
Inference API (Reasoners)
RDF FILESRDF FILES
APPLICATIONAPPLICATION
Converters Parser
Serializer
DBDBDBDB
RDB SDB AllegroGraph
RDF/Triple Stores
![Page 14: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/14.jpg)
Step 1 - Use Cases
◦Describe initial, most difficult requirements in conversational, informal English
◦Work with domain experts to create use cases required by a given domain
◦Use case examples Searching – “What are all the products that have a Vendor
of Microsoft and a product name of windows_nt?”
Equality – “Determine if two instances are equal”
![Page 15: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/15.jpg)
Step 2 - Ontology creation and validation
◦Use an ontology editor to create an ontology/schema based on the use cases created in Step 1
◦Ontology editor used: Protégé 4.0
◦External reasoner plug-in: Pellet
◦Creation of Classes and corresponding subclasses
Properties: Object properties as well as data properties
Individuals of a class
◦Run the reasoner to validate the correctness of model
![Page 16: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/16.jpg)
= <owl:Class>
= <rdfs:subClassOf>
ABC = <rdf:Property>
Identification concept hierarchy
Product category concept hierarchy
hasIdentification
Relationship connecting the two structures
![Page 17: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/17.jpg)
Step 3 - Ontology migration to Jena
◦ Create Java classes using Ontology generated in Step 2
◦ Java classes are created using Schemangen
Input to Schemagen: Ontology.owl
Output from Schemagen: Ontology.java
Step 4 - Data migration
◦ Perform Data Migration – Two approaches
◦ First approach
Mapping relational data to RDF with a mapping tool
◦ Second approach
Mapping relational data to RDF using database view
![Page 18: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/18.jpg)
Database to Relational Query (D2RQ) allows us to view the relational database as an RDF triples
D2RQ mapping file
◦ Maps database columns to predicates in the ontology
Use the mapping file to convert the relational database into triples
A triple is created as follows
◦primary key of table ---> subject
◦ column name ---> predicate
◦ value of the cell ---> object
![Page 19: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/19.jpg)
First approach limitations
◦D2RQ is not required when a combined view of different tables is used as is the case with the NVD-CPE database
◦D2RQ does not allow us to update database tables
Second approach
◦ Involves creating a new relational schema that is closely related to the ontology
◦This schema will serve as a stepping stone for the data along the path to the semantic store
![Page 20: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/20.jpg)
Create a view that combines required columns from various tables
Read tuples from this view (table) to convert the product information into triples
The triple is now created as
◦primary key ( cpe name ) ---> subject
◦predicate based on the ontology ---> predicate
◦ value of the cell ---> object
![Page 21: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/21.jpg)
Step 5 - Reasoning
◦The process by which new triples are systematically added to a graph based on patterns in existing triples.
◦ Inference rules Systematic patterns defining which of the triples should
be inferred.
◦Steps involved Choose a reasoner - Pellet (External reasoner)
Create inference rules as part of the ontology using OWL
Run the reasoner
Verify the correctness of the inference rules using inferred triples
![Page 22: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/22.jpg)
Step 6 - SPARQL queries
◦SPARQL queries are very similar to SQL queries.
◦Write SPARQL queries for each of the use cases from Step 1
Step 7 - Application ◦ Integrate the newly implemented functionality with the
web application.
◦Create user interface that enables Navigation
Search
Querying
![Page 23: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/23.jpg)
Step 8 - Performance with triple stores ◦ Performance metrics to test for
Load time - Load triples in to triple store Query times - Running time of the sparql queries for various
use cases
◦ Perform testing on triple stores like RDB, SDB and AllegroGraph and document corresponding performance metrics
Step 9 - Cyclic process ◦ Write additional use case scenarios and repeat the process
until all use cases have been modeled◦Refine model until correct inferences are being drawn.
![Page 24: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/24.jpg)
Strategy
![Page 25: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/25.jpg)
RDB,SDB and Allegrograph triple stores are optimized and indexed
Metrics measure performance on◦ 94216 products without reasoning
◦ 5961 products with reasoning
Example Queries◦ List all the vendors
◦ List all the products
◦ List products created in given range of time period
◦ List all products for a given vendor or given creation date
Example Queries with reasoning◦ Products containing TCP/IP devices
◦ Products containing a given shared library
![Page 26: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/26.jpg)
Metric Relational View RDB SDB AllegroGraph
Version SQL Server 05 Jena-2.5.6 SDB-1.1 AllegroGraph-3.2
Size(Rows/Triples) 96485 (R) 982403 (T) 982403 (T) 982403 (T)
Total Space (MB) 13.08 1044.00 302.63 387.00
Index Space (MB) 0.008 674.22 75.55 316.06
Log Space (MB) - 285.06 82.44 -
Load time - 231.6 s 284.6 s 164.8 s
![Page 27: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/27.jpg)
Metric RDB SDB AllegroGraph
Version Jena-2.5.6 SDB-1.1 AllegroGraph-3.2
Size(Rows/Triples) 97814 (T) 97814 (T) 97814 (T)
Total Space (MB) 118.31 61.38 38
Index Space (MB) 66.98 9.65 31.46
Log Space (MB) 13.31 38.38 -
Load time 18.58 hrs 17.62 hrs 19.06 hrs
![Page 28: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/28.jpg)
Query (Triples) RDBMS(ms) RDB(ms) SDB(ms) AllegroGraph(ms)
Vendors (9898 ) 53.2 737.4 711.2 945.6
Products (96216) 10.6 1013.2 723.4 5572.8
MS Products (2616) 12 26.4 30.0 141.4
‘win ce’ Agent (1) 27 74.8 8.4 11.0
All CPE names(96216) 11 1235.0 1274.6 7321.2
Given CPE name(1) 1 838.6 472.2 5425
All creation dates (96216)
8.2 1183.8 1499.4 5464.4
Given creation date (56811)
70.6 937.4 1427.4 5519
Type ‘a’ (82981) 34 749.6 1120.6 5325
Group by Type h=4941,o=8294,a=82981
92.6 768.4 1243.8 5406.2
![Page 29: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/29.jpg)
Reasoning Performed on 5961 products
Total Number of products - 96216
![Page 30: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/30.jpg)
![Page 31: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/31.jpg)
![Page 32: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/32.jpg)
![Page 33: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/33.jpg)
![Page 34: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/34.jpg)
Choice of semantic model instead of relational model enhances automation of Vulnerability management
Creating a comprehensive list of use cases at once is challenging.◦ Cyclical process makes incorporation of new use cases flexible
Efforts must be taken to optimize triple store performance
Implementation of a system must carefully choose a triple store/reasoner for their implementation ◦ Trade-off between speed and power
![Page 35: Towards A Semantic Web Application for NVD-CPE](https://reader036.fdocuments.us/reader036/viewer/2022062517/56813b19550346895da3cbd3/html5/thumbnails/35.jpg)
• http://jena.sourceforge.net/
• http://nvd.nist.gov/
• http://www.semanticsupport.org/
• http://www.w3.org/2007/03/RdfRDB/papers/d2rq-
positionpaper/
• http://www4.wiwiss.fu-berlin.de/bizer/D2RQ/spec/
• Dean Allemang, James Hendler: Semantic Web for the
Working Ontologist: Effective Modeling in RDFS and OWL
• John Hebeler , Matthew Fisher , Ryan Blace , Andrew Perez-
Lopez:Semantic Web Programming