1 Future Database Needs SC 32 Study Period February 5, 2007 Bruce Bargmeyer, Lawrence Berkley...

Future Database NeedsSC 32 Study Period

February 5, 2007

Bruce Bargmeyer, Lawrence Berkley National LaboratoryUniversity of CaliforniaTel: +1 510-495-2905bebargmeyer@lbl.gov

JTC1 SC32N1633

Topics

Study period purposeNew challengesA brief tutorial on Semantics and semantic

computingwhere XMDR fits

Semantic computing technologies Traditional Data Administration

Some limitations of current relational technologies Some input from other sources

Future Database NeedsStudy Period

A one-year study period to identify and understand case studies related to this area.

Bring together a small group of experts in a meeting on “Case Studies on new Database Standards Requirements”.

The workshop would provide input to existing SC32 projects and may provide background material for new proposals for upgrades or for new work within SC32 in time for 2007 SC32 Plenary

--Document 32N1451

The Internet Revolution

A world wide web of diverse content: The information glut is nothing new. The access to it is astonishing.

Challenge: Find and process non-explicit data

Analgesic Agent

Non-Narcotic Analgesic

AcetominophenNonsteroidal Antiinflammatory Drug

Analgesic and Antipyretic

DatrilAnacin-3 Tylenol

For example…

Patient data on drugs contains brand names (e.g. Tylenol, Anacin-3, Datril,…);

However, want to study patients taking analgesic agents

Challenge: Specify and compute across Relations, e.g., within a food web in an

Arctic ecosystem

An organism is connected to another organism for which it is a source of food energy and material by an arrow representing the direction of biomass transfer.

Source: http://en.wikipedia.org/wiki/Food_web#Food_web (from SPIRE)

Challenge: Combine Data, Metadata & Concept Systems

ID Date Temp Hg

A 06-09-13 4.4 4

B 06-09-13 9.3 2

X 06-09-13 6.7 78

Name Datatype Definition Units

ID textMonitoring Station Identifier

not applicable

Date date Date yy-mm-dd

Temp numberTemperature (to 0.1 degree C)

degrees Celcius

Hg numberMercury contamination

micrograms per liter

Inference Search Query:“find water bodies downstream from Fletcher Creek where chemical contamination was over 10 micrograms per liter between December 2001 and March 2003”

Metadata:

Biological Radioactive

Contamination

lead cadmiummercury

Chemical

Concept system:

Challenge: Use data from systems that record the same facts with different terms

Common Content

OASIS/ebXMLRegistries

Common Content

ISO 11179Registries

Common Content

OntologicalRegistries

Common Content

CASE ToolRepositories

Common Content

UDDIRegistries

CountryIdentifier

DataElement

XML Tag

TermHierarchy

Attribute

BusinessSpecification

TableColumn

SoftwareComponentRegistries

Common Content

DatabaseCatalogs

BusinessObject

DublinCore

Registries

Common Content

Coverage

Data Elements

ISO 3166English Name

ISO 31663-Numeric Code

ISO 31662-Alpha Code

Algeria

Belgium

Denmark

France

Zimbabwe

Name:Context:Definition:Unique ID: 4572Value Domain:Maintenance Org.Steward:Classification:Registration Authority:Others

ISO 3166French Name

L`Algérie

Belgique

Danemark

Egypte

La France

Zimbabwe

Same Fact, Different Terms

Algeria

Belgium

Denmark

France

Zimbabwe

Name: Country IdentifiersContext:Definition:Unique ID: 5769Conceptual Domain:Maintenance Org.:Steward:Classification:Registration Authority:Others

DataElementConcept

Challenge: Draw information together from a broad range of studies, databases, reports, etc.

Challenge: Gain Common Understanding of meaning between Data Creators and Data Users

Users Information systems

Data Creation

UsersUsers

EPAenvironagricultureclimatehuman healthindustrytourismsoilwaterair

123345445670248591308

3268082513485038270800002178

text data

environagricultureclimatehuman healthindustrytourismsoilwaterair

123345445670248591308

3268082513485038270800002178

ambienteagriculturatiemposalud hunanoindustriaturismotierraaguaaero

123345445670248591308

3268082513485038270800002178

text data

123345445670248591308

3268082513485038270800002178

text data

Others . . .

ambienteagriculturatiemposalud hunoindustriaturismotierraaguaaero

123345445670248591308

3268082513485038

3268082513485038270800002178

text data

A common interpretation of what the data represents

Challenge: Drawing Together Dispersed Data

Users Information systems

Data Creation

UsersUsers

EPAenvironagricultureclimatehuman healthindustrytourismsoilwaterair

123345445670248591308

3268082513485038270800002178

text data

123345445670248591308

3268082513485038270800002178

ambienteagriculturatiemposalud hunanoindustriaturismotierraaguaaero

123345445670248591308

3268082513485038270800002178

text data

123345445670248591308

3268082513485038270800002178

text data

Others . . .

ambienteagriculturatiemposalud hunoindustriaturismotierraaguaaero

123345445670248591308

3268082513485038

3268082513485038270800002178

text data

A common interpretation of what the data represents

Semantic Computing

We are laying the foundation to make a quantum leap toward a substantially new way of computing: Semantic Computing

How can we make use of semantic computing? What do organizations need to do to prepare for

and stimulate semantic computing?

Coming: A Semantic Revolution

Searching and rankingPattern analysisKnowledge discoveryQuestion answeringReasoningSemi-automated decision making

The Nub of It

Processing that takes “meaning” into account Processing based on the relations between things

not just computing about the things themselves.Computing that takes people out of the

processing, reducing the human toil Data access, extraction, mapping, translation,

formatting, validation, inferencing, …Delivering higher-level results that are more

helpful for the user’s thought and action

Semantics Challenges

Managing, harmonizing, and vetting semantics is essential to enable enterprise semantic computing

Managing, harmonizing and vetting semantics is important for traditional data management. In the past we just covered the basics

Enabling “community intelligence” through efforts similar to Wikipedia, Wikitionary, Flickr

A Brief Tutorial on Semantics

What is meaning?What are concepts?What are relations?What are concept systems?What is “reasoning”?

19C.K Ogden and I. A. Richards. The Meaning of Meaning.

Thought or Reference (Concept)

Referent Symbol

SymbolisesRefers to

Stands for“Rose”, “ClipArt”

Meaning: The Semiotic Triangle

Semiotic Triangle:Concepts, Definitions and Signs

CONCEPT

Referent

Refers To Symbolizes

Stands For

“Rose”,“ClipArt”

Definition

Definitions in the EPA Environmental Data Registry

http://www.epa/gov/edr/sw/AdministeredItem#MailingAddress

The exact address where a mail piece is intended to be delivered, including urban-style address, rural route, and PO Box

http://www.epa/gov/edr/sw/AdministeredItem#StateUSPSCode

The U.S. Postal Service (USPS) abbreviation that represents a state or state equivalent for the U.S. or Canada

http://www.epa/gov/edr/sw/AdministeredItem#StateName

The name of the state where mail is delivered

Mailing Address:

State USPS Code:

Mailing Address State Name:

SNOMED – Terms Defined by Relations

Computable Meaning

CONCEPT

Referent

Refers To Symbolizes

Stands For

“Rose”,“ClipArt”

rdfs:subClassOf owl:equivalentClass owl:disjointWith

If “rose” is owl:disjointWith “daffodil”, then a computer can determine that anassertion is invalid, if it states that a rose is also a daffodil (e.g., in a knowledgebase).

Fletcher CreekMerced

WaterBody

What are Relations?

Relation

Merced Lake

Fletcher CreekMerced River

isA isA

Concepts and relations can be represented as nodes and edges in formal graph structures, e.g., “is-a” hierarchies.

b a c d

Nodes represent concepts

Lines (arcs) represent relations

Concept Systems have Nodes and may have Relations

Concept systems are concepts and the relations between them.Concept systems can be represented & queried as graphs

A More Complex Concept Graph

From Supervaluation Semantics for an Inland Water Feature OntologyPaulo Santos and Brandon Bennett http://ijcai.org/papers/1187.pdf#search=%22terminology%20water%20ontology%22

Concept lattice of inland water features

Linear LargeNon-linear

Non-linear

Large linear Small linear Small non- linear

Deep Natural

Artificial

River Stream Canal Reservoir Lake Marsh Pond

Flowing Shallow Stagnant

Partial Order Tree

Ordered Tree

Faceted ClassificationDirected Acyclic GraphPartial Order Graph

Powerset of 3 element setBipartite Graph Clique

Compound Graph

Types of Concept System Graph Structures

Graph Taxonomy

Directed Graph

Directed Acyclic Graph

Undirected Graph

Bipartite Graph

Partial Order Graph

Faceted Classification

Clique

Partial Order Tree

Lattice

Ordered Tree

Note: not all bipartite graphsare undirected.

What Kind of Relations are There?Lots!

Relationship class: A particular type of connection existing between people related to or having dealings with each other.

acquaintanceOf - A person having more than slight or superficial knowledge of this person but short of friendship.

ambivalentOf - A person towards whom this person has mixed feelings or emotions.

ancestorOf - A person who is a descendant of this person. antagonistOf - A person who opposes and contends against this person. apprenticeTo - A person to whom this person serves as a trusted counselor or

teacher. childOf - A person who was given birth to or nurtured and raised by this person. closeFriendOf - A person who shares a close mutual friendship with this person. collaboratesWith - A person who works towards a common goal with this person.

Example of relations in a food web in an Arctic ecosystem

An organism is connected to another organism for which it is a source of food energy and material by an arrow representing the direction of biomass transfer.

Source: http://en.wikipedia.org/wiki/Food_web#Food_web (from SPIRE)

Ontologies are a type of Concept System

Ontology: explicit formal specifications of the terms in the domain and relations among them (Gruber 1993)

An ontology defines a common vocabulary for researchers who need to share information in a domain. It includes machine-interpretable definitions of basic concepts in the domain and relations among them.

Why would someone want to develop an ontology? Some of the reasons are: To share common understanding of the structure of information

among people or software agents To enable reuse of domain knowledge To make domain assumptions explicit To separate domain knowledge from the operational knowledge To analyze domain knowledge

http://www.ksl.stanford.edu/people/dlm/papers/ontology101/ontology101-noy-mcguinness.html

What is Reasoning?Inference

Polio Smallpox

Infectious Disease

Disease

is-a is-a

Diabetes Heart disease

Chronic Disease

Signifies inferred is-a relationship

Reasoning: Taxonomies & partonomies can be used to support inference queries

Oakland Berkeley

Alameda County

California

part-of

part-of part-of

part-of

Santa Clara San Jose

Santa Clara County

part-of

E.g., if a database containsinformation on events by city,we could query that database for events that happened in a particular county or state,even though the event data does not contain explicit state or county codes.

Reasoning: Relationship metadata can be used to infer non-explicit data

Analgesic Agent

AcetominophenNonsteroidal Antiinflammatory Drug

DatrilAnacin-3 Tylenol

For example…(1) patient data on drugs currently

being taken contains brand names (e.g. Tylenol, Anacin-3, Datril,…);

(2) concept system connects different drug types and names with one another (via is-a, part-of, etc. relationships);

(3) so… patient data can be linked and searched by inferred terms like “acetominophen” and “analgesic” as well as trade names explicitly stored as text strings in the database

Reasoning: Least Common Ancestor Query

Analgesic Agent

Acetominophen

Opioid

Opiate

Morphine Sulfate

Codeine Phosphate

Nonsteroidal Antiinflammatory Drug

What is the least common ancestor concept in the NCI Thesaurus for Acetominophen and Morphine Sulfate? (answer = Analgesic Agent)

Reasoning: Example “sibling” queries: concepts that share a common ancestor

Environmental:"siblings" of Wetland (in NASA SWEET ontology)

HealthSiblings of ERK1 finds all 700+ other kinase enzymesSiblings of Novastatin finds all other statins

11179 MetadataSibling values in an enumerated value

domain

HealthFind all the siblings of

Breast Neoplasm

EnvironmentalFind all chemicals that are acarcinogen (cause cancer) andtoxin (are poisonous) andterratogenic (cause birth

defects)

Reasoning: More complex “sibling” queries: concepts with multiple ancestors

site neoplasms breast disorders

Breast neoplasm

RespiratorySystem

neoplasm

Non-Neoplastic

Breast Disorder

Eye neoplasm

End of Tutorial about concept systems

What are the “Database Language” challenges?

Metadata Registries & Database Technologies – Which Does What?

Traditional Data Registries (11179 Edition 2) Register metadata which describes data—in databases,

applications, XML Schemas, data models, flat files, paper Assist in harmonizing, standardizing, and vetting metadata Assist data engineering Provide a source of well formed data designs for system designers Record reporting requirements Assist data generation, by describing the meaning of data entry

fields and the potential valid values Register provenance information that can be provided to end

users of data Assist with information discovery by pointing to systems where

particular data is maintained.

Data Elements

Algeria

Belgium

Denmark

France

Zimbabwe

ISO 3166French Name

L`Algérie

Belgique

Danemark

Egypte

La France

Zimbabwe

Traditional MDR:Manage Code Sets

Algeria

Belgium

Denmark

France

Zimbabwe

DataElementConcept

What Can XMDR Do?

Support a new generation of semantic computing Concept system management Harmonizing and vetting concept systems Linkage of concept systems to data Interrelation of multiple concept systems Grounding ontologies and RDF in agreed upon

semantics Reasoning across XMDR content (concept

systems and metadata) Provision of Semantic Services

We are trying to manage semantics in an increasingly complex content space

Structured dataSemi-structured dataUnstructured dataTextPictographicGraphicsMultimediaVoice video

Case Study

Combining Concept Systems, Data, and Metadata to answer queries.

Linking Concepts: Text Document

§ 141.62 Maximum contaminant levelsfor inorganic contaminants.(a) [Reserved](b) The maximum contaminant levelsfor inorganic contaminants specified inparagraphs (b) (2)–(6), (b)(10), and (b)(11)–(16) of this section apply to communitywater systems and non-transient,non-community water systems.The maximum contaminant level specifiedin paragraph (b)(1) of this sectiononly applies to community water systems.The maximum contaminant levelsspecified in (b)(7), (b)(8), and (b)(9)of this section apply to communitywater systems; non-transient, noncommunitywater systems; and transientnon-community water systems.Contaminant MCL (mg/l)(1) Fluoride ............................ 4.0(2) Asbestos .......................... 7 Million Fibers/liter (longerthan 10 μm).(3) Barium .............................. 2(4) Cadmium .......................... 0.005(5) Chromium ......................... 0.1(6) Mercury ............................ 0.002(7) Nitrate ............................... 10 (as Nitrogen)

§ 141.62 40 CFR Ch. I (7–1–02 Edition)

Title 40--Protection of Environment

CHAPTER I--ENVIRONMENTAL PROTECTION AGENCY PART 141--NATIONAL PRIMARY DRINKING WATER REGULATIONS

Thesaurus Concept System(From GEMET)

Chemical Contamination

Definition The addition or presence of chemicals to, or in, another substance to such a degree as to render it unfit for its intended purpose.

Broader Term contamination

Narrower Terms cadmium contamination, lead contamination, mercury contamination

Related Terms chemical pollutant, chemical pollution

Deutsch: Chemische Verunreinigung

English (US): chemical contamination

Español: contaminación química

SOURCE General Multi-Lingual Environmental Thesaurus (GEMET)

Concept System (Thesaurus)

Chemical

cadmium lead mercury

chemical pollutant

chemical pollution

Contamination

Name Acalypha ostryifolia

Mercury Mercury, bis(acetato-.kappa.O)(benzenamine)-

Mercury, (acetato-.kappa.O)phenyl-, mixt. with phenylmercuric propionate

Type Biological Organism

Chemical Chemical Chemical

CAS Number

7439-97-6 63549-47-3 No CAS Number

TSN 28189

EPA ID E17113275 E965269

Recent Additions | Contact Us

Environmental Data Registry

Chemicals in EPA Environmental Data Registry

Monitoring StationsName Latitude Longitude Location

A 41.45 N 125.99 W Merced Lake

B 43.23 N 120.50 WMerced

X 39.45 N 118.12 WFletcher

ID Date Temp Hg

A 2006-09-13 4.4 4

B 2006-09-13 9.3 2

X 2006-09-15 5.2 3

X 2006-09-13 6.7 78

Measurements

Merced Lake

Fletcher CreekMerced River

Metadata

System Data Element Definition Units Precision

Measurements ID Monitoring Station Identifier not applicable not applicable

Measurements Date Date sample was collected not applicable not applicable

Measurements Temp Temperature degrees Celcius 0.1

Measurements Hg Mercury contamination micrograms per liter 0.004

Monitoring Stations Name Monitoring Station Identifier

Monitoring Stations Latitude Latitude where sample was taken

Monitoring Stations LongitudeLongitude where sample was taken

Monitoring Stations Location Body of water monitored

Contaminants Contaminant Name of contaminant

Contaminants Threshold Acceptable threshold value

Metadata

ContaminantsContaminant Threshold

mercury 5

lead 42?

cadmium 250?

Relations among Inland Bodies of Water

Fletcher Creek

Merced Lake

Merced River

feeds into

feeds intoFletcher Creek Merced Lake

Merced River

fed from feeds into

Combining Data, Metadata & Concept Systems

ID Date Temp Hg

A 06-09-13 4.4 4

B 06-09-13 9.3 2

X 06-09-13 6.7 78

Name Datatype Definition Units

ID textMonitoring Station Identifier

not applicable

Date date Date yy-mm-dd

Temp numberTemperature (to 0.1 degree C)

degrees Celcius

Hg numberMercury contamination

micrograms per liter

Inference Search Query:“find water bodies downstream from Fletcher Creek where chemical contamination was over 2 parts per billion between December 2001 and March 2003”

Metadata

Contamination

lead cadmiummercury

Chemical

Concept system

Example – Environmental Text Corpus

Idea: Develop an environmental research corpus that could attract R&D efforts. Include the reports and other material from over $1b EPA sponsored research. Prepare the corpus and make it available

Research results from years of ORD R&D Publish associated metadata and concept

systems in XMDR Use open source software for EPA testing

Information Extraction & Semantic Computing

Segment

Classify

Associate

Normalize

Deduplicate

Discover patterns

Select models

Fit parameters

Inference

Report results

Actionable Information

Decision Support

ExtractionEngine

11179-3(E3)

Metadata Registries are Useful

Registered semantics For “training” extraction engines The“Normalize” function can make use of standard

code sets that have mapping between representation forms.

The “Classify” function can interact with pre-established concept systems.

Provenance High precision for proper nouns, less precision

(e.g., 70%) for other concepts -> impacts downstream processing, Need to track precision

Data Elements

Algeria

Belgium

Denmark

France

Zimbabwe

ISO 3166French Name

L`Algérie

Belgique

Danemark

Egypte

La France

Zimbabwe

Normalize – Need Registered and Mapped Concepts/Code Sets

Algeria

Belgium

Denmark

France

Zimbabwe

DataElementConcept

Challenge for Database Languages

The extraction database can contain graphs with > a billion nodes. Types of queries that can be done Query performance Linkage of “extract database” concepts and

relations to same concepts and relations in traditional databases.

Example – 11179-3 (E3) Support Semantic Web Applications

The address state code is “AB”. This can be expressed as a directedGraph e.g., an RDF statement:

Address

State Code

Subject

Predicate

Object

XMDR may be used to “ground” the Semantics of an RDF Statement.

Graph RDF

Example: Grounding RDF nodes and relations: URIs Reference a Metadata Registry

dbA:ma344

“AB”^^ai:StateCode

ai: StateUSPSCode

@prefix dbA: “http:/www.epa.gov/databaseA”@prefix ai: “http://www.epa.gov/edr/sw/AdministeredItem#”

dbA:e0139

ai: MailingAddress

Definitions in the EPA Environmental Data Registry

http://www.epa/gov/edr/sw/AdministeredItem#MailingAddress

The exact address where a mail piece is intended to be delivered, including urban-style address, rural route, and PO Box

http://www.epa/gov/edr/sw/AdministeredItem#StateUSPSCode

The U.S. Postal Service (USPS) abbreviation that represents a state or state equivalent for the U.S. or Canada

http://www.epa/gov/edr/sw/AdministeredItem#StateName

The name of the state where mail is delivered

Mailing Address:

State USPS Code:

Mailing Address State Name:

Ontologies for Data Mapping

Concept Concept

ConceptConceptGeographic Area

Geographic Sub-Area

Country

Country Identifier

Country Name Country Code

Short Name ISO 31662-Character

ISO 31663- Character

Long Name

DistributorCountry Name

Mailing AddressCountry Name ISO 3166

3-Numeric CodeFIPS Code

Ontologies can help to capture and express semantics

Example: Content Mapping Service

Collect data from many sources – files contain data that has the same facts represented by different terms. E.g., one system responds with Danemark, DK, another with DNK, another with 208; map all to Denmark.

XMDR could accept XML files with the data from different code sets and return a result mapped to a single code set.

Actions to Manage Enterprise Semantics

Define, data, concepts, and relationsHarmonize and vet data and concept

systemsGround semantics for RDF, concept

systems, ontologies Provide semantics services

Challenge: Concept System Store

Metadata Registry

Concept System Thesaurus Themes

DataStandards

Ontology GEMET

StructuredMetadata

UsersUsers

Concept systems:KeywordsControlled VocabulariesThesauriTaxonomiesOntologiesAxiomatized Ontologies

(Essentially graphs: node-relation-node + axioms)

Challenge: Management of Concept Systems

Metadata Registry

DataStandards

Ontology GEMET

StructuredMetadata

UsersUsers

Concept system:RegistrationHarmonization StandardizationAcceptance (vetting)Mapping (correspondences)

Challenge: Life Cycle Management

Metadata Registry

DataStandards

Ontology GEMET

StructuredMetadata

UsersUsers

Life cycle management:Data andConcept systems(ontologies)

Challenge: Grounding Semantics

Metadata Registry

DataStandards

Ontology GEMET

StructuredMetadata

UsersUsers

MetadataRegistries Semantic Web

RDF TriplesSubject (node URI)Verb (relation URI)Object (node URI)

Ontologies

Some Limitations of Relational Technologies & SQL

Limited graph computations Weak graph query language

Limited object computations Weak object query language

Inadequate linkage of metadata to data (underspecified “catalog”) CASE tools also disable, rather than enable

data administration & semantics management

Limitations (Cont.)

Limited linkage of concept system (graphs) to data (relational, graph, object)

Some Input From WG 2 and XMDR

Look at recent work on a graph query language by David Silberberg of Johns Hopkins University Applied Physics Lab.

Input from WG 2 and XMDR

David Jensen, of the University of Massachussetts Amherst ( http://kdl.cs.umass.edu/people/jensen/ ) has been developing a very interesting Proximity system and in the process has worked with complex patterns in very large data sets, including alternative query languages and database technologies. ( http://kdl.cs.umass.edu/proximity/index.html ). QGRAPH is a new visual language for querying and updating graph databases. A key feature of QGRAPH is that the user can draw a query consisting of vertices and edges with specified relations between their attributes. The response will be the collection of all subgraphs of the database that have the desired pattern.

Query languages are necessary to extract useful information from massive data sets. Moreover, annotated corpora require thousands of hours of manual annotation to create, revise and maintain. Query languages are also useful during this process. For example, queries can be used to find parse errors or to transform annotations into different schemes. However, they suffer from several problems. First, updates are not supported as query languages focus on the needs of linguists searching for syntactic

constructions. Second, their relationship to existing database query languages is poorly understood, making it difficult to

apply standard database indexing and query optimization techniques. As a consequence they do not scale well.

Finally, linguistic annotations have both a sequential and a hierarchical organization. Query languages must support queries that refer to both of these types of structure simultaneously. Such hybrid queries should have a concise syntax. The interplay between these factors has resulted in a variety of mutually-inconsistent approaches.

Catherine Lai and Steven Bird

Department of Computer Science and Software Engineering

University of Melbourne, Victoria 3010, Australia

Try to keep an eye on companies that are grappling with advanced database, knowledge management, information extraction, and analysis requirements, such as Metamatrix, I2, NetViz, Top Quadrant, OntologyWorks, Franz, Cogito, or Objectivity, with new ones cropping up very often.

Check out the EU sites given the large investments being made there in areas of interest. For example, KAON.

Watch the outcome of an NSF funded project on querying linguistic databases,including annotated corpora ( http://projects.ldc.upenn.edu/QLDB/ ). Steven Bird at U. Melbourne is one of the principals on that project.

Need for graph query languages that go beyond RDF and XML Frank Olken: Make SQL a strongly typed language with respect to

measurement dimensionality. Performance: project graph structured queries against graph

structured data. Express with great difficulty the query in SQL. Complex objects. Model gets complex. Putting humpty dumpty together again at query time.

Political problem in govt. Vendors on board, hard to pursue other technologies.

Object systems. OMG working on it? (OQL?). JAVA has ugly layer that maps into relational system. Franz has SPARQL built on top of a graph store.

Link Mining Applications: Progress and Challenges - Ted E. Senator

Link mining is a fairly new research area that lies at the intersection of link analysis, hypertext and web mining, relational learning and inductive logic programming, and graph mining. However, and perhaps more important, it also represents an important and essential set of techniques for constructing useful applications of data mining in a wide variety of real and important domains, especially those involving complex event detection from highly structured data. Imagine a complete “link mining

toolkit.” What would such a toolkit look like?

Most important, it would require a language that enabled the natural representation of entities and links. Such a language would also allow for the representation of pattern templates and for specifying matches between the templates and their instantiations.

The language would have to accept an arbitrary database schema as input, with a specified mapping between relations in the database and fundamental link types in the language.

It would have to compile into efficient and rapidly executable database queries.

It would need to be able to represent grouped entities and multiple abstraction hierarchies and reason at all levels.

It would have to enable the creation of new schema elements in the database to represent newly discovered concepts.

It would need to represent both pattern templates and pattern instances, and to have a mechanism for tracking matches between the two.

It would have to have constructs for representing fundamental relationships such as part-of, is-a, and connected-to (the most generic link relationship), as well as perhaps other high-level link types such as temporal relationships (e.g., before, after, during, overlapping, etc.), geo-spatial relationships, organizational relationships, trust relationships, and activities and events.

The toolkit would include at least one and possibly many pattern matchers. It would require tools for creating and editing patterns. It would have to include visualizations for many different types of structured data.

It would need mechanisms for handling uncertainty and confidence. It would have to track the dependence of any conclusion (e.g., pattern match or

discovered pattern) back to the underlying data, and perhaps incorporate backtracking so the impact of data corrections could be detected.

It would need configuration management tools to track the history of discovered and matched patterns.

It would need workflow mechanisms to support multiple users in an organizational structure.

It would need mechanisms for ingesting domain-specific knowledge. It would have to be able to deal with multiple data types including text and

imagery. And it would have to be able to rapidly incorporate new link mining techniques

as they are developed. Finally, it would need to include mechanisms for maximum privacy protection.

Where to Progress Semantics Management?

SC 32 in WG 2 and WG 3 as extensions to ongoing work or as New Work Items

W3C as XQuery, SPARQL, Semantic Web Deployment WG (RDF vocabularies, SKOS)

OMG as extensions to the MOF…

Thanks & Acknowledgements

John McCarthy Karlo Berket Kevin Keck Frank Olken Harold Solbrig

L8 and SC 32/WG 2 Standards Committees Major XMDR Project Sponsors and Collaborators

U.S. Environmental Protection Agency Department of Defense National Cancer Institute U.S. Geological Survey Mayo Clinic Apelon

1 Future Database Needs SC 32 Study Period February 5, 2007 Bruce Bargmeyer, Lawrence Berkley...

Documents

Transcript of 1 Future Database Needs SC 32 Study Period February 5, 2007 Bruce Bargmeyer, Lawrence Berkley...

2905 Dunhill III with price

Berkley Supplement

4227 Bacon Ave, Berkley, MI | Beautiful Berkley Bungalow

2905 Good Luck Lane Annapolis, MD 21401

2905 BetadinePREMIO2012

Berkley Joined

SAS Starter Berkley

The Berkley Center.Ppt

Berkley House Apartments

W. R. Berkley Corporation

BERKLEY PUBLISHING GROUP - Bookseller Servicesbooksellers.penguin.com/static/pdf/berkley-mm-jan15.pdf · berkley berkley sensation prime crime jove boulevard ace berkley publishing

BERKLEY - Catalogo 2010 USA

2011 Berkley Line

15 Berkley Hall Court

I berkley

Berkley B3ZPL Pump

Cooperating Registries Draft Content for OASIS/ebXML Reg/Rep f2f November 1, 2001 Bruce Bargmeyer (510) 495-2905 bebargmeyer@lbl.gov.

2905 Edgar Expansion Joints With Caesar II

Acupuncture Herbs Fertility From ... - The Berkley Center · •All herbs prescribed at The Berkley Center for Reproductive Wellness are done so by Mike Berkley, a New York State

W. R. Berkley Corporation/media/Files/W/WR-Berkley/documents... · W. R. Berkley Corporation ... 7 I Employment and the Workplace ... ` ignoring complaints or requests that a certain