1

Future Database NeedsSC 32 Study Period

February 5, 2007

Bruce Bargmeyer, Lawrence Berkley National LaboratoryUniversity of CaliforniaTel: +1 510-495-2905bebargmeyer@lbl.gov

JTC1 SC32N1633

2

Topics

Study period purposeNew challengesA brief tutorial on Semantics and semantic

computingwhere XMDR fits

Semantic computing technologies Traditional Data Administration

Some limitations of current relational technologies Some input from other sources

Future Database NeedsStudy Period

A one-year study period to identify and understand case studies related to this area.

Bring together a small group of experts in a meeting on “Case Studies on new Database Standards Requirements”.  

The workshop would provide input to existing SC32 projects and may provide background material for new proposals for upgrades or for new work within SC32 in time for 2007 SC32 Plenary

--Document 32N1451

3

4

The Internet Revolution

A world wide web of diverse content: The information glut is nothing new. The access to it is astonishing.

5

Challenge: Find and process non-explicit data

Analgesic Agent

Non-Narcotic Analgesic

AcetominophenNonsteroidal Antiinflammatory Drug

Analgesic and Antipyretic

DatrilAnacin-3 Tylenol

For example…

Patient data on drugs contains brand names (e.g. Tylenol, Anacin-3, Datril,…);

However, want to study patients taking analgesic agents

6

Challenge: Specify and compute across Relations, e.g., within a food web in an

Arctic ecosystem

                                        An organism is connected to another organism for which it is a source of food energy and material by an arrow representing the direction of biomass transfer.

Source: http://en.wikipedia.org/wiki/Food_web#Food_web (from SPIRE)

7

Challenge: Combine Data, Metadata & Concept Systems

ID Date Temp Hg

A 06-09-13 4.4 4

B 06-09-13 9.3 2

X 06-09-13 6.7 78

Name Datatype Definition Units

ID textMonitoring Station Identifier

not applicable

Date date Date yy-mm-dd

Temp numberTemperature (to 0.1 degree C)

degrees Celcius

Hg numberMercury contamination

micrograms per liter

Inference Search Query:“find water bodies downstream from Fletcher Creek where chemical contamination was over 10 micrograms per liter between December 2001 and March 2003”

Data:

Metadata:

Biological Radioactive

Contamination

lead cadmiummercury

Chemical

Concept system:

9

Challenge: Use data from systems that record the same facts with different terms

Common Content

OASIS/ebXMLRegistries

Common Content

ISO 11179Registries

Common Content

OntologicalRegistries

Common Content

CASE ToolRepositories

Common Content

UDDIRegistries

CountryIdentifier

DataElement

XML Tag

TermHierarchy

Attribute

BusinessSpecification

TableColumn

SoftwareComponentRegistries

Common Content

Common Content

DatabaseCatalogs

BusinessObject

DublinCore

Registries

Common Content

Coverage

10

Data Elements

DZ

BE

CN

DK

EG

FR

. . .

ZW

ISO 3166English Name

ISO 31663-Numeric Code

012

056

156

208

818

250

. . .

716

ISO 31662-Alpha Code

Algeria

Belgium

China

Denmark

Egypt

France

. . .

Zimbabwe

Name:Context:Definition:Unique ID: 4572Value Domain:Maintenance Org.Steward:Classification:Registration Authority:Others

ISO 3166French Name

L`Algérie

Belgique

Chine

Danemark

Egypte

La France

. . .

Zimbabwe

DZA

BEL

CHN

DNK

EGY

FRA

. . .

ZWE

ISO 31663-Alpha Code

Same Fact, Different Terms

Algeria

Belgium

China

Denmark

Egypt

France

. . .

Zimbabwe

Name: Country IdentifiersContext:Definition:Unique ID: 5769Conceptual Domain:Maintenance Org.:Steward:Classification:Registration Authority:Others

DataElementConcept

11

Challenge: Draw information together from a broad range of studies, databases, reports, etc.

12

Challenge: Gain Common Understanding of meaning between Data Creators and Data Users

Users Information systems

Data Creation

UsersUsers

EEA

USGS

DoD

EPAenvironagricultureclimatehuman healthindustrytourismsoilwaterair

123345445670248591308

123345445670248591308

3268082513485038270800002178

3268082513485038270800002178

text data

environagricultureclimatehuman healthindustrytourismsoilwaterair

123345445670248591308

123345445670248591308

3268082513485038270800002178

3268082513485038270800002178

text

ambienteagriculturatiemposalud hunanoindustriaturismotierraaguaaero

123345445670248591308

123345445670248591308

3268082513485038270800002178

3268082513485038270800002178

text data

data

environagricultureclimatehuman healthindustrytourismsoilwaterair

123345445670248591308

123345445670248591308

3268082513485038270800002178

3268082513485038270800002178

text data

Others . . .

ambienteagriculturatiemposalud hunoindustriaturismotierraaguaaero

123345445670248591308

123345445670248591308

3268082513485038

3268082513485038270800002178

text data

A common interpretation of what the data represents

13

Challenge: Drawing Together Dispersed Data

Users Information systems

Data Creation

UsersUsers

EEA

USGS

DoD

EPAenvironagricultureclimatehuman healthindustrytourismsoilwaterair

123345445670248591308

123345445670248591308

3268082513485038270800002178

3268082513485038270800002178

text data

environagricultureclimatehuman healthindustrytourismsoilwaterair

123345445670248591308

123345445670248591308

3268082513485038270800002178

3268082513485038270800002178

text

ambienteagriculturatiemposalud hunanoindustriaturismotierraaguaaero

123345445670248591308

123345445670248591308

3268082513485038270800002178

3268082513485038270800002178

text data

data

environagricultureclimatehuman healthindustrytourismsoilwaterair

123345445670248591308

123345445670248591308

3268082513485038270800002178

3268082513485038270800002178

text data

Others . . .

ambienteagriculturatiemposalud hunoindustriaturismotierraaguaaero

123345445670248591308

123345445670248591308

3268082513485038

3268082513485038270800002178

text data

A common interpretation of what the data represents

14

Semantic Computing

We are laying the foundation to make a quantum leap toward a substantially new way of computing: Semantic Computing

How can we make use of semantic computing? What do organizations need to do to prepare for

and stimulate semantic computing?

15

Coming: A Semantic Revolution

Searching and rankingPattern analysisKnowledge discoveryQuestion answeringReasoningSemi-automated decision making

16

The Nub of It

Processing that takes “meaning” into account Processing based on the relations between things

not just computing about the things themselves.Computing that takes people out of the

processing, reducing the human toil Data access, extraction, mapping, translation,

formatting, validation, inferencing, …Delivering higher-level results that are more

helpful for the user’s thought and action

17

Semantics Challenges

Managing, harmonizing, and vetting semantics is essential to enable enterprise semantic computing

Managing, harmonizing and vetting semantics is important for traditional data management. In the past we just covered the basics

Enabling “community intelligence” through efforts similar to Wikipedia, Wikitionary, Flickr

18

A Brief Tutorial on Semantics

What is meaning?What are concepts?What are relations?What are concept systems?What is “reasoning”?

19C.K Ogden and I. A. Richards. The Meaning of Meaning.

Thought or Reference (Concept)

Referent Symbol

SymbolisesRefers to

Stands for“Rose”, “ClipArt”

Meaning: The Semiotic Triangle

20

Semiotic Triangle:Concepts, Definitions and Signs

CONCEPT

Referent

Refers To Symbolizes

Stands For

“Rose”,“ClipArt”

Definition

Sign

24

Definitions in the EPA Environmental Data Registry

http://www.epa/gov/edr/sw/AdministeredItem#MailingAddress

The exact address where a mail piece is intended to be delivered, including urban-style address, rural route, and PO Box

http://www.epa/gov/edr/sw/AdministeredItem#StateUSPSCode

The U.S. Postal Service (USPS) abbreviation that represents a state or state equivalent for the U.S. or Canada

http://www.epa/gov/edr/sw/AdministeredItem#StateName

The name of the state where mail is delivered

Mailing Address:

State USPS Code:

Mailing Address State Name:

26

SNOMED – Terms Defined by Relations

30

Computable Meaning

CONCEPT

Referent

Refers To Symbolizes

Stands For

“Rose”,“ClipArt”

rdfs:subClassOf owl:equivalentClass owl:disjointWith

If “rose” is owl:disjointWith “daffodil”, then a computer can determine that anassertion is invalid, if it states that a rose is also a daffodil (e.g., in a knowledgebase).

31

Fletcher CreekMerced

Lake

WaterBody

What are Relations?

Relation

Merced Lake

Fletcher CreekMerced River

isA isA

Concepts and relations can be represented as nodes and edges in formal graph structures, e.g., “is-a” hierarchies.

32

A

2

b a c d

1

Nodes represent concepts

Lines (arcs) represent relations

Concept Systems have Nodes and may have Relations

Concept systems are concepts and the relations between them.Concept systems can be represented & queried as graphs

33

A More Complex Concept Graph

From Supervaluation Semantics for an Inland Water Feature OntologyPaulo Santos and Brandon Bennett http://ijcai.org/papers/1187.pdf#search=%22terminology%20water%20ontology%22

Concept lattice of inland water features

Linear LargeNon-linear

Non-linear

Large linear Small linear Small non- linear

Deep Natural

Artificial

River Stream Canal Reservoir Lake Marsh Pond

Flowing Shallow Stagnant

35

Tree

Partial Order Tree

Ordered Tree

Faceted ClassificationDirected Acyclic GraphPartial Order Graph

Powerset of 3 element setBipartite Graph Clique

Compound Graph

Types of Concept System Graph Structures

36

Graph Taxonomy

Directed Graph

Directed Acyclic Graph

Graph

Undirected Graph

Bipartite Graph

Partial Order Graph

Faceted Classification

Clique

Partial Order Tree

Tree

Lattice

Ordered Tree

Note: not all bipartite graphsare undirected.

37

What Kind of Relations are There?Lots!

Relationship class: A particular type of connection existing between people related to or having dealings with each other.

acquaintanceOf - A person having more than slight or superficial knowledge of this person but short of friendship.

ambivalentOf - A person towards whom this person has mixed feelings or emotions.

ancestorOf - A person who is a descendant of this person. antagonistOf - A person who opposes and contends against this person. apprenticeTo - A person to whom this person serves as a trusted counselor or

teacher. childOf - A person who was given birth to or nurtured and raised by this person. closeFriendOf - A person who shares a close mutual friendship with this person. collaboratesWith - A person who works towards a common goal with this person.

…

38

Example of relations in a food web in an Arctic ecosystem

                                        An organism is connected to another organism for which it is a source of food energy and material by an arrow representing the direction of biomass transfer.

Source: http://en.wikipedia.org/wiki/Food_web#Food_web (from SPIRE)

39

Ontologies are a type of Concept System

Ontology: explicit formal specifications of the terms in the domain and relations among them (Gruber 1993)

An ontology defines a common vocabulary for researchers who need to share information in a domain. It includes machine-interpretable definitions of basic concepts in the domain and relations among them.

Why would someone want to develop an ontology? Some of the reasons are: To share common understanding of the structure of information

among people or software agents To enable reuse of domain knowledge To make domain assumptions explicit To separate domain knowledge from the operational knowledge To analyze domain knowledge

http://www.ksl.stanford.edu/people/dlm/papers/ontology101/ontology101-noy-mcguinness.html

40

What is Reasoning?Inference

Polio Smallpox

Infectious Disease

Disease

is-a

is-a is-a

is-a

is-a

Diabetes Heart disease

Chronic Disease

is-a

Signifies inferred is-a relationship

41

Reasoning: Taxonomies & partonomies can be used to support inference queries

Oakland Berkeley

Alameda County

California

part-of

part-of part-of

part-of

part-of

Santa Clara San Jose

Santa Clara County

part-of

E.g., if a database containsinformation on events by city,we could query that database for events that happened in a particular county or state,even though the event data does not contain explicit state or county codes.

42

Reasoning: Relationship metadata can be used to infer non-explicit data

Analgesic Agent

Non-Narcotic Analgesic

AcetominophenNonsteroidal Antiinflammatory Drug

Analgesic and Antipyretic

DatrilAnacin-3 Tylenol

For example…(1) patient data on drugs currently

being taken contains brand names (e.g. Tylenol, Anacin-3, Datril,…);

(2) concept system connects different drug types and names with one another (via is-a, part-of, etc. relationships);

(3) so… patient data can be linked and searched by inferred terms like “acetominophen” and “analgesic” as well as trade names explicitly stored as text strings in the database

43

Reasoning: Least Common Ancestor Query

Analgesic and Antipyretic

Analgesic Agent

Non-Narcotic Analgesic

Acetominophen

Opioid

Opiate

Morphine Sulfate

Codeine Phosphate

Nonsteroidal Antiinflammatory Drug

What is the least common ancestor concept in the NCI Thesaurus for Acetominophen and Morphine Sulfate? (answer = Analgesic Agent)

44

Reasoning: Example “sibling” queries: concepts that share a common ancestor

Environmental:"siblings" of Wetland (in NASA SWEET ontology)

HealthSiblings of ERK1 finds all 700+ other kinase enzymesSiblings of Novastatin finds all other statins

11179 MetadataSibling values in an enumerated value

domain

45

HealthFind all the siblings of

Breast Neoplasm

EnvironmentalFind all chemicals that are acarcinogen (cause cancer) andtoxin (are poisonous) andterratogenic (cause birth

defects)

Reasoning: More complex “sibling” queries: concepts with multiple ancestors

site neoplasms breast disorders

Breast neoplasm

RespiratorySystem

neoplasm

Non-Neoplastic

Breast Disorder

Eye neoplasm

46

End of Tutorial about concept systems

What are the “Database Language” challenges?

49

Metadata Registries & Database Technologies – Which Does What?

Traditional Data Registries (11179 Edition 2) Register metadata which describes data—in databases,

applications, XML Schemas, data models, flat files, paper Assist in harmonizing, standardizing, and vetting metadata Assist data engineering Provide a source of well formed data designs for system designers Record reporting requirements Assist data generation, by describing the meaning of data entry

fields and the potential valid values Register provenance information that can be provided to end

users of data Assist with information discovery by pointing to systems where

particular data is maintained.

50

Data Elements

DZ

BE

CN

DK

EG

FR

. . .

ZW

ISO 3166English Name

ISO 31663-Numeric Code

012

056

156

208

818

250

. . .

716

ISO 31662-Alpha Code

Algeria

Belgium

China

Denmark

Egypt

France

. . .

Zimbabwe

Name:Context:Definition:Unique ID: 4572Value Domain:Maintenance Org.Steward:Classification:Registration Authority:Others

ISO 3166French Name

L`Algérie

Belgique

Chine

Danemark

Egypte

La France

. . .

Zimbabwe

DZA

BEL

CHN

DNK

EGY

FRA

. . .

ZWE

ISO 31663-Alpha Code

Traditional MDR:Manage Code Sets

Algeria

Belgium

China

Denmark

Egypt

France

. . .

Zimbabwe

Name: Country IdentifiersContext:Definition:Unique ID: 5769Conceptual Domain:Maintenance Org.:Steward:Classification:Registration Authority:Others

DataElementConcept

51

What Can XMDR Do?

Support a new generation of semantic computing Concept system management Harmonizing and vetting concept systems Linkage of concept systems to data Interrelation of multiple concept systems Grounding ontologies and RDF in agreed upon

semantics Reasoning across XMDR content (concept

systems and metadata) Provision of Semantic Services

52

We are trying to manage semantics in an increasingly complex content space

Structured dataSemi-structured dataUnstructured dataTextPictographicGraphicsMultimediaVoice video

53

Case Study

Combining Concept Systems, Data, and Metadata to answer queries.

54

Linking Concepts: Text Document

§ 141.62 Maximum contaminant levelsfor inorganic contaminants.(a) [Reserved](b) The maximum contaminant levelsfor inorganic contaminants specified inparagraphs (b) (2)–(6), (b)(10), and (b)(11)–(16) of this section apply to communitywater systems and non-transient,non-community water systems.The maximum contaminant level specifiedin paragraph (b)(1) of this sectiononly applies to community water systems.The maximum contaminant levelsspecified in (b)(7), (b)(8), and (b)(9)of this section apply to communitywater systems; non-transient, noncommunitywater systems; and transientnon-community water systems.Contaminant MCL (mg/l)(1) Fluoride ............................ 4.0(2) Asbestos .......................... 7 Million Fibers/liter (longerthan 10 μm).(3) Barium .............................. 2(4) Cadmium .......................... 0.005(5) Chromium ......................... 0.1(6) Mercury ............................ 0.002(7) Nitrate ............................... 10 (as Nitrogen)

§ 141.62 40 CFR Ch. I (7–1–02 Edition)

Title 40--Protection of Environment

CHAPTER I--ENVIRONMENTAL PROTECTION AGENCY PART 141--NATIONAL PRIMARY DRINKING WATER REGULATIONS

55

Thesaurus Concept System(From GEMET)

Chemical Contamination

Definition The addition or presence of chemicals to, or in, another substance to such a degree as to render it unfit for its intended purpose.

Broader Term contamination

Narrower Terms cadmium contamination, lead contamination, mercury contamination

Related Terms chemical pollutant, chemical pollution

Deutsch: Chemische Verunreinigung

English (US): chemical contamination

Español: contaminación química

SOURCE General Multi-Lingual Environmental Thesaurus (GEMET)

56

Concept System (Thesaurus)

Chemical

cadmium lead mercury

Biological Radioactive

chemical pollutant

chemical pollution

Contamination

57

Name Acalypha ostryifolia

Mercury Mercury, bis(acetato-.kappa.O)(benzenamine)-

Mercury, (acetato-.kappa.O)phenyl-, mixt. with phenylmercuric propionate

Type Biological Organism

Chemical Chemical Chemical

CAS Number

7439-97-6 63549-47-3 No CAS Number

TSN 28189

ICTV

EPA ID E17113275 E965269

Recent Additions | Contact Us

Environmental Data Registry

Chemicals in EPA Environmental Data Registry

58

Data

Monitoring StationsName Latitude Longitude Location

A 41.45 N 125.99 W Merced Lake

B 43.23 N 120.50 WMerced

River

X 39.45 N 118.12 WFletcher

Creek

ID Date Temp Hg

A 2006-09-13 4.4 4

B 2006-09-13 9.3 2

X 2006-09-15 5.2 3

X 2006-09-13 6.7 78

Measurements

A

BX

Merced Lake

Fletcher CreekMerced River

59

Metadata

System Data Element Definition Units Precision

Measurements ID Monitoring Station Identifier not applicable not applicable

Measurements Date Date sample was collected not applicable not applicable

Measurements Temp Temperature degrees Celcius 0.1

Measurements Hg Mercury contamination micrograms per liter 0.004

Monitoring Stations Name Monitoring Station Identifier

Monitoring Stations Latitude Latitude where sample was taken

Monitoring Stations LongitudeLongitude where sample was taken

Monitoring Stations Location Body of water monitored

Contaminants Contaminant Name of contaminant

Contaminants Threshold Acceptable threshold value

Metadata

ContaminantsContaminant Threshold

mercury 5

lead 42?

cadmium 250?

60

Relations among Inland Bodies of Water

Fletcher Creek

Merced Lake

Merced River

feeds into

feeds intoFletcher Creek Merced Lake

Merced River

fed from feeds into

61

Combining Data, Metadata & Concept Systems

ID Date Temp Hg

A 06-09-13 4.4 4

B 06-09-13 9.3 2

X 06-09-13 6.7 78

Name Datatype Definition Units

ID textMonitoring Station Identifier

not applicable

Date date Date yy-mm-dd

Temp numberTemperature (to 0.1 degree C)

degrees Celcius

Hg numberMercury contamination

micrograms per liter

Inference Search Query:“find water bodies downstream from Fletcher Creek where chemical contamination was over 2 parts per billion between December 2001 and March 2003”

Data

Metadata

Biological Radioactive

Contamination

lead cadmiummercury

Chemical

Concept system

62

Example – Environmental Text Corpus

Idea: Develop an environmental research corpus that could attract R&D efforts. Include the reports and other material from over $1b EPA sponsored research. Prepare the corpus and make it available

Research results from years of ORD R&D Publish associated metadata and concept

systems in XMDR Use open source software for EPA testing

63

Information Extraction & Semantic Computing

Segment

Classify

Associate

Normalize

Deduplicate

Discover patterns

Select models

Fit parameters

Inference

Report results

Actionable Information

Decision Support

ExtractionEngine

11179-3(E3)

XMDR

65

Metadata Registries are Useful

Registered semantics For “training” extraction engines The“Normalize” function can make use of standard

code sets that have mapping between representation forms.

The “Classify” function can interact with pre-established concept systems.

Provenance High precision for proper nouns, less precision

(e.g., 70%) for other concepts -> impacts downstream processing, Need to track precision

66

Data Elements

DZ

BE

CN

DK

EG

FR

. . .

ZW

ISO 3166English Name

ISO 31663-Numeric Code

012

056

156

208

818

250

. . .

716

ISO 31662-Alpha Code

Algeria

Belgium

China

Denmark

Egypt

France

. . .

Zimbabwe

Name:Context:Definition:Unique ID: 4572Value Domain:Maintenance Org.Steward:Classification:Registration Authority:Others

ISO 3166French Name

L`Algérie

Belgique

Chine

Danemark

Egypte

La France

. . .

Zimbabwe

DZA

BEL

CHN

DNK

EGY

FRA

. . .

ZWE

ISO 31663-Alpha Code

Normalize – Need Registered and Mapped Concepts/Code Sets

Algeria

Belgium

China

Denmark

Egypt

France

. . .

Zimbabwe

Name: Country IdentifiersContext:Definition:Unique ID: 5769Conceptual Domain:Maintenance Org.:Steward:Classification:Registration Authority:Others

DataElementConcept

Challenge for Database Languages

The extraction database can contain graphs with > a billion nodes. Types of queries that can be done Query performance Linkage of “extract database” concepts and

relations to same concepts and relations in traditional databases.

67

68

Example – 11179-3 (E3) Support Semantic Web Applications

The address state code is “AB”. This can be expressed as a directedGraph e.g., an RDF statement:

Address

AB

State Code

Node

Node

Edge

Subject

Predicate

Object

XMDR may be used to “ground” the Semantics of an RDF Statement.

Graph RDF

69

Example: Grounding RDF nodes and relations: URIs Reference a Metadata Registry

dbA:ma344

“AB”^^ai:StateCode

ai: StateUSPSCode

@prefix dbA: “http:/www.epa.gov/databaseA”@prefix ai: “http://www.epa.gov/edr/sw/AdministeredItem#”

dbA:e0139

ai: MailingAddress

70

Definitions in the EPA Environmental Data Registry

http://www.epa/gov/edr/sw/AdministeredItem#MailingAddress

The exact address where a mail piece is intended to be delivered, including urban-style address, rural route, and PO Box

http://www.epa/gov/edr/sw/AdministeredItem#StateUSPSCode

The U.S. Postal Service (USPS) abbreviation that represents a state or state equivalent for the U.S. or Canada

http://www.epa/gov/edr/sw/AdministeredItem#StateName

The name of the state where mail is delivered

Mailing Address:

State USPS Code:

Mailing Address State Name:

72

Ontologies for Data Mapping

Concept Concept

ConceptConceptGeographic Area

Geographic Sub-Area

Country

Country Identifier

Country Name Country Code

Short Name ISO 31662-Character

Code

ISO 31663- Character

Code

Long Name

DistributorCountry Name

Mailing AddressCountry Name ISO 3166

3-Numeric CodeFIPS Code

Ontologies can help to capture and express semantics

73

Example: Content Mapping Service

Collect data from many sources – files contain data that has the same facts represented by different terms. E.g., one system responds with Danemark, DK, another with DNK, another with 208; map all to Denmark.

XMDR could accept XML files with the data from different code sets and return a result mapped to a single code set.

74

Actions to Manage Enterprise Semantics

Define, data, concepts, and relationsHarmonize and vet data and concept

systemsGround semantics for RDF, concept

systems, ontologies Provide semantics services

75

Challenge: Concept System Store

Metadata Registry

Concept System Thesaurus Themes

DataStandards

Ontology GEMET

StructuredMetadata

UsersUsers

Concept systems:KeywordsControlled VocabulariesThesauriTaxonomiesOntologiesAxiomatized Ontologies

(Essentially graphs: node-relation-node + axioms)

}

76

Challenge: Management of Concept Systems

Metadata Registry

Concept System Thesaurus Themes

DataStandards

Ontology GEMET

StructuredMetadata

UsersUsers

Concept system:RegistrationHarmonization StandardizationAcceptance (vetting)Mapping (correspondences)

}

77

Challenge: Life Cycle Management

Metadata Registry

Concept System Thesaurus Themes

DataStandards

Ontology GEMET

StructuredMetadata

UsersUsers

Life cycle management:Data andConcept systems(ontologies)

}

78

Challenge: Grounding Semantics

Metadata Registry

Concept System Thesaurus Themes

DataStandards

Ontology GEMET

StructuredMetadata

UsersUsers

MetadataRegistries Semantic Web

RDF TriplesSubject (node URI)Verb (relation URI)Object (node URI)

Ontologies

Some Limitations of Relational Technologies & SQL

Limited graph computations Weak graph query language

Limited object computations Weak object query language

Inadequate linkage of metadata to data (underspecified “catalog”) CASE tools also disable, rather than enable

data administration & semantics management

79

Limitations (Cont.)

Limited linkage of concept system (graphs) to data (relational, graph, object)

80

Some Input From WG 2 and XMDR

Look at recent work on a graph query language by David Silberberg of Johns Hopkins University Applied Physics Lab.

81

Input from WG 2 and XMDR

David Jensen, of the University of Massachussetts Amherst ( http://kdl.cs.umass.edu/people/jensen/ ) has been developing a very interesting Proximity system and in the process has worked with complex patterns in very large data sets, including alternative query languages and database technologies. ( http://kdl.cs.umass.edu/proximity/index.html ). QGRAPH is a new visual language for querying and updating graph databases. A key feature of QGRAPH is that the user can draw a query consisting of vertices and edges with specified relations between their attributes. The response will be the collection of all subgraphs of the database that have the desired pattern.

82

Input from WG 2 and XMDR

Query languages are necessary to extract useful information from massive data sets. Moreover, annotated corpora require thousands of hours of manual annotation to create, revise and maintain. Query languages are also useful during this process. For example, queries can be used to find parse errors or to transform annotations into different schemes. However, they suffer from several problems. First, updates are not supported as query languages focus on the needs of linguists searching for syntactic

constructions. Second, their relationship to existing database query languages is poorly understood, making it difficult to

apply standard database indexing and query optimization techniques. As a consequence they do not scale well.

Finally, linguistic annotations have both a sequential and a hierarchical organization. Query languages must support queries that refer to both of these types of structure simultaneously. Such hybrid queries should have a concise syntax. The interplay between these factors has resulted in a variety of mutually-inconsistent approaches.

Catherine Lai and Steven Bird

Department of Computer Science and Software Engineering

University of Melbourne, Victoria 3010, Australia

83

Input from WG 2 and XMDR

Try to keep an eye on companies that are grappling with advanced database, knowledge management,  information extraction, and analysis requirements, such as Metamatrix, I2, NetViz, Top Quadrant, OntologyWorks, Franz, Cogito, or Objectivity, with new ones cropping up very often.    

Check out the EU sites given the large investments being made there in areas of interest. For example, KAON. 

Watch the outcome of an NSF funded project on querying linguistic databases,including annotated corpora ( http://projects.ldc.upenn.edu/QLDB/ ). Steven Bird at U. Melbourne is one of the principals on that project.

84

Input from WG 2 and XMDR

Need for graph query languages that go beyond RDF and XML Frank Olken: Make SQL a strongly typed language with respect to

measurement dimensionality. Performance: project graph structured queries against graph

structured data. Express with great difficulty the query in SQL. Complex objects. Model gets complex.  Putting humpty dumpty together again at query time.

Political problem in govt. Vendors on board, hard to pursue other technologies.

Object systems. OMG working on it? (OQL?). JAVA has ugly layer that maps into relational system. Franz has SPARQL built on top of a graph store.

85

Input from WG 2 and XMDR

Link Mining Applications: Progress and Challenges - Ted E. Senator

Link mining is a fairly new research area that lies at the intersection of link analysis, hypertext and web mining, relational learning and inductive logic programming, and graph mining. However, and perhaps more important, it also represents an important and essential set of techniques for constructing useful applications of data mining in a wide variety of real and important domains, especially those involving complex event detection from highly structured data. Imagine a complete “link mining

toolkit.” What would such a toolkit look like?

86

Input from WG 2 and XMDR

Link Mining Applications: Progress and Challenges - Ted E. Senator

Most important, it would require a language that enabled the natural representation of entities and links. Such a language would also allow for the representation of pattern templates and for specifying matches between the templates and their instantiations.

The language would have to accept an arbitrary database schema as input, with a specified mapping between relations in the database and fundamental link types in the language.

It would have to compile into efficient and rapidly executable database queries.

It would need to be able to represent grouped entities and multiple abstraction hierarchies and reason at all levels.

It would have to enable the creation of new schema elements in the database to represent newly discovered concepts.

87

Input from WG 2 and XMDR

Link Mining Applications: Progress and Challenges - Ted E. Senator

It would need to represent both pattern templates and pattern instances, and to have a mechanism for tracking matches between the two.

It would have to have constructs for representing fundamental relationships such as part-of, is-a, and connected-to (the most generic link relationship), as well as perhaps other high-level link types such as temporal relationships (e.g., before, after, during, overlapping, etc.), geo-spatial relationships, organizational relationships, trust relationships, and activities and events.

The toolkit would include at least one and possibly many pattern matchers. It would require tools for creating and editing patterns. It would have to include visualizations for many different types of structured data.

It would need mechanisms for handling uncertainty and confidence. It would have to track the dependence of any conclusion (e.g., pattern match or

discovered pattern) back to the underlying data, and perhaps incorporate backtracking so the impact of data corrections could be detected.

88

Input from WG 2 and XMDR

Link Mining Applications: Progress and Challenges - Ted E. Senator

It would need configuration management tools to track the history of discovered and matched patterns.

It would need workflow mechanisms to support multiple users in an organizational structure.

It would need mechanisms for ingesting domain-specific knowledge. It would have to be able to deal with multiple data types including text and

imagery. And it would have to be able to rapidly incorporate new link mining techniques

as they are developed. Finally, it would need to include mechanisms for maximum privacy protection.

89

Where to Progress Semantics Management?

SC 32 in WG 2 and WG 3 as extensions to ongoing work or as New Work Items

W3C as XQuery, SPARQL, Semantic Web Deployment WG (RDF vocabularies, SKOS)

OMG as extensions to the MOF…

90

91

Thanks & Acknowledgements

John McCarthy Karlo Berket Kevin Keck Frank Olken Harold Solbrig

L8 and SC 32/WG 2 Standards Committees Major XMDR Project Sponsors and Collaborators

U.S. Environmental Protection Agency Department of Defense National Cancer Institute U.S. Geological Survey Mayo Clinic Apelon