Presentation AgendaPresentation Agenda

- SAIC Introduction- Stanford (KSL)- SRI International - Stanford (Formal Reasoning Group)- NWU- MIT- CMU- TextWise- SAIC Summary

SAIC Integrated Knowledge SAIC Integrated Knowledge Environment (SIKE) Environment (SIKE)

ArchitectureArchitecture

Architecture exists at two levels - System Level Architecture

Transport Layer Syntactic Layer

Knowledge Architecture Semantic Layer

HPKB Integrated Knowledge HPKB Integrated Knowledge Environment (HIKE) Environment (HIKE)

ArchitectureArchitecture

Architecture exists at two levels - System Level Architecture

Transport Layer Syntactic Layer

Knowledge Architecture Semantic Layer

System Level ArchitectureSystem Level ArchitectureFeaturesFeatures

A distributed heterogeneous environment to solve Crisis Management Challenge Problem.

Federation of OKBC(Open Knowledge Base Connectivity) servers

Added power of component-based approach for the distribution of knowledge content

Web based graphical user interface

STARTSTART

Analyst

OntolinguaOntolingua

HIKEGUI

HIKEGUI

Ocelot&

PERK

Ocelot&

PERK

ATPATP

SNARKSNARK

SMEMAC/FAC

SMEMAC/FAC

GKB Editor

GKB Editor

TextWiseTextWise

WebKBWebKB

JOTJOT

ATPLATPL

Crisis Management -Crisis Management -Knowledge Level ArchitectureKnowledge Level Architecture

Knowledge Architecture design is an output of the Knowledge Architecture working group convened by SAIC

Includes the SAIC merged ontology The SAIC merged ontology contains the year 1

knowledge bases from KSL, NWU, FRG, SRI, SAIC, and CMU

Ontology merging effort led by Stanford KSL led to development of the KB merging tool

SAIC CM CP KnowledgeSAIC CM CP KnowledgeArchitectureArchitecture

HPKB Upper Level

SAIC Merged Ontology (Y1)

PQ CasesActionsInterests

Year 2 Domain Specific

Analogy ...

SAIC Merged Ontology (Y1) SAIC Merged Ontology (Y1) DomainsDomains

Capability Analysis Benefits/Risks analysis Terrorism World Fact Book International economics model A National interests model A model of economic, military, and diplomatic

support/opposition. World oil flow

SAIC Merged Ontology (Y1) SAIC Merged Ontology (Y1) DomainsDomains

Properties of multilateral organizations Capabilities and Resources International Organizations, Companies Military weapons, artillery, personnel Strike Capabilities EIA pages (oil quotas, etc) International Organizations,

memberships, goals Geographical information

Common Knowledge Common Knowledge ComponentsComponents

PQ Ontology Ontology used to define the vocabulary available for

the user to query the system. Actions

A model of international actions described in the International System Framework Document (ISF).

Interests A model of national interests and strategic interests

defined by the ISF.

Common Knowledge Common Knowledge Components (Cont’d)Components (Cont’d)

Analogy Ontology Case Library

Year 1 Scenario Year 2 Scenario 1998 Iranian-Taliban Crisis Abu Musa Incident Caspian Pipeline Consortium (CPC) Operation Desert Shield 1990-1 1984-8 Tanker War

…

Knowledge Base Knowledge Base Development StrategyDevelopment Strategy

Shared upper structure and SAIC merged ontology

Common components across developers Periodic KB merging into common

components

Knowledge ArchitectureKnowledge Architecture

Currently available in Ontolingua HPKB upper level SAIC merged Ontology (Y1) PQ Ontology Knowledge Components …..

http://ontolingua.stanford.edu

SAIC Crisis Management SAIC Crisis Management Year 2 PQ distribution Year 2 PQ distribution

Different technology developers assume responsibility for specific PQs, but make use of shared knowledge structures

PQ distribution as shown (next slide)

Parameterized Question Distribution

200 SRI 220 SRI 240 SAIC201 SRI 221 SRI202 SRI 222 SRI 251 SAIC203 SRI 223 NWU 252 KSL204 SRI 224 NWU 253 KSL205 FRG 225 NWU 254 SRI206 SRI 226 NWU 255 SAIC207 FRG

228 NWU 124 KSL, MIT209 SRI 125 KSL, MIT210 SRI 230 FRG 126 KSL, MIT211 KSL 231 FRG 127 KSL, MIT212 KSL 232 KSL 128 KSL213 KSL 233 KSL214 SAIC 234 SRI

216 MIT/START 236 SAIC217 MIT/START 237 SAIC

238 SAIC219 SRI 239 SAIC

Critical Component Critical Component Experiments (CCEs)Experiments (CCEs)

Theory Merging CCE Led by KSL. Merges CMU, FRG, KSL, NWU, SAIC and

SRI Knowledge Bases. Develops merging tools and techniques Merging evaluation (TBD)

Critical Component Critical Component Experiments (CCEs)Experiments (CCEs)

Knowledge Extraction (TextWise) TextWise parses a multi-year multi-source

corpus to produce output that populates terrorism templates defined by SAIC.

Phased approach Terrorist Group Template definitions loaded into

SNARK KB (currently available) Post January: Population of Terrorist Event and

Supporting Action templates

Critical Component Critical Component Experiments (CCEs)Experiments (CCEs)

Natural language interface to selected Parameterized Questions using START/SNARK MIT START team parses natural language and

converts this text into KIF formalizations that are then input to SRI SNARK theorem prover.

Server used for START queries also used by SAIC GUI interface.

Critical Component Critical Component Experiments (CCEs)Experiments (CCEs)

Analogical Reasoning Led by NWU NWU will answer the analogical reasoning PQs

for the SAIC integration team. The questions will be answered as follows

Analogy Ontology (NWU) SME, MAC/FAC (Analogical Reasoner) (NWU) Case Library (SAIC)

All Ontologies stored in Ontolingua

SAIC Crisis Management SAIC Crisis Management User InterfacesUser Interfaces

GUI interface to SNARK (live) remote version (Server at SRI) local (server on laptop)

GUI interface to ATP Lisp translator to facilitate batch interface

processing of PQs

Stanford KSLStanford KSL

Stanford KSLStanford KSL

Richard Fikes

Deborah McGuinness

James Rice

Gleb Frank

Yi Sun

Stanford KSL-ATP & ATPLStanford KSL-ATP & ATPL ATP is supported and in use for challenge

problem work Providing ATP for use by FRG ATP has been upgraded to handle larger KBs ATP client side listener developed for remote

building and testing of KBs (see demo!) ATPL available for SAIC challenge problem

use offered knowledge server support to NWU

KSL-Challenge Problem KSL-Challenge Problem WorkWork

PQ answers (over 1/4 of questions) KB diagnostics differential questions

Merging CCE Led merge of Y1 KBs Developed initial merging tool Providing knowledge library of individual and

merged Y1 (and Y2) KBs

Explanation Approach IExplanation Approach I Break queries and answers into components based

on their logical form

conjunctive antecedents are separated

follow-up queries are generated for those that are not

directly asserted

query bindings may be presented

Explanation Approach IIExplanation Approach II Present in pseudo natural language

Use documentation strings and internal templates

Axiom: Diplomatic-Opposition-Propagation-Due-To-Group-Membership

(=> (and (Opposed-Diplomatically ?group ?enemy ?time-range)

(Group-Members ?group ?member))

(Opposed-Diplomatically ?member ?enemy ?time-range))

Doc String: ?member diplomatically opposed ?enemy because

?member is a member of ?group, which opposed ?enemy.

Explanation Approach IIIExplanation Approach III Prune (and/or rewrite) internal axioms

delete internal axioms such as “if a class is known to be

non-primitive, its primitiveness is false” by setting

explanation-visibility to be internal

generate abstract presentation strings for axioms such as

taxonomic inheritance

Explanation Approach IVExplanation Approach IV

Present abstractions for multiple answers “members of the UN-Security Council opposed Iraq”

rather than listing all of the members

Provide meta language for contextual and

domain-oriented pruning explanation visibility, slots to use for abstraction,

“interesting” slots, etc.

TAA68 TAA68 What countries diplomatically opposed What countries diplomatically opposed Iraq after the Persian Gulf War?Iraq after the Persian Gulf War?

Incremental ExplanationsIncremental Explanations

Incremental Explanations IIIncremental Explanations II

Status and PlansStatus and Plans Status

Implemented for ATP Tested on KSL Y1 and some Y2 queries

Plans Implement pruning meta language based on description logic

foundation

Expand to other reasoners (e.g., SNARK)

Demonstrations available

SRISRI

SRI’s Contribution to SRI’s Contribution to IntegrationIntegration

Helped conceptualize the HIKE GUI Delivered a PC-based SNARK server Helped produce the SAIC merged ontology START/SNARK interface Loading information extracted by Textwise

Merging with Team SAICMerging with Team SAIC

Syntactic merge Semantic merge Computational merge

Syntactic MergeSyntactic Merge

KBs translated into the same language Different ways to write the same thing

(person ?x) or (instance-of ?x person) We converted our KBs into a syntax that

will be readable by KSL

Most (95%) of the work can be automated

Semantic MergeSemantic Merge

Semantic merge Identical terms should have the same

definitions Differences in representational choices

Mostly manual, but some tools possible

(Supporting-Terrorist-Attack ?action) =(and (instance-of ?action action) (supports ?action terrorist-attack))

Computational MergeComputational Merge

Merged KB can be as efficiently reasoned with as the original

Sorted vs unsorted language Consider (father ?x ?y)

The first argument must be a male

The second argument must be a person

In a sorted language, ?x will unify with only males

CMCP Knowledge BaseCMCP Knowledge Base

HPKB Upper Level

SAIC Merged Ontology (Y1)

PQ AgentsActionsInterests

ReadingComprehension

Option Generation

Option Evaluation

Cases

CMCP Knowledge BaseCMCP Knowledge Base

Responsibility for about 20 PQs Actively co-developing content with SAIC

Interface withInterface withProject Genoa Project Genoa

Structured Argumentation

Publish Arguments

Direct entry by SMEs

Fusion Fusion Fusion Fusion

Fusion

Q1.1.1 Q1.1.2 Q1.1.3 Q1.4.1 Q1.4..2 Q1.4.3Q1.2.1 Q1.2.2 Q1.2.3 Q1.3.1 Q1.3.2 Q1.3.3

A1.1 A1.2 A1.3 A1.4

A1

Final Conclusion

Is the project being managed according to the project plan?OK Caution Warning

Evidence:

Will the effort be completed on or ahead of schedule?

Will this effort be completed within the budget?

Will the technical solution be developed according to plan?

Will project resources for this effort be available according to plan?

Will operations be satisfied by the results of the project? OK Caution Warning

Evidence:

Will the projected capital & operating costs meet requirements?

Will the projected operating performance meet requirements?

Do projected operating benefits justify expected expenditures?

Are communications between project & operations staff satisfactory?

Argument

Templates

Interface with Project GenoaInterface with Project GenoaAccomplishments for 1998Accomplishments for 1998

OracleDB

SEAS Server

OKBC

WWW Browser

HTTP/HTML

Fusion Fusion Fusion Fusion

Fusion

Q1.1.1Q1.1.2Q1.1.3 Q1.4.1Q1.4..2Q1.4.3Q1.2.1Q1.2.2Q1.2.3 Q1.3.1Q1.3.2Q1.3.3

A1.1 A1.2 A1.3

A1

Ontology Manager

Grasper

Gister Engine

CL-HTTP Server

Oracle DBMSServer

SQL

Ocelot KBMS

Perk StorageSystem

Arg./Sit.Ontology

CWEST

SEAS HTML Generator

OKBC GKB-Browser

Interface with Project GenoaInterface with Project GenoaPlans for 1999Plans for 1999

Integration at content level Use situation ontology from HPKB for argument

indexing Multi-user editing of arguments

Use collaboration system for asynchronous editing Domain-specific GUI for editing argument

ontology Enhance GKB-Editor to be more accessible to SMEs

MIT - STARTMIT - START

MIT (START): MIT (START): Y2 Integration PlansY2 Integration Plans

Link START to other HPKB systems by translating English queries into PQ specifications, then forwarding the translated queries

Extend the START Server’s KB with background knowledge to support analyst’s activities

Support answering selected Parameterized Questions for the Y2 Crisis Management Challenge Problem

Increase START’s access to “live” information from the World Wide Web by incorporating robust access interfaces

MIT (START): New Coverage for Y2

• Material from the International System Framework and Agent-Specific Background Information documents, supporting PQs 216, 217, 124, 125, 126 and 127

• Background information on terrorist groups, including membership, activities, funding and locations

• Weapon strike capabilities between Persian Gulf regions and countries

• Information on Fortune 500 companies, including locations of headquarters, CEOs, assets, profits and stock prices

• Information on 30,000 U.S. cities, including areas, populations, coordinates, time zones and weather

MIT (START): New MIT (START): New Coverage for Y2Coverage for Y2

Material from the International System Framework and Agent-Specific Background Information documents, supporting PQs 216, 217, 124, 125, 126 and 127

Background information on terrorist groups, including membership, activities, funding and locations

Weapon strike capabilities between Persian Gulf regions and countries

Information on Fortune 500 companies, including locations of headquarters, CEOs, assets, profits and stock prices

Information on 30,000 U.S. cities, including areas, populations, coordinates, time zones and weather

NWUNWU

CMUCMU

CMU CM PlansCMU CM Plans Extract relevant ground facts from the Web

company instances name locations of operations economic sector products produced and raw materials consumed

(especially those on export-control lists) relations with other companies pieces of infrastructure

instances of <EconomicActionType>

CMU CM PlansCMU CM Plans

Deliver extracted facts to integration teams via OKBC.

Use facts to support PQs 200, 201, 203, 211, 216, etc. by representing economic interests, capabilities and actions of international agents, and links among agents.

Integration of Text ExtractionIntegration of Text Extractionwith SAIC Terrorism DBwith SAIC Terrorism DB

Ian Niles

TextWise, LLC

SAIC Terrorism DBSAIC Terrorism DB

(defobject ABU-NIDAL-ORGANIZATION"International terrorist organization led by Sabri al-Banna. Split from PLO in 1974. Made up of various functional committees, including political, military, and financial.(Source: 1996 Patterns of Global Terrorism:App. B: Background on Terrorist Groups, http://www.iet.com/Projects/HPKB/Web mirror/GLOB_terror/appb.html)”

(own-slot-value nick-name ABU-NIDAL-ORGANIZATION "ANO")

(individual ABU-NIDAL-ORGANIZATION)

(instance-of ABU-NIDAL-ORGANIZATION terrorist-group)

(residence-of-organization ABU-NIDAL-ORGANIZATION libya))

Integration of CRCs into DBIntegration of CRCs into DB

Terrorist Group template instances were automatically generated from KNOW-IT output in three steps:

A base template instance is created for each example of the proper noun category 54 (terrorist groups)

CRCs referencing terrorist groups are mapped to slots of the terrorist group template.

The automatically generated slots are inserted into the appropriate template instances.

Automatically Generated Template Automatically Generated Template InstancesInstances

(defobject HAMAS

"(Source: 1998 TextWise LLC Terrorism Database)"

(individual HAMAS)

(instance-of HAMAS terrorist-group)

(affiliated-with Palestine-Liberation-Organization)

(own-slot-value nick-name HAMAS Hamas)

(own-slot-value nick-name HAMAS Islamic-Resistance- Movement)

(residence-of-organization HAMAS Israel)

(residence-of-organization HAMAS United-States)

(residence-of-organization HAMAS West-Bank))

Automatically Generated Template Automatically Generated Template Instances (con’t)Instances (con’t)

(defobject Hizballah

"(Source: 1998 TextWise LLC Terrorism Database)"

(individual Hizballah)

(instance-of Hizballah terrorist-group)

(affiliated-with Islamic-Jihad)

(own-slot-value nick-name Hizballah Islamic-Jihad-for-the-Liberation-of-Palestine)

(own-slot-value nick-name Hizballah Lebanese-Hizballah)

(own-slot-value nick-name Hizballah Party-of-God)

(own-slot-value nick-name Hizballah Hezbollah)

(own-slot-value nick-name Hizballah Hizbollah)

(own-slot-value nick-name Hizballah Organization-of-the-Oppressed-on-Earth)

(own-slot-value nick-name Hizballah Revolutionary-Justice-Organization)

(residence-of-organization Hizballah Lebanon))

Future Integration WorkFuture Integration Work Crafting more rules to extract instances of the 54 (terrorist

group) proper name category

Automatic generation of instances of the two other Terrorism DB templates

Mapping more relations and combinations of relations to template slots

Making the ouput KIF 3.0 Compliant

•Carnegie Mellon University

•TextWise

•SRI International

•North Western University

•Stanford University (Knowledge Systems Laboratory)

•Stanford University (Formal Reasoning Group)

•Stanford University (Scaleable Knowledge Composition)

•Massachussets Institute of Technology

•Carnegie Mellon University

•TextWise

•SRI International

•North Western University

•Stanford University (Knowledge Systems Laboratory)

•Stanford University (Formal Reasoning Group)

•Stanford University (Scaleable Knowledge Composition)

•George Mason University

•Massachussets Institute of Technology

•Information Sciences Institute

•Stanford Medical Informatics

BackupsBackups

KB Development Time (Exluding TextWise)

0

5000

10000

15000

20000

25000D

ec

Jan

Feb

Mar

Apr

May

June

(SQ

)

TQ

A

TQ

B

TQ

C

TQ

D

Aug

Months

Axi

oms

SRI(SAIC)KSLNWUCMU

SAIC Crisis Management only

KB Development Time

010000

200003000040000

500006000070000

8000090000

Dec

Jan

Feb

Mar

Apr

May

June

(SQ

)

TQ

A

TQ

B

TQ

C

TQ

D

Aug

Months

Axi

oms

SRI(SAIC)KSLNWUTextWiseCMU

SAIC Crisis Management only

TextWiseTextWise 1. Create a terrorism database partition by retrieving a large multi-year, multi-

source corpus of documents which mention the terms"terrorism", "terrorist" or "terrorists" and running the document processing system over these documents (date of deliverable: 11/27).

2. Create an index from every canonicalized PN in the version of PNDBin /home/chess/CYC to all of its non-canonicalized variants (date ofdeliverable: 11/27).

3. Implement the pseudo-code for the Template Instance Generator (TIG)(date of deliverable: 12/31).

4. Design and implement component which will convert sets of CRCs intoinstances of Supporting Actions and Terrorist Attacks templates.

CreditsCMU - webKB• Tom Mitchell• Mark Craven

MIT - START• Boris Katz• Gary Borchardt

NWU - Flow Model• Ken Forbus• Jeff Usher

SRI - SNARK, GKB Editor• Vinay Chaudhri• Richard Waldinger• Mark Stickel

Stanford KSL - Ontolingua, ATP• Richard Fikes• Deborah McGuinness• James Rice• Gleb Frank

Stanford - FRG• John McCarthy• Tom Costello

TextWise - Know-IT• Liz Liddy• Woojin Paik

USC ISI - LOOM/EXPECT• Yolanda Gil• Jim Blythe

CreditsUSC ISI - LOOM• Bob Mcgregor• Hans Chalupsky• David Moriarty

Cycorp - Cyc• Doug Lenat• Ben Rode

Teknowledge - TFS• Adam Pease• John Li• Cleo Condoravdi

GMU - Disciple• George Tecucci• Katy Wright

SMI - Protege• Mark Musen• Natalya Fridman Noy• Bill Grosso

SAIC - SIKE• Dave Easter• Albert Lin• Barbara Starr• Don Henager• Henry Gunthardt• Ben Good• Brian Truong• Bryner Pancho• Lei Wang

HIKE N-tier ArchitectureHIKE N-tier Architecture

HIKEClient

HIKEClient

HIKEClient

HIKEClient

HIKEClient

HIKEClient

HIKEServer

HIKEServer

HIKEServer

HIKEServer

HIKEServer

HIKEServer

HIKEStub

HIKEStub

HPKBTechnologyComponent

HPKBTechnologyComponent

HPKBTechnologyComponent

HPKBTechnologyComponent

HPKBTechnologyComponent

HPKBTechnologyComponent

HIKEStub

HIKEStub

HPKBTechnologyComponent

HPKBTechnologyComponent

LoomOKBCServer

LoomOKBCServer

OKBC

OKBC

Java RMI

HTTP

Sockets (TCP/IP)

Three Levels of IntegrationThree Levels of Integration

There are 3 levels at which integration can occur: Transport layer (e.g. Sending information from

one server to another) Syntactic layer (Ensuring that information is in the

same syntax as that defined by another system) Semantic layer (Ensuring that all concepts and

theories are aligned) defined last year by Adam Pease

STARTSTART

OntolinguaOntolingua

HIKEGUI

HIKEGUI

OcelotOcelot

ATPATP

SNARKSNARK

SMEMAC/FAC

SMEMAC/FAC

GKB Editor

GKB Editor

TextWiseTextWise

WebKBWebKB

Analyst

Knowledge ArchitectureKnowledge Architecture

Currently available in Ocelot (Via GKB editor) HPKB upper level Actions Ontology Interests Ontology SAIC/SRI Y1 Ontology

lajolla.ai.sri.com:8000

Knowledge ServersKnowledge Servers A federation of OKBC Knowledge Servers

LOOM (USC ISI) Ontolingua (Stanford KSL) Ocelot (SRI) Cyc (Cycorp) ATP

Manual Knowledge Acquisition Tools GKB Editor (SRI) Ontolingua (Stanford KSL) JOT ATPL Ontosaurus (USC ISI) Expect (USC ISI)

Knowledge Servers (Cont’d)Knowledge Servers (Cont’d)

Semi- Automatic Knowledge Acquisition KNOW-IT (TextWise)

Text extraction from the web, newsfeeds and other sources

webKB (CMU) Knowledge Extraction (and discovery) from web

based sources.

Expect (USC ISI) Automatic generation of rules

Question AnsweringQuestion Answering

Natural Language Understanding START (MIT)

Parses natural language queries. Multimedia web based answers from annotated web sources.

TextWise Parses natural language queries. Returns answers from web

based sources by parsing textual information.

Theorem Provers SNARK (SRI) ATP (Stanford KSL)

Problem SolversProblem Solvers

Machine Learning Disciple Learning Agent (GMU)

multi-strategy learning methods Problem Solving Methods

Problem Solving Methods Stanford Medical Informatics (SMI)

Three layered PSM to detect, classify, and monitor battlefield activities.

Information Science Institute (ISI) Course of Action Generation problem solvers to

create alternative solutions to workarounds problems.

Problem Solvers (Cont’d)Problem Solvers (Cont’d)

Bayesian Networks SPOOK (Stanford Robotics Laboratory)

System for Probabalistic Object Oriented Knowledge - supports reasoning with uncertainty

Qualitative Reasoning NWU/KSL

supports construction of certain types of models such as flow models, e.g. :

World Oil flow model Common Sense reasoning about the battlespace, focusing on the

trafficability/terrain suitability task.

Problem Solvers (Cont’d)Problem Solvers (Cont’d) Monitoring Process

Massachusetts Institute of Technology (MIT) provides tools for constructing and controlling

networks of distributed monitoring processes

Crisis Management -Crisis Management -Knowledge Level ArchitectureKnowledge Level Architecture

Knowledge Architecture design is an output of the Knowledge Architecture working group convened by SAIC

Includes the SAIC merged ontology The SAIC merged ontology contains the year 1

knowledge bases from KSL, FRG, SRI/SAIC, and CMU

Ontology merging effort led by Stanford KSL led to development of the KB merging tool