Explanation in GILA 2
Stanford -> RPIMcGuinness, Ding
January 15, 2008
Motivation
Improve trust in recommendations from GILA components• Support evaluation
– Understand why GILA makes suggestions – Identify which prior knowledge is (re-)used– Identify which learned knowledge is learned and (re-)used– Summarize usage of external interaction information steps
• Support internal trust and reuse– Identify which component suggested what and why– Identify/ propagate dependencies
Provided Knowledgeruntime inputruntime input
prior input prior input
Learned Knowledge
Final Output
Flow of GILA Knowledge
expert execution trace
ontological knowledge
constraint-violation
pstep list as the final solution
Learning & practice
performance
context knowledge
facts embedded inthe input problem
problem/solution
constraints
conflict priority
GILA KnowledgeKnowledge Category Producer Consumer OWL class
Execution trace Provided Expert MRE, ILRs gilcore:ExecutionTrace
Input ACO problem Provided Blueforce MRE, ILRs gilaco:ProblemACO
Context knowledgeProvided Context
serviceMRE, ILRs Phase II
Ontological knowledge Provided Expert MRE, ILRs aco:*
Constraint Learned CL ILRs gilcore:Constraint
Constraint Violation Learned SC MRE, ILRs gilaco:ViolationStatement
An ACO problem state Learned MRE ILRs gilaco:ProblemACO
Cost of an ACO problem state Learned DTL MRE, ILRs? gilaco:CostStatement
Credit/Blame Assignment Learned SC + MRE ILRs Phase IIConflicts in an ACO problem state
Learned 4D MRE gilcore:Conflict
Order of conflicts Learned ILRs MRE Phase II
Intersection of conflicts Learned 4D MRE gkst:IntersectionDetails
Pseudo expert trace Learned MRE, ILRs ILRs gilcore:ExecutionTrace
Final solution Learned MRE-DM PstepManager gilcore:PstepList
Phase II Objectives
• Show usage of provided knowledge– Expert execution trace– Ontological knowledge
• Show usage of learned knowledge– Constraints– Blame (constraint violation) assignments
• Show (abstracted) problem-solving trace (with dependencies)– problem, learner, solution, …., final solution
Approach
• Ontology Representation– Reference knowledge at appropriate level of granularity– Provide interlingua for learners’ knowledge usage
• Extract some knowledge from KB• Derive new knowledge from existing knowledge
• Computational Components– Run time API for sharing knowledge of explanations– API for extracting, summarizing and visualizing problem-
solving and knowledge usage trace utilizing Inference Web approach
Example: Blame-assignment
#vstmt192 (gilaco:ViolationStatement)•Degree of importance of violation•How severe the violation is•Confidence
#constraint 7(gilcore:Constraint)
#pstep261(gilcore:Pstep)
#problem26(gilaco:ProblemTrySafetyConstraint)
#solution28 (gilaco:SolutionTrySafetyConstraint)
SafetyChecker
MRE-DM #solution23(gilaco:SolutionResolveConflict)
#solution26(gilaco:SolutionResolveConflict)
X-ILRDT-ILR
…
Learned knowledge
Learned knowledge
Learned knowledge
Learned knowledge Provided knowledgeProvided knowledge
Example: Learn and Solve Based on correspondence, santi Nov 30, 2007
Solution 5647 Case 101
PSTEP 8859
PSTEP 8860
PSTEP 8861
ExecutionTrace 1878
generartedfrom
learnedfrom
learnedfrom
learnedfrom
in
in
in
0.9054
confidence
0.9054
confidence
Performance Step 129Performance Step 129 Learning Step 126Learning Step 126
CBL ILRCBL ILR
Has conclusion Has antecedent Has conclusion Has antecedent
match case and solvematch case and solve Learn new caseLearn new caseHas learner
Has learnerUse rule Use rule
Directions• Determine appropriate granularity for representation and
propagation (initially coarse level moving to finer granularity where required)
• Design appropriate primitives for phase II topics – context, credit/blame, orderings, priorities, …
• Focus on dependencies initially (supporting explanations showing usage of prior knowledge, external interaction-gained info, use and re-use of learned knowledge, similarity knowledge, adaptation knowledge…)
• Design GILA-appropriate explanation templates exploiting our explanation interlingua
• Present knowledge provenance summaries (with follow up options)
PSTEP Manager
FujitsuSong
PSTEP Manager in Phase 1
SWebADSWebAD
PSTEP Manager (PMAN)
PSTEP List
PSTEP List
Translate into SWebAD
command
Translate into SWebAD
command
Backup PSTEP information
Backup PSTEP information
ExecutionExecution
Generate execution trace
• Known Issues– PMAN is widely used to check
whether a proposed solution is correct or not, each check takes long time.
– Most failures are due to some errors which can be checked before execution and avoided (e.g. NaN maximum altitude, minimum altitude > maximum altitude, etc.)
– The knowledge may not be discovered from the expert trace (since the expert does not make this type of mistakes), but can be learned from the execution result reported by SWebAD during the execution time.
PSTEP Manager in Phase 1 is a simple execution engine and it hides execution related details from other modules
PSTEP Manager in Phase 2
STBMCSSTBMCSPSTEP
ListPSTEP
List
Translate into STBMCS
command
Translate into STBMCS
command
Backup PSTEP information
Backup PSTEP information
Generate execution trace
Early Error DetectionEarly Error Detection
Execution Request
Execution Request
STBMCS Error
Learner
STBMCS Error
Learner
Constraint knowledge repository
Constraint knowledge repository
PSTEP List Optimization
PSTEP List Optimization
New ModulesNew Modules
Existing ModulesExisting Modules
PSTEP Manager (PMAN)
• STBMCS Error learner learns new knowledge from error report of STBMCS (previously SWebAD)
• Share constraint knowledge with 4DCL
• Use constraint knowledge to detect errors of proposed solution before real execution
Share with 4DCL/R
Error Report
Extra
Files/WWW Toolkit
Proof Markup Language (PML)
CWM (NSF TAMI)
JTP(DAML/NIMD)
SPARK(DARPA CALO)
UIMA(DTO NIMD
Exp Aggregation)
IW Explainer/Abstractor
IWBase
IWBrowser
IWSearch
Trust
Justification
Provenance
N3
KIF
SPARK-L
Text Analytics
IWTrust
provenanceregistration
search enginebased publishing
Expert friendlyVisualization
End-user friendly visualization
Trust computationSemantic Discovery Service
(DAML/SNRC)
OWL-S/BPEL
Framework for explaining question answering tasks by • abstracting, storing, exchanging, • combining, annotating, filtering, segmenting, • comparing, and rendering proofs and proof fragments provided by question answerers.
Inference Web Infrastructure McGuinness, Ding, Pinheiro da Silva, Chang, Fikes, Glass, Zeng
Top Related