Our analogical processing tools
Base
Target
SME
Inputs = propositional descriptions, w/ incremental updatesOutput = one or twomappings
Operates in polynomial time,by exploiting graph labels & greedy algorithms
Mappings = correspondences + structural evaluation + candidate inferences
SME
SME
SME
CVmatch
CVmatch
CVmatch
CVmatch
MemoryPool
Output = memory item+ SME results
Probe
Structure-Mapping Engineprovides analogical
matching
MAC/FAC providessimilarity-based
retrieval
Cheap, fast, non-structural
No hand-indexingof cases required
How SEQL Works
Exemplars…
GeneralizationsSME
NewStimulus
1. Compare against each generalization Gi. If close
enough, assimilate input into Gi by replacing Gi with the overlap of Gi and input and
halt.
2. Compare input against each exemplar Ei. If similar enough, create new generalization from overlap of Ei and input, halt. If
nothing similar enough, add input to set of exemplars
SEQL refines knowledge by progressive alignment of examples
New: The GEL algorithm
Case Mapper: An Analogy GUI
• Goal: Provide civilized interface for entering knowledge via analogy– Should be useful platform for experimenting with
dialogue moves
• Current state– Basic functionality showing signs of life– AI-expert friendly
• Next steps– Improved pidgin– Interface to inference machinery for candidate
inference evaluation– Explore using dialogue management, simple NLP for
interaction
Integrating into the E2E system
• Strategy: Provide analogy server– KQML for communication
– Strategies for analogical reasoning coded in next-generation reasoner
• Advantages– Neutral with respect to uniprocessor/distributed
operation
– Enables us to tune our strategies more easily
• Drawbacks– Sockets as bottleneck
– Need to keep KB in synch
• Alternative strategy: Assimilation
Domain Theory Environment (DTE)
KnowledgeBase
Reasoner
AnalogicalReasoner
SpatialReasoner
GIS
Uses ODBC, Relational database(Microsoft Access) to store
KB contents(inspired by Hendler’s PARKA-
DB)
Domain Theory Environment (DTE)
KnowledgeBase
Reasoner
AnalogicalReasoner
SpatialReasoner
GIS
Federated architecture,supports reasoning sourcesthat provide special-purpose
capabilities efficiently
Domain Theory Environment (DTE)
KnowledgeBase
Reasoner
AnalogicalReasoner
SpatialReasoner
GIS
Query-driven backchainerprovides basic reasoning services,
integration mechanism
Domain Theory Environment (DTE)
KnowledgeBase
Reasoner
AnalogicalReasoner
SpatialReasoner
GIS
KQML interface for building servers
(e.g., analogy server,geographic reasoner)
DTE Problems
KnowledgeBase
Reasoner
AnalogicalReasoner
SpatialReasoner
GIS
Too slow, not scaling well
High overhead,too many computational
cliffs
Solution: Build next-generation system
• Collaborating with Xerox PARC– John Everett, Reinhard Stolle, Bob Cheslow
• Keeping good ideas in DTE:– Federated architecture/Reasoning sources model
– Using database to implement KB
– Query mechanism with simple backchainer as glue
– Use of LTMS for justifications, reasoning
• Overall structure of interfaces to applications using it will be similar
• Internals will be very different
Next-generation system
KnowledgeBase
Reasoner
AnalogicalReasoner
SpatialReasoner
GIS
Special-purpose C++ database,written by PARC. Built-in
support for pattern matching.Adding new knowledge:
DTE DB: 4 assertions/secondNew DB: 98 assertions/secondRetrieval:2-3 msec, in 111K
assertion KB(preliminary data)
Next-generation System
KnowledgeBase
Reasoner
AnalogicalReasoner
SpatialReasoner
GIS
Working memory = LTRE + discrimination tree indexing.
Suggestions Architecture:Limit backchaining for “quick”
reasoning. Expensive operations queued as
suggestions, processed via agenda mechanism.
Multithreaded, to exploit time user spends doing other things.
Especially important for sketching, dialogue
management
Next-Generation System
KnowledgeBase
AnalogicalReasoner
SpatialReasoner
Gizmo Mk2
PerceptualInk
Processor
Reasoner
Streamlined reasoning source interface, with constraint posting
for query optimizer.
Provide qualitative reasoningservices by embedding QP
theory implementation
Create ink-based spatial reasoner, organized for
incremental processing from the ground up
Current schedule
• Halloween: First version turning over• Thanksgiving: DTE applications ported• Christmas: First round of performance tuning
finished
Everyday Physical Semantics domain theory
• Claim: There is a basic set of physical notions that need to be understood in order to interpret sketched explanations– e.g., Simple notions of
surfaces, volumes, forces, and materials
• Claim: Qualitative physics research can provide most of this knowledge– Much of it has already been
done, in isolated pieces– Needs to be integrated, gaps
filled– Tied to sketch-based spatial
representations
• Surface constraints on motion– Will use Nielsen’s qualitative
mechanics
• Fluid Ontologies– Collins’ molecular collection
ontology
– Kim’s bounded stuff ontology
– + usual contained stuff ontology
• Surface/fluid interactions– Kim’s qualitative streamline
theory
• Qualitative topology– Cohn’s spatial algebras
• Qualitative Statics– Nielsen & Kim’s qualitative
vectors
Multiple Perspectives: An example
• How to reason about liquids?• Two models, due to Hayes
– Contained stuff ontology: Individuate liquid via the space that it is in.
– Piece of stuff ontology: Individuate liquid as a particular collection of molecules.
Fluid ontologies
• Contained stuffs– Most detailed: Paper with John Collins, FSThermo
domain theory
• Pieces of stuff– Molecular collections (w/John Collins)
– Plugs (Gordon Skorstad)
• Bounded stuffs (H. Kim)
Molecular Collection ontology
• Idea: Follow a little piece of stuff around a system– So small that when it reaches a junction, it never splits
apart
• Provides the perspective gained by tracing through a system of changes
Bounded stuffs
• Specialization of contained stuff ontology• Where something is within the space matters
– Affects connectivity
Ontology zoo for liquids
Contained Stuff Piece of Stuff
PlugMolecularCollection
Bounded Stuff
Parasitic on
Qualitative Mechanics• Provides axioms for
interaction of solids and surfaces– Qualitative vector
representation
• Assumes visual parsing of 2D shapes– Center of gravity, center of
rotation critical
– Surfaces broken at corners, points of contact
not OkOk
Ok
not Ok
Qualitative Mechanics
• Qualitative angles and vectors• How forces interact with surfaces, constraints on
motion• Laminar flow fields
Engineering Thermodynamics
• Basics of heat, mass flow• In-depth KB for supporting design, analysis• KB for supporting textbook problem solving
– Includes control knowledge, analysis of roles for equations in problem-solving
– Pisan’s Ph.D. thesis solves most problems in typical engineering thermodynamics textbooks
• Teleological representations for thermodynamic cycles– No chemical interactions
Sketching for knowledge acquisition• sKEA: Sketch-based Knowledge Entry
Associate– Built on top of nuSketch + significant
extensions– Rich perceptual processing of digital
ink• Will support visual analogies and
analogies using diagrams Speech I/O and specialized Dialogue Manager
• Can be used standalone or as component in larger system
• Ink Interpretation is key problem– Collaborating with PARC vision
group (Eric Saund, Jim Mahoney) for perceptual processing
– Developing domain theories that bridge perception and conceptual knowledge
sKEA
Digital ink
Everyday Physical SemanticsDomain Theory
Graphical SymbologyDomain Theory
PerceptualInk Processor
High-Level VisualInterpreter(GeoRep II)
DTE + EvidentialReasoner
MultimodalIntegrator
Speech I/O
Current Sketches+ Interpretations
RKF Team System
Tools we will use in sketching
Lo w-le ve lre la tio ns
Hig h-le ve lre la tio ns(p la c e vo c a b ula ry)
LLRD HLRD
Dom ain-specific
rules
Visual operation
library
Lined ra wing
L1 L3
L4
L5
C 1 C 2
L2
GeoRep MAGI
MAGI models processes ofsymmetry and regularity detection
• Uses variation of structure-mapping laws to detect self-similarity
• Same software operates on visual, functional, conceptual, and mathematical representations
•Makes predictions consistent with human perceptual data
GeoRep provides high-levelvisual processing for
spatial reasoning
Provides equivalentof Ullman’s universal
visual routines
Provides bridgebetween the visualand the conceptual
Visual Symbology domain theory• Represents conventions for
displaying conceptual information graphically
• Includes
– What visual entities often depict
• boxes, blobs, arrows, etc.
– Conventional views • side/top/bottom, 2D/3D,
abstract/physical, cutaways
– Conceptual interpretation of visual relations
• proximity/alignment indicating grouping, inside indicating containment or partonomy,touching indicating contact
State(before) Process
State(after)
Arg2Arg1 Binary
Relationship
CellDNA
VirusDNA
(Part-of cellDNA cell)
(in-contact (protein-coat virus) (lysosome cell))
Approach: Blob Semantics
• Shape, object recognition irrelevant– Linguistic input provides labels and type information– Arrows may be exception wrt recognition
• Spatial relationships between blobs is central– Topology
• Touching or not, inside, overlap
– Proximity• What arrows refer to
– Orientation• Multiple reference frames• Quadrant plus relative inclination
– Conceptual interpretation of spatial relationships
• Hypothesis: Sufficient for– Process diagrams– Action sequences
Issues in blob semantics
• Adequacy of visual primitives• User-defined diagram types
– Kinds of objects participating
– Conceptual interpretation of spatial relationships
• Arrow recognition– Support different types of arrows?
Perceptual Ink Processor
• Will use next-generation reasoner for conceptual side of reasoning
• For visual reasoning, draw on three sources:– Our work on GeoRep and Magi (Ferguson’s Ph.D.
work)
– Eric Saund’s scale-space blackboard (Xerox PARC)• Stroke-based visual routines
• Should provide robust proximity detection
– Jim Mahoney’s MAPS ideas (Xerox PARC)• Bitmap-based visual routines
• Should provide robust qualitative descriptions of free space
Speech or not?
• Most multimodal systems use speech recognition– Hands, eyes busy with diagram– Potential problems with speech for RKF
• Novel nouns, phrases could lead to distracting speech training during knowledge entry
• How open-ended is grammar? Necessity versus user expectations
• Trying both in RKF– NLP support with speech
• LKB parser (Stanford CSLI)
– Experiment: Speechless multimodal interface• Type (or write) label for instance, collection• Draw button, as in nuSketch COA Creator• Sacrifice fluidity for expressiveness
Top Related