The SNePS Approach to Cognitive Robotics
description
Transcript of The SNePS Approach to Cognitive Robotics
S.C. Shapiro
cse@buff
alo
The SNePS Approach to Cognitive Robotics
Stuart C. Shapiro
Department of Computer Science and Engineering
and Center for Cognitive Science
University at Buffalo
S.C. Shapiro
cse@buff
alo
Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buff
alo
Goal
• A computational cognitive agent that can:– Understand and communicate in English; – Discuss specific, generic, and “rule-like” information;– Reason;– Discuss acts and plans;– Sense;– Act;– Remember and report what it has sensed and done.
S.C. Shapiro
cse@buff
alo
Embodied Cassie
• A computational cognitive agent– Embodied in hardware– or Software-Simulated– Based on SNePS and GLAIR.
S.C. Shapiro
cse@buff
alo
SNePS• Knowledge Representation and Reasoning
– Propositions as Terms
• SNIP: SNePS Inference Package– Specialized connectives and quantifiers
• SNeBR: SNePS Belief Revision
• SNeRE: SNePS Rational Engine
• Interface Languages– SNePSUL: Lisp-Like– SNePSLOG: Logic-Like– GATN for Fragments of English.
S.C. Shapiro
cse@buff
alo
GLAIR Architecture
Knowledge Level
Perceptuo-Motor Level
Sensory-Actuator Level
NL
ActionPercepts
Grounded Layered Architecture with Integrated Reasoning
SNePS
S.C. Shapiro
cse@buff
alo
Interaction with CassieEnglish
(Statement, Question, Command)
(Current) Set of Beliefs[SNePS]
(Updated) Setof Beliefs[SNePS]
Actions[SNeRE]
(New Belief)[SNePS]
English sentence expressingnew belief answering question reporting actions
Answer[SNIP]
GATN Parser
GATN Generator
ReasoningClarification DialogueLooking in World
Reasoning
S.C. Shapiro
cse@buff
alo
Cassie, the BlocksWorld Robot
S.C. Shapiro
cse@buff
alo
Cassie, the FEVAHR
S.C. Shapiro
cse@buff
alo
FEVAHR/Cassie in the Lab
S.C. Shapiro
cse@buff
alo
FEVAHRWorld Simulation
S.C. Shapiro
cse@buff
alo
UXO Remediation Cassie
CassieCorner flag
NonUXO object
Corner flag
UXO
Batterymeter
Corner flag
Drop-off zone
Field
Safe zone
RechargingStation
S.C. Shapiro
cse@buff
alo
Crystal Space Environment
S.C. Shapiro
cse@buff
alo
Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buff
alo
Entities, Terms, Symbols, Objects
• Cassie’s mental entity: a person named Bill
• SNePS term: B5• Object in world:
S.C. Shapiro
cse@buff
alo
Intensional Representation
Intensional entities are distincteven if coreferential.
“The morning star is the evening star.”
“George IV wondered if Scott was the author of Waverly.”
S.C. Shapiro
cse@buff
alo
McCarthy’s Telephone Number Problem
Mary's telephone number is Mike's telephone number.
I understand that Mike's telephone number is Mary's telephone number.
Pat knew Mike's telephone number.
I understand that Pat knew Mike's telephone number.
Pat dialed Mike's telephone number.
I understand that Pat dialed Mike's telephone number.
S.C. Shapiro
cse@buff
alo
Answering the Telephone Number Problem
Did Pat dial Mary's telephone number?
Yes, Pat dialed Mary's telephone number.
Did Pat know Mary's telephone number?
I don't know.
S.C. Shapiro
cse@buff
alo
Representing Propositions
Propositions must befirst-class entities of the domain
Represented by terms.
S.C. Shapiro
cse@buff
alo
Discussing Propositions
That Bill is sweet is Mary's favorite proposition.
I understand that Mary's favorite proposition is that Bill is sweet.
Mike believes Mary's favorite proposition.
I understand that Mike believes that Bill is sweet.
S.C. Shapiro
cse@buff
alo
Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buff
alo
Logic for NLU &Commonsense Reasoning
Either Pat is a man or Pat is a woman or Pat is a robot.
I understand that Pat is a robot or Pat is a woman or Pat is a man.
Pat is a woman.
I understand that Pat is a woman.
What is Pat?
Pat is a woman and Pat is not a robotand Pat is not a man.
S.C. Shapiro
cse@buff
alo
Representation in FOPL?
Man(Pat) Woman(Pat) Robot(Pat)
S.C. Shapiro
cse@buff
alo
Representation in FOPL?
Man(Pat) Woman(Pat) Robot(Pat)
but don’t want inclusive or
S.C. Shapiro
cse@buff
alo
Representation in FOPL?
Man(Pat) Woman(Pat) Robot(Pat)
but don’t want inclusive or
Man(Pat) Woman(Pat) Robot(Pat)+ +
T T T
F
TSo don’t want exclusive or either
S.C. Shapiro
cse@buff
alo
andor
andor(i, j){P1, ..., Pn}
True iff at least i, and at most j of the Pi are True
S.C. Shapiro
cse@buff
alo
thresh
thresh(i, j){P1, ..., Pn}
True iff either fewer than i,
or more than j
of the Pi are True
Note: thresh(i, j) ~andor(i, j)
S.C. Shapiro
cse@buff
alo
or-entailment
{P1, ..., Pn} v=> {Q1, ..., Qn}
True iff for all i, j Pi Qj
S.C. Shapiro
cse@buff
alo
and-entailment
{P1, ..., Pn} &=> {Q1, ..., Qn}
True iff for all j
P1 &…& Pn Qj
S.C. Shapiro
cse@buff
alo
Numerical entailment
{P1, ..., Pn} i=> {Q1, ..., Qn}
True iff for all j
andor(i, n){P1, …, Pn } Qj
S.C. Shapiro
cse@buff
alo
Universal Quantifier
all(ū)({R1(ū),..., Rn(ū)} &=> {C1(ū),..., Cm(ū)})
Every ā that satisfies
R1(ū)&…& Rn(ū)
also satisfies
C1(ū),..., Cm(ū)})
S.C. Shapiro
cse@buff
alo
Numerical Quantifiers
nexists(i,j,k)(x)
({P1(x),..., Pn(x)}: {Q(x)})}
There are k individuals that satisfy
P1(x) ... Pn(x)
and, of them, at least i and at most j also satisfy
Q(x)
S.C. Shapiro
cse@buff
alo
Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buff
alo
MENTAL ACTS
• Believe(proposition)• Disbelieve(proposition)
S.C. Shapiro
cse@buff
alo
Act Selection
• Do-One({act1 ... actn})
• Snif(if(condition, act),
...
if(condition, act)
[else(act)])
S.C. Shapiro
cse@buff
alo
Act Iteration• Do-All({act1 ... actn})• Sniterate(if(condition, act),
...
if(condition, act),
[else(act)])• Snsequence(act1, ..., actn)• Cascade(act1, ..., actn)• P-Do-All({act1, ..., act2})
S.C. Shapiro
cse@buff
alo
Entity Iteration
WithSome(var, suchthat,
do, [else])
WithAll(var, suchthat,
do, [else])
WithSome+(var, suchthat,
do, [else])
WithNew(vars, thatare, suchthat,
do, [else])
S.C. Shapiro
cse@buff
alo
Proposition/Act Transformers
• Achieve(proposition)• ActPlan(act, plan)• GoalPlan(proposition, act)• Precondition(act, proposition)• Effect(act, proposition)• WhenDo(proposition, act)• WheneverDo(proposition, act)• IfDo(proposition, act)
S.C. Shapiro
cse@buff
alo
Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buff
alo
Conditional Plans If a block is on a support then a plan to achieve
that the support is clear is to pick up the block and then put the block on the table.
all(x, y) ({Block(x), Support(y), On(x, y)} &=> {GoalPlan(Clear(y), Snsequence(Pickup(x), Put(x, Table)))})(STRIPS-like representation: No times)
S.C. Shapiro
cse@buff
alo
Use of Conditional Plan
GoalPlan(Clear(B),
Snsequence(Pickup(A),
Put(A, Table)))Remember (cache) derived propositions.
S.C. Shapiro
cse@buff
alo
Use of Conditional Plan
GoalPlan(Clear(B),
Snsequence(Pickup(A),
Put(A, Table)))???SNeBR to the rescue!
S.C. Shapiro
cse@buff
alo
A FEVAHR Acting Ruleall(a, o) ({Agent(a), Thing(o)}
&=> {Precondition(Follow(a, o), Near(a, o)),
GoalPlan(Near(a, o), Goto(a, o)),
Precondition(Goto(a, o), Lookat(a, o)),
ActPlan(Lookat(a, o), Find(a, o))})
Uses a temporal model.
S.C. Shapiro
cse@buff
alo
Acting According to the Rule
S.C. Shapiro
cse@buff
alo
Acting According to the Rule
I found a red robot.I am looking at a red robot.
Follow a red robot.
S.C. Shapiro
cse@buff
alo
Acting According to the Rule
I went to a red robot.I am near a red robot.I am following a red robot.
I found a red robot.I am looking at a red robot.
Follow a red robot.
S.C. Shapiro
cse@buff
alo
A Plan for Blowing up UXOsall(a)(Agent(a) =>ActPlan(Blowup(a, UXOs), Act(a,Cascade(SearchforUxo(a),
WithSome+(obj, Near(a, obj), WithNew({ch ex},
{Charge(ch), Explosion(ex)}, Possess(a, ch), Cascade(Place(a, ch, obj), Hide(a),
Waitfor(a, ex), SearchforUxo(a))),
goto(a, SafeZone))))))
S.C. Shapiro
cse@buff
alo
Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buff
alo
Representation and Use of Indexicals
• Words whose meanings are determined by occasion of use
• E.g. I, you, now, then, here, there
• Deictic Center <*I, *YOU, *NOW>
• *I: SNePS term representing Cassie
• *YOU: person Cassie is talking with
• *NOW: current time.
S.C. Shapiro
cse@buff
alo
Analysis of Indexicals(in input)
• First person pronouns: *YOU• Second person pronouns: *I• “here”: location of *YOU• Present/Past relative to *NOW.
S.C. Shapiro
cse@buff
alo
Generation of Indexicals
• *I: First person pronouns
• *YOU: Second person pronouns
• *NOW: used to determine tense and aspect.
S.C. Shapiro
cse@buff
alo
Come here.
Use of Indexicals 1
S.C. Shapiro
cse@buff
alo
Come here.I came to you, Stu.I am near you.
Use of Indexicals 2
S.C. Shapiro
cse@buff
alo
Who am I?Your name is ‘Stu’and you are a person.
Who have you talked to?
I am talking to you.Talk to Bill.
I am talking to you, Bill.Come here.
Use of Indexicals 3
S.C. Shapiro
cse@buff
alo
Come here.
I found you.I am looking at you.
Use of Indexicals 4
S.C. Shapiro
cse@buff
alo
Come here.
I came to you.I am near you.
I found you.I am looking at you.
Use of Indexicals 5
S.C. Shapiro
cse@buff
alo
Who am I?
I talked to Stuand I am talking to you.
Your name is ‘Bill’and you are a person.
Who are you?I am the FEVAHRand my name is ‘Cassie’.
Who have you talked to?
Use of Indexicals 6
S.C. Shapiro
cse@buff
alo
Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buff
alo
A Personal Sense of Time
• *NOW contains SNePS term representing current time.
• *NOW moves when Cassie acts or perceives a change of state.
S.C. Shapiro
cse@buff
alo
B6
Representation of Time
find
lex
action object
B1
!
agentact
eventtime
NOW
!!before after before after
?????????????
I
S.C. Shapiro
cse@buff
alo
Movement of Timet1 t2!before after t3!before after
NOW NOW NOW
S.C. Shapiro
cse@buff
alo
Performing a Punctual Actt1 t3!before after
NOW NOW
t2!before after
!
time
event
S.C. Shapiro
cse@buff
alo
Performing a Durative Actt1
NOW
!before after t2
!
time
event
NOW
t3!
supintsubint
S.C. Shapiro
cse@buff
alo
The Pacemaker
• PML process periodically increments variable COUNT.
• *COUNT = some PML integer.
• Reset to 0 when NOW moves.
• Provides bodily “feel” of passing time.
S.C. Shapiro
cse@buff
alo
Quantizing Time
Cannot conceptualize fine distinctions in time intervals.
So quantize, e.g. into half orders of magnitude (Hobbs, 2000).
S.C. Shapiro
cse@buff
alo
Movement of Time with Pacemaker
NOW COUNT n
hom
0
KL
PML
t1 t2q
!before after
time duration!
S.C. Shapiro
cse@buff
alo
The Problem of the Fleeting Now
How can you reason about “now”
if it never stands still?
S.C. Shapiro
cse@buff
alo
Fleeting Now Example 1
12:15:00: “Is John having lunch now?”
12:15:02: Agent walks to John’s office.
12:17:00: Agent sees John at his desk, eating.
12:19:00: Agent reports “yes”.
Appropriate granularity.
S.C. Shapiro
cse@buff
alo
Fleeting Now Example 2
12:15:00: “Is John having lunch now?”
Agent knows John is at home without a phone.
Agent contemplates driving to John’s home.
Don’t bother---inappropriate granularity.
S.C. Shapiro
cse@buff
alo
The Vagueness of “now”
I’m now giving a talk.I’m now supervising PhD students.I’m now visiting Paris.I’m now living in Buffalo.The agent is now walking to John’s office.The agent is now seeing if John is eating lunch.
Multiple now’s at different granularities.
S.C. Shapiro
cse@buff
alo
NOW-MTF
NOW
Semi-lattice of times, all of which contain *NOW,any of which could be meant by “now”Finite---only conceptualized times of conceptualized states
Maximal Temporal Frame based on *NOW
S.C. Shapiro
cse@buff
alo
Moving NOW with MTF
NOW
S.C. Shapiro
cse@buff
alo
Typical Durations
“If the walk light is on now, cross the street.”Relevant duration is typical duration of walk lights.
“Is John having lunch now?”Relevant duration is typical duration of lunch.
Use quantized typical durations when updating NOW-MTFs.
S.C. Shapiro
cse@buff
alo
Using Appropriate Granularity
NOW
Lunch time
Lunch? Lunch!
Yes!
S.C. Shapiro
cse@buff
alo
Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buff
alo
Goal
• A computational cognitive agent/robot
• That can communicate in natural language.
S.C. Shapiro
cse@buff
alo
Intensional Representation& Propositions
• SNePS terms represent mental entities.
• May assert that two entities are coreferential.
• Relations/acts may be declared transparent.
• Propositions are first-class entities.
S.C. Shapiro
cse@buff
alo
SNePS Connectives and Quantifiers
• Designed logical connectivesand rules of inference
More appropriate for NLU and Commonsense reasoningthan in standard FOPC.
S.C. Shapiro
cse@buff
alo
SNeRE Acting Constructs
• Separate, but Coordinated
Syntax and Semantics
For Acting and for Reasoning
S.C. Shapiro
cse@buff
alo
Representation and Useof Indexicals
• Use of Deictic Center for parser to interpret indexicals as current referents
• And for generator to generate indexicals from current referents.
S.C. Shapiro
cse@buff
alo
A Personal Sense of Time
• *NOW is current time.
• Updated when Cassie acts
or perceives a change of state.
• Points into MTF to support vagueness of “now”.
S.C. Shapiro
cse@buff
alo
For More Information
• Personnel
• Manual
• Tutorial
• Bibliography
• ftp’able SNePS source code
• etc.
• http://www.cse.buffalo.edu/sneps/