ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM...
-
Upload
randolf-george -
Category
Documents
-
view
240 -
download
9
Transcript of ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM...
![Page 1: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/1.jpg)
ThoughtTreasure, the hard common sense problem, and applications of common sense
Erik T. MuellerIBM Research*
http://www.research.ibm.com/people/e/etm/
November 2002*Work performed while at Signiform
![Page 2: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/2.jpg)
Talk plan ThoughtTreasure overview ThoughtTreasure and the hard
common sense problem Applications of common sense
- SensiCal- NewsForms
*Work performed at the MIT Media Lab and Signiform
*
![Page 3: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/3.jpg)
ThoughtTreasure
Commonsense knowledge base Architecture for natural language
understanding Uses multiple representations:
logic, finite automata, grids, scripts
![Page 4: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/4.jpg)
ThoughtTreasure KB
35,023 English words/phrases 21,529 French words/phrases 51,305 commonsense
assertions 27,093 concepts
![Page 5: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/5.jpg)
ThoughtTreasure in CycLhttp://www.signiform.com/tt/ttkb/tt0.00022.cycl.gz
Including linguistic knowledge 547,651 assertions 72,554 constants
![Page 6: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/6.jpg)
ThoughtTreasure architecture80,000 lines of code
Text agency Syntactic component Semantic component Generator Planning agency Understanding agency
![Page 7: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/7.jpg)
ThoughtTreasure applications Commonsense applications Simple factual question answering
> What color are elephants? They are gray.> What is the circumference of the earth?40,003,236 meters.> Who created Bugs Bunny?Tex Avery created Bugs Bunny.
Story understanding
![Page 8: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/8.jpg)
The hard problem:Story understandingTwo robbers entered Gene Cook’s furniture store in Brooklyn and forced him to give them $1200.Who was in the store initially?And during the robbery?Did Cook want to be robbed?Did the robbers tell Cook their names?Did Cook know he was going to be robbed?Does he now know he was robbed?What crimes were committed?(adapted from http://www-formal.stanford.edu/jmc/mrhug.html)
![Page 9: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/9.jpg)
ThoughtTreasure approachto story understanding
1. To build a computer that understands stories, build a computer that can construct simulations of the states and events described in the story.
20020217T194352wwwwwwwwwwwwwwwwwww vw ttt w LtttJw ttt w
Actor Jim49 PAs Sleep PA
![Page 10: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/10.jpg)
ThoughtTreasure approachto story understanding
2. Modularize the problem by having different agents work on different parts of the simulation.
20020217T194352wwwwwwwwwwwwwwwwwww vw ttt w LtttJw ttt w
Actor Jim49 PAs Sleep PA
Jim49Sleep
UA
Jim49Emotion
UA
SpaceUA
TimeUA
![Page 11: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/11.jpg)
Story understanding by simulation Read a story Maintain a simulation = model of
story Answer questions
![Page 12: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/12.jpg)
The debate in psychology People reason using mental
modelsJohnson-Laird, Philip N. (1993). Human and machine thinking.
People reason using inference rulesRips, Lance J. (1994). The psychology of proof.
![Page 13: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/13.jpg)
The debate in AI AI programs should use lucid
representationsLevesque, Hector (1986). Making believers out of computers.
AI programs should use indirect representationsDavis, Ernest (1991). (Against) Lucid representations.
AI programs should use diverse representationsMinsky, Marvin (1986). The society of mind.
![Page 14: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/14.jpg)
Story understanding by simulation A simulation is a sequence of
states A state is a snapshot of the mental
world of each story character and the physical world
![Page 15: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/15.jpg)
Physical world:Grids and wormholes
20020217T194352
wwwwwwwwwwwwwwwwwww vw ttt w LtttJw ttt w z wormhole
grid
![Page 16: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/16.jpg)
Physical world: Objects20020217T194352
wwwwwwwwwwwwwwwwwww vw ttt w LtttJw ttt w
Object TVSet31State OffChannel 24
![Page 17: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/17.jpg)
Mental world: Planning agents20020217T194352
wwwwwwwwwwwwwwwwwww vw ttt w LtttJw ttt w
Actor Jim49PAsEmotions
Sleep
Watch TV
awakeasleep
![Page 18: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/18.jpg)
Mental world: Emotions20020217T194352
wwwwwwwwwwwwwwwwwww vw ttt w LtttJw ttt w
Actor Jim49PAsEmotions
happy 0.7hopeful 0.6
![Page 19: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/19.jpg)
v J
Jim turned on the TV.
v J
On
Each input updates the simulation
Off
v J
Off
5
5 6
![Page 20: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/20.jpg)
Why this approach?
Easy question answering by reading answers off the simulation
Convenient modularization by components of the simulation
![Page 21: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/21.jpg)
Modularization
One understanding agent per simulation componentSpaceTimeActor – one per story characterDevice – one per deviceGoalEmotionSleepWatch TV…
– one per story character
![Page 22: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/22.jpg)
Understanding agents
Space UA – Jim walked to the table. Emotion UA – Jim was happy. Sleep UA – Jim was asleep. TV device agent – The TV was on.
![Page 23: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/23.jpg)
Detail on ThoughtTreasure representations Planning agents Grids and wormholes
![Page 24: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/24.jpg)
INITIAL off-hook dialtone dialed
ringing
pickedup
talkingdoneFINAL
off-hook
retry
on-hook
WAIT 30 sec
WAITdialtone dial
WAIT busy
WAIT ring
WAIT hello
say-hellosay-bye
talk
on-hook
WAIT 30 sec
PhoneCall planning agent
![Page 25: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/25.jpg)
SubwayPlatform grid==metro-RD-2d//|col-distance-of=5m|row-distance-of=10m|
level-of=-2u|orientation-of=north|GS=
GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
w9 w
w w
w BBBBC BBBB BBBB BBBB CBBBB uu3
wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww.
u:stairs-up
w:wall
B:bench
G:&metro-Creteil-tracks
3:&metro-RD-Creteil-entrance1
9.wuG:&metro-RD-Creteil-platform
![Page 26: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/26.jpg)
SubwayPlatform grid==metro-RD-2d//|col-distance-of=5m|row-distance-of=10m|
level-of=-2u|orientation-of=north|GS=
GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
w*****************************************w
w*****************************************w
w***************************************uu3
wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww.
u:stairs-up
w:wall
B:bench
G:&metro-Creteil-tracks
3:&metro-RD-Creteil-entrance1
9.wuG:&metro-RD-Creteil-platform
![Page 27: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/27.jpg)
SubwayPlatform grid==metro-RD-2d//|col-distance-of=5m|row-distance-of=10m|
level-of=-2u|orientation-of=north|GS=
GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
w w
w w
w BBBBC BBBB BBBB BBBB CBBBB uu3
wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww.
u:stairs-up
w:wall
B:bench
G:&metro-Creteil-tracks
3:&metro-RD-Creteil-entrance1
9.wuG:&metro-RD-Creteil-platform
wormhole
![Page 28: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/28.jpg)
wwwwwwwwwwwwwwwwwwwwwwwwwwwww
ddT uuu
w wwwwwwww w
w w Xw w
w BfA w w
w wwwwwwww w
w w
wwwTwwwwwwwwwwwwwwwwwwwwwwwww
d
3.
d:stairs-down
u:stairs-up
w:wall
A:employee-side-of-counter
B:customer-side-of-counter
T:automated-turnstile
X.wf:&metro-RD-ticket-booth
3:&metro-RD-Creteil-entrance1
SubwayTicketArea grid
wormhole
![Page 29: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/29.jpg)
Inferences from grids Distance between objects Relative position of objects (left, right,
front, back) Whether two actors can see or hear
each other Whether there is a path from one
location to another
![Page 30: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/30.jpg)
Text agency
Input
text
Syntactic component
nodes
filters base rules
syntactic parser
Semantic component
parsetrees semantic parser
anaphoric parser
generator
Understanding agencydiscourse NOW 20020312T102344 SPEAKER Tom62 LISTENER TT LANGUAGE English CONTEXTS
Planning agency
context SENSE 0.9 STORY TIME 20020217T115849 REPRESENTATIONSwwwwwwwwww J wwwwwwwww
Actor Jim49 PAs EMOTIONS
planning agent OBJ [sleep Jim49] STATE asleep
understanding agent
sleep und. agent
planning agent OBJ [...]
understanding agent
Outputtext
assertions [...]
ThoughtTreasure architecture
“He falls asleep...”
“Yes, ...”
![Page 31: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/31.jpg)
Simple stories handled by ThoughtTreasure Two children, Bertie and Lin, in
their playroom and bedrooms Jenny in her grocery store Jim waking up and taking a shower
in his apartment
![Page 32: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/32.jpg)
Story understanding by simulation
> Lin walks to her playroom.Ts = 20020217T115849
wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
w w BBBBw
w w BBBBw
w Lin :-) awake w
w wwwwwwwwwwwwwwwwww
w w
w w LLLL w
w w LLLL w
wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
![Page 33: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/33.jpg)
> She is where?
She was in the playroom of the apartment.
Story understanding by simulation
![Page 34: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/34.jpg)
> She walks to her bedroom.Ts = 20020217T115902
wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
w w BBBBw
w w BBBBw
w .. w
w ... wwwwwwwwwwwwwwwwww
w ........Lin :-) awake w
w w LLLL w
w w LLLL w
wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
Story understanding by simulation
![Page 35: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/35.jpg)
> She is where?
She was in her bedroom.
> She is near her bed?
Yes, she is in fact near the bed.
Story understanding by simulation
![Page 36: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/36.jpg)
> She falls asleep.Ts = 20020217T120034
wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
w w BBBBw
w w BBBBw
w w
w wwwwwwwwwwwwwwwwww
w . w
w w .LLL w
w w Lin :-) asleep w
wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
Story understanding by simulation
![Page 37: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/37.jpg)
> She is asleep?
Yes, she was in fact asleep.
> She is awake?
No, she is not awake.
> She was in the playroom when?
She was in the playroom of the apartment two minutes ago.
Story understanding by simulation
![Page 38: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/38.jpg)
Scaling up ThoughtTreasure Add more grids, planning agents,
understanding agents Use automated and semi-
automated techniques for acquisition
Use Open Mind for acquisition
![Page 39: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/39.jpg)
Applications of common sense SensiCal NewsForms
![Page 40: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/40.jpg)
SensiCal Smart calendar application Speeds entry by filling in information Points out obvious blunders
![Page 41: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/41.jpg)
SensiCal
![Page 42: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/42.jpg)
Operation of SensiCal
new or modified calendar item
extract information
commonsense reasoning
fill in missing informationpoint out potential
problems
![Page 43: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/43.jpg)
Information extracted
Type of item Participants
- role- name
Location- venue name and type- earth coordinates
![Page 44: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/44.jpg)
Text: lunch w/lin at frank's steakhouseStartTs: 20020528T120000 EndTs: 20020528T130000
ItemType: mealMealType: lunchParticipant: LinVenue: Frank's SteakhouseVenueType: steakhouse
Information extracted
![Page 45: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/45.jpg)
Common sense of calendaring People grocery shop in grocery stores Vegetables are found in grocery stores A person can’t be in two places at once Allow sufficient time to travel from one location to another People typically work during the day and sleep at night People do not usually attend business meetings on holidays Lunch is eaten around noon for an hour Don’t eat at restaurants that serve mostly food you avoid Vegetarians avoid meat A steak house serves beef Beef is meat You can’t visit a place that’s not open Restaurants do not generally serve dinner after 11 pm Museums are often closed on Mondays …
![Page 46: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/46.jpg)
Representation of common sense in ThoughtTreasure Assertions Scripts Grids Procedures
- Trip planning agent- Path planner
![Page 47: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/47.jpg)
Assertions
[performed-in eat-dinner restaurant][serve-meal steakhouse beef] [avoid vegetarian meat]
![Page 48: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/48.jpg)
Scripts Roles[role-of visit-museum museum-goer] [role-of wedding-ceremony bride]
Sequence of events[event01-of go-blading
[put-on blader rollerblades]][event02-of go-blading
[blade blader]] Time and duration[min-value-of eat-lunch 11am][max-value-of eat-lunch 2pm][duration-of eat-lunch 1hrs]
![Page 49: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/49.jpg)
Cost[cost-of take-subway $1]
Entry conditions[entry-condition-of sleep
[sleepy sleeper]]
Goals[goal-of telephone-call
[talk calling-party called-party]]
Scripts
![Page 50: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/50.jpg)
GridsVVVVVSSSwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwVVVVVSS2q fnffwVVVVVSSSq fxffwVVVVVSSSw ttt ffffmffff ffffwVVVVVSSSw tttB sssssssssssssssssssssssssssss ffffwVVVVVSSSw cArtt sssssssssssssssssssssssssssss fuffwVVVVVSSSw ttt z y ffffwVVVVVSSSw1 fpffwVVVVVSSSw ffffw VVVVVSSSwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww.A:employee-side-of-counter B:customer-side-of-counterc:side-chair f:store-refrigeratorm:Sprite n:carrotn:greenhouse-lettuce n:tomaton:onion …
![Page 51: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/51.jpg)
Trip planning agent
1. walk to subway stop in street grid (00:10)
2. take subway on subway grid (00:20)3. board plane in airport grid (01:00)4. fly along Atlantic Ocean in earth
grid(07:00)
Total (08:30)
![Page 52: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/52.jpg)
Path planner&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&& wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww&& wwwwwwwwwww tttwsss ht Aswcccccccccw&& wwwwwwwwwwwwwwwwwww w tttwmss wcccccccccw&&wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwccccccccw w w smwmss wcc w&&w llllllll ss g Ecccc ssssssssswccccccccw w w smwmss wcv w&&wa llllllll ss s ssssssssswccccccccw wwww wwwww wwmss wcc w&&wA Ec sswccccccccw wwwww wwwwwwwwww wwww&&w cc sswccccccccw w&&wr sswwwwwwwwww w&&wr w **************************************** w&&wr ************ wwwwwwwwwwwwwwwwwwwwwwwwwttttttttttttl* lw&&w *********** ssw wttttttttttttt* tw&&w ************ ssw wttttttttttttt* w&&w ****** w ssw wA s * EEEEEEEEEw&&w w Rsw w * EEEEEEEEEw&&w s w sw wls * EEEEEEEEEw&&w ttttttttt wwwwww wwww w s * EEEEEEEEEw&&wr sttttttttts ss ssw w w s * EEEEEEEEEw&&wr ttttttttt ss ssw w * EEEEEEEEEw&&wr s ss Aw w * EEEEEEEEEw&&w s ss w w E * EEEEEEEEEw&&w tttt sssssssssss ss rrw w ttttttEt EEEEEEEEEw&&w A sssssssssss ss rrw w w rrrrr rrrrr Aw&&wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
![Page 53: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/53.jpg)
SensiCal status Prototype implemented in Tcl and
Perl as extension to ical Communicates with
ThoughtTreasure using ThoughtTreasure server protocol
![Page 54: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/54.jpg)
SensiCal future work
Add more commonsense knowledge and reasoning
Hook up to Open Mind
![Page 55: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/55.jpg)
NewsForms NewsForms are XML
representations of news events NewsForms enable numerous
commonsense applications
![Page 56: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/56.jpg)
NewsForms for 17 types of events
competitions deals earnings reports economic releases Fed watching IPOs injuries and fatalities joint ventures legal events
![Page 57: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/57.jpg)
medical findings negotiations new products management successions trips and visits votes war weather reports
NewsForms for 17 types of events
![Page 58: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/58.jpg)
NewsForm defines
Elements for news event typesDeal, InjuryFatality
Child elementsTarget of DealCause of InjuryFatality
Standard values (#PCDATA)Earthquake, Fire
![Page 59: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/59.jpg)
NewsForm DTD<?xml version="1.0" encoding="UTF-8"?><!– NewsForm document type definition --><!ELEMENT Deal (Acquirer|Advisor|DealStatus| DealValue|SharePrice|Stake|StockRatio| Successor|Survivor|Target)*>
<!--Rumored/InTalks/Agreed/Approved/ Completed/Failed --><!ELEMENT DealStatus %STRING><!ELEMENT InjuryFatality (AccidentCar| AccidentPlane|AtLocation|Boat|Cause| CauseEvent|Hospitalized|Injured| InjuredCount|Killed|KilledCount| LandedPlane|Source|SurvivedBy)*>
…
![Page 60: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/60.jpg)
Sample NewsForm<NewsForm> <Head> <DatelineTime>19990125T181917Z</DatelineTime> </Head> <InjuryFatality> <Cause>Earthquake</Cause> <InjuredCount>900</InjuredCount> <KilledCount>143</KilledCount> <Source><Function>CivilDefenseOfficial </Function></Source> <AtLocation> <Country>COL</Country> <Latitude>4.29</Latitude> <Longitude>-75.68</Longitude> </AtLocation> </InjuryFatality></NewsForm>
![Page 61: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/61.jpg)
NewsForm applications E-commerce Portable devices Desktop applications
![Page 62: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/62.jpg)
Travel site informs user of earthquake
Price: 1 adult @ USD 269.50
Flight: Jetblue Airways flight 83 on an Airbus Industrie Jet
Departs: Thursday, March 01From: New York Kennedy (JFK) at 9:00pmTo: Seattle/Tacoma, WA (SEA) at 11:59pm
_____________________________Strong Earthquake Rocks Seattle(02/28/01, 3:47 p.m. ET) By Chris Stetkiewicz and Scott Hillis , Reuters SEATTLE—An earthquake measuring 7.0 rattled Seattle Wednesday, swaying buildings and forcing the evacuation of thousands from their offices, schools, homes, and hospitals, witnesses said.
New York Kennedy (JFK) to Seattle/Tacoma, WA (SEA)
![Page 63: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/63.jpg)
Agent redirects order to bankrupt partner
____________________________Thursday April 12, 12:57 pm Eastern Time
iTech Capital Notes PinPoint Has Ceased OperationsVANCOUVER, BRITISH COLUMBIA--PinPoint Corporation, a developer of local positioning system technologies, has notified all of its shareholders that it has been unsuccessful in its attempt to raise additional capital required to continue operations. As such, PinPoint has ceased operations and has filed for protection under the bankruptcy laws.
automatedagent
PinPointBUY Mobile Search
![Page 64: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/64.jpg)
Cell phone informs fan of nearby star
______________________________STAR TRACKS'DEATH' IN NEW YORK: Leading men ANTHONY HOPKINS and BRAD PITT are now in town shooting "Meet Joe Black," a remake of the after-life fantasy "Death Takes a Holiday." Pitt's stunt double gets hit by a car ... so the story can begin.
![Page 65: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/65.jpg)
Calendar program informs user of subway outage
______________________________TRAVELING ON THE T > TRANSIT UPDATEGreen Line:Green Line service on the E train is temporarily suspended this morning due to signal work.
Mon TueMeeting atMFA
Wed Thu Fri
E
![Page 66: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/66.jpg)
NewsForms status NewsForm DTD specified NewsExtract text-to-NewsForm
converter implemented NewsExtract search engine
implemented
![Page 67: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/67.jpg)
NewsExtractalpha2
Search for news stories
Type Date range Map Sort by
About NewsExtract
Questions or comments? [email protected] © 2000 Signiform. All Rights Reserved. Terms of use.
a Search
Any Last 30 days US Date
![Page 68: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/68.jpg)
NewsExtractalpha2614 matches for a (last 30 days)
word count: a:614Last updated July 14, 2000 04:01:44 EDT
X•••••••••••••••••••••••• • ••••• •
•••• ••• • •• ••••••••
•••X G •• • • •••••• •••
•• •••••• •• •• ••••••• ••••••••••• • •
•X••• •• ••• •••••••••• • G• •••••• X••X•••• •
• • •••• • • ••• •X•• ••• X •• ••••••••
•• • •••• •••••••••••••X•X•• • •X••• •••X•
•••••••••• •••••• • •• •• X•••XX•X••••••X•••••X•
•• X• • B •••••• X••• ••••••• • • •X XXX•• •
• • • • XX ••••••••X•X B•• • X•B X•••••XX•X•
•XXX ••• • • X • • •••••XX•X••• ••••X••
X• •• ••••••••••••••••••••••••••••••••••••••••••••••B•
••• X•• • • • X G • •• X •X••X •••
•••XX •• • • •••••••• G ••••••••X•• ••••X••
•X• • X • • •X•••••••• • •X •••••••
• ••••••••••X•••• X •• •• • X• •••
•• X X •• ••••••••••••X•
•••••• ••X••••••• •••• •
••• ••• X X•
••• •••X
The facts on this page are automatically parsed out of the story text and may be incorrect.
Gene Discovery in Mice May Lead to New Arthritis Treatments (washingtonpost.com)
WashingtonPost NewsForm MedicalFinding Illness: Arthritis
InternetNews - Streaming Media News -- AOL, RealNetworks Team for StreamingInternetNews.com NewsForm
JointVenture JointVentureType: Agreement
Convolve, MIT Sue Compaq, Seagate For $800 MillionTechWeb NewsForm
LegalEvent Forum: U S Industries Inc [USI.N] LegalAction: File LegalFiling: Suit
20000714
![Page 69: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/69.jpg)
NewsForms future work Improve precision/recall of
NewsExtract Launch distributed human project
for realtime NewsForm creation and correction
Build commonsense applications that use NewsForms
![Page 70: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/70.jpg)
ThoughtTreasure, the hard common sense problem, and applications of common sense
Erik T. MuellerIBM Research*
http://www.research.ibm.com/people/e/etm/
November 2002*Work performed while at Signiform
![Page 71: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/71.jpg)
Extra slides
![Page 72: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/72.jpg)
ThoughtTreasure vs. Cyc ThoughtTreasure inspired by Cyc Cyc mostly uses single representation (logic);
ThoughtTreasure uses multiple representations (logic, finite automata, grids)
Recovery of script information from Cyc is difficult:(=> (and (subEvents ?X ?U) (isa ?U Staining)) (isa ?X WoodRefinishing))(=> (and (isa ?U ShapingSomething) (subEvents ?U ?X)) (isa ?X CuttingSomething))
ThoughtTreasure scripts are easy to use:[event-of refinish-wood [stain human wood]][event-of shape [cut human physical-object]]
![Page 73: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/73.jpg)
ThoughtTreasure vs. OpenCyc
Analysis of nonlinguistic assertionshttp://www.signiform.com/tt/htm/opencyctt.htm
OpenCyc 0.6.0 ThoughtTreasure 0.00022
Hierarchical 62% 56%Typing 33% 2%Spatial 0% 4%Script 0% 4%Part 0% 2%Property 0% 1%Other 5% 31%
Assertions 60,878 51,305
![Page 74: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/74.jpg)
WordNet is lexical, not conceptual; world knowledge, scripts, object properties excluded
WordNet has separate databases for nouns, verbs, adjectives, adverbs; no prepositions; few relations between databases; agreement not connected to agree
WordNet lacks top-level ontology
ThoughtTreasure vs. WordNet
![Page 75: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/75.jpg)
ThoughtTreasure vs. WordNet
WordNet is monolingual (But see EuroWordNet)
WordNet weak on argument structurelet go of, let go, release *> Somebody ----s something*> Somebody ----s somebody
WordNet has no parser, generator, understanding agents
![Page 76: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/76.jpg)
Scripts in ThoughtTreasure Scripts proposed in 1970s by
Schank & Abelson, Minsky, Wilks Few attempts to build database of
scripts 100 scripts in ThoughtTreasure
![Page 77: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/77.jpg)
Scripts in ThoughtTreasure:Representation based on Schank & Abelson, 1977
Sequence of events Roles, props Places Entry conditions, goals, results Emotions Duration, frequency, cost
![Page 78: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/78.jpg)
ThoughtTreasure object: call
[English] telephone call, call, make a telephone call, make a phone call, make a call, give a ring to, give a call to,
telephone, ring up, ring, call up, call, phone up, phone; [French] appeler, téléphoner à
[ako ^ interpersonal-script]
[cost-of ^ NUMBER:USD:1]
[duration-of ^ NUMBER:second:600]
[event01-of ^ [pick-up calling-party phone-handset]]
[event02-of ^ [dial calling-party phone-number]]
[event03-of ^ [interjection-of-greeting called-party calling-party]]
[event04-of ^ [interjection-of-greeting calling-party called-party]]
[event05-of ^ [converse calling-party called-party]]
[event06-of ^ [interjection-of-departure calling-party called-party]]
[event07-of ^ [interjection-of-departure called-party calling-party]]
[event08-of ^ [hang-up calling-party phone-handset]]
[goal-of ^ [near-audible calling-party called-party]]
[performed-in ^ room]
[period-of ^ NUMBER:second:7200]
[r1 ^ human]
[r2 ^ human]
[related-concept-of ^ phonestate]
[role01-of ^ calling-party]
[role02-of ^ called-party]
[role02-script-of ^ handle-call]
[role02-script-of handle-call ^]
[role03-of ^ phone]
[role04-of ^ phone-handset]
[role05-of ^ phone-number]
![Page 79: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/79.jpg)
ThoughtTreasure object: mail-letter-at-post-office
[ako ^ mail-letter]
[cost-of ^ NUMBER:USD:0.33]
[duration-of ^ NUMBER:second:600]
[event01-of ^ [pick-up sender snail-mail-letter]]
[event02-of ^ [ptrans sender na post-office]]
[event03-of ^ [wait-in-line sender]]
[event04-of ^ [ptrans-walk sender na postal-counter]]
[event05-of ^ [pre-sequence postal-clerk sender]]
[event05-of ^ [pre-sequence sender postal-clerk]]
[event06-of ^ [hand-to sender postal-clerk snail-mail-letter]]
[event07-of ^ [weigh postal-clerk snail-mail-letter]]
[event08-of ^ [postmark postal-clerk snail-mail-letter]]
[event09-of ^ [post-sequence postal-clerk sender]]
[event09-of ^ [post-sequence sender postal-clerk]]
[event10-of ^ [ptrans sender post-office na]]
[goal-of ^ [owner-of snail-mail-letter recipient]]
[goal-of ^ [s-employment postal-clerk]]
[performed-in ^ post-office]
[period-of ^ NUMBER:second:604800]
[role01-of ^ sender]
[role02-of ^ recipient]
[role03-of ^ snail-mail-letter]
[role04-of ^ post-office]
[role05-of ^ postal-counter] [role06-of ^ postal-clerk]
![Page 80: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/80.jpg)
ThoughtTreasure object: have-filling-done
[English] have a filling, have a filling done; [French] se faire faire un plombage
[ako ^ dentist-appointment]
[cost-of ^ NUMBER:USD:200]
[duration-of ^ NUMBER:second:3600]
[emotion-of ^ [nervousness role-patient]]
[emotion-of ^ [pain role-patient]]
[event01-of ^ [ptrans role-patient na dental-office]]
[event02-of ^ [ptrans-walk role-patient na waiting-room]]
[event03-of ^ [wait role-patient]]
[event04-of ^ [action-call dental-assistant na role-patient]]
[event05-of ^ [ptrans-walk role-patient waiting-room dental-operatory]]
[event06-of ^ [sit-in role-patient dental-chair]]
[event07-of ^ [inject dentist novocaine mouth]]
[event08-of ^ [wait role-patient]]
[event09-of ^ [drill-tooth dentist tooth dental-drill]]
[event09-of ^ [listen role-patient elevator-music]]
[event10-of ^ [fill-tooth dentist tooth dental-filling]]
[event11-of ^ [ptrans role-patient dental-operatory na]]
[goal-of ^ [p-health role-patient]]
[goal-of ^ [s-profit dentist]]
[performed-in ^ dental-office]
[period-of ^ NUMBER:second:1.5768e+08]
[r1 ^ human]
[role01-of ^ role-patient]
[role02-of ^ dentist]
[role03-of ^ dental-assistant]
[role04-of ^ tooth]
[role05-of ^ mouth]
[role06-of ^ dental-office]
[role07-of ^ waiting-room]
[role08-of ^ dental-chair]
[role09-of ^ dental-operatory]
[role10-of ^ dental-filling]
[role11-of ^ novocaine]
![Page 81: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/81.jpg)
Scripts in ThoughtTreasure:Comparison to related work
Cyc: 185 events with 1 subevent (avg 1.7) FrameNet: 20 frames (0 subevents) Andrew Gordon’s EPs: 768 EPs (avg 3.2
subevents) ThoughtTreasure: 100 scripts (avg 8.4
subevents) WordNet: no scripts but 427 synsets with
outgoing entailment links (avg 1.06 subevents)
![Page 82: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/82.jpg)
Grids in ThoughtTreasure
restaurant, bar, grocery-store, theater-ground-floor, theater-hall, TV-studio, city-apartment1, small-apartment-building-floor, small-apartment-building-ground-floor, city-apartment2, large-apartment-building-ground-floor, country-house-ground-floor, hotel-ground-floor, hotel-room-and-floor, city-street1, city-street2, city-street-and-park, country-area, subway-ticket-area, subway-platform1, subway-platform2, subway-platform3, subway-tracks, airport1, airport2, highway-map1, highway-map2, highway-map3
![Page 83: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/83.jpg)
AAAAAAAAAAAss wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww ssGGGGGGGGGGG
AAAAAAAAAAAss w1 n n n SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS wuuuuw3 kwv k 3w ssGGGGGGGGGGG
AAAAAAAAAAAss w C C C C nwuuuuw vw w ssGGGGGGGGGGG
AAAAAAAAAAAss w wuuuuw w w ssGGGGGGGGGGG
AAAAAAAAAAAss w SSSSSfffffffffffffffffSSSSSSSSS w wwwpppwwpppwwwwwwwww ssGGGGGGGGGGG
AAAAAAAAAAAss w SJ S w p 0w ssGGGGGGGGGGG
AAAAAAAAAAAss wn S S w p w ssGGGGGGGGGGG
AAAAAAAAAAAss w S C C C C C C C C S nwVV p f ssGGGGGGGGGGG
AAAAAAAAAAAss w S8 ttttttttttttttttt 8S S wwwwwwww wwwww f ssGGGGGGGGGGG
AAAAAAAAAAAss w S w w w w f ssGGGGGGGGGGG
AAAAAAAAAAAss w FP T S wwwwwwww w w f ssGGGGGGGGGGG
AAAAAAAAAAAss wn C C C S p w w f ssGGGGGGGGGGG
AAAAAAAAAAAss w M ttLttLtLtLC HS p w w w ssGGGGGGGGGGG
AAAAAAAAAAAss w RRDRT T M tttttMttLC HS wwwwwwwwwwppppwwwww w ssGGGGGGGGGGG
AAAAAAAAAAAss w M tLttLMtLLC HSSS w w6 7w w ssGGGGGGGGGGG
AAAAAAAAAAAss wn FP C C C S nw w w w ssGGGGGGGGGGG
AAAAAAAAAAAss w 98 tC S w w 7w w ssGGGGGGGGGGG
AAAAAAAAAAAss w S C tt tC S w w w w ssGGGGGGGGGGG
AAAAAAAAAAAss w SCtttt CttZM T tC S w wpppwwwwwwww w ssGGGGGGGGGGG
AAAAAAAAAAAss w S tttt S w w5 w w ssGGGGGGGGGGG
AAAAAAAAAAAss wn S C C S nw w w w ssGGGGGGGGGGG
AAAAAAAAAAAss w SSSSSSS B B B B S w wYM tUU w w ssGGGGGGGGGGG
AAAAAAAAAAAss w SK ttttttt 8S w wY tUUc w w ssGGGGGGGGGGG
AAAAAAAAAAAss w S w wYM ttt w w ssGGGGGGGGGGG
AAAAAAAAAAAss wn SSSSSSSSSSSSSSSSSSSSSSSS nwuuuu wY tttc w w ssGGGGGGGGGGG
AAAAAAAAAAAss w wuuuu wYM ttt w w ssGGGGGGGGGGG
AAAAAAAAAAAss w n n n n n n wuuuu wYK tQQc w w ssGGGGGGGGGGG
AAAAAAAAAAAss wwwwwwwwwwwwwwwwwwwwppppwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwuuuu wYM tQQ w w ssGGGGGGGGGGG
AAAAAAAAAAAss w4 wY tttc w w ssGGGGGGGGGGG
AAAAAAAAAAAss w wYM ttt p w ssGGGGGGGGGGG
AAAAAAAAAAAss w wY9 tttc p w ssGGGGGGGGGGG
AAAAAAAAAAAss w wYM p w ssGGGGGGGGGGG
AAAAAAAAAAAss w w w w ssGGGGGGGGGGG
AAAAAAAAAAAss w wwwwwwwwwwww w ssGGGGGGGGGGG
AAAAAAAAAAAss w w wwwwssGGGGGGGGGGG
AAAAAAAAAAAss w w p pNsGGGGGGGGGGG
AAAAAAAAAAAss w p p pssGGGGGGGGGGG
AAAAAAAAAAAss w p wwwwssGGGGGGGGGGG
AAAAAAAAAAAss w p ptttssGGGGGGGGGGG
AAAAAAAAAAAss w w p c ssGGGGGGGGGGG
AAAAAAAAAAAss w w wwwwssGGGGGGGGGGG
AAAAAAAAAAAss w w p pssGGGGGGGGGGG
AAAAAAAAAAAss w w p pssGGGGGGGGGGG
AAAAAAAAAAAss w w wwwwssGGGGGGGGGGG
AAAAAAAAAAAss w w w ssGGGGGGGGGGG
AAAAAAAAAAAss w w w ssGGGGGGGGGGG
AAAAAAAAAAAss w w w ssGGGGGGGGGGG
AAAAAAAAAAAss w w w ssGGGGGGGGGGG
AAAAAAAAAAAss w w w ssGGGGGGGGGGG
AAAAAAAAAAAss w w w ssGGGGGGGGGGG
AAAAAAAAAAAss w w w ssGGGGGGGGGGG
AAAAAAAAAAAss w w w ssGGGGGGGGGGG
AAAAAAAAAAAss w w w ssGGGGGGGGGGG
AAAAAAAAAAAss wwwwwwwwwwwwwwwwwwwwwwwwwwwwppppwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww ssGGGGGGGGGGG
AAAAAAAAAAAss w w w2WWWWWW Hw w w ssGGGGGGGGGGG
AAAAAAAAAAAss w w p II Hw w w ssGGGGGGGGGGG
AAAAAAAAAAAss w w p II Xw w w ssGGGGGGGGGGG
AAAAAAAAAAAss w w w w w w ssGGGGGGGGGGG
AAAAAAAAAAAss w w wc M w w w ssGGGGGGGGGGG
AAAAAAAAAAAss wwwwwww wwwwwwwwwwww w w ssGGGGGGGGGGG
AAAAAAAAAAAss w w tt wwwww w w ssGGGGGGGGGGG
AAAAAAAAAAAssEEEp p tt c p p w w ssGGGGGGGGGGG
AAAAAAAAAAAssEEEp p ttttttt p p w w ssGGGGGGGGGGG
AAAAAAAAAAAssEEEp Hw ttttttt wwwww w w ssGGGGGGGGGGG
AAAAAAAAAAAssOEEp Hw w w w ssGGGGGGGGGGG
AAAAAAAAAAAssEEEp Hw wwwww w w ssGGGGGGGGGGG
AAAAAAAAAAAssEEEp p p p w w ssGGGGGGGGGGG
AAAAAAAAAAAssEEEp p p p w w ssGGGGGGGGGGG
AAAAAAAAAAAss w w wwwww w w ssGGGGGGGGGGG
AAAAAAAAAAAss wwwwwww w w w ssGGGGGGGGGGG
AAAAAAAAAAAss w w M wO w w ssGGGGGGGGGGG
AAAAAAAAAAAss wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww ssGGGGGGGGGGG
TVStudio grid
![Page 84: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/84.jpg)
CountryArea gridMMMMMMMMMMMMMMMMMMMMMMMMMMMM rrr
MMMMMTMMMMMTMMMMMTMMMMMM T rrr
MMMMMTMMMMTMMMMMM T hhh rrr
MMMMMMMMM BBB hhh rrr
BBB rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrN
MMM F T rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
MMMMMMMM F rrr hhh
MMMMTMMMTMMM F rrr hhh
MMMMTMMMMMTMMMMM BBB rrr
MMMMMMMMMMMMMMMMMMM BBB rrr
MMTMMMTMMMMMTMMMTMMM rrr
MMMMMTMMMTMMMTMMMMM BBBB rrr
MMMTMMMTMMMTMMMTMM BBBB rrr
MMMMMMMMMMMMMMMMMM T BBBB rrr
MMMMMMMTMMTMMM rrr
MMMMMMMMMMM HHH rrr
MMMMTMMTMM T HHH rrr
MMMMMMMMMM F BBB L rrr
MMTMMTMMM T F BBB rrrr
MMMMMMMM PPPP rrrr
MMTMMTMM CPPPDDDDDDDDDrrrrr
MMMMMMM PPPPDDDDDDDDrrrrrrr
MMMM PPPP rrrr rrrr
M rrrr rrrr
hhh rrrrr rrrrr
hhh rrrrr T rrrrrr
rrrrrrrrrrrrrrrrrrrrrrrrr T T rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
rrrrrrrrrrrrrrrrrrrrrr T rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
1 T T T T T T
T T T T T T T
T T T T T T T T T T
T T T T T T T T T T T T T T
T T T T T T T T T T T T T T T T T T T T
T T T T T T T T T T T T T T T T T
![Page 85: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/85.jpg)
Inferences from grids Distance between objects Relative position of objects (left, right,
front, back) Whether two actors can see or hear
each other Whether there is a path from one
location to another
![Page 86: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/86.jpg)
Planning agents in ThoughtTreasure
GRASPER: Grasp, Release, Holding, Connect, ConnectedTo, Rub, Pour, SwitchX, FlipTo, KnobPosition, GestureHere, HandTo, ReceiveFrom
CONTAINER: ActionOpen, ActionClose, Open, Closed, Inside MOVEMENT: MoveTo, MoveObject, SmallContainedObjectMove,
HeldObjectMove, GrasperMove, ActorMove, LargeContainerMove TRANSPORTATION: NearAudible, NearReachable, NearGraspable,
GridWalk, Sitting, Standing, Lying, Sit, Stand, Lie, Warp, Drive, GridDriveCar, MotorVehicleOn, MotorVehicleOff, Stay, Pack, Unpack
COMMUNICATION: Mtrans, ObtainPermission, HandleProposal, Converse, Call, HandleCall, OffHook, OnHook, PickUp, HangUp, Dial
MONEY: PayInPerson, PayCash, PayByCard, PayByCheck, CollectPayment, CollectCash
INTERPERSONAL RELATIONS: MaintainFriends, Appointment ENTERTAINMENT: AttendPerformance, WorkBoxOffice,
PurchaseTicket, WatchTV, TVSetOn, TVSetOff PERSONAL: Sleep, Shower, Dress, PutOn, Wearing, Strip, TakeOff
![Page 87: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/87.jpg)
Device agents in ThoughtTreasure
CAR: off, on, ignition TV: off, on, channel, antenna, plug TELEPHONE: hook, state, handset, dial, phone number,
connection SHOWER: off, on, faucet, shower head, washing hair, shampoo
![Page 88: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/88.jpg)
Planning agents (PAs)Finite automata augmented with the ability to perform arbitrary
computations
1. Do A2. If X else3. Wait for Y4. Do B5. If Z else6. Do C
![Page 89: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/89.jpg)
PhoneCall planning agentcall(A1, A2) :-1: T = FINDO(phone),
H = FINDP(phone-handset, T),
off-hook(H), WAIT FOR dialtone(T) AND GOTO 22
OR WAIT 30 seconds AND GOTO 777,22: RETRIEVE phone-number-of(CLD =
FINDO(phone NEAR A2), N = number); dial(FINDP(right-hand, A1),
FINDP(phone-dial, T), N), WAIT FOR busy-signal(T) AND GOTO 777 OR WAIT FOR audible-ring(T, CLD) AND GOTO 4, OR WAIT 10 seconds AND GOTO 7774: WAIT FOR voice-connection(T, CLD) AND GOTO 5 OR WAIT 30 seconds AND GOTO 777
![Page 90: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/90.jpg)
PhoneCall planning agent
5: WAIT FOR interjection-of-greeting(A3 = human, ?)
AND GOTO 61 OR WAIT 30 seconds AND GOTO 77761: ASSERT near-audible(A1, A3); IF A3 != A2 GOTO 990; calling-party-telephone-greeting(A1, A2),62: WAIT FOR mtrans(A1, A2) AND GOTO 7 OR WAIT 5 seconds AND GOTO 990,7: WAIT FOR mtrans(A2, A1) AND GOTO 990 OR WAIT 5 seconds AND GOTO 990,777: on-hook(H) ON SUCCESS GOTO 1,990: interjection-of-departure(A1, A3), WAIT FOR interjection-of-departure(A3, A1)
OR WAIT 5 seconds, RETRACT near-audible(A1, A3); on-hook(H).
![Page 91: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/91.jpg)
PhoneCall device agentH = FINDP(phone-handset, T)
IF condition(T, W) and W < 0 { /* T broken */
ASSERT idle(T)
} ELSE IF idle(T) {
IF off-hook(H) ASSERT dialtone(T)
} ELSE IF dialtone(T) {
IF on-hook(H) ASSERT idle(T) ...
} ELSE IF ringing(T, CLG = phone) {
IF off-hook(H) {
ASSERT voice-connection(CLG, T)
ASSERT voice-connection(T, CLG)
}
...
![Page 92: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/92.jpg)
PurchaseTicket planning agentpurchase-ticket(A, P) :- dress(A, purchase-ticket), RETRIEVE building-of(P, BLDG); near-reachable(A, BLDG), near-reachable(A, FINDO(box-office)), near-reachable(A, FINDO(customer-side-of-counter)),2: interjection-of-greeting(A, B = FINDO(human NEAR employee-side-of-counter)), WAIT FOR may-I-help-you(B, A) OR WAIT 10 seconds AND GOTO 2,5: request(A, B, P),6: WAIT FOR I-am-sorry(B) AND GOTO 13 OR WAIT FOR describe(B, A, TKT = ticket) AND GOTO 8 OR WAIT 20 seconds AND GOTO 5,8: WAIT FOR propose-transaction(B, A, TKT, PRC = currency), IF TKT and PRC are OK accept(A, B) AND GOTO 10 ELSE decline(A, B) AND GOTO 6,10: pay-in-person(A, B, PRC), receive-from(A, B, TKT) ON FAILURE GOTO 13, ASSERT owner-of(TKT, A), post-sequence(A, B), SUCCESS,13: post-sequence(A, B), FAILURE.
![Page 93: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/93.jpg)
WorkBoxOffice planning agentwork-box-office(B, F) :- dress(B, work-box-office), near-reachable(B, F), TKTBOX = FINDO(ticket-box); near-reachable(B, FINDO(employee-side-of-counter)),100: WAIT FOR attend(A = human, B) OR pre-sequence(A = human, B), may-I-help-you(B, A),103: WAIT FOR request(A, B, R) AND GOTO 104 OR WAIT FOR post-sequence(A, B) AND GOTO 110,104: IF R ISA tod { current-time-sentence(B, A) ON COMPLETION GOTO 103 } ELSE IF R ISA performance { GOTO 105 } ELSE { interjection-of-noncomprehension(B, A) ON COMPLETION GOTO 103 }105: find next available ticket TKT in TKTBOX for R; IF none { I-am-sorry(B, A) ON COMPLETION GOTO 103 } ELSE { describe(B, A, TKT) ON COMPLETION GOTO 106 },106: propose-transaction(B, A, TKT, TKT.price), WAIT FOR accept(A, B) AND GOTO 108 OR WAIT FOR decline(A, B) AND GOTO 105 OR WAIT 10 seconds AND GOTO 105,108: collect-payment(B, A, TKT.price, FINDO(cash-register)),109: hand-to(B, A, TKT),110: post-sequence(B, A) ON COMPLETION GOTO 100.
![Page 94: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/94.jpg)
Natural language processing:the basics
Text agency: word, phrase, name, time and date expression, phone number, media object, product, price, end of sentence, communicon, email header, attribution, table
Lexicon: part of speech, language, dialect, argument structure, selectional restriction, subcategorization restriction, inflection, prefix, suffix, derivational rule
Syntactic component: syntactic parser, constituent, noun phrase, verb phrase, sentence, base rule, filter, barrier, transformation, compound tense, relative clause
![Page 95: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/95.jpg)
Natural language processing:the basics
Semantic component: semantic parser, case frame, semantic Cartesian product, theta marking, argument, adjunct, copula, relative clause, appositive, genitive, nominalization, conjunction, tense, aspect, anaphoric parser, antecedent, salience, feature unification, article, intension, extension, c-command, deixis, speaker, listener, story time, now
Generator: English, French, indicative, subjunctive, generation advice, unit of measure, value range names
Question answering: Yes-No question, question-word question, location, time, temporal relation, degree, quantity, description, means, reason, explanation, clarification, narrowing down
![Page 96: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/96.jpg)
A lexical entry definition=pour-on//pour* on+.Véz/|r1=human|r2=physical-object|r3=physical-object|
+ = takes indirect objectV = verbé = takes direct objectz = English
![Page 97: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/97.jpg)
ú = subject assigned to slot 2ü = subject assigned to slot 3è = object assigned to slot 1é = object assigned to slot 2ë = object assigned to slot 3÷ = indicative (that) he goesO = subjunctive (that) he goÏ = infinitive (for him) to go± = present participle (him) going...
![Page 98: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/98.jpg)
Semantic parsingThe American who daydreams, ate.
[preterit-indicative [ingest [such-that [definite-article human] [present-indicative [daydream human na]] [nationality-of human US]] food]]
![Page 99: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/99.jpg)
Semantic parsingI want to buy a Fiat Spyder.
[present-indicative [active-goal subject-pronoun [buy subject-pronoun na Fiat-Spider na]]]
[present-indicative [active-goal Jim [buy Jim na Fiat-Spider na]]]
![Page 100: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/100.jpg)
Semantic parsing
How are you?
[how-are-you Jim TT]
What is your address?
[present-indicative [email-address-of TT object-interrogative-pronoun]]
![Page 101: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/101.jpg)
Anaphoric parsing Resolves I, you based on speaker,
listener Resolves him, her, they based on
previously mentioned actors Resolves the X based on objects of type
X previously mentioned or spatially near previously mentioned actors
Resolves the red X or the X that is red based on red X’s
![Page 102: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/102.jpg)
He poured Pert Plus on his hair.
[preterit-indicative [pour [such-that grasper [of grasper subject-pronoun]] Pert-Plus [possessive-determiner hair]]]
[preterit-indicative [pour Jim-left-hand Pert-Plus Jim-head-hair]]
![Page 103: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/103.jpg)
Implementation of a UAvoid UA_Emotion_FortunesOfOthers(Actor *ac, Ts *ts, Obj *a, Obj *in, Obj *other, Float weight, Obj *other_emot_class){ int found; Float weight1; Obj *other_emot_class1; ObjList *causes, *objs, *atts, *p, *q; /* Relate <a's emotion to <a's attitudes. */ if (0.0 != (weight1 = UA_FriendAttitude(ts, a, other, 1, &atts))) { if (FloatSign(weight1) == FloatSign(weight)) { /* The input emotion agrees with known attitudes. */ ContextSetRSN(ac-cx, RELEVANCE_TOTAL, SENSE_TOTAL, NOVELTY_HALF); ContextAddMakeSenseReasons(ac-cx, atts); } else { /* The input emotion disagrees with known attitudes. */ ContextSetRSN(ac-cx, RELEVANCE_TOTAL, SENSE_LITTLE, NOVELTY_TOTAL); ContextAddNotMakeSenseReasons(ac-cx, atts); } } else { /* Attitude of <a toward <other is unknown. */ ContextSetRSN(ac-cx, RELEVANCE_TOTAL, SENSE_MOSTLY, NOVELTY_MOSTLY); UA_Infer(ac-cx-dc, ac-cx, ts, L(N("like-human"), a, other, NumberToObj(weight), E), in); }...
![Page 104: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/104.jpg)
More stories handled by ThoughtTreasure
![Page 105: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/105.jpg)
GROCER and SPACE UAsJenny Powell was a grocer.
spin GROCER PA to AWAIT-CUSTOMERJenny walks to checkout counter
At seven am she stepped out into the street.Jenny walks to street
###############################################################** wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww## ** w scssw## *** ssssw## ** sFssw## w* ssssw## w* ttt ssssSssss ssssw## w * tttc sssssssssssssssssssssssssssss ssssw## w s*ctt sssssssssssssssssssssssssssss sCssw## w ttt C l ssssw## w swssw## w ssssw## w ssssw## w ssssw## wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww###############################################################
![Page 106: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/106.jpg)
Question answering
Q: Where was Jenny’s left foot?A: Her left foot was in the corner grocery.
Q: What salamis was she near?A: She was near Danish salami.
Q: When did she step out into the street?A: She walked to the street at seven am.
![Page 107: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/107.jpg)
SLEEP and SHOWER UAsJim was sleeping.
spin SLEEP PA to ASLEEPJim lies down in bed
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&& wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww&& wwwwwwwwwww tttwsss ht Aswcccccccccw&& wwwwwwwwwwwwwwwwwww w tttwmss wcccccccccw&&wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwccccccccw w w smwmss wcc w&&w llllllll ss g Ecccc ssssssssswccccccccw w w smwmss wcv w&&wa llllllll ss s ssssssssswccccccccw wwww wwwww wwmss wcc w&&wA Jc sswccccccccw wwwww wwwwwwwwww wwww&&w cc sswccccccccw w&&wr sswwwwwwwwww w&&wr w ************************************** w&&wr **wwwwwwwwwwwwwwwwwwwwwwwwwttttttttttttl* lw&&w * ssw wttttttttttttt * tw&&w ** ssw wttttttttttttt * w&&w J w ** ssw wA s *JJJJJJJJw&&w w** Rsw w JJJJJJJJJw&&w s w sw wls JJJJJJJJJw&&w ttttttttt wwwwww wwww w s JJJJJJJJJw&&wr sttttttttts ss ssw w w s JJJJJJJJJw&&wr ttttttttt ss ssw w JJJJJJJJJw&&wr s ss Aw w JJJJJJJJJw&&w s ss w w E JJJJJJJJJw&&w tttt sssssssssss ss rrw w ttttttJt JJJJJJJJJw&&w A sssssssssss ss rrw w w rrrrr rrrrr Aw&&wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
![Page 108: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/108.jpg)
He woke up.spin SLEEP PA to AWAKE
He poured Pert Plus on his hair.spin SHOWER PA to READY-TO-LATHER
Jim walks to showerJim turns on showerJim pours shampoo on hair
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&& wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww&& wwwwwwwwwww tttwsss ht * Aswcccccccccw&& wwwwwwwwwwwwwwwwwww w tttwmss * wcccccccccw&&wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwccccccccw w w smwmss * wcc w&&w llllllll ss g Ecccc ssssssssswccccccccw w w smwmss * wcv w&&wa llllllll ss s ssssssssswccccccccw wwww wwwww wwmss * wcc w&&wA Jc sswccccccccw wwwww *wwwwwwwwww wwww&&w cc sswccccccccw *** w&&wr sswwwwwwwwww * w&&wr w * w&&wr wwwwwwwwwwwwwwwwwwwwwwwwwttttttttttttl* lw&&w ssw wttttttttttttt * tw&&w ssw wttttttttttttt * w&&w J w ssw wA s *JJJJJJJJw&&w w Rsw w JJJJJJJJJw&&w s w sw wls JJJJJJJJJw&&w ttttttttt wwwwww wwww w s JJJJJJJJJw&&wr sttttttttts ss ssw w w s JJJJJJJJJw&&wr ttttttttt ss ssw w JJJJJJJJJw&&wr s ss Aw w JJJJJJJJJw&&w s ss w w E JJJJJJJJJw&&w tttt sssssssssss ss rrw w ttttttJt JJJJJJJJJw&&w A sssssssssss ss rrw w w rrrrr rrrrr Aw&&wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
![Page 109: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/109.jpg)
Question answering UAs
Q: Where was Jim?
LOCATION QUESTION understanding agent
A: He was in the bedroom while sleeping. He was in the bathroom while taking a shower.
![Page 110: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/110.jpg)
Understanding agency: Image
Emotion UA
Goal UA
Friend UA
TakeShower UA
Sleep UA
Time UA
Relation UA
Space UA
![Page 111: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/111.jpg)
Understanding agency: Reality
![Page 112: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/112.jpg)
Complex agent dependencies
Space UA
Emotion UA Goal UA
freezing outside
nice location happy falling
prevent fall
buy flowers florist
success happy
pain withdraw
![Page 113: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/113.jpg)
Complex updates
Jim went to sleep.1. Jim walks to bed.2. Jim lies down.3. Jim falls asleep.
![Page 114: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/114.jpg)
Many possible language inputs
Mary is sleeping.Mary is lying awake in her bed.Mary was lying asleep in her bed.Mary was asleep and Jim did not want to wake her.At ten in the morning, Mary was still asleep.Mary had only slept a few hours.…
![Page 115: ThoughtTreasure, the hard common sense problem, and applications of common sense Erik T. Mueller IBM Research*](https://reader031.fdocuments.us/reader031/viewer/2022013004/56649e455503460f94b3a1f9/html5/thumbnails/115.jpg)
ThoughtTreasure wish list
Better concept and lexical entry coverage Better coverage of stereotypical locations Better script coverage Better coverage of typical object properties Script recognition Script-based word sense disambiguation Dynamic location generation English annotation of knowledge base Graphical tools for commonsense knowledge entry Graphical display of simulation Compound noun understanding Question-based understanding Metaplanning