Natural Language Processing

• what it does• what is involved• why is it difficult• brief history

sentence structured repns of meaning"how old is my help3.doc file?"

Lisp: (query (file-detail 'date "C:/help3.doc"))

"the large cat chased the rat"Logic: (1s1 large(s1) feline(s1))

(1s2 rodent(s2)) chased(s1, s2 )

"the young boy ate a bad apple" CD Graph ...see next page...

CD graph "the young boy ate a bad apple"

what is involved

symbolic computationie: symbols manipulated by symbol processors

search & inference

knowledge representation techniques

prejudice, politics, etc

ambiguity...• syntactic• semantic• pragmatic

why is it difficult

example sentences

• the old man the boats• my car drinks petrol• I saw the Eiffel Tower flying to Paris• he opened the door with the key

he opened the door with the squeaking hinge• the boy kicked the ball under the tree

the boy kicked the wall under the tree• put the bottles in the box on the shelf by the door

1950s Russian English translation1956 Chomsky1960s Pattern matching1970s Parsing & some KnRep1980s Kn & inference1990s big dreams small results2000+ quietly promising

(brief) history of language processing

matching:Sir

matching:Student

matching:Elisa

a modernapproach

inputsentence

syntaxanalysis(parsing)

semanticanalysis

pragmaticanalysis

targetrepresentation

grammar

lexicon

semanticrules

contextualinformation

morphologicalprocessing

step 1- morphological processing

objective: strip words into roots & modifiers

issues• inflection (cat pl cat-s)• derivation (happy adj happiness noun)• compounding (toothpaste)

morphological processing - notes

• all(?) spoken lngs exhibit morphology• easier to handle in written lngs if not iconic• some morphology describes infm beyond syntax

eg: proximity (Tamil, Setswana, etc)casespeaker / listener peer relationship

morphology examples

Noun + Suffix Syntactic case Meaning Chennai-ukku dative: destination To Madras Chennai-ukku-irundu dative: source From Madras Chennai-le containment In Madras Chennai-ai object (formal) Madras Fig. 2.2. Suffix Attachment for Noun Cases (Tamil author's spelling)

Proximity Time Things (inanimate) Near i-ppa (this time: now) i-ndtha (this thing: this) Far a-ppa (that time: then) a-ndtha (that thing: that) Question e-ppa (what time: when) e-ndtha (what thing: which) Fig. 2.3. Proximity Information as Prefix Tags (Tamil)

Proximity Cow Student Near Speaker kgomo e (this cow) mo-ithuti yo (this student) Near Listener kgomo e-o (that cow) mo-ithuti yo-o (that student) Far kgomo e-le (that cow) mo-ithuti yo-le (that student) Fig. 2.4. Proximity by Demostrative Pronoun Inflection (Setswana)

step 2- syntax analysis

objectives: 1 check for correctness2 produce phrase structure

uses• parser a rule-based search engine• grammar context-free production rules• lexicon dictionary of words & their

categories

syntax rules

parts of speechrules of combination

consider• the cat chases the mouse• all large black dogs chase cats

example 1 - using Lkit

(build-lexicon

'((a determiner) (cat noun) (dog noun)

(the determiner) (chased verb)))

(build-grammar

'((s1 (sentence -> noun-phrase verb-phrase))

(np (noun-phrase -> determiner noun))

(vp (verb-phrase -> verb noun-phrase))

))

example 1 - output

(parse 'sentence '(the dog chased a cat))

complete-edge 0 5 s1 sentence (the dog ...) nil

s1 sentence -> (noun-phrase verb-phrase)

Syntax

(sentence

(noun-phrase (determiner the) (noun dog))

(verb-phrase

(verb chased)

(noun-phrase (determiner a) (noun cat))))

Semantics

(sentence)

so what ?

we want meaning

Remember: "the young boy ate a bad apple"

how can semantics be encoded as symbols?the boy / an apple?young/old, happy/sad, good/bad?how can semantics be generated?what can be inferred from semantics?

Reminder: "the young boy ate a bad apple"

symbolic representation of semantics

(actor (root boy) (id boy#732)

(tags animate human male)

(qual (age (val 5) (approx 3)))

(quant specific))

(action (primitve INGEST))

(object (root apple) (id nil)

(tags physob veg fruit food)

(qual (phy-state -4))

(quant non-specific))

semantics in lexicon

a simple example

(build-lexicon

'((a det any )

(cat noun feline )

(chased verb hunts )

(dog noun canine )

(the det specific)

))

semantics in grammar rules

(s1 (sentence -> noun-phrase verb-phrase)

(actor . noun-phrase)

(action . verb-phrase.action)

(object . verb-phrase.object)

)

(np (noun-phrase -> det noun)

(det . noun)

)

(vp (verb-phrase -> verb noun-phrase)

(action . verb)

(object . noun-phrase)

)

semantics - results

(parse 'sentence '(the dog chased a cat))

complete-edge 0 5 s1 sentence (the dog...) nil

s1 sentence -> (noun-phrase verb-phrase)

Syntax

(sentence (noun-phrase (det the) (noun dog))

(verb-phrase (verb chased)

(noun-phrase (det a) (noun cat))))

Semantics

(sentence (actor (specific canine))

(action hunts)

(object (any feline)))

semantics in lexicon - checks 1

(a det (sems . any))

(all det (sems . every))

(cat noun (sems . feline) (num . sing))

(cats noun (sems . feline) (num . plur))

(chase verb (sems . hunts) (num . plur))

(chases verb (sems . hunts) (num . sing))

(dog noun (sems . canine) (num . sing))

(dogs noun (sems . canine) (num . plur))

(the det (sems . specific))

semantics in grammar - checks 1

(s1 (sentence -> noun-phrase verb-phrase)

(actor . noun-phrase.sems)

(action . verb-phrase.action)

(object . verb-phrase.object)

; check number of noun-phrase & verb-phrase

(if (noun-phrase.number

= verb-phrase.number)

numeric-agreement-ok

numeric-agreement-bad

)

)

semantics - results

(parse 'sentence '(the dog chases a cat))

complete-edge 0 5 s1 sentence (the dog...) nil

s1 sentence -> (noun-phrase verb-phrase)

Syntax

(sentence (noun-phrase (det the) (noun dog))

(verb-phrase (verb chases)

(noun-phrase (det a) (noun cat))))

Semantics

(sentence (actor specific canine)

(action . hunts)

(object any feline)

numeric-agreement-ok)

semantics - results

(parse 'sentence '(the dogs chases a cat))

complete-edge 0 5 s1 sentence (the dog...) nil

s1 sentence -> (noun-phrase verb-phrase)

Syntax

(sentence (noun-phrase (det the) (noun dog))

(verb-phrase (verb chases)

(noun-phrase (det a) (noun cat))))

Semantics

(sentence (actor specific canine)

(action . hunts)

(object any feline)

numeric-agreement-bad)

semantics in grammar - checks 2

(s1 (sentence -> noun-phrase verb-phrase)

(fail if noun-phrase.number

/= verb-phrase.number)

(actor . noun-phrase.sems)

(action . verb-phrase.action)

(object . verb-phrase.object)

)

semantics - results

(parse 'sentence '(the dog chases a cat))

Semantics

(sentence (actor specific canine)

(action . hunts)

(object any feline))

(parse 'sentence '(the dogs chases a cat))

.... failed ....

semantics in grammar - checks 3

(s1 (sentence -> noun-phrase verb-phrase)

(glitch numeric-agreement

if not noun-phrase.number

= verb-phrase.number)

(actor . noun-phrase.sems)

(action . verb-phrase.action)

(object . verb-phrase.object)

)

semantics - results

(parse 'sentence '(the dogs chases a cat))

complete-edge 0 5 s1 sentence (the dogs...) nil

Glitches: (numeric-agreement)

s1 sentence -> (noun-phrase verb-phrase)

Syntax

(sentence (noun-phrase (det the) (noun dogs))

(verb-phrase (verb chases)

(noun-phrase (det a) (noun cat))))

Semantics

(sentence (actor specific canine)

(action . hunts)

(object any feline))

example 2 - lexicon

(a det any )

(cat noun feline )

(chase verb hunts )

(dog noun canine )

(the det specific)

(black adj (color black))

(large adj (size 7/10))

(small adj (size 3/10))

example 2 - grammar

(build-grammar

'((np (noun-phrase -> ?det *adj noun)

(if det

(quantification . det)

(quantification undefined))

(qualifiers . *.adj)

(object . noun)

))

))

example 2 - results

(parse 'noun-phrase '(small black dog))

complete-edge 0 3 np noun-phrase (small...) nil

np noun-phrase -> (?det *adj noun)

Syntax

(noun-phrase (adj small) (adj black) (noun dog))

Semantics

(noun-phrase

(quantification undefined)

(qualifiers ((size . 3/10)) ((color . black)))

(object canine))

example 2 - resultssmall dogs chase the small cats

and large dogs chase the large cats

(sentence conjunction

((actor (quant undefined) (qual (size . 3/10))

(object . canine))

(action . hunts)

(object (quant . specific) (qual (size . 3/10))

(object . feline)))

((actor (quant undefined) (qual (size . 7/10))

(object . canine))

(action . hunts)

(object (quant . specific) (qual (size . 7/10))

(object . feline))))

semantic processing (one approach)

• semantic rules in grammar 1st stage case frame• verb form primitive action case frame• disambiguate & fill additional case frame slots• check references with world and/or dialog• do statement level inference• integrate with dialog• do event sequence dialog

step-1: produce raw case frame

• verb casesthe cat chased the rat in the kitchenthe cat chased the rat into the kitchen

• common casessource start-time instrumentdestination end-time beneficiarylocation duration

the ambiguity problem

eg: the boy kicked the ball under the tree

grammar rulesS S PPS NP VPNP ?det *adj nounNP NP PP

example frame #1

actor (quant specific)

(tags animate male human)

(qual (age (range 3 13)))

(root boy)

action (root kick)

object (root ball)

(tags manip)

(posn-relative

(locator beneath)

(object (root tree)

...etc... )

example frame #2

actor (quant specific)

(tags animate male human)

(qual (age (range 3 13)))

(root boy)

action (root kick)

object (root ball)

(tags manip)

dest (posn-relative

(locator beneath)

(object (root tree)

...etc... )

example verb form #1

primitive strike

prohibited object (tags manip)

slots instrument (part-of $actor foot)

legal start-time, end-time, duration

instrument, beneficiary, location

illegal source, dest

example verb form #2

primitive push

required object (tags manip)

slots instrument (part-of $actor foot)

legal source, dest, start-time, end-time,

instr, beneficiary, locatn, duration

semantic processing (one approach)

× semantic rules in grammar 1st stage case frame× verb form primitive action case frame× disambiguate & fill additional case frame slots check references with world and/or dialog do statement level inference• integrate with dialog• do event sequence dialog

integration with dialog

dialogs have...• players (actors)• props (objects)• locations (from case frames)• themes (derived)• event sequences (from themes)• plans (from themes and/or derived)

event sequence

set of...• players (actors)• props (objects)

series of...• semantically encoded activities (matched)• escapes, exceptions & alternatives

reading – grammars, etc

 A good source of links & references...“Computational Analysis of Prepositions” http://knol.google.com/k/abdul-baqi-sharaf/computational-analysis-of-prepositions/3hc3uny2z7r41/4# if you only plan to read one article...Baldwin, T. Kordoni, V and Villavicencio, A. 2009. Prepositions in Applications: A Survey and Introduction to the Special Issue ". Computational Linguistics 35 (2): 119–149. also...

Litkowski, Kenneth C. and Orin Hargraves. 2007. SemEval-2007 task 06: Word-sense disambiguation of prepositions. In Proceedings of the 4th International Workshop on Semantic Evaluations, pages 24–29, Prague. Disambiguation of Preposition Sense Using Linguistically MotivatedFeatures, Stephen Tratz and Dirk Hovy. Proceedings of the NAACL HLT Student Research Workshop and Doctoral Consortium, pages 96–100,Boulder, Colorado, June 2009. c 2009 Association for Computational Linguistics 

reading – grammars, etc

 the NLP dictionary: www.cse.unsw.edu.au/~billw/nlpdict.html for practical help with building grammars check the following (it is about 10 years old but then so is the English language :o)A Grammar Writer’s Cookbook. Miriam Butt, Tracy Holloway King, Marma-Eugenia Niño and Fridirique Segond also (for writing larger grammars) it is useful to find a book on grammar for tutors and/or students of English as a second language.

for a broad (if a little formal) take on semantics try dipping into...Semantics-Oriented Natural Language Processing Mathematical Models and Algorithms. Vladimir Fomichov A. 2010

 

reading – kn rep for NLP

logic and knowledge representation – a guidehttp://dspace.dsto.defence.gov.au/dspace/bitstream/1947/9996/1/DSTO-TR-

2324%20PR.pdf representing events for NLPhttp://www.google.co.uk/url?sa=t&rct=j&q=knowledge%20representation

%20%22representing%20events%22&source=web&cd=6&sqi=2&ved=0CEgQFjAF&url=http%3A%2F%2Fwww.aaai.org%2Focs%2Findex.php%2FFSS%2FFSS10%2Fpaper%2Fdownload%2F2183%2F2819&ei=f6oWT_e7DeKC4gTMpaijBA&usg=AFQjCNFYmurwJR9oqfCRBimVprWRK45kew&cad=rja

 semantic networks & frames (2005)http://www.cs.bham.ac.uk/~jxb/IAI/w6.pdf VERL: An Ontology Framework for Video Events (2005)http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1524892