CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10,...
-
Upload
laurel-arnold -
Category
Documents
-
view
217 -
download
0
Transcript of CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10,...
![Page 1: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/1.jpg)
CPSC 422, Lecture 34 Slide 1
Intelligent Systems (AI-2)
Computer Science cpsc422, Lecture 34
Apr, 10, 2015Slide source: from Pedro Domingos UW & Markov Logic: An Interface Layer for Artificial Intelligence Pedro Domingos and Daniel Lowd University of Washington, Seattle
![Page 2: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/2.jpg)
CPSC 422, Lecture 34 2
Lecture Overview• Finish Inference in MLN
• Probability of a formula, Conditional Probability
• Markov Logic: applications• (422) Highlights from IUI conference• Watson…. • TA evaluation / Teaching Evaluations• Final Exam (office hours, samples)
![Page 3: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/3.jpg)
Markov Logic: Definition
A Markov Logic Network (MLN) is a set of pairs (F, w) where
F is a formula in first-order logic w is a real number
Together with a set C of constants, It defines a Markov network with
One binary node for each grounding of each predicate in the MLN
One feature/factor for each grounding of each formula F in the MLN, with the corresponding weight w
CPSC 422, Lecture 34 3
Grounding: substituting vars with constants
![Page 4: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/4.jpg)
MLN features
)()(),(,
)()(
ySmokesxSmokesyxFriendsyx
xCancerxSmokesx
1.1
5.1
Cancer(A)
Smokes(A)Friends(A,A)
Friends(B,A)
Smokes(B)
Friends(A,B)
Cancer(B)
Friends(B,B)
Two constants: Anna (A) and Bob (B)
CPSC 422, Lecture 34 4
![Page 5: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/5.jpg)
Computing Cond. Probabilities
Let’s look at the simplest case
P(ground literal | conjuction of ground literals, ML,C)
CPSC 422, Lecture 34 5
P(Cancer(B)| Smokes(A), Friends(A, B), Friends(B, A) )
To answer this query do you need to create (ground) the whole network?
Cancer(A)
Smokes(A)Friends(A,A)
Friends(B,A)
Smokes(B)
Friends(A,B)
Cancer(B)
Friends(B,B)
)()(),(,
)()(
ySmokesxSmokesyxFriendsyx
xCancerxSmokesx
1.1
5.1
![Page 6: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/6.jpg)
Computing Cond. ProbabilitiesLet’s look at the simplest case
P(ground literal | conjuction of ground literals, ML,C)
CPSC 422, Lecture 34 6
P(Cancer(B)| Smokes(A), Friends(A, B), Friends(B, A) )
You do not need to create (ground) the part of the Markov Network from which the query is independent given the evidence
![Page 7: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/7.jpg)
Computing Cond. Probabilities
CPSC 422, Lecture 34 7
P(Cancer(B)| Smokes(A), Friends(A, B), Friends(B, A) )
You can then perform Gibbs Sampling in
this Sub Network
The sub network is determined by the formulas
(the logical structure of the problem)
)()(),(,
)()(
ySmokesxSmokesyxFriendsyx
xCancerxSmokesx
1.1
5.1
![Page 8: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/8.jpg)
CPSC 422, Lecture 34 8
Lecture Overview• Finish Inference in MLN
• Probability of a formula, Conditional Probability
• Markov Logic: applications• (422) Highlights from IUI conference• Watson…. • TA evaluation / Teaching Evaluations• Final Exam (office hours, samples)
![Page 9: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/9.jpg)
9
Entity Resolution
CPSC 422, Lecture 34
• Determining which observations correspond to the same real-world objects
• (e.g., database records, noun phrases, video regions, etc)
• Crucial importance in many areas (e.g., data cleaning, NLP, Vision)
![Page 10: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/10.jpg)
10
Entity Resolution: ExampleAUTHOR: H. POON & P. DOMINGOSTITLE: UNSUPERVISED SEMANTIC PARSINGVENUE: EMNLP-09
AUTHOR: Hoifung Poon and Pedro DomingsTITLE: Unsupervised semantic parsingVENUE: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing
AUTHOR: Poon, Hoifung and Domings, PedroTITLE: Unsupervised ontology induction from textVENUE: Proceedings of the Forty-Eighth Annual Meeting of the Association for Computational Linguistics
AUTHOR: H. Poon, P. DomingsTITLE: Unsupervised ontology inductionVENUE: ACL-10
SAME?
SAME?
CPSC 422, Lecture 34
![Page 11: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/11.jpg)
11
Problem: Given citation database, find duplicate recordsEach citation has author, title, and venue fieldsWe have 10 relations
Author(bib,author)Title(bib,title)Venue(bib,venue)
HasWord(author, word)HasWord(title, word)HasWord(venue, word)
SameAuthor (author, author)SameTitle(title, title)SameVenue(venue, venue)
SameBib(bib, bib)
Entity Resolution (relations)
CPSC 422, Lecture 34
indicate which words are present in each field;
represent field equality;
represents citation equality;
relate citations to their fields
![Page 12: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/12.jpg)
12
Predict citation equality based on words in the fields
Title(b1, t1) Title(b2, t2) ∧ ∧HasWord(t1,+word) HasWord(t2,+word) ∧ ⇒SameBib(b1, b2)
(NOTE: +word is a shortcut notation, you actually have a rule for each word e.g., Title(b1, t1) Title(b2, t2) ∧ ∧HasWord(t1,”bayesian”) ∧HasWord(t2,”bayesian” ) SameBib(b1, b2) )⇒
Same 1000s of rules for author
Same 1000s of rules for venue
Entity Resolution (formulas)
CPSC 422, Lecture 34
![Page 13: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/13.jpg)
13
Transitive closureSameBib(b1,b2) SameBib(b2,b3) SameBib(b1,b3)∧ ⇒
SameAuthor(a1,a2) SameAuthor(a2,a3) SameAuthor(a1,a3)∧ ⇒Same rule for titleSame rule for venue
Entity Resolution (formulas)
CPSC 422, Lecture 34
Link fields equivalence to citation equivalence – e.g., if two citations are the same, their authors should be the same Author(b1, a1) Author(b2, a2) SameBib(b1, b2) ∧ ∧ ⇒SameAuthor(a1, a2)…and that citations with the same author are more likely to be the sameAuthor(b1, a1) Author(b2, a2) SameAuthor(a1, a2) ∧ ∧ SameBib(b1, b2)⇒
Same rules for titleSame rules for venue
![Page 14: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/14.jpg)
Benefits of MLN model
CPSC 422, Lecture 34 14
Standard non-MLN approach: build a classifier that given two citations tells you if they are the same or not, and then apply transitive closure
New MLN approach: • performs collective entity resolution, where
resolving one pair of entities helps to resolve pairs of related entities
e.g., inferring that a pair of citations are equivalent can provide evidence that the names AAAI-06 and 21st Natl. Conf. on AI refer to the same venue, even though they are superficially very different. This equivalence can then aid in resolving other entities.
![Page 15: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/15.jpg)
15
Other MLN applications
CPSC 422, Lecture 34
• Information Extraction
• Co-reference Resolution (see lecture 1!)
• Robot Mapping (infer the map of an indoor environment from laser range data)
• Link-based Clustering (uses relationships among the objects in determining similarity)
• Ontologies extraction from Text
• …..
![Page 16: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/16.jpg)
422 big picture
Query
Planning
Deterministic Stochastic
• Value Iteration• Approx.
Inference
• Full Resolution
• SAT
LogicsBelief Nets
Markov Decision Processes and
Partially Observable MDP
Markov Chains and HMMs
First Order LogicsDescription Logics/
OntologiesTemporal rep.
Applications of AI
More sophisticated reasoning
Undirected Graphical Models
Conditional Random Fields
Reinforcement Learning
Representation
ReasoningTechnique
Prob relational models
Prob CFGMarkov Logics
Hybrid
CPSC 422, Lecture 34 Slide 17
![Page 17: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/17.jpg)
422 big picture
Query
Planning
Deterministic Stochastic
• Value Iteration• Approx.
Inference
• Full Resolution
• SAT
LogicsBelief Nets
Markov Decision Processes and
Partially Observable MDP
Markov Chains and HMMs
First Order Logics
OntologiesTemporal rep.
Applications of AI
Approx. : Gibbs
Undirected Graphical Models
Conditional Random Fields
Reinforcement Learning
Representation
ReasoningTechnique
Prob CFGProb Relational
ModelsMarkov Logics
Hybrid: Det +Sto
Forward, Viterbi….Approx. : Particle
Filtering
CPSC 422, Lecture 34 Slide 18
![Page 18: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/18.jpg)
CPSC 422, Lecture 34 19
Lecture Overview• Finish Inference in MLN
• Probability of a formula, Conditional Probability
• Markov Logic: applications• (422) Highlights from IUI conference• Watson…. • TA evaluation / Teaching Evaluations• Final Exam (office hours, samples)
![Page 19: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/19.jpg)
CPSC 422, Lecture 34 Slide 20
AI and HCI meet
Keynote Speaker:Prof. Dan Weld, University of WashingtonIntelligent Control of CrowdsourcingCrowd-sourcing labor markets (e.g., Amazon Mechanical Turk) are booming, …… use of partially-observable Markov decision Processes (POMDPs) to control voting on binary-choice questions and iterative improvement workflows.
![Page 20: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/20.jpg)
Some papers from IUI
Unsupervised Modeling of Users' Interests from their Facebook Profiles and Activities (Page 191)Preeti Bhargava (University of Maryland)Oliver Brdiczka (Vectra Networks, Inc.)Michael Roberts (Palo Alto Research Center)
named entity recognition, document categorization, sentiment analysis, semantic relatedness and social tagging
Semantic Textual Similarity (STS) system [13] for computing the SR scores. STS is based on LSA along with WordNet knowledge and is trained on LDC Gigawords and Stanford Webbase corpora
CPSC 422, Lecture 34 Slide 21
![Page 21: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/21.jpg)
Some papers from IUI
BayesHeart: A Probabilistic Approach for Robust, Low-Latency Heart Rate Monitoring on Camera Phones Xiangmin Fan (University of Pittsburgh)Jingtao Wang (University of Pittsburgh)
BayesHeart is based on an adaptive hidden Markov model, requires minimal training data and is user-independent.
Two models, one with 2 states and one with 4 states, work in combination….
CPSC 422, Lecture 34 Slide 22
![Page 22: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/22.jpg)
Watson : analyzes natural language questions and content well enough and fast enough to compete and win against champion players at Jeopardy!
CPSC 422, Lecture 34 Slide 23Source:IBM
“This Drug has been shown to relieve the symptoms of ADD with relatively few side effects."
• 1000s of algorithms and KBs,
• 3 secs
![Page 23: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/23.jpg)
AI techniques in 422 / Watson• Parsing (PCFGs)• Shallow parsing (NP segmentation with CRFs)• Entity and relation Detection (NER with CRFs)• Logical Form Generation and Matching• Logical Temporal and Spatial Reasoning• Leveraging many databases, taxonomies,
and ontologies• Confidence…. Probabilities (Bnets to rank)• Strategy for playing Jeopardy…statistical
models of players and games, game-theoretic analyses … .. and application of reinforcement-learningCPSC 422, Lecture 34 Slide 24
![Page 24: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/24.jpg)
From silly project to $1 billion investment
2005-6 “IT’S a silly project to work on, it’s too gimmicky, it’s not a real computer-science test, and we probably can’t do it anyway.” These were reportedly the first reactions of the team of IBM researchers challenged to build a computer system capable of winning “Jeopardy!
CPSC 422, Lecture 34 Slide 25
On January 9th 2014, with much fanfare, the computing giant announced plans to invest $1 billion in a new division, IBM Watson Group. By the end of the year, the division expects to have a staff of 2,000 plus an army of external app developers …..Mike Rhodin, who will run the new division, calls it “one of the most significant innovations in the history of our company.” Ginni Rometty, IBM’s boss since early 2012, has reportedly predicted that it will be a $10 billion a year business within a decade.
………after 8-9 years…
![Page 25: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/25.jpg)
And interactive, collaborative question-answering / problem solving
More complex questions in the future…
Or something like: “Should Europe reduce its energy dependency from Russia and what would it take?”
CPSC 422, Lecture 34 Slide 27
![Page 26: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/26.jpg)
AI applications…….
• DeepQA• Robotics• Search Engines• Games• Tutoring Systems• Medicine / Finance / …..• …….
CPSC 422, Lecture 34 Slide 28
• Most companies are investing in AI and/or developing/adopting AI technologies
![Page 27: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/27.jpg)
CPSC 422, Lecture 34 29
Lecture Overview
• Finish Inference in MLN• Probability of a formula, Conditional
Probability
• Markov Logic: applications• (422) Highlights from IUI conference• Watson…. • TA evaluation / Teaching Evaluations• Final Exam (office hours, samples)
![Page 28: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/28.jpg)
CPSC 422, Lecture 34 Slide 30
TA evaluation
Issam Laradji [email protected]
Also if you have not done it yet, fill out the teaching evaluationshttps://eval.olt.ubc.ca/science. login to the site using your CWL
![Page 29: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/29.jpg)
CPSC 422, Lecture 34 Slide 31
• Learning Goals (look at the end of the slides for each lecture)
• Revise all the clicker questions, practice exercises, assignments and midterm
• Will post more practice material today• Office Hours – usual ones Me Mon 1-2, Issam
Fri 11-12• – if high demand we will add a few more.
• Can bring letter sized sheet of paper with anything written on it (double sided)
Final Exam, Sat, Apr 18, we will start at 12:00PM Location:
DMP 110How to prepare….
![Page 30: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/30.jpg)
32
Example: Coreference Resolution
Mentions of Obama are often headed by "Obama"Mentions of Obama are often headed by "President"Appositions usually refer to the same entity
Barack Obama, the 44th President of the United States, is the first African American to hold the office. ……
CPSC 422, Lecture 34
![Page 31: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/31.jpg)
33
Example: Coreference Resolution
1.5
0.8
100
Head(A,“Obama”)
MentionOf(A,Obama)
Head(A,“President”)
MentionOf(B,Obama)
Apposition(A,B)
Head(B,“Obama”)
Head(B,“President”)
Two mention constants: A and B
Apposition(B,A)
MentionOf ( ,Obama) Head( ,"Obama")
MentionOf ( ,Obama) Head( ,"President")
, , Apposition( , ) MentionOf ( , ) MentionOf ( , )
x x x
x x x
x y c x y x c y c
In general, they represent feature templates for Markov NetworksCPSC 422, Lecture 34
![Page 32: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/32.jpg)
34
Can also resolve fields:
HasToken(token,field,record)SameField(field,record,record)SameRecord(record,record)
HasToken(+t,+f,r) ^ HasToken(+t,+f,r’) => SameField(f,r,r’)SameField(f,r,r’) <=> SameRecord(r,r’)SameRecord(r,r’) ^ SameRecord(r’,r”) => SameRecord(r,r”)SameField(f,r,r’) ^ SameField(f,r’,r”) => SameField(f,r,r”)
More: P. Singla & P. Domingos, “Entity Resolution with Markov Logic”, in Proc. ICDM-2006.
Entity Resolution
CPSC 422, Lecture 34
![Page 33: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/33.jpg)
35
UNSUPERVISED SEMANTIC PARSING. H. POON & P. DOMINGOS. EMNLP-2009.
Unsupervised Semantic Parsing, Hoifung Poon and Pedro Domingos. Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing. Singapore: ACL.
Information Extraction
CPSC 422, Lecture 34
![Page 34: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/34.jpg)
36
UNSUPERVISED SEMANTIC PARSING. H. POON & P. DOMINGOS. EMNLP-2009.
Unsupervised Semantic Parsing, Hoifung Poon and Pedro Domingos. Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing. Singapore: ACL.
Information ExtractionAuthor Title Venue
SAME?
CPSC 422, Lecture 34
![Page 35: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/35.jpg)
37
Information Extraction
• Problem: Extract database from text orsemi-structured sources
• Example: Extract database of publications from citation list(s) (the “CiteSeer problem”)
• Two steps:– Segmentation:
Use HMM to assign tokens to fields– Entity resolution:
Use logistic regression and transitivityCPSC 422, Lecture 34
![Page 36: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/36.jpg)
38
Token(token, position, citation)InField(position, field!, citation)SameField(field, citation, citation)SameCit(citation, citation)
Token(+t,i,c) => InField(i,+f,c)InField(i,+f,c) ^ InField(i+1,+f,c)
Token(+t,i,c) ^ InField(i,+f,c) ^ Token(+t,i’,c’) ^ InField(i’,+f,c’) => SameField(+f,c,c’)SameField(+f,c,c’) <=> SameCit(c,c’)SameField(f,c,c’) ^ SameField(f,c’,c”) => SameField(f,c,c”)SameCit(c,c’) ^ SameCit(c’,c”) => SameCit(c,c”)
Information Extraction
CPSC 422, Lecture 34
![Page 37: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/37.jpg)
39
Token(token, position, citation)InField(position, field!, citation)SameField(field, citation, citation)SameCit(citation, citation)
Token(+t,i,c) => InField(i,+f,c)InField(i,+f,c) ^ !Token(“.”,i,c) ^ InField(i+1,+f,c)
Token(+t,i,c) ^ InField(i,+f,c) ^ Token(+t,i’,c’) ^ InField(i’,+f,c’) => SameField(+f,c,c’)SameField(+f,c,c’) <=> SameCit(c,c’)SameField(f,c,c’) ^ SameField(f,c’,c”) => SameField(f,c,c”)SameCit(c,c’) ^ SameCit(c’,c”) => SameCit(c,c”)
More: H. Poon & P. Domingos, “Joint Inference in InformationExtraction”, in Proc. AAAI-2007.
Information Extraction
CPSC 422, Lecture 34
![Page 38: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/38.jpg)
Inference in MLN• MLN acts as a template for a Markov
Network• We can always answer prob. queries
using standard Markov network inference methods on the instantiated network
• However, due to the size and complexity of the resulting network, this is often infeasible.
• Instead, we combine probabilistic methods with ideas from logical inference, including satisfiability and resolution.
• This leads to efficient methods that take full advantage of the logical structure.
CPSC 422, Lecture 34 40
![Page 39: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/39.jpg)
MAP Inference
• Reduces to finding the pw that maximizes the sum of weights of satisfied clauses
• Use weighted SAT solver(e.g., MaxWalkSAT [Kautz et al., 1997])
CPSC 422, Lecture 34 41
i
iipw
pwnw )(maxarg
Find most likely state of world
)(maxarg pwPpw
Probabilistic problem solved by logical inference method
![Page 40: CPSC 422, Lecture 34Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 34 Apr, 10, 2015 Slide source: from Pedro Domingos UW & Markov.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649ddc5503460f94ad41f4/html5/thumbnails/40.jpg)
Computing Probabilities
P(Formula,ML,C) = ?• Brute force: Sum probs. of possible
worlds where formula holds
• MCMC: Sample worlds, check formula holds
CPSC 422, Lecture 3442
F
CL
PW
M ,
FPWpw CLCL MpwPMFP ),(),( ,,
||
||),( , S
SMFP
S
S
FCL
F