Post on 17-Nov-2014
description
November 11, 2009 1Artificial Intelligence
An Introduction to
Natural Language Processingand
Machine Learning
Karthik SankarDepartment of CSE
NIT Trichy
Department of CSE, NIT Trichy
November 11, 2009 2Artificial Intelligence
Natural Language Processing
Department of CSE, NIT Trichy
November 11, 2009 3
Artificial Intelligence
A lot of human communication is by means of natural language
So computers could be a ton more useful if they could read our email, do ourlibrary research, chat to us, do all of these things involve dealing with naturallanguage
They're pretty good at dealing with machine languages that are made for them,but human languages, not so.
“Look. The computer just can't deal with the kind of stuff that humans produce,and how they naturally interact”
We’re exploiting human cleverness rather than working out how to havecomputer cleverness.
Department of CSE, NIT Trichy
November 11, 2009 4
Artificial Intelligence
NLP is a field of computer science and linguistics concerned with the interactions between computers and human (natural) languages
Department of CSE, NIT Trichy
Definition
Phonology - study of speech sounds Morphology - study of meaningful components of words Syntax - study of structural relationships between words Semantics - study of meaning
Categories
November 11, 2009 5
Artificial Intelligence
Modeling the pronunciation of a word as a string of symbols – PHONES
Articulatory Phonetics: How phones are produced as the various organs in the mouth, throat and nose modify the airflow from the lungs.
CanChairCoach
Department of CSE, NIT Trichy
Phonology
Syllables
November 11, 2009 6
Artificial Intelligence
Identification, analysis and description of the structure of words.
InflectionsNumberTenseCaseGenderPerson
Word Formationmother in lawhot dog
Finite State MachinesFinite State Transducers
Department of CSE, NIT Trichy
Morphology
dog/dogs : goose/geesehunt – huntedhis - hers
November 11, 2009 7
Artificial Intelligence
Part of Speech TaggingNounVerbAdjective …
I can write – aux. verb OR verb OR noun
Context Free Grammars
Department of CSE, NIT Trichy
Syntax
November 11, 2009 8
Artificial Intelligence
Understanding and representing the meaning
Department of CSE, NIT Trichy
Semantics
having
Who has What does he have
First Order Predicate Calculus
Has(Ram, book)
November 11, 2009 9
Artificial Intelligence
Adjective: the adjectives are associated with which of the two nouns ?“pretty little girls' school”
Pronoun: which noun does ‘they’ relate to ?We gave the monkeys the bananas because they were hungry.We gave the monkeys the bananas because they were over-ripe.
Emphasis: notice the change in meaning due to the change in stressI never said she stole my moneyI never said she stole my moneyI never said she stole my moneyI never said she stole my moneyI never said she stole my moneyI never said she stole my moneyI never said she stole my money
Department of CSE, NIT Trichy
Ambiguity
November 11, 2009 10
Artificial Intelligence
Fed raises interest rates half a percent in effort to control inflation
Department of CSE, NIT Trichy
Ambiguity - contd
She rates highlyOur water rates are high
Japanese movies interest meThe interest rate is 8 percent
Fed raisesThe raises we received was small
November 11, 2009 11
Artificial Intelligence
Department of CSE, NIT Trichy
Resolving Ambiguity
Part of Speech Tagging
Word Sense Disambiguation
Probabilistic Parsing
Speech Act Interpretation
November 11, 2009 12
Artificial Intelligence
Department of CSE, NIT Trichy
Perceptions
Perception provides agents with information about the world they inhabit.
Perception is initiated by sensors.
A sensor is anything that can record some aspect of the environment and pass itas input to an agent program.
The sensor could be as simple as a one-bit sensor that detects whether a switchis on or off or as complex as the retina of the human eye, which contains morethan a hundred million photosensitive elements
Image processing Computer Vision Speech recognition Facial recognition Object recognition
November 11, 2009 13
Artificial Intelligence
Department of CSE, NIT Trichy
Applications
Information retrieval & Web SearchInformation retrieval (IR) is the science of searching for documents,for information within documents, and for metadata about documents, as wellas that of searching databases and the World Wide Web.
Information Extraction Information extraction (IE) is a type of information retrieval whose goal is toautomatically extract structured information, i.e. categorized and contextuallyand semantically well-defined data from a certain domain, fromunstructured machine-readable documents
Question AnsweringType in keywords to Asking Questions in Natural Language.Response from documents to extracted or generated answer
Text SummarizationProcess of distilling most important information from a source to produce anabridged version
Machine Translationuse of computer software to translate text or speech from one natural language to another.
November 11, 2009 14
Artificial Intelligence
Department of CSE, NIT Trichy
Applications
Speech - recognition & synthesisDeriving a textual representation of a spoken utterance
Natural Language understanding and generationNLG system is like a translator that converts a computer based representationinto a natural language representation.
Human - Computer ConversationDialogue between humans and computers using natural language.
Text GenerationA method for generating sentences from “keywords” or “headwords”.
Hand writing recognitionAbility of a computer to receive and interpret intelligible handwritten inputfrom sources such as paper documents, photographs, touch-screens and otherdevices
November 11, 2009 15Artificial Intelligence
Machine Learning
Department of CSE, NIT Trichy
November 11, 2009 16
Artificial Intelligence
Department of CSE, NIT Trichy
Machine Learning
The ability to learn
Learning something new Learning something new about something you already knew Learning how to do something better, either more efficiently or with more
accuracy
A system can improve its problem solving accuracy (and possibly efficiency) bylearning how to do something better
November 11, 2009 17
Artificial Intelligence
Department of CSE, NIT Trichy
Types of Machine Learning - 1
SymbolicExplicitly represented Domain knowledge
Sub-Symbolic or Connectionist Networks• Neural Networks• simulate the structure and/or functional aspects of
biological neural networks• Simple processing elements (neurons), which can
exhibit complex global behaviour, determined bythe connections between the processing elementsand element parameters
Genetic and Evolutionary LearningLearning through adaptation
November 11, 2009 18
Artificial Intelligence
Department of CSE, NIT Trichy
Types of Machine Learning - 2 - “is there a teacher ???”
SupervisedTraining data is available
UnsupervisedTraining data is not available. Self learning process
Reinforcementhow an agent ought to take actions in an environment so as to maximize somenotion of long-term reward
November 11, 2009 19
Artificial Intelligence
Department of CSE, NIT Trichy
Types of Machine Learning - 3
Knowledge acquisition
Learning through problem solving
Explanation based learning
Analogy
November 11, 2009 20
Artificial Intelligence
Department of CSE, NIT Trichy
Framework for Symbol Based Learning
Data and the goals of the learning task
The representation of Learned Language
A set of operations
The concept space
Heuristic Search
November 11, 2009 21
Artificial Intelligence
Department of CSE, NIT Trichy
Framework for Symbol Based Learning
November 11, 2009 22
Artificial Intelligence
Department of CSE, NIT Trichy
Example - the goal is to build an arch
positive
negativenegative
positive
November 11, 2009 23
Artificial Intelligence
Department of CSE, NIT Trichy
Example
November 11, 2009 24
Artificial Intelligence
Department of CSE, NIT Trichy
Example
November 11, 2009 25
Artificial Intelligence
Department of CSE, NIT Trichy
Example
November 11, 2009 26
Artificial Intelligence
Department of CSE, NIT Trichy
Version Space Search
Concept space
November 11, 2009 27
Artificial Intelligence
Department of CSE, NIT Trichy
Version Space Search
Generalization Operations
Color(ball, red) generalizes to Color(X, red)
Shape(X, round) ^ Size(X, small) ^ Color(X, red)generalizes to Shape(X, round) ^ Size(X, small)
Covering
p covers q
November 11, 2009 28
Artificial Intelligence
Department of CSE, NIT Trichy
Version Space Search
Candidate Elimination Algorithm
Specific to general direction
General to specific direction
Bi-directional
November 11, 2009 29
Artificial Intelligence
Department of CSE, NIT Trichy
Version Space Search
Role of negative examples
November 11, 2009 30
Artificial Intelligence
Department of CSE, NIT Trichy
Version Space Search - Specific to general direction
November 11, 2009 31
Artificial Intelligence
Department of CSE, NIT Trichy
Version Space Search - Specific to general direction - example
November 11, 2009 32
Artificial Intelligence
Department of CSE, NIT Trichy
Version Space Search - General to specific direction
November 11, 2009 33
Artificial Intelligence
Department of CSE, NIT Trichy
Version Space Search - General to specific direction - example
November 11, 2009 34
Artificial Intelligence
Department of CSE, NIT Trichy
November 11, 2009 35
Artificial Intelligence
Department of CSE, NIT Trichy
Version Space Search
How the algorithm works
November 11, 2009 36
Artificial Intelligence
Department of CSE, NIT Trichy
Thank you