Chapter 8-Learning (2)
-
Upload
divya-shetty -
Category
Documents
-
view
233 -
download
0
Transcript of Chapter 8-Learning (2)
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 1/46
Chapter 8: Learning
By, Safa Hamdare
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 2/46
Learning
• Learning is essential for unknown environments,– i.e., when designer lacks omniscience
• Learning is useful as a system constructionmethod,– i.e., expose the agent to reality rather than trying to
write it down
• Learning modifies the agent's decisionmechanisms to improve performance
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 3/46
Learning agents
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 4/46
Learning Agent• Four Components
1. Performance Element: collection of knowledgeand procedures to decide on the next action.
E.g. walking, turning, drawing, etc.
2. Learning Element: takes in feedback from thecritic and modifies the performance elementaccordingly.
3. Critic: provides the learning element withinformation on how well the agent is doing based ona fixed performance standard.
E.g. the audience
4. Problem Generator: provides the performance
element with suggestions on new actions to take.
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 5/46
Designing Learning element
• Design of a learning element is affected by4 major issues:1. Which components of the performance
element to improve2. The representation of those
components3. Available feedback
4. Prior knowledge
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 6/46
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 7/46
Types of Inductive Learning1. Supervised learning: Inputs and Outputs
available. For every input, the learner is provided witha target; that is, the environment tells the learner what itsresponse should
2. Unsupervised learning: no hint of correctoutcome. the learner receives no feedback from the
world at all– just examples (e.g. – the same figures asabove , without the labels)
3. Reinforcement learning: evaluation of action. thelearner receives feedback about the appropriateness of itsresponse. i.e. occasional rewards
M M MF F F
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 8/46
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 9/46
Inductive Learning
• Key idea:– To use specific examples to reach
general conclusions
• Given a set of examples, the systemtries to approximate the evaluationfunction.
• Also called Pure Inductive Inference .
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 10/46
Recognizing Handwritten Digits
Learning Agent
Training Examples
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 11/46
Bi
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 12/46
Bias• Bias: Any preference for one hypothesis
over another, beyond mere consistencywith the examples.
• Since there are almost always a large
number of possible consistent hypotheses,all learning algorithms exhibit some sortof bias.
• Example:
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 13/46
Formal Definition for Inductive Learning
• Simplest form: learn a function from
examples• Example: a pair (x, f(x)), where
– x is the input,
– f(x) is the output of the function /target functionapplied to x.
• hypothesis: a function “h” that
approximates f, given a set of examples.• Task of induction: Given a set of examples,
find a function h that approximates the true
evaluation function f.
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 14/46
Inductive learning - Example 1
• f (x) is the target function
• An example is a pair [x, f (x)]
• Learning task: find a hypothesis h such that h(x) f (x) given a
training set of examples D = {[xi, f (xi) ]}, i = 1,2,…, N
1)(,
0
0
1
0
10
1
1
1
xx f
1)(,
0
0
1
1
10
0
1
1
xx f
1)(,
0
1
0
0
10
1
1
1
xx f
Etc...
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 15/46
Inductive learning method
• Construct/adjust h to agree with f on training set• (h is consistent if it agrees with f on all examples
• E.g., curve fitting:
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 16/46
Inductive learning method
• Construct/adjust h to agree with f on training set
• (h is consistent if it agrees with f on all examples)
• E.g., curve fitting:
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 17/46
Inductive learning method
• Construct/adjust h to agree with f on training set
• (h is consistent if it agrees with f on all examples)
• E.g., curve fitting:
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 18/46
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 19/46
Inductive learning method• Construct/adjust h to agree with f on training set
• (h is consistent if it agrees with f on all examples)• E.g., curve fitting:
How do we choose from among multiple consistenthypothesis?
• Ockham’s razor: prefer the simplest hypothesis
consistent with data
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 20/46
Th R t t D i
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 21/46
The Restaurant Domain Attributes Goal
Example Fri Hun Pat Price Rain Res Type Est WillWait
X1 No Yes Some $$$ No Yes French 0-10 YesX2 No Yes Full $ No No Thai 30-60 No
X3 No No Some $ No No Burger 0-10 Yes
X4 Yes Yes Full $ No No Thai 10-30 Yes
X5 Yes No Full $$$ No Yes French >60 NoX6 No Yes Some $$ Yes Yes Italian 0-10 Yes
X7 No No None $ Yes No Burger 0-10 No
X8 No Yes Some $$ Yes Yes Thai 0-10 Yes
X9 Yes No Full $ Yes No Burger >60 No
X10 Yes Yes Full $$$ No Yes Italian 10-30 No
X11 No No None $ No No Thai 0-10 No
X12 Yes Yes Full $ No No Burger 30-60 Yes
Will we wait, or not?
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 22/46
S li i E l b T i A ib ( ’ )
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 23/46
Splitting Examples by Testing on Attributes (con’t)
+ X1, X3, X4, X6, X8, X12 (Positive examples) - X2, X5,
X7, X9, X10, X11 (Negative examples)
+ X1, X3, X4, X6, X8, X12 (Positive ex)
- X2, X5, X7, X9, X10, X11 (Negative ex)
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 24/46
Splitting Examples by Testing onAttributes (con’t)
+ X1, X3, X4, X6, X8, X12 (Positive examples) - X2, X5,
X7, X9, X10, X11 (Negative examples)
Patrons?
+- X7, X11
nonesome
full
+X1, X3, X6, X8- +X4, X12- X2, X5, X9, X10
No Yes
l E l b b ( ’ )
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 25/46
Splitting Examples by Testing on Attributes (con’t)
+ X1, X3, X4, X6, X8, X12 (Positive examples) - X2, X5,X7, X9, X10, X11 (Negative examples)
Yes No
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 26/46
Decision tree learning example
Induced tree (from examples)
D l l
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 27/46
Decision tree learning example
True tree
Goal Predicate:
Will wait for a table?
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 28/46
Patrons?
WaitEst?
Hungry?
Yes
nonesome
full
>6030-60 10-30
0-10
noyes
Logical Representation of a Path
r [Patrons(r, full) Wait_Estimate(r, 10-30) Hungry(r, yes)] Will_Wait(r)
Ch i ib
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 29/46
Choosing an attribute• Idea: a good attribute splits the examples into
subsets that are (ideally) "all positive" or "allnegative"
• Patrons? is a better choice
Wh M k G d A ib ?
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 30/46
Patrons?
+
- X7, X11
nonesome
full
+X1, X3, X6, X8
-
+X4, X12
- X2, X5, X9, X10
Type?
+ X1
- X5
French
Italian Thai
+X6
- X10
+X3, X12
- X7, X9
+ X4,X8
- X2, X11
Burger
What Makes a Good Attribute?
Better
Attribute
Not As
Good AnAttribute
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 31/46
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 32/46
Decision tree learning example
16
3ln6
36
3ln6
312
6
63ln
63
63ln
63
12
6Entropy
Alternate?
3 T, 3 F 3 T, 3 F
Yes No
Entropy decrease for Alternate= 1 – 1= 0
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 33/46
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 34/46
Decision tree learning example
98.07
3ln7
37
4ln7
412
7
53ln
53
52ln
52
12
5Entropy
Fri
2 T, 3 F 4 T, 3 F
Yes No
Entropy decrease for Fri/Sat= 1 – 0.98 = 0.02
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 35/46
Decision tree learning example
804.05
4ln5
45
1ln5
112
5
72ln
72
75ln
75
12
7Entropy
Hungry?
5 T, 2 F 1 T, 4 F
Yes No
Entropy decrease for Hungry= 1 – 0.804 = 0.19
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 36/46
Decision tree learning example
18
4ln8
48
4ln8
412
8
42ln
42
42ln
42
12
4Entropy
Raining?
2 T, 2 F 4 T, 4 F
Yes No
Entropy decrease for Raining= 1 – 1 = 0
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 37/46
Decision tree learning example
978.07
4ln7
47
3ln7
312
7
52ln
52
53ln
53
12
5Entropy
Reservation?
3 T, 2 F 3 T, 4 F
Yes No
Entropy decrease for Reservation = 1 – 0.978 = 0.02
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 38/46
Decision tree learning example
456.06
4ln6
46
2ln6
212
6
40ln
40
44ln
44
12
4
22ln
22
20ln
20
12
2Entropy
Patrons?
2 F
4 T
None Full
Entropy decrease for Patrons= 1 – 0.456 = 0.543
2 T, 4 F
Some
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 39/46
Decision tree learning example
77.04
3ln4
34
1ln4
112
4
20ln
20
22ln
22
12
2
63ln
63
63ln
63
12
6Entropy
Price
3 T, 3 F
2 T
$ $$$
Entropy decrease for Price = 1 – 0.77 = 0.23
1 T, 3 F
$$
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 40/46
Decision tree learning example
14
2ln4
24
2ln4
212
4
42ln
42
42ln
42
12
42
1ln
2
1
2
1ln
2
1
12
2
2
1ln
2
1
2
1ln
2
1
12
2Entropy
Type
1 T, 1 F
1 T, 1 F
French Burger
Entropy decrease for Type= 1 – 1 = 0
2 T, 2 FItalian
2 T, 2 F
Thai
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 41/46
Decision tree learning example
792.02
2ln2
22
0ln2
012
2
21ln
21
21ln
21
12
22
1ln
2
1
2
1ln
2
1
12
2
6
2ln
6
2
6
4ln
6
4
12
6Entropy
Est. waitingtime
4 T, 2 F
1 T, 1 F
0-10 > 60
Entropy decrease for Est = 1 – 0.792 = 0.21
2 F10-30
1 T, 1 F
30-60
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 42/46
Entropy for each AttributeEntropy decrease for Alternate= 1 – 1= 0
Entropy decrease for Bar= 1 – 1= 0
Entropy decrease for Fri/Sat= 1 – 0.98 = 0.02
Entropy decrease for Hungry= 1 – 0.804 = 0.19
Entropy decrease for Raining= 1 – 1 = 0
Entropy decrease for Reservation = 1 – 0.978 = 0.02
Entropy decrease for Patrons= 1 – 0.456 = 0.543
Entropy decrease for Price = 1 – 0.77 = 0.23
Entropy decrease for Est = 1 – 0.792 = 0.21
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 43/46
Decision tree learning example
Patrons?
2 F
4 T
None Full
Largest entropy decrease (0.543)achieved by splitting on Patrons.
2 T, 4 F
Some
X? Continue like this, making new splits,
always purifying nodes.
Next step
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 44/46
Next stepGiven Patrons as root node, the next attribute chosen is
Hungry?
33.02
2ln2
22
0ln2
012
24
2ln4
24
2ln4
212
4Entropy
Entropy decrease for Hungry= 1 – 0.33= 0.666
Decision tree learning
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 45/46
Decision tree learning• Aim: find a small tree consistent with the training
examples
• Idea: (recursively) choose "most significant" attributeas root of (sub)tree.
8/12/2019 Chapter 8-Learning (2)
http://slidepdf.com/reader/full/chapter-8-learning-2 46/46