Enhancing learning through innovative technologies: observations from UCS & beyond
Learning from Observations Chapter 18 Section 1 – 4.
-
Upload
ariel-abbot -
Category
Documents
-
view
221 -
download
0
Transcript of Learning from Observations Chapter 18 Section 1 – 4.
Learning from Observations
Chapter 18
Section 1 – 4
Outline
• Learning agents
• Inductive learning
• Decision tree learning
• Boosting
This is about the best “off-the-shelves” classification algorithm!(and you’ll get to know it)
Learning agents
Sometimes we want to invest time and effort to observe the feedback from our environment to our actions in order to improve these actionsso we can more effectively optimize our utility in the future.
Learning element
• Design of a learning element is affected by– Which components of the performance element are to
be learned (e.g. learn to stop for traffic light)– What feedback is available to learn these components
(e.g. visual feedback form camera)– What representation is used for the components (e.g. logic, probabilistic descriptions, attributes,...)
• Type of feedback:– Supervised learning: correct answers for each
example (label).– Unsupervised learning: correct answers not given.– Reinforcement learning: occasional rewards
Two Examples of Learning Object Categories.Here is your training set (2 classes):
Here is your test set:Does it belong to one of the above classes?
S. Savarese, 2003Copied from P. Perona
talk slides.
Learning from 1 Example
P. Buegel, 1562
Inductive learning
• Simplest form: learn a function from examples
f is the target function
An example is a pair (x, f(x))
Problem: find a hypothesis h
such that h ≈ fgiven a training set of examples
Inductive learning method
• Construct/adjust h to agree with f on training set• (h is consistent if it agrees with f on all examples)• E.g., curve fitting:
Inductive learning method
• Construct/adjust h to agree with f on training set• (h is consistent if it agrees with f on all examples)• E.g., curve fitting:
Inductive learning method
Inductive learning method
Inductive learning method
which curve is best?
• Ockham’s razor: prefer the simplest hypothesis consistent with data
Inductive learning method
Decision Trees
Problem: decide whether to wait for a table at a restaurant, based on the following attributes:
1. Alternate: is there an alternative restaurant nearby?2. Bar: is there a comfortable bar area to wait in?3. Fri/Sat: is today Friday or Saturday?4. Hungry: are we hungry?5. Patrons: number of people in the restaurant (None, Some, Full)6. Price: price range ($, $$, $$$)7. Raining: is it raining outside?8. Reservation: have we made a reservation?9. Type: kind of restaurant (French, Italian, Thai, Burger)10. WaitEstimate: estimated waiting time (0-10, 10-30, 30-60, >60)
Attribute-based representations
• Examples described by attribute values (Boolean, discrete, continuous)
• E.g., situations where I will/won't wait for a table:
• Classification of examples is positive (T) or negative (F)
• General form for data: a number N of instances, each with attributes (x1,x2,x3,...xd) and target value y.
Decision trees• One possible representation for hypotheses• We imagine someone taking a sequence of decisions.• E.g., here is the “true” tree for deciding whether to wait:
Note you can use the same attribute more than once.
Expressiveness• Decision trees can express any function of the input attributes.• E.g., for Boolean functions, truth table row → path to leaf:
• Trivially, there is a consistent decision tree for any training set with one path to leaf for each example (unless f nondeterministic in x) but it probably won't generalize to new examples
• Prefer to find more compact decision trees: we don’t want to memorize the data, we want to find structure in the data!
Hypothesis spaces
How many distinct decision trees with n Boolean attributes?
= number of Boolean functions
= number of distinct truth tables with 2n rows = 22n
• E.g., with 6 Boolean attributes, there are 18,446,744,073,709,551,616 trees
n=2: 2^2 = 4 rows. For each row we can choose T or F: 2^4 functions.
Decision tree learning• If there are so many possible trees, can we actually
search this space? (solution: greedy search).
• Aim: find a small tree consistent with the training examples
• Idea: (recursively) choose "most significant" attribute as root of (sub)tree.
Choosing an attribute
• Idea: a good attribute splits the examples into subsets that are (ideally) "all positive" or "all negative"
• Patrons or type? To wait or not to wait is still at 50%.
Using information theory
• Entropy measures the amount of uncertainty
in a probability distribution:
Consider tossing a biased coin.If you toss the coin VERY often,the frequency of heads is, say, p, and hence the frequency of tails is 1-p. (fair coin p=0.5).
The uncertainty in any actual outcomeis given by the entropy:
Note, the uncertainty is zero if p=0 or 1and maximal if we have p=0.5.
Using information theory
• If there are more than two states s=1,2,..n we have (e.g. a die):
( ) ( 1)log[ ( 1)]
( 2)log[ ( 2)]
...
( )log[ ( )]
Entropy p p s p s
p s p s
p s n p s n
1
( ) 1n
s
p s
Using information theory
• Imagine we have p examples which are true (positive) and n examples which are false (negative).
• Our best estimate of true or false is given by:
• Hence the entropy is given by:
( , ) log logp p pn n n
Entropyp n p n p n p n p n p n
( ) /
( ) /
P true p p n
p false n p n
Using Information Theory
• How much information do we gain if we disclose the value of some attribute?
• Answer:
uncertainty before minus uncertainty after
Example
Before: Entropy = - ½ log(1/2) – ½ log(1/2)=log(2) = 1 bit: There is “1 bit of information to be discovered”.
After: for Type: If we go into branch “French” we have 1 bit, similarly for the others.French: 1bitItalian: 1 bitThai: 1 bitBurger: 1bit
After: for Patrons: In branch “None” and “Some” entropy = 0!, In “Full” entropy = -1/3log(1/3)-2/3log(2/3).
So Patrons gains more information!
On average: 1 bit ! We gained nothing!
Information Gain• How do we combine branches:
1/6 of the time we enter “None”, so we weight“None” with 1/6. Similarly: “Some” has weight: 1/3 and “Full” has weight ½.
1
( ) ( , )n
i i i i
i i i i i
p n p nEntropy A Entropy
p n p n p n
weight for each branch
entropy for each branch.
Information gain
• Information Gain (IG) or reduction in entropy from the attribute test:
• Choose the attribute with the largest IG
( )I G A Entropy before Entropy af ter
Information gain
For the training set, p = n = 6, I(6/12, 6/12) = 1 bit
Patrons has the highest IG of all attributes and so is chosen by the DTL algorithm as the root
bits 0)]4
2,
4
2(
12
4)
4
2,
4
2(
12
4)
2
1,
2
1(
12
2)
2
1,
2
1(
12
2[1)(
bits 0541.)]6
4,
6
2(
12
6)0,1(
12
4)1,0(
12
2[1)(
IIIITypeIG
IIIPatronsIG
Example contd.
• Decision tree learned from the 12 examples:
• Substantially simpler than “true” tree---a more complex hypothesis isn’t justified by small amount of data
What to Do if...
• In some leaf there are no examples:
Choose True or False according to the number of positive/negative examples at your parent.
• There are no attributes left
Two or more examples have the same attributes but different label: we have an error/noise. Stop and use majority vote.
Demo:http://www.cs.ubc.ca/labs/lci/CIspace/Version4/dTree/
Boosting
• Main idea: – train classifiers (e.g. decision trees) in a sequence.– a new classifier should focus on those cases which
were incorrectly classified in the last round.– combine the classifiers by letting them vote on the
final prediction.– each classifier could be (should be) very “weak”,
e.g. a DT, with only one node: decision stump.
Example
this line is one simple classifier saying that everything to the left + and everything to the rightis -
Decision Stump
• Data: {(X1,Y1),(X2,Y2),....,(Xn,Yn)}
attributes (e.g. temperature outside)
label (e.g. True or False, 0 or 1 -1 or +1)
Xi
threshold-1
+11( )
1i
i
if Xf x
if X
The Algorithm in Detail
(Z,H) = AdaBoost[Xtrain,Ytrain,Rounds]
weights = 1/N; (N = # datacases)
• For r=1 to Rounds Do• H[r] = LearnWeakClassifier[Xtrain,Ytrain,weights];• error = 0;• For i=1 to N Do
•If H[r,Xtrain(i)]=Ytrain(i) error = error + weight(i);• For i=1 to N Do
•If H[r,Xtrain(i)]=Ytrain(i) weight(i)=weight(i) error/(1-error);• Normalize Weights• Z[r] = log[(1-error)/error]
Final classifier:
1
( ) ( ) ( , )R
r
F x Z r H r x
( , ) 2 ( ) ( ( )) ( ) 0 1H r x sign r X axis r r
And in a Picture
training casecorrectlyclassified
training casehas large weightin this round
this DT has a strong vote.
And in animation
Original Training set : Equal Weights to all training samples
Taken from “A Tutorial on Boosting” by Yoav Freund and Rob Schapire
AdaBoost(Example)
ROUND 1
AdaBoost(Example)
ROUND 2
AdaBoost(Example)
ROUND 3
AdaBoost(Example)
k-Nearest Neighbors • Another simple classification algorithm
• Idea: Look around you too see how your neighbors classify data.
• Classify a new data-point according to a majority vote of your k-nearest neighbors.
k-NNdecision line
Distance Metric
• How do we measure what it means to be “close”?
• Depending on the problem we should choose an appropriate distance metric.
( , ) | | { discrete}; Hamming distance
( , ) ( ) ( ) { .}; Scaled EuclideanDistancen m n m
Tn m n m n m
D x x x x x
D x x x x A x x x cont
Algorithm in Detail
• For a new test example:
• Find the k closest neighbors
• Each neighbor predicts the class to be it sown class
• Take majority vote.
Demo: http://www.comp.lancs.ac.uk/~kristof/research/notes/nearb/cluster.html
Demo: MATLAB
Logistic Regression / Perceptron
• Fits a soft decision boundary between the classes.
1 dimension 2 dimensions
The logit / sigmoid
)exp(1
1)(
bWxxf
Determines the offset
Determines the angleand the steepness.
Objective
• We interpret f(x) as the probability of classifying a data case as positive.
• We want to maximize the total probability of the data-vectors:
)1( )0(
)(1log)(log
n nyexamplespositive
yexamplesnegative
nn xfxfO
Algorithm in detail
• Repeat until convergence (gradient descend):
b
Obb
W
OWW
)0()1(
)0()1(
)()(1
)()(1
nn
nn
yexamplesnegative
n
yexamplespositive
n
n
yexamplesnegative
n
yexamplespositive
nn
xfxfb
O
xxfxxfW
O
(Or better: Newton steps)
Cross-validation
• You are ultimately interested in good performance on new (unseen) test data.
• To estimate that, split of a (smallish) subset of the training data (called validation set).
• Train without validation data and “test” on validation data.
• This will give an indication of how long to run boosting.
Project
• Demo
• For this project I will obtain data from a company who want to predict activity of a chemical compound.
• Training data will be provided
• Test data will not!
• May the best win this competition.
Summary• Learning agent = performance element + learning
element
• For supervised learning, the aim is to find a simple hypothesis approximately consistent with training examples
• Decision tree learning + boosting
• Learning performance = prediction accuracy measured on test set