Statistical Learning (From data to distributions)
description
Transcript of Statistical Learning (From data to distributions)
STATISTICAL LEARNING(FROM DATA TO DISTRIBUTIONS)
AGENDA Learning a discrete probability distribution
from data Maximum likelihood estimation (MLE) Maximum a posteriori (MAP) estimation
MOTIVATION Agent has made observations (data) Now must make sense of it (hypotheses)
Hypotheses alone may be important (e.g., in basic science)
For inference (e.g., forecasting) To take sensible actions (decision making)
A basic component of economics, social and hard sciences, engineering, …
MACHINE LEARNING VS. STATISTICS Machine Learning automated statistics This lecture
Bayesian learning Maximum likelihood (ML) learning Maximum a posteriori (MAP) learning Learning Bayes Nets (R&N 20.1-3)
Future lectures try to do more with even less data Decision tree learning Neural nets Support vector machines …
PUTTING GENERAL PRINCIPLES TO PRACTICE Note: this lecture will cover general principles
of statistical learning on toy problems Grounded in some of the most theoretically
principled approaches to learning But the techniques are far more general Practical applications to larger problems requires
a bit of mathematical savvy and “elbow grease”
BEAUTIFUL RESULTS IN MATH GIVE RISE TO SIMPLE IDEAS
OR
HOW TO JUMP THROUGH HOOPS IN ORDER TO JUSTIFY SIMPLE IDEAS …
CANDY EXAMPLE• Candy comes in 2 flavors, cherry and lime, with identical
wrappers• Manufacturer makes 5 indistinguishable bags
• Suppose we draw• What bag are we holding? What flavor will we draw
next?
h1C: 100%L: 0%
h2C: 75%L: 25%
h3C: 50%L: 50%
h4C: 25%L: 75%
h5C: 0%L: 100%
BAYESIAN LEARNING Main idea: Compute the probability of each
hypothesis, given the data Data d: Hypotheses: h1,…,h5
h1C: 100%L: 0%
h2C: 75%L: 25%
h3C: 50%L: 50%
h4C: 25%L: 75%
h5C: 0%L: 100%
BAYESIAN LEARNING Main idea: Compute the probability of each
hypothesis, given the data Data d: Hypotheses: h1,…,h5
h1C: 100%L: 0%
h2C: 75%L: 25%
h3C: 50%L: 50%
h4C: 25%L: 75%
h5C: 0%L: 100%
P(hi|d)
P(d|hi)
We want this…
But all we have is this!
USING BAYES’ RULE P(hi|d) = a P(d|hi) P(hi) is the posterior
(Recall, 1/a = P(d) = Si P(d|hi) P(hi)) P(d|hi) is the likelihood P(hi) is the hypothesis prior
h1C: 100%L: 0%
h2C: 75%L: 25%
h3C: 50%L: 50%
h4C: 25%L: 75%
h5C: 0%L: 100%
LIKELIHOOD AND PRIOR Likelihood is the probability of observing the
data, given the hypothesis model
Hypothesis prior is the probability of a hypothesis, before having observed any data
COMPUTING THE POSTERIOR Assume draws are independent Let P(h1),…,P(h5) = (0.1, 0.2, 0.4, 0.2, 0.1) d = { 10 x }
P(d|h1) = 0P(d|h2) = 0.2510
P(d|h3) = 0.510
P(d|h4) = 0.7510
P(d|h5) = 110
P(d|h1)P(h1)=0P(d|h2)P(h2)=9e-8P(d|h3)P(h3)=4e-4P(d|h4)P(h4)=0.011P(d|h5)P(h5)=0.1
P(h1|d) =0P(h2|d) =0.00P(h3|d) =0.00P(h4|d) =0.10P(h5|d) =0.90
Sum = 1/a = 0.1114
POSTERIOR HYPOTHESES
PREDICTING THE NEXT DRAW P(X|d) = Si P(X|hi,d)P(hi|d)
= Si P(X|hi)P(hi|d)
P(h1|d) =0P(h2|d) =0.00P(h3|d) =0.00P(h4|d) =0.10P(h5|d) =0.90
H
D X
P(X|h1) =0P(X|h2) =0.25P(X|h3) =0.5P(X|h4) =0.75P(X|h5) =1
Probability that next candy drawn is a lime
P(X|d) = 0.975
P(NEXT CANDY IS LIME | D)
PROPERTIES OF BAYESIAN LEARNING If exactly one hypothesis is correct, then the
posterior probability of the correct hypothesis will tend toward 1 as more data is observed
The effect of the prior distribution decreases as more data is observed
HYPOTHESIS SPACES OFTEN INTRACTABLE To learn a probability distribution, a
hypothesis would have to be a joint probability table over state variables 2n entries => hypothesis space is 2n-1-
dimensional! 2^(2n) deterministic hypotheses
6 boolean variables => over 1022 hypotheses And what the heck would a prior be?
LEARNING COIN FLIPS Let the unknown fraction of cherries be q
(hypothesis) Probability of drawing a cherry is q Suppose draws are independent and
identically distributed (i.i.d) Observe that c out of N draws are cherries
(data)
LEARNING COIN FLIPS Let the unknown fraction of cherries be q
(hypothesis) Intuition: c/N might be a good hypothesis (or it might not, depending on the draw!)
MAXIMUM LIKELIHOOD Likelihood of data d={d1,…,dN} given q
P(d|q) = Pj P(dj|q) = qc (1-q)N-c
i.i.d assumption Gather c cherry terms together, then N-c lime terms
MAXIMUM LIKELIHOOD Likelihood of data d={d1,…,dN} given q
P(d|q) = qc (1-q)N-c
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
1.2
1/1 cherry
q
P(da
ta|q
)
MAXIMUM LIKELIHOOD Likelihood of data d={d1,…,dN} given q
P(d|q) = qc (1-q)N-c
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
1.2
2/2 cherry
q
P(da
ta|q
)
MAXIMUM LIKELIHOOD Likelihood of data d={d1,…,dN} given q
P(d|q) = qc (1-q)N-c
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.020.040.060.080.1
0.120.140.16
2/3 cherry
q
P(da
ta|q
)
MAXIMUM LIKELIHOOD Likelihood of data d={d1,…,dN} given q
P(d|q) = qc (1-q)N-c
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.01
0.02
0.03
0.04
0.05
0.06
0.07
2/4 cherry
q
P(da
ta|q
)
MAXIMUM LIKELIHOOD Likelihood of data d={d1,…,dN} given q
P(d|q) = qc (1-q)N-c
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.0050.01
0.0150.02
0.0250.03
0.0350.04
2/5 cherry
q
P(da
ta|q
)
MAXIMUM LIKELIHOOD Likelihood of data d={d1,…,dN} given q
P(d|q) = qc (1-q)N-c
0 0.10.20.30.40.50.60.70.80.9 10
0.0000002
0.0000004
0.0000006
0.0000008
0.000001
0.0000012
10/20 cherry
q
P(da
ta|q
)
MAXIMUM LIKELIHOOD Likelihood of data d={d1,…,dN} given q
P(d|q) = qc (1-q)N-c
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
1E-312E-313E-314E-315E-316E-317E-318E-319E-31
50/100 cherry
q
P(da
ta|q
)
MAXIMUM LIKELIHOOD Peaks of likelihood function seem to hover
around the fraction of cherries… Sharpness indicates some notion of
certainty…
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
1E-312E-313E-314E-315E-316E-317E-318E-319E-31
50/100 cherry
q
P(da
ta|q
)
MAXIMUM LIKELIHOOD P(d|q) be the likelihood function The quantity argmaxq P(d|q) is known as the
maximum likelihood estimate (MLE)
MAXIMUM LIKELIHOOD The maximum of P(d|q) is obtained at the
same place that the maximum of log P(d|q) is obtained Log is a monotonically increasing function
So the MLE is the same as maximizing log likelihood… but: Multiplications turn into additions We don’t have to deal with such tiny numbers
MAXIMUM LIKELIHOOD The maximum of P(d|q) is obtained at the
same place that the maximum of log P(d|q) is obtained Log is a monotonically increasing function
l(q) = log P(d|q) = log [ qc (1-q)N-c]
MAXIMUM LIKELIHOOD The maximum of P(d|q) is obtained at the
same place that the maximum of log P(d|q) is obtained Log is a monotonically increasing function
l(q) = log P(d|q) = log [ qc (1-q)N-c]= log [ qc ] + log [(1-q)N-c]
MAXIMUM LIKELIHOOD The maximum of P(d|q) is obtained at the
same place that the maximum of log P(d|q) is obtained Log is a monotonically increasing function
l(q) = log P(d|q) = log [ qc (1-q)N-c]= log [ qc ] + log [(1-q)N-c]= c log q + (N-c) log (1-q)
MAXIMUM LIKELIHOOD The maximum of P(d|q) is obtained at the
same place that the maximum of log P(d|q) is obtained Log is a monotonically increasing function
l(q) = log P(d|q) = c log q + (N-c) log (1-q) At a maximum of a function, its derivative is 0 So, dl/dq(q) = 0 at the maximum likelihood estimate
MAXIMUM LIKELIHOOD The maximum of P(d|q) is obtained at the
same place that the maximum of log P(d|q) is obtained Log is a monotonically increasing function
l(q) = log P(d|q) = c log q + (N-c) log (1-q) At a maximum of a function, its derivative is 0 So, dl/dq(q) = 0 at the maximum likelihood estimate => 0 = c/q – (N-c)/(1-q)
=> q = c/N
MAXIMUM LIKELIHOOD FOR BN For any BN, the ML parameters of any CPT
can be derived by the fraction of observed values in the data, conditioned on matched parent values
Alarm
Earthquake Burglar
E: 500 B: 200
N=1000
P(E) = 0.5 P(B) = 0.2
A|E,B: 19/20A|B: 188/200A|E: 170/500A| : 1/380
E B P(A|E,B)T T 0.95F T 0.95T F 0.34F F 0.003
OTHER MLE RESULTS Multi-valued variables: take fraction of counts
for each value Continuous Gaussian distributions: take
average value as mean, standard deviation of data as standard deviation
MAXIMUM LIKELIHOOD PROPERTIES As the number of data points approaches
infinity, the MLE approaches the true estimate
With little data, MLEs can vary wildly
MAXIMUM LIKELIHOOD IN CANDY BAG EXAMPLE hML = argmaxhi P(d|hi) P(X|d) P(X|hML)
hML = undefined h5
P(X|hML)
P(X|d)
BACK TO COIN FLIPS The MLE is easy to compute… but what
about those small sample sizes? Motivation
You hand me a coin from your pocket 1 flip, turns up tails Whats the MLE?
MAXIMUM A POSTERIORI ESTIMATION Maximum a posteriori (MAP) estimation Idea: use the hypothesis prior to get a better
initial estimate than ML, without resorting to full Bayesian estimation “Most coins I’ve seen have been fair coins, so I
won’t let the first few tails sway my estimate much”
“Now that I’ve seen 100 tails in a row, I’m pretty sure it’s not a fair coin anymore”
MAXIMUM A POSTERIORI P(q|d) is the posterior probability of the
hypothesis, given the data argmaxq P(q|d) is known as the maximum a
posteriori (MAP) estimate Posterior of hypothesis q given data d={d1,
…,dN} P(q|d) = 1/a P(d|q) P(q) Max over q doesn’t affect a So MAP estimate is argmaxq P(d|q) P(q)
MAXIMUM A POSTERIORI hMAP = argmaxhi P(hi|d) P(X|d) P(X|hMAP)
hMAP = h3 h4 h5
P(X|hMAP)
P(X|d)
ADVANTAGES OF MAP AND MLE OVER BAYESIAN ESTIMATION Involves an optimization rather than a large
summation Local search techniques
For some types of distributions, there are closed-form solutions that are easily computed
BACK TO COIN FLIPS Need some prior distribution P(q) P(q|d) P(d|q)P(q) = qc (1-q)N-c P(q)
Define, for all q, the probability that I believe in q
10 q
P(q)
MAP ESTIMATE Could maximize qc (1-q)N-c P(q) using some
optimization Turns out for some families of P(q), the MAP
estimate is easy to compute
10 q
P(q)
Beta distributions
BETA DISTRIBUTION Betaa,b(q) = g qa-1 (1-q)b-1
a, b hyperparameters > 0 g is a normalization
constant a=b=1 is uniform
distribution
POSTERIOR WITH BETA PRIORPosterior qc (1-q)N-c P(q)
= g qc+a-1 (1-q)N-c+b-1
MAP estimateq=(c+a-1)/(N+a+b-2)
Posterior is also abeta distribution!
POSTERIOR WITH BETA PRIORWhat does this mean? Prior specifies a “virtual
count” of a=a-1 heads, b=b-1 tailsSee heads, increment aSee tails, increment b
MAP estimate is a/(a+b)Effect of prior diminishes
with more data
MAP FOR BN For any BN, the MAP parameters of any CPT
can be derived by the fraction of observed values in the data + the prior virtual counts
Alarm
Earthquake Burglar
E: 500+100 B: 200+200
N=1000+1000
P(E) = 0.5 0.3 P(B) = 0.2 0.2
A|E,B: (19+100)/(20+100)A|B: (188+180)/(200+200)A|E: (170+100)/(500+400)A| : (1+3)/(380 + 300)
E B P(A|E,B)T T 0.95 0.99F T 0.94 0.92T F 0.34 0.3
F F 0.0030.006
EXTENSIONS OF BETA PRIORS Multiple discrete values: Dirichlet prior
Mathematical expression more complex, but in practice still takes the form of “virtual counts”
Mean, standard deviation for Gaussian distributions: Gamma prior
Conjugate priors preserve the representation of prior and posterior distributions, but do not necessary exist for general distributions
RECAP Bayesian learning: infer the entire
distribution over hypotheses Robust but expensive for large problems
Maximum Likelihood (ML): optimize likelihood Good for large datasets, not robust for small
ones Maximum A Posteriori (MAP): weight
likelihood by a hypothesis prior during optimization A compromise solution for small datasets Like Bayesian learning, how to specify the prior?
APPLICATION TO STRUCTURE LEARNING So far we’ve assumed a BN structure, which
assumes we have enough domain knowledge to declare strict independence relationships
Problem: how to learn a sparse probabilistic model whose structure is unknown?
BAYESIAN STRUCTURE LEARNING But what if the Bayesian network has
unknown structure? Are Earthquake and Burglar independent?
P(E,B) = P(E)P(B)?
Earthquake Burglar
?B E0 01 10 11 0… …
BAYESIAN STRUCTURE LEARNING Hypothesis 1: dependent
P(E,B) as a probability table has 3 free parameters
Hypothesis 2: independent Individual tables P(E),P(B) have 2 total free
parameters Maximum likelihood will always prefer
hypothesis 1!Earthquake Burglar
B E0 01 10 11 0… …
BAYESIAN STRUCTURE LEARNING Occam’s razor: Prefer simpler explanations
over complex ones Penalize free parameters in probability
tables
Earthquake Burglar
B E0 01 10 11 0… …
ITERATIVE GREEDY ALGORITHM Fit a simple model (ML), compute likelihood Repeat:
Earthquake Burglar
B E0 01 10 11 0… …
ITERATIVE GREEDY ALGORITHM Fit a simple model (ML), compute likelihood Repeat:
Pick a candidate edge to add to the BN
Earthquake Burglar
B E0 01 10 11 0… …
ITERATIVE GREEDY ALGORITHM Fit a simple model (ML), compute likelihood Repeat:
Pick a candidate edge to add to the BN Compute new ML probabilities
Earthquake Burglar
B E0 01 10 11 0… …
ITERATIVE GREEDY ALGORITHM Fit a simple model (ML), compute likelihood Repeat:
Pick a candidate edge to add to the BN Compute new ML probabilities If new likelihood exceeds old likelihood +
threshold, Add the edge
Otherwise, repeat with a different edge Earthquake Burglar
B E0 01 10 11 0… …
(Usually log likelihood)
OTHER TOPICS IN LEARNING DISTRIBUTIONS Learning continuous distributions using
kernel density estimation / nearest neighbor smoothing
Expectation Maximization for hidden variables Learning Bayes nets with hidden variables Learning HMMs from observations Learning Gaussian Mixture Models of continuous
data
NEXT TIME Introduction to machine learning R&N 18.1-2