Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6...
Transcript of Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6...
![Page 1: Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6 Jeongmin Lee Computer Science Department University of Pittsburgh CS 1675 Intro to Machine](https://reader034.fdocuments.us/reader034/viewer/2022052017/602f83514dcd066f2926a472/html5/thumbnails/1.jpg)
Week6JeongminLee
ComputerScienceDepartmentUniversityofPittsburgh
CS1675IntrotoMachineLearning– Recitation
![Page 2: Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6 Jeongmin Lee Computer Science Department University of Pittsburgh CS 1675 Intro to Machine](https://reader034.fdocuments.us/reader034/viewer/2022052017/602f83514dcd066f2926a472/html5/thumbnails/2.jpg)
Homework3
• Averagescore:85.27(std:20.34)•Median:92.50• Theproblemthatmanystudenthadmistake• DeriveMLestimateforexponentialdistribution(3b)• Creating3dplotforGaussiandistribution
![Page 3: Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6 Jeongmin Lee Computer Science Department University of Pittsburgh CS 1675 Intro to Machine](https://reader034.fdocuments.us/reader034/viewer/2022052017/602f83514dcd066f2926a472/html5/thumbnails/3.jpg)
Homework3
• DeriveMLestimateforexponentialdistribution(3b)
![Page 4: Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6 Jeongmin Lee Computer Science Department University of Pittsburgh CS 1675 Intro to Machine](https://reader034.fdocuments.us/reader034/viewer/2022052017/602f83514dcd066f2926a472/html5/thumbnails/4.jpg)
DensityEstimation
• GivenasetofobservationsX,estimateaprobabilitydistributionthatgeneratedX• (Assumption:)ObservationXisgeneratedfromanunknownprobabilitydistributionp(X)
![Page 5: Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6 Jeongmin Lee Computer Science Department University of Pittsburgh CS 1675 Intro to Machine](https://reader034.fdocuments.us/reader034/viewer/2022052017/602f83514dcd066f2926a472/html5/thumbnails/5.jpg)
TwoMethods
• MaximumLikelihoodEstimation(ML)<- ourfocus• BayesianParameterEstimation• MaximumAPosterioriEstimation(MAP)
![Page 6: Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6 Jeongmin Lee Computer Science Department University of Pittsburgh CS 1675 Intro to Machine](https://reader034.fdocuments.us/reader034/viewer/2022052017/602f83514dcd066f2926a472/html5/thumbnails/6.jpg)
MaximumLikelihoodEstimation(ML)
• Goal:Maximizethelikelihoodofdata
• Logismonotonicallyincreasingfunction
Θ"# = 𝑎𝑟𝑔𝑚𝑎𝑥*+,*-𝑝(𝐷|Θ, 𝜁)
Θ"# = 𝑎𝑟𝑔𝑚𝑎𝑥*+,*-𝑝 𝐷 Θ, 𝜁
= 𝑎𝑟𝑔𝑚𝑎𝑥*+,*- log 𝑝 𝐷 Θ, 𝜁
![Page 7: Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6 Jeongmin Lee Computer Science Department University of Pittsburgh CS 1675 Intro to Machine](https://reader034.fdocuments.us/reader034/viewer/2022052017/602f83514dcd066f2926a472/html5/thumbnails/7.jpg)
MaximumLikelihoodEstimation(ML)
©Hauskrecht
(Binomialdistribution)
![Page 8: Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6 Jeongmin Lee Computer Science Department University of Pittsburgh CS 1675 Intro to Machine](https://reader034.fdocuments.us/reader034/viewer/2022052017/602f83514dcd066f2926a472/html5/thumbnails/8.jpg)
MaximumLikelihoodEstimation(ML)
©Hauskrecht
Now,youcanjustplugyourobservations(N1,andN2)intothisequation=Knowingtheparameter=Knowingtheprobabilitydistribution
![Page 9: Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6 Jeongmin Lee Computer Science Department University of Pittsburgh CS 1675 Intro to Machine](https://reader034.fdocuments.us/reader034/viewer/2022052017/602f83514dcd066f2926a472/html5/thumbnails/9.jpg)
DeriveMLestimateforExponentialDist.
• ExponentialDistribution:
• MaximumLikelihood(ML)estimateofb:
• Log-likelihood:
©RachelMisbin
![Page 10: Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6 Jeongmin Lee Computer Science Department University of Pittsburgh CS 1675 Intro to Machine](https://reader034.fdocuments.us/reader034/viewer/2022052017/602f83514dcd066f2926a472/html5/thumbnails/10.jpg)
DeriveMLestimateforExponentialDist.
• Let’ssimplifyit:
(logofprod=sumoflog)
©RachelMisbin
![Page 11: Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6 Jeongmin Lee Computer Science Department University of Pittsburgh CS 1675 Intro to Machine](https://reader034.fdocuments.us/reader034/viewer/2022052017/602f83514dcd066f2926a472/html5/thumbnails/11.jpg)
DeriveMLestimateforExponentialDist.
• Optimizelog-likelihoodbytakingpartialderivativew.r.t b
©RachelMisbin
𝛿log(𝑥)𝛿𝑥
=1𝑥
𝛿(1/𝑥)𝛿𝑥
=1𝑥<
![Page 12: Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6 Jeongmin Lee Computer Science Department University of Pittsburgh CS 1675 Intro to Machine](https://reader034.fdocuments.us/reader034/viewer/2022052017/602f83514dcd066f2926a472/html5/thumbnails/12.jpg)
DeriveMLestimateforExponentialDist.
• Setthepartialderivativetozero:
• Solveforb:
©RachelMisbin
![Page 13: Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6 Jeongmin Lee Computer Science Department University of Pittsburgh CS 1675 Intro to Machine](https://reader034.fdocuments.us/reader034/viewer/2022052017/602f83514dcd066f2926a472/html5/thumbnails/13.jpg)
Homework3
• Creating3dplotforGaussiandistribution
![Page 14: Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6 Jeongmin Lee Computer Science Department University of Pittsburgh CS 1675 Intro to Machine](https://reader034.fdocuments.us/reader034/viewer/2022052017/602f83514dcd066f2926a472/html5/thumbnails/14.jpg)
Creating3dplotforGaussiandistribution
• Meanandcovariancewehave:
mean = [3.6377, 7.8506];cov =[3.6414,1.0779;1.0779,3.7831];
![Page 15: Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6 Jeongmin Lee Computer Science Department University of Pittsburgh CS 1675 Intro to Machine](https://reader034.fdocuments.us/reader034/viewer/2022052017/602f83514dcd066f2926a472/html5/thumbnails/15.jpg)
Creating3dplotforGaussiandistribution
• CreateXandYcoordinategrid
x=-5:0.1:15;y =-5:0.1:15;[X,Y]=meshgrid(x,y);%meshgrid:replicatestheinputgridvectorstoasetofcoordinatestorectangulargrid[X,Y]
![Page 16: Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6 Jeongmin Lee Computer Science Department University of Pittsburgh CS 1675 Intro to Machine](https://reader034.fdocuments.us/reader034/viewer/2022052017/602f83514dcd066f2926a472/html5/thumbnails/16.jpg)
Creating3dplotforGaussiandistribution
• ComputeZaxisusingmvnpdf
Z=mvnpdf([X(:)Y(:)],mean,cov);
• ChangetheformofZinto2Dgrid
Z=reshape(Z,length(y),length(x));
![Page 17: Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6 Jeongmin Lee Computer Science Department University of Pittsburgh CS 1675 Intro to Machine](https://reader034.fdocuments.us/reader034/viewer/2022052017/602f83514dcd066f2926a472/html5/thumbnails/17.jpg)
Creating3dplotforGaussiandistribution
• Figureon3dsurfaceusingsurffunction:
figure;surf(x,y,Z);
![Page 18: Week 6 - people.cs.pitt.edupeople.cs.pitt.edu/~jlee/teaching/cs1675/cs1675_week6.pdf · Week 6 Jeongmin Lee Computer Science Department University of Pittsburgh CS 1675 Intro to Machine](https://reader034.fdocuments.us/reader034/viewer/2022052017/602f83514dcd066f2926a472/html5/thumbnails/18.jpg)
Thanks!-
Questions?