Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.
-
Upload
adrian-copeland -
Category
Documents
-
view
213 -
download
0
Transcript of Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.
![Page 1: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/1.jpg)
Machine Learning for Computer graphics
Aaron HertzmannUniversity of Toronto
Bayesian
![Page 2: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/2.jpg)
Computers are really fast
• If you can create it, you can render it
![Page 3: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/3.jpg)
How do you create it?
Digital Michaelangelo Project
Steven Schkolne
![Page 4: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/4.jpg)
Two problems
• How do we get the data into the computer?
• How do we manipulate it?
![Page 5: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/5.jpg)
Data fitting
Pj ;t jjmj(qt) à sj ;tjj
2
+ w1P
j ;t jjqt à qtà1jj2
+ w2P
j ;t jj:::jj2
![Page 6: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/6.jpg)
Key questions
• How do you fit a model to data?– How do you choose weights and
thresholds?– How do you incorporate prior
knowledge?– How do you merge multiple
sources of information?– How do you model uncertainty?
Bayesian reasoning provides a solution
![Page 7: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/7.jpg)
Talk outline
• Bayesian reasoning• Facial modeling example• Non-rigid modeling from video
![Page 8: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/8.jpg)
What is reasoning?
• How do people reason?• How should computers do it?
![Page 9: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/9.jpg)
Aristotelian Logic
• If A is true, then B is true• A is true• Therefore, B is true
A: My car was stolenB: My car isn’t where I left it
![Page 10: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/10.jpg)
Real-world is uncertain
Problems with pure logic• Don’t have perfect information• Don’t really know the model• Model is non-deterministic
![Page 11: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/11.jpg)
Beliefs
• Let B(X) = “belief in X”,• B(¬X) = “belief in not X”
1. An ordering of beliefs exists2. B(X) = f(B(¬X))3. B(X) = g(B(X|Y),B(Y))
![Page 12: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/12.jpg)
Cox axioms
R.T. Cox, “Probability, frequency, and reasonable expectation,” American J. Physics, 14(1):1-13, 1946
p(A) =P
fB=bigp(A;B)
p(A;B) = p(AjB)p(B)
p(TRUE) = 1
![Page 13: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/13.jpg)
“Probability theory is nothing more than common sense reduced to calculation.”
- Pierre-Simon Laplace, 1814
![Page 14: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/14.jpg)
Bayesian vs. Frequentist
• Frequentist (“Orthodox”):– Probability = percentage of
events in infinite trials
• Medicine, biology: Frequentist• Astronomy, geology, EE,
computer vision: largely Bayesian
![Page 15: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/15.jpg)
Learning in a nutshell
• Create a mathematical model• Get data• Solve for unknowns
![Page 16: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/16.jpg)
Face modeling
• Blanz, Vetter, “A Morphable Model for the Synthesis of 3D Faces,” SIGGRAPH 99
![Page 17: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/17.jpg)
Generative model
• Faces come from a Gaussian
p(SjSö;þ) = (2ù)djþjp 1 eà(SàSö)Tþà1(SàSö)=2
p(fSigjSö;þ) =Q
i p(SijSö;þ)
Learning
argmaxSö;þ p(Sö;þjfSig) Si
![Page 18: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/18.jpg)
Bayes Rule
p(modeljdata) / p(datajmodel)p(model)
P(A;B) = p(AjB)p(B)
= p(BjA)p(A)
P(BjA) = p(AjB)p(B)=p(A)
Often: p(modeljdata) / p(datajmodel)
![Page 19: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/19.jpg)
Learning a GaussianargmaxSö;þ p(Sö;þjfSig)
Sö P
i Si=Nþ
Pi(Si à Sö)(Si à Sö)T=N
= argmaxSö;þ
Qi p(SijSö;þ)
= argmaxSö;þ p(fSigjSö;þ)
= argmaxSö;þ
Qi (2ù)djþjp 1 eà 2
1(SiàSö)Tþà1(SiàSö)=2
![Page 20: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/20.jpg)
Maximization trick
• Maximize <-> minimize
p(x)à lnp(x)
![Page 21: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/21.jpg)
Fitting a face to an image
Generative model
p(SjSö;þ) = (2ù)djþjp 1 eà(SàSö)Tþà1(SàSö)=2
p(I jS;ú;û2) = (2ù)djþjp 1 eà jjIàRender(S;ú)jj2=2û2
I = Render(S;ú) + n
![Page 22: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/22.jpg)
Fitting a face to an image
Maximize
jjI à Render(S;ú)jj2=2û2 + (S à Sö)Tþà1(S à Sö)=2
+ 2N ln2ùû2 + 2
1ln(2ù)djþj
à lnp(S;újI ;Sö;þ;û2) =
p(S;újI ;Sö;þ;û2)
minimize
![Page 23: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/23.jpg)
Why does it work?p(fxigjmodel) =
Qp(xijmodel)
p(x|model)
s:t:R
p(xjmodel) = 1
![Page 24: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/24.jpg)
General features
• Models uncertainty• Applies to any generative model• Merge multiple sources of
information• Learn all the parameters
![Page 25: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/25.jpg)
Caveats
• Still need to understand the model
• Not necessarily tractable• Potentially more involved than
ad hoc methods
![Page 26: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/26.jpg)
Applications in graphics
• Shape and motion capture• Learning styles and generating
new data
![Page 27: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/27.jpg)
Learning Non-Rigid 3D Shape from 2D Motion
Joint work with Lorenzo Torresani and Chris
Bregler (Stanford, NYU)
![Page 28: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/28.jpg)
Camera geometry
Orthographic projection
p ;̀t = RtS ;̀t + Tt
p ;̀tS ;̀t
![Page 29: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/29.jpg)
Non-rigid reconstruction
Input: 2D point tracksOutput: 3D nonrigid motion
pt = RtSt + Tt
Totally ambiguous!
![Page 30: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/30.jpg)
Shape reconstruction
Least-squares version
pt = RtSt + Tt
minimizeP
t jjpt à RtSt à Ttjj2 + wP
t jjSt à Söjj2
![Page 31: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/31.jpg)
Bayesian formulation
p(R;S;T;Sö:þjP)/ p(PjR;S;T)p(SjSö:þ)
maximize
p(StjSö;þ) = (2ù)djþjp 1
eà(StàSö)Tþà1(StàSö)=2
p ;̀t = RtS ;̀t + Tt + n n ø N(0;û2)
![Page 32: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/32.jpg)
How does it work?p(R;S;T;Sö:þjP)/ p(PjR;S;T)p(SjSö:þ)
àP
t lnp(PtjRt;St;Tt) àP
t lnp(StjSö;þ)
=P
t jjpt à RtSt à Ttjj2=2û2 + 2N lnû2
+P
t 21(St à Sö)Tþà1(St à Sö) + 2
T lnjþj
Maximize
Minimize
(actual system is slightly more sophisticated)
![Page 33: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/33.jpg)
Results
Least-squares, no reg.
Least-squares, reg.
Gaussian
LDS
View 2
Input
![Page 34: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.](https://reader034.fdocuments.us/reader034/viewer/2022042822/56649f135503460f94c271a9/html5/thumbnails/34.jpg)
Conclusions
• Bayesian methods provide unified framework
• Build a model, and reason about it
• The future of data-driven graphics