Reading: I. Goodfellow et al., Generative adversarial...
Transcript of Reading: I. Goodfellow et al., Generative adversarial...
![Page 1: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/1.jpg)
Generative Adversarial Networks
Reading: I. Goodfellow et al., Generative adversarial networks, NIPS 2014
![Page 2: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/2.jpg)
Adversarial Networks
Generative Model Real world Discriminative
Model
real or fake?
From https://www.cs.colorado.edu/~mozer/Teaching/syllabi/DeepLearningFall2017/lectures/gan.pptx
![Page 3: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/3.jpg)
Training Procedure: Basic Idea
• G tries to fool D
• D tries not to be fooled
• Models are trained simultaneously – As G gets better, D has a more
challenging task – As D gets better, G has a more
challenging task
• Ultimately, we don’t care about the D – Its role is to force G to work harder
Discriminative
Model
real or fake?
Generative Model
noise (Z)
Real world
From https://www.cs.colorado.edu/~mozer/Teaching/syllabi/DeepLearningFall2017/lectures/gan.pptx
![Page 4: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/4.jpg)
![Page 5: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/5.jpg)
GAN Training Algorithm
![Page 6: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/6.jpg)
From Goodfellow et al., 2014
![Page 7: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/7.jpg)
http://www.iangoodfellow.com/slides/2018-06-22-gan_tutorial.pdf
![Page 8: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/8.jpg)
http://www.iangoodfellow.com/slides/2018-06-22-gan_tutorial.pdf
![Page 9: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/9.jpg)
Kerras et al, PROGRESSIVE GROWING OF GANS FOR IMPROVED QUALITY, STABILITY, AND VARIATION
Gradually adds layers to both generator and discriminator
![Page 10: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/10.jpg)
![Page 11: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/11.jpg)
Vector arithmetic
![Page 12: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/12.jpg)
![Page 13: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/13.jpg)
Cycle GANs (Zhu et al., 2017; arXiv:1703:10593v2 [cs.CV])
Given two image collections, algorithm learns to translate an image from one collection to the other
From https://www.cs.colorado.edu/~mozer/Teaching/syllabi/DeepLearningFall2017/lectures/gan.pptx
![Page 14: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/14.jpg)
Photos to paintings
From https://www.cs.colorado.edu/~mozer/Teaching/syllabi/DeepLearningFall2017/lectures/gan.pptx
![Page 15: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/15.jpg)
From https://www.cs.colorado.edu/~mozer/Teaching/syllabi/DeepLearningFall2017/lectures/gan.pptx
![Page 16: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/16.jpg)
Image-to-Image Translation with Conditional Adversarial Nets Isola et al., 2017
![Page 17: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/17.jpg)
https://phillipi.github.io/pix2pix/ https://arxiv.org/pdf/1808.06601.pdf
![Page 18: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/18.jpg)
Image-to-Image Translation Image-to-Image Translation with Conditional Adversarial Networks
Phillip Isola Jun-Yan Zhu Tinghui Zhou Alexei A. Efros
![Page 19: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/19.jpg)
![Page 20: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/20.jpg)
![Page 21: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/21.jpg)
![Page 22: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/22.jpg)
![Page 23: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/23.jpg)
![Page 24: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/24.jpg)
Generative Adversarial Text to Image Synthesis Reed et al.
![Page 25: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/25.jpg)
![Page 26: Reading: I. Goodfellow et al., Generative adversarial ...web.cecs.pdx.edu/~mm/MachineLearningFall2018/GANs.pdfGenerative Adversarial Networks Reading: I. Goodfellow et al., Generative](https://reader033.fdocuments.us/reader033/viewer/2022053013/5f10c4af7e708231d44ab95b/html5/thumbnails/26.jpg)
https://github.com/reedscot/icml2016