CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line...
43
CSC413/2516 Winter 2021 Tutorial 9 - Information Theory March 11th, 2021 Presented by Jonathan Lorraine Original notes by Silviu Pitis Source: https://csc413-2020.github.io/assets/tutorials/tut09_infotheory.pdf
Transcript of CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line...
![Page 1: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/1.jpg)
CSC413/2516 Winter 2021Tutorial 9 - Information Theory
March 11th, 2021Presented by Jonathan Lorraine
Original notes by Silviu PitisSource:
https://csc413-2020.github.io/assets/tutorials/tut09_infotheory.pdf
![Page 2: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/2.jpg)
![Page 3: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/3.jpg)
![Page 4: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/4.jpg)
![Page 5: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/5.jpg)
![Page 6: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/6.jpg)
![Page 7: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/7.jpg)
![Page 8: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/8.jpg)
![Page 9: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/9.jpg)
![Page 10: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/10.jpg)
![Page 11: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/11.jpg)
![Page 12: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/12.jpg)
![Page 13: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/13.jpg)
![Page 14: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/14.jpg)
![Page 15: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/15.jpg)
![Page 16: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/16.jpg)
![Page 17: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/17.jpg)
![Page 18: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/18.jpg)
![Page 19: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/19.jpg)
![Page 20: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/20.jpg)
![Page 21: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/21.jpg)
![Page 22: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/22.jpg)
![Page 23: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/23.jpg)
![Page 24: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/24.jpg)
![Page 25: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/25.jpg)
![Page 26: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/26.jpg)
![Page 27: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/27.jpg)
![Page 28: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/28.jpg)
![Page 29: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/29.jpg)
![Page 30: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/30.jpg)
![Page 31: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/31.jpg)
![Page 32: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/32.jpg)
![Page 33: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/33.jpg)
![Page 34: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/34.jpg)
![Page 35: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/35.jpg)
![Page 36: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/36.jpg)
![Page 37: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/37.jpg)
![Page 38: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/38.jpg)
![Page 39: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/39.jpg)
![Page 40: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/40.jpg)
![Page 41: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/41.jpg)
![Page 42: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/42.jpg)
![Page 43: CSC413/2516 Winter 2021 Tutorial 9 - Information Theory ...uses Jensen's inequality. The final line is the variational or evidence lower bound (ELBO) on log p(x). While this "1.S.-Jensen"](https://reader035.fdocuments.us/reader035/viewer/2022070216/611c843203a5b868b470d443/html5/thumbnails/43.jpg)