Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf ·...
Transcript of Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf ·...
![Page 1: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/1.jpg)
Bayesian Computation
Andrew GelmanDepartment of Statistics and Department of Political Science
Columbia University
Class 3, 21 Sept 2011
Andrew Gelman Bayesian Computation
![Page 2: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/2.jpg)
Review of homework 3
I Skills:
1. Write the joint posterior density (up to a multiplicativeconstant)
2. Program one-dimensional Metropolis jumps3. Program the accept/reject rule4. Fit generalized linear models in R5. Display and summarize results
I And more . . .
Andrew Gelman Bayesian Computation
![Page 3: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/3.jpg)
Review of homework 3
I Skills:
1. Write the joint posterior density (up to a multiplicativeconstant)
2. Program one-dimensional Metropolis jumps3. Program the accept/reject rule4. Fit generalized linear models in R5. Display and summarize results
I And more . . .
Andrew Gelman Bayesian Computation
![Page 4: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/4.jpg)
Review of homework 3
I Skills:
1. Write the joint posterior density (up to a multiplicativeconstant)
2. Program one-dimensional Metropolis jumps3. Program the accept/reject rule4. Fit generalized linear models in R5. Display and summarize results
I And more . . .
Andrew Gelman Bayesian Computation
![Page 5: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/5.jpg)
Review of homework 3
I Skills:
1. Write the joint posterior density (up to a multiplicativeconstant)
2. Program one-dimensional Metropolis jumps3. Program the accept/reject rule4. Fit generalized linear models in R5. Display and summarize results
I And more . . .
Andrew Gelman Bayesian Computation
![Page 6: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/6.jpg)
Review of homework 3
I Skills:
1. Write the joint posterior density (up to a multiplicativeconstant)
2. Program one-dimensional Metropolis jumps3. Program the accept/reject rule4. Fit generalized linear models in R5. Display and summarize results
I And more . . .
Andrew Gelman Bayesian Computation
![Page 7: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/7.jpg)
Review of homework 3
I Skills:
1. Write the joint posterior density (up to a multiplicativeconstant)
2. Program one-dimensional Metropolis jumps3. Program the accept/reject rule4. Fit generalized linear models in R5. Display and summarize results
I And more . . .
Andrew Gelman Bayesian Computation
![Page 8: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/8.jpg)
Review of homework 3
I Skills:
1. Write the joint posterior density (up to a multiplicativeconstant)
2. Program one-dimensional Metropolis jumps3. Program the accept/reject rule4. Fit generalized linear models in R5. Display and summarize results
I And more . . .
Andrew Gelman Bayesian Computation
![Page 9: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/9.jpg)
Review of homework 3
I Skills:
1. Write the joint posterior density (up to a multiplicativeconstant)
2. Program one-dimensional Metropolis jumps3. Program the accept/reject rule4. Fit generalized linear models in R5. Display and summarize results
I And more . . .
Andrew Gelman Bayesian Computation
![Page 10: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/10.jpg)
Implementing Gibbs and Metropolis and improving theirefficiency
I Presentation by Wei Wang, Ph.D. student in statistics
I You can interrupt and discuss . . .
Andrew Gelman Bayesian Computation
![Page 11: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/11.jpg)
Implementing Gibbs and Metropolis and improving theirefficiency
I Presentation by Wei Wang, Ph.D. student in statistics
I You can interrupt and discuss . . .
Andrew Gelman Bayesian Computation
![Page 12: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/12.jpg)
Implementing Gibbs and Metropolis and improving theirefficiency
I Presentation by Wei Wang, Ph.D. student in statistics
I You can interrupt and discuss . . .
Andrew Gelman Bayesian Computation
![Page 13: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/13.jpg)
1. Write the joint posterior density (up to a multiplicativeconstant)
I Binomial model for #deaths given #rats
I Logistic model for Pr(death)
I Prior distribution for the logistic regression coefficients
I Discuss extensions to the model
I Steps 2, 3, 4 5 are straightforward
Andrew Gelman Bayesian Computation
![Page 14: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/14.jpg)
1. Write the joint posterior density (up to a multiplicativeconstant)
I Binomial model for #deaths given #rats
I Logistic model for Pr(death)
I Prior distribution for the logistic regression coefficients
I Discuss extensions to the model
I Steps 2, 3, 4 5 are straightforward
Andrew Gelman Bayesian Computation
![Page 15: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/15.jpg)
1. Write the joint posterior density (up to a multiplicativeconstant)
I Binomial model for #deaths given #rats
I Logistic model for Pr(death)
I Prior distribution for the logistic regression coefficients
I Discuss extensions to the model
I Steps 2, 3, 4 5 are straightforward
Andrew Gelman Bayesian Computation
![Page 16: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/16.jpg)
1. Write the joint posterior density (up to a multiplicativeconstant)
I Binomial model for #deaths given #rats
I Logistic model for Pr(death)
I Prior distribution for the logistic regression coefficients
I Discuss extensions to the model
I Steps 2, 3, 4 5 are straightforward
Andrew Gelman Bayesian Computation
![Page 17: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/17.jpg)
1. Write the joint posterior density (up to a multiplicativeconstant)
I Binomial model for #deaths given #rats
I Logistic model for Pr(death)
I Prior distribution for the logistic regression coefficients
I Discuss extensions to the model
I Steps 2, 3, 4 5 are straightforward
Andrew Gelman Bayesian Computation
![Page 18: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/18.jpg)
1. Write the joint posterior density (up to a multiplicativeconstant)
I Binomial model for #deaths given #rats
I Logistic model for Pr(death)
I Prior distribution for the logistic regression coefficients
I Discuss extensions to the model
I Steps 2, 3, 4 5 are straightforward
Andrew Gelman Bayesian Computation
![Page 19: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/19.jpg)
And more . . .
I Check convergence
I Debug program
I Check fit of model to data
I Understand model in context of data and alternative models
Andrew Gelman Bayesian Computation
![Page 20: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/20.jpg)
And more . . .
I Check convergence
I Debug program
I Check fit of model to data
I Understand model in context of data and alternative models
Andrew Gelman Bayesian Computation
![Page 21: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/21.jpg)
And more . . .
I Check convergence
I Debug program
I Check fit of model to data
I Understand model in context of data and alternative models
Andrew Gelman Bayesian Computation
![Page 22: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/22.jpg)
And more . . .
I Check convergence
I Debug program
I Check fit of model to data
I Understand model in context of data and alternative models
Andrew Gelman Bayesian Computation
![Page 23: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/23.jpg)
And more . . .
I Check convergence
I Debug program
I Check fit of model to data
I Understand model in context of data and alternative models
Andrew Gelman Bayesian Computation
![Page 24: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/24.jpg)
Optimizing the algorithm
I Scale of jumps in α and β
I Jumping distributions
I One-dimensional or two-dimensional jumps
I How to implement Gibbs here??
I Other computational strategies??
Andrew Gelman Bayesian Computation
![Page 25: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/25.jpg)
Optimizing the algorithm
I Scale of jumps in α and β
I Jumping distributions
I One-dimensional or two-dimensional jumps
I How to implement Gibbs here??
I Other computational strategies??
Andrew Gelman Bayesian Computation
![Page 26: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/26.jpg)
Optimizing the algorithm
I Scale of jumps in α and β
I Jumping distributions
I One-dimensional or two-dimensional jumps
I How to implement Gibbs here??
I Other computational strategies??
Andrew Gelman Bayesian Computation
![Page 27: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/27.jpg)
Optimizing the algorithm
I Scale of jumps in α and β
I Jumping distributions
I One-dimensional or two-dimensional jumps
I How to implement Gibbs here??
I Other computational strategies??
Andrew Gelman Bayesian Computation
![Page 28: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/28.jpg)
Optimizing the algorithm
I Scale of jumps in α and β
I Jumping distributions
I One-dimensional or two-dimensional jumps
I How to implement Gibbs here??
I Other computational strategies??
Andrew Gelman Bayesian Computation
![Page 29: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/29.jpg)
Optimizing the algorithm
I Scale of jumps in α and β
I Jumping distributions
I One-dimensional or two-dimensional jumps
I How to implement Gibbs here??
I Other computational strategies??
Andrew Gelman Bayesian Computation
![Page 30: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/30.jpg)
For next week’s class
I Homework 4 due 5pm Tues
I All course material is at http://www.stat.columbia.edu/~gelman/bayescomputation
I Next class:I Student presentation on missing-data imputation
Andrew Gelman Bayesian Computation
![Page 31: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/31.jpg)
For next week’s class
I Homework 4 due 5pm Tues
I All course material is at http://www.stat.columbia.edu/~gelman/bayescomputation
I Next class:I Student presentation on missing-data imputation
Andrew Gelman Bayesian Computation
![Page 32: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/32.jpg)
For next week’s class
I Homework 4 due 5pm Tues
I All course material is at http://www.stat.columbia.edu/~gelman/bayescomputation
I Next class:I Student presentation on missing-data imputation
Andrew Gelman Bayesian Computation
![Page 33: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/33.jpg)
For next week’s class
I Homework 4 due 5pm Tues
I All course material is at http://www.stat.columbia.edu/~gelman/bayescomputation
I Next class:I Student presentation on missing-data imputation
Andrew Gelman Bayesian Computation
![Page 34: Andrew Gelman Department of Statistics and Department of ...gelman/bayescomputation/class3.pdf · Department of Statistics and Department of Political Science Columbia University](https://reader035.fdocuments.us/reader035/viewer/2022070802/5f02d3cc7e708231d406355e/html5/thumbnails/34.jpg)
For next week’s class
I Homework 4 due 5pm Tues
I All course material is at http://www.stat.columbia.edu/~gelman/bayescomputation
I Next class:I Student presentation on missing-data imputation
Andrew Gelman Bayesian Computation