Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28,...
-
Upload
clyde-jeffry-potter -
Category
Documents
-
view
223 -
download
0
description
Transcript of Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28,...
![Page 1: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/1.jpg)
Bayesian Density Regression
Author: David B. Dunson and Natesh Pillai
Presenter: Ya XueApril 28, 2006
![Page 2: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/2.jpg)
Outline
• Key idea
• Proof
• Application to HME
![Page 3: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/3.jpg)
Bayesian Density Regression with Standard DP
• The regression model: (i=1,...,n)
• Two cases:1. 2.
Parametric model
Standard Dirichlet process mixture model
iiiiiii dpxyfxyf )(),|()|(
),|(,)( iiii xyfpGp i )(
![Page 4: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/4.jpg)
Bayesian Density Regression with Standard DP
• Model
• The algorithm automatically finds the shrinkage of parameters
.,...,1),,(~
,~),(),|1(
0
NiGDPG
Gxxyp
i
iTiiii
.,...,1, Nii
![Page 5: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/5.jpg)
Polya Urn Model
ij
ii jn
Gn
X ,)1
1()1
(),,|( 0)(
• Standard Polya urn model
• This paper proposed a generalized Polya urn model.
ijij
ij
ij
ijij
ii jw
wG
wX ,)()(),,|( 0
)(
where is a kernel function.),( jiij xxww 0ijw monotonically as increases.),( ji xxd
.1lim ijxx wij
(1)
![Page 6: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/6.jpg)
Idea – Spatial DPEquation (1) implies• The prior probability of setting decreases as
increases.
• The prior probability of increases as more neighbors are added that have predictor values xj close to xi.
• The expected prior probability of increases in proportion to the hyperparameter .
ji ),( ji xxd
)(ii
)(ii
![Page 7: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/7.jpg)
Outline
• Key idea
• Proof
• Application to HME
![Page 8: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/8.jpg)
Spatial Varying Regression Model
iixiiiii dGxyfxyfi
)(),|()|(
• At a given location in the feature space,
A mixture of an innovation random measure
and neighboring random measures
j~i indexes samples
![Page 9: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/9.jpg)
Theorem 1
![Page 10: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/10.jpg)
Hierarchical Model
• The hierarchical form
![Page 11: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/11.jpg)
• Let denote an index set for the subjects drawn from the jth mixture component, for j=1,...,n. Then we have for
• Conditioning on Z, we can use the Polya urn result to obtain the conditional prior
• Only the subvector of elements of belonging to are informative.
Conditional Distribution},...,1{}:{ njZiI ij
*~jxi G
.jIi
(2))(i
iZI
![Page 12: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/12.jpg)
Marginalize over Z
• We obtain the following generalization of the Polya urn scheme (a)
(b)if sample i and j belong to the same mixture component.1ijm
![Page 13: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/13.jpg)
Example
(a) (b)
For example, n=4,
p(mi)
![Page 14: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/14.jpg)
Rewrite Equation (2)
• Let
• Then Eqn.(2) can be expressed as
(3)
![Page 15: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/15.jpg)
Theorem 4
Hence, Eqn. (3) is equivalent to
ijij
ij
ij
ijij
ii jw
wG
wBX .)()(),,,|( 0
)(
![Page 16: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/16.jpg)
Predictive distribution
![Page 17: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/17.jpg)
Outline
• Key idea
• Proof
• Application to HME
![Page 18: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/18.jpg)
Mixture Model
• We simulate data from a mixture of two normal linear regression models
• Poor results obtained by using the standard DP mixture model.
)04.0,;()1()01.0,;()|( 42
22
2 22ii
xii
xii xyNexyNexyf ii
![Page 19: Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006.](https://reader036.fdocuments.us/reader036/viewer/2022062317/5a4d1b217f8b9ab059995555/html5/thumbnails/19.jpg)