Approximate Bayesian Inference for Survival Models · Rupali Akerkar1 Department of Mathematical...

Approximate Bayesian Inference

Approximate Bayesian Inference for SurvivalModels

Rupali Akerkar1

Department of Mathematical SciencesNTNU,Norway

15 May 2009

1With H.Rue/S.Martino

Introduction

Outline

Basic idea

I Survival model

I Present survival model as a latent Gaussian model

I Apply INLA

I Verify results with MCMC results

Introduction

Outline

Basic idea

I Survival model

I Apply INLA

Introduction

Outline

Basic idea

I Survival model

I Apply INLA

Introduction

Outline

Basic idea

I Survival model

I Apply INLA

Introduction

Outline

I Survival Analysis

I Some definitionsI CensoringI Likelihood

I Survival Models

I Parametric modelsI Semiparametric models for hazard

I piecewise constant modelsI piecewise linear models

I Spatial model

Survival Data

Censoring

Let T be a random survival time, the following functions aredefined:

I Density function:

T ∼ f (t)

I Survival function:

S(t) = 1− F (t) =

∫ ∞t

f (u)du

I Hazard function:

h(t) = limδt→0

δtP(t < T < t + δt | T > t)

h(t) = limδt→0

S(t)− S(t + δt)

Survival Data

Censoring

I Uncensored observation: The failure time is recorded

I Right censored observation: The censoring time C < T isrecorded

I Interval censored observation: The failure time is notobserved exactly but it is known that Tlo < T < Tup

Survival Data

Censoring

Each observation is described by a triple(Tlo , Tup, δ) with

Tlo = Tup = T , δ = 1 if the obs. is uncensored

Tlo = Tup = C , δ = 0 if the obs. is right censored

Tlo < Tup, δ = 0 if the obs. is interval censored

Survival Data

Likelihood

The likelihood function is:

L =∏

where:

Survival Data

Likelihood

I if i is uncensored

Li = h(T )S(T ) = h(T )exp{−∫ T

0h(u)du}

I if i is censored

Li = S(C ) = exp{−∫ C

0h(u)du}

Survival Data

Likelihood

I if i is interval censored

Li = S(Tlo)− S(Tup)

= exp{−∫ Tl0

0h(u)du}{1− exp(−

∫ Tup

h(u)du)}

Survival Data

Likelihood

The contribution to the log-likelihood of data i , (Tlo ,Tup,δi ), is ingeneral

li = δih(Tup,i )−∫ Tup,i

0h(u)du + log{1− exp(−

∫ Tup,i

h(u)du)}

Survival Data

Likelihood

∫ Tup,i

h(u)du)}

Only included for uncensored data

Survival Data

Likelihood

∫ Tup,i

h(u)du)}

Included for all data

Survival Data

Likelihood

∫ Tup,i

h(u)du)}

Only included for interval censored data

Survival Models

Cox Model

The model proposed by Cox in 1972 is

h(t|z1, ...., zp) = h0(t) exp(z1β1 + ....+ zpβp)

I h0 = baseline hazard

I zi = covariates

I βi = regression parameters

For this model, the covariates are assumed to have fixed effects onfailure pattern.But the covariates effects do change with time.

Survival Models

More comprehensive model

A more comprehensive model is achieved by assuming

h(t) = exp(z0β0(t) + ...+ z1βp(t))

where β0 = log(h0).

Survival Models

I Parametric modelsI ExponentialI Weibull

I Parametric models with frailtyI Semiparametric models

I Piecewise-constant baseline hazardI Piecewise-linear baseline hazard

Survival Models

Parametric models

The Weibull modelI The data : (Tlo,i , Tup,i , δi ), i = 1,...,nd

I The hazards rate :

h(u; z , α) = αuα−1exp(η)

with η = zTβI the log-likelihood :

l = δh(Tup)−∫ Tup

∫ Tup

h(u)du)}

= δ[logα + (α− 1)logTup + η]− eηTαup + log{1− e−eη(Tα

up−Tαlo )}

I Priors for parameters :

β ∼ N(0, τβI )

α ∼ π(α)

Survival Models

Parametric models

∫ Tup

h(u)du)}

up−Tαlo )}

β ∼ N(0, τβI )

α ∼ π(α)

Survival Models

Parametric models

∫ Tup

h(u)du)}

up−Tαlo )}

β ∼ N(0, τβI )

α ∼ π(α)

Survival Models

Parametric models

∫ Tup

h(u)du)}

up−Tαlo )}

β ∼ N(0, τβI )

α ∼ π(α)

Survival Models

Parametric models

The Weibull model as a latent Gaussian model

I The latent Gaussian field

x = {η1, ..., ηnd, β} ∼ N(0,Q−1)

I The hyperparametersθ = α

I The likelihood

π(data|x,θ) =

nd∏i=1

π(datai |ηi ,θ)

We can apply INLA to this model without problems!!

Survival Models

Parametric models

In more general term we can have

η =∑

fj(zj) +∑k

βj z̃j + ε

where fj(zj) can represent

I smooth effect of covariate

I time varying effect of covarite

I space effect

As long as the prior for f () is Gaussian we are still in the latentGaussian model framework!

Survival Models

The Weibull model

Example1 - Kidney data

(Nahman et al.,1992) The time to the first infection for kidneydialysis patients is analysed. The data are right censored. Onebinary covariate z is catheter placement.The model is

h(t; z) = αtα−1exp{β0 + β1z}

we assumeβ0, β1 ∼ N(0, 0.001)

α ∼ Γ(1, 0.001)

Survival Models

The Weibull model

Example

−3 −2 −1 0 1 2

posterior marginals of intercept

−2.0 −1.5 −1.0 −0.5 0.0 0.5 1.0

posterior marginal densities of fixed effect

0.6 0.8 1.0 1.2 1.4

Alpha.parameter.for.Weibull

Solid line from INLA (0.165sec and 0.0235 for inla.hyperpar),histogram from 3x106 samples from Winbugs(1900 sec)

Survival Models

Parametric models with frailty

Parametric models with frailty - Hazard function

Let tij be the survival times for the j th subject in the i th cluster,i=1,...,n,j=1,...,mi , the hazard function is given as:

h(tij ; zij ,wi , α) = αtα−1ij wiexp(zT

ij β)

= αtα−1ij exp(ηij)

withηij = zT

ij β + log(wi )

Survival Models

Parametric models with frailty - Log-likekihood function

The likelihood function for the generic data (Tlo ,Tup, δ) is then

∫ Tup

h(u)du)}

up−Tαlo )}

Survival Models

Log-normal model for frailty

If we assume log(wi ) = εi to have a Gaussian prior N(0, τw ), theparametric frailty model falls into the latent Gaussian family.

x = {η11, ..., ηnmn , ε1, ...εn, β} ∼ N(0,Q−1)

I the hyperparametersθ = (α, τw )

I The likelihood

π(data|x,θ) =

nd∏i=1

π(data|ηi ,θ)

...and so no problem to apply INLA!!

Survival Models

Example2 - Rat data

(Mantel et al.,1977) study time till tumor development in rats from50 distinct litters, the covarite z is a treatment( drug or placebo),w is the frailty variable (litter/cluster). The model is

h(tij ; zij ,wi , α) = αtα−1exp{β0 + β1z + log(wi )}

We assumeβ0, β1 ∼ N(0, 0.001)

α, τw ∼ Γ(1, 0.001)

Survival Models

Example2 - Rat data

−3.0 −2.5 −2.0 −1.5 −1.0 −0.5 0.0

posterior marginal densities of intercept

−1 0 1 2 3

posterior marginal densities of fixed effect

Solid line from INLA(1.422 sec) and histogram from 106 samplesfrom Winbugs(1687 sec)

Survival Models

Example - Rat data

2.5 3.0 3.5 4.0 4.5 5.0 5.5

posterior marginals of Weibull parameterx

0 1000 2000 3000 4000

posterior marginals of group−effectx

Solid line from INLA(1.422 sec) and histogram from 106 samplesfrom Winbugs(1687 sec)

Survival Models

Semiparametric model for hazard

Piecewise constant model for h0(t)

Divide the time line into J predefined intervals, Ik = (Sk ,Sk+1] fork = 1, ..., J with 0 = s1 < ... < sJ <∞, we define the baselinehazard as:

h0(t) = λk if t ∈ Ik = (sk , sk+1]

and the baseline survival is then:

S0(t) = exp{−∫ t

0h0(u)du} = exp{

k−1∑j=1

(sj+1 − sj)λj − (t − sk)λk}

Survival Models

Piecewise constant model for h0(t)

In general, let the hazard rate be

h(t; .) = h0(t)exp(zTβ) = exp{zTβ + logh0(t)}= exp{zTβ + logλk}; t ∈ Ik

and assume a RW prior for the piecewise baseline hazard

logλ1, ..., logλJ | τλ ∼ RW (τλ)

thenηk = zTβ + logλk | ... ∼ Gaussian

Survival Models

Log-likelihood for right censored data

The log-likelihood contribution for a (possibly) right censoredobservation t ∈ Ik is :

log [h(t; .)δS(t; .)] = δηk − {k−1∑j=1

(sj+1 − sj)eηj + (t − sk)eηk}

= δηk − (t − sk)eηk −k−1∑j=1

(sj+1 − sj)eηj

I This can be seen as the log likelihood from a Poisson withmean (t − sk)eηk observed to be 0 or 1 according to δ

I This can be seen as the likelihood from k − 1 Poisson withmean (sj+1 − sj)e

ηj observed to be 0

Survival Models

log [h(t; .)δS(t; .)] = δηk − {k−1∑j=1

= δηk − (t − sk)eηk −k−1∑j=1

(sj+1 − sj)eηj

Survival Models

log [h(t; .)δS(t; .)] = δηk − {k−1∑j=1

= δηk − (t − sk)eηk −k−1∑j=1

(sj+1 − sj)eηj

Survival Models

Each data point ti is written as k ”augmented data points”yi1, ...., yik coming from Poisson distribution

Survival Models

Semiparametric models for hazard

Piecewise constant model for h0(t) as latent Gaussian field

x = {logλ1, ..., logλJ , β, η11...}

I The hyperparametersθ

I The ”augmented” Poisson data

Survival Models

Example3 -Times to death for a Breast-cancer trial

(Sedmak et al., 1989) The time to death of 45 breast cancerpatients is analysed. The data are right censored. One binarycovariate z(immunohistochemical response) is also recorded.The model is:

h(t; z) = h0(t)exp{β0 + β1z}

Moreover we divide the time line into 2 equal intervals

h0(t) = λk t ∈ Ik

and assumelogλ1, logλ2 ∼ RW 1(τλ)

β0, β1 ∼ N(0, 0.001)

Survival Models

−1.5 −1.0 −0.5 0.0 0.5 1.0

posterior marginals of fixed effect

−7.0 −6.5 −6.0 −5.5

posterior marginals of intercept

The solid line is the INLA approximations(0.090sec) and thehistogram is from 5x106 Winbug samples(685sec)

Survival Models

−0.15 −0.10 −0.05 0.00 0.05 0.10 0.15

posterior marginals of hazard for first interval

−10 −5 0 5 10

posterior marginals of logprecision

The solid line is the INLA approximations(0.090sec) and thehistogram is from 5x106 Winbug samples(685sec)

Survival Models

Piecewise linear model for h0(t)

Divide the time line into J predefined intervals, Ik = (Sk ,Sk+1] fork = 1, ..., J with 0 = s1 < ... < sJ <∞, we define the baselinehazard as:

h0(t) = λj +λj+1 − λj

sj+1 − sjt if t ∈ Ij = (sj , sj+1]

Survival Models

The log-likelihood contribution for a (possibly) right censoredobservation t ∈ Ik

log [h(t; .)S(t; .)] = δηk − w1eη1 −

k−1∑j=2

wjeηj − wkeηk − wk+1e

where, w ’s, the weights are

w1 =s2 − s1

2, wj =

sj+1 − sj−1

wk = t − sk + sk−1

2− (t − sk)2

2(sk+1 − sk)and wk+1 =

(t − sk)2

2(sk+ − sk)

Survival Models

Example4-Times to death for a Breast-cancer trial

500 1000 1500 2000

baseline.hazard.linear

Posterior mean together with 0.025quant 0.975quant

500 1000 1500 2000

baseline.hazard.constant

Piecewise linear and constant baseline hazard

Survival Models

Semiparametric model with spatial effect

Example5- Leukemia survival data

(Henderson et al.,2002) We study time to death for 1043leukaemia patients.The covariates included are age, wbc, tpi, sex and spatialinformation on district level. Here η is

ηij = α+β.hazj +β.sex ∗ sexi +β.agei +β.tpii +β.wbci +β.spatiali

Survival Models

Assumeα, β.sex ∼ N(0, 0.001)

β.haz , β.age, β.tpi , β.wbc ∼ RW 1(τ ′s)

β.spatial ∼ besag

Survival Models

2 4 6 8 10

posterior mean and quantiles for age using inla(line) and winbugs(dotted)l

5 10 15

posterior mean and quantiles for tpi using inla(line) and winbugs(dotted)1:19

5 10 15 20

district means for inla(red colour) and winbugsx

Posterior means of age,tpi and spatial effect, Solid line(red) is theINLA approximations and dotted line from Winbugs samples.

Survival Models

Example5-Leukemia survival data

20 30 40 50 60 70 80 90

0 100 200 300 400 500

−5 0 5

0 5 10 15 20

district

0 1000 2000 3000 4000 5000

hazard

Compared these with results by Kneib et al.,2007.

Example5-Leukemia survival data-Demo

demo(Leuk)

New inla input

data=read.table(”data1.txt”, header = T) this is some datan=length(data$time)surv.time = list(truncation=rep(0,n), event =data$event,lower=data$time,upper = rep(0,n), time=data$time)d = c(as.list(data), surv.time = surv.time)formula = surv.time ∼ placementmodel=inla(formula,family=”weibull”, data= d, verbose=TRUE,keep=TRUE )h=inla.hyperpar(model)

Summary

I Many survival models fall in the latent Gaussian models family

I For such models INLA is a fast and reliable tool for estimate

I ... there is still lot to do to make INLA a more general tool forsurvival models

Approximate Bayesian Inference for Survival Models · Rupali Akerkar1 Department of Mathematical...

Documents

Transcript of Approximate Bayesian Inference for Survival Models · Rupali Akerkar1 Department of Mathematical...

Approximate Bayesian Inference I:

Latent Gaussian models: Approximate Bayesian inference (INLA)folk.ntnu.no/joeid/phdclass_30jan18.pdf · 2018. 1. 30. · Latent Gaussian models: Approximate Bayesian inference (INLA)

Approximate Bayesian Inference for Latent … · Approximate Bayesian Inference for Latent Gaussian Models Using Integrated Nested Laplace Approximations Havard Rue & Sara Martino

Expectation Propagation for Approximate Bayesian Inference · PDF file362 MINKA UAI2001 Expectation Propagation for Approximate Bayesian Inference Thomas P. Minka Statistics Dept.

Approximate Inference for Fully Bayesian Gaussian Process ...

Bayesian inference

Inference in Bayesian networks - courses.csail.mit.edu › 6.034s › handouts › spring12 › ...Approximate inference using MCMC \State" of network = current assignment to all variables.

An intro to ABC – approximate Bayesian computationAn intro to ABC – approximate Bayesian computation ... exact inference for model parameters , nor it is possible to approximate

Bayesian network inference - Computer Sciencelazebnik/fall10/lec19_bayes_net_inference.pdf · • Use the Bayesian network to generate samples from the joint distribution • Approximate

Approximate Inference: Variational InferenceVariational Inference CMSC 678 UMBC Outline Recap of graphical models & belief propagation Posterior inference (Bayesian perspective) Math:

Approximate Bayesian Computation in Evolution and Ecologyjtaylor/teaching/Fall2013/...techniques—often dubbed approximate Bayesian computation, or likelihood-free inference—has

FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

SUBSPACE INFERENCE FOR BAYESIAN DEEP LEARNING · SUBSPACE INFERENCE FOR BAYESIAN DEEP LEARNING TAKEAWAYS ‣ We can apply standard approximate inference methods in subspaces of parameter

Bayesian Inference!!!

Tutorial on Approximate Bayesian Computation · Inference for simulator-based models Exact inference Approximate inference Rejection ABC algorithm Approximations to make inference

ALGORITHMS FOR APPROXIMATE BAYESIAN INFERENCE WITH

Full Bayesian inference (Learning)...Learning paradigms Learning as inference Bayesian learning, full Bayesian inference, Bayesian model averaging Model identification, maximum likelihood

Lecture 16 • 1 Lecture 16 • 2 · 1 Lecture 16 • 1 6.825 Techniques in Artificial Intelligence Inference in Bayesian Networks •Exact inference •Approximate inference Lecture

Stochastic Gradient Descent as Approximate Bayesian Inferencejmlr.csail.mit.edu/papers/volume18/17-214/17-214.pdf · Keywords: approximate Bayesian inference, variational inference,

Approximate Bayesian inference for random effects meta ...