MSH3 Generalized linear model Ch. 6 Count data …SydU MSH3 GLM (2016) Second semester Dr. J. Chan...
Transcript of MSH3 Generalized linear model Ch. 6 Count data …SydU MSH3 GLM (2016) Second semester Dr. J. Chan...
MSH3 Generalized linear model Ch. 6 Count data models
Contents
§4 Count data model 123
§4.1 Introduction: The Children Ever Born Data . . . . . . . 123
§4.2 The Poisson Distribution . . . . . . . . . . . . . . . . . 124
§4.3 Log-Linear Models . . . . . . . . . . . . . . . . . . . . . 126
§4.4 ML estimation . . . . . . . . . . . . . . . . . . . . . . . 127
§4.5 Goodness of fit and hypothesis test . . . . . . . . . . . . 128
§4.6 Children ever born example . . . . . . . . . . . . . . . . 129
§4.6.1 Grouped Data and the Offset . . . . . . . . . . . 129
§4.6.2 The Deviance Table . . . . . . . . . . . . . . . . 130
§4.6.3 The Additive Model . . . . . . . . . . . . . . . . 132
§4.7 Distributions with over- and underdispersion . . . . . . . 135
§4.7.1 Negative binomial distribution . . . . . . . . . . . 135
§4.7.2 Generalized Poisson distribution . . . . . . . . . . 139
§4.8 Model for excess zero . . . . . . . . . . . . . . . . . . . 141
§4.8.1 Zero-inflated Poisson model . . . . . . . . . . . . 142
§4.8.2 Hurdle model . . . . . . . . . . . . . . . . . . . . 143
§4.9 Poisson mixture model . . . . . . . . . . . . . . . . . . . 143
§4.10Model selection example . . . . . . . . . . . . . . . . . . 148
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 122
MSH3 Generalized linear model Ch. 6 Count data models
§4 Count data model
§4.1 Introduction: The Children Ever Born Data
The following table, adapted from Little (1979), comes from the Fiji
Fertility Survey, typical in the reports of the World Fertility Survey.
Each cell in the table shows the mean, the variance and the number of
observations.
The unit of analysis is the individual woman, the response is the number
of children she has borne, and the 3 discrete predictors are the duration
since her first marriage, the type of place of residence (Suva the capital,
other urban and rural), and her educational level (none, lower primary,
upper primary, and secondary or higher).
Data such as these have traditionally been analyzed using ordinary lin-
ear models with normal errors. The key concern, however, is not the
normality of the errors but rather the assumption of constant variance.
0.0 0.5 1.0 1.5 2.0
−0.
50.
00.
51.
01.
52.
02.
5
log(mean)
log(
var)
Figure 6.1: The Mean-variance Relationship for the CEB Data
The plot of variance against mean in log scale for all cells with at least
20 observation shows clearly that, the assumption of constant variance is
not suitable and the relationship is close to proportional. Thus, a Poisson
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 123
MSH3 Generalized linear model Ch. 6 Count data models
regression models is more suitable than a ordinary linear models.
Table 6.1: Number of Children Ever Born to Women of Indian RaceBy Marital Duration, Type of Place of Residence and Educational Level(Each cell shows the mean, variance and sample size)
Marr. Suva Urban RuralDur. N LP UP S+ N LP UP S+ N LP UP S+
0-4 0.50 1.14 0.90 0.73 1.17 0.85 1.05 0.69 0.97 0.96 0.97 0.741.14 0.73 0.67 0.48 1.06 1.59 0.73 0.54 0.88 0.81 0.80 0.59
8 21 42 51 12 27 39 51 62 102 107 47
5-9 3.10 2.67 2.04 1.73 4.54 2.65 2.68 2.29 2.44 2.71 2.47 2.241.66 0.99 1.87 0.68 3.44 1.51 0.97 0.81 1.93 1.36 1.30 1.1910 30 24 22 13 37 44 21 70 117 81 21
10-14 4.08 3.67 2.90 2.00 4.17 3.33 3.62 3.33 4.14 4.14 3.94 3.331.72 2.31 1.57 1.82 2.97 2.99 1.96 1.52 3.52 3.31 3.28 2.5012 27 20 12 18 43 29 15 88 132 50 9
15-19 4.21 4.94 3.15 2.75 4.70 5.36 4.60 3.80 5.06 5.59 4.50 2.002.03 1.46 0.81 0.92 7.40 2.97 3.83 0.70 4.91 3.23 3.29 -14 31 13 4 23 42 20 5 114 86 30 1
20-24 5.62 5.06 3.92 2.60 5.36 5.88 5.00 5.33 6.46 6.34 5.74 2.504.15 4.64 4.08 4.30 7.19 4.44 4.33 0.33 8.20 5.72 5.20 0.5021 18 12 5 22 25 13 3 117 68 23 2
25-29 6.60 6.74 5.38 2.00 6.52 7.51 7.54 - 7.48 7.81 5.80 -12.40 11.66 4.27 - 11.45 10.53 12.60 - 11.34 7.57 7.07 -
47 27 8 1 46 45 13 - 195 59 10 -
§4.2 The Poisson Distribution
A random variable Y has a Poisson distribution with parameter µ > 0
if its pmf is
Pr(Y = y) =e−µµy
y!, y = 0, 1, 2, . . .
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 124
MSH3 Generalized linear model Ch. 6 Count data models
The mean and variance are
E(Y ) = Var(Y ) = µ.
Since the mean is equal to the variance, any factor that affects one will
also affect the other. Hence homoscedasticity fails and the variance tends
to increase with the mean for Poisson data.
The Poisson distribution can be derived as a limiting (approximate) form
of the binomial distribution. Specifically, if Y ∼ B(n, π), Yapprox∼ P (µ)
with fixed µ = nπ as n→∞ and π → 0 (rare events).
Alternatively, the Poisson distribution is defined in terms of a stochastic
process with the following conditions:
• The probability of at least one occurrence of the event in a given
time interval is proportional to the length of the interval.
• The probability of two or more occurrences of the event in a very
small time interval is negligible.
• The numbers of occurrences of the event in disjoint time intervals
are mutually independent.
Then the distribution for the number of events in a fixed time interval is
Poisson with mean µ = λt, where λ is the rate of event per unit time and
t is the length of the time interval. The proof is given in the Appendix.
A useful property of the Poisson distribution is that the sum of indepen-
dent Poisson random variables is also Poisson. Specifically, if Y1 and Y2
are independent with Yi ∼ P (µi) for i = 1, 2 then Y1 +Y2 ∼ P (µ1 +µ2).
This result generalizes in an obvious way to the sum of more than two
Poisson observations.
For a group of ni individuals with identical covariate values, let Yij denote
the number of events experienced by the j-th unit in the i-th group, and
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 125
MSH3 Generalized linear model Ch. 6 Count data models
let Yi denote the total number of events in group i. Then, under the
usual assumption of independence, if Yij ∼ P (µi) for j = 1, 2, . . . ni,
then Yi =ni∑j=1
Yij ∼ P (niµi). In terms of estimation, we obtain exactly
the same likelihood function if we work with the individual counts Yij or
the group counts Yi.
§4.3 Log-Linear Models
Let Yi ∼ P (µi) independently and y1, y2, . . . yn are n observed values.
Let the mean µi (variance as well) depend on a vector of explanatory
variables xi in a simple linear model:
µi = x′iβ.
However, while x′iβ can assume any real value, the Poisson mean µi has
to be non-negative. One solution is to model the logarithm of the µi, a
canonical link for Poisson distribution, using a linear model:
log(µi) = x′iβ
where βj in β represents the expected change in the log of the mean per
unit change in the predictor xij. Exponentiating the equation, we obtain
a multiplicative model for the mean:
µi = exp(x′iβ) = exp (p∑j=1
xijβj) =
p∏j=1
exp(xijβj) =
p∏j=1
[exp(βj)]xij
which agrees with what we observe that the size of effects is proportional
to the count itself. Increasing xij by one unit, the mean µi is multiplied
by exp(βj).
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 126
MSH3 Generalized linear model Ch. 6 Count data models
§4.4 ML estimation
The log-likelihood function for n independent Poisson observations is
`(β) =
n∑i=1
yi log(µi)− µi − log(yi!).
Taking derivatives of `(β) with respect to the elements of β, and setting
them zero
∂`
∂βj=
n∑i=1
yiµiµixij − µixij =
n∑i=1
xij(yi − µi),
the maximum likelihood estimates in log-linear Poisson models satisfy
the estimating equations
X ′y = X ′µ
where X is the design matrix, with row xi for each observation and
one column for each predictor, including the constant (if any). This
estimating equation arises in any generalized linear model with canonical
link.
Thus, in models with a constant, one of the estimating equations matches
the sum of observed and fitted values. In models with a discrete factor,
such as the Children Ever Born example, the observed and fitted total
number of children ever born will be matched in each category of marital
duration since first marriage. This result generalizes to higher order
terms. The interaction model A+B+AB would match the totals in each
combination of categories of A and B, or the AB margin.
To estimate the parameters, we will use the iteratively-reweighted least
squares (IRLS) algorithm the working dependent variable z
zi = ηi +yi − µiµi
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 127
MSH3 Generalized linear model Ch. 6 Count data models
since ∂ηi∂µi
= ∂ lnµi∂µi
= 1µi
and the diagonal matrix W of iterative weights
has elements
wii = [V ar(zi)]−1
= [V ar(yi) (∂ηi∂µi
)2]−1
= [µi(1
µi)
2]−1
= µi
where µi is the fitted values based on the current parameter estimates.
Initial values can be obtained by applying the link to the data, that is
taking the log of the response, and regressing it on the predictors using
OLS. The procedure usually converges in a few iterations.
§4.5 Goodness of fit and hypothesis test
1. Deviance: It measures the discrepancy between observed and fitted
values for Poisson responses:
D = 2
n∑i=1
[yi log
(yiµi
)− (yi − µi)
]n→∞∼ χ2
n−p
where p the number of parameters. The first term is identical to
the binomial deviance and the second term, a sum of differences
between observed and fitted values, is usually zero for MLE which
march marginal totals.
2. Pearson’s chi-squared statistic: It is the squared difference between
observed and fitted values to the variance of the observed value:
X2 =
n∑i=1
(yi − µi)2
µi
n→∞∼ χ2n−p
which the same for Poisson and binomial data.
3. Likelihood ratio test: It can be constructed in terms the difference
in deviances between two nested model:
−2 lnλ =D(ω1)−D(ω2)
φ
n→∞∼ χ2p2−p1
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 128
MSH3 Generalized linear model Ch. 6 Count data models
under the assumption that the smaller model ω1 is correct in H0.
4. Wald test: To test H0 : β = β0, the test statistics is
W0 = (β − β0)′[var(β)]−1(β − β0)n→∞∼ χ2
p
based on the fact that the MLE
β ∼ Np(β,X′WX)
where W is the diagonal matrix of estimation weights wii.
§4.6 Children ever born example
§4.6.1 Grouped Data and the Offset
Let Yijkl denote the number of children from the l-th woman in the
(i, j, k)-th group, where i denotes marital duration, j residence and k
education and with mean µijk. Let Yijk =∑
l Yijkl denote the group
total. The group total is a realization of a Poisson variate with mean
nijkµijk, where nijk is the number of observations in the (i, j, k)-th cell.
Suppose that a log-linear model is postulated for the individual means:
logE(Yijkl) = log(µijk) = x′ijkβ
where xijk is a vector of covariates. Then the log of the expected value
of the group total is
logE(Yijk) = log(nijkµijk) = log(nijk) + x′ijkβ.
Thus the group totals have the same coefficients β as the individual
means, except that the linear predictor includes the term log(nijk) called
an offset which is the log of exposure measures, the number of women
here.
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 129
MSH3 Generalized linear model Ch. 6 Count data models
Thus, we can fit log-linear models to the individual counts or group
totals with the log of the number of women in each cell as an offset. The
deviances is different because they measure goodness of fit to different
sets of counts but differences of deviances between nested models are
exactly the same.
§4.6.2 The Deviance Table
Table 7.2 reports the results of fitting a variety of Poisson models. The
null model assuming the same number of children for all the groups is
rejected based (D = 3732 on 69 df). Introducing marital duration
reduces the deviance to 165.8 on 64 df. The drop of 3566 on 5 df shows
that the number of CEB depends on the duration since the first marriage.
To test the effect of education controlling for duration, the additive
model D + E relative to the one-factor D model gives a chi-squared
statistic of 65.8 (165.84-100.01) on 3 (64-61) df and is highly significant.
We can also test the net effect of education controlling for residence &
duration, by comparing the 3-factor additive model D + R + E with the
2-factor model D + R. The difference in deviances of 50.1 (120.68-70.65)
on 3 (62-59) df is highly significant. This smaller Chi-square statistic
indicates that part of the education effect may due to the fact that more
educated women live in Suva or in other urban areas.
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 130
MSH3 Generalized linear model Ch. 6 Count data models
Table 6.2: Deviances for Poisson log-linear models fitted to CEB data
Type Model Deviance d.f.
Null 3731.52 69
1-factor Models Duration 165.84 64
Residence 3659.23 67
Education 2661.00 66
2-factor Models D + R 120.68 62
D + E 100.01 61
DR 108.84 52
DE 84.46 46
3-factor Models D + R + E 70.65 59
D + RE 59.89 53
E + DR 57.06 49
R + DE 54.91 44
DR + RE 44.27 43
DE + RE 44.60 38
DR + DE 42.72 34
DR + DE + RE 30.95 28
Does education make more of a difference in rural areas than in
urban areas? To answer this question we compare models D+RE to
D+R+E. The reduction in deviance is 10.8 (70.65-59.89) on 6 (59-53) df
and is not significant, with a P-value of 0.096.
Does the effect of education increase with marital duration? Adding
DE (model R+DE) to model D+R+E reduces the deviance by 15.7
(70.65-54.91) at the cost of 15 df (59-44), hardly a bargain. Thus, we
conclude that the additive model which has a deviance of 70.65 on 59
d.f. (an average of 1.2 per d.f.) is adequate for these data.
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 131
MSH3 Generalized linear model Ch. 6 Count data models
§4.6.3 The Additive Model
The associated P-value of the D+R+E model is 0.14 (Pr(χ259 > 70.65) =
0.14), so the model passes the goodness-of-fit test. Table 6.3 shows its
parameter estimates and standard errors.
Duration. The constant represents the log of the mean number of CEB
for the reference cell, which is Suvanese women with no education and
0-4 years marriage duration. Since exp(0.1173) = 0.89, these women
have 0.89 children on average at this time in their lives. The duration
parameters trace the increase in CEB with duration for any residence
and education group. The model is multiplicative in the original scale.
From duration 0-4 to 5-9 years, the number of CEB gets multiplied by
exp(0.9977) = 2.71. By duration 25-29 years, the number of CEB is
exp(1.977) = 7.22 times those at duration 0-4 for any residence type and
education level.
Residence. The effects of residence show that Suvanese women have
the lowest fertility. At any given marriage duration, women living in
other urban areas have 1.12 (exp(0.1123)) times of CEB than Suvanese
women with the same level of education. Similarly, at any fixed duration,
women who live in rural areas have 1.16 times CEB (exp(0.1512)) than
Suvanese women with the same level of education.
Education. Higher education is associated with smaller family sizes net
of duration and residence. At any given marriage duration, women with
upper primary education have 0.903 (exp(−0.1017)) times CEB, and
women with secondary or higher education have 0.734 (exp(−0.3096))
times CEB, than women with no education in the same type of residence.
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 132
MSH3 Generalized linear model Ch. 6 Count data models
Table 6.3: Estimates for additive log-linear model of CEB data
Parameter Estimate Std. Error z-ratio
Constant -0.1173 0.0549 2.14
Duration 0-4 -
5-9 0.9977 0.0528 18.91
10-14 1.3705 0.0511 26.83
15-19 1.6142 0.0512 31.52
20-24 1.7855 0.0512 34.86
25-29 1.9768 0.0500 39.50
Residence Suva -
Urban 0.1123 0.0325 3.46
Rural 0.1512 0.0283 5.34
Education None -
Lower 0.0231 0.0227 1.02
Upper -0.1017 0.0310 -3.28
Sec+ -0.3096 0.0552 -5.61
The educational differential of 0.734 times between these two groups
translates into 0.25 of a child at durations 0-4, increases to about 1 child
around duration 15, and reaches almost 1.75 children by duration 25+.
Thus, the effect of education increases with marital duration. (5-9, Sec+:
exp(−0.1173 + 0.9977− 0.3096) = 1.77)
Table 6.4: Fitted values for Suvanese women with no education and
with secondary or higher Education
Marital Duration 0-4 5-9 10-14 15-19 20-24 25+
No Education 0.89 2.41 3.50 4.47 5.30 6.42
Secondary+ 0.65 1.77 2.57 3.28 3.89 4.71
Difference 0.24 0.64 0.93 1.19 1.41 1.71
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 133
MSH3 Generalized linear model Ch. 6 Count data models
> rm(list=ls())
> data=read.table("CEB.txt",skip=1)
> mu=data[,1]
> ni=data[,3]
> ln=log(ni)
> dur=as.factor(data[,4])
> resid=as.factor(data[,5])
> edu=as.factor(data[,6])
> y=round(mu*ni,0)
> d0=glm(y~offset(ln),family=poisson)$dev
> d1=glm(y~offset(ln)+dur,family=poisson)$dev
> d2=glm(y~offset(ln)+resid,family=poisson)$dev
> d3=glm(y~offset(ln)+edu,family=poisson)$dev
> d4=glm(y~offset(ln)+dur+resid,family=poisson)$dev
> d5=glm(y~offset(ln)+dur+edu,family=poisson)$dev
> d6=glm(y~offset(ln)+dur*resid,family=poisson)$dev
> d7=glm(y~offset(ln)+dur*edu,family=poisson)$dev
> d8=glm(y~offset(ln)+dur+edu+resid,family=poisson)$dev
> d9=glm(y~offset(ln)+dur+edu*resid,family=poisson)$dev
> d10=glm(y~offset(ln)+dur*resid+edu,family=poisson)$dev
> d11=glm(y~offset(ln)+dur*edu+resid,family=poisson)$dev
> d12=glm(y~offset(ln)+dur*resid+edu*resid,family=poisson)$dev
> d13=glm(y~offset(ln)+dur*edu+resid*edu,family=poisson)$dev
> d14=glm(y~offset(ln)+dur*resid+dur*edu,family=poisson)$dev
> d15=glm(y~offset(ln)+dur*resid+dur*edu+resid*edu,family=poisson)$dev
> c(d0,d1,d2,d3,d4,d5,d6)
[1] 3731.8516 166.0717 3659.2791 2660.9976 120.6804 100.1917 108.8965
> c(d7,d8,d9,d10,d11,d12,d13)
[1] 84.53047 70.66530 59.92076 57.13495 54.80143 44.31104 44.52327
> summary(glm(y~offset(ln)+dur+resid+edu,family=poisson))
Call:
glm(formula = y ~ offset(ln) + dur + resid + edu, family = poisson)
Deviance Residuals:
Min 1Q Median 3Q Max
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 134
MSH3 Generalized linear model Ch. 6 Count data models
-2.2960 -0.6641 0.0725 0.6336 3.6782
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -0.11710 0.05491 -2.132 0.032969 *
dur2 0.99693 0.05274 18.902 < 2e-16 ***
dur3 1.36940 0.05107 26.815 < 2e-16 ***
dur4 1.61376 0.05119 31.522 < 2e-16 ***
dur5 1.78491 0.05121 34.852 < 2e-16 ***
dur6 1.97641 0.05003 39.501 < 2e-16 ***
resid2 0.11242 0.03250 3.459 0.000541 ***
resid3 0.15166 0.02833 5.353 8.63e-08 ***
edu2 0.02297 0.02266 1.014 0.310597
edu3 -0.10127 0.03099 -3.268 0.001082 **
edu4 -0.31015 0.05521 -5.618 1.94e-08 ***
---
Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1 1
(Dispersion parameter for poisson family taken to be 1)
Null deviance: 3731.852 on 69 degrees of freedom
Residual deviance: 70.665 on 59 degrees of freedom
AIC: 522.14
Number of Fisher Scoring iterations: 4
§4.7 Distributions with over- and underdispersion
§4.7.1 Negative binomial distribution
The number of successes Y before r failures in a sequence of iid Bernoulli
trials with success probability p follows a Negative binomial distribution
NB(r, p) if it’s pmf, mean and variance are
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 135
MSH3 Generalized linear model Ch. 6 Count data models
f (y) =
(y + r − 1
y
)(1− p)rpy,
E(Y ) =pr
1− p, and V ar(Y ) =
pr
(1− p)2=E(Y )
1− prespectively. As V ar(Y ) > E(Y ), it is overdispersed. Being a Poisson-
Gamma mixture,
Y ∼ NB(r, p) ⇔ Y ∼ P (λ), λ ∼ G(r,p
1− p)
it has higher variability than Poisson. The proof is:
f (y; r, p) =
∫ ∞0
λye−λ
y!
λr−1e−λ(1−p)/p
( p1−p)
rΓ(r)dλ
=(1− p)rp−r+r+y
y!Γ(r)
∫ ∞0
λr+y−1
pr+y−1e−λ/pd(
λ
p)
=(1− p)rpy
y!Γ(r)Γ(r + y)
=Γ(r + y)
y!Γ(r)py(1− p)r =
(y + r − 1
y
)(1− p)rpy.
since Γ(y) =∫∞
0 xy−1e−xdx. Poisson and NB distributions are related
as:
P (λ) = limr→∞
NB
(r,
λ
λ + r
)since taking expectation on both sides, λ = rp
1−p ⇒ p = λλ+r . Note that
E(Y ) = limr→∞
pr
1− p= lim
r→∞
rλ+r · λ
1− λλ+r
= λ
V ar(Y ) = limr→∞
pr
(1− p)2= lim
r→∞
rλ+r · λ
(1− λλ+r)
2= λ.
In fitting a negative binomial distribution to the CEB data,
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 136
MSH3 Generalized linear model Ch. 6 Count data models
> library(MASS)
> glm.nb(y ~ offset(ln)+dur+resid+edu)
Call: glm.nb(formula = y ~ offset(ln) + dur + resid + edu,
init.theta = 809399.1036, link = log)
Coefficients:
(Intercept) dur2 dur3 dur4 dur5 dur6
-0.11709 0.99694 1.36939 1.61375 1.78489 1.97640
resid2 resid3 edu2 edu3 edu4
0.11243 0.15166 0.02297 -0.10128 -0.31016
Degrees of Freedom: 69 Total (i.e. Null); 59 Residual
Null Deviance: 3730
Residual Deviance: 70.66 AIC: 524.1
There were 27 warnings (use warnings() to see them)
> data=read.table("CEBMeanVar.txt")
> mug=data[,1] #mean by group
> var=data[,2] #var by group
> mean(mug)
[1] 3.727609
> mean(var)
[1] 3.615435
There is no improvement in model fit (Residual Deviance & AIC) as the
data is equi-dispersed across groups.
For mean regression, we have
E(Yi) =pir
1− pi= exp(xiβ) ⇒ pi =
exp(xiβ)
r + exp(xiβ)
and Yi ∼ NB
(r,
exp(xiβ)
r + exp(xiβ)
)where the pi is expressed in terms of
regression parameter β and the shape parameter r is an independentparameter.
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 137
MSH3 Generalized linear model Ch. 6 Count data models
> #AIDS deaths
> no=c(0,1,2,3,1,5,10,17,23,31,20,25,37,45)
> time=c(1:14)
>
> #NB mean regression
> logl1 <- function(pars)
+ b0<-pars[1]
+ b1<-pars[2]
+ r<-pars[3]
+ p=exp(b0+b1*time)/(r+exp(b0+b1*time))
+ l1=sum(lgamma(no+r)-lgamma(no+1)-lgamma(r)+r*log(1-p)+no*log(p))
+ return(-l1)
+
> mreg=optim(c(0.37571,0.25365,10),logl1,method = "BFGS",hessian=T)
> #BFGS is a quasi-Newton method also known as variable metric algorithm
> OI1<-solve(mreg$hessian)
> se1<-sqrt(diag(OI1))
> par1=mreg$par
> rbind(par1,se1)
[,1] [,2] [,3]
par1 0.04184276 0.28891875 12.59239
se1 0.39073410 0.03941281 11.06966
> AIC1=mreg$value*2+length(par1)*2
> AIC1
[1] 83.76235
> library(MASS) #alternative way using glm.nb
> summary(glm.nb(no~time, link = log))
Estimate Std. Error z value Pr(>|z|)
(Intercept) 0.04179 0.34097 0.123 0.902
time 0.28892 0.03308 8.734 <2e-16 ***
---
(Dispersion parameter for Negative Binomial(12.5919) family taken as 1)
AIC: 83.762
Theta: 12.59
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 138
MSH3 Generalized linear model Ch. 6 Count data models
Std. Err.: 9.81
Warning while fitting theta: alternation limit reached
The AIC of 83.762 is lower than 85.358 for Poisson regression in Chapter 1 showingimproved model fit after allowing for model complexity. Equivalently, one mayconsider logistic regression for pi. Then E(Yi) = exp(ln r+xiβ) implying that β1remains unchange and β0 drops by ln r.
§4.7.2 Generalized Poisson distribution
A random variable Y follows a GP(r, λ) distribution if its pmf is given by
fGP (y) =
λ(λ+ ry)y−1 exp[−(λ+ ry)]
y!, y = 0, 1, . . . ;
0, y > s when r < 0
where λ, |r| < 1 and s ≥ 4 is the largest natural number such that λ+ rs > 0 toensure λ + ry > 0 when r < 0. The two parameters r and λ are subject to thefollowing constraints:
1. λ ≥ 0,
2. max
−1,−λ
s
< r < 1.
The mean and variance for Y are given by respectively
E(Y ) =λ
1− rand V ar(Y ) =
λ
(1− r)3=
E(Y )
(1− r)2. (1)
The shape of GP distribution is controlled by r. A negative, zero and positive rindicates underdispersion, equidispersion and overdispersion, respectively.
0 3 6 9 13 17 21 25 29 33 37 41 45 49
0.00
0.05
0.10
0.15
0.20
r=−0.1, m=4.5, v=3.8r=0.3, m=7.1, v=14.6r=0.6, m=12.5, v=78.1r=0.9, m=50, v=5000
pmf of GP across r when lam=5
0 3 6 9 13 17 21 25 29 33 37 41 45 49
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
lam=1, m=1.7, v=4.6lam=2, m=5, v=13.9lam=5, m=11.7, v=32.4lam=7, m=16.7, v=46.3
pmf of GP across lam when r=0.4
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 139
MSH3 Generalized linear model Ch. 6 Count data models
For mean regression, we have
E(Yi) =λi
1− r= exp(xiβ) ⇒ λi = (1− r) exp(xiβ)
and Yi ∼ GP (r, (1− r) exp(xiβ)) where the λi is expressed in terms of regressionparameter β and the shape parameter r is an independent parameter.
> #GP regression
> logl3 <- function(pars)
+ b0<-pars[1]
+ b1<-pars[2]
+ r<-pars[3]
+ lam=(1-r)*exp(b0+b1*time)
+ l3=sum(log(lam)+(no-1)*log(lam+r*no)-(lam+r*no)-lgamma(no+1))
+ return(-l3)
+
> gpreg=optim(c(0.3,0.25,0.01),logl3,method = "BFGS",hessian=T)
Some Warning messages:
...
> OI3<-solve(gpreg$hessian)
> se3<-sqrt(diag(OI3))
> par3=gpreg$par
> rbind(par3,se3) #r>0 implies overdispersion
[,1] [,2] [,3]
par3 0.3320324 0.25763373 0.3288926
se3 0.3443768 0.02997835 0.1419665
> AIC3=gpreg$value*2+length(par3)*2
> AIC3
[1] 82.36633
> preg=glm(no~time, family=poisson(link=log)) #Poisson reg
> par0=preg$coeff
> c0=function(time) exp(par0[1]+par0[2]*time) #Poisson reg
> c1=function(time) exp(par1[1]+par1[2]*time) #NB reg
> c1v=function(time) exp(par1[1]+par1[2]*time)*
(par1[3]+exp(par1[1]+par1[2]*time))/par1[3]
> c3=function(time) exp(par3[1]+par3[2]*time) #GP reg
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 140
MSH3 Generalized linear model Ch. 6 Count data models
> c3v=function(time) exp(par3[1]+par3[2]*time)/(1-par3[3])^2
> plot(time,no,xlab="3 month periods: Jan-March 1983 to April-June 1986",
ylab="Number of AIDS deaths", cex=2,pch=20,col=’black’ )
> curve(c0,1,14,add=TRUE,col=’blue’,lwd=2)
> curve(c1,1,14,add=TRUE,col=’gray50’,lwd=2)
> curve(c1v,1,14,add=TRUE,col=’gray50’,lwd=2,lty=2)
> curve(c3,1,14,add=TRUE,col=’magenta’,lwd=2)
> curve(c3v,1,14,add=TRUE,col=’magenta’,lwd=2,lty=2)
> abline(lsfit(time,no),col=’red’,lwd=2)
> legend(0.1*max(time),0.9*max(no),text.width=1.2,y.intersp=1.2,bty="n",
c("linear mean","Poisson mean","NB mean","NB var","GP mean","GP var"),
col = c("red","blue","gray50","gray50","magenta","magenta"),
lty = c(1,1,1,2,1,2),lwd=c(2,2,2,2,2,2),cex=1)
The AIC improves further from 83.762 (NB) to 82.366 (GP) and the estimates arenearly the same as those of Poisson regression. The following diagram comparesall fits for mean and variance.
2 4 6 8 10 12 14
010
2030
40
3 month periods: Jan−March 1983 to April−June 1986
Num
ber
of A
IDS
dea
ths
linear meanPoisson meanNB meanNB varGP meanGP var
linear meanPoisson meanNB meanNB varGP meanGP var
§4.8 Model for excess zero
The model concerns a random event containing excess zero counts.
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 141
MSH3 Generalized linear model Ch. 6 Count data models
§4.8.1 Zero-inflated Poisson model
Proposed by Lambert (1992), the model contains two mixture components fortwo zero generating processes: a degenerate distribution that generates structuralzeros and a Poisson, Negative Binomial or GP distributions that generates counts,some of which may be zero. With a Poisson mixture, it is given by:
Pr(Y = 0) = π + (1− π)e−λ
Pr(Y = y) = (1− π)λye−λ
y!, y ≥ 1
where λ is the expected Poisson count and π is the probability of extra zeros. Anexample is the number of certain insurance claims which is zero-inflated by thosepeople who have not bought any insurance and thus are unable to claim. Themean and variance are
E(Y ) = (1− π)λ and V ar(Y ) = λ(1− π)(1 + λπ)
since
V ar(Y ) = E(Y 2)− [E(Y )]2 = (1− π)(λ2 + λ)− (1− π)2λ2
= (1− π)[λ2 + λ− (1− π)λ2] = λ(1− π)(1 + λπ).
The method of moments estimators are given by
λ =s2 + y2 − y
yand π =
s2 − ys2 + y2 − y
where y is the sample mean and s2 is the sample variance. The maximum likeli-hood estimator for λ can be found by solving the following equation
y(1− e−λ) = λ(
1− n0n
)(2)
for λ where n0n is the observed proportion of zeros. This can be solved by iterations
or using the R module uniroot, and the maximum likelihood estimator for π isgiven below (proofs in assignment):
π = 1− y
λ.
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 142
MSH3 Generalized linear model Ch. 6 Count data models
§4.8.2 Hurdle model
Proposed by Mullahy (1986), the model contains two mixture components: adegenerate model for measuring whether the outcome overcomes a hurdle (say zerofor zero-inflated data) and a truncated model like Poisson or negative binomial(NB) distributions explaining those outcomes that pass the hurdle. The model isgiven by:
Pr(Y = 0) = π
Pr(Y = y) = (1− π)λye−λ
y!(1− e−λ), y ≥ 1
The loglikelihood function is
`(π, λ) = n0 ln π + n1 ln(1− π) + lnλ
n1∑i=1
yi − n1λ−n1∑i=1
yi!− n1 ln(1− e−λ)
where n0 is the count of 0 and n1 is the count of yi > 0. Hence the MLE areπ = n0
n and λ is the solution to (2).
§4.9 Poisson mixture model
Assume that there are m subjects each with n repeated measurement Yij. If
subject i comes from group k at probability πk, k = 1, . . . , G such thatG∑k=1
πk = 1,
then Yij ∼ P (µjk) where µjk = exp(xjβk). Then the observed data loglikelihood
`(y|θ) =m∑i=1
log
[G∑k=1
πk
(n∏j=1
e−µjkµyijjk
yij!
)](3)
where θ = (βT1 , . . . ,βTG, π1, . . . , πG−1) and y is a vector of yij. Define the group
membership Iik = 1 if subject i belongs to group k. Then an estimate of Iik is
Iik =
πk
(n∏j=1
e−µjkµyijjk
yij!
)G∑
k′=1
πk′
(n∏j=1
e−µjk′µyijjk′
yij!)
) . (4)
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 143
MSH3 Generalized linear model Ch. 6 Count data models
Parameters can be estimated using the ML method by maximizing `(y|θ) in (3)with respect to θ applying optim again. Other R package for mixture modelssay flexmix can’t handle group membership for subject (cluster) rather thanindividual observation. Alternatively, one may use the EM method:
E-step: estimate the unobserved group membership Iik in (4) in E-step and
πk =m∑i=1
Iik/n.
M-step: estimate β = (βT1 , . . . ,βTG) using the complete data likelihood
`c(y|β, I) = ln
m∏i=1
G∏k=1
[πk
(n∏j=1
e−µjkµyijjk
yij!
)]Iik=
m∑i=1
G∑k=1
Iik ln πk +m∑i=1
G∑k=1
Iik
[n∑t=1
(−µjk − yij lnµjk − ln yij!)
]where I is a vector of Iik. We consider a Cannabis offences data in Sydney andfit a 2-group mixture model using the ML method.
> ym=read.table("CannabisData2.txt")
> y1 #high
V1 V2 V3 V4 V5 V6 V7 V8
11 7 8 11 16 9 9 3 9
12 22 17 12 20 22 13 19 15
13 18 9 13 16 17 22 17 26
14 18 13 5 17 24 12 9 6
15 47 9 13 15 17 8 14 5
17 18 22 13 16 34 35 5 16
25 22 15 9 18 15 12 8 7
28 30 5 13 10 18 5 3 8
29 30 25 26 19 14 16 13 5
42 9 11 7 26 6 5 3 4
43 5 4 21 5 13 7 6 4
> y2 #low
V1 V2 V3 V4 V5 V6 V7 V8
1 6 2 4 2 3 1 1 2
2 2 1 1 2 0 1 1 2
3 7 5 6 4 7 8 5 5
5 8 3 6 8 8 2 4 1
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 144
MSH3 Generalized linear model Ch. 6 Count data models
6 4 7 13 6 12 1 2 3
7 2 0 0 1 0 0 0 4
8 4 8 10 3 2 3 5 13
9 4 2 5 4 4 3 2 1
10 5 4 6 6 5 3 2 5
16 1 2 2 2 1 2 1 3
18 5 3 0 2 1 0 0 1
19 4 5 6 2 2 1 1 2
20 1 2 2 5 2 1 2 4
21 3 2 0 5 1 0 2 2
22 6 3 0 3 1 0 1 1
23 2 0 3 2 7 5 3 6
24 6 11 3 9 5 9 3 7
26 3 5 3 4 0 2 6 2
27 9 6 4 2 7 3 2 3
30 0 0 2 0 1 0 0 0
31 0 0 0 0 0 0 0 1
32 2 0 1 0 4 1 0 0
33 4 5 1 2 2 2 1 1
34 10 11 3 3 1 4 0 1
35 3 0 1 2 2 0 1 0
36 11 2 2 2 5 14 12 3
37 8 6 7 6 4 4 0 3
38 2 0 3 0 2 3 1 0
39 3 0 1 3 1 4 0 5
40 1 1 6 0 1 1 0 1
41 5 10 2 3 4 2 2 7
> ym=ym[-4,] #delete outlier
> m=nrow(ym)
> time=1:8
> l1=rep(0,m)
> l2=rep(0,m)
> l=rep(0,m)
>
> logl <- function(pars)
+ b01<-pars[1]; b11<-pars[2]; b02<-pars[3]; b12<-pars[4]; pi<-pars[5]
+ mu1=exp(b01+b11*time)
+ mu2=exp(b02+b12*time)
+ for (i in 1:m)
+ l1[i]=prod(exp(-mu1)*mu1^ym[i,]/factorial(ym[i,]))
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 145
MSH3 Generalized linear model Ch. 6 Count data models
+ l2[i]=prod(exp(-mu2)*mu2^ym[i,]/factorial(ym[i,]))
+ l[i]=pi*l1[i]+(1-pi)*l2[i]
+
+ ll=sum(log(l))
+ return(-ll)
+
>
> mixreg=optim(c(1.87,-0.25,1.00,-0.05,0.7),logl,method = "BFGS",hessian=T)
Warning messages: ...
> OIm<-solve(mixreg$hessian)
> sem<-sqrt(diag(OIm))
> parm=mixreg$par
> rbind(parm,sem)
[,1] [,2] [,3] [,4] [,5]
parm 2.9755017 -0.08009246 1.43787948 -0.07483189 0.26166373
sem 0.0588963 0.01260854 0.07516145 0.01600812 0.06780302
> AICm=mixreg$value*2+length(parm)*2
> AICm
[1] 2041.832
> ind=rep(0,m)
> b01=parm[1]; b11=parm[2]; b02=parm[3]; b12=parm[4]; pi=parm[5]
> mu1=exp(b01+b11*time)
> mu2=exp(b02+b12*time)
> for (i in 1:m)
+ l1[i]=prod(exp(-mu1)*mu1^ym[i,]/factorial(ym[i,]))
+ l2[i]=prod(exp(-mu2)*mu2^ym[i,]/factorial(ym[i,]))
+ ind[i]=pi*l1[i]/(pi*l1[i]+(1-pi)*l2[i])
+
> ind=round(ind,2)
> ind
[1] 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 1 0 0 0 0 0 0 0 1 0 0 1 1 0 0 0 0 0 0 0 0 0 0
[39] 0 0 1 1
> sum(ind)/m
[1] 0.2619048
> mi=1:m
> gp1=mi[ind==1]
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 146
MSH3 Generalized linear model Ch. 6 Count data models
> gp2=mi[ind==0]
> y1=ym[gp1,]
> y2=ym[gp2,]
> ym1=apply(y1,2,mean)
> ym2=apply(y2,2,mean)
> n1=length(gp1)
> n2=length(gp2)
> c(n1,n2)
[1] 11 31
1 2 3 4 5 6 7 8
010
2030
40
time
Num
ber
of c
anna
bis
offe
nces
high fitted meanhigh obs meanhigh obs varlow fitted meanlow obs meanlow obs var
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
0.00
0.05
0.10
0.15
0.20
0.25
t=2t=4t=6t=8
pmf of the 2 groups for time=2,4,6,8
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
0.00
0.05
0.10
0.15
high level grouplow level groupoverall
pmf of the 2 groups and overall for time=1
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 147
MSH3 Generalized linear model Ch. 6 Count data models
§4.10 Model selection example
Maxell (1961) discusses a 5 × 4 table giving the number of boys with 4 differentratings for disturbed dreams in 5 different age groups. The higher the rating themore the boys suffer from disturbed dreams.
RatingAge 4 3 2 15-7 7 3 4 78-9 13 11 15 10
10-11 7 11 9 2312-13 10 12 9 2814-15 3 4 5 32
One way of modeling the data is to let the number in the (i, j)-th cell be Yij andassume that Yij ∼ Poisson(µij).
Given∑5
i=1
∑4j=1 Yij = 223, the joint conditional distribution of the Yij’s is multi-
nomial. We look for a pattern in the µij relating to the factors age and rating.We use log link.
The saturated model Ω : lnµij = µ0 + αi + βj + (αβ)ij
where αi is the age group effect, α1 = β1 = 0,
βj is the dream rating effect,
(αβ)ij is the interaction effects and (αβ)1j = 0 = (αβ)i1.
The null model φ : lnµij = µ0 where
µij = y =223
20= 11.15, µ0 = lnµij = ln(11.15) = 2.411
> y=c(7,3,4,7,13,11,15,10,7,11,9,23,10,12,9,28,3,4,5,32)
> age=factor(c(1,1,1,1,2,2,2,2,3,3,3,3,4,4,4,4,5,5,5,5))
> rate=factor(c(1,2,3,4,1,2,3,4,1,2,3,4,1,2,3,4,1,2,3,4))
> glm1=glm(y~age+rate,family=poisson)
> summary(glm1)
Call:
glm(formula = y ~ age + rate, family = poisson)
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 148
MSH3 Generalized linear model Ch. 6 Count data models
Deviance Residuals:
Min 1Q Median 3Q Max
-2.86383 -0.71932 -0.05721 0.59370 2.53159
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 1.32623 0.26103 5.081 3.76e-07 ***
age2 0.84730 0.26082 3.249 0.00116 **
age3 0.86750 0.26004 3.336 0.00085 ***
age4 1.03302 0.25410 4.065 4.80e-05 ***
age5 0.73967 0.26523 2.789 0.00529 **
rate2 0.02469 0.22224 0.111 0.91153
rate3 0.04879 0.22093 0.221 0.82522
rate4 0.91629 0.18708 4.898 9.69e-07 ***
---
Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1
(Dispersion parameter for poisson family taken to be 1)
Null deviance: 94.607 on 19 degrees of freedom #n-p=20-1
Residual deviance: 32.457 on 12 degrees of freedom #n-p=20-1-3-4
AIC: 129.57
Number of Fisher Scoring iterations: 5
> ey=glm1$fitted.values
> ey
1 2 3 4 5 6 7 8
3.766816 3.860987 3.955157 9.417040 8.789238 9.008969 9.228700 21.973094
9 10 11 12 13 14 15 16
8.968610 9.192825 9.417040 22.421525 10.582960 10.847534 11.112108 26.457399
17 18 19 20
7.892377 8.089686 8.286996 19.730942
> n=length(y)
> mu=mean(y)
> mu
[1] 11.15
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 149
MSH3 Generalized linear model Ch. 6 Count data models
> D.null=2*sum(y*log(y/mu)-(y-mu))
> D.null
[1] 94.60676
> D.add=2*sum(y*log(y/ey)-(y-ey)) #additive age_rate model
> D.add
[1] 32.4571
> AIC.add=-2*sum(y*log(ey)-ey-log(factorial(y)))+2*(1+3+4)
> AIC.add
[1] 129.5738
To test if β1 = β2 = β3, i.e., rating can be collapsed to 2-levels, the change indeviance, as compared to the saturated model with 0 deviance and 0 d.f., is 4.2581which distributed as χ2
10. Since the p-value > 0.05, accept H0.
> r1=factor(c(1,1,1,2,1,1,1,2,1,1,1,2,1,1,1,2,1,1,1,2))
> glm3=glm(y~age*r1,family=poisson)
> summary(glm3)
Call:
glm(formula = y ~ age * r1, family = poisson)
Deviance Residuals:
Min 1Q Median 3Q Max
-0.8260 -0.3434 0.0000 0.1203 1.0049
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 1.5404 0.2673 5.764 8.22e-09 ***
age2 1.0245 0.3116 3.288 0.00101 **
age3 0.6568 0.3293 1.994 0.04613 *
age4 0.7949 0.3220 2.469 0.01356 *
age5 -0.1542 0.3934 -0.392 0.69517
r12 0.4055 0.4629 0.876 0.38108
age2:r12 -0.6678 0.5830 -1.145 0.25203
age3:r12 0.5328 0.5430 0.981 0.32644
age4:r12 0.5914 0.5313 1.113 0.26567
age5:r12 1.6740 0.5735 2.919 0.00351 **
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 150
MSH3 Generalized linear model Ch. 6 Count data models
---
Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1
(Dispersion parameter for poisson family taken to be 1)
Null deviance: 94.6068 on 19 degrees of freedom
Residual deviance: 4.2581 on 10 degrees of freedom #n-p=20-1-4-1-4
AIC: 105.37
Number of Fisher Scoring iterations: 4
For residual checks in GLM, the mean and variance are functionally related ininterpreting the residuals plot. To help in this problem, we generally use thescaled residuals of some form:
1. Ordinary residuals: Ri = Yi − µi but it is useful only in normal case.
2. Pearson (or standardized) residuals: Rpi =Yi − µi√a(φ)b′′(θi)
where Var(Yi) = a(φ)b′′(θi). For the Poisson case,
Rpi =Yi − µi√
µi
as Var(Yi) = E(Yi).
> r=residuals.glm(glm3,type="pearson")
> r
1 2 3 4 5
1.080123e+00 -7.715167e-01 -3.086067e-01 -2.014199e-15 -1.478018e-15
6 7 8 9 10
-5.547002e-01 5.547002e-01 -5.617334e-16 -6.666667e-01 6.666667e-01
11 12 13 14 15
-1.776357e-15 -2.222376e-15 -1.036952e-01 5.184758e-01 -4.147807e-01
16 17 18 19 20
6.713998e-16 -5.000000e-01 -1.332268e-15 5.000000e-01 -2.512148e-15
> par=glm3$coeff
> r1=(y[1]-exp(par[1]))/sqrt(exp(par[1]))
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 151
MSH3 Generalized linear model Ch. 6 Count data models
> r1 #agree with the first pearson residual
(Intercept)
1.080123
Results of the four models are summarized below:
Model Dev df Diff dev Diff df p-value Conclusion1. Saturated age*rate 0 0 - - - -2. Reduced sat. age*rate 4.2581 10 4.2581 10 0.935 Accept M23. Additive age+rate 32.457 12 32.457 12 0.001 Reject M3, favor M14. Null 94.607 19 62.15 7 0.000 Reject M4, favor M3
Note that M2 and M3 are not nested.
Reference:
1. Little, J.A. (1979) The General Linear Model and Direct Standardization AComparison. Sociological methods and research, 7, 475-501.
2. Lambert, D. (1992) Zero-Inflated Poisson Regression, with an Application toDefects in Manufacturing. Technometrics, 34: 1-14.
3. Mullahy, J. (1986) Specification and testing of some modified count datamodels. Journal of Econometrics, 33, 341-365.
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 152
MSH3 Generalized linear model Ch. 6 Count data models
Appendix
Definition: A stochastic process N(t), t ≥ 0 is a counting process if N(t)represents the total number of events that have occured by time t. Note: N(t) ≥ 0.If s < t, N(s) ≤ N(t).
Definition: A stochastic process is independent increment the number of eventsin disjoint intervals are independent.
Definition: A stochastic process is stationary increment if the distributions ofN(t1 + s)−N(t2 + s) and N(t1)−N(t2) are equal.
Lemma: The counting process N(t), t ≥ 0 is a Poisson process with rate λ if
1. N(0) = 0
2. The process has stationary and independent increment.
3. Pr(N(h) = 1) = λh+ o(h)
4. Pr(N(h) ≥ 2) = o(h)
where f(x) = o(x) if f(x)x → 0 as x→ 0 and o(h)± o(h) = o(h).
Proof: Let pn(t) = Pr(N(t) = n). We have
p0(t+ h) = Pr(N(t) = 0, N(t+ h)−N(t) = 0)
= Pr(N(t) = 0) Pr(N(h) = 0) = p0(t)(1− λh+ o(h)) by (2-4)
Hence
p′0(t) = limh→0
p0(t+ h)− p0(t)h
= limh→0−λp0(t) +
o(h)
hp0(t) = −λp0(t)
p′0(t)
p0(t)= −λ ⇒ ln p0(t) = −λt+ c ⇒ p0(t) = ke−λt
Since p0(0) = Pr(N(0) = 0) = 1 ⇒ k = 1, we have p0(t) = e−λt. Then assuming
pn−1(t) =(λt)n−1e−λt
(n− 1)!using mathematical induction, we have
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 153
MSH3 Generalized linear model Ch. 6 Count data models
pn(t+ h) = Pr(N(t) = n,N(t+ h)−N(t) = 0)
+ Pr(N(t) = n− 1, N(t+ h)−N(t) = 1)
+n∑k=2
Pr(N(t) = n− k,N(t+ h)−N(t) = k)
= Pr(N(t) = n) Pr(N(h) = 0) + Pr(N(t) = n− 1) Pr(N(h) = 1) + o(h)
= pn(t)(1− λh+ o(h)) + pn−1(t)(λh+ o(h)) + o(h)
pn(t+ h)− pn(t)h
= −pn(t)λ+ pn(t)o(h)
h+ pn−1(t)λ+ pn−1(t)
o(h)
h+o(h)
hp′n(t) = −λpn(t) + λpn−1(t)
Implies
p′n(t) + λpn(t) = λ(λt)n−1e−λt
(n− 1)!=λntn−1e−λt
(n− 1)!
The solution to the differential equation
dy
dt+ h(t)y = g(t)
is
y =
∫e∫h(t)dtg(t)dt+ c
e∫h(t)dt
.
Now h(t) = λ and g(t) =λntn−1e−λt
(n− 1)!. We have
pn(t) =
∫(e∫λdt)[λ
ntn−1e−λt
(n−1)! ]dt+ c
e∫λdt
=
∫(eλt)[λ
ntn−1e−λt
(n−1)! ]dt+ c
eλt
=λn∫
tn−1
(n−1)!dt+ c
eλt=λn tn
n(n−1)! + c
eλt
= e−λt(
(λt)n
n!+ c
).
Since pn(0) = Pr(N(0) = n) = 0⇒ c = 0, n ≥ 1, we have
pn(t) =e−λt(λt)n
n!
which is the pmf of Poisson distribution.
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 154