Comparison of spline- and loess-based approaches for the ... · 2.2 Loess-based approach The...
Transcript of Comparison of spline- and loess-based approaches for the ... · 2.2 Loess-based approach The...
Comparison of spline- and loess-based approaches for the estimation of
child mortality
Richard Silverwood and Simon Cousens
London School of Hygiene and Tropical Medicine
16th April 2008
Contents
1 Introduction 2
2 Overview of methods 2
2.1 Spline-based approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.2 Loess-based approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3 General issues with the methods 4
3.1 Spline-based approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.2 Loess-based approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
4 Comparison of spline- and loess-based approaches 6
4.1 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
5 Incorporation of uncertainty 22
5.1 Spline-based approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
5.1.1 Random draw simulation approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
5.1.2 Analytic approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
5.1.3 Comparison with uncertainty intervals obtained using the loess-based approach . . . . . . . . . 23
5.2 Loess-based approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.2.1 Analytic approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
6 Further extensions and alternative approaches 38
6.1 Incorporation of sampling variability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
6.2 Multilevel modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
7 Summary 41
Appendix: Comparison of the datasets 42
1
1 Introduction
There are currently (at least) two approaches towards the estimation of childhood mortality: a ‘spline-based approach’
favoured by the Inter-agency Coordination Group on Child Mortality Estimation and detailed by Hill et al (the ‘Green
Book’) [1], and a ‘loess-based approach’ described by Murray et al [2].
This report aims to provide a brief overview of these methods (Section 2), discussing any outstanding issues
with their application (Section 3), and to compare the results obtained under each (Section 4). The incorporation of
uncertainty into both estimating procedures is examined (Section 5), and further extensions and alternative approaches
discussed (Section 6). The findings are briefly summarised, and the key issues remaining to be addressed highlighted
(Section 7).
2 Overview of methods
The spline-based approach (Section 2.1) and the loess-based approach (Section 2.2) are briefly summarised.
2.1 Spline-based approach
For each country, the spline-based approach proceeds as follows:
1. Assign weights to each observed value of infant or under-5 mortality:
• Assign weights based on data/study type, length of time before survey to which estimate refers, age group
of mother, etc.;
• If one survey contributes two types of estimates then reduce both sets of estimates to half their standard
weight.
2. Define knots:
• Work backwards in time from the most recent observation;
• Weights summed and a knot defined every time the sum of the weights reaches a multiple of 5;
• For last knot defined (i.e. earliest knot), remaining weights must sum to at least 5.
3. Using weighted least squares regression fit the linear spline model,
log(y) = β0 + β1x +K∑
k=1
bk(x− κk)+ + ε, (2.1)
where y is childhood mortality, x is year, κ1, . . . , κK are the K knot locations, (x− κk)+ is equal to 0 if x < κk
and x− κk if x ≥ κk, and ε is an error term which is assumed to be normally distributed.
4. Critically examine results:
• Identify any datasets that are clearly aberrant;
• Reduce weights for the entire aberrant dataset(s) by a constant factor — generally 0.5, 0.25 or 0.
5. Refit spline model using revised weights.
2
6. Decide whether to select the infant or under-5 mortality sequence of estimates as the more consistent series.
7. Corresponding values of the other indicator (the derived indicator) can be obtained using a model life table.
2.2 Loess-based approach
The general approach is to fit loess regression curves to the data using a variety of smoothing parameters to vary the
sensitivity to recent data trends.
The basic loess function is
log(y) = β0 + β1x + β2z + ε, (2.2)
where y is under-5 mortality, x is calendar year, z is an indicator variable taking value 1 if the observed value comes
from a vital registration system and value 0 otherwise, and ε is an error term which is assumed to be normally
distributed.
The loess function (2.2) is fitted using weighted least squares regression, with the weights corresponding to each
observed under-5 mortality value calculated using a separate weighting function. This weighting function is tuned by
a single parameter, α.
Let x0 be the time point at which a fitted loess curve is required and define ψ = ||x − x0|| to be the separation
between a time point x and x0. For α < 1, the weighting function is calculated using only the 100α% of observations
closest to x0 (i.e. with smallest values of ψ), using
w =
(1−
(ψ
Ψ
)3)3
, (2.3)
where w is the weight corresponding to the time point x, and Ψ is the maximum value of ψ among the 100α% of
observations with the smallest values of ψ. For α ≥ 1, all data are included and the weighting function becomes
w =
(1−
(ψ
Ψα1/2
)3)3
. (2.4)
These weighting functions are illustrated in Fig. 1 for Armenia.
For each country, the loess-based approach proceeds as follows:
1. Decide upon the minimum value of α which will be used (αmin):
• Calculate the minimum value of α which will ensure that at least 3 data points are always included in the
loess regression (as is required for the variance-covariance matrix associated with the regression coefficients
to be estimable);
• Examine the fitted loess curve corresponding to this α value. If this α value does not provide a sufficiently
smooth fit to the data then increase α until a sufficiently smooth fit is achieved.
• The resulting α value should be used as αmin, unless the country under consideration has fewer than 100,000
children younger than 5 years, in which case αmin is the maximum of this α value and 0.4.
2. Decide upon the maximum value of α which will be used (αmax):
3
1980 1985 1990 1995 2000 2005
020
4060
80
Year
Und
er−
5 m
orta
lity
(per
100
0)
1980 1985 1990 1995 2000 2005
0.0
0.2
0.4
0.6
0.8
1.0
Year
Wei
ght
Fig. 1: Data (left-hand plot; blue points represent vital registration data, black points represent non-vital registration data) and weight
functions calculated at 1990 (right-hand plot; red lines represent smaller α values, yellow lines represent larger α values) for Armenia.
• Examine the correlations between the fitted loess curves and the ordinary least squares fits for various
values of α;
• If correlation becomes almost perfect once a given value of α is passed, set αmax to be this value. Otherwise
use αmax = 2.
3. For each value of α in {αmin, αmin + 0.05, αmin + 0.10, . . . , 1.0, 1.1, . . . , αmax}:
• Calculate the weights associated with each observed under-5 mortality value using (2.3) or (2.4) as appro-
priate;
• Fit the loess function (2.2) using weighted least squares regression;
• Simulate 1000 random draws from the multivariate normal distribution defined by the estimated regression
coefficients and their variance-covariance matrix;
• For each of the 1000 random draws, calculate the estimated/predicted under-5 mortality at the required
time point, assuming non-vital registration data.
4. Pool the 1000 estimates/predictions per α value across the set of α values.
5. Calculate the final estimated/predicted under-5 mortality at the required time point as the median of these
pooled estimates/predictions, with an uncertainty interval corresponding to the 2.5th and 97.5th centiles of these
pooled estimates/predictions.
3 General issues with the methods
There remain some outstanding issues with both the spline- and loess-based approaches.
4
3.1 Spline-based approach
In the spline-based approach the fitted models are restricted to being piecewise linear on the log scale, which is
unlikely to be an accurate representation of the actual trend. The down-weighting of aberrant datasets in step 4 of
the procedure described in Section 2.1 is rather ad-hoc and introduces an element of subjectivity into the procedure,
as does the decision over whether the infant or under-5 mortality sequence of estimates is the more consistent series
in step 6. For a truly transparent and reproducible method, these areas of subjectivity would ideally be formalised.
Additionally, the spline-based approach does not currently include any quantification of the uncertainty surrounding
each estimate, for example through the use of uncertainty intervals.
3.2 Loess-based approach
There are several issues regarding the α values which are used. For example, setting αmin as the first acceptable value
above 0.10 and αmax as the last acceptable value below 2.00 is somewhat arbitrary, as is increasing α in increments
of 0.05 when less than 1 and 0.1 when greater than 1. The selection of αmin and αmax, as described in Section 2.2,
is also a little subjective. Additional subjectivity is introduced by the manner in which ‘extreme outliers that clearly
differ from the rest of the datapoints’ [2] are excluded from the analysis.
The effects of varying αmin and αmax are illustrated in Fig. 2–Fig. 5. In each, loess curves are fitted to the same
dataset (Belize) for α = {αmin, αmin+0.05, αmin+0.10, . . . , 1.0, 1.1, . . . , αmax} and under-5 mortality in 2015 predicted.
This dataset shows an overall decreasing trend, but with an apparently more recent increasing trend. In each case, the
left-hand plot shows the fitted loess curves and the right-hand plot shows the distribution of the simulated random
draws with the final predicted under-5 mortality and uncertainty interval in 2015.
In Fig. 2, αmin = 0.30 and αmax = 2.00. The loess curves corresponding to the lower values of α follow the more
recent increasing trend in the data. The loess curves corresponding to the higher α values ignore the more recent
increasing trend and are all very similar to one another. This results in a skewed distribution of the simulated random
draws and a wide, skewed uncertainty interval.
In Fig. 3, αmax is still 2.00, but αmin is increased to 0.60. The loess curves corresponding to the lower values of α
in Fig. 2 which follow the more recent increasing trend in the data are no longer present. This results in a less skewed
distribution of the simulated random draws and a narrower, less skewed uncertainty interval. Although the upper
bound of the uncertainty interval is reduced drastically, relatively little change is seen in the median and no change is
seen in the lower bound.
In Fig. 4, αmin = 0.30, similarly to in Fig. 2, but now αmax is decreased to 1.50. Some of the loess curves
corresponding to the higher α values in Fig. 2 which ignore the more recent increasing trend are no longer present.
This results in a lower density of simulated random draws at the lower end of the distribution, although the distribution
still remains highly skewed. Neither bound of the uncertainty interval changes greatly, but the median value increases
somewhat.
In Fig. 5, αmin = 0.60 and αmax = 1.50. The loess curves corresponding to both the lower α values and the higher
α values in Fig. 2 are no longer present. This again results in a less skewed distribution of the simulated random draws
and a narrower, less skewed uncertainty interval. Although the upper bound of the uncertainty interval is reduced
drastically, relatively little change is seen in either the median or the lower bound.
Thus it can be seen that varying αmin and αmax can in some circumstances make a large difference to the final
5
1970 1980 1990 2000 2010
2.0
2.5
3.0
3.5
4.0
4.5
5.0
Year
Log
unde
r−5
mor
talit
y (p
er 1
000)
2
3
4
5
Density
Pre
dict
ed lo
g un
der−
5 m
orta
lity
(per
100
0) in
201
5
0.0 0.5 1.0 1.5
(12.6)
(56.3)
(8.6)
Fig. 2: Fitted loess curves (left-hand plot; all data are vital registration data) and distribution of simulated random draws with final
predicted under-5 mortality and uncertainty interval in 2015 (right-hand plot; values in brackets are transformed back to the original scale)
using α = {0.30, 0.35, . . . , 1.00, 1.10, . . . , 2.00} in Belize.
predicted under-5 mortality and associated uncertainty interval. Generally, the lower the αmin value is set, the more
likely it is that (potentially spurious) trends in the most recent data which differ from the overall trend will be picked
up. The higher the αmax value is set, the more likely it is that the loess curves fitted for α values close to αmax are
essentially the same. When both low α values and high α values are included then the result may be wide, highly
skewed uncertainty intervals. As the selection of αmin and αmax is somewhat subjective, the potential consequences
should be borne in mind.
4 Comparison of spline- and loess-based approaches
4.1 Method
Estimated/predicted under-5 mortality under the spline- and loess-based approaches in the years 2000, 2005, 2010 and
2015 are compared in the 60 country datasets available in the UNICEF database. The countries included are shown
in Table 1.
The spline-based approach proceeds as described in Section 2.1. The final weights (those used in step 5) are
provided with the datasets and used here. For some countries (Lesotho and Zimbabwe), recently published spline-
based estimates are set to be constant, but this is ignored here.
The loess-based approach ignores the weights used for the spline-based approach. Instead, data which come from
a vital registration system are identified using the documentation provided with the data. The loess-based approach
then proceeds as described in Section 2.2.
For illustration, fitted spline and loess curves are shown in Fig. 6 and Fig. 7 for Armenia and Zimbabwe, respectively.
The fitted curves are seen to be very similar for Armenia, but for Zimbabwe they differ greatly, particularly since the
most recent knot in the spline-based approach.
6
1970 1980 1990 2000 2010
2.0
2.5
3.0
3.5
4.0
4.5
5.0
Year
Log
unde
r−5
mor
talit
y (p
er 1
000)
2
3
4
5
DensityP
redi
cted
log
unde
r−5
mor
talit
y (p
er 1
000)
in 2
015
0.0 0.5 1.0 1.5 2.0
(11.5)
(22.4)
(8.5)
Fig. 3: Fitted loess curves (left-hand plot; all data are vital registration data) and distribution of simulated random draws with final
predicted under-5 mortality and uncertainty interval in 2015 (right-hand plot; values in brackets are transformed back to the original scale)
using α = {0.60, 0.65, . . . , 1.00, 1.10, . . . , 2.00} in Belize.
1970 1980 1990 2000 2010
2.0
2.5
3.0
3.5
4.0
4.5
5.0
Year
Log
unde
r−5
mor
talit
y (p
er 1
000)
2
3
4
5
Density
Pre
dict
ed lo
g un
der−
5 m
orta
lity
(per
100
0) in
201
5
0.0 0.5 1.0
(14.2)
(59.1)
(8.9)
Fig. 4: Fitted loess curves (left-hand plot; all data are vital registration data) and distribution of simulated random draws with final
predicted under-5 mortality and uncertainty interval in 2015 (right-hand plot; values in brackets are transformed back to the original scale)
using α = {0.30, 0.35, . . . , 1.00, 1.10, . . . , 1.50} in Belize.
7
1970 1980 1990 2000 2010
2.0
2.5
3.0
3.5
4.0
4.5
5.0
Year
Log
unde
r−5
mor
talit
y (p
er 1
000)
2
3
4
5
DensityP
redi
cted
log
unde
r−5
mor
talit
y (p
er 1
000)
in 2
015
0.0 0.5 1.0 1.5
(12.2)
(23.1)
(8.7)
Fig. 5: Fitted loess curves (left-hand plot; all data are vital registration data) and distribution of simulated random draws with final
predicted under-5 mortality and uncertainty interval in 2015 (right-hand plot; values in brackets are transformed back to the original scale)
using α = {0.60, 0.65, . . . , 1.00, 1.10, . . . , 1.50} in Belize.
Armenia Fiji Maldives Somalia
Belarus Gambia Mexico Syria
Belize Georgia Micronesia, Fed. States Tajikistan
Brazil Ghana Moldova Thailand
Burkina Faso Guinea Mongolia Timor Leste
Burundi Guinea Bissau Nepal Togo
Cambodia Haiti Palau Trinidad & Tobabgo
Central African Republic Honduras Papua New Guinea Turkmenistan
Chad India Peru Tuvalu
China Iraq Russian Federation Ukraine
Colombia Jamaica Rwanda United Arab Emirates
Congo Kazakhstan Sao Tome & Principe Uruguay
Cote d’Ivoire Kyrgyzstan Senegal Uzbekistan
Egypt Lesotho Sierra Leone Venezuela
Ethiopia Malawi Solomon Islands Zimbabwe
Table 1: Countries included in the comparison of the spline- and loess-based approaches.
8
1980 1990 2000 2010
020
4060
8010
0
Year
Und
er−
5 m
orta
lity
(per
100
0)
1980 1990 2000 2010
020
4060
8010
0Year
Und
er−
5 m
orta
lity
(per
100
0)
Fig. 6: Fitted spline curve (left-hand plot; dashed vertical line indicates knot location) and loess curve (right-hand plot; blue points
represent vital registration data, black points represent non-vital registration data, dashed lines represent uncertainty intervals) for Armenia.
1960 1970 1980 1990 2000 2010
050
100
150
200
Year
Und
er−
5 m
orta
lity
(per
100
0)
1960 1970 1980 1990 2000 2010
050
100
150
200
Year
Und
er−
5 m
orta
lity
(per
100
0)
Fig. 7: Fitted spline curve (left-hand plot; dashed vertical lines indicate knot locations) and loess curve (right-hand plot; all data are
non-vital registration data, dashed lines represent uncertainty intervals) for Zimbabwe.
9
4.2 Results
Estimated/predicted under-5 mortality in the years 2000, 2005, 2010 and 2015 under the two different methods are
presented in Tables 2–4. Although it is clear that there are often discrepancies between the spline- and loess-based
estimated/predicted values, these differences are more easily interpreted when the results are presented graphically.
An important aspect of the spline-based approach is the down-weighting of aberrant datasets in step 4 of the
procedure described in Section 2.1, referred to henceforth as ‘ad-hoc weight adjustment’. As this ad-hoc weight ad-
justment is conducted on a largely subjective basis, it is perhaps likely that for countries with a great deal of ad-hoc
weight adjustment the final estimates/predictions of under-5 mortality will be less comparable with the loess-based
equivalents, for which no additional weighting has been applied. Thus when analysing the results, countries with no
ad-hoc weight adjustments and countries with more complex ad-hoc weight adjustments are examined separately. In
some countries the ad-hoc weight adjustment involves only a down-weighting of data coming from vital registration
systems. As the loess-based approach also effectively downweights the influence of vital registration data, countries in
which this is the case are included with those where there is no ad-hoc weight adjustment.
Fig. 8–15 show the loess-based estimates/predictions of under-5 mortality plotted against the spline-based esti-
mates/predictions for each year separately. Countries where the estimated/predicted under-5 mortality using one
approach is 20–33.3% greater than the mean estimated/predicted under-5 mortality for the two different approaches
are highlighted with solid markers. This corresponds to the larger estimate/prediction being 1.5–2 times greater than
the smaller. Countries where the estimated/predicted under-5 mortality using one approach is more than 33.3% greater
than the mean estimated/predicted under-5 mortality for the two different approaches are labelled. This corresponds
to the larger estimate/prediction being more than twice the smaller.
It can be seen that in countries with either no ad-hoc weight adjustments or merely a down-weighting of the
vital registration data (Fig. 8, Fig. 10, Fig. 12 and Fig. 14) the estimated/predicted under-5 mortality under the two
different approaches is generally more similar than in countries with more complex ad-hoc weight adjustments (Fig. 9,
Fig. 11, Fig. 13 and Fig. 15). In countries with either no ad-hoc weight adjustments or merely a down-weighting of the
vital registration data there appears to be a reasonably similar proportion of countries where the spline-based method
provides the greater estimate/prediction and where the loess-based method provides the greater estimate/prediction.
This is not so true for countries with more complex ad-hoc weight adjustments, though interpretation of this is more
difficult. Also, within each category of ad-hoc weight adjustment, the estimations/predictions under the two different
approaches tend to get less similar as time progresses.
Table 5 summarises the observations from Fig. 8–15 by tabulating, for each category of ad-hoc weight adjustment
and each year separately, the differences between the spline- and loess-based estimates/predictions.
For countries with no ad-hoc weight adjustments or merely a down-weighting of the vital registration data, in
2000 all estimates/predictions are within 10% of the mean, showing good agreement between the two approaches.
As time progresses, however, predictions become a little less similar — by 2015 only 56% remain within 10% of the
mean, with 8% more than 20% from the mean. In 2000 approximately as many countries have a greater spline-based
estimate/prediction as have a greater loess-based estimate/prediction. The proportion of countries where the loess-
based prediction is greater than the spline-based prediction increases so that by 2015 this is true for 68% of countries.
Thus, although the numbers involved are small, there is some evidence of a tendency for the loess-based approach to
10
Cou
ntry
2000
.520
05.5
2010
.520
15.5
Splin
eLoe
ssSp
line
Loe
ssSp
line
Loe
ssSp
line
Loe
ss
Arm
enia
36.3
35.3
(32.
2,39
.0)
25.5
27.1
(21.
8,30
.8)
17.9
21.3
(14.
3,25
.3)
12.5
16.7
(9.1
,21
.2)
Bel
arus
17.3
13.1
(11.
1,17
.8)
13.9
10.7
(8.9
,13
.0)
11.2
8.8
(6.0
,11
.5)
9.0
7.4
(3.8
,10
.4)
Bel
ize1
19.3
21.2
(19.
8,23
.3)
12.8
15.3
(13.
7,18
.5)
8.5
11.1
(9.5
,14
.8)
5.6
8.0
(6.6
,11
.8)
Bra
zil
29.6
34.2
(30.
6,38
.5)
21.2
26.7
(22.
9,31
.1)
15.3
20.9
(17.
1,25
.1)
11.0
16.3
(12.
6,20
.3)
Bur
kina
Faso
193.
919
6.9
(184
.9,21
2.7)
202.
319
1.1
(173
.6,22
8.0)
211.
118
4.7
(162
.8,24
8.0)
220.
217
8.3
(152
.3,27
1.7)
Bur
undi
180.
818
3.8
(171
.4,19
6.0)
181.
018
1.5
(165
.5,20
4.1)
181.
117
8.9
(158
.8,21
4.2)
181.
217
5.9
(152
.5,22
5.3)
Cam
bodi
a10
4.2
104.
8(9
6.3,
114.
1)85
.491
.0(6
3.3,
107.
3)70
.080
.9(3
8.3,
102.
5)57
.472
.1(2
2.9,
98.0
)
Cen
tral
Afr
ican
Rep
ublic
186.
118
6.3
(165
.2,20
1.7)
176.
618
7.6
(158
.1,21
5.6)
167.
618
8.4
(150
.7,23
2.8)
159.
018
9.3
(143
.9,25
1.9)
Cha
d220
5.3
203.
6(1
93.0
,21
4.5)
208.
720
3.5
(185
.3,22
3.7)
212.
220
3.7
(175
.4,23
6.1)
215.
720
3.9
(162
.6,24
8.0)
Chi
na36
.631
.1(2
6.4,
36.7
)25
.427
.5(2
2.1,
35.0
)17
.624
.3(1
7.2,
35.5
)12
.321
.5(1
2.9,
36.2
)
Col
ombi
a25
.924
.5(2
1.4,
28.8
)21
.419
.9(1
6.4,
25.7
)17
.716
.0(1
2.7,
23.1
)14
.612
.9(9
.8,
20.8
)
Con
go11
7.0
114.
0(1
00.7
,12
8.2)
124.
811
5.8
(95.
3,14
7.2)
133.
211
6.1
(89.
2,17
6.9)
142.
211
6.9
(83.
0,21
3.1)
Cot
ed’
Ivoi
re2
137.
913
8.8
(125
.5,15
4.6)
131.
612
8.8
(102
.9,15
1.2)
125.
512
0.3
(80.
3,15
0.5)
119.
711
2.6
(61.
7,14
9.8)
Egy
pt2
50.5
54.4
(48.
8,63
.2)
37.7
40.3
(32.
7,50
.5)
28.1
30.2
(21.
7,40
.6)
21.0
22.6
(14.
4,32
.7)
Eth
iopi
a15
0.6
159.
5(1
46.1
,17
2.3)
127.
114
6.2
(117
.1,16
5.2)
107.
313
4.9
(91.
5,15
8.4)
90.6
124.
4(7
2.2,
152.
2)
Fiji
113
.115
.0(1
2.9,
20.4
)9.
811
.8(9
.6,
19.5
)7.
49.
3(7
.2,
18.2
)5.
67.
3(5
.3,
17.4
)
Gam
bia
131.
912
9.6
(119
.5,13
8.3)
116.
311
5.6
(103
.5,12
8.9)
102.
510
2.6
(89.
0,12
1.5)
90.3
91.2
(76.
8,11
5.2)
Geo
rgia
36.6
37.6
(30.
0,46
.7)
32.7
36.4
(28.
3,51
.2)
29.3
34.9
(24.
9,68
.5)
26.2
33.3
(21.
6,93
.1)
Gha
na11
2.7
114.
2(1
08.2
,12
0.6)
118.
710
9.2
(101
.0,13
2.4)
124.
910
3.8
(94.
1,14
8.8)
131.
598
.7(8
7.4,
168.
1)
Gui
nea
184.
318
5.5
(178
.9,19
3.1)
164.
816
5.3
(144
.7,17
5.9)
147.
314
8.1
(110
.2,16
2.8)
131.
713
2.6
(83.
2,15
1.2)
Table
2:
Est
imate
d/pre
dic
ted
under
-5m
ort
ality
(per
1000).
Spline-
base
dm
ethod
pro
vid
espoin
tes
tim
ate
sonly
.Loes
s-base
dm
ethod
pro
vid
espoin
tes
tim
ate
sand
95%
unce
rtain
tyin
terv
als
.
1Spline-
base
des
tim
ate
sdo
not
corr
espond
topublish
edvalu
esdue
topublish
edvalu
esbei
ng
der
ived
from
the
fitt
edin
fant
mort
ality
spline.
2Spline-
base
des
tim
ate
sdiff
ersl
ightl
yfr
om
publish
ed
valu
es.
3R
ecen
tpublish
edsp
line-
base
des
tim
ate
sare
const
ant
—th
isis
ignore
dher
e.
11
Cou
ntry
2000
.520
05.5
2010
.520
15.5
Splin
eLoe
ssSp
line
Loe
ssSp
line
Loe
ssSp
line
Loe
ss
Gui
nea
Bis
sau
217.
521
6.0
(211
.4,22
0.7)
202.
920
4.2
(195
.9,21
3.6)
189.
219
3.1
(179
.4,20
9.0)
176.
418
2.5
(164
.1,20
4.8)
Hai
ti10
9.2
116.
7(1
08.0
,12
6.2)
84.2
101.
6(8
2.3,
114.
3)64
.988
.9(6
1.8,
103.
4)50
.177
.8(4
6.5,
93.6
)
Hon
dura
s39
.640
.2(3
8.2,
42.5
)28
.732
.3(3
0.1,
35.4
)20
.825
.9(2
3.6,
29.9
)15
.120
.7(1
8.5,
25.2
)
Indi
a188
.294
.8(8
6.9,
114.
3)77
.286
.0(7
6.4,
130.
2)67
.577
.9(6
7.2,
146.
8)59
.170
.4(5
9.1,
162.
6)
Iraq
47.5
46.8
(42.
3,51
.9)
46.6
41.2
(35.
4,47
.9)
45.7
36.5
(27.
7,45
.5)
44.9
32.4
(21.
4,43
.8)
Jam
aica
31.9
21.9
(17.
2,30
.6)
31.3
19.5
(14.
1,35
.4)
30.6
17.2
(11.
3,42
.2)
29.9
15.1
(9.0
,51
.1)
Kaz
akhs
tan
42.9
39.1
(34.
4,45
.2)
31.0
31.0
(22.
7,37
.8)
22.3
25.3
(13.
3,33
.2)
16.1
20.7
(7.8
,29
.0)
Kyr
gyzs
tan
51.4
51.6
(45.
5,57
.7)
42.5
42.5
(35.
5,49
.5)
35.2
35.4
(27.
3,43
.3)
29.1
29.6
(20.
7,37
.9)
Les
otho
310
8.4
93.1
(85.
3,11
1.0)
131.
886
.3(7
6.2,
127.
2)16
0.2
79.8
(68.
1,14
7.5)
194.
773
.9(6
1.0,
171.
1)
Mal
awi
155.
316
8.9
(155
.0,17
9.4)
125.
315
0.0
(121
.8,16
2.5)
101.
113
3.5
(93.
9,14
7.5)
81.6
118.
8(7
3.0,
134.
1)
Mal
dive
s54
.153
.6(4
8.9,
58.8
)33
.335
.6(2
7.0,
42.3
)20
.424
.3(1
4.3,
30.8
)12
.616
.5(7
.6,
22.5
)
Mex
ico1
35.8
35.3
(31.
8,40
.4)
29.7
29.4
(25.
7,35
.9)
24.7
24.4
(20.
8,31
.9)
20.5
20.3
(16.
7,28
.2)
Mic
rone
sia,
Fed.
Stat
es46
.547
.1(4
2.6,
52.3
)41
.842
.9(3
6.5,
50.5
)37
.539
.1(3
1.0,
49.0
)33
.735
.6(2
6.6,
47.5
)
Mol
dova
24.3
27.5
(25.
3,29
.9)
19.8
22.6
(17.
7,25
.4)
16.1
18.9
(11.
8,22
.0)
13.1
15.9
(7.8
,19
.1)
Mon
golia
61.5
57.1
(51.
5,64
.8)
45.2
39.0
(29.
0,50
.2)
33.3
27.1
(15.
0,39
.7)
24.5
18.8
(7.8
,31
.1)
Nep
al86
.196
.5(8
6.6,
110.
8)63
.181
.8(6
5.8,
99.5
)46
.269
.8(4
9.7,
89.4
)33
.959
.5(3
7.5,
80.7
)
Pal
au1
9.9
3.1
(1.4
,6.
6)6.
71.
5(0
.4,
7.0)
4.5
0.7
(0.1
,9.
4)3.
00.
3(0
.0,
14.1
)
Pap
uaN
ewG
uine
a279
.683
.3(7
2.5,
95.5
)72
.977
.5(6
3.5,
95.8
)66
.872
.2(5
5.6,
94.9
)61
.267
.2(4
8.0,
94.7
)
Per
u41
.348
.0(4
2.6,
53.0
)27
.337
.7(2
9.7,
43.3
)18
.029
.6(2
0.5,
35.4
)11
.923
.2(1
4.2,
29.0
)
Rus
sian
Fede
rati
on23
.922
.0(1
7.0,
27.1
)16
.919
.4(1
3.5,
24.3
)11
.517
.3(1
1.5,
22.0
)7.
815
.5(9
.7,
19.8
)
Table
3:
Est
imate
d/pre
dic
ted
under
-5m
ort
ality
(per
1000)
conti
nued
.Spline-
base
dm
ethod
pro
vid
espoin
tes
tim
ate
sonly
.Loes
s-base
dm
ethod
pro
vid
espoin
tes
tim
ate
sand
95%
unce
rtain
ty
inte
rvals
.1
Spline-
base
des
tim
ate
sdo
not
corr
espond
topublish
edvalu
esdue
topublish
edvalu
esbei
ng
der
ived
from
the
fitt
edin
fant
mort
ality
spline.
2Spline-
base
des
tim
ate
sdiff
ersl
ightl
yfr
om
publish
edvalu
es.
3R
ecen
tpublish
edsp
line-
base
des
tim
ate
sare
const
ant
—th
isis
ignore
dher
e.
12
Cou
ntry
2000
.520
05.5
2010
.520
15.5
Splin
eLoe
ssSp
line
Loe
ssSp
line
Loe
ssSp
line
Loe
ss
Rw
anda
183.
418
6.3
(175
.4,19
8.9)
162.
718
2.5
(168
.1,20
3.6)
144.
417
8.9
(158
.0,21
0.2)
128.
117
5.3
(147
.6,21
6.8)
Sao
Tom
e&
Pri
ncip
e97
.271
.1(5
4.1,
88.7
)95
.962
.3(3
2.5,
86.4
)94
.655
.5(1
9.5,
84.7
)93
.349
.5(1
1.5,
83.3
)
Sene
gal
132.
613
2.0
(122
.0,14
3.1)
118.
711
9.5
(106
.9,13
7.6)
106.
310
7.8
(93.
6,13
3.2)
95.2
97.1
(81.
7,12
9.9)
Sier
raLeo
ne27
6.5
273.
9(2
62.6
,29
0.5)
271.
126
4.1
(249
.5,30
4.3)
265.
925
4.1
(236
.7,32
5.0)
260.
724
4.4
(224
.6,34
8.7)
Solo
mon
Isla
nds1
79.2
56.5
(42.
4,75
.2)
68.8
57.8
(38.
8,91
.8)
59.9
58.7
(34.
9,11
7.2)
52.1
59.3
(31.
5,15
1.5)
Som
alia
164.
815
8.6
(143
.6,17
5.2)
148.
513
3.8
(112
.5,16
1.8)
133.
811
4.7
(82.
1,15
4.2)
120.
698
.5(5
9.2,
145.
0)
Syri
a19
.921
.7(1
6.3,
25.4
)14
.516
.1(1
0.4,
19.7
)10
.711
.9(6
.5,
15.1
)7.
88.
8(4
.2,
11.7
)
Taj
ikis
tan
93.3
80.6
(71.
2,90
.9)
71.4
67.4
(54.
0,81
.6)
54.6
57.1
(36.
1,76
.0)
41.8
48.5
(23.
8,69
.8)
Tha
iland
12.9
14.4
(12.
4,16
.4)
8.4
10.3
(7.9
,12
.2)
5.5
7.4
(4.9
,9.
1)3.
65.
3(3
.0,
6.8)
Tim
orLes
te10
6.8
113.
7(9
8.6,
131.
8)61
.392
.5(6
3.1,
122.
9)35
.275
.4(3
7.9,
115.
8)20
.262
.5(2
2.8,
110.
6)
Tog
o12
4.0
129.
0(1
23.5
,13
5.6)
110.
611
9.6
(112
.5,13
1.5)
98.5
110.
5(1
01.8
,12
9.6)
87.8
102.
3(9
1.8,
127.
0)
Tri
nida
d&
Tob
ago
34.2
30.5
(25.
1,38
.4)
37.0
28.5
(22.
1,42
.9)
39.9
26.6
(19.
2,49
.4)
43.1
24.8
(16.
4,55
.8)
Tur
kmen
ista
n70
.664
.2(5
5.8,
74.0
)54
.142
.2(3
3.0,
52.7
)41
.428
.8(1
6.1,
40.2
)31
.719
.8(7
.7,
30.5
)
Tuv
alu1
35.7
34.9
(17.
2,69
.0)
30.3
29.2
(11.
2,73
.7)
25.7
24.5
(7.5
,78
.3)
21.8
20.6
(4.8
,82
.7)
Ukr
aine
22.8
22.3
(20.
4,25
.3)
23.6
20.6
(18.
2,23
.3)
24.4
18.8
(14.
8,21
.7)
25.2
17.2
(11.
7,20
.7)
Uni
ted
Ara
bE
mir
ates
10.3
9.5
(7.3
,11
.8)
8.5
6.9
(4.9
,9.
1)7.
14.
9(3
.2,
7.2)
5.8
3.5
(2.1
,5.
8)
Uru
guay
116
.019
.5(1
7.2,
25.1
)16
.216
.4(1
4.2,
22.4
)16
.513
.8(1
1.6,
20.0
)16
.811
.5(9
.5,
17.7
)
Uzb
ekis
tan
62.3
55.9
(49.
2,66
.0)
46.0
46.1
(40.
9,51
.3)
34.0
36.9
(31.
0,43
.6)
25.1
30.0
(21.
8,38
.0)
Ven
ezue
la24
.525
.6(2
3.6,
29.3
)21
.322
.1(1
9.9,
28.1
)18
.419
.1(1
6.8,
27.2
)16
.016
.5(1
4.1,
26.0
)
Zim
babw
e313
5.3
73.0
(64.
9,85
.0)
185.
568
.5(5
8.6,
88.2
)25
4.2
63.9
(52.
3,92
.6)
348.
559
.5(4
6.8,
96.0
)
Table
4:
Est
imate
d/pre
dic
ted
under
-5m
ort
ality
(per
1000)
conti
nued
.Spline-
base
dm
ethod
pro
vid
espoin
tes
tim
ate
sonly
.Loes
s-base
dm
ethod
pro
vid
espoin
tes
tim
ate
sand
95%
unce
rtain
ty
inte
rvals
.1
Spline-
base
des
tim
ate
sdo
not
corr
espond
topublish
edvalu
esdue
topublish
edvalu
esbei
ng
der
ived
from
the
fitt
edin
fant
mort
ality
spline.
2Spline-
base
des
tim
ate
sdiff
ersl
ightl
yfr
om
publish
edvalu
es.
3R
ecen
tpublish
edsp
line-
base
des
tim
ate
sare
const
ant
—th
isis
ignore
dher
e.
13
010
020
030
0Lo
ess−
base
d es
timat
e/pr
edic
tion
0 100 200 300Spline−based estimate/prediction
Fig. 8: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in the year 2000.5 in countries with either
no ad-hoc weight adjustments or merely a down-weighting of the vital registration data. Countries where estimated/predicted under-5
mortality using one approach is 20–33.3% greater than the mean estimated/predicted under-5 mortality for the two different approaches
are highlighted with solid markers. Countries where the estimated/predicted under-5 mortality using one approach is more than 33.3%
greater than the mean estimated/predicted under-5 mortality for the two different approaches are labelled.
Palau010
020
030
0Lo
ess−
base
d es
timat
e/pr
edic
tion
0 100 200 300Spline−based estimate/prediction
Fig. 9: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in the year 2000.5 in countries with more
complex ad-hoc weight adjustments. Countries where estimated/predicted under-5 mortality using one approach is 20–33.3% greater than
the mean estimated/predicted under-5 mortality for the two different approaches are highlighted with solid markers. Countries where
the estimated/predicted under-5 mortality using one approach is more than 33.3% greater than the mean estimated/predicted under-5
mortality for the two different approaches are labelled.
14
010
020
030
0Lo
ess−
base
d ap
proa
ch
0 100 200 300Spline−based approach
Fig. 10: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in the year 2005.5 in countries with either
no ad-hoc weight adjustments or merely a down-weighting of the vital registration data. Countries where estimated/predicted under-5
mortality using one approach is 20–33.3% greater than the mean estimated/predicted under-5 mortality for the two different approaches
are highlighted with solid markers. Countries where the estimated/predicted under-5 mortality using one approach is more than 33.3%
greater than the mean estimated/predicted under-5 mortality for the two different approaches are labelled.
Palau
Zimbabwe
010
020
030
0Lo
ess−
base
d ap
proa
ch
0 100 200 300Spline−based approach
Fig. 11: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in the year 2005.5 in countries with more
complex ad-hoc weight adjustments. Countries where estimated/predicted under-5 mortality using one approach is 20–33.3% greater than
the mean estimated/predicted under-5 mortality for the two different approaches are highlighted with solid markers. Countries where
the estimated/predicted under-5 mortality using one approach is more than 33.3% greater than the mean estimated/predicted under-5
mortality for the two different approaches are labelled.
15
050
100
150
200
250
Loes
s−ba
sed
appr
oach
0 50 100 150 200 250Spline−based approach
Fig. 12: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in the year 2010.5 in countries with either
no ad-hoc weight adjustments or merely a down-weighting of the vital registration data. Countries where estimated/predicted under-5
mortality using one approach is 20–33.3% greater than the mean estimated/predicted under-5 mortality for the two different approaches
are highlighted with solid markers. Countries where the estimated/predicted under-5 mortality using one approach is more than 33.3%
greater than the mean estimated/predicted under-5 mortality for the two different approaches are labelled.
Lesotho
Palau
ZimbabweTimor Leste
050
100
150
200
250
Loes
s−ba
sed
appr
oach
0 50 100 150 200 250Spline−based approach
Fig. 13: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in the year 2010.5 in countries with more
complex ad-hoc weight adjustments. Countries where estimated/predicted under-5 mortality using one approach is 20–33.3% greater than
the mean estimated/predicted under-5 mortality for the two different approaches are highlighted with solid markers. Countries where
the estimated/predicted under-5 mortality using one approach is more than 33.3% greater than the mean estimated/predicted under-5
mortality for the two different approaches are labelled.
16
010
020
030
040
0Lo
ess−
base
d es
timat
e/pr
edic
tion
0 100 200 300 400Spline−based estimate/prediction
Fig. 14: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in the year 2015.5 in countries with either
no ad-hoc weight adjustments or merely a down-weighting of the vital registration data. Countries where estimated/predicted under-5
mortality using one approach is 20–33.3% greater than the mean estimated/predicted under-5 mortality for the two different approaches
are highlighted with solid markers. Countries where the estimated/predicted under-5 mortality using one approach is more than 33.3%
greater than the mean estimated/predicted under-5 mortality for the two different approaches are labelled.
Jamaica
Lesotho
Palau
ZimbabweTimor Leste
010
020
030
040
0Lo
ess−
base
d es
timat
e/pr
edic
tion
0 100 200 300 400Spline−based estimate/prediction
Fig. 15: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in the year 2015.5 in countries with more
complex ad-hoc weight adjustments. Countries where estimated/predicted under-5 mortality using one approach is 20–33.3% greater than
the mean estimated/predicted under-5 mortality for the two different approaches are highlighted with solid markers. Countries where
the estimated/predicted under-5 mortality using one approach is more than 33.3% greater than the mean estimated/predicted under-5
mortality for the two different approaches are labelled.
17
produce slightly higher predictions. This is borne out by p-values in the region 0.05 to 0.1 depending on the statistical
test used.
For countries with more complex ad-hoc weight adjustments, in 2000 83% of estimates/predictions are within
10% of the mean, but the 17% not within 10% of the mean all have a lower loess-based estimate/prediction than
spline-based. As time progresses, predictions get further from the mean until by 2015 only 35% are within 10% of the
mean and 34% are more than 20% from the mean. However, by 2015 approximately as many countries have a greater
spline-based estimate/prediction as have a greater loess-based prediction.
Loess-based estimate/prediction as a % of the
mean of the loess- and spline-based estimates/predictions
YearAd-hoc weighting
<80% 80–90% 90–100% 100–110% 110–120% >120%adjustment?
2000.5None/VR only 0 (0%) 0 (0%) 12 (48%) 13 (52%) 0 (0%) 0 (0%)
More complex 2 (6%) 4 (11%) 12 (34%) 17 (49%) 0 (0%) 0 (0%)
2005.5None/VR only 0 (0%) 1 (4%) 7 (28%) 16 (64%) 1 (4%) 0 (0%)
More complex 5 (14%) 3 (9%) 9 (26%) 14 (40%) 3 (9%) 1 (3%)
2010.5None/VR only 0 (0%) 1 (4%) 7 (28%) 12 (48%) 4 (16%) 1 (4%)
More complex 5 (14%) 6 (17%) 6 (17%) 9 (26%) 6 (17%) 3 (9%)
2015.5None/VR only 1 (4%) 1 (4%) 6 (24%) 8 (32%) 8 (32%) 1 (4%)
More complex 7 (20%) 6 (17%) 3 (9%) 9 (26%) 5 (14%) 5 (14%)
Table 5: Loess-based estimate/prediction as a % of the mean of the loess- and spline-based estimates/predictions. ‘None/VR only’ is
either no ad-hoc weight adjustments or merely a down-weighting of the vital registration data. ‘More complex’ is more complex ad-hoc
weight adjustments.
Fig. 16–21 plot the spline-based estimate/prediction for each country and each year alongside the loess-based
estimate/prediction and associated uncertainty interval. Again, countries with no ad-hoc weight adjustments or
merely a down-weighting of the vital registration data (Fig. 16–18) are examined separately to countries with more
complex ad-hoc weight adjustments (Fig. 19–21). In most countries, the spline-based estimates/predictions lie within
the loess-based estimated uncertainty intervals. However, for several countries with more complex ad-hoc weighting
adjustments there are more severe discrepancies.
4.3 Conclusions
When there are complex ad-hoc weight adjustments, it is understandable that discrepancies between the spline- and
loess-based estimates/predictions occur, thus it is perhaps more informative to concentrate on the results obtained in
countries where these is little or no ad-hoc weight adjustments. In these countries the estimated/predicted under-5
mortality is often very similar for both the spline- and loess-based approaches, particularly at time points closer to
the range of years over which data are observed. However, there is some evidence of a tendency for the loess-based
approach to produce slightly higher estimates than the spline-based approach, particularly at time points further from
18
010
2030
4050
Und
er−
5 m
orta
lity
(per
100
0)
Arm
enia
Bel
ize
Col
ombi
a
Fiji
Hon
dura
s
Mex
ico
Mic
rone
sia
Mol
dova
Fig. 16: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in countries with either no ad-hoc weight
adjustments or merely a down-weighting of the vital registration data. Black points are loess-based estimates/predictions with uncertainty
intervals displayed as bars. Red points are spline-based estimates/predictions. For each country the estimates/predictions correspond,
from left to right, to the years 2000.5, 2005.5, 2010.5 and 2015.5.
050
100
150
Und
er−
5 m
orta
lity
(per
100
0)
Cam
bodi
a
Gam
bia
Geo
rgia
Mal
dive
s
Per
u
Tog
o
Tur
kmen
ista
n
Tuv
alu
Fig. 17: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in countries with either no ad-hoc weight
adjustments or merely a down-weighting of the vital registration data. Black points are loess-based estimates/predictions with uncertainty
intervals displayed as bars. Red points are spline-based estimates/predictions. For each country the estimates/predictions correspond,
from left to right, to the years 2000.5, 2005.5, 2010.5 and 2015.5.
19
010
020
030
040
0U
nder
−5
mor
talit
y (p
er 1
000)
Bur
undi
C. A
f. R
ep.
Con
go
Gui
nea
Gui
nea
Bis
sau
Rw
anda
Sen
egal
Sie
rra
Leon
e
Som
alia
Fig. 18: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in countries with either no ad-hoc weight
adjustments or merely a down-weighting of the vital registration data. Black points are loess-based estimates/predictions with uncertainty
intervals displayed as bars. Red points are spline-based estimates/predictions. For each country the estimates/predictions correspond,
from left to right, to the years 2000.5, 2005.5, 2010.5 and 2015.5.
010
2030
40U
nder
−5
mor
talit
y (p
er 1
000)
Bel
arus
Bra
zil
Chi
na
Pal
au
Rus
sian
Fed
.
Syr
ia
Tha
iland
Ukr
aine
U.A
.E.
Uru
guay
Ven
ezue
la
Fig. 19: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in countries with more complex ad-hoc weight
adjustments. Black points are loess-based estimates/predictions with uncertainty intervals displayed as bars. Red points are spline-based
estimates/predictions. For each country the estimates/predictions correspond, from left to right, to the years 2000.5, 2005.5, 2010.5 and
2015.5.
20
050
100
Und
er−
5 m
orta
lity
(per
100
0)
Egy
pt
Iraq
Jam
aica
Kaz
akhs
tan
Kyr
gyzs
tan
Mon
golia
Nep
al
Pap
ua N
. G.
S. T
. & P
.
Taj
ikis
tan
T. &
T.
Uzb
ekis
tan
Fig. 20: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in countries with more complex ad-hoc weight
adjustments. Black points are loess-based estimates/predictions with uncertainty intervals displayed as bars. Red points are spline-based
estimates/predictions. For each country the estimates/predictions correspond, from left to right, to the years 2000.5, 2005.5, 2010.5 and
2015.5.
010
020
030
040
0U
nder
−5
mor
talit
y (p
er 1
000)
Bur
kina
F.
Cha
d
Cot
e d’
Iv.
Eth
iopi
a
Gha
na
Hai
ti
Indi
a
Leso
tho
Mal
awi
Sol
omon
Is.
Tim
or L
este
Zim
babw
e
Fig. 21: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in countries with more complex ad-hoc weight
adjustments. Black points are loess-based estimates/predictions with uncertainty intervals displayed as bars. Red points are spline-based
estimates/predictions. For each country the estimates/predictions correspond, from left to right, to the years 2000.5, 2005.5, 2010.5 and
2015.5.
21
the range of years over which data are observed.
One potential explanation for this observation could be the apparent shift of births and deaths in DHS birth history
data from the most recent 5-year period to the previous period in some countries acknowledged elsewhere. This can
lead to an underestimate of mortality for the most recent 5-year period and an overestimate for the previous period,
resulting in an overestimate of the trend between these two time points. The spline-based approach, using a fixed set
of weights, is likely to be more sensitive to this feature of the data than the loess-based approach, using a variety of
smoothing parameters.
The loess- and spline-based approaches are likely to provide similar estimates/predictions if the data points lie on
a very obvious trajectory, if there are no vital registration data, and if, in the spline-based approach, the data points
are (finally, if not initially) weighted similarly to the average weighting in the loess-based approach. The loess-based
approach places more emphasis on the long-term trend than the spline-based approach. In the instances where there
is a large difference between the estimated/predicted under-5 mortality under the two approaches it is often because
a more recent deviation from the long-term trend is having a large effect on the fitted spline, but the deviation is only
acknowledged in smaller α values in the loess-based approach so the overall effect is diluted.
5 Incorporation of uncertainty
5.1 Spline-based approach
One disadvantage of the spline-based method as it currently stands is the lack of any indication of the level of
uncertainty associated with any estimate/predication of child mortality. One possible way to incorporate uncertainty
into the spline-based approach is through a similar random draw simulation method to that used in the loess-based
approach. This is detailed in Section 5.1.1. An alternative approach is to calculate ‘analytic’ uncertainty intervals
directly, as described in Section 5.1.2. In Section 5.1.3 uncertainty intervals are calculated using the random draw
simulation method and compared to those found under the loess-based approach.
5.1.1 Random draw simulation approach
For each country, a possible method for the incorporation of uncertainty into the spline-based approach proceeds as
follows:
1. Fit the infant or under-5 mortality spline as per steps 1–5 in Section 2.1.
2. Simulate 10,000 random draws from the multivariate normal distribution defined by the estimated coefficients
and their variance-covariance matrix.
3. For each required time point:
• For each of the 10,000 random draws, calculate the estimated/predicted mortality at the required time
point;
• Pool the 10,000 estimates/predictions;
• The uncertainty interval corresponds to the 2.5th and 97.5th centiles of these pooled estimates/predictions.
22
5.1.2 Analytic approach
As an alternative to the random draw simulation approach detailed in Section 5.1.1, ‘analytic’ uncertainty intervals
can be calculated directly.
Consider the linear spline model (2.1),
log(y) = β0 + β1x +K∑
k=1
bk(x− κk)+ + ε.
From this expression it is possible to calculate for any fitted point, log(y), the corresponding variance, var(log(y)). A
95% confidence interval for the fitted point can then be constructed as
log(y)± 1.96√
var(log(y)), (5.1)
or using the t-distribution analogously if the sample size is small.
For example, if there are K = 2 knots then (2.1) becomes
log(y) = β0 + β1x + b1(x− κ1)+ + b2(x− κ2)+ + ε.
Fitted values are then
log(y) = β0 + β1x + b1(x− κ1)+ + b2(x− κ2)+ (5.2)
with variance
var(log(y)) = var(β0) + x2var(β1) + (x− κ1)2var(b1) + (x− κ2)2var(b2) + 2xcov(β0, β1)
+ 2(x− κ1)cov(β0, b1) + 2x(x− κ1)cov(β1, b1) + 2(x− κ2)cov(β0, b2)
+ 2x(x− κ2)cov(β1, b2) + 2(x− κ1)(x− κ2)cov(b1, b2),
(5.3)
and a 95% confidence interval can be constructed by substituting (5.2) and (5.3) into (5.1).
However, as the number of knots K increases, the length of the expression for var(log(y)) increases rapidly.
As the approach detailed in Section 5.1.1 is simulating random draws from the multivariate normal distribution
defined by the same estimated coefficients, variances and covariances used in (5.2) and (5.3), with sufficient random
draws the estimated uncertainty intervals under the two approaches should be identical.
5.1.3 Comparison with uncertainty intervals obtained using the loess-based approach
Method
Estimated/predicted under-5 mortality uncertainty intervals under the spline- and loess-based approaches in the years
2000, 2005, 2010 and 2015 are compared in the same 60 datasets detailed in Section 4.1.
The spline-based approach proceeds as described in Section 2.1, with uncertainty intervals created using the random
draw simulation approach detailed in Section 5.1.1. The final weights (those used in step 5 of Section 2.1) are provided
23
1980 1990 2000 2010
020
4060
8010
0
Year
Und
er−
5 m
orta
lity
(per
100
0)
1980 1990 2000 2010
020
4060
8010
0
Year
Und
er−
5 m
orta
lity
(per
100
0)Fig. 22: Fitted spline curve (left-hand plot; dashed vertical line indicates knot location, dashed curves represent uncertainty intervals)
and loess curve (right-hand plot; blue points represent vital registration data, black points represent non-vital registration data, dashed
curves represent uncertainty intervals) for Armenia.
with the dataset and used here. For some countries (Lesotho and Zimbabwe), recently published spline-based estimates
are set to be constant, but this is ignored here.
The loess-based approach ignores the weights used for the spline-based approach. Instead, data which come from
a vital registration system are identified using the documentation provided with the data. The loess-based approach
then proceeds as described in Section 2.2.
For illustration, fitted spline and loess curves with uncertainty intervals are shown in Fig. 22 and Fig. 23 for
Armenia and Papua New Guinea, respectively. The estimated uncertainty intervals under the two approaches are seen
to be reasonably similar for Armenia, but for Papua New Guinea they differ greatly, particularly since the most recent
knot in the spline-based approach.
Results
Estimated under-5 mortality uncertainty intervals, as well as the corresponding estimates/predictions, in the years
2000, 2005, 2010 and 2015 under the two different methods are presented in Tables 6–8. Although it is clear that there
are often discrepancies between the spline- and loess-based estimated uncertainty intervals, these differences are more
easily interpreted when the results are presented graphically.
As in Section 4.2, countries with no ad-hoc weight adjustments or merely a down-weighting of data coming from
vital registration systems are examined separately to countries with more complex ad-hoc weight adjustments.
Fig. 24–29 plot the spline-based estimate/prediction and associated uncertainty interval for each country and each
year alongside the loess-based estimate/prediction and associated uncertainty interval. Countries with no ad-hoc
weight adjustments or merely a down-weighting of the vital registration data are shown in Fig. 24–26 and countries
with more complex ad-hoc weight adjustments in Fig. 27–29. There is much variability in the relative sizes of the
estimated uncertainty intervals, with the spline-based uncertainty interval being much wider in some cases and much
24
Countr
y2000.5
2005.5
2010.5
2015.5
Spline
Loes
sSpline
Loes
sSpline
Loes
sSpline
Loes
s
Arm
enia
36.3
(33.1
,39.9
)35.3
(32.2
,39.0
)25.5
(20.2
,32.3
)27.1
(21.8
,30.8
)17.9
(11.9
,27.0
)21.3
(14.3
,25.3
)12.5
(7.0
,22.7
)16.7
(9.1
,21.2
)
Bel
aru
s17.3
(15.8
,18.8
)13.1
(11.1
,17.8
)13.9
(11.7
,16.5
)10.7
(8.9
,13.0
)11.2
(8.5
,14.7
)8.8
(6.0
,11.5
)9.0
(6.1
,13.1
)7.4
(3.8
,10.4
)
Bel
ize1
19.3
(15.4
,24.0
)21.2
(19.8
,23.3
)12.8
(8.6
,18.9
)15.3
(13.7
,18.5
)8.5
(4.8
,14.9
)11.1
(9.5
,14.8
)5.6
(2.7
,11.8
)8.0
(6.6
,11.8
)
Bra
zil
29.6
(24.5
,35.7
)34.2
(30.6
,38.5
)21.2
(16.0
,28.3
)26.7
(22.9
,31.1
)15.3
(10.5
,22.3
)20.9
(17.1
,25.1
)11.0
(6.8
,17.7
)16.3
(12.6
,20.3
)
Burk
ina
F.
193.9
(185.1
,202.9
)196.9
(184.9
,212.7
)202.3
(177.3
,230.7
)191.1
(173.6
,228.0
)211.1
(167.7
,266.0
)184.7
(162.8
,248.0
)220.2
(158.3
,307.4
)178.3
(152.3
,271.7
)
Buru
ndi
180.8
(167.9
,194.2
)183.8
(171.4
,196.0
)181.0
(151.9
,215.9
)181.5
(165.5
,204.1
)181.1
(135.2
,244.0
)178.9
(158.8
,214.2
)181.2
(119.9
,275.7
)175.9
(152.5
,225.3
)
Cam
bodia
104.2
(93.5
,115.4
)104.8
(96.3
,114.1
)85.4
(67.2
,108.2
)91.0
(63.3
,107.3
)70.0
(47.5
,102.6
)80.9
(38.3
,102.5
)57.4
(33.4
,97.8
)72.1
(22.9
,98.0
)
C.A
f.R
ep.
186.1
(172.5
,201.1
)186.3
(165.2
,201.7
)176.6
(146.7
,212.4
)187.6
(158.1
,215.6
)167.6
(123.7
,226.4
)188.4
(150.7
,232.8
)159.0
(104.3
,242.7
)189.3
(143.9
,251.9
)
Chad2
205.3
(194.3
,216.8
)203.6
(193.0
,214.5
)208.7
(185.3
,235.0
)203.5
(185.3
,223.7
)212.2
(176.1
,255.6
)203.7
(175.4
,236.1
)215.7
(167.2
,278.5
)203.9
(162.6
,248.0
)
Chin
a36.6
(34.0
,39.4
)31.1
(26.4
,36.7
)25.4
(21.9
,29.5
)27.5
(22.1
,35.0
)17.6
(13.1
,23.7
)24.3
(17.2
,35.5
)12.3
(7.7
,19.1
)21.5
(12.9
,36.2
)
Colo
mbia
25.9
(22.4
,30.0
)24.5
(21.4
,28.8
)21.4
(16.3
,28.1
)19.9
(16.4
,25.7
)17.7
(11.7
,26.7
)16.0
(12.7
,23.1
)14.6
(8.3
,25.4
)12.9
(9.8
,20.8
)
Congo
117.0
(108.1
,126.7
)114.0
(100.7
,128.2
)124.8
(110.8
,140.8
)115.8
(95.3
,147.2
)133.2
(112.8
,157.5
)116.1
(89.2
,176.9
)142.2
(114.7
,176.9
)116.9
(83.0
,213.1
)
Cote
d’Iv.2
137.9
(126.1
,151.0
)138.8
(125.5
,154.6
)131.6
(114.8
,150.9
)128.8
(102.9
,151.2
)125.5
(104.4
,151.1
)120.3
(80.3
,150.5
)119.7
(94.8
,151.2
)112.6
(61.7
,149.8
)
Egypt2
50.5
(46.1
,55.4
)54.4
(48.8
,63.2
)37.7
(28.9
,49.1
)40.3
(32.7
,50.5
)28.1
(17.4
,45.2
)30.2
(21.7
,40.6
)21.0
(10.4
,41.9
)22.6
(14.4
,32.7
)
Eth
iopia
150.6
(140.9
,161.1
)159.5
(146.1
,172.3
)127.1
(113.3
,142.6
)146.2
(117.1
,165.2
)107.3
(90.8
,126.9
)134.9
(91.5
,158.4
)90.6
(72.7
,112.9
)124.4
(72.2
,152.2
)
Fiji1
13.1
(11.4
,15.1
)15.0
(12.9
,20.4
)9.8
(8.1
,11.9
)11.8
(9.6
,19.5
)7.4
(5.8
,9.4
)9.3
(7.2
,18.2
)5.6
(4.2
,7.4
)7.3
(5.3
,17.4
)
Gam
bia
131.9
(122.9
,141.6
)129.6
(119.5
,138.3
)116.3
(98.6
,137.1
)115.6
(103.5
,128.9
)102.5
(78.4
,134.2
)102.6
(89.0
,121.5
)90.3
(62.4
,131.8
)91.2
(76.8
,115.2
)
Geo
rgia
36.6
(33.2
,40.2
)37.6
(30.0
,46.7
)32.7
(27.7
,38.7
)36.4
(28.3
,51.2
)29.3
(22.8
,37.4
)34.9
(24.9
,68.5
)26.2
(18.8
,36.4
)33.3
(21.6
,93.1
)
Ghana
112.7
(108.2
,117.5
)114.2
(108.2
,120.6
)118.7
(105.2
,134.0
)109.2
(101.0
,132.4
)124.9
(99.7
,156.7
)103.8
(94.1
,148.8
)131.5
(94.1
,183.5
)98.7
(87.4
,168.1
)
Guin
ea184.3
(175.6
,193.7
)185.5
(178.9
,193.1
)164.8
(145.4
,186.1
)165.3
(144.7
,175.9
)147.3
(119.5
,180.2
)148.1
(110.2
,162.8
)131.7
(98.0
,174.7
)132.6
(83.2
,151.2
)
Table
6:
Est
imate
d/pre
dic
ted
under
-5m
ort
ality
(per
1000).
Spline-
base
dm
ethod
pro
vid
espoin
tes
tim
ate
sonly
.Loes
s-base
dm
ethod
pro
vid
espoin
tes
tim
ate
sand
95%
unce
rtain
tyin
terv
als
.
1Spline-
base
des
tim
ate
sdo
not
corr
espond
topublish
edvalu
esdue
topublish
edvalu
esbei
ng
der
ived
from
the
fitt
edin
fant
mort
ality
spline.
2Spline-
base
des
tim
ate
sdiff
ersl
ightl
yfr
om
publish
ed
valu
es.
3R
ecen
tpublish
edsp
line-
base
des
tim
ate
sare
const
ant
—th
isis
ignore
dher
e.
25
Countr
y2000.5
2005.5
2010.5
2015.5
Spline
Loes
sSpline
Loes
sSpline
Loes
sSpline
Loes
s
Guin
eaB
issa
u217.5
(211.1
,224.1
)216.0
(211.4
,220.7
)202.9
(189.4
,217.5
)204.2
(195.9
,213.6
)189.2
(169.2
,211.9
)193.1
(179.4
,209.0
)176.4
(151.0
,206.7
)182.5
(164.1
,204.8
)
Hait
i109.2
(101.6
,117.8
)116.7
(108.0
,126.2
)84.2
(72.8
,98.0
)101.6
(82.3
,114.3
)64.9
(51.5
,82.4
)88.9
(61.8
,103.4
)50.1
(36.4
,69.4
)77.8
(46.5
,93.6
)
Hondura
s39.6
(36.6
,42.8
)40.2
(38.2
,42.5
)28.7
(22.4
,36.7
)32.3
(30.1
,35.4
)20.8
(13.5
,32.3
)25.9
(23.6
,29.9
)15.1
(8.0
,28.3
)20.7
(18.5
,25.2
)
India
188.2
(77.2
,100.8
)94.8
(86.9
,114.3
)77.2
(62.9
,94.6
)86.0
(76.4
,130.2
)67.5
(51.4
,88.9
)77.9
(67.2
,146.8
)59.1
(41.8
,83.4
)70.4
(59.1
,162.6
)
Iraq
47.5
(43.0
,52.5
)46.8
(42.3
,51.9
)46.6
(37.1
,58.7
)41.2
(35.4
,47.9
)45.7
(27.7
,74.8
)36.5
(27.7
,45.5
)44.9
(20.7
,96.1
)32.4
(21.4
,43.8
)
Jam
aic
a31.9
(27.8
,36.8
)21.9
(17.2
,30.6
)31.3
(24.1
,40.7
)19.5
(14.1
,35.4
)30.6
(20.5
,45.6
)17.2
(11.3
,42.2
)29.9
(17.5
,51.1
)15.1
(9.0
,51.1
)
Kaza
khst
an
42.9
(37.2
,49.4
)39.1
(34.4
,45.2
)31.0
(22.7
,42.6
)31.0
(22.7
,37.8
)22.3
(13.4
,37.8
)25.3
(13.3
,33.2
)16.1
(7.9
,33.8
)20.7
(7.8
,29.0
)
Kyrg
yzs
tan
51.4
(45.9
,57.4
)51.6
(45.5
,57.7
)42.5
(36.5
,49.4
)42.5
(35.5
,49.5
)35.2
(29.1
,42.6
)35.4
(27.3
,43.3
)29.1
(23.0
,36.8
)29.6
(20.7
,37.9
)
Les
oth
o3
108.4
(99.6
,117.6
)93.1
(85.3
,111.0
)131.8
(108.6
,158.8
)86.3
(76.2
,127.2
)160.2
(117.5
,216.1
)79.8
(68.1
,147.5
)194.7
(127.2
,294.2
)73.9
(61.0
,171.1
)
Mala
wi
155.3
(146.3
,164.7
)168.9
(155.0
,179.4
)125.3
(109.2
,143.6
)150.0
(121.8
,162.5
)101.1
(81.1
,126.0
)133.5
(93.9
,147.5
)81.6
(60.1
,110.7
)118.8
(73.0
,134.1
)
Mald
ives
54.1
(49.6
,58.9
)53.6
(48.9
,58.8
)33.3
(26.9
,40.9
)35.6
(27.0
,42.3
)20.4
(14.1
,29.2
)24.3
(14.3
,30.8
)12.6
(7.4
,21.0
)16.5
(7.6
,22.5
)
Mex
ico1
35.8
(28.9
,44.2
)35.3
(31.8
,40.4
)29.7
(21.5
,41.1
)29.4
(25.7
,35.9
)24.7
(15.9
,38.1
)24.4
(20.8
,31.9
)20.5
(11.8
,35.4
)20.3
(16.7
,28.2
)
Mic
rones
ia46.5
(37.9
,57.4
)47.1
(42.6
,52.3
)41.8
(30.0
,58.7
)42.9
(36.5
,50.5
)37.5
(23.8
,60.0
)39.1
(31.0
,49.0
)33.7
(18.8
,61.5
)35.6
(26.6
,47.5
)
Mold
ova
24.3
(20.7
,28.5
)27.5
(25.3
,29.9
)19.8
(15.8
,24.8
)22.6
(17.7
,25.4
)16.1
(11.9
,21.7
)18.9
(11.8
,22.0
)13.1
(9.0
,19.0
)15.9
(7.8
,19.1
)
Mongolia
61.5
(53.6
,70.5
)57.1
(51.5
,64.8
)45.2
(32.5
,62.4
)39.0
(29.0
,50.2
)33.3
(19.5
,56.5
)27.1
(15.0
,39.7
)24.5
(11.6
,51.0
)18.8
(7.8
,31.1
)
Nep
al
86.1
(81.2
,91.4
)96.5
(86.6
,110.8
)63.1
(56.1
,71.0
)81.8
(65.8
,99.5
)46.2
(38.4
,55.6
)69.8
(49.7
,89.4
)33.9
(26.2
,43.7
)59.5
(37.5
,80.7
)
Pala
u1
9.9
(4.2
,23.2
)3.1
(1.4
,6.6
)6.7
(1.7
,26.2
)1.5
(0.4
,7.0
)4.5
(0.7
,30.0
)0.7
(0.1
,9.4
)3.0
(0.3
,33.9
)0.3
(0.0
,14.1
)
Papua
N.G
.279.6
(58.6
,107.6
)83.3
(72.5
,95.5
)72.9
(43.4
,121.4
)77.5
(63.5
,95.8
)66.8
(32.2
,137.5
)72.2
(55.6
,94.9
)61.2
(23.9
,155.6
)67.2
(48.0
,94.7
)
Per
u41.3
(37.9
,44.8
)48.0
(42.6
,53.0
)27.3
(22.9
,32.4
)37.7
(29.7
,43.3
)18.0
(13.7
,23.5
)29.6
(20.5
,35.4
)11.9
(8.2
,17.1
)23.2
(14.2
,29.0
)
Russ
ian
Fed
.23.9
(23.1
,24.7
)22.0
(17.0
,27.1
)16.9
(16.0
,17.9
)19.4
(13.5
,24.3
)11.5
(9.9
,13.3
)17.3
(11.5
,22.0
)7.8
(6.1
,10.0
)15.5
(9.7
,19.8
)
Table
7:
Est
imate
d/pre
dic
ted
under
-5m
ort
ality
(per
1000)
conti
nued
.Spline-
base
dm
ethod
pro
vid
espoin
tes
tim
ate
sonly
.Loes
s-base
dm
ethod
pro
vid
espoin
tes
tim
ate
sand
95%
unce
rtain
ty
inte
rvals
.1
Spline-
base
des
tim
ate
sdo
not
corr
espond
topublish
edvalu
esdue
topublish
edvalu
esbei
ng
der
ived
from
the
fitt
edin
fant
mort
ality
spline.
2Spline-
base
des
tim
ate
sdiff
ersl
ightl
yfr
om
publish
edvalu
es.
3R
ecen
tpublish
edsp
line-
base
des
tim
ate
sare
const
ant
—th
isis
ignore
dher
e.
26
Countr
y2000.5
2005.5
2010.5
2015.5
Spline
Loes
sSpline
Loes
sSpline
Loes
sSpline
Loes
s
Rw
anda
183.4
(171.1
,196.6
)186.3
(175.4
,198.9
)162.7
(139.9
,188.8
)182.5
(168.1
,203.6
)144.4
(113.3
,184.1
)178.9
(158.0
,210.2
)128.1
(91.4
,179.4
)175.3
(147.6
,216.8
)
S.T
.&
P.
97.2
(81.9
,115.2
)71.1
(54.1
,88.7
)95.9
(77.3
,118.7
)62.3
(32.5
,86.4
)94.6
(72.8
,122.7
)55.5
(19.5
,84.7
)93.3
(68.8
,126.5
)49.5
(11.5
,83.3
)
Sen
egal
132.6
(125.8
,139.5
)132.0
(122.0
,143.1
)118.7
(106.5
,131.9
)119.5
(106.9
,137.6
)106.3
(89.5
,125.7
)107.8
(93.6
,133.2
)95.2
(75.2
,119.8
)97.1
(81.7
,129.9
)
Sie
rra
Leo
ne
276.5
(262.1
,291.1
)273.9
(262.6
,290.5
)271.1
(241.7
,304.0
)264.1
(249.5
,304.3
)265.9
(221.5
,318.7
)254.1
(236.7
,325.0
)260.7
(202.5
,334.7
)244.4
(224.6
,348.7
)
Solo
mon
Is.1
,479.2
56.5
(42.4
,75.2
)68.8
57.8
(38.8
,91.8
)59.9
58.7
(34.9
,117.2
)52.1
59.3
(31.5
,151.5
)
Som
alia
164.8
(146.3
,184.9
)158.6
(143.6
,175.2
)148.5
(123.3
,177.1
)133.8
(112.5
,161.8
)133.8
(103.4
,171.1
)114.7
(82.1
,154.2
)120.6
(86.2
,165.9
)98.5
(59.2
,145.0
)
Syri
a19.9
(15.5
,25.4
)21.7
(16.3
,25.4
)14.5
(8.9
,23.9
)16.1
(10.4
,19.7
)10.7
(5.0
,22.6
)11.9
(6.5
,15.1
)7.8
(2.9
,21.5
)8.8
(4.2
,11.7
)
Tajikis
tan
93.3
(88.0
,98.9
)80.6
(71.2
,90.9
)71.4
(61.4
,83.3
)67.4
(54.0
,81.6
)54.6
(42.1
,71.4
)57.1
(36.1
,76.0
)41.8
(28.8
,61.2
)48.5
(23.8
,69.8
)
Thailand
12.9
(11.0
,15.0
)14.4
(12.4
,16.4
)8.4
(6.0
,11.9
)10.3
(7.9
,12.2
)5.5
(3.2
,9.6
)7.4
(4.9
,9.1
)3.6
(1.7
,7.8
)5.3
(3.0
,6.8
)
Tim
or
Les
te106.8
(93.8
,121.8
)113.7
(98.6
,131.8
)61.3
(36.2
,105.2
)92.5
(63.1
,122.9
)35.2
(13.3
,94.8
)75.4
(37.9
,115.8
)20.2
(4.8
,86.9
)62.5
(22.8
,110.6
)
Togo
124.0
(118.5
,129.7
)129.0
(123.5
,135.6
)110.6
(100.6
,121.3
)119.6
(112.5
,131.5
)98.5
(84.9
,113.9
)110.5
(101.8
,129.6
)87.8
(71.7
,107.1
)102.3
(91.8
,127.0
)
T.&
T.
34.2
(28.8
,40.6
)30.5
(25.1
,38.4
)37.0
(24.2
,56.5
)28.5
(22.1
,42.9
)39.9
(19.8
,80.7
)26.6
(19.2
,49.4
)43.1
(16.1
,115.8
)24.8
(16.4
,55.8
)
Turk
men
ista
n70.6
(66.1
,75.4
)64.2
(55.8
,74.0
)54.1
(45.1
,64.4
)42.2
(33.0
,52.7
)41.4
(30.3
,55.7
)28.8
(16.1
,40.2
)31.7
(20.4
,48.5
)19.8
(7.7
,30.5
)
Tuvalu
135.7
(17.7
,70.6
)34.9
(17.2
,69.0
)30.3
(12.2
,72.5
)29.2
(11.2
,73.7
)25.7
(8.4
,75.0
)24.5
(7.5
,78.3
)21.8
(5.8
,77.9
)20.6
(4.8
,82.7
)
Ukra
ine
22.8
(20.4
,25.3
)22.3
(20.4
,25.3
)23.6
(17.3
,31.7
)20.6
(18.2
,23.3
)24.4
(14.0
,41.8
)18.8
(14.8
,21.7
)25.2
(11.2
,55.4
)17.2
(11.7
,20.7
)
U.A
.E
.10.3
(9.0
,11.7
)9.5
(7.3
,11.8
)8.5
(7.2
,10.1
)6.9
(4.9
,9.1
)7.1
(5.6
,8.9
)4.9
(3.2
,7.2
)5.8
(4.3
,7.9
)3.5
(2.1
,5.8
)
Uru
guay1
16.0
(14.6
,17.5
)19.5
(17.2
,25.1
)16.2
(14.0
,18.9
)16.4
(14.2
,22.4
)16.5
(11.6
,23.4
)13.8
(11.6
,20.0
)16.8
(9.6
,29.3
)11.5
(9.5
,17.7
)
Uzb
ekis
tan
62.3
(57.7
,67.3
)55.9
(49.2
,66.0
)46.0
(36.7
,58.2
)46.1
(40.9
,51.3
)34.0
(22.4
,52.5
)36.9
(31.0
,43.6
)25.1
(13.7
,47.5
)30.0
(21.8
,38.0
)
Ven
ezuel
a24.5
(22.3
,26.8
)25.6
(23.6
,29.3
)21.3
(18.5
,24.2
)22.1
(19.9
,28.1
)18.4
(15.3
,22.0
)19.1
(16.8
,27.2
)16.0
(12.7
,19.9
)16.5
(14.1
,26.0
)
Zim
babw
e3135.3
(105.4
,173.4
)73.0
(64.9
,85.0
)185.5
(123.2
,279.4
)68.5
(58.6
,88.2
)254.2
(143.6
,448.8
)63.9
(52.3
,92.6
)348.5
(167.0
,721.6
)59.5
(46.8
,96.0
)
Table
8:
Est
imate
d/pre
dic
ted
under
-5m
ort
ality
(per
1000)
conti
nued
.Spline-
base
dm
ethod
pro
vid
espoin
tes
tim
ate
sonly
.Loes
s-base
dm
ethod
pro
vid
espoin
tes
tim
ate
sand
95%
unce
rtain
ty
inte
rvals
.1
Spline-
base
des
tim
ate
sdo
not
corr
espond
topublish
edvalu
esdue
topublish
edvalu
esbei
ng
der
ived
from
the
fitt
edin
fant
mort
ality
spline.
2Spline-
base
des
tim
ate
sdiff
ersl
ightl
yfr
om
publish
edvalu
es.
3R
ecen
tpublish
edsp
line-
base
des
tim
ate
sare
const
ant
—th
isis
ignore
dher
e.4
Insu
ffici
ent
data
poin
tsw
ith
non-z
ero
wei
ghts
for
calc
ula
tion
ofsp
line-
base
dunce
rtain
tyin
terv
al.
27
1960 1970 1980 1990 2000 2010
050
100
150
200
250
Year
Und
er−
5 m
orta
lity
(per
100
0)
1960 1970 1980 1990 2000 2010
050
100
150
200
250
Year
Und
er−
5 m
orta
lity
(per
100
0)Fig. 23: Fitted spline curve (left-hand plot; dashed vertical lines indicate knot locations, dashed curves represent uncertainty intervals)
and loess curve (right-hand plot; all data are non-vital registration data, dashed curves represent uncertainty intervals) for Papua New
Guinea.
narrower in others. However, for every year and every country (except Zimbabwe) the spline- and loess-based uncer-
tainty intervals overlap to some extent.
Fig. 30–37 show the range of the loess-based estimated under-5 mortality uncertainty interval plotted against the
range of the spline-based estimated uncertainty interval for each year separately. Countries where the range of the
estimated uncertainty interval using one approach is 20–33.3% wider than the mean of the ranges of the estimated
uncertainty intervals for the two different approaches are highlighted with solid markers. This corresponds to the
wider estimated uncertainty interval being 1.5–2 times wider than the narrower. Countries where the range of the
estimated uncertainty interval using one approach is more than 33.3% wider than the mean of the ranges of the
estimated uncertainty intervals for the two different approaches are labelled. This corresponds to the wider estimated
uncertainty interval being more than twice the width of the narrower.
It can be seen that in countries with either no ad-hoc weight adjustments or merely a down-weighting of the vital
registration data (Fig. 30, Fig. 32, Fig. 34 and Fig. 36) the ranges of the estimated under-5 mortality uncertainty
intervals under the two different approaches are perhaps generally more similar than in countries with more complex
ad-hoc weight adjustments (Fig. 31, Fig. 33, Fig. 35 and Fig. 37), though this is not so marked as when examining the
estimates/predictions in Section 4. In both countries with no ad-hoc weight adjustments or merely a down-weighting
of the vital registration data and countries with more complex ad-hoc weight adjustments there appears to be a
reasonably similar proportion of countries where the spline-based method provides the wider range of the estimated
uncertainty interval and where the loess-based method provides the wider range. Also, within each category of ad-hoc
weight adjustment, the ranges of the estimated uncertainty intervals under the two different approaches tend to get
somewhat less similar as time progresses.
Table 9 summarises the observations from Fig. 30–37 by tabulating, for each category of ad-hoc weight adjustment
28
010
2030
4050
Und
er−
5 m
orta
lity
(per
100
0)
Arm
enia
Bel
ize
Col
ombi
a
Fiji
Hon
dura
s
Mex
ico
Mol
dova
Per
u
Fig. 24: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in countries with either no ad-hoc weight
adjustments or merely a down-weighting of the vital registration data. Black points are loess-based estimates/predictions with uncertainty
intervals displayed as bars. Red points are spline-based estimates/predictions with uncertainty intervals displayed as bars. For each country
the estimates/predictions correspond, from left to right, to the years 2000.5, 2005.5, 2010.5 and 2015.5.
050
100
150
Und
er−
5 m
orta
lity
(per
100
0)
Cam
bodi
a
Gam
bia
Geo
rgia
Mal
dive
s
Mic
rone
sia
Tim
or L
este
Tur
kmen
ista
n
Tuv
alu
Fig. 25: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in countries with either no ad-hoc weight
adjustments or merely a down-weighting of the vital registration data. Black points are loess-based estimates/predictions with uncertainty
intervals displayed as bars. Red points are spline-based estimates/predictions with uncertainty intervals displayed as bars. For each country
the estimates/predictions correspond, from left to right, to the years 2000.5, 2005.5, 2010.5 and 2015.5.
29
010
020
030
040
0U
nder
−5
mor
talit
y (p
er 1
000)
Bur
undi
C. A
f. R
ep.
Con
go
Gui
nea
Gui
nea
Bis
sau
Rw
anda
Sen
egal
Sie
rra
Leon
e
Som
alia
Fig. 26: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in countries with either no ad-hoc weight
adjustments or merely a down-weighting of the vital registration data. Black points are loess-based estimates/predictions with uncertainty
intervals displayed as bars. Red points are spline-based estimates/predictions with uncertainty intervals displayed as bars. For each country
the estimates/predictions correspond, from left to right, to the years 2000.5, 2005.5, 2010.5 and 2015.5.
010
2030
4050
Und
er−
5 m
orta
lity
(per
100
0)
Bel
arus
Bra
zil
Chi
na
Kaz
akhs
tan
Pal
au
Rus
sian
Fed
.
Syr
ia
Tha
iland
U. A
. E.
Uru
guay
Ven
ezue
la
Fig. 27: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in countries with more complex ad-hoc weight
adjustments. Black points are loess-based estimates/predictions with uncertainty intervals displayed as bars. Red points are spline-based
estimates/predictions with uncertainty intervals displayed as bars. For each country the estimates/predictions correspond, from left to
right, to the years 2000.5, 2005.5, 2010.5 and 2015.5.
30
050
100
150
Und
er−
5 m
orta
lity
(per
100
0)
Egy
pt
Hai
ti
Iraq
Jam
aica
Kyr
gyzs
tan
Mon
golia
Nep
al
S. T
. & P
.
T. &
T.
Taj
ikis
tan
Ukr
aine
Uzb
ekis
tan
Fig. 28: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in countries with more complex ad-hoc weight
adjustments. Black points are loess-based estimates/predictions with uncertainty intervals displayed as bars. Red points are spline-based
estimates/predictions with uncertainty intervals displayed as bars. For each country the estimates/predictions correspond, from left to
right, to the years 2000.5, 2005.5, 2010.5 and 2015.5.
010
020
030
040
0U
nder
−5
mor
talit
y (p
er 1
000)
Bur
kina
F.
Cha
d
Cot
e d’
Iv.
Eth
iopi
a
Gha
na
Indi
a
Leso
tho
Mal
awi
Pap
ua N
. G.
Sol
omon
Is.
Tog
o
Zim
babw
e
Fig. 29: Comparison of loess- and spline-based estimates/predictions of under-5 mortality in countries with more complex ad-hoc weight
adjustments. Black points are loess-based estimates/predictions with uncertainty intervals displayed as bars. Red points are spline-based
estimates/predictions with uncertainty intervals displayed as bars. For each country the estimates/predictions correspond, from left to
right, to the years 2000.5, 2005.5, 2010.5 and 2015.5.
31
Belize
MicronesiaFiji
Georgia
010
2030
4050
Loes
s−ba
sed
appr
oach
0 10 20 30 40 50Spline−based approach
Fig. 30: Comparison of loess- and spline-based uncertainty interval ranges for estimates of under-5 mortality in the year 2000.5 in countries
with either no ad-hoc weight adjustments or merely a down-weighting of the vital registration data. Countries where the range of the
estimated under-5 mortality uncertainty interval using one approach is 20–33.3% wider than the mean of the ranges of the estimated
under-5 mortality uncertainty intervals for the two different approaches are highlighted with solid markers. Countries where the range of
the estimated under-5 mortality uncertainty interval using one approach is more than 33.3% wider than the mean of the ranges of the
estimated under-5 mortality uncertainty intervals for the two different approaches are labelled.
Palau
Papua N. G.
Belarus
Nepal
Russian Fed.Uruguay
010
2030
4050
Loes
s−ba
sed
appr
oach
0 10 20 30 40 50Spline−based approach
Fig. 31: Comparison of loess- and spline-based uncertainty interval ranges for estimates of under-5 mortality in the year 2000.5 in countries
with more complex ad-hoc weight adjustments. Countries where the range of the estimated under-5 mortality uncertainty interval using
one approach is 20–33.3% wider than the mean of the ranges of the estimated under-5 mortality uncertainty intervals for the two different
approaches are highlighted with solid markers. Countries where the range of the estimated under-5 mortality uncertainty interval using
one approach is more than 33.3% wider than the mean of the ranges of the estimated under-5 mortality uncertainty intervals for the two
different approaches are labelled. Zimbabwe is excluded to aid clarity.
32
BelizeHonduras
Micronesia
Fiji
Georgia
020
4060
80Lo
ess−
base
d ap
proa
ch
0 20 40 60 80Spline−based approach
Fig. 32: Comparison of loess- and spline-based uncertainty interval ranges for estimates of under-5 mortality in the year 2005.5 in countries
with either no ad-hoc weight adjustments or merely a down-weighting of the vital registration data. Countries where the range of the
estimated under-5 mortality uncertainty interval using one approach is 20–33.3% wider than the mean of the ranges of the estimated
under-5 mortality uncertainty intervals for the two different approaches are highlighted with solid markers. Countries where the range of
the estimated under-5 mortality uncertainty interval using one approach is more than 33.3% wider than the mean of the ranges of the
estimated under-5 mortality uncertainty intervals for the two different approaches are labelled.
Palau
Papua N. G.
Ukraine
Uzbekistan
Nepal
Russian Fed.
020
4060
80Lo
ess−
base
d ap
proa
ch
0 20 40 60 80Spline−based approach
Fig. 33: Comparison of loess- and spline-based uncertainty interval ranges for estimates of under-5 mortality in the year 2005.5 in countries
with more complex ad-hoc weight adjustments. Countries where the range of the estimated under-5 mortality uncertainty interval using
one approach is 20–33.3% wider than the mean of the ranges of the estimated under-5 mortality uncertainty intervals for the two different
approaches are highlighted with solid markers. Countries where the range of the estimated under-5 mortality uncertainty interval using
one approach is more than 33.3% wider than the mean of the ranges of the estimated under-5 mortality uncertainty intervals for the two
different approaches are labelled. Zimbabwe is excluded to aid clarity.
33
Burundi
HondurasMexico
Micronesia
Fiji
Georgia
020
4060
8010
0Lo
ess−
base
d ap
proa
ch
0 20 40 60 80 100Spline−based approach
Fig. 34: Comparison of loess- and spline-based uncertainty interval ranges for estimates of under-5 mortality in the year 2010.5 in countries
with either no ad-hoc weight adjustments or merely a down-weighting of the vital registration data. Countries where the range of the
estimated under-5 mortality uncertainty interval using one approach is 20–33.3% wider than the mean of the ranges of the estimated
under-5 mortality uncertainty intervals for the two different approaches are highlighted with solid markers. Countries where the range of
the estimated under-5 mortality uncertainty interval using one approach is more than 33.3% wider than the mean of the ranges of the
estimated under-5 mortality uncertainty intervals for the two different approaches are labelled.
Iraq
Palau
Papua N. G.
Syria
T. & T.
Ukraine
Uzbekistan
India
Nepal
Russian Fed.
020
4060
8010
0Lo
ess−
base
d ap
proa
ch
0 20 40 60 80 100Spline−based approach
Fig. 35: Comparison of loess- and spline-based uncertainty interval ranges for estimates of under-5 mortality in the year 2010.5 in countries
with more complex ad-hoc weight adjustments. Countries where the range of the estimated under-5 mortality uncertainty interval using
one approach is 20–33.3% wider than the mean of the ranges of the estimated under-5 mortality uncertainty intervals for the two different
approaches are highlighted with solid markers. Countries where the range of the estimated under-5 mortality uncertainty interval using
one approach is more than 33.3% wider than the mean of the ranges of the estimated under-5 mortality uncertainty intervals for the two
different approaches are labelled. Zimbabwe is excluded to aid clarity.
34
Burundi
HondurasMexico
Micronesia
Congo
Fiji
Georgia
050
100
150
200
Loes
s−ba
sed
appr
oach
0 50 100 150 200Spline−based approach
Fig. 36: Comparison of loess- and spline-based uncertainty interval ranges for estimates of under-5 mortality in the year 2015.5 in countries
with either no ad-hoc weight adjustments or merely a down-weighting of the vital registration data. Countries where the range of the
estimated under-5 mortality uncertainty interval using one approach is 20–33.3% wider than the mean of the ranges of the estimated
under-5 mortality uncertainty intervals for the two different approaches are highlighted with solid markers. Countries where the range of
the estimated under-5 mortality uncertainty interval using one approach is more than 33.3% wider than the mean of the ranges of the
estimated under-5 mortality uncertainty intervals for the two different approaches are labelled.
IraqPalau
Papua N. G.
Syria
T. & T.
UkraineUruguayUzbekistan
China
India
Nepal
Russian Fed.
050
100
150
200
Loes
s−ba
sed
appr
oach
0 50 100 150 200Spline−based approach
Fig. 37: Comparison of loess- and spline-based uncertainty interval ranges for estimates of under-5 mortality in the year 2015.5 in countries
with more complex ad-hoc weight adjustments. Countries where the uncertainty interval range estimated under one approach is more than
50% wider than that estimated under the other approach are highlighted with solid markers. Countries where the uncertainty interval
range estimated under one approach is more than 100% wider than that estimated under the other approach are labelled. Zimbabwe is
excluded to aid clarity.
35
and each year separately, the differences between the ranges of the spline- and loess-based estimated under-5 mortality
uncertainty intervals.
For countries with no ad-hoc weight adjustments or merely a down-weighting of the vital registration data, in 2000
40% of estimated uncertainty intervals have ranges within 10% of the mean and 64% are within 20%. Approximately
as many countries have a wider spline-based estimated uncertainty interval as have a wider loess-based estimated
uncertainty interval. As time progresses, estimated uncertainty interval ranges get a little further from the mean. By
2015 only 36% remain within 10% of the mean, with 44% more than 20% from the mean. The proportion of countries
where the loess-based estimated uncertainty interval is narrower than the spline-based estimated uncertainty interval
also increases so that by 2015 almost half the countries have a loess-based estimated uncertainty interval less than
90% the width of the mean. Thus, whilst it may be expected that the loess-based approach would provide wider
uncertainty intervals due it allowing for some ‘model’ uncertainty by considering different α values, this is not seen
here.
For countries with more complex ad-hoc weight adjustments, in 2000 almost 40% of estimated uncertainty intervals
have ranges within 10% of the mean and over 60% are within 20%. Over three quarters of countries have a loess-based
estimated uncertainty interval that is wider than the spline-based estimated uncertainty interval. As time progresses,
estimated uncertainty interval ranges get further from the mean until by 2015 less than 20% are within 10% of the
mean and nearly 60% are more than 20% from the mean. However, by 2015 almost as many countries have a wider
spline-based estimated uncertainty interval as have a wider loess-based estimated uncertainty interval.
Loess-based uncertainty interval range as a % of the
mean of the loess- and spline-based uncertainty interval ranges
YearAd-hoc weighting
<80% 80–90% 90–100% 100–110% 110–120% >120%adjustment?
2000.5None/VR only 4 (16%) 4 (16%) 6 (24%) 4 (16%) 2 (8%) 5 (20%)
More complex 3 (9%) 2 (6%) 3 (9%) 10 (29%) 6 (18%) 10 (29%)
2005.5None/VR only 7 (28%) 4 (16%) 5 (20%) 5 (20%) 1 (4%) 3 (12%)
More complex 8 (24%) 5 (15%) 3 (9%) 5 (15%) 7 (21%) 6 (18%)
2010.5None/VR only 6 (24%) 5 (20%) 4 (16%) 6 (24%) 0 (0%) 4 (16%)
More complex 9 (26%) 7 (21%) 4 (12%) 3 (9%) 4 (12%) 7 (21%)
2015.5None/VR only 7 (28%) 5 (20%) 2 (8%) 7 (28%) 0 (0%) 4 (16%)
More complex 13 (38%) 3 (9%) 4 (12%) 2 (6%) 5 (15%) 7 (21%)
Table 9: Loess-based uncertainty interval range as a % of the mean of the loess- and spline-based uncertainty interval ranges. ‘None/VR
only’ is either no ad-hoc weight adjustments or merely a down-weighting of the vital registration data.‘More complex’ is more complex
ad-hoc weight adjustments.
Conclusions
Although it is again perhaps more informative to concentrate on the results obtained in countries where these is little
or no ad-hoc weight adjustment, there is much variability in the spline- and loess-based uncertainty intervals for both
36
categories of ad-hoc weight adjustment. Whilst some spline-based uncertainty intervals correspond almost exactly to
the loess-based equivalent, for others there is only a small overlap. The relative widths of the uncertainty intervals
gives no real suggestion that one approach may generally provide either wider or narrower intervals, although the
differences between the two approaches generally increase a little as time progresses.
This method for incorporating uncertainty into the spline-based approach only deals with uncertainty about the
fitted spline at the time point of interest, whereas in the loess-based approach both the uncertainty about each fitted
loess curve and the uncertainty surrounding the appropriate level of smoothing to use are handled through the pooling
of the simulated random draws across the set of α values. Although this may lead to the expectation of the loess-based
approach providing wider uncertainty intervals, this is not consistently seen here.
5.2 Loess-based approach
The loess-based approach detailed by Murray et al [2] includes a random draw simulation approach to the estimation
of uncertainty intervals for each estimate of childhood mortality. However, an analytic approach analogous to that
detailed for the spline-based approach in Section 5.1.2 could be utilised instead. This is described in Section 5.2.1.
5.2.1 Analytic approach
For a given value of α, consider the basic loess function (2.2),
log(y) = β0 + β1x + β2z + ε.
From this expression, fitted values can be seen to be
log(y) = β0 + β1x + β2z (5.4)
with variance
var(log(y)) =var(β0) + x2var(β1) + z2var(β2) + 2xcov(β0, β1) + 2zcov(β0, β2) + 2xzcov(β1, β2). (5.5)
However, as non-vital registration data are assumed (i.e. z is set to 0 for prediction purposes), (5.4) and (5.5)
reduce to
log(y) = β0 + β1x (5.6)
and
var(log(y)) = var(β0) + x2var(β1) + 2xcov(β0, β1). (5.7)
Thus log(y) can be thought of as following the normal distribution
log(y) ∼ N(β0 + β1x, var(β0) + x2var(β1) + 2xcov(β0, β1)),
or again using the t-distribution analogously if the sample size is small.
If this distribution is combined across the set of α values then an analytic uncertainty interval can be created. The
probability density function (PDF) of log(y) across the set of α values can be found as the mean of the PDFs for each
37
3.4 3.5 3.6 3.7 3.8
02
46
810
Log under−5 mortality (per 1000)
Pro
babi
lity
dens
ity fu
nctio
n
3.40 3.45 3.50 3.55 3.60 3.65 3.70 3.75
0.0
0.2
0.4
0.6
0.8
1.0
Log under−5 mortality (per 1000)
Cum
ulat
ive
dist
ribut
ion
func
tion
0.025
0.5
0.975
3.47 (32.2) 3.56 (35.3) 3.67 (39.0)
Fig. 38: Probability density functions (left-hand plot; black curves represent probability density functions for individual α values, red
curve is their mean) and cumulative distribution functions (right-hand plot; black curves represent cumulative distribution functions for
individual α values, red curve is their mean, dashed blue line corresponds to a cumulative distribution of 0.5, dashed red lines correspond
to a cumulative distribution of 0.025 and 0.975, values in brackets are transformed back to the original scale) for Armenia in 2000.
α value. Similarly, the cumulative distribution function (CDF) of log(y) across the set of α values can be found as the
mean of the CDFs for each α value. From this overall CDF, the estimate of the mortality can be simply defined as
the value of log(y) at which the CDF is equal to 0.5, and the limits of the uncertainty interval as the values of log(y)
at which the CDF is equal to 0.025 and 0.975.
This approach is illustrated for Armenia in 2000 in Fig. 38. It can be seen that the estimated under-5 mortality
per 1000 is 35.3, with an uncertainty interval from 32.2 to 39.0. These figures are identical to those calculated using
the random draw simulation approach given in Table 6.
More generally, as the random draw simulation approach is simulating random draws from the multivariate normal
distributions defined by the same estimated coefficients, variances and covariances used in (5.6) and (5.7), with sufficient
random draws the estimated uncertainty intervals under the two approaches should be identical.
6 Further extensions and alternative approaches
6.1 Incorporation of sampling variability
Sampling variability affects any data collection where the sample of subjects is not the entirety of the population.
From the Eritrea 2002 Demographic and Health Survey (DHS) Final Report [3]:
The sample of respondents selected in the [DHS] is only one of many samples that could have been selected
from the same population, using the same design and expected size. Each of these samples would yield
results that differ somewhat from the results of the actual sample selected. Sampling errors are a measure
of the variability between all possible samples. Although the degree of variability is not known exactly, it
can be estimated from the survey results.
38
Currently, neither the spline- nor loess-based approach make any explicit attempt to take sampling variability into
account. One way to incorporate sampling variability into the modelling could be to use inverse variance weighting.
Estimated sampling error can be obtained for each survey, then an appropriate scaled sampling error calculated for
each derived value of child mortality from the survey. Vital registration and census data can be assumed to have
minimal random sampling error. Both the spline- and loess-based approaches need little modification to incorporate
sampling variability in this way. The weights in the weighted least squares regressions would simply become the
product of the existing weights and the inverse of the sampling variance for each data point. Incorporating sampling
variability should, on average, reduce uncertainty.
As an example, predicted under-5 mortality in 2015 using the loess-based approach is recalculated, using inverse
variance weighting to incorporate sampling variability, for three countries (Congo, Eritrea and Turkmenistan). The
results are presented in Table 10. It can be seen that the width of the uncertainty interval is reduced in each instance.
The predicted under-5 mortality is also reduced, though this would not be expected to be generally the case.
Observations Predicted 2015 under-5 mortality (per 1000)
Country VR DHSIgnoring Incorporating Percentage change
sampling variability sampling variability Estimate UI width
Congo 0 12 138 (103, 185) 133 (102, 173) −4% −13%
Eritrea 0 10 43 (26, 62) 41 (28, 55) −5% −25%
Turkmenistan 16 11 49 (33, 73) 38 (23, 58) −22% −13%
Table 10: A comparison of predicted 2015 under-5 mortality using the loess-based approach when sampling variability is ignored and
when sampling variability is incorporated. VR is vital registration data, DHS is Demographic and Health Survey, UI is uncertainty interval.
6.2 Multilevel modelling
Within each country, observed values of child mortality coming from the same surveys are likely to be more highly
correlated than values coming from different studies. However, both the spline- and loess-based approaches to esti-
mating childhood mortality assume independence of the observations. One approach for handling the dependencies
between observations would be to use a multilevel model within each country with individual observations as the level
1 variable and survey as the level 2 variable.
The simplest multilevel approach is a random intercepts model. The general random intercepts spline model is
y = (β0 + ui) + β1x +K∑
k=1
bk(x− κk)+ + ε,
where ui ∼ N(0, σu) is the random intercept for survey i.
As an example, the random intercepts splines model is fitted for three countries (Micronesia, Papua New Guinea and
Armenia) and compared to the fitted conventional spline curve in Fig. 39–41. For Micronesia and Papua New Guinea,
where the within-survey trends in under-5 mortality often differ from the overall trend assuming independence of the
data, the fitted conventional spline and random intercepts spline curves differ greatly. For Armenia, where the within-
survey trends in under-5 mortality (in the surveys with non-zero weighting) are similar the overall trend assuming
independence of the data, the fitted curves differ little.
39
3040
5060
70U
nder
−5
mor
talit
y (p
er 1
000)
1980 1985 1990 1995 2000 2005 2010 2015Year
Conventional spline Random intercepts spline
Fig. 39: Comparison of conventional spline and random intercepts spline approaches for Fed. States of Micronesia.
5010
015
020
0U
nder
−5
mor
talit
y (p
er 1
000)
1960 1970 1980 1990 2000 2010Year
Conventional spline Random intercepts spline
Fig. 40: Comparison of conventional spline and random intercepts spline approaches for Papua New Guinea.
40
1020
3040
5060
7080
90U
nder
−5
mor
talit
y (p
er 1
000)
1975 1980 1985 1990 1995 2000 2005 2010 2015Year
Conventional spline Random intercepts spline
Fig. 41: Comparison of conventional spline and random intercepts spline approaches for Armenia.
7 Summary
The spline- and loess-based approaches as currently implemented both have limitations. Both include some level
of subjectivity, meaning that any estimates are less reproducible and transparent. In the loess-based approach, the
selection of the minimum and maximum values of α to use has an element of subjectivity. Whilst this will not usually
have a major effect on the central point estimate, it may have an important impact on the uncertainty range obtained.
Additional subjectivity is introduced by the manner in which ‘extreme outliers’ are excluded from the analysis. In the
spline-based approach subjectivity arises in the down-weighting of datasets which are deemed to be ‘clearly aberrant’
and in the choosing the ‘more consistent’ of the fitted infant and under-5 mortality curves.
However, the loess-based approach does offer several advantages over the spline-based approach. For example,
under the loess-based approach the estimates lie on a smooth rather than piecewise linear (on the log scale) curve.
Model uncertainty, or at least one aspect of it, is incorporated through the use of different smoothing parameters in
the loess-based approach, whereas in the spline-based approach the weights are fixed, aside from the ad-hoc weight
adjustments. The loess-based approach handles vital registration (VR) data in a more appropriate way, through an
additive parameter which allows VR data to influence the shape of the fitted curve but not the position, if it is believed
that VR coverage is less than 100%. Also, the loess-based approach includes the estimation of an uncertainty interval
whereas the spline-based approach does not currently, although the inclusion of this is a simple extension.
In practice, estimated/predicted child mortality using the spline- and loess-based approaches has been seen to often
be very similar, especially in countries where these is little or no ad-hoc weight adjustment in the spline-based ap-
proach, although discrepancies between the approaches generally increase as time since the last observation increases.
In 2000 there appears to be no systematic difference between the approaches, but by 2015 there is some evidence of
the loess-based approach being more likely to provide slightly greater predicted values. Predicted childhood mortality
is most likely to differ between the two approaches when there is a more recent deviation from a long-term trend in
the data. Incorporating uncertainty into the spline-based approach provides uncertainty intervals which are often rela-
41
tively similar to those obtained under the loess-based approach, though in some countries there are sizeable differences.
Thus there remain several general issues:
• How should sampling variability be incorporated into the estimation process? Through inverse variance weight-
ing?
• How should correlations between data points be incorporated into the estimation process? Through multilevel
modelling?
• How should uncertainty be quantified in the estimation process? Through a random draw simulation approach
or analytic limits?
Appendix: Comparison of the datasets
A comparison of the datasets used with the spline-based approach by UNICEF and with the loess-based approach by
Murray et al [2] (both accessed November 2007) may highlight differences which would cause discrepancies between
estimates of childhood mortality even if the same modelling approach was used.
Comparing the 60 available country datasets used for the spline-based approach (detailed in Section 4.1) with the
equivalent datasets used for the loess-based approach (downloaded from www.healthmetricsandevaluation.org/
mortality.xls) shows that whilst in some cases the datasets appear very similar (for example Brazil, see Fig. 42),
more often there are obvious differences. Sometimes there are data points in the dataset used for the spline-based
approach which are not present in the dataset used for the loess-based approach (for example Ukraine, see Fig. 43)
and sometimes the opposite is the case (for example Mexico, see Fig. 44). For a minority of countries there appears
to be little overlap at all between the two datasets (for example Somalia, see Fig. 45).
Given these differences, it is clear that discrepancies in published childhood mortality estimates under the two
approaches are not entirely due to the modelling approaches utilised.
References
[1] K. Hill, R. Pande, M. Mahy, and G. Jones. Trends in Child Mortality in the Developing World: 1960 to 1996.
UNICEF, New York, 1999.
[2] C. J. Murray, T. Laakso, K. Shibuya, K. Hill, and A. D. Lopez. Can we achieve Millennium Development Goal 4?
New analysis of country trends and forecasts of under-5 mortality to 2015. Lancet, 370(9592):1040–54, 2007.
[3] National Statistics and Evaluation Office (NSEO) [Eritrea] and ORC Macro. Eritrea Demographic and Health
Survey 2002. NSEO and ORC Macro, Calverton, Maryland, USA, 2003.
42
050
100
150
200
Und
er−
5 m
orta
lity
(per
100
0)
1960 1970 1980 1990 2000 2010Year
Spline dataset0
5010
015
020
0U
nder
−5
mor
talit
y (p
er 1
000)
1960 1970 1980 1990 2000 2010Year
Loess dataset
Fig. 42: Comparison of the datasets provided for the spline- and loess-based approaches for Brazil.
43
1015
2025
3035
Und
er−
5 m
orta
lity
(per
100
0)
1970 1980 1990 2000 2010Year
Spline dataset10
1520
2530
35U
nder
−5
mor
talit
y (p
er 1
000)
1970 1980 1990 2000 2010Year
Loess dataset
Fig. 43: Comparison of the datasets provided for the spline- and loess-based approaches for Ukraine.
44
050
100
150
Und
er−
5 m
orta
lity
(per
100
0)
1960 1980 2000Year
Spline dataset0
5010
015
0U
nder
−5
mor
talit
y (p
er 1
000)
1960 1980 2000Year
Loess dataset
Fig. 44: Comparison of the datasets provided for the spline- and loess-based approaches for Mexico.
45
3040
5060
7080
Und
er−
5 m
orta
lity
(per
100
0)
1975 1980 1985 1990 1995 2000Year
Spline dataset30
4050
6070
80U
nder
−5
mor
talit
y (p
er 1
000)
1975 1980 1985 1990 1995 2000Year
Loess dataset
Fig. 45: Comparison of the datasets provided for the spline- and loess-based approaches for Tuvalu.
46