Two bootstrapping routines for obtaining uncertainty measurement around the nonparametric distribution obtained
in NONMEM VIPaul G. Baverel 1, Radojka M. Savic 2 and Mats O. Karlsson 1
1 Division of Pharmacokinetics & Drug Therapy, Uppsala University, Sweden2 INSERM U738, University Paris Diderot Paris 7, Paris, France
Two novel bootstrapping routines intended for nonparametric estimation methods are proposed. Their evaluation with a simple PK model in the case of informative sampling design was performed when applying FOCE- NONP in NONMEM VI but it is easily transposable to other nonparametric applications.
These tools can be used for diagnostic purpose to help detecting misspecifications with respect to the distribution of random effects.
From the sampling distribution obtained, standard uncertainty metrics, such as standard errors and correlation matrix can be derived in case reporting uncertainty is intended. References: [1]. Perl-speaks-NONMEM (PsN software): L. Lindbom, M. Karlsson, N. Jonsson.
http://psn.sourceforge.net[2]. Savic RM, Baverel PG, Karlsson MO. A novel bootstrap method for obtaining uncertainty around the nonparametric distribution. PAGE 18 (2008) Abstract 1390.
[3]. Efron B. Bootstrap methods: another look at the jackknife. Ann Stat 1979; 7:1-26.
Overall, the trend and the magnitude of the 95% CI derived with the full and the simplified nonparametric bootstrapping methods (N=100 and N=500) matched the true 95% CI in all distributional cases and regardless of individual numbers in original data.
The simplified version induced slightly less bias in quantifying uncertainty (prediction errors of 95% CI width) than the full version. This is expected as the former methodology derives uncertainty from the original data.
Quantifying uncertainty in parameter estimates is essential to support
decision making throughout model building process.
Despite providing enhanced estimates of parameter distribution,
nonparametric algorithms do not yet supply uncertainty metrics.
Two different permutation methods automated in PsN [1] were developed to quantify uncertainty around NPD (95% confidence interval) and nonparametric estimates (SEs and variance-covariance matrix):
o The full method [2] relies on N bootstraps of the original data and a re-analysis of both the preceding parametric as well as the nonparametric stepo The simplified method relies on N bootstrap samples of the vectors of individual probabilities associated with each unique support point of the NPD
Six informative datasets of 50 or 200 individuals were simulated from an
IV bolus PK model in which CL and V conformed to various underlying
distributional shapes (log-normal, bimodal, heavy-tailed). Residual
variability was set to 10% CV.
Re-estimation was conducted assuming normality under FOCE, and NPDs were estimated by applying FOCE-NONP method in NONMEM VI.
Figure 1: Sequential steps of the operating procedure of both the full and simplified nonparametric bootstrapping methods intended for nonparametric estimation methods.
Figure 2. On the left : 95% confidence intervals obtained based on the full and simplified nonparametric bootstrapping methodologies in case of various underlying distributions of CL. The true 95% CI around the true parameter distribution is also represented for comparison, as well as the parametric cumulative density function.On the right: Prediction errors of the 95% CI width are displayed for each quartile of parameter distribution, the true uncertainty being taken as reference.
UNDERLYING BIMODAL CLEARANCE DISTRIBUTION
UNDERLYING HEAVY-TAILED CLEARANCE DISTRIBUTION (200 IDs)
Simplified nonparametric bootstrapping method: 5-step procedure (ca 2 mn.)
Full nonparametric bootstrapping method: 7-step procedure (CPU time: ca 4 hr.)
BOOTSTRAP* N times
original data DJ
Bootstrapped
data B1...BN
PARAMETRIC ESTIMATION:
B1... BN
N sets (θ,Ω,σ) each defined at
<J support points
NONPARAMETRIC
ESTIMATION:
DJ given (θ,Ω,σ)
N sets NPDN
defined at J support points for <J
individuals in B1...BN
PARTITIONING
NPDN into J
individual probability densities IPDN
BOOTSTRAP*
IPDN according to
sample scheme B1...BN
NxN matrices MbootN of
bootstrapped IPD<J
RE-ASSEMBLING
bootstrapped IPD<J From NPDnewN construct
nonparametric 95% CI around NPD Derive SEs and correlation
matrix of nonparametric estimates
1 2 3 4 5
6
7 5
PARTITIONING
NPD into J
individual probability densities IPDJ
NONPARAMETRIC ESTIMATION:
Original data DJ
Matrice M (JxJ) of
individual probabilities: Row entries J individuals
Column entries J support points
NPD defined at J support points
BOOTSTRAP
IPDJ
N times
RE-ASSEMBLING
bootstrapped IPDJ
N matrices MbootN
of bootstrapped IPDJ
1 2 3 4
50 IDs200 IDs
The true uncertainty was derived by standard nonparametric bootstrapping (N=1000) [3] of the true individual parameters and used as reference for qualitative and quantitative assessment of the uncertainty measurements derived from both techniques.
CL distributionC
um
ula
tive d
ensi
ty f
unct
ion
10 20 30 40 50 60
0.0
0.2
0.4
0.6
0.8
1.0
Q1 Q2 Q3 Q4
TrueParametric95% CI true95% CI simplified (N=100)95% CI full (N=100)
CL distribution
Cum
ula
tive d
ensi
ty f
unct
ion
5 10 15 20 25 30 35
0.0
0.2
0.4
0.6
0.8
1.0
Q1 Q2 Q3 Q4
TrueParametric95% CI true95% CI simplified (N=100)95% CI full (N=100)
N sets NPDnewN defined at J support
points of NPDN
N matrices MN (JxJ) of individual
probabilities: Row entries J individuals
Column entries J support points
A single set of NPDnewN defined at J support points of NPD
Prediction errors
-0.05
0.00
0.05
CL
200 IDs
-0.10
-0.05
0.00
0.05
CL
50 IDs
Simplified Full
Q1 Q2 Q3 Q4
CL distribution
10 20 30 40
0.0
0.2
0.4
0.6
0.8
1.0
Q1 Q2 Q3 Q4
TrueParametric95% CI true95% CI simplified (N=100)95% CI full (N=100)
N=100 Prediction errorsN=500
Table 1. SEs of parameter estimates obtained from 100 stochastic simulations followed by estimations given the true model under FOCE, FOCE-NONP and the analytical solution under FOCE ($COVARIANCE) in NONMEM VI. The simplified methodology was applied to each simulated dataset; SEs were computed and average SEs were reported for comparison with SSE.
SE100 Stochastic Simulations followed by Estimations (SSE)
Simplified nonparametric
bootstrap version
True empirical FOCE
Asymptotic ($COV)FOCE
True empirical FOCE-NONP
(N=100)FOCE-NONP
Θ CL 0.022 0.022 0.021 0.021Θ V 0.024 0.022 0.024 0.02
Ω CL 0.01 0.009 0.01 0.008 Ω CL,V 0.007 0.006
Ω V 0.011 0.01 0.011 0.008
SEs obtained with the simplified methodology matched the ones obtained by SSE.
CL distribution
5 10 15 20 25 30 35
0.0
0.2
0.4
0.6
0.8
1.0
Q1 Q2 Q3 Q4
TrueParametric95% CI true95% CI simplified (N=500)95% CI full (N=500)
-0.05
0.00
0.05
CL
N=100
-0.05
0.00
0.05
CL
Simplified Full
N=500
Q1 Q2 Q3 Q4
Top Related