Post on 18-Dec-2015
Parameter Redundancy and Identifiability
Diana Cole and Byron MorganUniversity of Kent
Initial work supported by an EPSRC grant to the National Centre for Statistical Ecology
Herring Gull Wandering Albatross Striped Sea BassGreat Crested Newts
Introduction
• If a model is parameter redundant you cannot estimate all the parameters in the model.
• Parameter redundancy equivalent to non-identifiability of the parameters.
• A model that is not parameter redundant will be identifiable somewhere (could be globally or locally identifiable).
• Parameter redundancy can be detected by symbolic algebra.• Ecological models and models in other areas are getting more
complex – then computers cannot do the symbolic algebra and numerical methods are used instead.
• In this talk we show some of the tools that can be used to overcome this problem.
Example 1- Cormack Jolly Seber (CJS) ModelCapture-Recature
Herring Gulls (Larus argentatus) capture-recapture data for 1983 to 1986 (Lebreton, et al 1995)
Numbers Ringed:
Numbers Recaptured:
111
123
78
R
9100
31030
2467
N
83
84
85Ringing yr
83
84
85
Recapture yr
84 85 86
Example 1- Cormack Jolly Seber (CJS) Model
i – probability a bird survives from occasion i to i+1pi – probability a bird is recaptured on occasion i = [1, 2, 3, p2, p3, p4 ]
recapture probabilities
Can only ever estimate 3 p4 - model is parameter redundant or non-identifiable.
etc1
00
0
22
43
433232
433221322121
pp
p
ppp
pppppp
Q
9100
31030
2467
N
r cn
ij ijicr c
ij
n
i
NRn
ij ij
n
i
n
ij
Nij QQL
11
1
111
123
78
R
Derivative Method (Catchpole and Morgan, 1997)
Calculate the derivative matrix D
rank(D) = 5 rank(D) = 5 Number estimable parameters = rank(D). Deficiency = p – rank(D) no. est. pars = 5, deficiency = 6 – 5 = 1
T
p
pp
p
ppp
pp
p
)ln(
)ln(
)ln(
)ln(
)ln(
)ln(
43
4332
32
433221
3221
21
κ
432321 ppp
j
i
D
T
ppppppR
pppR
ppR
pR
)1( 4332213221211
4332211
32211
211
μ
i
j
D
323211
4232112211
433211321111
432211
4323113211
432321322121
00
0
00
0
ppR
ppRpR
ppRpRR
pppR
pppRppR
pppRppRpR
14
14
14
13
13
13
13
12
12
12
13
13
13
12
12
12
12
11
11
11
000
00
000
000
00
000
ppp
pppp
ppp
Derivative or Jacobian Rank Test
• Jacobian is the transpose of the derivative matrix, so two are interchangeable.
• Uses of rank test:– Catchpole and Morgan (1997) exponential family models,
mostly used in ecological statistics.– Rothenberg (1971) original general use, examples
econometrics.– Goodman (1974) latent class models.– Sharpio (1986) non-linear regression models.– Pohjanpalo (1982) first use for compartment models.
Derivative or Jacobian Rank Test• The key to the symbolic method for detecting parameter
redundancy is to find a derivative matrix and its rank.• Models are getting more complex.• The derivative matrix is therefore structurally more complex.• Maple runs out of memory calculating the rank.• Examples: Hunter and Caswell (2009), Jiang et al (2007)
• How do you proceed?– Numerically – but only valid for specific value of parameters.
But can’t find combinations of parameters you can estimate. Not possible to generalise results.
– Symbolically – involves extending the theory, again it involves a derivative matrix and its rank, but the derivative matrix is structurally simpler.
Wandering AlbatrossMulti-state models for sea birds
Striped Sea BassAge-dependent tag-return
models for fish
Exhaustive Summaries• An exhaustive summary, , is a vector that uniquely defines
the model (Walter and Lecoutier, 1982).
• Derivative matrix
• r = Rank(D) is the number of estimable parameters in a model.• p parameters; d = p – r is the deficiency of the model (how
many parameters you cannot estimate). If d = 0 model is full rank (not parameter redundant , identifiable somewhere) . If d > 0 model is parameter redundant (non-identifiable).
• More than one exhaustive summary exists for a model• CJS Example:
i
j
D
T
p
pp
p
ppp
pp
p
)ln(
)ln(
)ln(
)ln(
)ln(
)ln(
43
4332
32
433221
3221
21
κ
T
ppppppR
pppR
ppR
pR
)1( 4332213221211
4332211
32211
211
μ
Exhaustive Summaries• Choosing a simpler exhaustive summary will simplify the
derivative matrix.• CJS Example:
• Computer packages, such as Maple can find the symbolic rank of the derivative matrix if it is structurally simple.
• Exhaustive summaries can be simplified by any one-one transformation such as multiplying by a constant, taking logs, and removing repeated terms.
• A simpler exhaustive summary can also be found using reparameterisation.
323211
4232112211
433211321111
432211
4323113211
432321322121
00
0
00
0
ppR
ppRpR
ppRpRR
pppR
pppRppR
pppRppRpR
D
14
14
14
13
13
13
13
12
12
12
13
13
13
12
12
12
12
11
11
11
000
00
000
000
00
000
ppp
pppp
ppp
D
Methods For Use With Exhaustive SummariesWhat can you estimate?
(Catchpole and Morgan, 1998, developed separately for compartment models in Chappell and Gunn,1998 and Evans and Chappell , 2000
extended to exhaustive summaries in Cole and Morgan, 2009a)
• A model: p parameters, rank r, deficiency d = p – r• There will be d nonzero solutions to TD = 0. • Zeros in s indicate estimable parameters. • Example: CJS, regardless of which exhaustive summary is used
• Solve PDEs to find full set of estimable pars.
• Example: CJS, PDE:
Can estimate: 1, 2, p2, p3 and 3p4
djfp
i iij ,...,10
1
10000
4
3
pT
α
0434
3
p
ff
p
432321 ppp
Methods For Use With Exhaustive SummariesExtension Theorem
(Catchpole and Morgan, 1997 extended to exhaustive summaries in Cole and Morgan, 2009a)
• Suppose a model has exhaustive summary 1 and parameters 1.
• Now extend that model by adding extra exhaustive summary terms 2, and extra parameters 2. (eg. add more years of ringing/recovery) New model’s exhaustive summary is = [1 2]T and parameters are = [1 2]T.
• If D1 is full rank and D2 is full rank, the extended model will be full rank. The result can be further generalised by induction.
• Method can also be used for parameter redundant models by first rewriting the model in terms of its estimable set of parameters.
i
j
,1
,11
D
2
,1
,21
,2
,2
,1
,2
,1
,1
00 D
DDD
i
j
i
j
i
j
i
j
Methods For Use With Exhaustive Summaries The PLUR decomposition
• Write derivative matrix which is full rank r as D = PLUR (P is a square permutation matrix , L is a lower diagonal square matrix, with 1’s on the diagonal, U is an upper triangular square matrix, R is a matrix in reduced echelon form).
• If Det(U) = 0 at any point, model is parameter redundant at that point (as long as R is defined). The deficiency of U evaluated at that point is the deficiency of that nested model (Cole and Morgan, 2009a).
• Example 2: Ring-recovery model:
Rank(D) = 5
10000
01000
00100
00010
00001
R)1)(1(
)(Det2,11,112,11,1
2,11,1
aa
U
0)(DetIf 2,11,1 U
Therefore nested model is parameter redundant with deficiency 1
4)(Rank2,11,1
U
aa 12,11,1
aa
aaaaa
2,112,1
1,11,111,1
0Q
Finding simpler exhaustive summaries Reparameterisation
1. Choose a reparameterisation, s, that simplifies the model structure.CJS Model (revisited):
2. Reparameterise the exhaustive summary. Rewrite the exhaustive summary, (), in terms of the reparameterisation - (s).
43
32
32
21
21
5
4
3
2
1
p
p
p
p
p
s
s
s
s
s
s
)ln(
)ln(
)ln(
)ln(
)ln(
)ln(
)(
43
4332
32
433221
3221
21
p
pp
p
ppp
pp
p
θ
)ln(
)ln(
)ln(
)ln(
)ln(
)ln(
)(
5
54
3
542
32
1
s
ss
s
sss
ss
s
s
43
433232
433221322121
00
0
p
ppp
pppppp
Q
Reparameterisation3. Calculate the derivative matrix Ds.
4. The no. of estimable parameters = rank(Ds)
rank(Ds) = 5, no. est. pars = 5
5. If Ds is full rank ( Rank(Ds) = Dim(s) ) s = sre is a reduced-form exhaustive summary. If Ds is not full rank solve set of PDE to find a reduced-form exhaustive summary, sre.There are 5 si and the Rank(Ds) = 5, so Ds is full rank. s is a reduced-form exhaustive summary.
15
15
15
14
14
13
13
12
12
11
000
0000
0000
0000
00000
)(
sss
ss
ss
ss
s
s
s
i
js
D
.)(DimRankif
si
js
Reparameterisation
6. Use sre as an exhaustive summary.
A reduced-form exhaustive summary is
Rank(D2) = 5; 5 estimable parameters.Solve PDEs: estimable parameters are 1, 2, p2, p3 and 3p4
43
32
32
21
21
s
p
p
p
p
p
re
3
22
11
4
33
22
2
0000
000
000
0000
000
000
p
pp
pp
s
i
rejD
ReparameterisationExample 2
• Hunter and Caswell (2009) examine parameter redundancy of multi-state mark-recapture models, but cannot evaluate the symbolic rank of the derivative matrix (developed numerical method).
• 4 state breeding success model:
1)...()(
1
loglog
1211
1),(
1
1 1
4
1
4
1
),(),(,
rc
rc
mL
Trrcccc
Trrcr
N
r
N
rc i j
crij
crji
II
)1(0)1(0
0)1(0)1(
)1()1()1()1(
4422
3311
444333222111
444333222111
0000
0000
000
000
2
1
p
psurvival breeding given survival successful breeding recapture
Wandering Albatross
1 3
2 4
1 success
2 = failure
3 post-success
4 = post-failure
Reparameterisation
1. Choose a reparameterisation, s, that simplifies the model structure.
2. Rewrite the exhaustive summary, (), in terms of the reparameterisation - (s).
2
1
333
222
111
14
13
3
2
1
p
p
s
s
s
s
s
s
)1(
)1(
)1(
)(
121
21
211
2222
2221
1112
1111
pp
p
p
p
p
θ
)1(
)(
131321
146
132
145
131
sss
ss
ss
ss
ss
s
Reparameterisation3. Calculate the derivative matrix Ds.
4. The no. of estimable parameters =rank(Ds)
rank(Ds) = 12, no. est. pars = 12, deficiency = 14 – 12 = 2
5. If Ds is full rank s = sre is a reduced-form exhaustive summary. If Ds is not full rank solve set of PDE to find a reduced-form exhaustive summary, sre. Tre sssssssssssssssss 104934837141312116521 //
139
13145513
13131113
0000
)(000
)22(000
)(
ss
sssss
sssss
sD
i
js
s
.)(DimRankif
si
js
Reparameterisation Method
6. Use sre as an exhaustive summary.T
re pp
2244411333
4
4
3
3214433222111222111
s
Breeding Constraint
Survival Constraint
1= 2=
3= 4
1= 3,
2= 4
1= 2,
3= 4
1, 2,
3,4
1= 2= 3= 4 0 (8) 0 (9) 1 (9) 1 (11)
1= 3 ,2= 4 0 (9) 0 (10) 0 (10) 2 (12)
1= 2, 3= 4 0 (9) 0 (10) 1 (10) 1 (12)
1,2,3,4 0 (11) 0 (12) 0 (12) 2 (14)
Reparameterisation MethodFurther Examples
Multi-state models - general exhaustive summary has been developed if there is more than one observable state (Cole, 2009). Maple procedures for finding this exhaustive summary and the derivative matrix. Able to come up with general rules.
Jiang et al (2007) age-dependent fisheries model is more complex, but essentially uses reparameterisation method (Cole and Morgan, 2009b). Able to give general results, whereas Jiang et al (2007) result only applies for 3 years of data.
Wandering Albatross
Striped Sea Bass
Reparameterisation MethodFurther Examples
Multi-state analysis of Great Crested Newts. The parameter redundancy of the more complex models can be examined using the reparameterisation method to find a simpler exhaustive summary. This example consists of 2 states, one observable and one unobservable, so required development of another simpler exhaustive summary (McCrea and Cole work in progress).
Parameter redundancy in Pledger et al (2009)'s stopover models (Matechou and Cole unpublished work).
Clint – a male great crested newt
Sandpiper
Reparameterisation MethodFurther Examples
• Audoly et al (1998) consider a linear compartment model :
Reparameterisation MethodFurther Examples
• In Catchpole and Morgan (2009a) we use the reparameterisation method to show that the model is not redundant.
)(),(),(
),(),(
0100
0010
000/1
0001
)(0
)(0
0)(
0)(
1
0434144341
3403432332
23321221
14124121
ttxtxt
txtV
kkkkk
kkkkk
kkkk
kkkk
T
BuθAθ
θCθyCB
A
TVkkkkkkkkkk 104031441433423321221
Reparameterisation MethodFurther Examples
• We show that an exhaustive summary is:
• By solving si() = bi i = 1,...,11 we find there is a unique solution with k21 = b9, k12 = b5/b9,...,V1 = b11. Hence the model is globally identifiable.
1
3441
21
43134
3223
4114
2112
043414
034323
3212
4121
11
10
9
8
7
6
5
4
3
2
1
)(
)(
)(
)(
V
kk
k
kk
kk
kk
kk
kkk
kkk
kk
kk
s
s
s
s
s
s
s
s
s
s
s
s
Reparameterisation MethodFurther Examples
• Dochain et al (1995) examine the identifiability of models for the activated sludge process using a non-linear compartment model.
• Symbolic method possible for k=1.
• However for k=2 model too complex.
• Using the reparameterisation method with the extension theorem Cole and Morgan (2009a) show that for any k there are 3k estimable parameters (out of 4k+1) of the form
k
ii
i dt
tSYU
1
)()1( ki
tSK
tS
Y
X
t
tS
im
i
i
i ,...,1)(
)()(
1,
1max,
kiYSYSKY
YXiiiiim
i
ii ,...,1)1)(0(),1()0(,)1(
,max,
Conclusion• Exhaustive summaries offer a more general framework for
symbolic detection of parameter redundancy.• Parameter redundancy can be investigated symbolically by
examining a derivative matrix and its rank.• In the symbolic method we can find the estimable parameter
combinations (via PDEs).• The symbolic method can easily be generalised using the
extension theorem.• Parameter redundant nested models can be found using a
PLUR decomposition of any full rank derivative matrix.• The use of reparameterisation allows us to produce structurally
much simpler exhaustive summaries, allowing us to examine parameter redundancy of much more complex models symbolically.
• Methods are general and can in theory be applied to any parametric model.
References– Audoly, S. D’Angio, L., Saccomani, M. P. and Cobelli, C. (1998) IEEE Transactions on
Biomedical Engineering 45, 36-47.– Catchpole, E. A. and Morgan, B. J. T. (1997) Biometrika, 84, 187-196– Catchpole, E. A., Morgan, B. J. T. and Freeman, S. N. (1998) Biometrika, 85, 462-468
– Chappell, M. J. and Gunn, R. N. (1998) Mathematical Biosciences, 148 21-41.– Dochain, D., Vanrolleghem, P. A. and Van Dale, M. (1995) Water Research, 29, 2571-
2578.
– Evans, N. D. and Chappell, M. J. (2000) Mathematical Biosciences 168, 137-159.– Goodman, L. A. (1974) Biometrika, 61, 215-231.– Hunter, C.M. and Caswell, H. (2009). Ecological and Environmental Statistics Volume 3.
797-825– Jiang, H. Pollock, K. H., Brownie, C., et al (2007) JABES, 12, 177-194– Lebreton, J. Morgan, B. J. T., Pradel R. and Freeman, S. N. (1995). Biometrics, 51, 1418-
1428.– Pledger, S., Efford, M. Pollock, K., Collazo, J. and Lyons, J. (2009) Ecological and
Environmental Statistics Series: Volume 3. – Pohjanpalo, H. (1982) Technical Research Centre of Finland Research Report No. 56.– Rothenberg, T. J. (1971) Econometrica, 39, 577-591.– Shapiro, A. (1986) Journal of the American Statistical Association, 81, 142-149.– Walter, E. and Lecoutier, Y (1982) Mathematics and Computers in Simulations, 24, 472-
482
ReferencesRecent Work
– Cole, D. J. (2009) Determining Parameter Redundancy of Multi-state Mark-Recapture Models for Sea Birds. Presented at Eurings 2009 to appear in Journal of Ornithology.
– Cole, D. J. and Morgan, B. J. T (2009a) Determining the Parametric Structure of Non-Linear Models IMSAS, University of Kent Technical report UKC/IMS/09/005
– Cole, D. J. and Morgan, B. J. T. (2009b) A note on determining parameter redundancy in age-dependent tag return models for estimating fishing mortality, natural mortality and selectivity. IMSAS, University of Kent Technical report UKC/IMS/09/003 (To appear in JABES)
– See http://www.kent.ac.uk/ims/personal/djc24/parameterredundancy.htm for papers and Maple code