Covariance matrix estimation and linear process bootstrap ...
COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF...
22
COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF SIMULATIONS Linda Blot Fundamental Cosmology Meeting Teruel - 12/09/2017
Transcript of COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF …...COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF...
COVARIANCE ESTIMATION FROM LARGE ENSEMBLES OF
SIMULATIONSLinda Blot
Fundamental Cosmology Meeting
Teruel - 12/09/2017
MOTIVATIONS
➤ LSS data can constrain cosmological parameters with precision comparable to CMB data and are complementary
➤ Non-linearities are important modelling is harder
EUCLID collaboration
FROM OBSERVATIONS TO CONSTRAINTS
➤ Ideal world: full multivariate probability distribution of the observable for all the models
➤ If we assume multivariate Gaussian -> mean and covariance
➤ Estimation of the covariance:
✦ internal: from data
✦ external: from simulations
✦ model: from the theoretical model
FROM OBSERVATIONS TO CONSTRAINTS
➤ Ideal world: full multivariate probability distribution of the observable for all the models
➤ If we assume multivariate Gaussian -> mean and covariance
➤ Estimation of the covariance:
✦ internal: from data
✦ external: from simulations
✦ model: from the theoretical model
cov(k1, k2) =2
Nk1
P 2(k1)�k1,k2 +
1
V
Z
�k1
Z
�k2
d3k01
Vk1
d3k02
Vk2
T (k01,�k0
1,k02,�k0
2)
non-linear regime, bias + mask
FROM OBSERVATIONS TO CONSTRAINTS
➤ Ideal world: full multivariate probability distribution of the observable for all the models
➤ If we assume multivariate Gaussian -> mean and covariance
➤ Estimation of the covariance:
✦ internal: from data
✦ external: from simulations
✦ model: from the theoretical model
Sample covariance}
dcov(k1, k2) =1
Ns � 1
NsX
i=1
[
ˆPi(k1)� ¯P (k1)][ ˆPi(k2)� ¯P (k2)]
DEUS PARALLEL UNIVERSE RUN SIMULATIONS
Set A Set BSet C
WMAP-7 ΛCDM Set A: 12288 simulations
Set B: 96 simulations Set C: 512 simulations
Total: ~1.5M cpu hours on the ADA supercomputer at IDRIS
AMADEUS application:
• IC: optimised version of MPGRAFIC (Prunet 2008)
• N-body: improved version of RAMSES (Teyssier 2002)
• Halo finder: PFOF (Roy et al.2014)
DEUS PARALLEL UNIVERSE RUN SIMULATIONS
MASS RESOLUTION EFFECT
➤ Matter power spectrum
Heitmann et al. 2010Rasera et al. 2014
100
101
102
cov(
k,k
)/(P
2 lin(k
)/N
k/2
)
A
B
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
�0.4�0.2
0.00.20.40.60.81.0
�co
v(k,k
)z = 0.00
z = 0.30
z = 0.50
z = 1.00
z = 2.00
MASS RESOLUTION EFFECT
➤ Matter power spectrum variance
LB et al. 2015
Map the spectrum from the PDF of set A into the one of set B using only the first two moments
MASS RESOLUTION EFFECT CORRECTION
P̂ corr
A (k) =hP̂A(k)� P̄A(k)
i �ˆPB(k)
�ˆPA(k)
+ P̄B(k)
Prob
ability
P(k)
AB
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.40.80.91.01.11.21.31.4
�B(k
)/�
A(k
)
z = 0.00
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.40.80.91.01.11.21.31.4
z = 0.30
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.40.80.91.01.11.21.31.4
�B(k
)/�
A(k
)
z = 0.50
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4k (h/Mpc)
0.80.91.01.11.21.31.4
z = 1.000.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4
k (h/Mpc)
0.80.91.01.11.21.31.4
�B(k
)/�
A(k
)
z = 2.00
100
101
102
cov(
k,k
)/(P
2 lin(k
)/N
k/2
)
A
B
corrected
�0.4�0.2
0.00.20.4
�co
v A(k
,k)
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
�0.4�0.2
0.00.20.4
�co
v B(k
,k)
z = 0.00
z = 0.30
z = 0.50
z = 1.00
z = 2.00
CORRECTED MATTER POWER SPECTRUM VARIANCE
P̂ corr
A (k) =hP̂A(k)� P̄A(k)
i �ˆPB(k)
�ˆPA(k)
+ P̄B(k)
Blot et al. 2015
0.03 0.2 0.4 0.6 0.8 1.0k1 (h/Mpc)
0.03
0.2
0.4
0.6
0.8
1.0
k 2(h
/Mpc)
z = 2.00
�0.30
�0.15
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.2 0.4 0.6 0.8 1.0k1 (h/Mpc)
0.03
0.2
0.4
0.6
0.8
1.0k 2
(h/M
pc)
z = 1.00
�0.30
�0.15
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.2 0.4 0.6 0.8 1.0k1 (h/Mpc)
0.03
0.2
0.4
0.6
0.8
1.0
k 2(h
/Mpc)
z = 0.00
�0.30
�0.15
0.00
0.15
0.30
0.45
0.60
0.75
0.90
CORRELATION MATRIX
r(k1, k2) =cov(k1, k2)p
cov(k1, k1) cov(k2, k2)
Blot et al. 2015
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
S3
z = 0.00 z = 0.30
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
S3
z = 0.50
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
z = 105.00
PDF OF THE MATTER POWER SPECTRUM
Skewness
𝛘2 distribution with Nk d.o.f. → Gaussian for Nk >> 1
0.0
0.5
1.0
1.5
2.0
2.5
3.0
S4
z = 0.00 z = 0.30
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
0.0
0.5
1.0
1.5
2.0
2.5
3.0
S4
z = 0.50
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
z = 105.00
Blot et al. 2015
0.0
0.5
1.0
1.5
2.0
2.5
3.0
S4
z = 0.00 z = 0.30
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
0.0
0.5
1.0
1.5
2.0
2.5
3.0
S4
z = 0.50
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
z = 105.00
PDF OF THE MATTER POWER SPECTRUM
Kurtosis
𝛘2 distribution with Nk d.o.f. → Gaussian for Nk >> 1
0.0
0.5
1.0
1.5
2.0
2.5
3.0
S4
z = 0.00 z = 0.30
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
0.0
0.5
1.0
1.5
2.0
2.5
3.0
S4
z = 0.50
0.0 0.2 0.4 0.6 0.8 1.0k (h/Mpc)
z = 105.00
Blot et al. 2015
COSMOLOGICAL DEPENDENCE OF THE COVARIANCE MATRIX
512
(328.125)3
5123
2 x 1010
Cosmo
Symmetric variation of parameters wrt fiducial WMAP7
COSMOLOGICAL DEPENDENCE OF THE COVARIANCE MATRIX
0.0 0.2 0.4 0.6 0.8 1.0k
10�1
100
101
102
103
104
105
106
107
108
109
�2 (
P(k
))
z = 2.0
z = 1.0
z = 0.0
fiducial
w = �1.2
w = �0.8
Preliminary
0.0 0.2 0.4 0.6 0.8 1.0k
0
100
200
300
400
500
600
700
800
�2 (
P(k
))/�
2 L(P
(k))
fiducial w=-1 Ωm = 0.2573
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 2.00 w = �1.2
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 2.00 w = �0.8
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 2.00
�0.02
0.00
0.02
0.04
0.06
0.08
0.10
0.12
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 1.00 w = �1.2
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 1.00 w = �0.8
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 1.00
�0.01
0.00
0.01
0.02
0.03
0.04
0.05
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 0.00 w = �1.2
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 0.00 w = �0.8
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 0.00
�0.002
0.000
0.002
0.004
0.006
0.008
0.010
COSMOLOGICAL DEPENDENCE OF THE COVARIANCE MATRIX
Preliminary
COSMOLOGICAL DEPENDENCE OF THE COVARIANCE MATRIX
0.0 0.2 0.4 0.6 0.8 1.0k
10�1
100
101
102
103
104
105
106
107
108
109
�2 (
P(k
))
z = 2.0
z = 1.0
z = 0.0
fiducial⌦m = 0.3100
⌦m = 0.2046
Preliminary
0.0 0.2 0.4 0.6 0.8 1.0k
0
500
1000
1500
2000
2500
3000
�2 (
P(k
))/�
2 L(P
(k))
fiducial w=-1 Ωm = 0.2573
COSMOLOGICAL DEPENDENCE OF THE COVARIANCE MATRIX
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 2.00 ⌦m = 0.3100
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 2.00 ⌦m = 0.2046
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 2.00
�0.04
0.00
0.04
0.08
0.12
0.16
0.20
0.24
0.28
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 1.00 ⌦m = 0.3100
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 1.00 ⌦m = 0.2046
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 1.00
0.00
0.04
0.08
0.12
0.16
0.20
0.24
0.28
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 0.00 ⌦m = 0.3100
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 0.00 ⌦m = 0.2046
0.00
0.15
0.30
0.45
0.60
0.75
0.90
0.03 0.26 0.50 0.74 0.98 1.22k1 (h/Mpc)
0.03
0.26
0.50
0.74
0.98
1.22
k 2(h
/Mpc
)
z = 0.00
0.00
0.04
0.08
0.12
0.16
0.20
0.24
Preliminary
FOR THE FUTURE
➤ Impact on cosmological parameter constraints of
➤ Skewness of the PDF
➤ Cosmological dependence of the likelihood (covariance)
CONCLUSIONS
➤ Covariances are affected by numerical systematics
➤ Non-linearities skew the PDF of the matter power spectrum
➤ Covariance depend on cosmological parameters
COVARIANCE WITH APPROXIMATE METHODS
2.61 2.62 2.63 2.64 2.65
0.435
0.440
0.445
b1
f¥s8
kmax=0.25Fisher Forecast
MinervaCOLAPinocchio
Euclid Likelihood fitting WP
