Bayesian Large Scale Structure inference

46
J. Jasche, Bayesian LSS Inference Jens Jasche La Thuile, 11 March 2012 Bayesian Large Scale Structure inference

TAGS:

description

Jens Jasche La Thuile, 11 March 2012. Bayesian Large Scale Structure inference. Motivation. Goal: 3D cosmography homogeneous evolution Cosmological parameters Dark Energy Power-spectrum / BAO non-linear structure formation Galaxy properties Initial conditions - PowerPoint PPT Presentation

Transcript of Bayesian Large Scale Structure inference

Page 1: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Jens Jasche

La Thuile, 11 March 2012

Bayesian Large Scale Structureinference

Page 2: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Goal: 3D cosmography• homogeneous evolution

• Cosmological parameters• Dark Energy• Power-spectrum / BAO

• non-linear structure formation• Galaxy properties• Initial conditions• Large scale flows

Motivation

Tegmark et al. (2004)

Jasche et al. (2010)

Page 3: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Motivation

No ideal Observations in reality

Page 4: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Motivation

No ideal Observations in reality• Statistical uncertainties

Noise, cosmic variance etc.

Page 5: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Motivation

No ideal Observations in reality• Statistical uncertainties • Systematics

Noise, cosmic variance etc.

Kitaura et al. (2009)

Survey geometry, selection effects,biases

Page 6: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Motivation

No ideal Observations in reality• Statistical uncertainties • Systematics

Noise, cosmic variance etc.

Kitaura et al. (2009)

Survey geometry, selection effects,biases

No unique recovery possible!!!

Page 7: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Bayesian Approach

“Which are the possible signals compatible with the observations?”

Page 8: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Object of interest: Signal posterior distribution

“Which are the possible signals compatible with the observations?”

Bayesian Approach

Page 9: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Object of interest: Signal posterior distribution

“Which are the possible signals compatible with the observations?”

Prior

Bayesian Approach

Page 10: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Object of interest: Signal posterior distribution

“Which are the possible signals compatible with the observations?”

Likelihood

Bayesian Approach

Page 11: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Object of interest: Signal posterior distribution

“Which are the possible signals compatible with the observations?”

Evidence

Bayesian Approach

Page 12: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Object of interest: Signal posterior distribution

“Which are the possible signals compatible with the observations?”

We can do science!• Model comparison• Parameter studies• Report statistical summaries• Non-linear, Non-Gaussian error propagation

Bayesian Approach

Page 13: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

LSS Posterior Aim: inference of non-linear density fields

• non-linear density field lognormal

See e.g. Coles & Jones (1991), Kayo et al. (2001)

Page 14: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

LSS Posterior Aim: inference of non-linear density fields

• non-linear density field lognormal

• “correct” noise treatment full Poissonian distribution

See e.g. Coles & Jones (1991), Kayo et al. (2001)

Credit: M. Blanton and the

Sloan Digital Sky Survey

Page 15: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

LSS Posterior

Problem: Non-Gaussian sampling in high dimensions • direct sampling not possible• usual Metropolis-Hastings algorithm inefficient

Aim: inference of non-linear density fields• non-linear density field

lognormal

• “correct” noise treatment full Poissonian distribution

See e.g. Coles & Jones (1991), Kayo et al. (2001)

Credit: M. Blanton and the

Sloan Digital Sky Survey

Page 16: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

LSS Posterior

Problem: Non-Gaussian sampling in high dimensions • direct sampling not possible• usual Metropolis-Hastings algorithm inefficient

Aim: inference of non-linear density fields• non-linear density field

lognormal

• “correct” noise treatment full Poissonian distribution

See e.g. Coles & Jones (1991), Kayo et al. (2001)

HADES (HAmiltonian Density Estimation and Sampling)

Jasche, Kitaura (2010)

Credit: M. Blanton and the

Sloan Digital Sky Survey

Page 17: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

LSS inference with the SDSS

Application of HADES to SDSS DR7• cubic, equidistant box with sidelength 750 Mpc• ~ 3 Mpc grid resolution• ~ 10^7 volume elements / parameters

Jasche, Kitaura, Li, Enßlin (2010)

Page 18: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Application of HADES to SDSS DR7• cubic, equidistant box with sidelength 750 Mpc• ~ 3 Mpc grid resolution• ~ 10^7 volume elements / parameters

Goal: provide a representation of the SDSS density posterior• to provide 3D cosmographic descriptions• to quantify uncertainties of the density distribution

LSS inference with the SDSS

Jasche, Kitaura, Li, Enßlin (2010)

Page 19: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Application of HADES to SDSS DR7• cubic, equidistant box with sidelength 750 Mpc• ~ 3 Mpc grid resolution• ~ 10^7 volume elements / parameters

Goal: provide a representation of the SDSS density posterior• to provide 3D cosmographic descriptions• to quantify uncertainties of the density distribution

3 TB, 40,000 density samples

LSS inference with the SDSS

Jasche, Kitaura, Li, Enßlin (2010)

Page 20: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

What are the possible density fields compatible with the data ?

LSS inference with the SDSS

Page 21: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

What are the possible density fields compatible with the data ?

LSS inference with the SDSS

Page 22: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Page 23: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

ISW-templates

LSS inference with the SDSS

Page 24: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Redshift uncertainties Photometric surveys

• deep volumes• millions of galaxies• low redshift accuracy

Page 25: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Redshift uncertainties

Galaxy positions

Matter density

Photometric surveys• deep volumes• millions of galaxies• low redshift accuracy

Page 26: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Redshift uncertainties

Galaxy positions

Matter density

We know that:• Galaxies trace the matter distribution• Matter distribution is homogeneous and isotropic

Jasche et al. (2010)

Photometric surveys• deep volumes• millions of galaxies• low redshift accuracy

Page 27: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Redshift uncertainties

Galaxy positions

Matter density

We know that:• Galaxies trace the matter distribution• Matter distribution is homogeneous and isotropic

Jasche et al. (2010)

Photometric surveys• deep volumes• millions of galaxies• low redshift accuracy

Page 28: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Redshift uncertainties

Galaxy positions

Matter density

We know that:• Galaxies trace the matter distribution• Matter distribution is homogeneous and isotropic

Jasche et al. (2010)

Joint inference

Photometric surveys• deep volumes• millions of galaxies• low redshift accuracy

Page 29: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Joint sampling

Page 30: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Joint sampling

Metropolis Block sampling:

Page 31: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Application of HADES to artificial photometric data • cubic, equidistant box with sidelength 1200 Mpc• ~ 5 Mpc grid resolution• ~ 10^7 volume elements / parameters

Photometric redshift inference

Page 32: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Application of HADES to artificial photometric data • cubic, equidistant box with sidelength 1200 Mpc• ~ 5 Mpc grid resolution• ~ 10^7 volume elements / parameters• ~ 2x10^7 radial galaxy positions / parameters

• (~100 Mpc)

Photometric redshift inference

Page 33: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Application of HADES to artificial photometric data • cubic, equidistant box with sidelength 1200 Mpc• ~ 5 Mpc grid resolution• ~ 10^7 volume elements / parameters• ~ 2x10^7 radial galaxy positions / parameters

• (~100 Mpc)• ~ 3x10^7 parameters in total

Photometric redshift inference

Page 34: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Application of HADES to artificial photometric data • cubic, equidistant box with sidelength 1200 Mpc• ~ 5 Mpc grid resolution• ~ 10^7 volume elements / parameters• ~ 2x10^7 radial galaxy positions / parameters

• (~100 Mpc)• ~ 3x10^7 parameters in total

Photometric redshift inference

Page 35: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Photometric redshift sampling

Jasche, Wandelt (2011)

Page 36: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Deviation from the truth

Jasche, Wandelt (2011)

Before After

Page 37: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Deviation from the truth

Jasche, Wandelt (2011)

Raw data Density estimate

Page 38: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Summary & Conclusion

LSS 3D density inference Correction of systematic effects, survey geometry, selection effects Accurate treatment of Poissonian shot noise Precision non-linear density inference Non-linear, non-Gaussian error propagation

Joint redshift & 3D density inference Positive influence on mutual inference Improved redshift accuracy Treatment of joint uncertainties

Also note: methods are general complementary data sets

• 21 cm, X-ray, etc

Page 39: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

The End …

Thank you

Page 40: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Resimulating the SGW

Building initial conditions for the Sloan Volume• resimulate the Sloan Great Wall (SGW)

Preliminary Work

Page 41: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Marginalized redshift posterior

Jasche, Wandelt (2011)

Page 42: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

IDEA: Follow Hamiltonian dynamics

Conservation of energy Metropolis acceptance probability is unity

All samples are accepted

Persistent motion

Hamiltonian Sampling

= 1

HADES (Hamiltonian Density Estimation and Sampling)HADES (Hamiltonian Density Estimation and Sampling)

Page 43: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

HADES vs. ARES

HADES

Variance

Mean

Page 44: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

ARESHADES

HADES vs. ARES

Page 45: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Lensing templates

LSS inference with the SDSS

Page 46: Bayesian Large Scale Structure inference

J. Jasche, Bayesian LSS Inference

Extensions of the sampler Multiple block Metropolis Hastings sampling

Example redshift sampling


Aspects of Bayesian Inference

Aspects of Bayesian Inference

Bayesian large-scale structure inference and cosmic web ...Bayesian large-scale structure inference and cosmic web analysis Florent Leclercq Institut d’Astrophysique de Paris Institut

Bayesian large-scale structure inference and cosmic web ...Bayesian large-scale structure inference and cosmic web analysis Florent Leclercq Institut d’Astrophysique de Paris Institut

Bayesian Inference for Categorical Data Analysis: A Surveypeople.stat.sc.edu/Hitchcock/bayesfinal.pdf · Bayesian Inference for Categorical Data Analysis: ... Bayesian Inference for

Bayesian Inference for Categorical Data Analysis: A Surveypeople.stat.sc.edu/Hitchcock/bayesfinal.pdf · Bayesian Inference for Categorical Data Analysis: ... Bayesian Inference for

Bayesian Inference!!!

Bayesian Inference!!!

Object Perception as Bayesian Inference · 2020-06-29 · Object Perception as Bayesian Inference 1 Object Perception as Bayesian Inference Daniel Kersten Department of Psychology,

Object Perception as Bayesian Inference · 2020-06-29 · Object Perception as Bayesian Inference 1 Object Perception as Bayesian Inference Daniel Kersten Department of Psychology,

Qualitative Robustness in Bayesian Inference - arXiv · Qualitative Robustness in Bayesian Inference ... Abstract The practical implementation of Bayesian inference requires numerical

Qualitative Robustness in Bayesian Inference - arXiv · Qualitative Robustness in Bayesian Inference ... Abstract The practical implementation of Bayesian inference requires numerical

Bayesian Inference (II) - University of California, Santa Cruzabrsvn/intro_bayes_2.pdf · Bayesian Inference in a Nutshell (Again) In Bayesian inference, uncertainty or degree of

Bayesian Inference (II) - University of California, Santa Cruzabrsvn/intro_bayes_2.pdf · Bayesian Inference in a Nutshell (Again) In Bayesian inference, uncertainty or degree of

Bayesian Inference 2019 - Courses

Bayesian Inference 2019 - Courses

Bayesian Inference (II)

Bayesian Inference (II)

Collapsed Variational Bayesian Inference of the Author ...€¦ · Collapsed Variational Bayesian Inference of the Author-Topic Model: Application to Large-Scale Coordinate-Based

Collapsed Variational Bayesian Inference of the Author ...€¦ · Collapsed Variational Bayesian Inference of the Author-Topic Model: Application to Large-Scale Coordinate-Based

Bayesian Inference for Structured Spike and Slab Priorspapers.nips.cc/paper/5464-bayesian-inference-for... · 2014-12-02 · Bayesian Inference for Structured Spike and Slab Priors

Bayesian Inference for Structured Spike and Slab Priorspapers.nips.cc/paper/5464-bayesian-inference-for... · 2014-12-02 · Bayesian Inference for Structured Spike and Slab Priors

Bayesian inference and generative models - TNU should I know about Bayesian inference? Because Bayesian principles are fundamental for • statistical inference in general • system

Bayesian inference and generative models - TNU should I know about Bayesian inference? Because Bayesian principles are fundamental for • statistical inference in general • system

J. Jasche, Bayesian LSS Inference Jens Jasche La Thuile, 11 March 2012 Bayesian Large Scale Structure inference.

J. Jasche, Bayesian LSS Inference Jens Jasche La Thuile, 11 March 2012 Bayesian Large Scale Structure inference.

Bayesian Inference (I)

Bayesian Inference (I)

Bayesian inference method

Bayesian inference method

Bayesian Inference - Michael Clark

Bayesian Inference - Michael Clark

Multi-scale Inference of Interaction Rules in Animal Groups Using Bayesian Model …garnett/files/papers/mann_et... · 2016. 12. 20. · Multi-scale Inference of Interaction Rules

Multi-scale Inference of Interaction Rules in Animal Groups Using Bayesian Model …garnett/files/papers/mann_et... · 2016. 12. 20. · Multi-scale Inference of Interaction Rules

Introduction to Bayesian Inference

Introduction to Bayesian Inference

Bayesian network inference

Bayesian network inference

Bayesian inference. - UFRGS

Bayesian inference. - UFRGS