The Mare Nostrum Universe

Gustavo YepesUniversidad Autónoma de Madrid

What is Mare Nostrum?

• IBM JS20 Blade Center:– 4812 PPC processors– 9.6 Tbytes of Memory– Myrinet interconection.– 233 Tbytes disk

What is Mare Nostrum?

• IBM JS20 Blade Center:– 4812 PPC processors– 9.6 Tbytes of Memory– Myrinet interconection.– 233 Tbytes disk

Still the most powerfullsupercomputer in Europe.It is located at the BSC in Barcelona

What is Mare Nostrum?

• IBM JS20 Blade Center:– 4812 PPC processors– 9.6 Tbytes of Memory– Myrinet interconection.– 233 Tbytes disk

Still the most powerfullsupercomputer in Europe.It is located at the BSC in Barcelona

A perfect place to create a Universe, not only because of its power…

The MareNostrum UniverseTREEPM+SPH simulation

ΛCDM model (WMAP1)• 500/h Mpc3 volume• GADGET 2 code (Springel 2005)• Adiabatic SPH+TREEPM Nbody

– 10243 FFT for the PM force.– 15 kpc force resolution.

• 2x10243 dark and sph particles– 109.33 partículas– 8x109 M dark matter– 109 M for gas particles

• 1 million dark halos bigger thana typical galaxy (1012 Mo)

• Simulation done at MareNostrum– 512 processors (1/8 total power)– 1Tbyte ram– 500 wallclock hrs (29 cpu years)– Output: 8600 Gbytes of data. – Same computing power than the

Millenium Run.

SpringelSpringel et al 2005et al 2005

Moore`sMoore`s LawLaw::

CapacityCapacity ofof professorsprofessorsdoubledouble everyevery 18 18 monthsmonths

N.BodyN.Body simulationssimulationsdoubledouble thethe numbernumber ofofparticlesparticles everyevery 16.4 16.4 monthsmonths

ExtrapolatingExtrapolating::

10101010 partpartíículas in 2008culas in 2008

……butbut itit waswas done in done in 20042004

“Hubbel Volume”

“Millenium Run”

SpringelSpringel et al 2005et al 2005

“Hubbel Volume”

“Millenium Run”Moore`sMoore`s LawLaw::

CapacityCapacity ofof professorsprofessorsdoubledouble everyevery 18 18 monthsmonths

N.BodyN.Body simulationssimulationsdoubledouble thethe numbernumber ofofparticlesparticles everyevery 16.4 16.4 monthsmonths

ExtrapolatingExtrapolating::

10101010 partpartíículas in 2008culas in 2008

……butbut itit waswas done in done in 20042004

MareNostrumMareNostrum UniverseUniverse (SPH) (SPH)

SpringelSpringel et al 2005et al 2005

“Hubbel Volume”

“Millenium Run”Moore`sMoore`s LawLaw::

CapacityCapacity ofof professorsprofessorsdoubledouble everyevery 18 18 monthsmonths

N.BodyN.Body simulationssimulationsdoubledouble thethe numbernumber ofofparticlesparticles everyevery 16.4 16.4 monthsmonths

ExtrapolatingExtrapolating::

10101010 partpartíículas in 2008culas in 2008

……butbut itit waswas done in done in 20042004

MareNostrumMareNostrum UniverseUniverse (SPH) (SPH)

TheoreticalTheoretical NN--bodybody limitlimit atat MN MN

Collaborators:• S. Gottlober (AIP Germany)• R. Sevilla (UAM, Spain)• M. G. Rivero (UAM, Spain)• M. Hoeft (Bremen, Germany)• Massimo Meneghetti (ZAH, Heildelberg)• Christian Wagner (AIP Germany)• Arman Khalatyan (AIP, Germany)• V. Turchaninov (IAM Moscow)• A. Faltenbacher (USC, California)• M. Plionis, S. Basilakos (NOA, Greece)• F. Atrio (USAL, Spain)• A. Doroshkevich (Moscow)• …

Halo finder• Hierarchical Friend of Friends analysis delivers

structures at virial overdensities and allsubstructures (Klypin,Gottlober,Kravtsov 1997)

• Minimal Spanning Tree technique:– Sorting particles in a cluster ordered sequence

• Mass, shapes, orientations, angular momentum, particle list, are obtained from the MST for eachobject.

• Full MPI implementation of the halo finder:– Timing: Analysis of 10243 particles is 4 hrs with 32

cpus.

Halo Mass function

• 4063 clusters with M > 1014 h-1 Msun• 58167 groups+clusters with M >1013 h-1 Msun• 506000 objects with M > 1012 h-1 Msun• More than 1 million objects with more than 100 particles

Spin parameter

Dark halosλ = 0.023 for all overdensitiesno hint that substructures have a differentspin distribution

“gas” halosλ ~ 0.05

Shape of the dark matter distribution atvirial overdensity:

halos with more than 500 particles

Shapes of halos

Shapes of halos

Shape of dark matter halos atVirial overdensity

Shape of dark matter halos at 64 times virial overdensity

Shape of halos

Shape of dark matter halos atVirial overdensity

Shape of the corresponding gas distribution in same halos

Baryon oscillations

Dark HalosDark matter particles

Baryon oscillations

Dark HalosDark matter particles

Baryon oscillations

Dark HalosDark matter particles

Baryon phase space diagram

Baryon phase space diagram

Baryon phase space diagram

Baryon phase space diagram

HOT

Warm-Hot

COLD

(T > 107 K)

(105 K < T < 107 K)

(T < 105 K)

Evolution of baryon phases

Evolution of baryon phases

All baryons WHIM baryons

Clusters of galaxies

Most massive cluster:

Baryon fraction in clusters

X-ray Temperature function

Scaling relations:Resolution effects

Comparison of Lx for two versionsof the same simulation:

Colored points: MareNostrum run

Black points: low-res version with2x5123 particles:(Basilakos et al 2005Ascasibar et al 2004-2005Rubiño’s talk this meeting)

Scaling relations:Resolution effects

Comparison of Lx for two versionsof the same simulation:

Colored points: MareNostrum run

Black points: low-res version with2x5123 particles:(Basilakos et al 2005Ascasibar et al 2004-2005Rubiño’s talk this meeting)

Green Points are data..

Evolution of scaling relations

Evolution of scaling relations

Evolution of scaling relations

Evolution of scaling relations

Evolution of scaling relations

(R. Sevilla’s Thesis)

Strong Lensing

• Arcs from our most massive cluster :• Composite U,B,V using ray tracing technique.• Collaboration with Massimo Meneghetti:

•Study role of cluster mergers on strong lensing efficiency.•Create the largest database of lensed simulations from 4000clusters between z=0 and 1.5.

SUMMARY• The MareNostrum Universe simulation uses 2 billion particles to

simulate both the dark and baryonic matter. It is one of the largestSPH simulations done up to now.

• It constitutes a very useful data base to do different studies of large-scale structure formation and distribution in both components.

• We have discussed here several of the analyses that are beingcarried out:– Halo identification and internal properties.– Baryon oscillations in the LSS of halos.– Baryon distribution.– Clusters of galaxies: scaling relations, baryon fractions, lensing..– S-Z signal from different structures (clusters, WHIM..)– etc..

• We are open to any other interesting study you may have thought– So please contact us if you’d like to put your hands on the tons of Gbytes of

data that the MareNostrum Universe is made of..

Thank you

– gustavo.yepes@uam.es– sgottloeber@aip.de

• The MareNostrum supercomputer is run by• We thank BSC for giving access to this facility• http://www.bsc.es

mailto:Gustavo.yepes@uam.eshttp://www.bsc.es/index.html

The Mare Nostrum UniverseWhat is Mare Nostrum?What is Mare Nostrum?What is Mare Nostrum?The MareNostrum Universe�TREEPM+SPH simulationCollaborators:Halo finderSpin parameterShapes of halosShapes of halosShape of halosBaryon oscillationsBaryon oscillationsBaryon oscillationsBaryon phase space diagramBaryon phase space diagramBaryon phase space diagramBaryon phase space diagramEvolution of baryon phasesEvolution of baryon phasesClusters of galaxiesBaryon fraction in clustersX-ray Temperature functionScaling relations:�Resolution effectsScaling relations:�Resolution effectsEvolution of scaling relationsEvolution of scaling relationsEvolution of scaling relationsEvolution of scaling relationsEvolution of scaling relationsStrong LensingSUMMARYThank you