Astronum2006 - Palm Springs1March 27-30, 2006 Numerical simulations of astrophysical plasmas :...
-
Upload
esther-deveraux -
Category
Documents
-
view
215 -
download
1
Transcript of Astronum2006 - Palm Springs1March 27-30, 2006 Numerical simulations of astrophysical plasmas :...
March 27-30, 2006Astronum2006 - Palm Springs 1
Numerical simulations of astrophysical plasmas :Numerical simulations of astrophysical plasmas :status and perspectives of the Saclay/DAPNIA status and perspectives of the Saclay/DAPNIA
software projectsoftware project
E. Audit, D. Pomarède, R. Teyssier, B. ThoorisCEA/DAPNIA - Saclay
Astronum2006 2D. Pomarède, CEA/DAPNIA
•Overview of numerical simulations in Saclay
•Cosmological structures formation : the RAMSES code
•Dynamics of the interstellar medium : the HERACLES code
•Numerical simulations software project
data handling
visualization
•Perspectives
Outline
Astronum2006 3D. Pomarède, CEA/DAPNIA
Numerical simulations in Saclay•An extensive program of simulations of astrophysical plasmas is
conducted at the Saclay/DAPNIA Laboratory
COSMOLOGICALSTRUCTURESFORMATION
RAMSES
STELLAREVOLUTION
ASH
DYNAMICSOF THE ISMHERACLES
FORMATION OFPROTO-PLANETARY
DISKSFARGO
•It covers a vast spectrum of challenging problems at various scales and has its foundations in a suite of independent numerical codes :
Astronum2006 4D. Pomarède, CEA/DAPNIA
Numerical simulations in Saclay : overview•Cosmological structures
– The formation of the large structures in the Universe– Interplay of the dark matter component and the baryon gas– RAMSES code (R. Teyssier) : N-body and hydrodynamical
simulations
•The Interstellar Medium– The formation and dynamics of molecular clouds– Radiation Hydrodynamics– HERACLES code (E. Audit et al.) : radiative transfer coupled to
hydrodynamics
•Stellar evolution– MHD of the Sun– Participation (A.S. Brun) to ASH developments (J. Toomre et al.)
•Proto-planetary systems– Study of the disk-planet tidal interactions– FARGO code (F. Masset) : 2D & 3D hydrodynamics
Astronum2006 5D. Pomarède, CEA/DAPNIA
Numerical simulations in Saclay : overview•Computing resources
– CCRT (CEA Supercomputing Center) :• HP/OSF1 256 quadri-processors alpha nodes• Linux 128 quadri-processors AMD Opteron
•“Horizon Project” resources (cosmology)– IDRIS (CNRS Supercomputing Center)
• 1024 processors Power4 cluster– MareNostrum (Barcelona Supercomputing Center)
• 2406 dual 64-bit processor nodes @ 2.2GHz, ~42 TeraFlops• the most powerful supercomputer in Europe
•Software Project– Developments are now addressed in the context of a
software engineering project aiming at the optimization and sharing of expertise for various components :
Algorithms parallelization/optimization Data Handling Database Post-treatment Visualization
Astronum2006 6D. Pomarède, CEA/DAPNIA
The RAMSES code•A hybrid simulation code
– N-body treatment to solve the Dark Matter (DM) dynamics– Hydrodynamical treatment of the baryonic component
•Position of the problem :– Objective is to study the structure formation in the Universe with high
spatial resolution– DM is believed to be the dominant component in mass of the
cosmological density field, with only a small fraction ~10% in baryons– At intermediate scales such as galaxy clusters DM stills dominates
but a gaseous component is introduced (constraints from observations of hot intracluster medium)
– At smaller scales, gas-cooling and fluid dynamics play a dominant role in the structure of galaxy-size objects
• at 1st order, baryons = hydrostatic ionized plasma trapped in DM gravitational potential wells
• complex hydrodynamical processes require accurate treatment : shock heating atomic radiation cooling star formation
Astronum2006 7D. Pomarède, CEA/DAPNIA
The RAMSES code•Resolution requirements
– a volume of 100 Mpc h-1 side with haloes simulated with 10 resolution elements requires a spatial resolution of 10 kpc h-1
dynamical range of 104
– 100 particles/galaxy 2563 particles in total
•Algorithms– F90 parallelized with MPI or OpenMP
– Based entirely on Adaptive Mesh Refinement (AMR) technique
– Tree based data structure allowing recursive refinements on a cell-by-cell basis
– DM particles considered as a collisionless N-body system described by the Vlasov-Poisson equations
– Hydrodynamical solver based on a 2nd-order Godunov method
– With 4.1107 cells a formal resolution of 81923 is reached at the 14th Level
•Ref : R. Teyssier, A&A 385, 337-364 (2002)
The RAMSES AMRlevel 2
level 3
level 5
level 9
level 14
basic element of AMR structure :group of 2dim
sibling cells called “octs”
level 11
RAMSES simulations of cosmological structuresRecent results obtained with RAMSES :“The History of the Baryon Budget – Cosmic Logistics in a Hierarchical Universe”, Y. Rasera and R.
Teyssier, accepted for publication in A&A, astro-ph/0505473 (23 May 2005)
“Kinematic Dynamos using Constrained Transport with High Order Godunov Schemes and Adaptive Mesh Refinement”, R. Teyssier, S. Fromang, E. Dormy, accepted for publication in Journal of Computa-tional Physics, astro-ph/0601715 (31 Jan 06)
extension to the modeling of magnetic field evolution (MHD)
gas density zooming sequence : from large scale filamentary structures to spiral discs
Astronum2006 11D. Pomarède, CEA/DAPNIA
The HERACLES code
•A 3-D RHD code written in F90 parallelized with MPI•Implemented in Cartesian, cylindrical, and spherical coordinates•In current implementation only regular mesh grids•Solves the equations of radiative transfer coupled to
hydrodynamics : fluid evolution is determined by the classical conservations
equations (mass, momentum, energy) + source terms characterizing the momentum and energy
exchanges between the fluid and the radiation
•The transfer equations are solved by a second order Godunov type method and integrated implicitly using iterative solvers
•Reference : “HERACLES: a three dimensional radiation hydrodynamics code” ,M. González, E. Audit, and P. Huynh, to be published
Astronum2006 12D. Pomarède, CEA/DAPNIA
HERACLES - simulations of ISM dynamics•Current studies focus on the thermal fragmentation of turbulent
flows of interstellar hydrogen neutral atomic hydrogen (HI) represents more than 50% of the ISM molecular clouds form through the condensation of a warm neutral
phase (WNM) into a cold medium (CNM)
•Thermal processes : heating : photo-electric effect on small grains and poly-aromatic
hydrocarbons due to the far-ultraviolet galactic radiation cooling by molecules and various processes
•Initial conditions : collision between two turbulent streams (converging flow) of WNM
triggers the formation of CNM structures
•Resolution requirements : 4 different spatial scales are involved1. cooling length of the WNM typically ~ 10-20 pc2. typical size of CNM fragments ~ 0.1 pc3. conduction length in the 10-1-10-3 pc range4. size of the shocked layer ~ 10-3 pc dynamic range of 104
Astronum2006 13D. Pomarède, CEA/DAPNIA
HERACLES - simulations of ISM dynamics•First studies conducted with 2D simulations
•Analysis of structure morphology, internal velocity dispersion, fraction of gas in the various states in
E. Audit and P. Hennebelle, A&A 443,1-13 (2005)
weakly turbulent forcing very turbulent forcing
Fragments of cold, high-density gas connected by thin layers of low-density gas. Sharp thermal fronts bound the structures and connect them to the warm surrounding medium.
Complex density field. The different phases are highly interwoven with pockets of warm gas embedded in filaments of cooler gas.
HERACLES simulations of ISM dynamics
2D slice
the limited dynamical range leads to under-estimation of the highest density reached during supersonic collisions
Astronum2006 16D. Pomarède, CEA/DAPNIA
Numerical simulations software project
•Software developments are managed in the context of an indepen-dent engineering project. The objective is to provide a core of software modules useable by the various simulations tools. This includes data handling, post-treatment, visualization, parallelization optimization,…
RAMSESA
SH
HE
RA
CL
ES
FARGO
CORE SOFTWAREMODULESI/O, Graphics,
Algorithms
Astronum2006 17D. Pomarède, CEA/DAPNIA
Data Handling
•A unique format has been chosen for the data produced by all the simulation codes : HDF5, the Hierarchical Data Format developed by the NCSA National Center for Supercomputing Applications.
•HDF5 is a general purpose library and file format for storing scientific data
•Two primary objects are handled : Datasets = multidimensional array of data elements Groups = structures
•Efficient storage and I/O : the library is tuned and adapted to read and write data efficiently on parallel computing systems
•Free, open source software, including utilities (browser)•Plenty of applications have interfaces to HDF5, in particular in the
graphics/analysis area : IDL, ParaView, MATLAB, …
Astronum2006 20D. Pomarède, CEA/DAPNIA
Visualization•Visualization is performed in the framework of IDL•Baseline is to use the object-oriented programming offered by IDL’s Object
Graphics•The visualization tool is a graphical widget
users act on the objects to modify their attributes through menus, droplist, sliders, buttons & dialog fields
•Input data : scalar and vector fields on regular grids and on AMR tree, particles•Once the data are loaded in memory, various implementations of the visualization
objects are proposed based on different classes of the IDL library : IDLgrPolygon : set of polygons and vertices, to hold an iso-surface IDLgrVolume : mapping from a 3D array of data to a 3D array of voxel colors projected
to two dimensions IDLgrPolyline : useful for hedgehog displays of vector fields IDLgrSurface, IDLgrImage, IDLgrPlot, IDLgrSymbol
•Basic graphics objects are inserted in Models that can be transformed (rotated, scaled and/or translated) : this forms the basis for spatial navigability around the 3D objects on display.
IDLgrLight objects represent sources of illumination : ambient, positional, directional, spotlight
•Two methods of rendering objects : hardware accelerator using OpenGL, highly efficient on a local machine software rendering applies when running on a distant computer
the graphical widget
menusslidersbuttonsdroplistdialog fields
surface object
axis object
plot objectimage object
navigation
profile
Select :densitypressurevelocityradiative energyradiative fluxparticlescustom variable
Visualization of regular grids and AMR data
Max grid :regular 3D 1200x1200x1200 = 1.728 109 cellsregular 2D 10000x10000AMR up to level 10 projected in grid 1024x1024x1024
image of 2D 10000x10000 HERACLES simulation
surface of a density slice on a restricted region of the RAMSES AMR (level 7 to level 11)
Visualization of iso-surfaces
Simulation of ISM hydrogen turbulencesdensity field - volume of size 15 pc
grid 12003
Interactive setting of the contour value on the density histogram
Visualization of 3D vector fields
RAMSES cosmological simulationbox of size 100 h-1 Mpc
HERACLES simulation of ISM turbulences
Hedgehog display of hydrodynamical velocity fieldBlue semi-transparent volume = iso-surface of the density
Visualization of particles
particle cloud display of a dark matter sampleyellow semi-transparent volume = iso-surface of the hydrodynami-cal density field
DM haloes
Astronum2006 27D. Pomarède, CEA/DAPNIA
Conclusions and outlooks
•An extensive program of simulations of astrophysical plasmas is pursued•New developments are managed in the framework of a software engineering
project•Solutions to current limitations will be studied :
– HERACLES : implementation of multiple grid algorithm to reach the effective 104 mesh resolution in 3D simulations
– RAMSES : algorithm optimization to improve load balancing– Visualization :
• optimization of memory management to access larger data sets• parallelization to improve speed (fastDL/mpiDL solutions from RSI)• implementation of multiple grid
•New projects :– Extend the use of the core software modules to FARGO and ASH– Database : development of a Virtual Observatory for cosmological
simulations (Horizon project)– Openings towards other domains :
• HERACLES application to laser fusion and plasma physics