Cosmic Microwave Background Data Analysis At NERSC
description
Transcript of Cosmic Microwave Background Data Analysis At NERSC
Cosmic Microwave Background Data Analysis At NERSC
Julian Borrillwith
Christopher CantalupoTheodore Kisner
What Is The CMB ?
A snapshot of the Universe when it first became neutral 400,000 years after the Big Bang.
Cosmic - filling all of space.
Microwave - redshifted by the expansion of the Universe from 3000K to 3K.
Background - primordial photons coming from “behind” all astrophysical sources.
Why Do We Care About The CMB ?
The CMB is a unique probe of the very early Universe.Its tiny (1:105-8) fluctuations carry information about - the fundamental parameters of cosmology - ultra-high energy physics beyond the Standard
Model
What Does The CMB Look Like ?
CMB Work At NERSC
• Started in 1997:– 2 separate allocations for Maxima & Boomerang– together 5 users & 30,000 CPU-hours
• Developed into premier world center for CMB analysis:– single allocation shared by O(10) experiments– O(100) users & O(1,000,000) MPP-hrs/year
• Now includes "Big Science" satellite mission– split into two allocations
• mp107 - 13 sub-orbital experiments 40 users & 500,000 MPP-hrs
• planck: Planck satellite 60 users & 2,000,000 MPP-hrs*
The Planck Satellite
•The primary driver for current NERSC CMB work.
•A joint ESA/NASA mission due to launch in the fall of 2008.
•An 18+ month all-sky survey at 9 microwave frequencies from 30 to 857 GHz.
•O(1012) observations, O(108) sky pixels,O(104) spectral multipoles.
Data Management
• Dominated by time-ordered data– O(1-10) TB, O(10,000- 100,000) files
• Each data set must be analyzed as a whole.• Each data analysis needs O(100x) storage.• Each data set may have its own format/distribution.• Each data set must be selectively shared.
• Requires– Pre-fetching & active disk quota management– Efficient & abstracted run-time reading– Project account
Task Management
• Any member of a team must be able to– Access all the data– Access all the general & project-specific codes– Generate and execute standard analyses– Share the results with the team
• Some members of a team must be able to– Control the overall team work-load/-distribution– Manage software versioning and access
• Requires– Project account with individual user certification– Limited capability for most; full capability for
some.– Synchronized data-for-task management
A Framework for CMB Analysis At NERSC
Data Management
Data staging
Run-time IO
Memory
Task Management
User
Project
Critical Components & Issues
• NERSC Global Filesystem– access from Franklin
• Storage Resource Manager– optimal transfer protocols
• Project quotas– separation from UNIX groups
• Project accounts– appropriate queue limits
• User accounts– maintain (unique) accessibility
• Modules– work just fine
Conclusions
• NERSC has developed into the world's leading center for HPC for CMB data analysis– Recognized as such by the recent NASA/NSF/DOE
Weiss report on the future of CMB research.
• This reflects the NERSC resources'– capacity and capability,– accessibility,– long-range development plan.
• Long may it continue !