Computing in IceCubeGeorges Kohnen, Université de Mons-Hainaut, Belgium

for the IceCube Collaborationgeorges.kohnen@umh.ac.be

The IceCube neutrino telescopeThe Amundsen-Scott South Pole Station (Antarctica) is home to IceCube, the world’s largest high-energy neutrino telescope, and its smaller predecessor AMANDA (Antarctic Muon And Neutrino Detector Array). They are sensitive to the Cherenkov light emitted by high-energy charged leptons and cascades resulting from the interaction of neutrinos. The main goal of the detectors is the search for high-energy extraterrestrial neutrinos, which may reveal the origin of cosmic rays and offer insight into the most energetic phenomena in the universe.Upon completion in 2011, IceCube will instrument a volume of one km3, and will consist of up to 4800 DOMs (Digital Optical Modules) arranged in a hexagonal pattern of 80 strings in the ice, at depths between 1450 m and 2450 m below the surface at the South Pole. The ice layer above the detector efficiently shields it from low-energy atmospheric muons, the main background. Individual DOMs are vertically spaced by 17 m, and the interstring distance is ~125 m.AMANDA was completed in 2000 and consists of 677 optical modules on 19 strings in a denser arrangement, contained in a cylinder of radius 100 m. It complements IceCube at lower energies (~a few 10’s of GeV).IceCube also includes a surface component, the IceTop air shower array. It can perform cosmic ray and air shower physics up to 1018 eV, but can also be used in conjunction with the in-ice DOMs for composition studies of high-energy showers, vetoes or calibration on high-energy shower background. Each of its planned 80 stations will be made up from two tanks, each containing 90 cm of frozen water and instrumented with two DOMs.After the 2006/2007 austral summer, IceCube now consists of 22 operational strings and 26 surface stations, resulting in a total of 1424 DOMs. The AMANDA detector, fully contained within its larger successor, has been integrated into IceCube operations, with combined triggering, common event building and a unified data analysis strategy.IceCube is an international collaboration, led by the University of Wisconsin-Madison, with collaborating institutions in the United States, Germany, Belgium, Sweden, Switzerland, the United Kingdom, Japan and New Zealand.

09/2007http://www.icecube.wisc.edu

OutlineThe IceCube neutrino observatory is a cubic kilometer Cherenkov detector currently under construction in the deep ice at the geographic South Pole. As of 2007, it has reached more than 25 % of its final instrumented volume and is actively taking data. We will briefly describe the design and current status, as well as the physics goals of the detector. The main focus will, however, be on the unique computing structure of the experiment, due to the remoteness of the detector and the limited access to satellites for data transfer. We will also outline the structure of the software framework used for filtering, reconstruction and simulation.

International Conference on Computing in High Energy and Nuclear Physics 2007 – Victoria, Canada

Data acquisition, filtering, transfer and storageEach DOM contains a 10-inch Hamamatsu photomultiplier tube and supporting hardware housed inside a glass pressure sphere. The firmware can be reprogrammed from the surface, and the DOMs are synchronized to a GPS clock to achieve a timing precision of the order of two nanoseconds. Two digitizers capture the waveform signal received from the photomultiplier tube. Moreover, calibration runs for neighboring DOMs can be taken using the integrated LEDs. The DOMs capture, digitize, timestamp and buffer signals independently.

Cables from all strings converge into the IceCube Laboratory (ICL), a two-story elevated building at the surface, in the middle of the array. It was commissioned in January 2007 and houses all the electronics needed for data-taking, archiving, filtering and data reduction. Three winterover scientists maintain the systems during the austral winter (from February to October).

The data acquisition software (written in Java) collects the digitized waveforms from all hit DOMs, triggers and builds events, without any detector deadtime. This system, totally redesigned over the course of the last year, now provides near to 100 % uptime running all three sub-detectors together: InIce, IceTop and AMANDA.

Due to the limited bandwidth for transfer over the NASA’s Tracking and Data Relay Satellite System (TDRSS, ~25 GB/day), the data needs to be processed and filtered on-line. Only level 1 (triggered and filtered) physics data is sent to the northern hemisphere. Additionally, all raw data is being written to tapes.

A central data warehouse using SAN storage systems is maintained at the University of Wisconsin-Madison (UW), housing experimental and simulated data. Two data processing clusters with a total of ~1500 CPUs are available for processing. More computing power is spread throughout the collaboration. UW also houses a replica of the South Pole Computing Systems, that is used for testing software before it is deployed on site.Data, Software and the IceTray frameworkIceCube analysis software (written in C++) is a suite of software that provides a uniform environment for simulation, reconstruction, analysis and filtering. It includes the modular IceTray framework (control of flow), the data storage classes (in-memory representation of the data) as well as various modules and services (such as data I/O, logging and database access). The framework allows branching a chain of modules to generate separate outputs.

The data itself is contained in a ”frame” that represents the state of the detector at the time of an event. It contains data from four streams, one for each type of data: Geometry, Calibration, Status and Physics (events).

The software is distributed by a central version tracking system (SVN) and organized into meta-projects for reconstruction, simulation, filtering, etc. Auxiliary software packages, such as Root and MySQL, are provided by a tools distribution. Code documentation uses the Doxygen system, and scripting is done using the Python language.

All non-physics data is stored in a time-based MySQL database, that provides a current snapshot of the state of the detector at any given time.

IceCube software is used in online processing and filtering at South Pole, for reconstruction and analysis, as well as simulation production in cluster environments.

The IceCube Laboratory at South Pole

A simulated and reconstructed muon eventin the 80-string detector