v5-02-Release

Peter Hristov02/04/2012

Root & Geant3• Problems with v5-30-06

– It has to be patched for THn-related classes– Cannot “drain” the geometry created with v5-30-02 (AliEn -> local)

• Problems with v5-32-01– Issue with openssh on the build servers– The same problems as with v5-30-06

• Problems with v5-33-02– Crash in the geometry

• All these except the “draining issue” are solved in v5-33-02a• Geant3 v1-14 recreated to fix a duplication in the declaration of deuteron,

triton, He4– It contains also the fixes in Gheisha => no “Geant3/Fluka” correction should be

needed in the future– v5-01-Rev-29: rebuild with the latest Root v5-33-02a and Geant3 v1-14

Changes: v4-20-Rev-25a

• #92223: Modify AliGenPythia.cxx, .h to enable triggering with specific species within a given pt window. v4-20-Rev-25 should be used for reference

• It will be build using Root v5-28-00c and Geant3 v1-11-23-1 (the same as before, see the packages table)

Changes: v5-02-Rev-05• #92975 Please port to the next realese to be installed at P2 the trunk rev. 55338

(AliACORDEQAChecker&AliACORDEDataMakerRec)• #90625: Memory problem in AliTPCtrackerMI. From rev. 55270,55281,55346• #92820 Patch to fix the use of gMC in TOF. From rev. 55280• #93109: Port change in AliGRPPreprocessor.cxx to the release (rev. 55370)• #93117: VZERO OCDB: Update of the time delays for few channels. From rev.

55405• #93184: Request to port ZDC code to the release. From rev. 55432• #90436: Porting request: Misuse of TClonesArray containing AliESDMuonCluster.

From rev. 55434• #93101: Request to port to release TPC code - update of DQM. From rev. 55394• #93172: Porting request: Improve trigger chamber efficiency calculation. From rev

55449• Technical fix: avoid error messages in DA. From rev. 55450

Requests: v5-02-Release

• #93216 Request to port to Release: AliESDEvent.h

• #93218 port to Release AliT0PreprocessorOffline and AliT0CalibSeasonTimeShift

• #93224 port to Release AliT0Reconstructor.cxx with new pedestal subtraction

• #93257 EMCAL: Port updates in tender, QA and analysis to v5.02

Requests: OCDB

• #93007 Improve OCDB snapshot• #93153 Request to upload new TRD OCDB

object TRD/Calib/TrapConfig• #93276 Request to upload new TRD OCDB

object TRD/Calib/Krypton_2011-02 and TRD/Calib/Krypton_2011-03

• #93285 SPD Dead to RAW OCDB 2012

Requests: v5-01-Release

• #93161 PbPb simulation LHC11h (reconstruction part) - crash in AliITStrackerSA

Other reports

• #93013 Bias in MeanVertexSPD of LHC11a data• #93021 Online Reconstruction and Visualisation Design• #93171 Online Event display crashes on consecutive runs• #93203 problem in saving ESDMuonClusters in HLT tree• #93227 simulation crashes in AliTOFcalib because of run

number -1• #93264 Rule Checker gives wrong recommendations for

include vs. forward declaration• #93265 Patch to change sequence in AliReconstruction

Old slides

CPass0/CPass1/VPass/Pass1• #92847 Sequenced timeline of all steps in CPass0/CPass1/VPass• #92845 Implementation of the automatic production chain

CPass0/CPass1 + OCDB updates/snapshots• #92839 Validation of CPass0/CPass1 per detector with flags• #92837 Prepare configuration/Steering/makeOCDB macros

update/new code for CPass0/CPass1• #92836 Calibration macros update/new code for CPass0/CPass1• #92836 Calibration macros update/new code for CPass0/CPass1• #92835 Prepare reconstruction macros for CPass1 scenario b)• #92834 Implement trigger aliases in AliRoot, RAW data reconstruction• #92833 Implement trigger aliases in Shuttle/GRP• #91510: Reconstruction able to deal with different triggers

DQM• #92849 Implementation of AliZDCQAChecker for DQM required• #92848 Migration of TRD DQM shifter plots from custom AMORE

agent to QA framework• #92846 Implementation of AliSSDQAChecker for DQM required• #92844 Implementation of AliPMDQAChecker for DQM required• #92843 Implementation of AliPHOSQAChecker for DQM required• #92842 Logic for CPass0/CPass1 running and failures• #92841 Implementation of AliHMPIDQAChecker for DQM

required• #92840 Implementation of AliFMDQAChecker for DQM required• #92838 Implementation of AliTOFQAChecker for DQM required

Other reports (26/03)

• #92884 ZDC digitizer bug: spectators not added for AMPT generator

• #92794 root v5-33-02 and alien on Mac OS X Snow Leopard?

• #92765 Implement Trigger Aliases• #92723 request changes in STEER needed

for PID and track selection

OCDB

• #92456 TPC MC RecoParams for LHC11 pp simulation

Changes: v5-01-Rev-29

• #92604 Request to port AliTPCv*.cxx to the release. From rev. 55159

• #92528 port to Release and AN-tag patch to T0 part of AliTriggerSelection. From rev. 55154

• #92223: Modify AliGenPythia.cxx, .h to enable triggering with specific species within a given pt window. From rev. 55080,55252

• Technical fix from rev. 53259

Changes: v5-02-Rev-04• #92654 fix bug in AliAODPid copy constructor port to v5-02-

Release. From rev. 55214• #92650 request to port AliTOFTender from trunk to release. From

rev. 55209• #92604 Request to port AliTPCv*.cxx to the release. From rev.

55159• #92223: Modify AliGenPythia.cxx, .h to enable triggering with

specific species within a given pt window. From rev. 55080,55252• #92722: port to Release AliT0CalibTimeEq.cxx. From rev. 55238• #91699: Request on more detailed ITS cluster information in the

ESDs: Porting request for 55200

Changes: v5-02-Rev-04

• #92772: Port updates in CaloTrackCorr and EMCAL to release 5.02. From rev. 54897,54921,55071,55073,55082,55178,55179,55180,55181,55182,55183,55184,55185,55206,55210,55211,55220,55223,55224,55251

LHC11h PbPb data: plans• MC production: task #25489, #26646

– New tag from v5-01-Release– The updated macros are prepared by Andreas– Two reconstruction passes (with/without TPC bug)– A short test will be run– The full production will take ~2 months (to be reevaluated after the tests)– This MC should be OK also for pass3 (compatible reconstruction?)

• AOD re-filtering for pass2: task #26717– Should start this week with v5-02-Rev-04– Relatively long processing (2 weeks?)– Streaming of “interesting” raw event (selected by the task of Jacek): task #26736

• Pass3: no task yet– The memory consumption is an issue– Main improvements expected from TPC– Compatibility => changes only in v5-01-Release?– We have to start in a month from now– Detector readiness, etc. from PWGPP

Recent tasks• #27319 pass4 of LHC11a pp 2.76: detector readiness• #27312 Re-reconstruction of events with C0OM2 and CCUP2 triggers: validation• #27271 MC Production Request 6 UPC Samples for the central arm: validation• #27104 EMCAL Geant4 tuning to test beam and collision data: placeholder• #27084 MC production request for Lambda, K0Short analyses in p+p at 2.76

TeV: needs macros and JDL• #27076 30M pp events reconstructed without cleaning: needs test• #26886 MC Production Request 1M Pythia+cocktail events: validation• #26881 TPC Kr data processing in 2012: done?• #26878 Full processing of LHC11b/c minbias for calorimeters: start?• #26818 MC Pythia+HF+FlatMultiplicity, 10M anchored to LHC10e: AOD filtering,

needs v5-02-Rev-04• #26759 Embedding production of beauty on LHC10 PbPb: staging

v5-01-Rev-28

• Additional fix for #92162: Porting request - AliTPCtrackerMI.cxx from rev. 54843 + option to reintroduce the bug in AliTPCtrackerMU::AddCovariance/AddCovarianceAdd. It is needed for the LHC11h MC production.

• Changes for #92556: TRD request :: Porting updates of PWGPP/TRD to Release. From rev. 55133,55134,55138 + patch for AliTRDresolution.cxx

Changes: v5-02-Rev-03• #92466 port to Release AliT0AnalysisTaskQA. From rev. 55075• #92461 Requests for LHC11h re-filtering of AODs. From rev.

54999,55020 • #92393 request to port r55045 (tune of muon esd filter) to the

release• #92346 port to Release AliT0QAChecker. From rev. 54998• #92333 Porting codes to Release (DQM PMD). From rev. 55028• #92248 Porting into the release of rev. 55001• #92528: port to Release and AN-tag patch to T0 part of

AliTriggerSelection. From rev. 55154

Changes: v5-02-Rev-03• #92480: Classes for the new ZDC geometry in the release.

From rev. 55078• #92488: Porting request for 11h AOD production. From rev.

55083• #92495: Request to port TOF trunk rev. to Release (support to

LHC10c pass2). From rev. 55090• #92498: request to port AliTOFTender from trunk to release.

From rev. 54955,55048,55092• Technical fix from rev. 55057• #92556: TRD request :: Porting updates of PWGPP/TRD to

Release. From rev. 55133,55134,55138

Changes: v5-02-Rev-03

• #92479: Request to update ZDC initialization in macros/ConfigRaw2012.C. From rev. 55186,55188

• #92610: please port EMCal DQM checker update to release (to be deployed at P2). From rev. 55174

• #92625: please port EMCal DA update to release (to be deployed at P2). From rev. 55177

Visualization: v5-02-Rev-03

• #92237 AliACORDEQAChecker keeps adding "function" to the plot. From rev. 55016

• #92288 SDD QA produces a weird plot SDDRawDataCheck leading to a crash. From rev. 55097

• #92173 cannot use event display during cosmics run

Other reports (12/03/12)• #91510 Reconstruction able to deal with different triggers• #91960 wish documentation for cmake detector

algorithms• #92351 recover TOF tender calibration for pass2 on

LHC10c for analysis• #92331 preparing to move AliTOFHeader to AOD• #92250 Need to reset ObjectCount for TRef used for calo

in ESD ?• #92223 Modify AliGenPythia.cxx, .h to enable triggering

with specific species within a given pt window

v5-02-Rev-02• #88827: Request for porting updates to TOF QA task into release.

From rev. 54971• #91071: Running on AODs very slow on AAF. From rev.

54891,54920• #91977 Tracking cosmic muons in the TPC and momentum

calculation. From rev. 54984• #92135 Port TPC event filtering task to AliRoot release v5-02. From

rev. 54892,54901• #92162 Porting request - AliTPCtrackerMI.cxx. From rev. 54942• #92238: New raw-data event header format. From rev. 54983• #92242: Request of porting in the release. From rev.

54905,54906,54931,54949,54961,54981

Changes: v5-01-Rev-27• #91718 port AliTOFTenderSupply to release. From rev.

50837,50882,50990,51795,52834,52893,53029,53364,53431,53600,54541

• #91883 Request to port to release (5.01 and 5.02) the AliQAThresholds related classes. From rev. 50769,54709,54758

• #91889 porting CDB snapshot commits to v5-01-Release. From rev. 51502,53986,54066,54114,54283,54301,54366,54664,54704,54959

• #91931 port to Release AliT0QAChecker. From rev. 54595,54831• #91932 commit to trunk and port to Release AliAODTZERO. From rev.

54860• #91954 VZERO: Changes related to DQM/QA checker. From rev. 54845

Changes: v5-01-Rev-27

• #91962 Porting into the release of TOF/AliTOFTrigger update. From rev. 54814

• #91977 Tracking cosmic muons in the TPC and momentum calculation. From rev. 54984

• #92162 Porting request - AliTPCtrackerMI.cxx. From rev. 54942

• #92238: New raw-data event header format. From rev. 54983

• #88827: Request for porting updates to TOF QA task into release. From rev. 54971

v5-02-Rev-01

• Corresponds to the trunk rev. 54887• Works with Root v5-32-01 and Geant3 v1-12• All tests are OK• No significant difference in the memory

consumption with respect to v5-01-Release• Known issues

– Creation of AliMDC RPM with shared libraries. Ongoing

– Merging of raw tags. Under investigation

v5-02-Release• Modifications from rev. 54391 to 54845 tested

– main tests are OK, no need for manual selection– this makes the new release suitable for AOD filtering with

tenders– all porting requests will be solved “in one go”

• Issues– Creation of AliMDC RPM with shared libraries. In progress– Creation of PAR files.– Merging of raw tags. Under investigation, not yet reproduced

• Next tag: tomorrow, 28/02– Moving to the usual Savannah porting requests after it

Changes: v5-01-Rev-26

• Adding protection against not initialized corrected ZDCTDC values from reconstruction. From rev. 54116

• #91648 To be ported to the release: Fix needed for simulation of strange baryons in HIJING+Signals. From rev. 54646

• #91695 Request to port AliQAThresholds.h to the Release. From rev. 54254

• #91322 Request to port bugfix in the AliPHOSReconstructor.cxx to the v5-01-Release. From rev. 54496

Requests: v5-02-Release

• #91482 EMCAL: Port fix for new 2 SM 1/3 to release 5.02

• #91465 Moved calibration object for ESD initialization• #91421 Request to port PHOS trigger QA task to the

Release• #91322 Request to port bugfix in the

AliPHOSReconstructor.cxx to the v5-02-Release• #91253 Request to port bugfix in the

AliAnalysisTaskPHOSPbPbQA.cxx to the v5-02-Release• #91071 Running on AODs very slow on AAF

Other reports

• #91699 Request on more detailed ITS cluster information in the ESDs

• #91685 ITS QA porting request• #91592 ZDC: new geometry to be implemented• #91512 Trigger aliases for reconstruction• #91510 Reconstruction able to deal with different

triggers• #91500 stored TPC only information in AOD049 -

LHC10h

v5-02-Release

• Branch v5-02-Release is created on 03/02• All tests OK (Root v5-30-00-patches)• Tests with Root v5-32-00-patches: ongoing• Several modifications to be ported• Issues

– Creation of AliMDC RPM with shared libraries– Creation of PAR files– Merging of raw tags

Changes: v5-01-Rev-24

• #24646 Re-produce AODs for cascades in pass2 PbPb 2010 (data & MC). Change in the configuration of vertexingHF from rev. 54533,54561

• #90456: Request for modification of AliSimulation::ConvertRaw2Sdigits to select events for embedding. From rev. 54508

• Technical fixes for Root 5-32. From rev. 51946• Technical fixes for Coverity 11390, 11389, 11388.

From rev. 54588

v5-01-Rev-25

• Technical fix from rev. 50674: Patch for recent TRefArray->TObjArray usage. Needed for the MC AOD filtering

v5-01-Rev-23

• #91061: AOD production very slow. From rev. 54460

Changes: v5-01-Rev-22• #88827: Request for porting updates to TOF QA task into

release. From rev. 54242• #90320: Request to port additional consistency checks in HLT

TPC cluster decoding to v5-01-Release. From rev. 54055,54056• #90546: Filtering crashes when processing 2010 MC data.

From rev. 54115• #90738 Request to port a fix to the release in AliZDCDigitizer.

From rev. 54035• #90749 ESD Porting Request: GetTPCClusterInfo with

additional switch. From rev. 54081• #90812 Porting overlap fixes to the release. From rev. 54092

Changes: v5-01-Rev-22• #90817 Please commit PHOS trigger part in the

AliAnalysisTaskESDfilter.cxx. From rev. 54439• #90870 Request to port ZDC code to the release. From rev.

54133• #90916 Request: porting to v5-01-Release of the new ESD-

>AOD filter. From rev. 52540,53853,54188,54210• #91005 Fix in AliCTPRawStream.cxx. From rev. 54234• #91126: Request to commit a patch for AMPT. From rev. 54424• #91159: Vertex generation in AliGenCorrHF. From rev. 54417• #91030: 2.76 TeV LHC11a pass3 dca problem. From rev. 54304

Other reports

• #90615 Problems in the material budget, eta<0.9 and 0.9<eta<1.4

• #90944 adding alignment objects for the additional supermodules in the geometry setup 2012

• #90939 Request to include (anti)hypertriton in ALICE GEANT3

Requests/Additional fixes

• #90625 Memory problem in AliTPCtrackerMI• #90622 Logic flaw in AliTPCseed. From rev.

53997• #90616 Worrying message from TPC

reconstruction. From rev. 54237• Changes in RAW (TClonesArray usage)

v5-02-Release

• Coverity: 129 defects to be fixed• AliRoot tests: mostly OK• Root v5-32-00-patches: needs tests• PWGs transition: PWG0 and PWG2 still

ongoing• One library per subdirectory: not yet ready• Savannah bug reports: many old bugs are still

open

GDB on Grid

• Some potential problems detected and fixed (ITS, TPC, HLT)

• Some jobs fail in the beginning (event 0-10), ~4%– Not reproducible locally, even if we run many

reconstruction jobs in parallel– Always caused by std_badalloc in different places

• Other jobs are killed by the system (memory) ~20%

Changes: v5-01-Rev-21• #90324: Exception in

AliITStrackerMI::FollowProlongationTree. From rev. 53978• #90549: Request to port r53948 to the release (MUON

small leak fix)• #90658: For v5-01: Option to isolate heavy flavor part of a

Pythia event. From rev. 53959• #84578: Request to extend AliGenBox for using Yrange.

From rev. 53996• Optional RB/PX 24 shielding and scoring. From rev.

53955,53956

Changes: v5-01-Rev-21

• #90461: Request to port a new feature for ZDC to the release. From rev. 53705

• #90504: EVE muon_init.C update r53875• #25142: Commit and porting to Release of the

new ESD->AOD filter. From rev. 54021• #90540: Port 53910,53911 and 53912 to the

Release (Full MC Header in the AOD)

Changes: OCDB

• #90756 Request to port object in RAW OCDB (for realistic MUON simulations)

• #90736 Calibration of the TRD cosmics of May,Jun and August

Reconstruction of RAW (LHC11h)

• Back trace problem solved• Clean-up of the PATH and LD_LIBRARY_PATH

on the GRID• Clean-up of the AliEn libraries• Deterministic splitting of the failed jobs (in

preparation)• New tests in parallel with the Grid production

Changes: v5-01-Rev-20• #90319: Segmentation violation in

AliPHOSRawFitterv1::~AliPHOSRawFitterv1. From rev. 53869• #90053: Request: Port bug fix TRD calibration code to release. From rev.

53734• #90292: Add line ConvertZDC() in

AliAnalysisTaskESDfilter::ConvertESDtoAOD(). From rev. 53895• #90307: ZDC QA update. From rev. 52738,53081,53271• #90309: ZDC request to port code to the release. From rev. 52616• #90024: port changes in PYTHIA6 for pyquen production (pyquen-

1.5.F,CMakelib6.4.21.pkg updated), rev.53645• #90359: Request: fix cached values in ESD. From rev. 53900• #90013: Vertexing task crashing in trunk. From rev. 53793• Additional protection. From rev. 53904

LHC11h Pass2 – reconstruction details• Use v5-01-Rev-19 in the production• Start in inverse time order (last runs first, “LIFO”): OK• Use MB trigger for CPass0: OK• Exercise the full production setup on runs from “grey area”:

special “gdb” production, run 170593: OK• Run with TPC pools: OK• Work on a local raw file: OK• Use OCDB snapshot: OK• Keep only the rec. points for the current event: OK• Switch off QA: OK• Switch off MUON, if the memory consumption is still too

high

48

49

Results• CPass0: 185 jobs, 523,509 out of 539,890 raw files successfully reconstructed => 97% efficiency•All runs with mag.field configuration (+ +) ready (170593-169628)

• Details on losses follow

• Pass2 current status: 131 jobs, 225,568 out of 362,790 files successfully reconstructed => 62.2% efficiency

50

Losses – Pass2• G_exception – average 6.5%

1698

3516

9838

1698

5516

9859

1699

1916

9922

1699

2416

9961

1699

6916

9981

1700

3617

0040

1700

8317

0085

1700

8917

0152

1701

5917

0193

1702

0317

0205

1702

0817

0230

1702

6817

0270

1703

0817

0311

1703

1317

0387

1703

8917

0546

1705

93

0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%

35.00%G_exception (%)

Strong run dependency

51

Losses – Pass2 (2)• Memory overrun – average 16.8%

1698

3516

9838

1698

5516

9859

1699

1916

9922

1699

2416

9961

1699

6916

9981

1700

3617

0040

1700

8317

0085

1700

8917

0152

1701

5917

0193

1702

0317

0205

1702

0817

0230

1702

6817

0270

1703

0817

0311

1703

1317

0387

1703

8917

0546

1705

93

0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%

35.00%

40.00%

Memory overrun (%)

Strong run dependencyFunction of number of events/chunk and data taking configuration

52

Losses• G_exception

• Debugging hard as there is no traceback• Seems to be random (from syswatch.log)• Irreproducible in local tests• No related issues shown by Valgrind• Appears in the first events of the chunks• Working with ROOT experts, at least to get the

exception in the logs => special “gdb” run• Memory overrun

• Additional profiling ongoing• All external sources are out – gain only possible

through changes in reconstruction

Special “gdb” run

• “catch throw” mode• Several problems discovered, to be submitted

to Savannah. Most probably uninitialized memory is used as index in an array– TClonesArray new with placement, where the

index come from GetEntriesFast– corrupted (?) raw data– deletion of arrays

Plans

• Continue the investigation of G__exception on the GRID

• Understand the difference between CPass0 and Pass2 (MB trigger, V0s, cascades?)

• Try to reproduce completely the GRID execution flow on a local machine

• Resubmit the failed jobs in “split” mode

v5-02-Release

• Complete the transition of the analysis code to the new modules

• Move every library to a sub-directory and get rid of *.pkg (native CMake)

• Fix the Coverity defects and compilation warnings• Solve as much as possible Savannah issues• Create the branch at the end of January• First stable tag in February