CC-IN2P3 : Status & perspectives
Transcript of CC-IN2P3 : Status & perspectives
Centre de Calcul de l’Institut National de Physique Nucléaire et de Physique des Particules
CC-IN2P3 : Status & perspectivesJFPPL Computing WS - March 2015
2015-03-10FJPLL Computing workshop
Organization
2
2015-03-10FJPLL Computing workshop 3
} Steering committee
} + 1 internal projects evaluation committee
} Goals : • list every transversal IT projects • ensure they have the necessary
ressources • make some recommandations, if
any, to the Direction } ~1 expert of each IT teams } First results
• 15 projects evaluated • good feedback from project leaders • a better knowledge
Organization
2015-03-10FJPLL Computing workshop 4
} 3 permanent positions opened in 2014 storage, system administration, administration
} Temporary contracts mainly funded on EU project or various agreements
} Globally : less WFTE (because of some leaves : retirements…)
} Regulation constraints : } strong turnover… } less candidates…
} 2015 forecast : } +1 permanent position (DBA) } 7 contracts end : some can be renewed
Staff
Worked FTE
0,0
20,0
40,0
60,0
80,0
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
Permanent Non-Permanent
78 people : 30 % temporary contracts 1 « chargé de recherche » et 77 technicians and engineers (ITA) :
61 BAP E : IT engineers 8 BAP J : administration 5 BAP G : facility management 3 BAP F : communication / mediation
2015-03-10FJPLL Computing workshop
Budget
5
2015-03-10FJPLL Computing workshop 6
} From 2011 to 2013 : more or less constant • no capacity growth, only renewal
} In 2014 : + 550 k€ • a 900 k€ increase coming from the CNRS allocation (TGIR/IR) • but lower funding coming from CEA = -200 k€
} Other incomes (ANR, european projects…) : ~ 383 k€
Resources : budget evolution
Bud
get
en k€
0
3000
6000
9000
12000
2005 2006 2007 2008 2009 2010 2011 2012 2013 2014
900110011001100
1364
13641364
1300
17001600
18501745179917002000
545050004587
20001350
4050370536563500422337503750375037503400
TGIR non LHCTGIR LCGCEA
Strong investments corresponding with the LHC startup : impact on capacity maintaining 3 to 5years later
non LCG budget more or less stable :
but running costs in increase (electricity: x3,3 in 8 years)
less investments for others
2015-03-10FJPLL Computing workshop 7
Budget 2014 : by destination
IN2P3 Labs4 %
Runnning costs4 %
HR5 %
FM26 %
Non LCG36 %
LCG (except op.)25 %
Running costs4 %
HR5 % FM
26 %
IT @ IN2P3 labs13 %
IT @ CC51 %
Maintenance21 %
Services11 %
Network17 %
Storage38 %CPU
12 %
Tertiary5 %Infra
20 %
Maintenance29 %
Fluids46 %
=
Half of the budget is dedicated to IT investments at CC-IN2P3
A quarter of the budget goes into electricity
TertiaryRunning costs
HRFluids
Infrastructure
IT increase
Maintening capacity
2015-03-10FJPLL Computing workshop
LHC Computing Grid
8
FJPLL Computing workshop 2015-03-10 9
WLCG and LCG-France
LCG-France :
MoU between IN2P3 and the Tiers sites for 2013-2016 : to keep the french contribution to WLCG at least at ~10% to keep the 100% of the T1 and its growth (~10 to 20%) maintain the T2-T3s by funding them to up 70% of the needed money
With annual budget forecast : ~1 800 €
FJPLL Computing workshop 2015-03-10 10
CC-IN2P3 within WLCG
107 kHS06 ~18 PB~10 PB
2015 T1 Pledged CPU Capcacity
22 %
11 %
2 %4 %
9 % 3 %4 %6 %
12 %
12 %
10 %4 %
Canada France Germany Italy Netherlands Nordic Korea Russia Spain Taïwan UK USA
2015 T1 Pledged Disk Capacity
20 %
11 %
3 %4 %
9 %1 %5 % 6 %
14 %
14 %
10 %4 %
2015 T1 Pledged Tape Capacity
25 %
12 %
2 %5 %
7 %1 %2 %5 %
15 %
12 %
10 %4 %
0
20,000
40,000
60,000
80,000
100,000
120,000
0
2000
4000
6000
8000
10000
12000
2006 2007 2008 2009 2010 2011 2012 2013 2014
TB
HEP
-SPE
C06
Evolution Ressources LCG-France Sites T2-T3 hors CC-IN2P3
Disk [TB] CPU [HEP-SPEC06]
% of 2014 pledges CPU Disk Tapes
French T1 / ∑ T1 10,15 10,29 10,11
French T1+T2s / ∑ T1+T2s
9,61 10,3 10,11
FJPLL Computing workshop 2015-03-10 11
Network connectivity for WLCG and LCG-France
2013 Renater report status:
LCG is 46 % of total french trafic
2015-03-10FJPLL Computing workshop
Capacities and usages
12
FJPLL Computing workshop 2015-03-10 13
CPU capacity
2011-13 : core number is constant
2014 : 20 000 cores for a global power of 208 866 HEPSpec06 2011 2013
Available power is growing :
- newer servers bought to maintain capacity - OS is more efficient
FJPLL Computing workshop 2015-03-10
Disk storage
Standard performance disk > 12 PB High performance disk (for GPFS) = 1,6 PB
Tapes
Storage used on magnetic tape : 25 PB out of 340 PB nominal capacity
Backup (TSM)
Stored volume : about 5,5 Po
14
Storage capacities and global consumption
2015 Allocation
CPU (HS06.h) 1 947 949 840
MSS (To) 24 957
dCache (To) 8 586
iRods (To) 524
GPFS (To) 1 673
xRootd (To) 2 545
FJPLL Computing workshop 2015-03-10 15
Users : distribution and CPU comsumption (Nov. 2014)
21,5 %0,3 %
6,4 %
0,4 %1,7 %10,4 %
59,3 %
Particle PhysicsHadron PhysicsTheoryLabsIN2P3 Multidisciplinary & openingsNuclear PhysicsAstroparticles + Neutrino
0,5 %
LHCb21,4 %
D05,7 %
CMS22,4 %
ATLAS50,0 %
CTA13 %
dchooz6 %
Virgo1 %
AMS13 %
km3net2 %
GLAST16 %
CRAL2,1 %Planck
12,7 %HESS15,3 %Auger3,3 %Antares15,1 %
Top 10 Astroparticle + Neutrino sharing
Top HEP
2015-03-10FJPLL Computing workshop
Perspectives
16
FJPLL Computing workshop 2015-03-10 17
Electricity cost…
0200400600800
100012001400
012345678
2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014
k€ HT Cons. (en 10 GWh) kWh price (€ cents/kWh)
1000
1350
1700
janv. 12févr. 12m
ars 12avr. 12m
ai 12juin 12juil. 12août 12sept. 12oct. 12nov. 12déc. 12janv. 13févr. 13m
ars 13avr. 13m
ai 13juin 13juil. 13août 13sept. 13oct. 13nov. 13déc. 13jan. 14fév. 14m
ars 14m
ai 14juin 14juil. 14août 14sept. 14oct. 14
Average consumed power (kW)
Average consumed power by month is decreasing :
- hardware renewal- energetic optimisation (setpoints
tuning)- VIL 2 : best PUE
But the global bill is more or less constant :
- kWh price increases- threshold effects of the EDF contracts
Estimation
FJPLL Computing workshop 2015-03-10 18
Computing rooms : VIL2
Base infrastructure designed to fit the final configuration (wires, pipes…)
Everything else is modular (transformers, UPS, chillers, etc.) and can be installed later as a function of the needs
Best PUE : ~ 1.47 Capacity : 80 racks
28 installed - 52 left
1 rack = 730 TB or 20,5 kHS06 (2014)
2*250 m2 to use
Investment for 80 racks : from 1.5 to 2 M€ ( Tier II) :
wires, pipes, transformers, chillers without racks and servers
FJPLL Computing workshop 2015-03-10
LSST (Large Synoptic Survey Telescope) : - reprocessing of half of the data- CC will host all the processed data
EUCLID : CC-IN2P3 will be one of the 8 “Sciences Data Centers” of this European space mission and should provide 30% of the resources (CPU and storage)
CTA : CC-IN2P3 may be led to operate for it, also by 2030, at least a 25% of 88 kHS06 for CPU, 207 PB for disk and 507 PB for tapes.
LHC : ~+25% per year indicates that at the end of the LHC Run 3 a T1 capacity of
19
Estimated needs of future (and on-going) experiments
En 2024 CPU kHS06
DisquePo
MSSPo
LHC 1 000 80 150
x2014 5 6 6
In 2030 CPUkHS06
DiskPB
MSSPB
LSST 2 400 100 266
EUCLID 67 150 52
CTA 22 52 127
∑ 2 489 302 445
x2014 12 22 18
Cumulated capacities by year for the 4 experiments
PB
0,00
200,00
400,00
600,00
800,00
kHS0
6
0
1 250
2 500
3 750
5 000
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
CPU kHS06 Disk PB Tapes PB
MOA Signed !
FTP Submitted
Not only a infrastructure challenge. Stronger collaboration with experiments, with similar T1s, new softwares, interfaces between data scientists & computer scientists …
FJPLL Computing workshop 2015-03-10
} IN2P3 computing budget 2015 is constant over 2014 : 6 200 k€ • it should permit to face the experiments needs (as far as we know them today) and to
continue infrastructure upgrades
} New projects begin } Indigo-Datacloud, EGI-Engage, EU-T0, CPER
} Human resources
• 2015 should be « calm » year : not much leaves expected (except projects) but some concern for the future
• 2016 : new cycle of retirements, end of the on-going CEA agreement • 2016+ : very strong concerns about CNRS budget
Many things to do …
20
2015 Forecast