T.Strizh (LIT, JINR) 1 Распределенная грид-инфраструктура для...

T.Strizh (LIT, JINR)T.Strizh (LIT, JINR)

11

Распределенная грид-инфраструктура для обработки и анализа данных с Большого адронного коллайдера

Кореньков В.В. (ЛИТ ОИЯИ)Кореньков В.В. (ЛИТ ОИЯИ)

ДубнаДубна, , 0707..1100..20112011


Some historySome history 2001-2003 - EU DataGrid project2001-2003 - EU DataGrid project

• middleware & testbed for an operational gridmiddleware & testbed for an operational grid 2002-2005 – LHC Computing Grid – LCG2002-2005 – LHC Computing Grid – LCG

• deploying the results of DataGrid to provide adeploying the results of DataGrid to provide aproduction facility for LHC experiments production facility for LHC experiments

2004-2006 – EU EGEE project phase 12004-2006 – EU EGEE project phase 1• starts from the LCG gridstarts from the LCG grid• shared production infrastructureshared production infrastructure• expanding to other communities and sciencesexpanding to other communities and sciences

2006-2008 – EU EGEE-II 2006-2008 – EU EGEE-II • Building on phase 1Building on phase 1• Expanding applications and communitiesExpanding applications and communities … …

2008-2010 – EU EGEE-III2008-2010 – EU EGEE-III 2010-2012 - 2010-2012 - EGI-InSPIREEGI-InSPIRE

CERN


350 sites55 countries150,000 CPUs150 PetaBytes>15,000 users>300 VOs>1 mln jobs/day

ArcheologyAstronomyAstrophysicsCivil ProtectionComp. ChemistryEarth SciencesFinanceFusionGeophysicsHigh Energy PhysicsLife SciencesMultimediaMaterial Sciences…


Tier 0 at CERN: Acquisition,Tier 0 at CERN: Acquisition, First pass reconstruction, First pass reconstruction,

Storage & Distribution Storage & Distribution

[email protected]

1.25 GB/sec (ions)

4


Tier 0 – Tier 1 – Tier 2Tier 0 – Tier 1 – Tier 2

[email protected] 5

Tier-0 (CERN):•Data recording•Initial data reconstruction

•Data distribution

Tier-1 (11 centres):•Permanent storage•Re-processing•Analysis

Tier-2 (>200 centres):• Simulation• End-user analysis

T.Strizh (LIT, JINR)T.Strizh (LIT, JINR)66


Russian Data Intensive Grid infrastructure (RDIG)

RDIG Resource Centres:– ITEP– JINR-LCG2 (Dubna)– RRC-KI– RU-Moscow-KIAM– RU-Phys-SPbSU– RU-Protvino-IHEP– RU-SPbSU– Ru-Troitsk-INR– ru-IMPB-LCG2– ru-Moscow-FIAN– ru-Moscow-MEPHI– ru-PNPI-LCG2 (Gatchina)– ru-Moscow-SINP- Kharkov-KIPT (UA)- BY-NCPHEP (Minsk) - UA-KNU

The Russian consortium RDIG (Russian Data Intensive Grid), was set up in September 2003 as a national federation in the EGEE project. Now the RDIG infrastructure comprises 17 Resource Centers with > 10000 kSI2K CPU and > 3500 TB of disc storage.


- support of basic grid-services;- support of basic grid-services; - Support of Regional Operations Center (ROC);- Support of Regional Operations Center (ROC); - Support of Resource Centers (RC) in Russia;- Support of Resource Centers (RC) in Russia; - RDIG Certification Authority;- RDIG Certification Authority; - RDIG Monitoring and Accounting;- RDIG Monitoring and Accounting; - participation in integration, testing, certification of grid-- participation in integration, testing, certification of grid-

software; software; - support of Users, Virtual Organization (VO) and application;- support of Users, Virtual Organization (VO) and application; - User & Administrator training and education;- User & Administrator training and education; - Dissemination, outreach and Communication grid activities.- Dissemination, outreach and Communication grid activities.

The main directions in development and maintenance of RDIG e-infrastructure are as

the following:


The tasks of the Russia & JINR in the WLCGThe tasks of the Russia & JINR in the WLCG ((20201111 years)years): : • Task 1. MW (gLite) testing (supervisor Task 1. MW (gLite) testing (supervisor O. KeebleO. Keeble) ) • Task 2. LCG vs Experiments (supervisor I. Bird)Task 2. LCG vs Experiments (supervisor I. Bird)• Task 3. LCG monitoring (supervisor J. Andreeva)Task 3. LCG monitoring (supervisor J. Andreeva)• Task 4. Tier3 monitoring (supervisor J. Andreeva, A.Klementov)Task 4. Tier3 monitoring (supervisor J. Andreeva, A.Klementov)• Task 5/6. Task 5/6. Genser/ Genser/ MCDBMCDB ( supervisor ( supervisor W. PokorskiW. Pokorski))

Worldwide LHC Computing Grid Project (WLCG)

The protocol between CERN, Russia and JINR on participation in LCG Project was approved in 2003. MoU on Worldwide LHC Computing Grid (WLCG) signed by Russia and JINR in October, 2007

CICC comprises CICC comprises 1584 Core1584 Core

Total performance Total performance ~4100~4100 kSI2K kSI2K

Disk storage capacity Disk storage capacity 1068 TB 1068 TB

2011 2012-2013

CPU (kSI2k) 4500 7000

Disk systems (TB)

1500 25000

500

1000

1500

2000

2500

3000

2003 2004 2005 2006 2007 2008 2009 2010

Perfo

rman

ce/C

apac

ity (k

SI2K

/TB)

Estimates of the JINR CICC resources Estimates of the JINR CICC resources increase in the futureincrease in the future

Performance

Disk storage capacityDisk storage capacity

Growth of the JINR CICC

resources in 2003 - 2010

~ 300 CPU and ~320 TB disk storage will be added

during the next few month

JINR Central Information and Computing JINR Central Information and Computing Complex (CICC)Complex (CICC)

Availability and Reliability = 99%


Integration with Google EarthIntegration with Google Earth

USER- INTERFACE AND VISUALIZATION USER- INTERFACE AND VISUALIZATION SERVICE DEVELOPMENT FOR VIRTUAL SERVICE DEVELOPMENT FOR VIRTUAL ORGANIZATION SUPPORT IN HIGH ENERGY ORGANIZATION SUPPORT IN HIGH ENERGY PHYSICSPHYSICS


Tier 3 sites monitoring projectTier 3 sites monitoring project Tier-3 sites consist of resources mostly dedicated for the data analysis by the Tier-3 sites consist of resources mostly dedicated for the data analysis by the

geographically close or local scientific groups. Set of Tier 3 sites can be joined to geographically close or local scientific groups. Set of Tier 3 sites can be joined to federation.federation.

Many Institutes and National Communities built (or have plans to build) Tier-3 Many Institutes and National Communities built (or have plans to build) Tier-3 facilities. Tier-3 sites comprise a range of architectures and many do not possess facilities. Tier-3 sites comprise a range of architectures and many do not possess Grid middleware, which would render application of Grid monitoring systems Grid middleware, which would render application of Grid monitoring systems useless.useless.

Joined effort of ATLAS, JINR and CERN IT (ES group)Joined effort of ATLAS, JINR and CERN IT (ES group) Objectives for Tier3 monitoringObjectives for Tier3 monitoring

• Monitoring of Tier 3 site.Monitoring of Tier 3 site.• Monitoring of Tier 3 sites federation.Monitoring of Tier 3 sites federation.

Monitoring of Tier 3 siteMonitoring of Tier 3 site• Detailed monitoring of the local fabric (overall cluster or clusters monitoring, Detailed monitoring of the local fabric (overall cluster or clusters monitoring,

monitoring each individual node in the cluster, network utilization)monitoring each individual node in the cluster, network utilization)• Monitoring of the batch system.Monitoring of the batch system.• Monitoring of the mass storage system (total and available space, number of Monitoring of the mass storage system (total and available space, number of

connections, I/O performance)connections, I/O performance)• Monitoring of VO computing activities at a siteMonitoring of VO computing activities at a site

Monitoring of Tier 3 sites federationMonitoring of Tier 3 sites federation• Monitoring of the VO usage of the Tier3 resources in terms of data transfer and Monitoring of the VO usage of the Tier3 resources in terms of data transfer and

job processing and the quality of the provided service based on the job job processing and the quality of the provided service based on the job processing and data transfer monitoring metrics. processing and data transfer monitoring metrics.


13131313

Country Normalized CPU time per Country LHC VO (July 2009 - October 2011)

Всего - 3,520,919,756 часов Россия- 88,014,873 (2.5%)


14141414

Russia Normalized CPU time per SITE LHC VO (September 2009 - September 2011)

Всего по РДИГ - 84,057,552 часов

ОИЯИ - 34,244,089 часов (40.7%)КИ - 18,073,012 часов (21.5%)ИФВЭ - 10,610,480 часов (12.6%) ПИЯФ - 5,509,617 часов (6.6%)ИТЭФ - 5,150,221 часов (6.1%)НИИЯФ МГУ - 4,522,040 часов (5.4%)ИЯИ - 2,093,048 часов (2.5)


15151515

Russia Normalised CPU time per LHC VO (September 2009 – September 2011)


Russia Normalised CPU time by SITE and VO (September 2009 – September 2011)


Member States of JINR Normalized CPU time per countries (September 2009 - September 2011)

JINR member states

NormalisedNormalised CPU time CPU time

Armenia 21

Belarus 60,225

Bulgaria 1,623,837

Czech Republic 21,948,110

Poland 43,682,563

Romania 13,391,632

Russia JINR

84,057,552 34,244,089

Slovakia 3,428,272

Ukraine (Kharkov-KIPT-LCG2) 700+ 1,312,299


JINR Grid-infrastructure for training andJINR Grid-infrastructure for training and education – first step towards education – first step towards

construction of the JINR Member States construction of the JINR Member States grid-infrastructuregrid-infrastructure

Consists of three grid sites at Consists of three grid sites at JINR and one at each of the JINR and one at each of the following sites: following sites:

Institute of High-Energy Institute of High-Energy Physics - IHEP (Protvino), Physics - IHEP (Protvino),

Institute of Mathematics Institute of Mathematics and Information and Information Technologies AS of Republic Technologies AS of Republic of Uzbekistan - IMIT of Uzbekistan - IMIT (Tashkhent, Uzbekistan), (Tashkhent, Uzbekistan),

Sofia University "St. Sofia University "St. Kliment Ohridski" - SU Kliment Ohridski" - SU (Sofia, Bulgaria), (Sofia, Bulgaria),

Bogolyubov Institute for Bogolyubov Institute for Theoretical Physics - BITP Theoretical Physics - BITP (Kiev, Ukraine), (Kiev, Ukraine),

National Technical National Technical University of Ukraine "Kyiv University of Ukraine "Kyiv Polytechnic Institute" - KPI Polytechnic Institute" - KPI (Kiev, Ukraine). (Kiev, Ukraine).

Letters of Intent with Moldova Letters of Intent with Moldova “MD-GRID”, Mongolia “MD-GRID”, Mongolia “Mongol-Grid”, Kazakhstan“Mongol-Grid”, Kazakhstan

Project with Cairo UniversityProject with Cairo University

https://gridedu.jinr.ruhttps://gridedu.jinr.ru


WEB-PORTAL “GRID AT JINR” – “WEB-PORTAL “GRID AT JINR” – “ГРИД В ОИЯИГРИД В ОИЯИ”: http://grid.jinr.ru”: http://grid.jinr.ru A new informational resourcehas been created at JINR:web-portal “GRID AT JINR”.The content includes the detailed information on the JINR grid-site and JINR’s participation in grid projects:

•Grid Conception •Grid-technologies •Grid-projects •RDIG Consortium

•JINR Grid-site •Infrastructure and services •Scheme •Statistics •VO and experiments support

•ATLAS •CMS •CBM and PANDA •HONE

•How to start •JINR in Grid-Projects

•WLCG •GridNNN •EGEE •RFBR Projects •INTAS Projects •SKIF-Grid

•Grid Middleware testing •Cooperation with JINR Member-States •Monitoring&Accounting

•RDIG-monitoring •dCache-monitoring •Dashboard •FTS-monitoring •Н1 МС-monitoring

•Grid- Conferences of JINR •GRID •NEC

•Education •Grid-infrastructure for education •Courses & Lectures •Text books

•Documentations •Articles•Materials for education

•News


2121

Frames for Grid cooperation of JINR Frames for Grid cooperation of JINR Worldwide LHC Computing Grid (WLCG); Enabling Grids for E-sciencE (EGEE) - Now is EGI-InSPIRE EGI-InSPIRE RDIG Development Now is E-ARENA CERN-RFBR project “Grid Monitoring from VO perspective” BMBF grant “Development of the Grid-infrastructure and tools to provide joint investigations

performed with participation of JINR and German research centers” “Development of Grid segment for the LHC experiments” was supported in frames of JINR-

South Africa cooperation agreement; Development of Grid segment at Cairo University and its integration to the JINR GridEdu Development of Grid segment at Cairo University and its integration to the JINR GridEdu

infrastructureinfrastructure NATO project "DREAMS-ASIA“ (Development of gRid EnAbling technology in

Medicine&Science for Central ASIA); JINR - FZU AS Czech Republic Project “The GRID for the physics experiments”JINR - FZU AS Czech Republic Project “The GRID for the physics experiments” NASU-RFBR project “Development and support of LIT JINR and NSC KIPT grid-NASU-RFBR project “Development and support of LIT JINR and NSC KIPT grid-

infrastructures for distributed CMS data processing of the LHC operation”infrastructures for distributed CMS data processing of the LHC operation” JINR-Romania cooperation Hulubei-Meshcheryakov programme JINR-Moldova cooperation (MD-GRID, RENAM)(MD-GRID, RENAM) JINR-Mongolia cooperation (Mongol-Grid)(Mongol-Grid) JINR-Slovakia cooperation JINR- Kazakhstan Kazakhstan cooperation (ENU Gumelev) Project "SKIF-GRID" (Program of Belarussian-Russian Union State). Project GridNNN (National Nanotechnological Net)


2222

PerspectivesPerspectives of JINR GRID Activities of JINR GRID Activities - - Tier 3 sites monitoring projectTier 3 sites monitoring project - Common JINR-CERN Project “- Common JINR-CERN Project “Global data transfer monitoring system Global data transfer monitoring system

for WLCG infrastructurefor WLCG infrastructure”” - development of unified Grid-environment of the JINR Member States- development of unified Grid-environment of the JINR Member States - Participation in project “WLCG Tier1 center in Russia”- Participation in project “WLCG Tier1 center in Russia” Proposal to create the LCG Tier1 center in Russia (official letter by Minister of Proposal to create the LCG Tier1 center in Russia (official letter by Minister of

Science and Education of Russia A. Fursenko has been sent Science and Education of Russia A. Fursenko has been sent to CERN DG R. Heuer in March 2011).to CERN DG R. Heuer in March 2011). The corresponding point to include in the agenda of next 5x5 meeting The corresponding point to include in the agenda of next 5x5 meeting

Russia-CERN. Russia-CERN. - for all four experiments ALICE, ATLAS, CMS and LHCb- for all four experiments ALICE, ATLAS, CMS and LHCb - ~10% of the summary Tier1 (without CERN) resources- ~10% of the summary Tier1 (without CERN) resources - increase by 30% each year- increase by 30% each year - draft planning (proposal under discussion) to have prototype in the end- draft planning (proposal under discussion) to have prototype in the end of 2011 - beginning 2012, and full resources in 2013 to meet the start ofof 2011 - beginning 2012, and full resources in 2013 to meet the start of next working LHC session.next working LHC session. Discussion about distributed Tier1 in Russia for LHC and FAIRDiscussion about distributed Tier1 in Russia for LHC and FAIR


The Fourth International Conference "Distributed Computing and The Fourth International Conference "Distributed Computing and Grid-technologies in Science and Education“ – GRID’2010Grid-technologies in Science and Education“ – GRID’2010

252 participants from 21 countries: Armenia, 252 participants from 21 countries: Armenia, Belarus, Bulgaria, Hungary, Germany, Greece, Belarus, Bulgaria, Hungary, Germany, Greece, Georgia, Iceland, Kazakhstan, Moldova, Myanmar, Georgia, Iceland, Kazakhstan, Moldova, Myanmar, Poland, Russia, Romania, USA, Uzbekistan, Poland, Russia, Romania, USA, Uzbekistan, Ukraine, France, Czechia, Switzerland, Sweden as Ukraine, France, Czechia, Switzerland, Sweden as well as from CERN and JINR.well as from CERN and JINR.

56 universities and research centers of Russia.56 universities and research centers of Russia. 8 sections: WLCG - worldwide Grid for processing 8 sections: WLCG - worldwide Grid for processing

data from LHC at CERN, Grid-applications, Grid in data from LHC at CERN, Grid-applications, Grid in business, distributed computing and Grid-business, distributed computing and Grid-technologies in education, Gridtechnologies in education, GridННСННС – Grid of the – Grid of the national nanotechnology network, methods and national nanotechnology network, methods and algorithms for distributed computing, Grid-algorithms for distributed computing, Grid-infrastructure and "cloud" computing.infrastructure and "cloud" computing.

Round tables on using grid-technologies in Round tables on using grid-technologies in business and on training in grid-technologies and business and on training in grid-technologies and their application in education.their application in education.

36 36 plenary talks,plenary talks, 78 78 sectional talks sectional talks. .

T.Strizh (LIT, JINR) 1 Распределенная грид-инфраструктура для...

Documents

Transcript of T.Strizh (LIT, JINR) 1 Распределенная грид-инфраструктура для...