Lecture 2: Servers and Services Network Design & Administration.
Data Import Data Export Mass Storage & Disk Servers Database Servers Tapes Network from CERN Network...
10
Data Import Data Export Mass Storage & Disk Servers Database Servers Tapes Network from CERN etwork rom Tier 2 and imulation centers Physics Software Development R&D Systems and Testbeds Info servers Code servers Web Servers Telepresence Servers Training Consulting Help Desk Production Reconstruction Raw/Sim-->ESD Scheduled, predictable experiment/ physics groups Production Analysis ESD-->AOD AOD-->DPD Scheduled Physics groups Individual Analysis AOD-->DPD and plots Chaotic Physicists Desktops Tier 2 Local institutes CERN Tapes
-
Upload
cornelia-richard -
Category
Documents
-
view
219 -
download
1
Transcript of Data Import Data Export Mass Storage & Disk Servers Database Servers Tapes Network from CERN Network...
DataImport
DataExport
Mass Storage & DiskServers
Database Servers
Tapes
Network from CERN
Networkfrom Tier 2 andsimulation centers
PhysicsSoftware
Development
R&D Systemsand Testbeds
Info serversCode servers
Web ServersTelepresence
Servers
TrainingConsultingHelp Desk
ProductionReconstruction
Raw/Sim-->ESD
Scheduled, predictable
experiment/physics groups
ProductionAnalysis
ESD-->AODAOD-->DPD
Scheduled
Physics groups
Individual Analysis
AOD-->DPDand plots
Chaotic
Physicists Desktops
Tier 2
Local institutes
CERN
Tapes
DataImport
DataExport
Mass Storage & DiskServers
Database Servers
Tapes
Network from CERN
Networkfrom Tier 2 andsimulation centers
PhysicsSoftware
Development
R&D Systemsand Testbeds
Info serversCode servers
Web ServersTelepresence
Servers
TrainingConsultingHelp Desk
ProductionReconstruction
Raw/Sim-->ESD
Scheduled, predictable
experiment/physics groups
ProductionAnalysis
ESD-->AODAOD-->DPD
Scheduled
Physics groups
Individual Analysis
AOD-->DPDand plots
Chaotic
Physicists Desktops
Tier 2
Local institutes
CERN
Tapes
WORKFLOW
Experiment
PhysicsGroups Individual
Physicists
DataImport
DataExport
Mass Storage & DiskServers
Database Servers
Tapes
Network from CERN
Networkfrom Tier 2 andsimulation centers
PhysicsSoftware
Development
R&D Systemsand Testbeds
Info serversCode servers
Web ServersTelepresence
Servers
TrainingConsultingHelp Desk
ProductionReconstruction
Raw/Sim-->ESD
Scheduled, predictable
experiment/physics groups
ProductionAnalysis
ESD-->AODAOD-->DPD
Scheduled
Physics groups
Individual Analysis
AOD-->DPDand plots
Chaotic
Physicists Desktops
Tier 2
Local institutes
CERN
Tapes
DATAFLOW Robotic Mass Storage
Central Disk Cache
Local Disk Cache
Local Disk Cache
Local Disk Cache
Local Disk Cache
DataImport
DataExport
Mass Storage & DiskServers
Database Servers
Tapes
Network from CERN
Networkfrom Tier 2 andsimulation centers
PhysicsSoftware
Development
R&D Systemsand Testbeds
Info serversCode servers
Web ServersTelepresence
Servers
TrainingConsultingHelp Desk
ProductionReconstruction
Raw/Sim-->ESD
Scheduled, predictable
experiment/physics groups
ProductionAnalysis
ESD-->AODAOD-->DPD
Scheduled
Physics groups
Individual Analysis
AOD-->DPDand plots
Chaotic
Physicists Desktops
Tier 2
Local institutes
CERN
Tapes
Data Input Rate from CERN: Raw Data - 5% 50TB/yr ESD Data - 50% 50TB/yr AOD Data - All 10TB/yr Revised ESD - 20TB/yr
Data Input from Tier 2: Revised ESD and AOD - 10TB/yr
Data Input from Simulation Centers: Raw Data - 100TB/yr
DataImport
DataExport
Mass Storage & DiskServers
Database Servers
Tapes
Network from CERN
Networkfrom Tier 2 andsimulation centers
PhysicsSoftware
Development
R&D Systemsand Testbeds
Info serversCode servers
Web ServersTelepresence
Servers
TrainingConsultingHelp Desk
ProductionReconstruction
Raw/Sim-->ESD
Scheduled, predictable
experiment/physics groups
ProductionAnalysis
ESD-->AODAOD-->DPD
Scheduled
Physics groups
Individual Analysis
AOD-->DPDand plots
Chaotic
Physicists Desktops
Tier 2
Local institutes
CERN
Tapes
Total Storage: Robotic Mass Storage - 300TB Raw Data: 50TB 5*10**7 events (5% of 1 year) Raw (Simulated) Data: 100TB 10**8 events EDS (Reconstructed Data): 100TB - 10**9 events (50% of 2 years) AOD (Physics Object) Data: 20TB 2*10**9 events (100% of 2 years) Tag Data: 2TB (all) Calibration/Conditions data base: 10TB (only latest version of most data types kept here)Central Disk Cache - 100TB (per user demand)
CPU Required for AMS database servers: ??*10**3 SI95 power
DataImport
DataExport
Mass Storage & DiskServers
Database Servers
Tapes
Network from CERN
Networkfrom Tier 2 andsimulation centers
PhysicsSoftware
Development
R&D Systemsand Testbeds
Info serversCode servers
Web ServersTelepresence
Servers
TrainingConsultingHelp Desk
ProductionReconstruction
Raw/Sim-->ESD
Scheduled, predictable
experiment/physics groups
ProductionAnalysis
ESD-->AODAOD-->DPD
Scheduled
Physics groups
Individual Analysis
AOD-->DPDand plots
Chaotic
Physicists Desktops
Tier 2
Local institutes
CERN
Tapes
Data Output Rate to CERN: AOD Data - 8 TB/yr Recalculated ESD - 10 TB/yr Simulation ESD data - 10 TB/yr
Data Output to Tier 2: Revised ESD and AOD - 15 TB/yr
Data Output to local institutes: ESD, AOD, DPD data - 20TB/yr
DataImport
DataExport
Mass Storage & DiskServers
Database Servers
Tapes
Network from CERN
Networkfrom Tier 2 andsimulation centers
PhysicsSoftware
Development
R&D Systemsand Testbeds
Info serversCode servers
Web ServersTelepresence
Servers
TrainingConsultingHelp Desk
ProductionReconstruction
Raw/Sim-->ESD
Scheduled, predictable
experiment/physics groups
ProductionAnalysis
ESD-->AODAOD-->DPD
Scheduled
Physics groups
Individual Analysis
AOD-->DPDand plots
Chaotic
Physicists Desktops
Tier 2
Local institutes
CERN
Tapes
Farms of low cost commodity computers, limited I/O rate, modest local disk cache-----------------------------------------------------Reconstruction Jobs: Reprocessing of raw data: 10**8 events/year (10%) Initial processing of simulated data: 10**8/year
1000 SI95-sec/event ==> 10**4 SI95 capacity: 100 processing nodes of 100 SI95 power
DataImport
DataExport
Mass Storage & DiskServers
Database Servers
Tapes
Network from CERN
Networkfrom Tier 2 andsimulation centers
PhysicsSoftware
Development
R&D Systemsand Testbeds
Info serversCode servers
Web ServersTelepresence
Servers
TrainingConsultingHelp Desk
ProductionReconstruction
Raw/Sim-->ESD
Scheduled, predictable
experiment/physics groups
ProductionAnalysis
ESD-->AODAOD-->DPD
Scheduled
Physics groups
Individual Analysis
AOD-->DPDand plots
Chaotic
Physicists Desktops
Tier 2
Local institutes
CERN
Tapes
Event Selection Jobs: 10 physics groups * 10**8 events (10%samples) * 3 times/yr based on ESD and latest AOD data 50 SI95/evt ==> 5000 SI95 power
Physics Object creation Jobs: 10 physics groups * 10**7 events (1% samples) * 8 times/yr based on selected event sample ESD data 200 SI95/event ==> 5000 SI95 power
Derived Physics data creation Jobs: 10 physics groups * 10**7 events * 20 times/yr based on selected AOD samples, generates “canonical” derived physics data 50 SI95/evt ==> 3000 SI95 power
Total 110 nodes of 100 SI95 power
DataImport
DataExport
Mass Storage & DiskServers
Database Servers
Tapes
Network from CERN
Networkfrom Tier 2 andsimulation centers
PhysicsSoftware
Development
R&D Systemsand Testbeds
Info serversCode servers
Web ServersTelepresence
Servers
TrainingConsultingHelp Desk
ProductionReconstruction
Raw/Sim-->ESD
Scheduled, predictable
experiment/physics groups
ProductionAnalysis
ESD-->AODAOD-->DPD
Scheduled
Physics groups
Individual Analysis
AOD-->DPDand plots
Chaotic
Physicists Desktops
Tier 2
Local institutes
CERN
Tapes
Derived Physics data creation Jobs: 200 physicists * 10**7 events * 20 times/yr based on selected AOD and DPD samples 20 SI95/evt ==> 30,000 SI95 power
Total 300 nodes of 100 SI95 power
DataImport
DataExport
Mass Storage & DiskServers
Database Servers
Tapes
Network from CERN
Networkfrom Tier 2 andsimulation centers
PhysicsSoftware
Development
R&D Systemsand Testbeds
Info serversCode servers
Web ServersTelepresence
Servers
TrainingConsultingHelp Desk
ProductionReconstruction
Raw/Sim-->ESD
Scheduled, predictable
experiment/physics groups
ProductionAnalysis
ESD-->AODAOD-->DPD
Scheduled
Physics groups
Individual Analysis
AOD-->DPDand plots
Chaotic
Physicists Desktops
Tier 2
Local institutes
CERN
Tapes
