EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP...
-
Upload
conner-shelburne -
Category
Documents
-
view
217 -
download
3
Transcript of EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP...
![Page 1: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/1.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1
WP1
Speaker name (Speaker function and WP )
speaker@
Presentation address e.g. https://edms.cern.ch/document/366530
![Page 2: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/2.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 2
Review of WP1 architecture
WP1 WMS architecture reviewed To apply the “lessons” learned and addressing the problems emerged with the
first release of the software, deployed in the EDG testbed E.g. reduce of persistent job info repositories, delegate some functionalities to
pluggable modules, etc.
To increase the reliability of the system To support new functionalities:
User APIs Job checkpointing
Shown at the WP1 demo Job partitioning Interactive jobs Dependencies of jobs Parallel jobs Grid Accounting Resource reservation and co-allocation
To make easier the interoperability with other Grid frameworks
![Page 3: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/3.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 3
WP1 reviewed architecture
Details in D1.4 …
![Page 4: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/4.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 4
Job submission example (for a “simple” job)
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
![Page 5: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/5.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 5
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
edg-job-submit myjob.jdlMyjob.jdl
JobType = “Normal”;Executable = "$(CMS)/exe/sum.exe";InputData = "LF:testbed0-00019";ReplicaCatalog = "ldap://sunlab2g.cnaf.infn.it:2010/rc=WP2 INFN Test Replica Catalog,dc=sunlab2g, dc=cnaf, dc=infn, dc=it";DataAccessProtocol = "gridftp";InputSandbox = {"/home/user/WP1testC","/home/file*”, "/home/user/DATA/*"};OutputSandbox = {“sim.err”, “test.out”, “sim.log"};Requirements = other. GlueHostOperatingSystemName == “linux" && other. GlueHostOperatingSystemRelease == "Red Hat 6.2“ && other.GlueCEPolicyMaxWallClockTime > 10000;Rank = other.GlueCEStateFreeCPUs;
submitted
Job Status
UI: allows users to access the functionalitiesof the WMS
Job Description Language(JDL) to specify job characteristics and requirements
![Page 6: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/6.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 6
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
RBstorage
Input Sandboxfiles
Job
waiting
submitted
Job StatusNS: network daemon responsible for acceptingincoming requests
![Page 7: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/7.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 7
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
RBstorage
waiting
submitted
Job Status
WM: responsible to takethe appropriate actions to satisfy the request
Job
![Page 8: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/8.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 8
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
RBstorage
waiting
submitted
Job Status
Match-maker
Where does thisjob must be executed ?
![Page 9: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/9.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 9
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
RBstorage
waiting
submitted
Job Status
Match-Maker/ Broker
Matchmaker: responsible to find the “best” CE where to submit a job
![Page 10: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/10.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 10
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
RBstorage
waiting
submitted
Job Status
Match-Maker/ Broker
Where are (which SEs) the needed data ?
What is thestatus of the
Grid ?
![Page 11: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/11.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 11
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
RBstorage
waiting
submitted
Job Status
Match-maker
CE choice
![Page 12: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/12.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 12
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
RBstorage
waiting
submitted
Job Status
JobAdapter
JA: responsible for the final “touches” to the job before performing submission(e.g. creation of wrapper script, etc.)
![Page 13: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/13.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 13
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
RBstorage
Job Status
JC: responsible for theactual job managementoperations (done via CondorG)
Job
submitted
waiting
ready
![Page 14: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/14.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 14
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
RBstorage
Job Status
Job
InputSandboxfiles
submitted
waiting
ready
scheduled
![Page 15: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/15.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 15
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
RBstorage
Job Status
InputSandbox
submitted
waiting
ready
scheduled
running
“Grid enabled”data transfers/
accesses
Job
![Page 16: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/16.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 16
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
RBstorage
Job Status
OutputSandboxfiles
submitted
waiting
ready
scheduled
running
done
![Page 17: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/17.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 17
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
RBstorage
Job Status
OutputSandbox
submitted
waiting
ready
scheduled
running
done
edg-job-get-output <dg-job-id>
![Page 18: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/18.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 18
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
RBstorage
Job Status
OutputSandboxfiles
submitted
waiting
ready
scheduled
running
done
cleared
![Page 19: EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1 WP1 Speaker name (Speaker function and WP ) speaker@ Presentation address e.g. .](https://reader031.fdocuments.us/reader031/viewer/2022013003/551c1091550346a84f8b53c3/html5/thumbnails/19.jpg)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 19
Job monitoring
UI
Log Monitor
Logging &Bookkeeping
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ComputingElement
RB node
LM: parses CondorG logfile (where CondorG logsinfo about jobs) and notifies LB
LB: receives and stores job events; processes corresponding job status
Log ofjob events
edg-job-status <dg-job-id>
Job status