Data integration in real time (pvmanager, graphene and all of that) Gabriele Carcassi.
STAR scheduling future directions Gabriele Carcassi 9 September 2002.
-
Upload
darlene-day -
Category
Documents
-
view
217 -
download
0
Transcript of STAR scheduling future directions Gabriele Carcassi 9 September 2002.
![Page 1: STAR scheduling future directions Gabriele Carcassi 9 September 2002.](https://reader036.fdocuments.us/reader036/viewer/2022082505/56649e7f5503460f94b82864/html5/thumbnails/1.jpg)
STAR schedulingfuture directions
Gabriele Carcassi9 September 2002
![Page 2: STAR scheduling future directions Gabriele Carcassi 9 September 2002.](https://reader036.fdocuments.us/reader036/viewer/2022082505/56649e7f5503460f94b82864/html5/thumbnails/2.jpg)
Directions
Current system refine, add other features
Queue system Condor, Condor-g/Globus
File catalog Active policies and connection to file movers (HRM,
GDMP), migration to other tools (MAGDA, GDMP, RLS, ...)
![Page 3: STAR scheduling future directions Gabriele Carcassi 9 September 2002.](https://reader036.fdocuments.us/reader036/viewer/2022082505/56649e7f5503460f94b82864/html5/thumbnails/3.jpg)
Current system
Optimize the policy as the scheduler is being used, fine tune the policy
so that the farm is used at top efficiencyOptimize the queries to the file catalog
make the queries smarter: for example, when a limit on the file exists, take the files from the machine that is less busy
![Page 4: STAR scheduling future directions Gabriele Carcassi 9 September 2002.](https://reader036.fdocuments.us/reader036/viewer/2022082505/56649e7f5503460f94b82864/html5/thumbnails/4.jpg)
Current system
Other refinement produce less garbage (all the temporary scripts and
file lists) better installation make it available to both BNL and LBL
![Page 5: STAR scheduling future directions Gabriele Carcassi 9 September 2002.](https://reader036.fdocuments.us/reader036/viewer/2022082505/56649e7f5503460f94b82864/html5/thumbnails/5.jpg)
Queue system
Investigating the use of Condor first deploy just Condor, and gather experience with
that migrate to Condor-g and Globus, and start
submitting jobs across sites integrate DAGMAN with the file catalog and the file
movers (?)
![Page 6: STAR scheduling future directions Gabriele Carcassi 9 September 2002.](https://reader036.fdocuments.us/reader036/viewer/2022082505/56649e7f5503460f94b82864/html5/thumbnails/6.jpg)
Queue system
Setup a testbed to gather experience with Condor by itself RCF is investigating on how to deploy Condor deploy Condor on some nodes to gather experience integrate Condor in our resource broker when everything works, we can start migrating the
whole farm
![Page 7: STAR scheduling future directions Gabriele Carcassi 9 September 2002.](https://reader036.fdocuments.us/reader036/viewer/2022082505/56649e7f5503460f94b82864/html5/thumbnails/7.jpg)
Queue system
Setup a testbed to gather experience with Condor-g setup two farms with Condor-g and gather
experience make changes to the resource broker, to be able to
submit across farms needs a file catalog that works on both sites
![Page 8: STAR scheduling future directions Gabriele Carcassi 9 September 2002.](https://reader036.fdocuments.us/reader036/viewer/2022082505/56649e7f5503460f94b82864/html5/thumbnails/8.jpg)
File catalog
Currently analyzing the optionsKeep for now the STAR catalog as it is
making the query smarter interface with tools like HRM/DRM
![Page 9: STAR scheduling future directions Gabriele Carcassi 9 September 2002.](https://reader036.fdocuments.us/reader036/viewer/2022082505/56649e7f5503460f94b82864/html5/thumbnails/9.jpg)
File catalog
Migration to other grid tools right now, the only part to be rewritten is the method
that resolves the query MAGDA’s catalog is not completely different, though
some issues has to be addressed integration with a grid catalog might provide other
feature (such as file migration) for free