OSG site utilization by VOs

Ilya Narsky, Caltech

Ilya Narsky Seattle OSG Meeting, Aug 2006 2

Help VOs utilize OSG sites Query VO reps at regular time intervals (monthly or

quarterly) Learn what prevents VOs from running jobs on OSG

sites, help solve problems, report status to the OSG Council

Motivated by the fact that utilization of OSG resources is far below 100%

Effort on my part started in early June Communication through osg-users list (low traffic). 11 VOs currently on the list, 2 more may be added

in the near future.

Ilya Narsky Seattle OSG Meeting, Aug 2006 3

Surveyed VOs VO Rep

ATLAS Torre Wenaus

CMS Frank Wuerthwein

CDF Matt Norman

D0 Parag Mhashilkar

LIGO Kent Blackburn

David Meyers

GADU Dina Sulakhe

NANOHUB Steve Clark

SDSS/DES Nickolai Kouropatkine

STAR Jerome Lauret

GLOW Dan Bradley

FERMILAB Keith Chadwick

iVDGL ?

MARIACHI ?

Ilya Narsky Seattle OSG Meeting, Aug 2006 4

Feedback All feedback summarized at VO twiki:

http://osg.ivdgl.org/twiki/bin/view/VO/WebHome (look at the right “status” column)

VO enthusiasm and response time vary Various problems, both technical and sociological I classify VOs into 3 groups:

1) ATLAS, CMS 2) LIGO, GADU, NANOHUB (fighting technical issues) 3) everyone else (planning to expand their usage of OSG

sites some time in the future but no immediate requests/complaints)

Ilya Narsky Seattle OSG Meeting, Aug 2006 5

LIGO and GADU VOs that are actively trying to run on more OSG

sites but need to solve various technical problems LIGO (see Kent Blackburn’s talk today)

Rely on VDS tools. Working with the VDS team for development/support of these tools.

New site-verify.pl in ITB-0.5.0 Jobs need 1-1.5 TB data on disk. Work on partitioning jobs. 1% LIGO jobs on OSG, 99% on LIGO grid

GADU Also rely on VDS tools. Unable to stage out to worker node

location (because it Is not reported by VDS); problems similar to LIGO’s. Interacting with LIGO and VDS to solve this.

Ilya Narsky Seattle OSG Meeting, Aug 2006 6

NANOHUB Jobs take too long (6-10 days on a typical

OSG site). Does not work so well on sites with runtime limits.

Jobs run successfully at Purdue and UNM. They recently implemented suspend-and-

resume functionality at the application level. A single simulation job is split into dozens sweeps. Can now run jobs on 3 sites with runtime limits (GRASE-CCR-U2, OSG_LIGO_PSU, VAMPIRE-Vanderbilt). At the moment, ¾ of their jobs are evicted.

Ilya Narsky Seattle OSG Meeting, Aug 2006 7

Talks at Seattle Meeting Torre – OSG extensions (slides) Ajit – CMS MC Production on OSG (slides) Kent – LIGO experience (slides) Michael Miller – STAR

For now STAR jobs are confined to STAR sites. MC Production will likely need OSG sites. Analysis users not likely to start using OSG resources out of STAR soon. Need a substantial amount of disk space for STAR apps.

Maytal – TACC Learning curve for scientists to start using grid resources

Notes will be posted at http://osg.ivdgl.org/twiki/bin/view/VO/SeattleUsersGroupMtgAugust2006

Ilya Narsky Seattle OSG Meeting, Aug 2006 8

Future It has been agreed that one VO will be

selected for reporting at every Monday Operations meeting

We also asked VOs to make web pages: VO scientific goals, rules of membership, info on

how to join, how to submit jobs, who to complain to when jobs are not running

CDF, ATLAS, D0, and LIGO have some approximations to those web pages

Hopefully more feedback from VOs

Ilya Narsky Seattle OSG Meeting, Aug 2006 9

Questions Is our way of collecting info from VOs (my

reports and Operations meetings) adequate? What else can/should we do?

Does your VO want to participate in these surveys?