2003 Annual CrossGrid Project Workshop in Santiago10th – 12th February 2003

1

Task 3.5 Activities in the first year• Month 3:Month 3:

– WP3 Integration Team was formed– Definition of the software requirements (tools and services) for each

task in WP3 was done. (SRS Deliverable D3.1)– Each task delivered the information to WP4 for initial infrastructure

set up (Definition of the reference platform)• Meeting at CERN in April 2002 with WP4 and EDG Integration

Team– Each task defined requirements from the applications in WP1 and

WP2– CVS central CrossGrid in Karlsruhe was available (accesible at

anoncvs@gridportal.fzk.de:/cvs/crossgrid):• PSNC created such repository for WP3 in Poznan• IFIC (CSIC-Valencia) started to do back-up (accessible at

anoncvs@cg1.ific.uv.es:/cvs/crossgrid)

2003 Annual CrossGrid Project Workshop in Santiago10th – 12th February 2003

2

2003 Annual CrossGrid Project Workshop in Santiago10th – 12th February 2003

3

2003 Annual CrossGrid Project Workshop in Santiago10th – 12th February 2003

4

Task 3.5 Activities in the first year• Month 6:Month 6:

– Each partner worked on designing the architecture of the new grid services and tools. (SDD Deliverable D3.2)

– Each task prepared a document about test procedures to validate its own software (sent to WP3 Integration Team and included in CG3.5-D3.2 deliverable)

– Co-ordination established with the other WP’s and Grid projects– WP3 IT is collaborating with WP4 IT:

• First meeting took place at CERN (July 2002), to get an in-site knowledge of the EDG testbed 1 setup

• Second meeting at CERN (October 2002), to review issues regarding testbed setup and computing centre operation

– A local testbed (see WP4 deliverable for Month6) was placed with Globus and DataGrid’s tools (version 1.2.x) in several sites, where each task has tested and developed their tools and services.

2003 Annual CrossGrid Project Workshop in Santiago10th – 12th February 2003

5

A local testbed

• CrossGrid Testbed:– LCFGLCFG

– CECE

– WNWN

– SESE

– UIUI

– Local RB

2003 Annual CrossGrid Project Workshop in Santiago10th – 12th February 2003

6

Task 3.5 Activities in the first year• Month 10-11:Month 10-11:

– Each task has worked on first prototype document of the new grid tools and services (First prototype Deliverable D3.3)

– An update of the test procedures for each module has been done by each task

– Each task has released the first version of the software to the WP3 IT. This software can be obtained from CVS repository hosted by FZK at Karlsruhe

– Each task has included in its D3.3 deliverable a User Guide explaining how to install and use their tools and services

– Each task and subtask has prepared an installation guide: • These documents are intended for Site Administrator of

CrossGrid Testbed resources• With these documents a WP3 general installation middleware

procedure document has been done by WP3 IT

2003 Annual CrossGrid Project Workshop in Santiago10th – 12th February 2003

7

Task 3.5 Activities in the first year• Month 12:Month 12:

– WP3 Integration meeting at Santiago (from 5th to 9th February)– Integrate and test the whole WP3 software in Santiago Testbed has been

done– Once everything is working, packages have been distributed (RPM files)

http://gridportal.fzk.de/distribution/crossgrid/crossgrid/wp3/wp3_x-name/ (x=1,…,5)

– First WP3 prototype is delivered– The WP3 IT has provided documentation for system administrators who

are installing the WP3 software:• Test procedures (CG3.5-D3.3 document)• User Guide (In each D3.3 task document)• Installation Guide (http://ific.uv.es/~sgonzale/integration/)

– Portal- Roaming access and SANTA-G monitoring tool will be implemented in the WP4 deliverable Testbed (Demo in Brussels, beginning of March)

2003 Annual CrossGrid Project Workshop in Santiago10th – 12th February 2003

8

Task 3.5 and WP3 IT actions

• Before Month 11 (December 2002)Before Month 11 (December 2002)

– Each partner made and tested well its software according to test procedures proposed by each Task

– Compatibility between WP3 software, Globus and EDG software was guaranteed. Vital evaluation with consequence serious for the success of the integration phase

– Co-ordination with WP4 and its time scheduling

– IT followed the progress of each task

2003 Annual CrossGrid Project Workshop in Santiago10th – 12th February 2003

9

Task 3.5 and WP3 IT Actions• Month 11 (January 2003)Month 11 (January 2003)

– Started Integration and Test phase– WP3 IT obtained the software from the CVS server– Each member installed and tested one of the tools produced by one

external partner following the WP3 Installation Guide• PSNC (task 3.1) tested Grid Resourced Management (task 3.2)• UAB (task 3.2) tested Portals and Roaming Access (task 3.1)• CYFRONET (task 3.3.1) tested Jiro-based monitoring tool (task 3.3.3)• CYFRONET (task 3.3.3) tested OCM-G monitoring tool (task 3.3.1)• CYFRONET (task 3.4) tested SANTA-G monitoring tool (task 3.3.2)• CSIC-Valencia (task 3.5) tested Optimisation Data Access (task 3.4)

– Report with bugs, comments and problems found during installation-test process was done and sent to WP3 task leaders

2003 Annual CrossGrid Project Workshop in Santiago10th – 12th February 2003

10

First PrototypeTask 3.5 Test and Integration

Santiago González de la Hoz

IFIC – València (Spain)

Partners:

2003 Annual CrossGrid Project Workshop in Santiago10th – 12th February 2003

11

Introduction• Goals (Annex 1):Goals (Annex 1):

– To deliver a release version of WP3 before each prototype

– Test all tools and services of WP3

– Integration and validation phase

• WP3 Integration Team WP3 Integration Team (IT):(IT):– It is formed by people from

each task of this WP and by people from each partner of this task.

Task 3.5 (WP3) Integration Team:

-Pawel Wolniewicz (PSNC, Task3.1)-Marcin Plóciennik (PSNC, Task3.1)-Miquel A. Senar (UAB, Task 3.2)-Bartek Balis (CYFRONET, Task 3.3)-Slawek Zielinski (CYFRONET, Task 3.3)-Lukasz Dutka (CYFRONET, Task 3.4)-Álvaro Fernández (CSIC, Task 3.2)-Javier Sánchez (CSIC, Task 3.5)-Santiago González (CSIC, Task 3.5)

Many thanks for your help!!All WP3 membersNorbert Meyer (WP3), Jesús Marco (WP4) and Marian Bubak (WP5)

2003 Annual CrossGrid Project Workshop in Santiago10th – 12th February 2003

12

Task 3.5 and WP3 IT Actions• Month 12 (February 2003 TODAY)Month 12 (February 2003 TODAY)

– All WP3 IT members have been physically in Santiago (from 5from 5thth to 9 to 9thth February February):

• At CESGA we have had the hardware and software required for a real CrossGrid testbed. (Many thanks to CESGA people for your support)

– IT has installed and tested WP3 software in CESGA Testbed

– RPM files have been done and distributed to packages repository (in Karlsruhe)

– The WP3 installation guide has been and will be improved

2003 Annual CrossGrid Project Workshop in Santiago10th – 12th February 2003

13

Task 3.5 and WP3 IT Actions• Month 12 (February 2003)Month 12 (February 2003)

– Integration with WP4 (TestBed):Integration with WP4 (TestBed):• Portal and Roaming Access (with the help of Pawel Wolniewicz

and Marcin Plóciennik)• Grid Resource Management (Submission for MPI Jobs hasn’t

been finished on time for the demo)• Santa-G (with the help of David O’Callaghan)• OCM-G (1) problems with kernel, patch needed: -> Solved by R.

Wismueller and B. Balis. 2) At the moment it is using NFS: -> First prototype is a local “grid” Testbed)

• Jiro-Based (it is not available at CVS, not RPM files at packages repository, see integration report by M. Hardt)

• Optimisation of data Access (it requires a HSM server and Castor is not yet in)

2003 Annual CrossGrid Project Workshop in Santiago10th – 12th February 2003

14

Conclusions• Who and how tested the tools in WP3?Who and how tested the tools in WP3?

– Who: Each task tested its software before releasing code to CVS server

• The WP3 Integration Team– See test procedurestest procedures (CG3.5-D3.2 and D3.3)

– See User GuidesUser Guides (D3.3 for each task)

– See General WP3 Installation GuideGeneral WP3 Installation Guide (http://ific.uv.es/~sgonzale/integration/)

– How: First, each IT member in its local testbed

• The final test and integration before delivering a release version of WP3 has been done at CESGA

2003 Annual CrossGrid Project Workshop in Santiago10th – 12th February 2003

15

Conclusions

• How the release version has been prepared?How the release version has been prepared?– Source and binary code can be obtained from CVSCVS Central Repository

hosted by FZK in Karlsruhe. This repository is intended primarily for developers

http://gridportal.fzk.de/cgi-bin/viewcvs.cgi/crossgrid/crossgrid/wp3/

– All software has been packaged as source and binary RPM filesRPM files

• All of the packages are available directly through a web interface either via direct browsing or via formatted list:

http://gridportal.fzk.de/distribution/crossgrid/crossgrid/wp3/

WP4 rpm release:

http://gridportal.fzk.de/distribution/crossgrid/releases/

2003 Annual CrossGrid Project Workshop in Santiago10th – 12th February 2003

16

Future Plans (WP3 and IT)

• Task3.1- submit some interactive and MPI jobs• Task3.2- prepare the RB to effectively submit MPI jobs

into the CE’s and deploy the Scheduling Agent entity• Task3.3.1- integration with Grid information services,

getting rid of NFS• Task3.3.2- add functionality & security• Task3.3.3- add monitoring events aggregation

and filtering functionality • Task3.4- more collaboration with DataGrid (To provide

one unified Replica Manager for DataGrid and CrossGrid; Estimator for Castor HSM )