Visit to AIST, Tokyo, Japan 23 April, 2004

GridLab 2003/4 „Steady leadership in changing

times!”

Jarek Nabrzyski

Project Coordinator

Poznan Supercomputing and Networking Center

Visit to AIST, Tokyo, Japan 23 April, 2004

GridLab Project

Funded by the EU (5+ M€), January 2002 – March 2005Application and Testbed oriented

Cactus Code, Triana Workflow, all the other applications

Main goal: to develop a Grid Application Toolkit (GAT) and set of grid services and tools (GridSuite):

Resource management (GRMS), Data management (GDMS),Monitoring (Mercury) and information services, Adaptive components (Pythia), Mobile user support and remote visualization, Security services (GAS),Portals (GridSphere),

... and test them on a real testbed with real applications

Visit to AIST, Tokyo, Japan 23 April, 2004

GridLab Members

PSNC (Poznan) - coordination AEI (Potsdam) ZIB (Berlin) Univ. of Lecce Cardiff University Vrije Univ. (Amsterdam) SZTAKI (Budapest) Masaryk Univ. (Brno) NTUA (Athens)

Sun Microsystems GmbHHP France

ANL (Chicago, I. Foster) ISI (LA, C.Kesselman) UoWisconsin (M. Livny)

collaborating with:Users!

EU Astrophysics Network,

DFN TiKSL/GriKSL

NSF ASC Project

other Grid projectsGlobus, Condor,

GrADS,

PROGRESS, Clusterix

GriPhyn/iVDGL,

European Grid Projects (GRIDSTART)

KISTI, Canadian Grid, LSU…

Visit to AIST, Tokyo, Japan 23 April, 2004

GridLab Aims

Get Computational Scientists using the “Grid” and Grid services for real, everyday, production work (AEI Relativists, EU Network, Grav Wave Data Analysis, Cactus User Community, ...),

Make it easier for applications to make flexible, efficient, robust, use of the resources available to their virtual organizations

Dream up, prototype, and test new application scenarios which make adaptive, dynamic, wild, and futuristic uses of resources.

Visit to AIST, Tokyo, Japan 23 April, 2004

What GridLab isn’t

We are not developing low level Grid infrastructure - we do more actually, we want to be infrastructure independent,We do not want to repeat work which has already been done (want to incorporate and assimilate it …)

Globus APIs, OGSA/OGSI/…, ASC Portal (GridSphere/Orbiter), GPDK, GridPort, DataGrid, GriPhyn,...

Visit to AIST, Tokyo, Japan 23 April, 2004

The Same Application …

Application

GAT

Application

GAT

Application

GAT

Laptop The GridSuper Computer

No network! Firewall issues!

Visit to AIST, Tokyo, Japan 23 April, 2004

GAT: What is It?

GAT: Grid Application ToolkitImplements the GAT-API

Used by applications (different languages)

GAT Adaptors Connect to capabilities/services

GAT EngineProvides the function bindings for the GAT-API

Visit to AIST, Tokyo, Japan 23 April, 2004

GAT

More or less …Set of calls GAT_ToolOrService(arguments)

Your chosen tools/services: resource brokers, information servers, application managers, grid monitoring, data managers, notification, etc.

Set of APIs for dealing with the GAT (registration, information, errors, fault tolerance)

Visit to AIST, Tokyo, Japan 23 April, 2004

Project Goals Achieved in 2003

Integration of all the GridLab software,“Migration Scenario” driving all the developments and integration,First release of GridLab software scheduled for December 2003, Make the testbed more stable and ready for any demo action and/or development,SC2003 was chosen to demo the GridLab software,Strong dissemination of the project in order to get attention of interesting Grid applications,Start the GAT and other GridLab standardization work under GGF,Strengthen management of the project, with clear responsibilities to ensure better control of resources, fast identification of the risks, ensure progress, GridLab Open Source License, Plan for exploitation of the results,

Visit to AIST, Tokyo, Japan 23 April, 2004

What we have done during last 8 months...

GridLab meeting in Eger (April) and Olomouc (October)

More developments and integration

More and more WPs brought to the scenario

Today most of the WPs were integrated with each other

SC2003 in Phoenix (2 big „demos”, many „small” demos)

Painful demo, but successful: GRMS + portal + GAT application + Testbed + WP5 information service + Adaptive with some last minute problems with other services

„The devil is in the details” - the most true sentence.

Visit to AIST, Tokyo, Japan 23 April, 2004

Major achievements

Integration efforts and devotion of project staff,Meeting in Eger in March/April, meeting in Olomouc in October,Many small integration meetings (Poznan-Berlin-Potsdam, Potsdam-Budapest, Potsdam-Prague, Cardiff, GGF meetings and at other confs.),

Getting a major attention towards the GAT and GridLab services of many application communities,

GRIDSTART, GGF, application communities and 6FPP (EGEE, DEISA, HPC-Europa, InteliGrid…)Expectations are high!

Many successful implementations of servicesSingle GridLab services being used by other projects,Most of the services are fully integrated with each other, but can be configured as stand aloneGAT can connect to most of the services. All adaptors will be available by mid of May

Visit to AIST, Tokyo, Japan 23 April, 2004

Major achievement (cont.)

GridLab middleware being exploited by other projects and production testbeds

GridSphere: tens of projects (Grid Infrastructure Project, GEON, HPC-Europa, Physics Portal Development Project, …)

GRMS (Clusterix, SGI Grid, Progress, Canadian Grid, InteliGrid)

Mercury Monitor (DataGrid, P-GRADE)

Visualization - GriKSL

Open Source LicenseGridSphere has it’s own Open Source license, GTPL-like

The rest of GridLab software is based on the GPL-like license

Visit to AIST, Tokyo, Japan 23 April, 2004

Technical Progress

GAT API specification and GAT adaptor developer’s guide

GAT tested by Cactus and a GAT subset implementation for Triana (GAP)

Strenghten the collaboration with ISI, ANL, Wisconsin

New Portal Framework (GridSphere) - 1st release in 2003 (2nd in 2004) - being used by tens of projects!

Services ready and interoperable

GAT ready for standardization process, already under the community comment process

Visit to AIST, Tokyo, Japan 23 April, 2004

Technical Progress (cont.)

Extending the GridLab scenario

Successful meeting in Eger (new application users involved, new developments) and in Olomouc (integration)

Problems/IssuesGAT development delayed - underestimated efforts needed for the GAT development

Will be revisited and refactored! User’s input is needed. Good docs is needed.

Hiring a new person, experienced with Cactus development to speed up the CGAT integration.

GAT documentation is behind! We are shifting efforts a bit to speed up the process.

Visit to AIST, Tokyo, Japan 23 April, 2004

Plans for 2004

Continue the development of GAT and servicesmore info in the following presentations

More complicated scenarioscollaborative environments,submitting, controling and steering jobs from mobile devices,more dynamic behavior of applications

Two more GridLab Workshop Meetings (Lecce: 16-22 May, Zakopane: early December)Organizing the GridLab/GT3.2/WSRF integration meeting with US partners (to ensure GAT compatibility)Prepare for the Supercomputing demosGGF BOF (GAT) and finally GGF WGGGF Scheduling Architecture Working Group now started (GRMS)Exploitation of the project results

Close work with GridLab’s commercial partnersGlobal Grid Application Alliance with GridLab’s leadershipGGF activities (long term)GridSuite (based on GridLab plus Gridstart Open Source, PSNC plus partners)

Open Source + commercial supportEuropean Grid Support Centre (PSNC)

Visit to AIST, Tokyo, Japan 23 April, 2004

Global Grid Application Alliance

Global Europe - GridLab, Jarek to organizeUS - Ed SeidelAsia Pacific - both Satoshi’s, KISTI

GoalsBuild the global community of Grid Application Developers and UsersBuild a Grid Application Toolkit, get the requirements from the GGAAFocus on Grid application level: tools, programming environments, PSEsWork on the new scenarios that the Alliance will bringWork on portable applications on a global scale using the state-of-the art technologies

Provide a background for the GridLab follow-up proposal (GridLab2, Matrix - the working titles)

Visit to AIST, Tokyo, Japan 23 April, 2004

Matrix Project Proposal

Submission: mid of 2005, Start: beginning of 2006.3 year, ~40 participants, ~17MEuroMatrix takes as a starting point work done both in the GridLab project, and importantly in other leading EU, US, and Asian projects, that aim to develop services and application-level technologies needed by the researchers who will be using the Grid.Its major focus will be on developing tools needed by application groups to grid-enable present day applications, while working with these groups to research and deploy a new generation of Grid application scenarios.

Visit to AIST, Tokyo, Japan 23 April, 2004

Matrix cont.

The grand vision of Matrix is a set of widely adopted application level tools that provide transparent access to Grid services, regardless of where they are deployed, that enable pressing problems to be solved on Grids.

As these tools will undergo an international standardization process, they should become the mechanism for application communities to harness the power of Grid computing, and to work together with interoperable components.

Visit to AIST, Tokyo, Japan 23 April, 2004

Matrix Applications

Visit to AIST, Tokyo, Japan 23 April, 2004

Matrix - Research Components