MONITORING, ACCOUNTING AND REGISTRATION SERVICES FOR RUSSIAN

GRID NETWORKq

Sergey Belov, Tatiana Goloskokova, Vladimir Korenkov, Nikolay Kutovskiy, Danila Oleynik, Artem Petrosyan, Roman

Semenov, Alexander Uzhinskiy LIT JINR

The 5th International Conference "Distributed Computing and Grid-technologies in Science and Education“

(GRID’2012)20 July 2012, JINR, Dubna, Russia

RGN projectRussian National Grid Network

Development of grid-based networks for high performance computing

To create a network infrastructure and connection to it of largest supercomputer centers, enterprises, high-tech industries and research organizations

Computing power for high-priority industrial tasks and scientific projects

2010 – start of network backbone’s development2011 – core grid services development, application’s

interfaces, grid infrastructure prototype2012 – further system development, more

supercomputers, more applications

2/14

Grid and network infrastructureMain aim

Integration of unique, medium and small supercomputers into a unified distributed computing environment

Development of high-speed backbone network for the infrastructure

Provide secure and clear remote access for users to these resources

Heterogeneous grid environment (hardware, software)

Oriented to parallel tasks rather than single batch tasks

Workflow management Jobs consist of tasks

Technical points is a set of grid services with unified interfacespartially based on Globus Toolkit 4, VOMSGSI based security model, mainly RESTful grid services

3/14

Core grid servicesWeb User InterfaceResource Broker/metascheduler + Workflow

management (RESTful)Grid gateway to the resources (WS)Information Service (RESTful)Monitoring & accountingRegistration service (RESTful)GSI services: CA, MyProxy, VOMSGridFTP serversNon-grid: applications licensing service, billing

and audit, application solvers images management

4/14

Goals of monitoringTracking state of sites and services

AvailabilityReal operational state

Monitoring of user's jobs and tasks Keeping history on different system's

parameters Information representation

General infrastructure state in wholeRunning jobs and tasksSeparate sites and services (real-time and history)Visualization of job events

5/14

Monitoring of resources and servicesState of computational resources by site

(based on data from information indexes)Slots available for tasksStructure and properties of clusters

Subclusters, nodes, slots, operation system, architecture

Application softwareSupported VOs (with ACLs, Access Control

Lists)Monitoring of jobs running on sites (by

information from Job management servers)6/14

Accounting and job monitoringGoal: to get information, both real-time and

historical, on resources utilization and jobs running on RGN infrastructure (by users, VOs, sites)

Information sources: JMS servers, Grid Gateways and local resources managers

Collecting data on jobs and tasks in the systemAll jobs events timestamps, real consumed CPU time

Accounting information reports in different views:by sites, VOs and single users

Aggregation of actual job's execution time from all sites

7/14

Monitoring and accounting information flows

8/14

Monitoring andaccounting

datastorage

Informationcollector

JMSJob

managementservices

Monitoringwebsite

Monitoring dataprovisioning

(Web Services)

AccountingInformationpublisher

Infosyscentral

Informationindex

Registrationservice

Accounting and jobs monitoring screenshots

9/14

Resource centers on the mapMain sites of the

infrastructure prototype (2011)

Other supercomputers are in process of connection and testing

Will be more supercomputers from different Russian regionsby the end of 2012

10 resource centers are already connected to high-speed VPN

10/14http://grid.voskhod.ru

Infrastructure operation visualization with Google Earth

11/14

Visualization package from the ARDA Dashboard project was used

Registration serviceService for registration of resources and grid servicesMain task is to store and provide information about

resources and grid services in RGNSite contacts and other formal informationServices’ endpoints, type, DN, etc.Resources operational state

Registration and information modification procedures support

Web-interface for users and RESTful interface for system services

Authorization and authentication along with rights and roles of access to add, modify or access information

Registration service’s web interface

Conclusion and resultsWere developed architecture and software

solution for monitoring , accounting and registration service for the Russian National Grid Network

Experience of developing of the similar services within GridNNN project was very helpful and was heavily used

Created tools were successfully deployed and are in production

Russian National Grid Network project is currently developing, so further work on the services will be continued

14/14