Enabling access to global grid
description
Transcript of Enabling access to global grid
![Page 1: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/1.jpg)
Enabling access to global grid
Cezary Mazurek
Poznań Supercomputing and Networking Center,
Poland
![Page 2: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/2.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 2
Agenda
• Introduction• PROGRESS grid-portal
environment• PROGRESS GSP (Grid Service
Provider)• Enabling access to global grid
with open source GSP• Conclusions• Demo
![Page 3: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/3.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 3
R&D Center
• PSNC was established in 1993 and is an R&D Center in:– New Generation Networks
• POZMAN and PIONIER networks• 6-NET, SEQUIN, ATRIUM projects
– HPC and Grids• GRIDLAB, CROSSGRID, VLAB,
PROGRESS projects
– Portals and Content Management Tools• Polish Educational Portal "Interkl@sa",
Multimedia City Guide, Digital Library Framework,Interactive TV
![Page 4: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/4.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 4
SUN Center of Excellence
• PSNC became the Sun CoE in New Generation Networks, Grids and Portals in November 2002
![Page 5: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/5.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 5
PROGRESS (1)
• Duration: December 2001 – May 2003 (R&D)
• Budget: ~4,0 MEuro• Project Partners
– SUN Microsystems Poland– PSNC IBCh Poznań– Cyfronet AMM, Kraków– Technical University Łódź
• Co-funded by The State Committee for Scientific Research (KBN) and SUN Microsystems Poland
![Page 6: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/6.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 6
PROGRESS (2)
• Deployment: June 2003 – ....– Grid constructors– Computational applications
developers– Computing portals operators
• Enabling access to global grid through deployment of PROGRESS open source packages
![Page 7: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/7.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 7
PROGRESS (3)
• Cluster of 80 processors
• Networked Storage of 1,3 TB
• Software: ORACLE, HPC Cluster Tools, Sun ONE, Sun Grid Engine, Globus
Wrocław
Gdańsk
![Page 8: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/8.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 8
PROGRESS GPE modules
• grid resource broker: enables the execution of PROGRESS grid jobs in a distributed cluster of Sun computers; uses XRSL language for grid job descriptions
• grid security: SSO authentication, authorisation• checkpointing• user interfaces
– Grid Service Provider• web portal: grid job management, application management,
provider management, short news reading and editing, DMS file system management
• migrating desktop: grid job management and DMS file system management
• data management system: storing the input and output files, metadata, multiple media data containers
• visualisation
![Page 9: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/9.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 9
Motivation (1)
• grid access environments lacked flexibility: one grid -> one portal
• user must have relied on the grid (e.g. job execution history)
• user interfaces were not that much functional
• what if we need more than one user interface to grid services?
![Page 10: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/10.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 10
Motivation (2)
• Grid constructors– „We have to build a new portal to enable
our grid resources to users...”
• Application developers– „We have to build a new portal to enable
our grid-enabled applications to users...”
• Portal operators– „It might work but probably we have to
redesign our portal to enable this new grid service...”
![Page 11: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/11.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 11
PROGRESS GPEPORTAL
HPCRESOURCES
GRIDMANAGEMENT
SYSTEM
GRID SERVICEPROVIDER
4-tier new4-tier newgrid-portal environmentgrid-portal environment
PORTAL
HPCRESOURCES
GRIDMANAGEMENT
SYSTEM
3-tier classical3-tier classicalgrid-portal grid-portal
environmentenvironment
![Page 12: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/12.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 12
PROGRESS GPE
![Page 13: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/13.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 13
PROGRESS GSPUser environment
(eg. Portal)
User environment (eg. Portal)
HPCRESOURCES
GRIDMANAGEMENT
SYSTEM
GRID SERVICEPROVIDER
HPCRESOURCES
GRIDMANAGEMENT
SYSTEM
GRID SERVICEPROVIDER
User environment (eg. Portal)
User environment (eg. Portal)
User environment(eg. Java GUI)
User environment(eg. Java GUI)
![Page 14: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/14.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 14
Grid Service Provider (1)
• the use of the grid resources most comfortable to the end users
• allows for easy building of numerous portals and other user interfaces; users can switch from one to another and use the same GSP services
• various web portals sharing the same distributed grid services and resources
• possibility of providing all clients (user interfaces) with computing resources belonging to two or more different grids
![Page 15: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/15.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 15
Grid Service Provider (2)
• Provided services:– job submission service
• managing the creation of user jobs, their submission to the grid and the monitoring of their execution (typically through reverse reporting performed by the Grid Management System about events connected with the execution of jobs)
– application management service• storing information about applications available for
running in the grid• assisting application developers in adding new
applications to the application repository
– provider management service• keeping up‑to‑date information on the services
available within the provider
Po
rtlet
We
bse
rvice
Se
ssion
Be
an
En
tity Be
an
s
![Page 16: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/16.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 21
PROGRESS Communication
HPC PortalGrid
Service Provider
DataManagement System
GridResource Broker
saveJob()getApplications()getTemplates()saveTaskOfJob()saveStdOfTask()submitJob()getUserJobs()getJobStatus()
listUserDirectory()addUserFile()
getUserFileLocation()
submitJob() changeTaskStatus()
![Page 17: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/17.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 22
Web Services and Grid
PORTLETSPORTLETS GRID SERVICEGRID SERVICEPROVIDERPROVIDER
DATADATAMANAGEMENTMANAGEMENT
GRID GRID RESOURCERESOURCE
BROKERBROKER
WSWS
![Page 18: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/18.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 23
GSP Release & Future• The Grid Service Provider software has been
released under an open source license (SC 2003)• It is tested and deployed within PROGRESS GPE
together with Grid Resource Broker, PROGRESS Portal and Migrating desktop, Data Management System, Test Applications and Visualisation
• We will continue the R&D on the Grid Service Provider– cooperation with multiple grid resource brokers/grid
management systems (e.g. the GridLab one)– additional functionality (e.g. OGSA interface,
cooperation with virtual laboratories)– apply the concept for access to visual grids, data grids
and digital library tools and services
![Page 19: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/19.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 24
Enabling global grid
![Page 20: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/20.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 25
Enabling global grid
![Page 21: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/21.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 26
Enabling global grid
![Page 22: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/22.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 27
Enabling global grid
• Moreover, to enable global grid within a single GSP installation, we developed a concept of grid resource broker plug-ins
• The plug-in will be responsible for communication with the grid resource broker
• The job submission service will use the proper plug-in whenever there’s a need to submit a computing job to a particular grid
![Page 23: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/23.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 28
GRB plug-in
• A java class capable of communicating with and submitting computing jobs for execution to a particular grid
• To enable the GRB plug-in concept, there will be a set of entity beans prepared to manage the information about available plug-ins and grids
![Page 24: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/24.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 29
Available plug-ins
• The GRB plug-in mechanisms are under development
• New release will include a plug-in for the PROGRESS grid resource broker
• Next plug-ins to follow:– GridLab– SGE?– Globus?
![Page 25: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/25.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 30
![Page 26: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/26.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 31
PROGRESS HPC Portal
![Page 27: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/27.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 32
Conclusions
• Grid constructors– „Finally, we can forget about computing portals
development”
• Application developers– „Now, we can easily make our application
available to new computing portals”
• Portal operators– „Well, there are portlets available which we
might include in our portal. It will work”
• End user– „Now, I really don’t care where is the grid
service. I can access it from the portal I used to work with”
![Page 28: Enabling access to global grid](https://reader036.fdocuments.us/reader036/viewer/2022081519/5681444f550346895db0eca9/html5/thumbnails/28.jpg)
Cezary Mazurek, PSNC, Enable access to global grid, Supercomputing 2003, Phoenix, AZ 33
http://progress.psnc.pl/
http://progress.psnc.pl/portal/
Demo:Demo:
SUPERCOMPUTING 2003, Phoenix, SUPERCOMPUTING 2003, Phoenix, SUN Microsystems booth: SUN Microsystems booth: Stand No 14Stand No 14