Post on 27-Mar-2015
CPSCG:Constructive Platform for Specialized
Computing Grid
Institute of High Performance ComputingDepartment of Computer Science
Tsinghua UniversityMing CHEN, Guangwen YANG
cm01@mails.tsinghua.edu.cn, ygw@tsinghua.edu.cn
Advanced Grid Research Workshops through European and Asian Co-operation
Outline
1. Introduction
2. Architecture
3. Applications
4. Summary
Advanced Grid Research Workshops through European and Asian Co-operation
Enabling fast, straightforward construction of grid system
• CPSCG is not only another grid middleware, but also a lightweight and ready-to-use platform for directly deploying grid service
• CPSCG provides flexible system-level services to make legacy applications grid-service ready
Advanced Grid Research Workshops through European and Asian Co-operation
Key Features
1. Streamline deployment2. Support both plain web service and WSRF 3. Automate application service visualization4. Fully distributed while maintaining single image
Advanced Grid Research Workshops through European and Asian Co-operation
Design Consideration
1. Enable fast deployment of, and easy access to application services
2. Endorse domain autonomy3. Adapt to dynamic, unreliable, unpredictable Internet
loose-coupled domains, fully distributed system, best effort services
Advanced Grid Research Workshops through European and Asian Co-operation
Outline
1. Introduction
2. Architecture
3. Applications
4. Summary
Advanced Grid Research Workshops through European and Asian Co-operation
Overview
One Platform– Information Center– Front End– Back End
Two Support Tool– Encapsulation– FiTMPI
Advanced Grid Research Workshops through European and Asian Co-operation
Platform: Deployment
DomainFront end
Back EndWindows
Back EndLinux
Information Center
Linux ClusterDomain
Front End
Back EndIA32
Back EndIA64
LAN
WAN/LAN
WAN/LAN
DomainFront End
Back EndPC
Back EndCluster
Information Center
Domain A
Domain B
Domain C
Support Tool
Advanced Grid Research Workshops through European and Asian Co-operation
Se
curi
ty MonitorMonitor ScheduleSchedule
LocalUser`
LocalUser`
LocalServices
LocalServices
LocalJobs
LocalJobs
Local Service Engine Remote Service ProxyDynamic Service Render
Remote DataAgent
Remote DataAgent
Local DataManagement
Local DataManagement
Back EndAgent
Back EndAgent
Back EndAgent
Back EndAgent
Service Layer
Function Layer
SynchronizerService List
Domains List Maintainer
Domains List Maintainer
Information Center
Platform Components
Back EndLocal SchedulerMonitor Local SchedulerMonitor
Advanced Grid Research Workshops through European and Asian Co-operation
Information Centers
• Maintain information of all registered services for service query
• Support UDDI to be compatible with Web Service• Fully duplicated to be resilient to single failure• Synchronize with each other using Bloom Filter to reduce
network traffics and timestamp to keep consistent
Advanced Grid Research Workshops through European and Asian Co-operation
Synchronization
InformationCenter A
ServiceList
Bloom Filter
ServiceList Digest
InformationCenter B
Exchange Digest
Synchronize
Advanced Grid Research Workshops through European and Asian Co-operation
Service QueryJob Management
Status Query
Account ManagementNode Management
Service ManagementAccess Policy Management
Domain Front End
DomainFront End
Back EndPC
Back Endcluster
Domain Administrator
End User
Application
Web service/WSRF
Web Pages
Every front end represents a single-sign point
Advanced Grid Research Workshops through European and Asian Co-operation
One-hop Routing: Improve Reliability
Remote Service Proxy Local Service Engine
Domain A
Remote Service Proxy
Intermediate Domain C
Domain B
Advanced Grid Research Workshops through European and Asian Co-operation
Monitor Architecture
sensor-II data stream interface over LAN (Level-B Communication - TCP)
ComputerCluster
Super ComputerOr other PC
ClusterNodes
sensor-I
FrontEndNode
sensor-II
UDPInterconnection
Level-ACommunication
sensor-I
sensor-IILevel-A
Communication
Aggregator
Advanced Grid Research Workshops through European and Asian Co-operation
Snapshot
Advanced Grid Research Workshops through European and Asian Co-operation
Back End
• Deployed in computing PCs/Clusters, hidden from other domains
• Communicate with its CPSCG front end and execute job from front end
• Support MPI/PBS• Can be another CPSCG front end
Advanced Grid Research Workshops through European and Asian Co-operation
Overview
One Platform– Information Center– Front End– Back End
Two Support Tool– Encapsulation– FiTMPI
Advanced Grid Research Workshops through European and Asian Co-operation
Support Tool: Legacy Applications Encapsulation
• Make legacy applications, especially large volume of command-line based applications, grid service ready
• Fast, straightforward, but flexible encapsulation
Advanced Grid Research Workshops through European and Asian Co-operation
Process of Encapsulation
Command-line Application
Command-line Application
Interpreter&
ServicePackage
Generator
Interpreter&
ServicePackage
Generator
GT4
Tomcat/JBoss
Web Service Package
WSRFPackage
Web Service Package
WSRFPackage
Web Service Package
Web Service Package
InteractiveUser Input
Advanced Grid Research Workshops through European and Asian Co-operation
FitMPI
• Heterogeneous environment• Resource Monitoring• Auto deployment• Communication between intranets• Input/Output Management• Process Management
Advanced Grid Research Workshops through European and Asian Co-operation
A “Hello World”-Like Demo
Install a system
Publish a back end
Deploy a service (Encapsulation): PClustalW
Invoke the service and get the result
Advanced Grid Research Workshops through European and Asian Co-operation
Install
Advanced Grid Research Workshops through European and Asian Co-operation
Publish A Back End
Advanced Grid Research Workshops through European and Asian Co-operation
Deploy Service Step 1: Select An Installed Legacy Application
Advanced Grid Research Workshops through European and Asian Co-operation
Deploy Service Step 2: Edit Service Interface
Advanced Grid Research Workshops through European and Asian Co-operation
Deploy Service Step 2: Edit Service Interface (Cont.)
Advanced Grid Research Workshops through European and Asian Co-operation
Deploy Service Step 3: Publish!
Advanced Grid Research Workshops through European and Asian Co-operation
Invoke ServiceStep 1: Select An Intended Service
Service can also be accessed via Web Service or WSRF
Advanced Grid Research Workshops through European and Asian Co-operation
Invoke ServiceStep 2: Input Parameters
Advanced Grid Research Workshops through European and Asian Co-operation
Get the Result
Advanced Grid Research Workshops through European and Asian Co-operation
Analyze Result with Other Tool
Advanced Grid Research Workshops through European and Asian Co-operation
Application: BioInformatic Grid
Phrap
Cap3
Blaster
PClustalw (MPI)/Clustalw
…
Advanced Grid Research Workshops through European and Asian Co-operation
Summary
1. CPSCG provides a platform helping end users fast and efficiently deploy grid system and grid services
2. CPSCG Supports domain autonomy via loose-coupled architecture, while offering global single image
Thank you!