The UCLA Grid Portal (UGP) and the University of...
Transcript of The UCLA Grid Portal (UGP) and the University of...
1
The UCLA Grid Portal (UGP)and the University of California
Grid (UC Grid)Joan Slottow, Prakashan Korambath, Kejian Jin
UCLA Academic Technology Services
2
The University of California
• UC Office of the President (UCOP)• 10 autonomous campuses• Medical Centers• Lawrence Berkeley National Laboratory
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 2
3
UC-Wide IT Policy and Planning
• IT Leadership Council (ITLC)– UC-wide IT Governing Body– Membership = CIOs and IT leaders from UC
entities.– Strategic planning and priorities
• IT Guidance Committee (ITGC) chargedwith recommending strategic directions toguide IT investments– Issued Report December 2007
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 3
4
ITGC Report
• IT infrastructure is critical– UC Grid is a key enabling technology
• Scalable data center services• Sustainable funding models• Collaboration• Networking (CENIC)
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 4
5
UC must operate as one andleverage the strengths of the 10
campuses• NEED Multi-campus initiatives and
coordination• UC Grid has to be implemented in
collaboration at the campus level
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 5
6
Grid
• Multiple, independent, geographicallydispersed compute resources -- networkedtogether -- used cooperatively
• Autonomous units under local control• Named after the electricity grid
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 6
7
UC Grid• Started at UCLA and currently
implemented at UCLA, UCOP, UCSB, andUCI
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 7
8CENIC 08: Lightpath to the Stars • cenic08.cenic.org 8
9
UCLA Grid Portal (UGP)
• Joins computational clusters into a Grid.• Under development at UCLA since 2002.• Built on top of open source components.• A Globus Incubator Project
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 9
10
Motivation
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 10
HoffmanCluster
HostedCluster1
HostedCluster2
HostedCluster3
HostedCluster4
HostedCluster5
Professor 1’sCluster
Professor 2’sCluster
Professor3’sCluster
Professor4’sCluster
Professor5’sCluster
MAC OS
Myrinet
InfiniBand
???
11
Motivation
• Common interface• Single sign-on• Work with/see all resources at once• Transfer files• Portal makes it easy for users to use
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 11
12CENIC 08: Lightpath to the Stars • cenic08.cenic.org 12
Head nodeCluster II
Head nodeCluster I
ION Visualization ServerStorage ServerMyProxy Server
Grid Portal
Grid Appliance
Credentialmapped tolocal ID
Credentialmapped tolocal ID
Grid Appliance
https
Uniform browserbased interface
Single Campus Architecture
13
UC Grid Architecture
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 13
UC Grid Portal
UCLA Grid Portal others
C1 C2 C3 C4 C5 C6 ………
•User always applies from Campus Portal•One Certificate Authority(CA) throughout UC Grid•User CA is automatically created and pushed to myproxy server•Simple account creation process (approval/denial workflow)•Once the admin approves, the user is able to login to two portals instantly•Enabled by Register Web Service and Sync Web Service in UC Grid Portal
… …
UCSB Grid Portal UCI Grid Portal
14
The UC Grid Portal
• Each user of a Campus Grid isautomatically able to use the UC GridPortal
• From the UC Grid Portal, users can:– work with clusters from all the campuses– transfer files across the campuses.– Use pooled resources across the
campuses.
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 14
15
UGP Services
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 15
16
Resource Discovery
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 16
17
Data Manager
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 17
18
File Transfer
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 18
19
Visualization
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 19
20
Batch Jobs
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 20
Generic
21
Batch Jobs
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 21
Applications and pool Jobs
22
Pool Job Submission• A way to share/use unused cycles.• UGP:
– Chooses the target cluster for the job.– Submits the job to run under a guest
login id there.– Stages the input files.
• UGP sends jobs to local scheduler.Decision for sharing resources made bythe cluster owner.
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 22
23
Application Input Forms
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 23
24
Data Manager and Batch Jobsto Other Grids
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 24
25
Interactive GUI Applications
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 25
26
Xterm
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 26
27
UC System Wide Mandate
• The CIOs of the campuses have agreed
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 27
28
Has to be Carried Out on eachof the 10 Campuses
• Implementation/maintenance/development hasto happen on each of 10 campuses
• People doing the work need:– a connection with the researchers– time to do the work– resources
• ATS at UCLA has the leadership role.
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 28
29
Campus have very DistributedResearch IT Infrastructures
• Individual researchers acquire andmaintain their own HPC resources.
• Most campuses do not have:– central research computing organizations– HPC facilities/resources for the campus
• This is not sustainable• Required:
– Data centers, HP Networks, support staffsCENIC 08: Lightpath to the Stars • cenic08.cenic.org 2
9
30
• UCLA is unique in that it has ATS– Emphasis on research computing
• HPC, Scientific Visualization, Statistics• Close relationship with researchers
– Started as the campus computer center, but:• Remained throughout the era of decentralization• Was on the forefront of: cluster building, cluster
hosting, grid• Has the ability to effect changes• UGP Development
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 30
31
Who to work with on theCampuses
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 31
32
UC Grid Implementers• Monthly Meetings (Skype, ichat, VNC)• Discussion/Mailing List• ATS Provides Support for implementers
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 32
33
The UC Grid Provides for theSharing of System-wide
Resources• The majority of campuses are planning
data center expansions• Research computing needs are growing• The grid provides a way to use virtual
(networked) resources no matter wherethey are.
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 33
34
January 2008 -- UCLA ->Shared Cluster Hosting Model
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 34
New Hoffman2 Cluster
35
Motivation for the SharedCluster Hosting Model
• Manpower to administer multiple hostedclusters >> to administer 1 cluster
• > 50% cycles are idle• Power use by idle clusters is not
significantly lower than for full operation• The ability for others to use your idle
cycles is the price you pay to join theshared cluster
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 35
36
The UCLA Shared ClusterModel scales to the UC
• Cost of data center space/location• Cost of electricity/location• Redundant and wasted capacity• Great need
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 36
Sharing must be attractive to the resourceowners. Only they can make the decision toshare.
37
February 2008 -- UC ResearchCyberinfrastructure Pilot -- Draft
Document
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 37
Phase I:Provide computing resources via two moderate institutionalclusters housed at LBNL and SDSC
Network-based services via UC Grid
Phase II:Expand and develop
38
Learn more about UGP
• UGP documentation, download and wiki
CENIC 08: Lightpath to the Stars • cenic08.cenic.org 38
http://www.ucgrid.org