Constructing Campus GridsExperiences adapting myVocs to UABgrid
John-Paul RobinsonHigh Performance Computing Services
Office of the Vice President for Information TechnologyUniversity of Alabama at Birmingham
Internet2 Spring Member MeetingApril 2007
Overview
UAB CyberInfrastructure UABgrid myVocs myVocs box myVocs box on UABgrid Setting Up a VO Future Directions
UAB CyberInfrastructure
UAB HPC Resources Shared HPC Facility has 4 clusters Computer Science HPC Facility has 2 clusters UAB overall HPC computing power has been
tripling approximately on a 2 year cycle during the past 4 years
Optical Networks – campus & regional UABgrid – a campus computing and
collaboration environment
UAB HPC Resources
IBM BlueGene/L System (most recent) 2 Dell Xeon 64-bit Linux Clusters
128 nodes 4 TB disk storage Gigabit and Infiniband interconnect
2 Verari Opteron 64-bit Linux Clusters 64 and 32 nodes 2 GB RAM per node Gigabit interconnect
IBM Xeon 32-bit Linux Cluster 64 Nodes, Gigabit interconnect
UAB 10GigE Research Network
Build high bandwidth network linking UAB compute clusters
Leverage network for staging and managing grid-based compute jobs
Connect directly to high-bandwidth regional networks
UABgrid
Common interface for access to HPC infrastructure
Leverage UAB identity management system for consistent identity across resources
Provide access to regional, national, and international collaborators using Shibboleth identity framework
Support research collaboration through autonomous virtual organizations
UABgrid Architecture
Leverages IdM investments via InCommon
Provides collaboration environment for autonomous virtual organizations
Supports integration of local, shared, and regional resources
UAB Office of the VP of IT CyberInfrastructure Vision
10 Gigabit Ethernet optical network links major research areas in state
High performance computation resources distributed across state
Campus grids like UABgrid provide uniform access to computational resources
Regional grids like SURAgrid provide access to aggregate computational power and unique resources
Alabama Regional Optical Network
Alabama RON is a very high bandwidth lambda network. Operated by SLR.
Connects major research institutions across state
Connects Alabama to National Lambda Rail and Internet2 – projected completion for 2007
Aggregating Resources
UABgrid 2.0, powered by myVocs, to begin pilot operation Summer 2007
Exploring grid interconnection with Alabama Supercomputer Authority and UA System to aggregate resources in state
Continuing participation with SURAgrid to aggregate resources in region
UABgrid Background
Project grew out of NMI Testbed participation, complemented by participation in developing SURAgrid
Initially an integration of campus identity with grid credentials using Pubcookie to issue certificates from UABgrid CA
Initial tool integration based exclusively on identity
UABgrid CA: credentials used by grid computing courses; part of SURAgrid Bridge CA
Limitations of Initial Version
No virtual organization support or other authorization attributes
UABgrid CA key escrow limits trust Support for non-UAB users limited Inter-domain trust via web user interface
doesn't scale well
Complimentary Activities
“NMI Enabled Open Source Collaboration Tools for Virtual Organization” grant explores middleware integration (2003)
Mailing list system integration discussions in Internet2 Mlist working group leads to “Shibboleth Systems” insights (2004)
myVocs.org developed as demonstration of Shibboleth system (2005)
GridShib collaboration expands system reach to Globus-based grid resources (2006)
myVocs box built to ease deployment (2006)
“Shibboleth System”
Simplified, strict “federation” of one identity provider (IdP) with many resources providers reflects trust model of traditional system environments
Using Shibboleth for intra-system attribute transfer supports applications distributed across domain boundaries
The system can receive outside attributes from standard Shibboleth IdP federations
Essentially a proxy identity provider
myVocs
Demonstration virtual organization collaboration environment at myVocs.org
Use Shibboleth for identity management and attribute distribution
Leverage wealth of open source web applications for VO collaboration tools
Globus provides distributed computation foundation
GridShib binds Shibboleth and Globus for common attribute foundation
myVocs Solves the Attribute Puzzle
IdP1IdP1
IdP1IdP2
IdP1IdPn
Identity Providers
UnivAttributes
VOAttributes
myVocs Solves the Attribute Puzzle
IdP1IdP1
IdP1IdP2
IdP1IdPn
Identity Providers
UnivAttributes
VOAttributes
myVocs Solves the Attribute Puzzle
IdP1IdP1
IdP1IdP2
IdP1IdPn
App1
Appn
App2
Identity Providers Applications
UnivAttributes
VOAttributes
A Look Inside myVocs
UABIdP
OtherIdPs
OpenIdP
UIUCIdP
Shibboleth SP
VO IdP with GridShib
VO Attribute Store
VO SPVO SPVO SP
MailList
VO SP
Wiki
VO SP VO SPVO SPVO SP
CMS
VO SP VO SPVO SPVO SP
GridApps
Globus SP
myVocs
myVocs
myVocs is a “modern application environment” (in spirit of RL Bob's Middleware picture from this morning)
Collaboration application scalability Many users, many organizations, many tools,
many kinds of existing infrastructure Deployment manages application access
myVocs box
A virtual machine instance of myvocs.org Instantiates working federated platform Allows stand-alone exploration of
federation middleware Simplify construction of federated system
environments Support development of federated
applications Conceptualize complex federations as
simple federations in layers
myVocs box Contents
Debian GNU/Linux minimal system install Shibboleth IdM infrastructure Simplified group management with
Sympa Dynamically allocated collaboration tools GridShib CA and IdP interfaces Short-circuit identity provider Basic tools to support stand-alone
operation
Running myVocs box
Download virtual machine image from http://myvocs-box.myvocs.org
Run it with VMware Player or Server Put myvocs-box IP in /etc/hosts Point browser at http://myvocs-box Explore VO management & sample web
tools
UABgrid 2.0
Use of myVocs collaboration environment architecture resolves limitations of initial version
Leverage myVocs box instance as the VO management platform
UABgrid CA aligned with PKI-lite GridShib CA supports grid credential
assignment without key escrow InCommon federation supplies identities
and other useful attributes
UABgrid and myVocs
UABIdP
OtherIdPs
Shibboleth SP
VO IdP with GridShib
VO Attribute Store
VO SPVO SPVO SP
WebApps
VO SP VO SPVO SPVO SP
GridApps
Globus SP
UABgrid running myVocs box
Know the network profile configuration Import myVocs box into local namespace Integrate with local trust environment Hook in identity providers Establish virtual organizations Migrate existing resources Integrate new resources
Network Profile
Default ports HTTP, HTTPS, SSH. OK No firewall rules. OK Public default root password. Not OK
Import into Namespace
“Import” into namespace means assign appropriate local host name
Host name change affects system, web server, Shibboleth, and messaging
System name is standard host name change process
Web server has static rule with default host name Shibboleth has host name in config and metadata Messaging requires Sendmail to masquerade as
new host name and to listen on external interface
Integrate with Local Trust Environment
UABgrid CA defines PKI trust environment for hosts and users on UABgrid
UABgrid CA will define trust foundation for myVocs box and UABgrid metadata
Migration from default myVocs box trust configuration delayed temporarily to speed exploration of other parts of implementation
Default myVocs config “works” with a false sense of self
Hook in Identity Providers
The goal is to make UABgrid an InCommon application
InCommon will be primary identity federation for UABgrid
UABgrid operating policy for InCommon is being developed
Initial draft awaiting review Two levels of access with different attribute
requirements: collab tools & compute resources
OpenIdP.org in use for initial testing
Establish Virtual Organization
VOs are easy to create by way of the Sympa interface
HPC Services group has existing virtual organization called the Advanced Technology Lab (@lab)
@lab selected for migration to UABgrid VO (Drupal, mailing list, Connotea, Trac, etc)
6 core members with additional affiliates @lab will be used to manage UABgrid
using UABgrid (eat own dog food)
UABgrid Management Project
cfengine for configuration management All nodes will need Globus + GridShib
stack to accept “management” jobs Authorization to execute jobs comes from
@lab VO role Taking system perspective provides a
simplistic model to support construction of infrastructure
Still early on, but grid management using the grid infrastructure is the goal
Experience: Authentication
Shibboleth clearly sufficient for web applications
User certs via GridShib CA interface good for non-web applications
Flexible yet consistent session lifetime management needed – can be achieved for now via published practices
Essentially, authentication needs can be pretty well satisfied with existing technology
Experience: Authorization
Default myVocs authz roles OK for smaller groups (only 3 roles)
No central PDP (each app decides meaning of roles) good for enabling integration rather than enforcing it (applications just receive consistent attributes)
Managing multiple apps independently can be time consuming, use a small number
Experience: Applications
Sample applications in myVocs box are OK for working groups due to scale
Sample web applications dated – the current sample apps need to be updated to latest releases and modernized
Management of some application features requires file system access – need owner/admin file UI for web applications
Need registration UI for additional apps GridShib for Globus is for WS (ie. not SSH)
Experience: Final Thought
Don't get lost in the technology. Shibboleth and Globus are just the means
to building user-driven, federated system environments
Remaining Tasks
Integrate myVocs box with UABgrid trust fabric
Migrate existing applications used by @lab – requires some development work to address Shibboleth support
Integrate additional resources – on-going evaluation of application needs for this and other VOs
Migrate other existing working groups to UABgrid 2.0 (a.k.a. buy-in)
The Future
UABgrid 2.0 Pilot begins summer 2007 Explore grid-based integration with UA System
and Alabama Supercomputer Authority Recruiting additional manpower
myVocs box Will continue to be leveraged on UABgrid for
development efforts and improved as VO management platform
Performance of VM analyzed Ease of administration improved
Shibboleth trust management, additional attributes
Acknowledgments
NSF ANI-0330543 “NMI Enabled Open Source Collaboration Tools for Virtual Organization”
Office of the Vice President for Information Technology, University of Alabama at Birmingham
Projects: SURAgrid, GridShib, Internet2 People: Jill Gemmill, Tom Scavo, Von
Welch, Jim Phelps, Michael Schiffers, David Shealy
Top Related