What is campus bridging and what is XSEDE doing about it? Presented by Craig Stewart ([email protected])
Executive Director, Pervasive Technology Institute, Indiana University and XSEDE Campus Bridging Manager
The beginnings of Campus Bridging as a concept….• In early 2009 National Science Foundation’s (NSF) Advisory Committee for
Cyberinfrastructure (ACCI) charged six different task forces: one of those was called Campus Bridging.
• Cyberinfrastructure consists of computational systems, data and information management, advanced instruments, visualization environments, and people, all linked together by software and advanced networks to improve scholarly productivity and enable knowledge breakthroughs and discoveries not otherwise possible.
• The goal of campus bridging is to enable virtual proximity:– the seamlessly integrated use among a scientist or engineer’s personal
cyberinfrastructure; cyberinfrastructure on the scientist’s campus; cyberinfrastructure at other campuses; and cyberinfrastructure at the regional, national, and international levels; as if they were proximate to the scientist.
– When working within the context of a Virtual Organization (VO), the goal of campus bridging is to make the ‘virtual’ aspect of the organization irrelevant (or helpful) to the work of the VO.
3
http://pti.iu.edu/campusbridging/
4
5
Commercial cloud (Iaas and Paas)
Volunteer computing
Workstations at Carnegie research
universities
Campus HPC/ Tier 3 systems
Track 2 and other major facilities
NSF Track 1
0 2000 4000 6000 8000 10000 12000
Nationally accessible cyberinfrastruc-ture as of 2011 - Estimated Computing
Capacity (TFLOPS)
From Welch, V., Sheppard, R., Lingwall, M.J., Stewart, C.A. 2011. Current structure and past history of US cyberinfrastructure (data set and figures). http://hdl.handle.net/2022/13136
(In)adequacy of Research CI
Key observations from ACCI Campus Bridging Task Force (paraphrased):• Aggregate US cyberinfrastructure
inadequate to meet needs• Existing CI not optimally utilized• Many of the challenges have to
do with the existing state of software, security, and policy
• Some reasonable choices well executed now are better than perfect solutions implemented later
Never (10.6%)
Some of the time (20.2%)
Most of the time (40.2%)
From: NSF Advisory Committee for Cyberinfrastructure Task Force on Campus Bridging. Final Report. March 2011. http://www.nsf.gov/od/oci/taskforces/TaskForceReport_CampusBridging.pdf.
XSEDE campus bridging vision & strategy
• Promote better and easier use, via XSEDE and campus bridging tools, of the nation’s aggregate CI resources
• Campus Bridging is more a mindset that should affect most of what XSEDE does rather than a specific set of software modules within the overall set of XSEDE services and products.
• Campus Bridging could be thought of as a very technical and broad approach to useability
• Our goal is going to be to work with the various groups in XSEDE (particularly XAUS, Campus Champions, Documentation / Training) to align activities and communications so that XSEDE collectively does things in a way that achieves the goals set in campus bridging area
• To be conscientiously targeted at Data, HPC, and HTC – probably in that order• Strategy: conscientiously make a small number of reasoned choices, pursue them
with diligence, and reap economies of scale (if things go right) or clear learning experiences (otherwise)
7
XSEDE, its components, ‘within’ and ‘beyond’ XSEDE• XSEDE is “the most advanced, powerful, and robust collection of integrated
advanced digital resources and services in the world. It is a single virtual system that scientists can use to interactively share computing resources, data, and expertise.” The XSEDE grant award from NSF funds its core organizing, support, and management entity.
• Service Providers (part of SP Forum):– ‘Level 1’ Service Providers meet all XSEDE integration requirements . . .
access through the XSEDE allocation process. [e.g. Kraken & Stampede]– ‘Level 2’ Service Providers make one or more digital services accessible
via XSEDE services and interfaces, share one or more digital services with the XSEDE community along with the organization’s local users [IU Quarry, Cornell Red Cloud are examples]
– ‘Level 3’ Service Providers are the most loosely coupled within the XSEDE Federation; they will advertise the characteristics of one or more digital services via XSEDE mechanisms [The CI for the Ocean Observatory Initiative is a possible example]
Challenges regarding campus bridging• It’s not a specific thing. You can’t point to a ‘campus bridge’
the way you can a supercomputer• There is no such thing as a ‘campus bridger’ the way there is a
Campus Champion. • It may make sense to talk about a ‘bridged resource’ • It’s more a mindset toward a particular form of technical
interoperability and usability than it is a specific thing• The hardest thing about campus bridging: explaining a set of
use cases that affects several types of XSEDE activities as campus bridging rather than having A thing to point to
• The second hardest thing: getting colleagues to abandon the idea that groups interested in campus bridging are XSEDE Service Provider wannabes.
XSEDE and systems engineering
• XSEDE is “an organization that delivers a series of instantiations of services to the US research community.”
• Development of these instantiations takes place via a particular systems engineering methodology called Architecture-centric systems engineering
• Use case descriptions => Use case quality attribute scenarios => Level 3 Decomposition documents (UML) => and then on to code and documents!
Campus Bridging use cases • UCCB 1.0. InCommon-based Authentication. Consistent use of
community-accepted authentication mechanisms.• UCCB 2.0. Economies of scale in training and usability • UCCB 3.0. Long-term remote interactive graphic session • UCCB 4.0. Use of data resources from campus on XSEDE, or from
XSEDE at a campus (Two approaches – one of which we hear about from Ian Foster in the next talk)
• UCCB 5.0. Support for distributed workflows spanning XSEDE and campus-based data, computational, and/or visualization resources
• UCCB 6.0. Shared use of computational facilities mediated or facilitated by XSEDE
• UCCB 7.0 Access to resources on a service for money basis (___ on demand).
• CB Prerequisite. XSEDE-wide unified trouble ticket handling
Economies of scale in training and usability from more consistency in cluster configurations
13
• In reality, the four cluster admins depicted here being in agreement are all right.
• Experienced cluster admins all learned how to use what they learn when the tools were still developing, so the tool each sysadmin knows the best is the tool that lets that sysadmin do their work the best
• The only way to develop consistency is to provide installers that will make their work easier
• The XSEDE architecture group is developing installers for file management tools
• XSEDE campus bridging is developing Rocks Rolls cluster build setups (and documentation)
Economies of scale in training and outreach
Image from TeraGridEOT: Education, Outreach, and Training 2010. https://www.teragrid.org/web/news/news#2010scihigh
• Consistency in system setups – local becoming more like XSEDE – should also lead to economies of scale in training
• Materials and trainer expertise will be more easily transportable and extensible
• The campus bridging group plans to work very closely with the campus champions
Shared Virtual Compute Facilities• SVCF – virtual cluster independent of XSEDE
– Can we provide tools that will create authentication screens that look and work like XSEDE login
– Doing this requires supporting multiple authentication mechanisms– Remember: not everyone one wants to have an XSEDE label on their
organization!• SVCF – accepting jobs from XSEDE
– Requires ability for SVCFs to accept jobs (and trust) XSEDE– Requires ability for XSEDE to trust SVCFs– Requires trouble ticket exchange and security notification / response
processes– This sort of SVCF may be a type of entity that one could meaningfully call a
‘bridged resource.’• This use case has High Performance Computing and High Throughput
Computing as two variants. The Open Science Grid = the solution to the High Throughput Variants of this use case!
16
Campus Bridging GFFS pilot program
Texas A&M – use of GFFS with Brazos Cluster Kansas – data transfer within Great Plans and with Indiana
University CUNY – spanning campus to XSEDE resources Univ of Miami – data sharing within campus and across WAN Pilot projects are currently working with XSEDE Operations to
prepare Genesis II Infrastructure for beta usage
Five year goals
• At the end of 5 years, have implemented and socialized a community vision of an integrated national cyberinfrastructure with XSEDE as a critical component – but just a component
• Working with XSEDE and the community as a whole, implement the technology required to support as many of the use cases described here as possible
Next deliverables and next steps
• Paper in the proceedings!• Template for hybrid system description – generalization of the
TACC / IU templates• Podcast• Some real progress on pilots• ROCKS Rolls – mid PY2 first visible results likely• Continued work with Open Science Grid• Campus Bridging objectives go in to the prioritization process
with everyone else’s priorities and we arrive at some estimate of what is feasible over next four years
19
XSEDE Campus Bridging Staff
• Craig Stewart (reports to Scott Lathrop in Scott’s role leading outreach)• Jim Ferguson (works 25% on campus bridging)• Therese Miller – IU overall project lead for XSEDE activities (will be aiding,
expected particularly in regards to campus champions)• Rich Knepper – Manager, Core Services, RT/PTI (0.25 FTE starting next
year)• 1.0 FTE for “ROCKS Rolling” for PY2
20
Please cite as:
Stewart, C.A., R. Knepper, J.W. Ferguson, F. Bachmann, I. Foster, A. Grimshaw, V. Hazlewood and D. Lifka. What is campus bridging and what is XSEDE doing about it? 2012. Presentation. Presented at: XSEDE12 (Chicago, IL, 16-20 Jul 2012). http://hdl.handle.net/2022/14599
All slides (except where explicitly noted) are copyright 2012 by the Trustees of Indiana University, and this content is released under the Creative Commons Attribution 3.0 Unported license (http://creativecommons.org/licenses/by/3.0/)
Thanks
• Campus Champions – whose input has already shaped Campus Bridging activities greatly.• All XSEDE staff• Guy Almes, Von Welch, Patrick Dreher, Jim Pepin, Dave Jent, Stan Ahalt, Bill Barnett,
Therese Miller, Malinda Lingwall, Maria Morris• Gabrielle Allen, Jennifer Schopf, Ed Seidel, all of the NSF program officers involved in the
campus bridging task force activities• All of the IU Research Technologies and Pervasive Technology Institute staff who have
contributed to this entire 2+ year process• Special thanks to CASC members who have participated in one of n information gathering
exercises (where n is large)• NSF for funding support (Awards 040777, 1059812, 0948142, 1002526, 0829462; this
material and ongoing work supported by Award 1053575)• Funding support provided by Lilly Endowment and the Indiana University Pervasive
Technology Institute• Any opinions presented here are those of the presenter or collective opinions of members
of the Task Force on Campus Bridging and do not necessarily represent the opinions of the National Science Foundation or any other funding agencies
22
Top Related