Post on 31-Dec-2015
description
“How to Connect Campus Grids to the NGS”
Neil GeddesDirector, GOSC
The UK's National Grid Service is a project to deploy and operate a grid infrastructure for computing and data
access across the UK. This development will be a cornerstone of the development of the UK's "e-
Infrastructure" over the coming decade. The goals, current status and plans for the National Grid Service and the Operations Support Centre will be described.
http://www.ngs.ac.ukhttp://www.grid-support.ac.uk
Outline
• Overview of GOSC and NGS• Services & Getting Access• Joining the NGS
– How and why
• The Future– Roadmap for the future
• Summary
GOSC
The Grid Operations Support Centre is a distributed “virtual centre” providing deployment and operations support for the NGS and the wider UK e-Science programme.
- started October 2004
GOSC Roles
UK Grid Services National Services
Authentication, authorization, certificate management, VO management, security, network monitoring, help desk + support centre.
support@grid-support.ac.uk NGS Services
Job submission, simple registry, data transfer, data access and integration, resource brokering, monitoring and accounting, grid management services, workflow, notification, operations centre.
NGS core-node Services CPU, (meta-) data storage, key software
Services coordinated with others (eg OMII, NeSC, LCG, EGEE): Integration testing, compatibility & Validation Tests, User Management,
training
Administration: Security Policies and acceptable use conditions SLA’s, SLD’s Coordinate deployment and Operations
The National Grid Service:Towards the UK's
e-infrastructure
• The UK's National Grid Service is a project to deploy and operate a grid infrastructure for computing and data access across the UK. – Learn what it means– Learn how to do it
• This development will be a cornerstone of the development of the UK's "e-Infrastructure" over the coming decade
NGS “Today”
Projectse-Mineralse-MaterialsOrbital Dynamics of GalaxiesBioinformatics (using BLAST) GEODISE projectUKQCD Singlet meson projectCensus data analysis MIAKT projecte-HTPX project.RealityGrid (chemistry)
Users LeedsOxfordUCLCardiffSouthamptonImperialLiverpoolSheffieldCambridgeEdinburghQUBBBSRCCCLRC.Nottingham…
Interfaces
OGSI::LiteOGSI::Lite
If you need something else, please say!
Country providing resourcesCountry anticipating joining
In EGEE: 113 sites, 30 countries >10,000 cpu ~5 PB storage
Includes non-EGEE sites:• 9 countries• 18 sites
EGEE Resources: Feb 2005
GOSC Management Board - NGS StatusEGEE Third Conference, Athens, 19.04.2005 8
Enabling Grids for E-sciencE
INFSO-RI-508833
Applications
• HEP Applications• Biomed Applications
– imaging, drug discover– mri simulation– protein sequence analyis
• Generic Applications– Earth Observation,
Seismology, Hydrology, Climate, Geosciences
– Computational Chemistry– Astrophysics
• Applications “behind the corner” – R-DIG– BioDCV
0
2000
4000
6000
8000
10000
12000
6/24
/2004
7/1/
2004
7/8/
2004
7/15
/2004
7/22
/2004
7/29
/2004
8/5/
2004
8/12
/2004
8/19
/2004
8/26
/2004
9/2/
2004
9/9/
2004
9/16
/2004
9/23
/2004
9/30
/2004
10/7/
2004
10/14
/200
4
10/21
/200
4
10/28
/200
4
11/4/
2004
11/11
/200
4
11/18
/200
4
11/25
/200
4
12/2/
2004
12/9/
2004
12/16
/200
4
12/23
/200
4
12/30
/200
4
1/6/
2005
1/13
/2005
1/20
/2005
1/27
/2005
2/3/
2005
2/10
/2005
2/17
/2005
2/24
/2005
3/3/
2005
3/10
/2005
3/17
/2005
3/24
/2005
3/31
/2005
Jo
bs
/da
y
LCG/CondorGLCG/OriginalNorduGridGrid3
NGS core nodes:
Need UK e-Science certificate (1-2 days)
Apply through NGS web site (1-2 weeks)
http://www.ngs.ac.uk
Service and Access
What we provide and
how to get permission
NGS Core Services
• Globus Toolkit version 2• Job submission, File transfer, Shell
• Storage Resource Broker• Oracle (9i)• OGSA-DAI• Certificate Authority• Information Services (MDS/GIIS)• MyProxy server• Integration tests and database• Cluster monitoring• LCG-VO
In testing:• VOMS• EDG Resource Broker• Portal(s)
Gaining Access
NGS Partner Sites• Data nodes at RAL + Manchester• Compute nodes at Oxford + Leeds• Compute nodes at Cardiff +
Bristol• Free at point of use• Apply through NGS web site• Accept terms and conditions of
use• Light-weight peer review
– 1-2 weeks
• To do: project or VO-based application and registration
National HPC services
• Must apply separately to research councils
• Digital certificate and Conventional (username/ password) access supported
Affiliate Sites• Access approved projects/VO’s
• All access is through digital X.509 certificates • From UK e-Science CA or recognized peer
Joining the NGS
How, Whyand
The Vision Thing
How to JoinResource providers join the NGS by• Defining level of service commitments through
SLDs• Adopting NGS acceptable use and security
policies• Run compatible middleware
– as defined by NGS Minimum Software Stack– and verified by compliance test suite
• Support monitoring and accounting
Two levels of membership1. Affiliation
• a.k.a. connect to NGS2. Partnership
How to Join - 11. Ask
• Grid Support Centre, Stephen Pickles• Process currently limited only by available efforthttp://www.ngs.ac.uk
How to Join - 21. Ask
• Grid Support Centre, Stephen Pickles• Process currently limited only by available effort
2. Get assigned a buddy• added to rollout mailing list• read the joining guidehttp://www.ngs.ac.uk/man/documents/NGS_Partner_joining_procedure_0.4.pdf
How to Join - 31. Ask
• Grid Support Centre, Stephen Pickles• Process currently limited only by available effort
2. Get assigned a buddy• added to rollout mailing list• read the joining guide
3. Install the minimum software stack• On the resource, or on a gatewayhttp://www.ngs.ac.uk/man/documents/NGS_Minimum_software_stack_0.7.pdf
How to Join - 41. Ask
• Grid Support Centre, Stephen Pickles• Process currently limited only by available effort
2. Get assigned a buddy• added to rollout mailing list• read the joining guide
3. Install the minimum software stack• On the resource, or on a gateway
4. Pass monitoring tests for 7 days
How to Join - 51. Ask
• Grid Support Centre, Stephen Pickles• Process currently limited only by available effort
2. Get assigned a buddy• added to rollout mailing list• read the joining guide
3. Install the minimum software stack• On the resource, or on a gateway
4. Pass monitoring tests for 7 days5. Agree to security and acceptable use policies
• security contact + part of campus security• developing operational security with GridPP, EGEE, OSG
How to Join - 61. Ask
• Grid Support Centre, Stephen Pickles• Process currently limited only by available effort
2. Get assigned a buddy• added to rollout mailing list• read the joining guide
3. Install the minimum software stack• On the resource, or on a gateway
4. Pass monitoring tests for 7 days5. Agree to security and acceptable use policies6. Define a Service Level
How to Join - 71. Ask
• Grid Support Centre, Stephen Pickles• Process currently limited only by available effort
2. Get assigned a buddy• added to rollout mailing list• read the joining guide
3. Install the minimum software stack• On the resource, or on a gateway
4. Pass monitoring tests for 7 days5. Agree to security and acceptable use policies6. Define a Service Level7. Set up service level monitoring
How to Join - 81. Ask
• Grid Support Centre, Stephen Pickles• Process currently limited only by available effort
2. Get assigned a buddy• added to rollout mailing list• read the joining guide
3. Install the minimum software stack• On the resource, or on a gateway
4. Pass monitoring tests for 7 days5. Agree to security and acceptable use policies6. Define a Service Level7. Set up service level monitoring8. Approval by GOSC Board
– Get representation on Technical Board– Process took 6-12 months for core sites– Process took 3-6 months for Cardiff +Bristol– Process should take less than 3 months for Lancaster
Why the NGS
• Common tools, procedures and interfaces– Reduce total cost of ownership for providers– Lower threshold for users
• Early adopter system for UK research grids– technology evaluation (in production)– technology choices– pool expertise– drive interface standards and requirements
• both a voice and a target
– etc.
Why Join• (your) Users increasingly want resources as services and not as
complicated bits of kit– common interfaces across a range of facilities
• Funders of regional and national facilities want common interfaces to lower barriers to access– e.g. Hector
• By joining you leverage the national expertise in running these services– technical advice and support– security procedures and incident response– tools to help monitor and patch
• All of the above required in any TCO calculation– Get it at lower cost by joining the NGS
• Members get a say in the technical decisions …
The Future
More Visionand
Vision Meets Reality
UK e-Infrastructure
LHC
ISIS TS2
HPCx + HECtoR
Users get common access, tools, information, Nationally supported services, through NGS
Integratedinternationally
VRE, VLE, IE
Regional and Campus grids
Community Grids
Maintaining Compatibility
• Operating a production grid means valuing robustness and reliability over fashion.
– bug uncovered in OpenSSH in October 2004• NGS cares about:
– alignment/compatibility with leading international Grid efforts– special requirements of UK e-Science community– easy migration/upgrade paths– proven robustness/reliability– based on standards or standards-track specifications
• NGS cannot support everything• Everyone wants service-oriented grids
– but still settling out: WS-I, WS-I+, OGSI, WSRF, GT3, GT4, gLite• Caution over OGSI/WSRF has led to wide convergence on GT2 for
production grids and hence some inter-Grid compatibility– but there are potentially divergent forces at work
• Significant changes to NGS Minimum Software Stack will require approval by NGS Management Board on conservative time scales
Strategic Framework
• GOSC/NGS UK e-Science project– support other UK (e-)science projects
• International Compatibility – EGEE
• European infrastructure (and possible funding)• LHC at most UK universities
– only user group who want to build the grid– GridPP committed to common w/s plan in 2005
• GEANT
– Others• TeraGrid – US cyberinfrastructure $$$ (unlikely to pay us)• Open Science Grid – will develop compatibility with LCG• RoW e.g. China
– Want use other software, but must be EGEE compatible– Also driven by user requirements – Sets framework for relationship with OMII and others
• Other factors– JISC and Shibboleth
Process for Moving Forward
1. New developments evaluated by ETF– must have some longer term support likely
2. User requests treated on case by case basis3. NGS Technical Board consider against needs
• user demand• new functionality• improved functionality• improved security/performace/managability
4. Proposal brought to GOSC Board • Prepared by GOSC “executive” • N.Geddes, S.Pickles, A.Richards, S.Newhouse
Current Roadmap
• April 2005 Extensions of NGS membership
• July 2005 First assessment of NGS WS infrastructure Outline plans for 2005/2006 NGS AAA devt.Migration and/or interoperability of GT2/GT4
• Oct 2005 Deployment of Resource Brokering and VO mgmtClient (compute element) compatibility with EGEE Decision on support for GT4 infrastructure
• Jan 2006 Evaluation of Shibboleth interoperability solutions
• April 2006 Approval of Shibboleth interoperability plans
• July 2006 Interoperability with Shibboleth infrastructure
• Oct 2006 Technology refresh for core NGS nodes
The beginning …
– Backup slides to follow …
© 2004 Electronic Data Systems Corporation. All rights reserved.
Step 2: Consolidate (Server, Network, Storage, etc)Step 2: Consolidate (Server, Network, Storage, etc)
Step 5: Utility ServiceStep 5: Utility Service
Step 1: Migrate & Manage (Regional Facilities)Step 1: Migrate & Manage (Regional Facilities)
Agility Agility DriversDrivers
• Standards
• Visibility
• Quality
• Security
• Efficiency
Step 4: Virtual Service SuiteStep 4: Virtual Service Suite
Step 3: Automated Operations & Managed StorageStep 3: Automated Operations & Managed Storage
Step 6: GridStep 6: Grid
Reduce Risk
Improve Utilisation
Improve Scalability,
Service Quality/Levels, Productivity &
more
Reduce TCO
EDS is Transforming Clients to Agile Enterprise – Virtualised Computing PlatformEDS is Transforming Clients to Agile Enterprise – Virtualised Computing Platform
EDS Services Transition Roadmap
The GOSC BoardDirector, GOSC (Chair) Neil GeddesTechnical Director, GOSC Stephen PicklesCollaborating Institutions
CCLRC Prof. Ken PeachLeeds Prof. Peter DewOxford Prof. Paul JeffreysManchester Mr. Terry HewittEdinburgh/NeSC Prof. Malcolm AtkinsonUKERNA Dr. Bob Day London College tbd
ETF Chair Dr. Stephen NewhouseGridPP Project Leader Prof. Tony DoyleOMII Director Dr. Alistair DunlopEGEE UK+I Federation Leader Dr. Robin MiddletonHEC Liaison Mr. Hugh Pilcher-Clayton
Also invitede-Science User Board Chair. Prof. Jeremy Frey Director, e-Science Core Programme Dr. Anne Trefethen