Grid Services at NERSC

Shreyas CholiaOpen Software and Programming Group, NERSC

scholia@lbl.govNERSC User Group Meeting

September 17, 2007

NERSC User Group Meeting, September 17, 2007 2

Introduction to Grids

Q: What is a Grid?

A: A Grid is a system that:1) coordinates resources that are not

subject to centralized control …

2) … using standard, open, general-purpose protocols and interfaces

3) … to deliver nontrivial qualities of service.

- Ian Foster, 2002, http://www-fp.mcs.anl.gov/~foster/Articles/WhatIsTheGrid.pdf

NERSC User Group Meeting, September 17, 2007 3

Introduction to Grids

• Grid computing provides the ability to share and aggregate heterogeneous, distributed computational capabilities and deliver them as a service.

• Uniform set of software interfaces to access non-uniform and physically distributed compute and storage resources.

• Resources are part of a larger whole:– Compute Grid– Data Grid

NERSC User Group Meeting, September 17, 2007 4

Software Overview

• Globus Toolkit 4.0.3– GridFTP for data transfer across the

Grid.– GRAM for remote job submission and

batch access.– GSI and X509 certificates for security.

• Using OSG Software Stack based on the Virtual Data Toolkit (VDT).

NERSC User Group Meeting, September 17, 2007 5

Core Grid Services at NERSC

• GRAM Gatekeeper– Allows you to run / submit, monitor and query

jobs on any NERSC machine remotely.– Interfaces with various backend batch systems

(PBS, Loadleveler, SGE) while presenting a uniform interface to the user (Globus RSL).

• GridFTP– Provides high performance data movement

between NERSC and remote site / client system.

– Supports striped, third party and parallel transfers.

NERSC User Group Meeting, September 17, 2007 6

Certificate Services

• All Grid services at NERSC use X509 Grid certificates for authentication and access.

• NERSC Users can acquire a grid certificate from the DOEGrids site (https://pki1.doegrids.org/ca/).

• User must register their certificate with NERSC in NIM.

• Once the certificate has been approved in NIM, user can access any of the NERSC Grid Services with this.

NERSC User Group Meeting, September 17, 2007 7

Certificate Services

• MyProxy Server– Allows users to store their certificates

in a secure fashion, and generate short term proxy certificates that can be accessed from anywhere.

– Users do not have to leave long term certificates lying around on all their client systems.

NERSC User Group Meeting, September 17, 2007 8

System Deployment

PDSF

Cln

GF

HPSS

GK

GF

Cln

Jacquard

Cln

Davinci/NGF

Cln

Bassi

GK

GF

GridFTP Gatekeeper

GK

GF

GK

GF

GK

GF Cln

SGE PBS PBS LoadLev

Cln Globus Client

MPI Jobs

NERSC User Group Meeting, September 17, 2007 9

Grid Clients

• Job Submission Tools– globus-job-submit, globus-job-query– Condor-G

• GridFTP Clients– globus-url-copy– uberftp

• Certificate Management Tools– grid-proxy-init– myproxy-init, myproxy-logon– grid-cert-info

NERSC User Group Meeting, September 17, 2007 10

Open Science Grid

• Distributed computing infrastructure for large-scale scientific research, built and operated by a consortium of universities, national laboratories, scientific collaborations and software developers.

• Researchers from many fields, including astrophysics, bioinformatics, computer science, medical imaging, nanotechnology and physics, use the OSG infrastructure to advance their research.

NERSC User Group Meeting, September 17, 2007 11

NERSC and the OSG

• OSG users have access to compute/storage resources including NERSC.

• NERSC resources are made available to OSG users via Grid Interfaces.

• NERSC users may be able to access the OSG by making a request to the OSG Engagement team.

NERSC User Group Meeting, September 17, 2007 12

Coming Soon …

• Grid Services on Franklin• SRM Access to NERSC Systems

– NGF– HPSS

• NERSC Online CA– Simplify the process of acquiring cert– User will be given a short lived cert

based on LDAP password

NERSC User Group Meeting, September 17, 2007 13

Sample Users

• STAR

• NREL

• Planck

• OSG - WRF

• OSG - LIGO

• ESG

• Incite 8 and 13

NERSC User Group Meeting, September 17, 2007 14

What it means to YOU!

• You can submit, manage and coordinate jobs across NERSC systems (and across other grid sites) using a common interface directly from a client machine.

• Especially useful for managing workflows across multiple systems / sites.

• Job control can be integrated into software running on your desktop.

• The ability to move data across sites using high performance parallel transfers.

NERSC User Group Meeting, September 17, 2007 15

Example

HPSS

Bassi

NGF

DaVinci

RemoteWeb Server

MyProxy

• Get Certificate• Submit Job to Bassi• GridFTP to HPSS• Do analysis on DaVinci• Move results to web server