GCRC Meeting 2004

BIRN Coordinating Center Software Development

Vicky Rowley

Incorporate software from any organization, source or binary, via CVS or data grid

Track version control at both entire system level and sub-component level

Allow continual improvement of the BIRN software infrastructure in terms of performance, robustness and interoperability

Support demonstration of latest development efforts without disruption to development effort or production operation

BIRN Development Environment Objectives

BIRN Development Infrastructure

Shared CVS Repository facilitates collaborative development of core system tools and utilities and accomplishes version tracking

Separately instantiated development, staging and production environments provide robust, yet flexible areas for developing, testing, and deploying BIRN system software

Simple Cluster

Frontend

General Compute

General Compute

Networked Storage

General Compute

General Compute

General Compute

General Compute

General Compute

User Workstation

System Deployment

BIRN specific extensions to Rocks grid management software, also under CVS, means automated, repeatable deployment of any version of the BIRN system

Rocks, with BIRN extensions, includes automated deployment mechanism for• Middleware (Security, Computational, Data)• Data mediation/integration• Application codes• Portal and other Workflows

BIRN Grid

Web/Portal

App/Mediator

DB Server

Frontend/GPOP

Nettools

Gen Compute

Networked Storage

Frontend/GPOP

Nettools

Gen Compute

Networked Storage

Frontend/GPOP

Nettools

Gen Compute

Networked Storage

Frontend/GPOP

Nettools

Gen Compute

Networked Storage

…

..

.

System Testing

Provides flexibility necessary to support testing efforts• Hardware can be reinstalled and reconfigured quickly and

easily• Supports short-term specialized testing (network issues,

database upgrades, etc.)• Supports “rolling baseline”, or instantiation of everyone’s

latest and greatest, until software release testing begins

BIRN System Software Releases

Standardizing software delivery process provides semi-annual software releases April & October• Developing internal processes for alpha, beta and

production releases• Improving documentation and tutorials for all components

BIRN 2.0 due in April• Software from testbeds and open source from the

community, as well as software for data grid, security, grid operations, etc., is included as part of the integrated BIRN software release

GCRC Meeting 2004

BIRN Coordinating Center Data Grid

Vicky Rowley

Local workstation

Simple FTP

FTP Server

User: Anonymous

Passwd: None

Path: /pub/dir1/dir/2 . . .

FTP Collaboration

Local workstation

FTP Server

FTP Server

FTP Server

FTP Server

FTP Server

User: vickyr

Passwd: *****

Path /pub/vickyr

User: rowleyv

Passwd: *****

Path /pub/rowleyv

User: vrowley

Passwd: *****

Path /pub/vrowley

User: vmrowley

Passwd: *****

Path /pub/vmrowley

User: vicky

Passwd: *****

Path /pub/vicky

Data Sharing Issues

Authentication – who is requesting my data? Authorization – are they allowed to access it? Auditing – who has used my data for what? Organization – where is it? Network transmission – how long will it take? Scalability – How many times do I need to repeat

this?

FTP vs. data grid

Account on each system, or anonymous access

Need knowledge of each systems directory structure

Read/write access Individual/group/all

. Directories represent

physical storage Doesn’t scale well

One account recognized on all systems

Individual directory structures abstracted away

Read/write/annotate/audit Individual, group, domain,

public Folders represent

collections of related data Better scalability

BIRN Data Grid

SRB Server

SRB Server

SRB Server

SRB Server

SRB Server

MCAT

BIRN Portal

BIRN Data Grid

Client/Server with data catalog GSI enabled Access control beyond Windows/Unix

• Read, write, annotate, audit, etc.• Individual and/or group access

Easy to use interfaces (programmable, full-featured interfaces, too)

Functionality to ease network loading (e.g. proxy commands and bulk transfers)

Scalable (with improved scalability on it’s way)

Monitoring Grid Status

0.7 TB

5.2 TB

0 TB

1.6 TB

0.8 TB

0.8 TB

3.2 TB

0.8 TB

2.4 TB

0.8 TB

0.8 TB

2.4 TB

1.6 TB

0.8 TB

5.0 TB

0.78 TB

0.08 TB

Performance and Trouble Shooting

3.6 Million Files

5.6 Terabytes Used

Logical Resources

Stability

BIRN Grid is up and available approximately 98%

Example of typical site uptime: