September 9, 2002 Grid Technology PanelPatrick Dreher 09/02 ID099-1

Technical Panel Discussion:Progress in Developing a Web Services Data Analysis Grid

Patrick DreherResearch Scientist & Associate Director

MIT Laboratory for Nuclear Science

Common Solutions Group MeetingSeattle, WashingtonSeptember 18, 2002

September 9, 2002 Grid Technology PanelPatrick Dreher 09/02 ID099-2

General Goals for Production Grid

• End-to-end integration and deployment using existing and prototype grid services

• Seemless deployment and extensions of grid middleware and extensions between application

• 24x7 service and access to grid• High availability of grid in production mode under

stressful loads

September 9, 2002 Grid Technology PanelPatrick Dreher 09/02 ID099-3

Web Services Data Analysis Grid • Web services data analysis grid is being written,

developed and tested jointly by Thomas Jefferson National Lab (Chip Watson, Walt Akers, Jie Chen, Ying Chen) and MIT (myself)

• Web Services approach chosen because there is wide spread distribution of browsers and powerful tools and applications from multiple vendors

• Traditional 3 tier architecture– 1st tier is the browser– 2nd tier is the Web services– 3rd tier applications, databases, servers, etc.

September 9, 2002 Grid Technology PanelPatrick Dreher 09/02 ID099-4

A Three Tier Web Services Architecture

Web Browser

XML to HTML servlet

Web Service

Application

Web Service

Web Service

Grid Service

Local Backend Services

(batch, file, etc.)

Web Server (Portal)

Authenticated connections

Remote Web Server

Web Service

Storage systemGrid resources,

e.g. Condor

Batch system

September 9, 2002 Grid Technology PanelPatrick Dreher 09/02 ID099-5

Distributed Analysis Environment

• Describe and publish data set (GFN, tag)

• Track and move data (sets) on requests

• Authenticate and authorize

• Submit, control, monitor access to resources

September 9, 2002 Grid Technology PanelPatrick Dreher 09/02 ID099-6

Certificates

September 9, 2002 Grid Technology PanelPatrick Dreher 09/02 ID099-7

September 9, 2002 Grid Technology PanelPatrick Dreher 09/02 ID099-8

Current certificate procedure

• Build strong authentication by combining1. Personal certificate used to create a proxy certificate

2. Permanent certificate w/o the private key

3. CA authority certificate (DOE Grids or JLab)

• Present this strong authentication triplicate to the Web server

• JLab server issues temporary 24 hour certificate• Web server then grants SSL connection• User registration for multi-site access is next step

September 9, 2002 Grid Technology PanelPatrick Dreher 09/02 ID099-9

Authorizations

September 9, 2002 Grid Technology PanelPatrick Dreher 09/02 ID099-10

Authorization Issues• Local vs. grid-wide account management

– Closely tied to local site acceptable use policies – Can either have local or group accounts

• Not every site allows group or pooled accounts

– Determine whether sites be willing (able) to delegate account management to a trusted CA?

– Grid-mapfiles • Needs to be generated and distributed

• Local site needs to be able to access and download a grid-mapfile

September 9, 2002 Grid Technology PanelPatrick Dreher 09/02 ID099-11

Replication and Transfers

• Prototype Web Services Data Analysis Grid is operational at the present time

• Current design allows for automated monitoring of repository directories and disk caches (currently not activated)

September 9, 2002 Grid Technology PanelPatrick Dreher 09/02 ID099-12

Operation of the Replica Catalog

September 9, 2002 Grid Technology PanelPatrick Dreher 09/02 ID099-13

DEMO

September 9, 2002 Grid Technology PanelPatrick Dreher 09/02 ID099-14

Lattice Portal

• Technologies Employed– Apache web server– Tomcat servlet engine (periodically queries

cluster batch systems and produces an XML data structure)

– 2nd servlet applies a style sheet to this XML document

• DEMO (cluster stats and job submission for authorized users)

September 9, 2002 Grid Technology PanelPatrick Dreher 09/02 ID099-15

Theoretical Physics Computational Clusters and

Grid Interconnects• Fermilab and JLAB/MIT designing and operating

clusters for lattice QCD since 1999• Currently 160 Pentium III processors (80 duals) at

Fermilab, 88 ALPHA processors (singles, duals, quads) at JLAB and MIT; connected with Myrinet

• Newly installed 128 node system at JLAB and purchasing 256 node system at FNAL, using dual Pentium IV and Myrinet 2000 interconnect

• Preliminary schedule:

• ~1/4 TFlops sustained in 2002• ~10 TFlops sustained in 2005

September 9, 2002 Grid Technology PanelPatrick Dreher 09/02 ID099-16

Development Milestones

Deploy prototype grid environment in 2002

Improve the robustness, reliability and maintainability of grid software through early use in production application environments.

Integration, coordination, and inter-networking with functional working systems

Integrate with international groups

September 9, 2002 Grid Technology PanelPatrick Dreher 09/02 ID099-17