Open Grid Computing Environments: Advanced

Gateway Support Activities

Marlon Pierce, Suresh Marru, Raminder Singh, Gerald Guo, Archit Kulshrestha

and collaborators.

Software Description

OGCE Gadget Container

Google Gadget/Open Social compatible software for building Web-based user interfaces.

Xbaya A visual user interface for composing, launching and monitoring workflows

GFAC An application factory service for wrapping command-line tools as Web services

XRegistry; Registry Gadget

A service and workflow registry and its user interface

Experiment Builder User interface for creating online experiments with registered workflows

Software Strategy• We build downloadable software• Focus on gadget container and tools for running

science applications on grids and clouds.• Provide a tool set that can be used in whole or in

part.– If you just want GFac, then you can use it without

buying an entire framework.• Outsource security, information services, data

and metadata, etc to other providers.– MyProxy, TG IIS, Globus, Condor, XMC Cat, iRods, etc.

NSF SDCI Award: 2010-2013

• IU: Marlon Pierce (PI), Suresh Marru (Co-PI), Raminder Singh, Rob Quick, Gregor von Laszewski, Gerald Guo

• Purdue: Carol Song (Co-PI), David Braun, Lan Zhao

• NCSA/UIUC: Sudhakar Pamidighantam (Co-PI), Shaowen Wang, Yan Liu

• UTHSCSA: Emre Brookes (Co-PI), Borries Demeler

Award Highlights

• Full Circle Development– Directly fund both software developers and gateway

consumers.• Directly supported (non-IU) gateways:

– UltraScan (UTHSCSA), GridChem (NCSA), SimpleGrid (UIUC), Purdue CCSM and Environmental Gateways

– Among the most used TG gateways.• Sustainability strategy: Apache Incubator for

workflow suite of tools – XBaya, GFac, and supporting services.

UTHSCSA JacintoTerascale storage

Web Server

US LIMS

MySQL DB

User

High Performance Computing Clusters

TeraGrid

TIGRE/Globus Network

GridControl

UltraScan High Level Overview

UltraScan Collaboration

• Immediate Goals: Use GFAC as a replacement job submission service. – GRAM 2, 4, 5 independence– Significant effort into GRAM5

testing on Ranger.• Longer term goals

– Integrate with TG information services to provide better job scheduling.

• OGCE Resource Prediction Service– Support UNICORE job

management. Current Architecture

Gram5 Testing• Objective

– Ultrascan (biochemistry gateway based in UT San Antonio) want to use ranger for their job submission. Ranger does not have GT4 and they don’t have GT2 adaptor and then GT5 was installed on ranger and we considered that as a better option.

– Ultrascan wants to use GFAC to remove their job submission code dependency on GT4

• Steps– Developed Testing harness to run different cases.– Started with small number of jobs and increased the concurrency later– Watched job behavior of the job on resource and monitored the gram log

• There were lot of issue which we found from the logs and working with Globus team to fix them

– Recorded all the job run data to create a google gadget to create graph for different runs on different resources.

Patterns:

TG Resources and patternsVersion Resource EndpointGT 5.0.2 QueenBee queenbee.loni-lsu.teragrid.org:2120/jobmanager-pbs

GT 5.0.2 Ranger login5.ranger.tacc.teragrid.org:2120/jobmanager-sge

GT 5.0.2 Lonestar gatekeeper.lonestar.tacc.teragrid.org:2120/jobmanager-lsf

Concurrent jobs Batch Size Total jobs Job Status Pass : Fail

1 10 10 10:0

3 10 30 30:0

5 10 50 50:0

10 10 100 20:0

20 10 200 40:0

50 10 500 100:0

100 10 1000 200:0

200 5 1000 Not tested (Need allocation)

500 2 1000 Not tested (Need allocation)

• There were problems with sge.pm to load system environment and user environment.

• You can't load modules using Gram parameters. Ranger all the libraries are as modules and user may want to load some special module version for his job. Example is if you have different version of your code using different modules.

• On Ranger user default shell is c shell and Gram runs using bash shell. This is not a problem but users need to be informed where to add user environment.

• You can look at detailed results on followinghttp://teragridforum.org/mediawiki/index.php?title=GRAM_5_Testing

Learning from this exercise on Ranger

GridChem Science Gateway• A chemistry/material Science Gateway for running computational

chemistry codes, workflows, and parameter sweeps.• Integrates molecular science applications and tools for community

use. • 400+ users heavily using TeraGrid. One of the consistent top5

TeraGrid Gateway users.• Supports all popular Chemistry applications including Gaussian,

GAMESS, NWChem, QMCPack, Amber and MolPro, CHARMM• ParamChem is a follow-on project to develop workflows for

chemical parameter studies and provide the infrastructure to execute them.

Cyberenvironments for ParameterizationComputational Reference Data Generation

GridChem , ParamChem Integration

• Immediate Goals: integrate XBaya workflow composer with GridChem user interface and middleware.– Target SC10

• Longer term goals: provide part of GridChem next generation middleware; support large scale ParamChem workflows.

SimpleGrid (UIUC)

SimpleGrid Gadgets

Yan and Gerald’s work from our hacking session.

SimpleGrid, GISolve

• Short term goal: develop SimpleGrid Gadgets deployable into gadget container.– Must meet security requirements– Support PHP development– Support interactivity requirements

• Integrate YUI JavaScript libraries with Gadget JavaScript.

• Longer term goals: investigate workflow, job management tools. Apply to GISolve

Purdue CCSM and Data Portals

• Short terms goals: Develop CCSM and data management gadgets and necessary backing middleware.– Interactivity and security requirements.– Significant requirements overlap with SimpleGrid

• Longer term goals: Build gateways out of gadgets hosted by multiple containers; examine workflow and other tools.

Conclusion• We focus initially on one component per

gateway.– SimpleGrid, CCSM, Data Portal: gadgets

• Other gadget based gateways at UC– GridChem: Xbaya– UltraScan: GFac

• Goal is to establish an Apache-style meritocracy for contributed code.

• Making distributed teams work: hacking retreats.

Backup Slides

2020

OVP/RST/ MIG

OGCERe-engineer, Generalize,

Build, Test and Release

LEAD

OGCE Gateway Tool Adaption & Reuse

GridChem

TeraGridUser Portal

OGCE Team

GridChem

Ultrascan

BioVLab

ODI

Bio Drug Screen

EST Pipeline

Future Grid

GFac, XBaya, XRegistry, FTR

Eventing System

LEAD

Resource Discovery Service

GPIR, File Browser

Gadget Container, GTLab, Javascript Cog,

XRegistry Interface, Experiment Builder, Axis2 Gfac, Axis2 Eventing System,

Resource Prediction Service, Swarm

Experiment Builder, XRegistry Interface

Xbaya, GC Middleware

GFac, Eventing System

XBaya, GFac

Workflow Suite, Gadget Container

Swarm->GFac

Swarm->GFac

GFac, Xbaya, …

Putting It All Together

Software Strategy

• Focus on gadget container and tools for running science applications on grids and clouds.

• Provide a tool set that can be used in whole or in part.– If you just want GFac, then you can use it without

buying an entire framework.

• Outsource security, information services, data and metadata, etc to other providers.– MyProxy, TG IIS, Globus, Condor, XMC Cat, iRods, etc.

Advanced Support Scenarios

• GridChem/ParamChem workflow support• UltraScan Job Submission (GFAC)• EST Pipeline

– Bioinformatics pipeline for managing mass job submission.

More Information• This is downloadable, packaged software.

– Apache Maven build system provides everything you need to to build the gadget container, gadgets, workflow composer, and backing services.

– Get code by anonymous SVN checkout.• Email: mpierce@cs.indiana.edu,

smarru@cs.indiana.edu, ogce-discuss@googlegroups.com

• OGCE Web Site: www.collab-ogce.org• Blog/News Feed:

http://collab-ogce.blogspot.com/

Acknowledgements and People

• Funding by TeraGrid GIG, RP and by OCI SDCI• IU: Marlon Pierce, Suresh Marru, Raminder

Singh, Archit Kulshrestha, Zhenhua Guo• TACC: Maytal Dahan, Rion Dooley• SDSC: Nancy Wilkins-Diehr, Jeff Sale• SDSU: Mary Thomas

GridChem Advanced Support

• IU Personal – Suresh Marru, Marlon Pierce, Raminder Singh, Ye Fan

• Gateways Area Director – Nancy Wilkins-Diehr• GridChem Team – Sudhakar Pamidigantam,

Rion Dooley, Vikram Gazula & Rest of GridChem team.

Empirical ForceFields Parameterization Need Process

Vanommeslaeghe et al. J. Comp.Chem 2010, 31, 671-690

Published by AAAS

A. J. Stone Science 321, 787 -789 (2008)

Fig. 1. Errors (V) in electrostatic potential on a surface at 1.8 times van der Waals radii around N-methyl propanamide for two models. (Left) Point charges; (right) charge, dipole, and quadrupole on C, N, and O; charge and dipole on H. The errors are much reduced in the multipole approach

Lack of Accurate Force Fields Produce Erroneous Property Estimation

Cyberenvironments for ParameterizationComputational Reference Data Generation

Molecular Force Field CyberenvironmentsParameter Initialization and optimization Workflow

Parameter definitions

Model/Reference Data Definition

Merit Function Specification

Consistency Checker

Optimization Methods Choice

Optmization Job Launcher

Update Parameter Database with new set

Workflow Manager

Optimization Incomplete?

Paramater testing Model

Successful Testing

Optimization Monitor

Optimization Job Completed?

Paramater Sensitivity Analysis

Notification of End of Workflow

Expert Interface

3030

OVP/RST/ MIG

OGCERe-engineer, Generalize,

Build, Test and Release

LEAD

OGCE Gateway Tool Adaption & Reuse

GridChem

TeraGridUser Portal

OGCE Team

GridChem

Ultrascan

BioVLab

ODI

Bio Drug Screen

EST Pipeline

Future Grid

GFac, XBaya, XRegistry, FTR

Eventing System

LEAD

Resource Discovery Service

GPIR, File Browser

Gadget Container, GTLab, Javascript Cog,

XRegistry Interface, Experiment Builder, Axis2 Gfac, Axis2 Eventing System,

Resource Prediction Service

Experiment Builder, XRegistry Interface

XBaya

Gfac, Eventing System

XBaya, GFac

Workflow Suite

Workflow Suite

Workflow Suite

???

Compute ResourcesCompute Resources

Resource Middleware

Resource Middleware Cloud Interfaces Grid Middleware SSH & Resource

Managers

Computational Clouds

Computational Grids

Gateway SoftwareGateway Software

User Interfaces

User Interfaces

Web/Gadget

Container

Web Enabled Desktop

Applications

User Managemen

t

Auditing & Reporting

Fault Tolerance

Application Abstractions

Workflow System

Information ServicesMonitoring

Registry Security

Provenance & Metadata Managemen

t

Local Resources

Web/Gadget

Interfaces

Gateway Abstraction Interfaces

Cyberinfrastructure Layers

Color Coding

Dependent resource provider components

Complimentary Gateway Components

OGCE Gateway Components

LEAD/OGCE Layered Workflow Architecture

Workflow Execution &

Control Engines

Workflow Execution &

Control Engines

Apache ODE

Workflow Specification

Workflow Specification

Workflow Interfaces (Design

& Definition)

Workflow Interfaces (Design

& Definition)

PythonBPEL 2.0

BPEL 1.0 Java Code Pegasus DAG

Scufl

XBaya GUI (Composition,

Deploying, Steering & Monitoring) Gadget Interface for

Input Binding

Condor DAGMan

Taverna

Dynamic Enactor

Jython InterpreterGBPEL

Flex/Web Composition

GFac Current & Future Features

Input Handlers

Input Handlers

Scheduling Interface

Scheduling Interface

AuditingAuditing

Monitoring InterfaceMonitoring Interface

Data Management AbstractionData Management Abstraction

Job ManagementAbstraction

Job ManagementAbstraction

Fault Tolerance Fault Tolerance

Output HandlersOutput Handlers

Registry InterfaceRegistry Interface

Checkpoint Support

Checkpoint Support

GlobusGlobus

Campus Resources

Campus Resources

UnicoreUnicore

CondorCondor

Amazon Eucalyptus

Amazon Eucalyptus

Color Coding

Planned/Requested Features

Existing Features

Gateway Computing Environments (GCE10)