Using the VL-EProof of Concept Environment

Connecting Users to the e-Science Infrastructure

David Groep, NIKHEF

Virtual Laboratory for e-Science (NL)

• To boost e-Science by – the creation of an e-Science environment – and doing research on methodologies

• To carry out concerted research – along the complete e-Science technology

chain, – ranging from applications to networking, – focused on new methodologies and reusable

components.

Grid ServicesHarness multi-domain distributed

resources

XXXXXXXXVL-e Application Oriented Services

Food Informatics

Dutch Telescience

Medical Diagnosis &

Imaging

Bio-Informatics

Data Intensive Science/

Bio-Diversity

Virtual Laboratory for e-Science

VL-E in a nutshell

• Experiments become more complex– more than just coping with the data– Computer is integrated part of the experiment– support the experimental process end-to-end

Technology (push)

…Grid

Resource SharingWeb

Networks

Application Needs (pull)

Experiment validationPapers and associated dataProvenance meta-dataInformation modelingData/Resource Collection Access…

interpretation

Rationalization of the experiment and processes via protocols

processingprocessed data

conversion, filtering,analyses, simulation, …experiment

parameters/settings,algorithms,

intermediate results,…

Parameter settings,Callibrations,

Protocols…

software packages,algorithms

…

raw dataacquisition

sensors,amplifiers imaging devices,, …

presentationvisualization, animationinteractive exploration, …

MetadataMuch of this is lost when an experiment is completed.

The Experimental Process

Combining data sources

Key element for all users: Data Combination• From different organisations

– data ownership preserved– data correctness maintained by preventing

‘forks’

• Extracting common meaning– need for workflow definition and

ontologies in collaborative experiments

Combining data in Cognition Science

• Collaborative scientific research– Information sharing– Metadata modeling

• Allows for experiment validation– Independent confirmation

of results• Statistical methodologies

– Access to large collections of data and metadata

• Training– Train the next generation

using peer reviewed publications and the associated data

Combining Acquisition and Simulation

• Robert: kun je hier een mooi plaatje voor maken? Het lijkt me de goede plaats om ook in-silico experimenten even te noemen

Role of the Proof-of-Concept (PoC)

• Platform for user application development• Provisioning network & grid infrastructure

– stable releases of common tools – tested ‘external’ middleware– stable releases of internal developments

• Support for users & dissemination– infrastructure installations– end-user helpdesk– on-site aid in migration

Application development

NL-Grid production clusterCentral mass-storage

facilities+SURFnet

Initial computeplatform

Stable, reliable, testedCert. releases Grid MW & VL-

software

VL-e Proof of Concept EnvironmentVL-e Rapid Prototyping Environment

DAS-2, local resources

VL-e Certification Environment

NL-Grid FabricResearch Cluster

Test & Cert.Grid MW & VL-software

Compatibility

Flexible, test environment

Environments

Usage

Characteristics

Virtual Lab.rapid prototyping

(interactive simulation)

Flexible, ‘unstable’

Download RepositoryPoC InstallerCluster Tools

Developer CVSNightly builds

Unit tests

stable, tested releases

Integration testsFunctionality testsAdventurous application people

PoC Release n

Release Candidate n+1

Developers Heaven/Haven

Tagged Release Candidates

LCG2.x + SRB + LCG2.x + others

GT3.2 + *

externalmiddleware

products

Food Informatics

Dutch Telescience

Medical Diagnosis &

Imaging

Bio-Informatics

Data Intensive Science/

Bio-Diversity

Involving Users

• Training via tutorials on middleware– good attendance, but slow uptake later on

• On-site support in integration– good technology update, but people intensive

• User driven integration: application pull– rapid update, good attendance– requires an ICT scientist to work long-term

with the domain scientists to recognize and extract generic elements

Tutorials

• Grid, LCG2 tutorials • Hands-on event series

‘Grid Admin Nerd Group’

‘After Sales Service’• Documentation• User help-desk (by phone & mail)

User Experience: nice, but information quickly ‘lost’

On-site support

• EMUTD exampleMaurice to provide image & input

• Effective use of EDG/EGEE tools for job submission, SRB for data access

User experience:problem effectively solved!but with high manpower investment by PoC

Grid ServicesHarness multi-domain distributed resources

Managementof comm. & computing

Managementof comm. & computing

Managementof comm. & computing

Potential Genericpart Potential

Genericpart

Potential Genericpart

ApplicationSpecific

Part

ApplicationSpecific

Part

ApplicationSpecific

Part

Virtual Laboratory Application Oriented Services

App

lica

tion

pul

lApplication Pull VL-E methodology

Can we keep our users content?

• Take care of grid & generic aspects– collaboration community building & security– policy-constraint & dynamic resource sharing

• Software Integration– there are many tools already … ‘just integrate them’– but only wide deployment will show the weaknesses

• Make it work– consistent software engineering practices– hide changes lower layers by use of standard interfaces– Easy-to-use installers (PoC Installer, Quattor)– and teach us how to scale up to a grid service provider

http://www.vl-e.nl/