KAREN

What you can do with an advanced research and education network!

2

Introductions

John and Sam We do not know your science We want to facilitate discussion This is an opportunity to report back to

REANNZ on issues and barriers Who are you?

3

Today’s Plan

Introduction Collaboration – now and in the future Lunch Tools Capability Development Wrap up

4

Introduction

Motivation A paradigm shift

Research Networks E-Research

What is it? International trends

Examples

5

The New Research Paradigm

Credit: GEANT2

6

Case Study: Serious Disease Genes Revealed Wellcome Trust Case Control Consortium 50 research groups 200 scientists DNA from 17,000 patients 15,000 polymorphic

markers Learned more in 12

months than last 15 years

7

Case Study:Functional MRI (fMRI) Data Center Online repository of

neuroimaging data A typical study comprises

3 groups 20 subjects/group 5 runs/subject 300 volumes/run 90,000 volumes, 60

GB raw data 1.2 million files

processed 100s of such studies in total

Credit Ian Foster, University of Chicago

8

www.fmridc.org

9

Global R&E Network Pathways

DISCLAIMER - This network map was a best estimate of expected connectivity for 2005, several changes in connectivity and planned connectivity have happened since it was created

Credit: John Silvester, USC, Chair CENIC

10

Kiwi Advanced Research and Education Network

Credit: KAREN. http://www.karen.net.nz

11

KAREN Went live Dec 2006 Went live Dec 2006 10Gb/s NZ Backbone10Gb/s NZ Backbone $40million, Government Funding$40million, Government Funding $5million Capability Build Programme$5million Capability Build Programme Linking all 8 Universities and all 9 Crown Linking all 8 Universities and all 9 Crown

Research Institutes, + National LibraryResearch Institutes, + National Library ~622Mb/s link to US~622Mb/s link to US ~133Mb/s link to Australia~133Mb/s link to Australia

Credit: KAREN. http://www.karen.net.nz

12

Advanced Research and Education Networks (ARENs) Credit: GEANT2

13

What is e-Research?

Collaboration Access to and management of data and

knowledge Advanced computing methods Shared resources New research techniques

14

Characterising e-ResearchCharacteristic Traditional Research E-Research

Participants Individual researcher or small local research team

Diversely skilled, distributed research team

Data Locally generated, stored and accessible

Generated, stored and accessible from distributed locations

Computation and Instrumentation

Batch compute jobs or jobs run on researcher’s own computers or research instruments

Large-scale, or on demand computation or access to shared instruments

Networking Not reliant on networks Reliant on research networks and middleware

Dissemination of Research

Via print publications or conference presentations

Via web sites and specialized web portals

Credit: Bill Appelbe and David Bannon, Victorian Partnership for Advanced Computing. eResearch: Paradigm Shift or Propaganda? http://www.jrpit.acs.org.au/jrpit/JRPITVolumes/JRPIT39/JRPIT39.2.83.pdf

15

Discussion

Where does your research fit into this characterisation of traditional research and e-research?

How does this compare with the research that you were doing 5 years ago?

16

Current Environment - Set of Tools

experiment

datastorage

analysisemail

websites

videoconference

scientist

instrument

HPC

Credit: BeSTGrid. http://www.bestgrid.org

17

Future EnvironmentResearch Collaboratories

experiment

datastorage

HPC

analysis

messaging

webportals

videoconference

scientistG

rid

Mid

dle

war

e

scientist

scientist

scientist

instrument

Credit: BeSTGrid. http://www.bestgrid.org

18

The Researcher’s View Why do I care?

New collaborative opportunities New funding opportunities NZ competitiveness

What’s in it for me? Key resource is often somewhere else More data, more tools Collaborating with the best

How do I get involved? Move from silo to GRID

Credit: BeSTGrid. http://www.bestgrid.org

19

Example e-Research Projects

BioCoRE SCOOP SEEK/EcoGrid

20

21

BioCoRE Seamlessly access local and remote technology Co-author papers Access high performance computing Share molecular visualisations Chat room Lab book Notifications, etc. http://www.ks.uiuc.edu/Research/biocore/

22

The Control Panel

23

Projects

24

Project Summary Review

State of recent job submissions

Who is logged in What tasks

members are working on

Recent discussion topics

Recent files added to BioFS

25

Project Status See

Current work Future work

Modify Schedule of

upcoming tasks

Display Current task

26

Publishing VMD Sessions

27

Configuring NAMD Simulations

28

Job Management A Grid Portal

Submit web form Monitor progress

BioCoRE Obtains resources Moves files Executes jobs Places results

29

Message Board

30

Lab Book

31

Website Library

32

BioCoRE File System

33

SURA Coastal Ocean Observing and Predicting Programme

34

SCOOP

Promote effective and rapid fusion of observed oceanic data with numerical models

Facilitate the rapid dissemination of information to operational, scientific, and public or private users

http://scoop.sura.org/

35

SCOOP Goals Create an open access, distributed

lab for oceanography by: Supporting data standards development and

implementation Demonstrating benefits/added value of

diverse communities moving to common standards for info exchange

Creating an environmental prediction system –a research tool that can also support relevant agency decision-making to improve society

36

The results of the analysis are visualized and disseminated in a form that can be readily incorporated into decision-support tools used by emergency response personnel.

For verification, all relevant and available observations are aggregated and compared with predictions, which provides a real-time measure of accuracy and quality for the predictions.

Real-Time EnsemblePrediction

Results from each of the predictions in the ensemble are then aggregated for analysis. Results include maps that show the probability of inundation with street level detail.

Each forecast wind field is used as input for numerical predictions of storm surge and wave fields. Because each individual element in this ensemble of surge and wave predictions involves a numerical calculation that could take many hours on a large supercomputer cluster, they are farmed out to the available computational resources within the distributed network.

Hurricane warnings issued by the NOAA National Hurricane Center (NHC) are used to create an ensemble of forecast wind fields.Each of these wind fields represents a plausible set of forecast winds over the entire region of interest for several days into the future.

37

Distributed Facility forCoastal Prediction

windforecastswater level

modelwave watchmodel

OpenIOOSdata

38

Science Environment forEcological Knowledge Aims to extend ecological and biodiversity

research capabilities by fundamentally improving how researchers: gain global access to ecological data and

information find and use distributed computational services exercise powerful new methods for capturing,

reproducing & analysing data http://seek.ecoinformatics.org/

39

SEEK’s Integrated Systems EcoGrid

Next generation internet architecture enables data storage, sharing, access and analysis

Semantic Mediation System Advanced reasoning system determines if

data and analytical components can be automatically used in a selected workflow

Analysis and Modeling System Ecologists design, modify and incorporate

analyses to compose new workflows and models in a visual, automated environment

40

EcoGrid Seamless access to and manipulation of data

and metadata stored at different nodes Authentication via single sign-on Web services for executing analytical pipelines Registry of data and compute nodes Rapid ingest of new data sources as well as

decades of legacy data Extensible relevant metadata based on the

Ecological Metadata Language Data replication provides fault tolerance,

disaster recovery and load balancing

41

Kepler Workflow Tool Example of the 'R' system in a Kepler workflow

42

Things to take away The research lifecycle is changing – an

evolution rather than a sea-change Bigger and more complex problems require

new methodologies and relationships Policy and funding are increasingly

dictating collaboration Advanced networks are essential It’s more about data than technology Many social and organisational factors

43

A Final Message

Credit: GEANT2