www.csiro.au

OGC and Grid in AU

Dr Robert Woodcock

Executive Manager, e-Science

Stream Leader, Exploration and Mining

Application Access and Workflows

Applications• Web portals

• Desktop applications

Pipelines: data / tool interoperability

• semantics: community agreed standards

• structure: GML, GeosciML for data interchange

• syntax: XML language

• service: WxS, mediator

Traditional Mechanical Modelling Workflow

• “Powerful” desktop computes several models at a time

• Limitations are in the order of ~2 models per week

• Models (mesh + data files) are individually and laboriously constructed

• The manual process is error prone

• Results are manually visualised one at a time

• Screenshots are manually taken and made into “movies”

• Very little, if any, standardised data archiving is done. This results in potential confusion or loss of the originating conditions of the experiments, making it unrepeatable in the long termSlide courtesy of Robert Cheung and Warren Potma

New Refined Workflow

• Parameterised template or wizard driven model geometry/mesh creation

• Boundary condition & model properties parameter sweep utilities

• automatically creates a “family” of model, data files based on varying a set of parameters

• Inversion algorithms

• determine input parameters of future iterations automatically based on the user ranking of previous results

Automated generation of visualisations

Automated movie generation

Automated archiving

3D Time varying volume visualisation

Parameterised Geometry Creation

Multi-site data storage via Storage Resource Broker

Slide courtesy of Robert Cheung and Warren Potma

Client

The Grid Application… Service Interactions

Resource Registry

Data Management Service

HPC Repository

LoginJob

MonitorRun

SimulationEdit Problem Description

Local Repository

Archive Search

Geology S.A

Geology W.AGeochem

N.S.W

Geochem W.A

Information

Authentication Job Management Service

Escript Service

FastfloRT Service

User

Workflow...

Computation

Community Infrastructure

Physical Resource Physical Resource

Semantic Web Semantic Grid

Classical Grid

Increasing scale of data and computation

Sca

le o

f In

tero

per

abil

ity Structured

data

Flat files, unstructured data, images

(Geochem Test Bed)

(NCRIS 5.13)

(APAC Grid Projects)

Classical Web

(SEE Grid)

Minimum accessible through NCRIS 5.13, 5.16: Bandwidth > 1 Gbit/sec, HPC > 800 Gflops, 300-400 TB online storage High

Low

Incr

easi

ng

Str

uct

ure

(Current Web)

Research organisations pressing forward…

AuScope:Earth Model for the Australian Continent

AuScope Infrastructure System

A toolkit for Geoscience Research and Geoscience Applications

minerals, energy, groundwater discovery and management; hazard prediction; environmental monitoring and management

Multiple workflows from observational data to user application

Progressive refinement and enrichment by a community of practice

QLD NSW

APAC Data &Knowledge Gateways

Govt Data &Knowledge Gateways

Industry Data & Knowledge

Gateways

Data and Knowledge Services

AuScope Compute Services

TAS VIC

NT WA

SAGA

QPSF AC3

Environment

MineralsIndustry

Insurance

Expert Scientist

Non-Expert Scientist

UniversityStudent

General Public

A variety of interfaces to suit user capabilities School

Students

TPAC VPAC

IVEC

SoftwareHardware

Finite ElementSolvers Reactive

TransportCode

Fluid FlowModelling

MechanicalModelling

VisualisationTools 3D

ModellingTools

PetroleumIndustry

AuScope

SAPAC APAC

AP

AC

AC

cES

ST

era

wu

lf

APAC

TP

AC

Computing ServicesV

PA

CC

SIR

OA

C3

IVE

C

ChemicalReactions

Coupled Processes

Rock Mechanics

Fluiddynamics

DeformationSurface

ProcessesFluidFlow

SocietalNeed

NumericalBasis

Modelling Codes

ModellingServices

Base Scientific Concepts

Finite Volumes

Inundation

Natural HazardsNatural Hazards

Resources & Mining

Resources & Mining

SustainableEnergy

SustainableEnergy GeodynamicsGeodynamics

TsunamiStorm Surge

Finite Element

Solver(Fastflo)

Reactive Transport

Predictive ore deposition

Mine Waste Disposal

Finite Element Solver (Finley)

Scripting environment for Finite element simulations

(E-Script)

Mine design

Earthquake

AuScope Modelling Framework

MantleConvection

FiniteElements

Chemistry(Log K)

Geodynamics Modelling code

Underworld

Finite Element

Solver(StGFEM)

ESysCrustal

Coupled Mechanics & Reactive Transport

Particle in Cell

Slabsubduction

MantleConvection

Code(CitcomS)

FiniteDifference

Visual-isation

(gLucifer)

SurfaceModelingPackage

(SPModel)

Chemistry(Gibbs)

Particle

Vulnerability Modelling

Interpolation Mesh

The benefits

Benefits for CSIRO and the nation

• Creation of new fields of science enabled by the combination of datasets from many research disciplines.

• Greater science agility by enabling the creation of larger more diverse science teams across Australia and globally

• Greater access to science knowledge by the public

• Greater reuse of science data assets

• Creation of a data processing and distribution models to support CSIRO’s management of Australia’s national science facilities like ATNF and AAHL

eRepository

Gateway

to

Science

Key Challenges

Data acquisition, management and storage

Large data sets (potentially 20 petabytes!)

Structured, spatial scientific databases

Access to

facilities – data stores, HPC, sensors, networks

processing services – simulation, modelling, inversion, and data mining tools

Information services – discovery; structured, spatially enabled, scientific databases

applications – web portals, desktop applications

Integration and Collaboration (separation of concerns)

Community of practice

What do we need?

Unification

OGC and Grid: issues with service chaining and authentication

Adhoc virtual organisations

supporting secure networks over public networks

High performance data transfer

appropriate protocols on appropriate networks

Stateful and asynchronous versions of OGC specs

especially for large data transfers or computational tasks

Toolkits

workflows, User Interfaces…deployment support

Management controls and Scheduling

www.csiro.au

Thank You

Name Dr Robert Woodcock

Title Executive Manager, e-Science

Phone +61 8 6436 8780

Email Robert.Woodcock@csiro.au

Web www.csiro.au

www.seegrid.csiro.au

Contact CSIRO

Phone 1300 363 400

+61 3 9545 2176

Email enquiries@csiro.au

Web www.csiro.au