SCIRun and SPA integration status

Steven G. Parker

Ayla Khan

Oscar Barney

Scientific Computing and Imaging Institute, University of UtahScientific Computing and Imaging Institute, University of Utah

SCIRun with PtolemyII

Scientific Computing and Imaging Institute, University of UtahScientific Computing and Imaging Institute, University of Utah

SCIRun with PtolemyII

SCIRunJNIActor declares native methods implemented in SCIRun Ptolemy package using JNI interface

Actor accepts ObjectToken encapsulating SCIRunData class: contains type information and raw data

Actor calls native functions that access data object in Java

Scientific Computing and Imaging Institute, University of UtahScientific Computing and Imaging Institute, University of Utah

SCIRun with PtolemyII

SCIRun contains modules that:

• Convert multi-dimensional arrays from Java object to nrrd (nearly raw raster data) format (Ptolemy package)

• Convert nrrd data to SCIRun types (Teem package)

Example of data stored in a nrrdArrow points to the module that reads data from Java

Scientific Computing and Imaging Institute, University of UtahScientific Computing and Imaging Institute, University of Utah

Nearly Raw Raster Data (NRRD) Format

Library and file format that supports image processing involving N-dimensional raster data and supports a large range of data types

Distributed with Gordon Kindlmann’s Teem library• Teem is used for manipulation of structured

scientific dataCan be thought of as wrappers for raw data and are

easier to create than SCIRun data objects for simple datatypes

SCIRun’s Teem interface has modules that convert nrrds to SCIRun data types

Scientific Computing and Imaging Institute, University of UtahScientific Computing and Imaging Institute, University of Utah

Current StatusSimple initial test case

• randomly generated mesh converted to SCIRun mesh via Teem nrrd to field converter

• requires pre-existing SCIRun network file containing instance of Ptolemy converter module (name and path of saved network set by SCIRunJNIActor)

Scientific Computing and Imaging Institute, University of UtahScientific Computing and Imaging Institute, University of Utah

Current Caveats

• SCIRun thread library vs. native threads used by JNI: interface is currently somewhat unstable

• Data from other actors must be encapsulated in SCIRun data object

• Teem package converters may limit access to full range of SCIRun datatypes

Scientific Computing and Imaging Institute, University of UtahScientific Computing and Imaging Institute, University of Utah

Next Steps

• Insert SCIRun into SPA workflow• Target specific workflows• Expand support for SCIRun data types• Improve SCIRun synchronization with Java• Add ability to send data from SCIRun to Java

actors• Increase efficiency of reading large data sets

from Java• Allow processing of multiple data sets in one

iteration or over multiple iterations

Scientific Computing and Imaging Institute, University of UtahScientific Computing and Imaging Institute, University of Utah

What are Components?No universally accepted definition in computer science research …yet

A unit of software development/deployment/reuse

• i.e. has interesting functionality

• Ideally, functionality someone else might be able to (re)use

• Can be developed independently of other components

Interacts with the outside world only through well-defined interfaces

• Implementation is opaque to the outside world

Can be composed with other components

• “Plug and play” model to build applications

• Composition based on interfaces

Scientific Computing and Imaging Institute, University of UtahScientific Computing and Imaging Institute, University of Utah

What is a Component Architecture?

A set of standards that allows:• Multiple groups to write units of software (components)…• And have confidence that their components will work with other

components written in the same architecture

These standards define…• The rights and responsibilities of a component• How components express their interfaces• The degree to which components are protected from each other• The environment in which are composed to form an application and

executed (framework)• The rights and responsibilities of the framework

Scientific Computing and Imaging Institute, University of UtahScientific Computing and Imaging Institute, University of Utah

What is the CCA?

CCA is a specification of a component environment designed for high performance scientific computing• Specification is decided by the CCA Forum

CCA Forum membership open to all

• “CCA-compliant” just means conforming to the specification

Doesn’t require using any of our code!

A tool to enhance the productivity of scientific programmers• Make the hard things easier, make some intractable

things tractable• Support & promote reuse & interoperability• Not a magic bullet

Scientific Computing and Imaging Institute, University of UtahScientific Computing and Imaging Institute, University of Utah

CCA Philosophy and ObjectivesLocal and remote components

• Support local, HPC parallel, and distributed computing

High Performance• Design should support high-performance mechanisms

wherever possible (i.e. minimize copies, extra communications, extra synchronization)

• Support SPMD and MPMD parallelism• Allow user to choose parallel programming models

Heterogeneity• Multiple architectures, languages, run-time systems

used simultaneously in an application

Integration• Components should be easy to make and easy to use

Openness and simplicity• CCA spec should be open & usable with open software

Scientific Computing and Imaging Institute, University of UtahScientific Computing and Imaging Institute, University of Utah

CCA Concepts: Language Interoperability

Existing language interoperability approaches are “point-to-point” solutions

Babel provides a unified approach in which all languages are considered peers

Babel used primarily at interfaces

C

C++

f77

f90

Python

Java

Babel

C

C++

f77

f90

Python

Java

Few other component models support all languages and data types important for scientific computing

Scientific Computing and Imaging Institute, University of UtahScientific Computing and Imaging Institute, University of Utah

CCA Implementations

• Caffeine (SCMD parallel, Babel or C++)• SCIRun2 (Distributed, MCMD parallel, Babel

or C++, incomplete)• XCAT (Grid-based, C++)• LegionCCA (Grid-based, uses Legion)• Decaf (Single process, Babel)

Scientific Computing and Imaging Institute, University of UtahScientific Computing and Imaging Institute, University of Utah

CCA myths (debunked)CCA is too complicated

• No, code integration is complicated

• Producing quality reusable code is complicated

• CCA makes both easier

• Some implementations are more complicated than they should be

CCA is buggy

• No, but some (all) of the implementations are

• Some are better than others

CCA has high overheads

• No, this has been measured to be about the cost of a C++ virtual function call for most cases

CCA is a data model

• No, but you could use CCA to build one

CCA is a workflow engine

• No, but you could use CCA to build one

CCA is perfect

• No, but it is an open forum

Scientific Computing and Imaging Institute, University of UtahScientific Computing and Imaging Institute, University of Utah

Should I use CCA?

• Of course!

• If you care about interoperable software for high-performance computing

• It won’t solve all of your problems

• If you are trying to build a complex system involving multiple languages

• If you don’t, you might end up creating something just like it

• Please participate in CCA forum and help us improve the specification

Scientific Computing and Imaging Institute, University of UtahScientific Computing and Imaging Institute, University of Utah