Problem-Solving Environments:

The Next Level in Software Integration

David W. Walker

Cardiff University

31/8/99 David W. Walker, Cardiff University

2

Objectives of this talk

To review the purpose and scope of PSEs

To discuss the requirements for constructing PSEs

To identify the major technologies that will serve as the infrastructure of PSEs

To review the PSEs currently in development and in use

31/8/99 David W. Walker, Cardiff University

3

Why PSEs? Need: enhanced scientific insight; reduced

development costs; improved product quality and industrial efficiency.

Need: transparent means of integrating distributed computers, instruments, sensors, and people.

Need: improved software productivity to extract maximum benefit from advances in computers, networks, and algorithms.

31/8/99 David W. Walker, Cardiff University

4

Why Now?

Confluence of complementary technologies.

Faster networks and communications. Network software technologies such as

CORBA, Java, and XML. “Big Science” is inherently distributed

and collaborative, and needs to migrate to WAN environments to progress.

31/8/99 David W. Walker, Cardiff University

5

What’s the Problem?

High-level, problem-specification languages, often coupled with expert system. For example, PDE solvers, numerical integration, etc.

Problem composition in form of dataflow graph using a GUI. Typically used in modelling and simulation of physical systems.

31/8/99 David W. Walker, Cardiff University

6

PSE Requirements

Expert assistance in problem specification and input.

Transparent access to distributed heterogeneous resources.

Interactivity and computational steering. Advanced/immersive visualisation. Integration with other knowledge

repositories and databases.

31/8/99 David W. Walker, Cardiff University

7

Technologies for PSEs

Hardware: Increasingly powerful computers Increasingly fast networks - gigabit

ethernet, vBNS, etc. Immersive visualisation platforms -

CAVEs, ImmersaDesks, etc.

31/8/99 David W. Walker, Cardiff University

8

Technologies for PSEs

Software: CORBA for transparent interaction

between distributed resources. Java for platform-independent

programming. XML interface specification. MPI for message-passing in SPMD

codes.

31/8/99 David W. Walker, Cardiff University

9

An Example PSE Architecture

Main PSE sub-systems are: Visual Program Composition

Environment (VPCE) for graphically composing applications.

Intelligent Resource Management System (IRMS) for scheduling applications on distributed resources.

31/8/99 David W. Walker, Cardiff University

10

VPCE Overview

GUI is used to build an application from software components - either a java or CORBA object with its interface specified in XML.

Each component may have a performance model and help file.

An annotated dataflow graph is produced that is passed to the IRMS.

31/8/99 David W. Walker, Cardiff University

11

IRMS Overview

IRMS locates software and hardware resources through information servers.

IRMS then schedules components on appropriate resources based on performance models and database of experience from previous runs. Genetic and neural network algorithms may be used.

31/8/99 David W. Walker, Cardiff University

12

The PSE Research Community

European Research Conference on PSEs took place June 1999 in Spain. Next one in summer 2001. http://www.cs.cf.ac.uk/euresco99/

EuroTools SIG on PSEs. http://www.irisa.fr/EuroTools/Sigs/

Cardiff PSE project web site. http://www.cs.cf.ac.uk/PSEweb/

31/8/99 David W. Walker, Cardiff University

13

US Software Infrastructure

Globus: provides core services for grid-enabled computing. http://www.globus.org/

Legion: an object-based metacomputing project. http://legion.virginia.edu/

The Grid is a computational and network infrastructure providing pervasive, uniform, and reliable access to distributed resources.

31/8/99 David W. Walker, Cardiff University

14

European Software Infrastructure

UNICORE: Uniform access to Computing Resources. Aimed at providing uniform, secure, batch access to distributed resources. http://www.genias.de/unicore/unicore.html

POLDER: a more ambitious metacomputing project. http://www.wins.uva.nl/projects/polder/

31/8/99 David W. Walker, Cardiff University

15

European Software Infrastructure

CODINE: resource management system targeted at optimal use of all software and hardware resources in a heterogeneous networked environment. http://www.genias.de/products/codine/

CCS: Computing Centre Software - resource management for networked high-performance computers. http://www.uni-paderborn.de/pc2/projects/ccs/

31/8/99 David W. Walker, Cardiff University

16

European Software Infrastructure

GRD: Global Resource Director for distributed environments featuring policy management and dynamic scheduling. http://www.genias.de/products/grd/

NWIRE: Netwide resources - management system for WAN-based resources. http://www-ds.e-technik.uni-dortmund.de/

31/8/99 David W. Walker, Cardiff University

17

COVISE Visualisation Environment

The Collaborative Visualisation and Simulation Environment is a distributed software environment that seamlessly integrates simulations, post-processing, and visualisation.

COVISE supports collaborative working, and is available commercially. http://www.hlrs.de/structure/organisation/vis/covise/

31/8/99 David W. Walker, Cardiff University

18

Ctadel and PDE Problems

Code-generation tool for applications based on differential equations using high-level language specifications is an environment for the automatic generation of efficient Fortran or HPF programs for PDE-based problems.

Used in HIRLAM numerical weather forecast system. http://www.wi.leidenuniv.nl/CS/HPC/ctadel.html

31/8/99 David W. Walker, Cardiff University

19

An Environment for Cellular Automata

CAMELCAMEL is a CA environment designed for message-passing parallel computers. It hides parallelism issues from a user.

User specifies only the transition function of a single cell of the system with CARPETCARPET, a high-level cellular language. http://isi-cnr.deis.unical.it:1080/~talia/CA.html

31/8/99 David W. Walker, Cardiff University

20

A PSE for Numerical General Relativity

CACTUS is a collaborative software environment for composing applications for the solution of general relativity problems.

Has been used in distributed computing experiments using Globus.

Interactive visualisation important. http://cactus.aei-potsdam.mpg.de

31/8/99 David W. Walker, Cardiff University

21

JACO3: Industrial Design PSE

Java and CORBA based collaborative environment for coupled simulations.

A CORBA based high performance distributed computing environment for coupling simulation codes.

Optimal design of complex and expensive products like airplanes, satellites, or cars. http://www.arttic.com/projects/jaco3/

31/8/99 David W. Walker, Cardiff University

22

A PSE for Stochastic Analysis

Promenvir: Probabilistic mechanical design environment - a metacomputing tool for stochastic analysis.

It can automatically generate a series of stochastic computational experiments, and run them on the available resources

It has been used for optimal design problems in the automobile industry. http://www.cepba.upc.es/promenvir.html

31/8/99 David W. Walker, Cardiff University

23

PSE for Engineering Simulations

JULIUS: Joint Industrial Interface for End-User Simulations.

Integrated HPC environment for multi-disciplinary engineering simulations.

Aimed at reducing design time for industrial products.

End-users are engineers. http://www.6s.org/

31/8/99 David W. Walker, Cardiff University

24

Summary

There is an active body of PSE researchers and developers in Europe.

PSEs are used in science, engineering, finance, and manufacturing.

Current emphasis is on PSE infrastructure and prototypes.

31/8/99 David W. Walker, Cardiff University

25

Future Challenges

Maintaining good, reliable performance in distributed environments important.

Need to integrate third party software. Need visualisation environments that

scale from PC up to immersive systems. Needs standards for interfaces and

interaction between PSEs.