A framework for immersive FEM visualisation using transparent object

communication in a distributed network environment

Mike Connell*, Odd Tullberg

Department of Structural Mechanics, Chalmers University of Technology, 412 96 Gothenburg, Sweden

Received 14 November 2000; accepted 1 July 2002

Abstract

We present the implementation of a software framework for conducting interactive FE simulations within Virtual Environments (VEs),

and show how this can be used for the visualisation of loading on large-scale structures. Approximation methods that allow results to be

displayed within the VE before the FEA is complete are also discussed. The framework is built using modular object orientated technology,

and exists as a distributed application running over a WAN. Use of modern distributed object libraries allows this parallelism to be largely

transparent within the framework. The use of strictly enforced software interfaces provides a clean separation between our framework and the

modules that provide services and functionality. q 2002 Civil-Comp Ltd and Elsevier Science Ltd. All rights reserved.

Keywords: OOP; Virtual reality; Design; Visualisation; Distributed computing; Java; RMI

1. Introduction

Data visualisation is a vital tool in the process of

presenting the results from Finite Element Analysis (FEA)

simulations to the user. Over the previous years, virtual

reality (VR) systems have begun to allow the visualisation

process to be taken into a virtual setting, where the FE

model exists within some realistic context. The ease of

navigation in the VR environment encourages investigation

and offers a more intuitive viewing of the model.

Additionally collaboration is facilitated by the use of

immersive VR systems, (such as CAVE or CUBE displays)

where a number of people can view the same model

together. The use of VR toolkits, which include support for

remote sessions, even offer the possibility of virtual users:

for example, whilst several people use a CUBE system (and

are physically in the same location), remote participants

may be present in the VE and represented to the other users

as mannequins within it. The use of VR also makes the

results of the analyses much more accessible for people

without a technical background or deep knowledge of the

particular problem, and this consistently proves valuable in

giving demonstrations, and illustrating aspects of the

analyses to visitors.

1.1. Integration in design

We have previously introduced work (for example [1])

regarding the integration of modelling, simulation, visual-

isation and communication tools. It can be summarised as

the conceptual integrated flow as shown in Fig. 1, a

tetrahedron, where the base represents the modelling,

simulation and visualisation in an overall adaptive design

process, with advanced data communication at the top of the

tetrahedron controlling and unifying data flow in the system

under the overall control of the design team. This integrated

VR simulation work forms a part of this larger project—

integrating the simulation and visualisation into a single

user environment.

1.2. Interactive VR

We have worked with the visualisation of a cable-stay

bridge (the Uddevalla Bridge, in south-western Sweden), as

shown in Fig. 2. The first step in this project was the

visualisation of pre-computed FE results in a virtual

environment. After generating an appropriate CAD model

of the bridge, we performed both a linear static analysis and

a modal dynamic analysis of the bridge. We then developed

[2] custom software enabling the visualisation of both scalar

results (as contour bands or iso-surfaces within solid

elements), and displacements (including animated

0965-9978/02/$ - see front matter q 2002 Civil-Comp Ltd and Elsevier Science Ltd. All rights reserved.

PII: S0 96 5 -9 97 8 (0 2) 00 0 63 -7

Advances in Engineering Software 33 (2002) 453–459

www.elsevier.com/locate/advengsoft

* Corresponding author. Fax: þ46-31-772-1976.

E-mail address: mikec@sm.chalmers.se (M. Connell).

eigenmode analyses results). Once this was achieved, we

concentrated on increasing the interactivity of the model

whilst in the virtual environment. It is this interactive

development which will be presented here.

We have been studying the effect of a large load as it

moves across the road-surface of the bridge. We are

interested in the performance of the bridge under this

load, and in particular, we are inspecting the bridge for areas

with an unexpected response. With an interactive VR

environment, we have developed a system, where a user

within the VR model can move the load as they stand upon

the (virtual) bridge—this allows a very natural investigation

of the model. Our current interaction with the model via the

load movement is an early stage in an ongoing investigation

about how we can interact with complex models in a virtual

environment. As the work proceeds, we are intending to be

able to modify both model geometry and analysis

attributes—thus bringing us towards our goal of a real-

time, interactive FE-analysis in a collaborative, immersive

VR environment.

Brooks [3] describes the improved simulation of the VE

as an important technology for VR in general, and that

virtual objects should behave realistically. This is a more

generalised case of what we attempt to achieve—the

accurate behaviour of a specific object within the VE. The

task of achieving a real-time interaction to complex

simulation problems is found in almost every field, where

these simulations occur (for example, surgical training

simulation in Rosen et al. [4]). Closer to our work is that of

Taylor et al. [5,6] or Liverani et al. [7]. However, these

systems work by the close coupling of the customised

software components (visualisation, simulation, VE-inter-

action, etc.), and not within a dynamic reusable software

system such as the one we present.

1.2.1. Immersive VE—the CUBE

There exist a plethora of different hardware platforms for

working with VR. These range from the well-known head

mounted display devices, where the user wears a special

helmet or visor display system, through to larger 3D

projection systems. We have the opportunity to work with a

CUBE projection system that has become central to our

visualisation work. The CUBE system is essentially similar

to a CAVE [8] system—a large-scale VR installation

consisting of a small cube shaped room. A number of walls

and possibly the floor and ceiling are used as projection

surfaces by video projectors coupled to a computer system1

thus people within the room can see only those images

projected by the computer and so can be immersed in a VR.

Our CUBE extends three meters in each dimension. Every

face, with the exception of the top face, is constructed from

a blank material upon which images can be projected. Fig. 3

shows the CUBE system currently installed at Chalmers

University. One side of the CUBE is a hinged door that

Fig. 1. Integrated design.

Fig. 2. The virtual bridge.

Fig. 3. Chalmers CUBE system.

1 In the system we have used, all four walls and the floor are used as

projection surfaces.

M. Connell, O. Tullberg / Advances in Engineering Software 33 (2002) 453–459454

allows entry and exit from the system. To produce the

images that are subsequently projected onto the CUBE

sides, we use a large SGI Onyx2 system, and five high

power video projectors. The images are bounced off mirrors

onto the back face of each surface in order to keep the inside

of the CUBE empty and available for users. Each projection

is done in frame sequential stereo, and lightweight liquid

crystal display shutter glasses are used to decode the display

into a true 3D image. In addition, one user of the CUBE

wears a tracking device, which is used in determining the

projections necessary to create the five images. This user

will have an optically correct view, whilst other users will

perceive visual distortion proportional to their distance from

the tracking unit.

1.3. Frameworks and software design

Rather than to develop a fixed piece of software for this

particular problem and FEA system, we have instead opted

to concentrate on developing a framework within which we

can ‘plug in’ the necessary software modules. That is, we

design a program, where the necessary system modules that

perform the basic tasks are independent and interchangeable

of the program. Our first attempt at such a framework [9]

was a success in terms of the interactive simulation, but was

found to have insufficient separation of the framework and

the modules, and so implementations were prone to ‘infect’

the framework with module dependencies. Subsequently we

re-implemented the framework with the goal of removing

this problem by enforcing greater separation between

framework and modules, and increasing the functionality

of the framework at the same time. Our goal was to perform

these improvements without degrading the efficiency of the

previous framework code. These changes are presented in

Section 2.

2. Framework and modules

We illustrate the architecture of the framework, which

we call iFEM, in Fig. 4. We have partitioned the program

into five modules as follows:

1. FEM—The FEA simulation code is usually operated in

batch mode. We have used the standard FEA tool

ABAQUS ([10]) for the implementation of the FEM

module in this interactive work.

2. VE—The control software is used to render the virtual

world and handle interaction with hardware. We used an

implementation based upon the VR-toolkit dVISE from

PTC ([11]). The use of the dVISE toolkit frees us from

the intricacies of low level graphic programming, and

isolates us from the idiosyncrasies of the VR systems we

use (which may change as VR display technologies

develop [12,13]).

3. Visualisation—Software to convert the numerical results

into a suitable geometric format: the conversion of the

numerical data present in the FEA results files as output

from the FEM module into a visualised form (for

example a 3D world with deformed geometry and/or

stress contours). It is within this module that the user

options (type of scalar result, threshold values, contour

keys, etc.) are applied. Whilst there exist cross platform

specification languages for VR (most noticeably VRML

[14]), they do not offer the full flexibility that we would

like to have available, and so we output data in the native

format for our VE module. We have written our own

Fig. 4. The iFEM framework.

M. Connell, O. Tullberg / Advances in Engineering Software 33 (2002) 453–459 455

visualisation software [2] in the Java programming

language which reads FEA result files (ABAQUS or

ANSYS) and produces VR data files suitable for dVISE.

4. Approximation—The module used to generate inter-

mediate virtual objects for viewing whilst accurate

simulation takes place. This is discussed in more detail

in the following section.

5. Control—A centralised module to handle concurrency

issues, bootstrapping and synchronisation.

Modules typically comprise of two or three distinct parts.

The first is the interface—this is the signature of the module,

and determines its functionality. In terms of the program

code, the interface specifies the method signatures which an

implementation must provide; it is up to the programmer to

develop the program logic in keeping with the intent of the

method as specified by the interface. There is only one

interface for each module. Accompanying the interface is

the implementation, which provides actual code to perform

each method specified in the interface. There can be a

number of implementations for each interface, for example,

the FEM interface has ANSYS and ABAQUS implemen-

tations, both providing similar functionality, but only one of

which is in use at any time. The third component relates to

the accessing of external programs outside the framework.

For example, where we use the ABAQUS implementation

of the FEM interface, it is the task of the ABAQUS

implementation to communicate directly with the actual

ABAQUS program, and so the module acts as a translator.

In doing so, it isolates the framework from any reference to

ABAQUS, as the only reference made to the FEM module is

through the FEM interface, and not through the ABAQUS

implementation.

The interfaces of the framework modules are shown in

Table 1. For example, the new Control module exposes just

six methods (four of which are very similar). In comparison,

ComputeServer module of the first framework [9], which

performs equivalent functionality, exposed 13 methods.

Whereas the ComputeServer performed actual computation,

Control now delegates responsibility to the other modules.

The ComputeServer became infected with implemen-

tation details, and so dependant on specific implementations

because of an insufficiently rigorous enforcement of the

separation between interface and implementation.

Examples of the dependencies that were observed include:

† The process method was responsible for starting a new

analysis task. It had a direct reference to an ABAQUSJob

object, and thus was directly linked to the ABAQUS

implementation. In the new Control module, this task is

carried out directly within the ABAQUS module

abstracted behind the FEM interface (Table 1, FEM,

compute).

† The showCachedResult method was a direct implemen-

tation of an approximation function. In the new frame-

work, this task is abstracted by the Approximation

interface (and the Cached implementation), (Table 1,

Approximation, approximate).

† Local variables referencing directories for the cache,

dVISE data files, FEM file templates. In the new

framework, these are encapsulated within the specific

implementations of the Approximation, VE and FEM

interfaces, respectively.

With sufficient effort, the ComputeServer module could

have been kept free of these dependencies. However, this

Table 1

Module interfaces

Interface Methods Function

Control bindModule(String,Module)

startup

Initialisation methods used at boot time

Module getFEM( )

Module getVisualiser( )

Module getVE( )

Module getApproximator( )

Return references to these modules, allowing modules

to communicate directly with each other

Approximation VR approximate(DLoad) Returns an approximated result of a given input case

double getDistance(DLoad) Returns a measure of the distance

between a given input case, and

the best result the approximation module

can offer

add(Dload, UCD) Add a new case (input parameters

and computed results) for future use

FEM UCD compute(DLoad) Perform a new FEA on specified input parameters

VE reload(String)

move(Point)

Modify specified objects in the virtual world

Visualiser Mesh viz.(Mesh, UCD) Convert numerical results to virtual world objects

M. Connell, O. Tullberg / Advances in Engineering Software 33 (2002) 453–459456

would require a positive conscious effort on behalf of the

person implementing modules, something not always

available with a casual user, or when time is short. In

contrast, the iFEM system naturally provides this separation

without user effort: instead of having to expend effort to

maintain the separation between the framework and specific

implementations, the user must now expend it in order to

override this separation. This is analogous to encapsulation

in object orientated programming—it is possible to break it

by a deliberate act, but it generally does not happen by

accident.

2.1. The approximation module

The FEA of our model is generally too time consuming

for us to be able to merely wait for the analysis to complete

when a user changes the loading pattern—we require the

model to respond in real-time. In order to achieve this, we

have used the concept of an Approximator—a module,

whose task is to provide approximate results in a timely

manner, ideally basing these upon knowledge of previous

similar analyses results. The nature of the approximation

depends on the concrete implementation of our module, but

could for example include:

† None—Modifying the world to contain a 3D represen-

tation of the well-known 2D ‘hourglass’ icon that

represents that the system is computing. Note however

that, whilst a 2D application is usually unresponsive to

the user at this point, the user in the VE can still navigate

and interact with the VE.

† Cached—Using the closest known results from a

previous simulation.

† Linear—Using a numerical interpolation of the results of

similar previous simulations.

† Decimation—Using a mesh of reduced complexity.

We have implemented the Cached method for our

interactive simulation. In pre-computing, we execute the

FEM module several times before entering the VR system,

each time using a different load position. These positions are

spread over the surface of the bridge. It is possible that the

user will position the load at one of these pre-set points, and

so we will conveniently have a result set ready for display.

However, this is unlikely, and the main purpose is to provide

reference data, which can be used to construct some

approximation of the analyses results as the load is moved.

Whilst in the VR system, movement of the load instigates

the execution of a new analysis using the new load position.

This analysis is executed in parallel with the user’s actions

in the bridge—the user continues to interact with the VE

whilst the analysis runs. As it will be some time before the

results from these analyses are available, we display

approximate results immediately—thus giving the user

immediate feedback of the effect of the load movement.

Naturally, when the FEM module has completed the

simulation for the current loading condition, the approxi-

mated results can be replaced with the exact results and

these are also added to the set of initial results—thus

building up the number of known cases that can be used for

approximation. In this way, the approximated results can (in

general) be expected to become more accurate as the model

is used—giving the impression that the system is learning to

simulate the behaviour of the model more accurately.

2.2. Interfaces and customisation

Each interface must be able to offer methods which can

control features unique to an implementation, or common

amongst a subset of implementations. For example,

specifying the position of a point load on a node can be

treated as a common attribute for the FEM interface, and so

we can add this as a direct method to the interface—

meaning that every implementation of the interface must

supply this functionality. In this case, we must be careful to

only include methods which we can be sure will be available

under every implementation. However, where the feature is

only applicable to a single implementation, we provide a

method, where individual parameters can be set as proper-

ties in the implementation. For example, to the interface we

would add the methods:

String getProperty(String name);void setProperty(String name, Stringvalue);

These allow us to pass arbitrary pairs of string values to

each implementation. They are then used as follows:

impl.setProperty(“ABAQUS.output.fil.binary”, “true”);

So, from any module, we can set the value of

“ABAQUS.output.fil.binary” to “true”, regardless of the

actual underlying implementation class. If that class is the

ABAQUS implementation, it will understand the meaning

of the above property and act accordingly, otherwise the

setting will be ignored. It is desirable to use these properties

as infrequently as possible in order to allow the interoper-

ability of modules based upon their interface, and we have

currently found that our use of them is minimal—the above

example being the only case for our ABAQUS

implementation.

2.3. Module communication

We have implemented communication between modules

using remote method invocation. This is a high-level

communication system that allows a method on one

machine to call methods on a remote machine, with no

added complexity in the program logic. For example, within

the VE module running on machine A, in JVMA:

M. Connell, O. Tullberg / Advances in Engineering Software 33 (2002) 453–459 457

···// Instruct Control to start a simulation// with the current parameterscontrol.start ( );

And in our Control module running on machine B, in JVMB:

···// Method that begins a simulationpublic void start ( ) {fe.writeInputFile( ); // with current

// detailsfe.start ( );···

}

When the VE module running on machine A calls the

start( ) method, the function will execute on machine B,

returning any results transparently to machine A. It should

be noted that this is no ‘magic bullet’—although the method

provides an excellent opportunity for distributed design, the

fundamental problem of communication speed between

machines still remains. Thus, we are careful to design our

interfaces so that large data structures are not passed as

parameters. As each module in our framework is largely

self-contained, this has been relatively easy to accomplish.

Parameter passing and the return of values are handled

through object serialisation. This is a standardised method

of reading and writing the state of an object through a

stream, and thus allows objects to be transmitted during

module communication.

3. Results

We present results for the Control, FEM, Visualisation

and VE modules all running on the same SGI Origin 2000

machine equipped with 22 CPUs and 9 Gb of RAM. We

also provide results with the VE module running remotely

on a SGI O2 machine, connected via 100 MB Ethernet, and

with the Visualisation module running on a single processor

Pentium-II PC. Example output of the software in use is

shown in Figs. 5 and 6. Table 2 contains the timed results for

various modules, and the overall latency of the system in

various configurations. Each result is numbered for

reference in the analysis that follows.

Result 1 shows the value of Tsim: the time required to

execute the FEA, from the time the FEM interface receives

the command to start, to the time the job terminates. This

was measured using the system timer, and was a consistent

value.

Results 2 and 3 show the value of Tvis: the time required

to take the numerical result files as created by the FEM

module and convert them to a suitable format for loading

into the VE. The much larger value, in the case where the

module runs on the Origin machine (result 2) instead of the

PC (result 3), clearly demonstrates that Java applications are

at the mercy of an efficient JVM implementation for the

platform upon which they run. The time is also extended by

the method used to interface with the VE module—during

visualisation we must generate text files which are then

processed by a number of standalone executables that are

part of the dVISE toolkit.

Qinteract is the total time lag between the load movement

in the VE, and the corresponding updating of the world with

a new model based upon the updated load position, and

hence the most important value—the ‘bottom line’. As this

value is the one of greatest interest, we present 4 values for

different configurations as results 4–7.

Results 4 and 5 show the cases for uncached viewing

(where the Approximation module cannot deliver a suitable

approximate result, and so the system must wait for the FEA

to complete). Here the time is dominated by the Tsim

component. In result 4, the visualisation module is executed

Fig. 5. Altering the bridge load in a VE. Fig. 6. Altering the bridge load in a VE.

M. Connell, O. Tullberg / Advances in Engineering Software 33 (2002) 453–459458

on the Origin machine, and so the value of Qinteract is mainly

a product of Tsim (result 2). In result 5 the visualisation

module is moved to the PC, and value of Qinteract is much

smaller, as we would expect from the difference between

results 2 and 3.

Perhaps more interesting, the time for cached viewing is

shown in results 6 and 7. In result 6, the VE module executes

on a different machine to the FEM module and so the

Qinteract time is dominated by the transfer of the VE object

files from the cache on the machine running the FEM

module to the machine running the VE module. In result 7,

all modules run on the same machine, and so this operation

is significantly faster (copying from one file system to

another residing on the same machine). This could further

be optimised by (for example) the use of symbolic links [15]

instead of the actual duplication of the data.

We expect that the Qview (latency of changes in head

position and orientation to the updating of a new view of the

VE) in our system to be comparable to that in other

examples, being in both cases dwarfed by the Tsim

component. Indeed, one of the benefits of utilising the VE

module is that we can use commercial software that is

designed to minimise the Qview value. For example, dVISE

contains code to lower the visual quality of objects within

the VE rather than reduce the interactive framerate below a

certain threshold.

4. Conclusions

For interactive work, the performance on non-cached

cases is less than ideal, the cached times are quite acceptable

for the remote case, and excellent for the native case, and

both show some promise as a teaching, illustration or

discussion tool even without further improvements. Whilst,

a pre-cached environment is essentially the same as a VE,

where all the simulation possibilities are done as a pre-

processing step. The fact that the cached entries are

generated on demand is a definite advantage (in that no

set up stage is required. If the cache is deleted, or trimmed or

capped to a specific size, the re-creation of the cache is both

automatic, and demand driven).

The results show that when the time required for

simulation (Tsim) and visualisation (Tvis) can be minimised,

high levels of interaction are possible. From the results, (in

particular Table 2, result 7) it can be seen that the

framework system itself does not impose prohibitive

overhead. The software framework has proved beneficial

in the development of modules, and increased the ease of

replacement of modules providing duplicate functionality.

References

[1] Connell M, Kettil P, Tagnfors H, Tullberg O, Wiberg N-E. Integrated

modelling, simulation and visualisation in immersive virtual environ-

ments—a tool in bridge design. Computational methods for shell and

spatial structures: IASS-IACM 2000, Chania-Crete, Greece; 2000.

[2] Connell M, Kettil P. Visualisation in virtual environments. Proceed-

ings of Modern Konstruktionsteknik’99; 9 November 1999. Report

99:2. Goteborg, Sweden: Department of Structural Mechanics,

Chalmers University of Technology.

[3] Brooks FP. What’s real about virtual reality? IEEE Comput Graph

Appl 1999;19(6):16–27.

[4] Rosen JM, Saltonian H, Redett RJ, Laub DR. Evolution of virtual

reality. IEEE Engng Med Biol 1996;15(2):16–22.

[5] Taylor VE, Stevens R, Canfield T. Performance models of interactive

immersive visualization for scientific application. International

workshop on high performance computing for computer graphics

and visualisation, Swansea, UK; 3–4 July 1995.

[6] Taylor VE, Chen J, Huang M, Canfield T, Stevens R. Identifying and

reducing critical lag in finite element simulations. IEEE Comput

Graph Appl 1996;16(4):67–71.

[7] Liverani A, Kuester F, Hamann B. Towards interactive finite element

analysis of shell structures in virtual reality. Proc IEEE Inform Visual

1999;340–6.

[8] Cruz-Neira C, Sandin DJ, DeFanti T. Surround-screen projection-

based virtual reality: the design and implementation of the CAVE.

Proc SIGGRAPH 1993;135–42.

[9] Connell MA, Tullberg O. A framework for the interactive

investigation of finite element simulations within virtual environ-

ments. In: Topping BHV, editor. Proceedings of Engineering

Computational Technology, Leuven 2000: developments in engin-

eering computational technology, 3/1. Civil-Comp Press Ltd; 2000.

p. 23–8. ISBN 0-948749-70-9.

[10] Hibbitt, Karlsson, Sorensen, Inc. ABAQUS: standard user’s manual;

1996, www.abaqus.com.

[11] Division Ltd. dVISE user guide: geometry tools for UNIX work-

stations; 1998, www.division.com, www.ptc.com.

[12] Connell MA. Virtual reality simulations for finite/discrete element

problems. PhD Thesis. Department of Civil Engineering, University

of Wales, Swansea, UK; October 1998.

[13] Sowizral H. Scene graphs in the new millenium. IEEE Comput Graph

Appl 2000;56–7.

[14] Carey R, Bell G, Marrin C. ISO/IEC 14772-1:1997. The virtual reality

modelling language (VRML), http://www.vrml.org.

[15] Richard WR. Advanced programming in the UNIX environment.

Reading, MA: Addison-Wesley; 1992. ISBN 0-201-56317-7.

Table 2

Results

Result Job Mean time (s) Platform

1 Tsim (time required for FEA) 14 ^ 1 FEM—origin 2000

2

3

Tvis (time to convert FEA

results into virtual objects)

100 ^ 5

13 ^ 0.5

Visualisation—origin 2000

Visualisation—PC

4 Qinteract (total delay between user

modifications, and the display

of results)

Uncached 119 ^ 5 Control, FEM, Visualisation, Approximation—origin 2000, VE—O2

5 Uncached ,31 ^ 2 Control, FEM, Approximation—origin 2000, VE—02, Visualisation—PC

6 Cached 5 ^ 0.5 Control, FEM, Visualisation, Approximation—origin 2000, VE—O2

7 Cached ,1 Control, FEM, Visualisation, Approximation, VE—origin 2000

M. Connell, O. Tullberg / Advances in Engineering Software 33 (2002) 453–459 459