CST STUDIO SUITETM 2011MPI Computing Guide

Contents

1 Introduction 3

2 Nomenclature 3

3 Terms 4

4 Prerequisites 44.1 Interconnection Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44.2 Hardware Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54.3 Operating System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

5 Supported Solvers and Features 6

6 Installation of CST STUDIO SUITETM on a Cluster System 66.1 Installation of MPI Frontend Node . . . . . . . . . . . . . . . . . . . . . . 76.2 Installation of MPI Client Node . . . . . . . . . . . . . . . . . . . . . . . . 96.3 Configuration of Remote Login . . . . . . . . . . . . . . . . . . . . . . . . 106.4 Testing your Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

7 Starting MPI Simulations 147.1 Starting MPI Simulations in Interactive Mode . . . . . . . . . . . . . . . . 147.2 Starting MPI Simulations from Command Line . . . . . . . . . . . . . . . 15

8 Updating Your Cluster 158.1 Frontend Node = WindowsR© , Client Nodes = Windows R© or LinuxR© . . . 168.2 Frontend Node = LinuxR©, Client Nodes = LinuxR© . . . . . . . . . . . . . 17

9 Licensing 17

10 Unattended Installation of MPI Service on WindowsR© 18

11 High Speed Interconnect Configuration 1811.1 InfiniBandTM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1811.2 MyrinetR© . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

12 History of Changes 21

October 26, 2010 2

1 Introduction

The term ”MPI Computing” describes a software parallelization approach for cluster sys-tems, i.e. computers connected via a fast interconnection network and used as a single”supercomputer”. A computational task, like an EM simulation, is divided into severalparts and those parts are handled by the machines in the cluster in parallel. Typically, theparts are not completely independent and, thus, the different computers need to exchangedata via the interconnection network and, additionally, have to be synchronized at certainpoints. For this purpose the so-called ”Message Passing Interface” (MPI), which describesa programming interface, has become a de facto standard and because of its importancethis type of parallel computing is often refered to as ”MPI Computing”. This manual con-tains an overview of the MPI Computing capabilities of CST STUDIO SUITETM as wellas detailed installation instructions which help you to properly install and set up parallelsimulations on your cluster system. Please do not hesitate to contact us in case of anyproblems during the setup process or during the simulation itself. In such a case pleasesend detailed information about the problem you are facing to info@cst.com.

2 Nomenclature

The following section explains the nomenclature used in this document.

command Commands you have to enter on a command prompt (either cmd onMicrosoft R©Windows R©or your favorite shell on LinuxR©) are typesetusing typewriter fonts.

<...> Within commands, the sections you should replace according to yourenvironment are enclosed in ”<...>”. For example ”<CST DIR>”should be replaced by the directory where you have installed CSTSTUDIO SUITETM (e.g. ”c:\Program Files\CST STUDIO SUITE

2011”).

Those icons indicate that the following section of the text appliesonly to a certain operating system:

= MS Windows R© = LinuxR©

October 26, 2010 3

3 Terms

The following section contains the definition of some important terms needed to under-stand the rest of the manual.

Cluster Computers connected via a fast interconnection networkand used as a single ”supercomputer”.

Node An arbitrary computer in your cluster system.

MPI Frontend Node The cluster node on which the simulation is started usingthe CST front end (CST DESIGN ENVIRONMENTTM ).

MPI Client Node A cluster node which takes part in a CST MPI simulation.An MPI Frontend Node can be an MPI Client Nodeat the same time.

4 Prerequisites

This section provides you with information about the hardware as well as the softwarerequirements necessary to successfully perform CST simulations on your cluster system.Please ensure that your system is configured according to the following points to avoidproblems during the setup or operation of CST STUDIO SUITETM .

4.1 Interconnection Network

4.1.1 Network Technology

The nodes in your cluster must be connected via a TCP/IP network (Ethernet).CST STUDIO SUITETM 2011 supports high speed interconnection networks for MPI Com-puting such as InfiniBandTM and Myrinet R© , i.e. the data exchange mechanism can circum-vent the network stack of the operating system and work directly on the network adaptersof the high speed interconnect taking advantage of the low latency and high bandwidth ofthe network. However, if your cluster is equipped with such a high speed interconnect makesure that the nodes can also communicate via a TCP/IP link (e.g. Ethernet interconnect)as this is needed by the mechanism which starts simulations using MPI Computing on thecluster.

4.1.2 High Speed Interconnects

• Windows R©

Under the Windows R© operating system, high speed interconnects are supportedthrough the DAPL (Direct Access Provider Library) interface. A DAPL interfaceis provided for example by the OpenFabrics WindowsR© drivers.

October 26, 2010 4

• Linux R©

Under the LinuxR© operating system, InfiniBandTM is supported through theOFEDTM (OpenFabrics Enterprise Distribution) verbs interface. QLogic R© PSM andMyricom R© MX interconnects are supported through the TMI (Tag Matching Inter-face). Other interconnects are supported through DAPL.

4.1.3 Name Resolution

The names of the cluster nodes must be correctly resolved to IPv4 addresses (not IPv6addresses) either using a DNS or appropriate entries in the ”hosts” file of your nodes ifyou use static IP addresses.

4.1.4 Unique Network

All nodes participating in an MPI simulation must be part of the same network, i.e. theMPI Frontend Node must not be a machine outside the cluster which cannot directlyaccess the cluster nodes. A simple check if all the cluster nodes are in the same networkcan be performed using the ping command. To check if <nodeA> can ”see” <nodeB> justenter the following command on a terminal on <nodeA>:

ping <nodeB>

If <nodeB> answers correctly the two nodes are in the same network and can communicateduring an MPI simulation.

4.2 Hardware Requirements

There are no special hardware requirements for the cluster nodes. However, if you intendto use GPU Computing together with MPI please refer to the GPU Computing Guide tolearn more about the special requirements for the GPU hardware.

4.3 Operating System

The following operating systems are supported (32 bit and 64 bit versions).

• Windows R© XP, Windows R© Vista, Windows R© 7

• Windows R© Server 2003 R2

• Windows R© Server 2008 R2

• Red Hat Enterprise LinuxR© 4 and 5

Additionally, MPI Client Nodes must be homogeneous in terms of the operating system(either MS Windows R© (32/64 bit) or LinuxR© (32/64 bit)). The following configurationsare currently supported:

October 26, 2010 5

MPI Frontend Node MPI Client Nodes

5 Supported Solvers and Features

The following table gives you an overview of the solvers which currently support MPIComputing and of the features which are supported with MPI for these solvers.

CST Product Solver Unsupported Features

CST MICROWAVE STUDIO R© T-Solver • Periodic Boundaries

• Network Co-simulation

• Subgridding

CST MICROWAVE STUDIO R© I-Solver Direct (MoM) and Iterative(MLFMM) solvers are sup-ported.

CST MICROWAVE STUDIO R© F-Solver The direct solver is supported.

CST PARTICLE STUDIO R© WAK-Solver • Indirect wake integrationover interfaces

• The features not sup-ported for the T-solverare also not supported inthe WAK-solver.

6 Installation of CST STUDIO SUITETM on a Cluster

System

The following section provides you with step-by-step installation instructions for severaltypical cluster setups. There are different install sets needed to install an MPI FrontendNode and an MPI Client Node.A typical installation is a three step procedure:

1. Installation and configuration of a frontend node, as described in section 6.1.

2. Installation and configuration of all the compute nodes, as described in section 6.2.

3. Configuration of the remote login, as described in section 6.3.

October 26, 2010 6

6.1 Installation of MPI Frontend Node

Please note that an MPI Frontend Node is a machine in your clusterwhich you want to be able to start CST MPI simulations from. Thus, ifyou want to be able to start CST MPI simulations from an arbitrary nodein your cluster (e.g., because this is the requirement of a queuing systemyou are using), then each of your cluster nodes is an MPI FrontendNode and you need to perform the following steps on each of your nodes.

6.1.1 Installation on WindowsR©

An MPI Frontend Node needs a full installation of the CST STUDIO SUITETM programfiles. Thus, install at least the feature set shown in figure 1 on each Windows R© node ofyour cluster which should be able to start a CST MPI simulation. Note that the ”LicenseServer” as well as the ”Dongle Driver” option must only be selected if the node should actas license server for your installation.1

Figure 1: Please install at least the following install set on an MPI Frontend Node

running on Windows R© .

1It will not harm the setup if you select the other features as well.

October 26, 2010 7

6.1.2 Installation on LinuxR©

Please notice that the following workflow assumes that you own a DVD which contains thecomplete LinuxR© version of CST STUDIO SUITETM . The standard DVD containing theWindows R© version does not contain all the files necessary for the LinuxR© MPI FrontendNode installation (it contains only the files needed to install an MPI Client Node).Please contact your CST sales representative or send an email to info@cst.com if you needthe DVD containing the complete CST STUDIO SUITETM files for LinuxR© systems.An MPI Frontend Node needs a full installation of the CST STUDIO SUITETM programfiles. Thus, install at least the feature set shown in figure 2 on each LinuxR© node of yourcluster which should be able to start a CST MPI simulation. Note that the ”LicenseServer” as well as the ”Dongle Driver” option must only be selected if the node should actas license server for your installation.2 If your MPI Frontend Node should also act asan MPI Client Node please also select the ”MPI Client Node” option.

Figure 2: Linux R© install set for MPI Frontend Node. If the node will also act as licenseserver please select additionally the option ”License Server” as well as ”Dongle Drivers”. Ifthe MPI Frontend Node should also be an MPI Client Node select additionally theoption ”MPI Client Node”.

2It will not harm the setup if you select the other features as well.

October 26, 2010 8

6.2 Installation of MPI Client Node

6.2.1 Installation on WindowsR©

To install a machine as MPI Client Node three steps are necessary.

1. The first step is to install the CST MPI service which allows the MPI FrontendNode to connect to the client nodes. For this purpose a special installer executableis provided within the CST installation directory on your MPI Frontend Node.This executable is called ”mpi install.exe”. Double clicking on the executable willstart the installation process. Please follow the instructions of the installer. If youneed to install it automatically on a lot of nodes, please see section 10 for unattendedinstallation. Please note that you need administrative privileges to run the installer.

2. The second step in the installation process is to make the solver executables accessi-ble. This can be performed in two different ways:

(a) The first possible way is to share the CST installation folder on your MPI Fron-tend Node such that this folder may be accessed from the MPI Client Node.This does only work if you have a server operating system on your MPI Fron-tend Node. Desktop operating systems such as WindowsR© 7, Windows R© Vistaor Windows R© XP will produce errors with too many connections to a sharedfolder.

(b) The second possible way is to perform an installation of the CST STUDIOSUITETM executables on the MPI Client Node as well (the same install setas for the MPI Frontend Node in section 6.1.1).

3. The third step is to create a temporary folder, which will be used by the solversduring simulation and which will be specified in the GUI or a configuration file whenstarting the simulation.

6.2.2 Installation on LinuxR©

To install a machine as MPI Client Node two steps are necessary:

1. The first step in the installation process is to make the solver executables accessible.This can be performed in two different ways. Whichever way you choose, please makesure that the installation is visible on all nodes at the same path. The two possibleways are:

(a) Execute the LinuxR© installer which can be found on both the WindowsR© and

October 26, 2010 9

LinuxR© DVD. Select the install option ”MPI Client Node”3. This will install allLinuxR© solver executables. If the MPI Client Node should also act as MPIFrontend Node please refer to section 6.1.2.

(b) Mount an existing CST LinuxR© installation using a remote file system, forexample nfs. Please note that the central CST installation needs to reside on aLinuxR© machine. A Windows R© installation of CST STUDIO SUITETM mountedvia e.g. Samba to a LinuxR© MPI Client Node is not a valid configuration.

You can also mix these two ways of making the solver executables accessible withinone cluster, as long as the installation is available on all nodes with the same path.So for example you can perform an installation on one node to /opt/cst and mountthat to /opt/cst using nfs on all other nodes.

2. The second step is to create a temporary folder, which will be used by the solversduring simulation and which will be specified in the GUI or a configuration file whenstarting the simulation. /tmp will work fine.

6.3 Configuration of Remote Login

6.3.1 Pure WindowsR© Cluster

To start MPI simulations on other WindowsR© machines you have to provide the credentialsof an user account which will be used by CST STUDIO SUITETM to log in to the MPIClient Nodes and start the simulation processes. Make sure that the user account youprovide is known by MPI Client Node machines. This is typically ensured if the useraccount is a domain account. However, if you are providing a local user account for thispurpose please make sure that this account is defined on all MPI Client Nodes identicallyand the password for the account is also the same on all nodes. To register the user accountplease open a terminal (Start→Run→cmd) on the MPI Frontend Node and go to theinstallation directory of CST STUDIO SUITETM then enter the command:

mpiexec -register

You will then be asked for the user credentials and the information you provide will bewritten to the registry (see fig. 3). Please note that this step is mandatory even if yourMPI Client Nodes are all running on LinuxR© . In this case, the user credentials behavejust as a ”dummy” setting. Actually, ssh is used to log in and start processes on MPIClient Nodes running a LinuxR© operating system. Make sure that the user account youregister has at least read access to the installation folder.

3If your MPI Frontend Node is also running on Linux R© please select the ”GUI Frontend” option aswell, as this is necessary to be able to update your installation later on with the service packs provided byCST.

October 26, 2010 10

Figure 3: Register the user credentials.

6.3.2 LinuxR© Cluster

For remote login on a LinuxR© cluster, ssh is used. You need to configure your cluster toallow passwordless ssh access to all MPI Client Node machines. The typical procedureto configure passwordless ssh remote login is:

1. Create a pair of a private and a public key for ssh. Do not use a passphrase:

ssh-keygen -t rsa

This should have created the two files id rsa and id rsa.pub in a directory named.ssh in your home directory.

2. Copy the public key to the file ∼/.ssh/authorized keys on each MPI ClientNode. For example by using

ssh-copy-id -i ∼/.ssh/id rsa.pub <nodename>

As an example:If you have four MPI Client Nodes named node1, node2, node3 and node4, youhave to perform the four commands

ssh-copy-id -i ∼/.ssh/id rsa.pub node1

ssh-copy-id -i ∼/.ssh/id rsa.pub node2

ssh-copy-id -i ∼/.ssh/id rsa.pub node3

ssh-copy-id -i ∼/.ssh/id rsa.pub node4

If you log into your MPI Client Nodes for the first time, it will complain thatthe authenticity of the host can’t be established and it will ask you if you want tocontinue. If this is the case, select yes.

October 26, 2010 11

3. Now you should be able to login to each of your nodes without a password. To testthat, query the hostname of each node using ssh:

ssh <nodename> hostname

This should give you the hostname without prompting for a password.

4. If you have more than one MPI Frontend Node, repeat this procedure on each ofthe frontend nodes.If you want to use your LinuxR© Cluster with a Windows R© Frontend (see section6.3.3), repeat this procedure on each MPI Client Nodes.

6.3.3 LinuxR© Cluster with WindowsR© Frontend

If you use a Windows R© MPI Frontend Node and LinuxR© MPI Client Nodes, moreconfiguration work is necessary.

1. First you need to configure your WindowsR© MPI Frontend Node as described insection 6.3.1 .

2. Second configure your LinuxR© MPI Client Nodes as described in section 6.3.2.Check that you can login from each MPI Client Node to each other MPI ClientNode.

3. You need to allow your WindowsR© MPI Frontend Node remote access to all of theLinuxR© MPI Client Nodes through the software package ’putty’, which is partlyincluded in the installation of CST STUDIO SUITETM .

(a) Create a second pair of a public and a private key for the Windows R© MPIFrontend Node to login to the LinuxR© MPI Client Nodes. To do that,execute puttygen.exe from the installation folder of CST STUDIO SUITETM .Click on Generate (see fig. ??), then move the mouse over the blank areato create some randomness. Then save both the generated public and privatekey to your hard disk. You will be asked for your private key when starting asimulation. Also copy the public key from the text field named "Public key

for pasting into OpenSSH authorized keys file" (see fig. 4) somewhereto store it. You will need it to configure the nodes.

(b) Append the public key from the field named "Public key for pasting into

OpenSSH authorized keys file" to the .ssh/authorized keys file on everyLinuxR© MPI Client Node.

(c) To make the host keys of all LinuxR© MPI Client Nodes known to the WindowsR©

MPI Frontend Node, login to each of the MPI Client Nodes once fromthe Windows R© MPI Frontend Node. To do that, start putty.exe from theinstallation folder of CST STUDIO SUITETM and specify the hostname of theMPI Client Node. Click on Open. The first time you do that, a window will

October 26, 2010 12

Figure 4: Copy the public key somewhere to store it.

pop up asking whether or not to accept the host key of the host. Accept thekey by clicking on Yes. Then you can close the terminal that opens up. Repeatthis for every MPI Client Node. Note that you need to specify the hostnamehere as you will later specify it in the CST STUDIO SUITETM GUI. So if youuse <hostname.domainname> here, you cannot use just <hostname> in the GUIand vice versa.

6.4 Testing your Installation

As soon as you have finished the installation, it is a good idea to test if all your nodes are cor-rectly configured. Please open the CST STUDIO SUITETM frontend on your MPI Fron-tend Node. Either create a small test model, e.g., using ”Macros→Construct→Online”or load a model you would like to test. Open the ”MPI Computing” dialog of your solver(i.e. ”Solve→Transient Solver...→Acceleration→MPI Computing”, see fig. 5). Enter thenames of your MPI Client Nodes as well as the CST installation directory on each node4,and a temporary directory which will be used to store files on the MPI Client Nodesduring the simulation (Please note that this directory must already exist on the nodes asit will not be created automatically!). You can either enter this information for each nodemanually or just enter global settings and apply them to all nodes stored in the list by

4In case of a shared folder (see section 6.2.1) enter the UNC path (network path) to this shared folder.

October 26, 2010 13

Figure 5: MPI Configuration Dialog

right-clicking on the head of each column and selecting ”Inherit Default”. Then click on”Test active nodes” to test all nodes in your list or right-click on the name of a certainnode and select ”Test Node” in the context menu to test a special node, respectively. Thistest checks the configuration of each active node in terms of the MPI simulation require-ments, i.e. the given hostname, installation folder, and temporary folder. Depending onthe number of nodes the test may last for some seconds. If the test was successful you canstart your first MPI simulation.

7 Starting MPI Simulations

7.1 Starting MPI Simulations in Interactive Mode

To activate MPI Computing the MPI option has to be checked in the ”Acceleration”settings of the solver. In addition to this the MPI Client Nodes have to be specifiedin the ”MPI Properties”. For each node, the CST installation folder, a temporary folder,and an architecture has to be specified.5 The cluster settings can be saved to a file withthe ”Export” button. The settings can then be restored using the ”Load File...” button.This exported file corresponds to the CST machine file format that will be described insection 7.2.

5Currently only homogeneous architecture settings for the MPI Client Nodes are supported.

October 26, 2010 14

7.2 Starting MPI Simulations from Command Line

If you start your simulation from the command line you can either use the MPI settingspreviously made in the ”MPI Properties” dialog of your model as described in section 7.1 orprovide the MPI settings using a machinefile. A machinefile can be created by the ”Export”function in the MPI dialog but also a machinefile created by a job queuing system can beused.6 To start your simulation from the command line using a machinefile you can specifythe option

-withmpi -machinefile "<FULL PATH TO MACHINEFILE>" -queuesys "CST"

which will load the MPI settings from the specified machinefile (CST format).7 The quotesare mandatory. The format of the CST machinefile is as follows:

# CSTMachineFile 20080509

# Template for machine configuration in internal CST format.

# The very first line should be as shown, with version numbering.

# It is possible to set common default settings, to be applied to

# all nodes in the cluster, unless differently specified.

# All lines beginning with "#" are comments

#

# The default settings are specified here:

#

# DefaultInstallationFolder <PATH_TO_INSTALLATION_FOLDER>

# DefaultTempFolder <PATH_TO_TEMP_FOLDER>

# DefaultMachineArchitecture <ARCHITECTURE>

#

# Then each machine node is specified as follows:

# HostName # <PATH_TO_INSTALLATION_FOLDER> ; <PATH_TO_TEMP_FOLDER> ; <ARCHITECTURE> ; <ActiveFlag>

#

# Architecture can have following values:

# Windows IA32, Windows AMD64, Linux IA32, Linux AMD64

#

# The <ActiveFlag> can have the following values: Active, Inactive

#

# The following list provides an example for a valid list of machines

# machine1 uses its own settings

machine1 # "C:\Program Files (x86)\CST STUDIO SUITE 2011" ; "C:\temp" ; Windows AMD64 ; Active

# machine2 uses the default settings

machine2 # ; ; ; Active

8 Updating Your Cluster

The software version of your CST installation must be identical on all nodes of the cluster.The way how you install the service packs you get from CST depends on the configurationof your cluster system, i.e. the operating system installed on the nodes.

6Please refer to the Application Note ”Interaction of CST STUDIO SUITETM with External Job Queu-ing Systems” to learn more about the correct configuration of your queuing system for CST simulations.

7Please note that the settings in the machinefile have precedence over the settings made in the ”MPIComputing” dialog.

October 26, 2010 15

8.1 MPI Frontend Node , MPI Client Nodes

In case you have set up your cluster such that the CST STUDIO SUITETM files are ac-cessed by the MPI Client Nodes using a shared folder you only need to update thissingle shared installation. This may be performed automatically or manually using theCST Update Manager.In case you have performed a local installation of CST STUDIO SUITETM on each nodeyou can install the service pack on the MPI Frontend Node and then send the up-date to the MPI Client Nodes using the following procedure. First update your MPIFrontend Node using the automatic update or the CST Update Manager. This updatedinstallation will now act as a reference installation for your MPI Client Nodes. Thecluster update uses the Distributed Computing (DC) system. Thus, you need to startthe CST DC Main Controller on your MPI Frontend Node. For this purpose start the”Main Controller Administration” found in the WindowsR© Start Menu folder of your CSTinstallation. In the DC Main Controller enter the path to your updated CST STUDIOSUITETM installation in the ”Reference Installation” field and a temporary folder. Thenstart the DC Main Controller by clicking on ”Start Server”. Figure 6 shows you how thedialog looks like. After you have started the DC Main Controller open the ”MPI Comput-ing” (i.e. ”Solve→Transient Solver...→Acceleration→MPI Computing”) dialog and pressthe ”Set Main Controller” button. Enter the name of your MPI Frontend Node whereyou have started the DC Main Controller before and press OK. Then press the ”UpdateActive Nodes” button to send the updated executables to the active MPI Client Nodes.

Figure 6: The DC Main Controller GUI must contain the path to your reference installationas well as a temporary folder.

October 26, 2010 16

8.2 MPI Frontend Node , MPI Client Nodes

Please note that the update of your LinuxR© installation can only be performed either bythe user who installed it or by the ”Main User”.8 The update of your LinuxR© installationcan be performed using a script called ”update with supfile” which can be found in yourCST installation folder. Download the service pack from the CST fileserver and make thefile accessible to all your cluster nodes. Log in as root or as ”Main User” to all the clusternodes and enter the command

./update with supfile "<FULL PATH TO SUPFILE>"

Please note that the full path to the file is required and that the quotes are mandatory.

9 Licensing

The number of acceleration tokens in your license determines the maximum number ofnodes which you can use in a single simulation. The number of tokens required to performa simulation is listed in the ”Acceleration” dialog of your solver. Table 1 shows you themaximum number of MPI nodes with a certain number of tokens.

Number of Tokens Max. Number of MPI Nodes

0 1

1 2

2 4

3 8

4 16

5 32

Table 1: The table shows the maximum number of MPI nodes with a certain number oftokens.

8This is the user account you specified during the installation process.

October 26, 2010 17

10 Unattended Installation of MPI Service on Windows R©

If you need to install the CST MPI Service using mpi install.exe on a large number ofMPI Client Nodes, you might consider performing an unattended installation of thisservice. To perform an unattended installation, first create a response file, which storesthe installation settings. Then you can start the installer with that response file. It willinstall the service without any user interaction.

• To create the response file, you need to execute the installer once interactively. Ex-ecute it with the -r option:

mpi install.exe -r

Perform the installation. During the installation, a file with the nameinstaller.propertieswill be created in the same folder as the file mpi install.exe

is in. Make sure you have write access to that directory.

• To perform an unattended installation, make the installer mpi install.exe and thefile installer.properties accessible to the MPI Client Nodes. Then performon each node:

mpi install.exe -i silent -f installer.properties

This will launch the installer in an unattended mode. The installer will not be visible.After some minutes, the installation will be completed.

11 High Speed Interconnect Configuration

Since version 2011, CST STUDIO SUITETM is built on top of the MPI library by IntelR© ,which supports various high speed interconnects. The best available interconnect is choosenautomatically. The configuration necessary for the interconnects to be used is described inthe following section. You can influence the interconnect selection using environment vari-ables. The most important environment variables are given here, for a complete overviewplease see the Intel R© MPI reference manual available athttp://software.intel.com/en-us/articles/intel-mpi-library-documentation/.

11.1 InfiniBandTM

11.1.1 WindowsR©

Under the Windows R© operating system, InfiniBandTM interconnects are supported throughthe DAPL interface. Please make sure your installed driver supports the DAPL interfaceand test that it is working using the tools provided with your driver. Some drivers pro-vided by MellanoxR© do not support the DAPL interface. In that case you might consider

October 26, 2010 18

installing the OpenFabrics drivers from www.openfabrics.org. If you have OpenFabricsdrivers installed, you can verify your installation with the tool dapl2test.exe providedby Openfabrics, as detailed in the following section. If dapl2test.exe succeeds, alsoInfiniBandTM support of CST STUDIO SUITETM should work.

11.1.2 Testing DAPL Support of OpenFabrics Drivers on Windows R©

The tool dapl2test.exe is bundled with the OpenFabrics drivers. This tool can performsome tests between two computers through the DAPL interface. If you have chosen thedefault options during the installation of the OpenFabrics drivers, these tools reside inc:\Program Files\WinOF. To perform a test, you start it in server mode on one of thenodes and run it in client mode on another node:

1. Run the tool in server mode on one of the nodes. To do this, start it without anyoptions:

c:\Program Files\WinOF\dapl2test.exe

2. Then you can perform the test from one of the other nodes. Try

dapl2test.exe /?

For a short usage guide. To perform some basic tests, execute the two commands

dapl2test.exe -T P -s <servername> RR

dapl2test.exe -T P -s <servername> RW

Replace <servername> by the hostname or the IP of the computer on which youstarted dapl2test.exe in server mode. Note that you need to specify the IP overInfiniBandTM

address or a hostname which resolves to the IP over InfiniBandTM address. Thesecommands perform an RDMA Read and an RDMA Write tests.

11.1.3 LinuxR©

InfiniBandTM is supported under LinuxR© through the OFEDTM verbs interface. Thus, if youhave installed the OpenFabrics Enterprise Distribution CST STUDIO SUITETM should al-ready be able to access your InfiniBand network. You can test your InfiniBandTM installationas described in the following section.

October 26, 2010 19

11.1.4 Testing InfiniBandTM on Linux R©

If you have OpenFabrics Enterprise Distribution installed, you have the six tools ib read bw,ib read lat, ib send bw, ib send lat, ib write bw, ib write lat installed on your ma-chine. With these tools, you can test if your InfiniBandTM interconnect is working and youcan test its speed. Usage of all of these tools is identical and shown here on the exampleof ib read lat.

1. Start the executable in server mode on one of your nodes. This is done by juststarting the executable without any options:

ib read lat

2. To start the test, execute the same executable on another node and specify the host-name of the first node, on which the server process is running, on the commandline:

ib read lat <servername>

This should measure the latency of read operations over InfiniBandTM between thesetwo nodes. If the measurement is successful, your InfiniBandTM interconnect works.On some new hardware, it complains about different CPU speeds. To ignore thesewarnings, specify -F on the commandline.

11.2 Myrinet R©

Please contact CST support for more detailed information or consult the Intel R© MPIreference manual.

October 26, 2010 20

12 History of Changes

The following changes have been applied to the document in the past.

Date Description

Apr. 7 2010Updated the list of supported operating systems insection 4.3

.

Sep. 7 2010Adapted this document to CST STUDIOSUITETM 2011

.

October 26, 2010 21