ALEAR - uni-osnabrueck.deneurokyberne...The simulator should provide interfaces to other scienti c...

EU-Project Number ICT – 214856

ALEAR

– Artificial Language Evolution on Autonomous Robots –

Deliverable D 3.1

– A Simulator for the Humanoid A-Series –

(September 30, 2008)

UOFM - Group (Lead)HUB - Group

Authors: Christian Rempis, Verena Thomas, Christian Thiele, Frank Pasemann

CONTENTS CONTENTS

Contents

1 Introduction 2

2 Simulator Architecture Overview 4

2.1 Modularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.2 Event Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.3 Variable Repository . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.4 Physical Abstraction Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.5 Customization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3 Simulation Model for the A-Series 7

3.1 Actuators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3.2 Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

4 Comparison with Physical Robot 11

5 Simulator Interfaces 19

5.1 ORCS Standard Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

5.2 ORCS Direct Control Protocol . . . . . . . . . . . . . . . . . . . . . . . . . 20

5.3 XML Environment Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . 21

6 ORCS User Manual 22

6.1 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

6.2 Graphical User Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

6.2.1 Camera Control and Keyboard Features . . . . . . . . . . . . . . . . 24

6.2.2 Control Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

6.2.3 Tools Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

6.2.4 View Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

6.3 XML Environment Description V 1.0 . . . . . . . . . . . . . . . . . . . . . . 32

6.4 Working with the Simulator . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

6.4.1 A-SeriesSimulator Basic Usage . . . . . . . . . . . . . . . . . . . . . 36

6.4.2 Interaction with the MotionEditor . . . . . . . . . . . . . . . . . . . 37

6.4.3 A-SeriesClientSimulator Synchronous Control . . . . . . . . . . . . . 39

7 Planned Extensions 42

7.1 Motor Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

7.2 Randomization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

8 Acknowledgments 43

D 3.1 Simulator for the A-Series 1

1 INTRODUCTION

1 Introduction

For its language games the ALEAR project will use humanoids as autonomous robots.There will be two types of robots: the so called A-Series, and a newly designed humanoiddeveloped especially for the ALEAR project, called the M-Series. The behaviors of theserobots will be controlled – amongst others – by neural networks which will be generated byapplying methods of evolutionary robotics. In this context a physical simulator has to beused

• to study gesturing and walking of a given robot for certain given control commands,

• to study these behaviors in dependence of mechanical and electronic properties of arobot,

• and finally, it has to be used to evolve the neural control, which is able to deliver thedesired basic behaviors of the humanoids.

This deliverable describes the features of the simulator (version 1.0, date September 3o). Itincludes a handbook for installing and using the simulator as well as references to availabledemonstrations and videos.

The basic requirements for the proposed physical simulator were given as follows:

• Because different robot platforms will be used by the ALEAR project the simulator hasto be adaptable to the perhaps varying properties of employed sensors and actuators.Therefore the simulator should be highly modularized; and it should be easy to modifyall relevant physical and morphological parameters of the simulated robot.

• For use in an artificial evolution scenario, there should be a trade-off between simu-lation speed and precision. Precision should be such that evolved neural controllersrun comparably good on the physical systems. Because evolution will always find so-lutions with respect to simulation errors, precision of the simulator must be such thatthese solutions can be avoided. On the other hand a certain lack in precision is seenas a plus for the development of robust neural controllers. For that, the simulator isdemanded to allow switching between simulated and physical robots and to run thesame controller on simulated and physical robots at the same time.

• For different scenarios, for different behaviors and for avoiding the generation of spe-cialists, it must be easy to change the simulated environments by placing differentobjects (boxes, spheres, etc.), changing colors and surface conditions.

• Synchronous simulation of a group of (different) robots should be realizable.

• The simulator should provide interfaces to other scientific applications, so that it canbe used by an interested scientific community.

• It should provide the possibility to use different physics engines (for instance ODE[9], IBDS [2])).

• Because it will be released to the public its handling should be easy.


1 INTRODUCTION

The presented version (1.0) of the Open Robot Control and Simulation Library (ORCS)matches all these demands. The simulator was written by the UOFM-group (VerenaThomas, Christian Rempis), the MotionEditor (section 6.4.2) by the HUB-group (Chris-tian Thiele). The physical model of the humanoid robot was defined in close collaborationof both groups.

More information on ORCS and the latest version of the library is available at the ORCShome page http://www.cogsci.uni-osnabrueck.de/~neurokybernetik/tools/orcs.


2 SIMULATOR ARCHITECTURE OVERVIEW

2 Simulator Architecture Overview

The Open Robot Control and Simulation Library (ORCS) is based on the platform indepen-dent C++ framework Qt (Trolltech) [11]. Qt provides a wide variety of classes, ranging fromoperating system independent types and containers to high-level components, like graphi-cal user interfaces, OpenGL support, XML parsers, scripting engines, a plug-in mechanism,networking and much more. Dependencies to other libraries – except of dependencies toexternal libraries required for the simulation of rigid body dynamics – have been avoided.Therefore ORCS compiles with only little effort on most common operating systems, likedifferent versions of Windows, Linux and Mac. A schematic overview of the ORCS archi-tecture is illustrated in figure 1. All important parts of the library are described in theremainder of this document.

CoreEvent

ManagementVariable

RepositoryPlugInSystem

ObjectCoordination

Physics

AbstractionLayer

ODEIBDS

Virtual Simulation Objects

OpenGLVisualization

EnvironmentXML

Analysis CollisionProcessing

Configuration

GUIConfigurationDialogs

GraphPlotting

InteractionDialogs

QT

UtilityKeyFrame

Control

CommandLine Parsing

ModuleCollections

InterfacingOSP 1.1

UDP Protocol

ODCP 1.0 UDP Protocol

CustomProtocols

Figure 1: Architecture of the Open Robot Control and Simulation Library (ORCS).

2.1 Modularity

To achieve a high degree of modularity and extendability, ORCS uses several mechanisms inaddition to common object oriented programming techniques that proved their usability andtheir potential to reduce the implementation effort for applications with rapidly changingrequirements (e.g. in the SIMBA Evolution Library [6]). These mechanisms facilitate notonly the usage of the library in a wide variety of applications but also provide a simple wayto extend existing applications with user specific plug-ins. In the next release users will forinstance be able to extend the simulators with own motor models, agents and embeddedcontrollers.

There are two main mechanisms used to enhance modularity and to decouple functionalmodules from each other: an event processor and a variable repository.


2.2 Event Processing 2 SIMULATOR ARCHITECTURE OVERVIEW

2.2 Event Processing

ORCS uses an event system for the interaction of functional modules. This system pro-vides a run-time interface for modules, using the classical observer pattern. Modules cancreate events at a globally available manager instance and trigger these events to signalthe occurrence of relevant events, e.g. the start or finalization of a simulation phase orthe detection of a collision in the simulator. Other modules interested in such an event canregister at the global instance to be notified at its occurrence. In difference to the signal-slotmechanism provided by Qt, ORCS events do not require a module to know the owner of anevent at compile time. Instead ORCS events are identified by simple strings and resolved atrun-time. Therefore it is easy to exchange both, event owners and event subscribers, withminimal changes to the application. Furthermore highly portable plug-ins and modules canbe implemented that allow users to select the required events manually at run-time andtherefore can be used in many different contexts.

2.3 Variable Repository

Modules in the library can exchange information using a global variable repository. Thisrepository provides access to variables identified by path names, similar to directory namesin an operating system. Any module can publish variables in this value repository andread variables published by other modules. To access a variable a module requires to knowthe name of the variable, not its owner. Therefore modules can easily be exchanged – aspreviously described for event processing – as long as the replacement of a module providesthe same set of variables at the variable repository.

Variables in the repository support a notification mechanism. In case a variable is changed,all modules registered for notification are informed about the change.

All variables encapsulate a specific data type (e.g. bool, int, double, string, color,vector3d) and provide corresponding methods to read and write the variable. Variables inthe repository additionally provide a set of methods to read and write the variable by usingsimple strings. Thus each variable comes with a build-in string parser and string composer,to handle such strings correctly for each specific type. By using strings all variables in therepository can easily be stored to files, displayed in user interfaces, modified with simpletext fields or loaded from files, independently of their encapsulated type.

The string interface of variables enables the implementation of general user interfaces (sec-tion 6.2.3) allowing the observation and modification of all repository variables in a uniformway. New modules therefore automatically get a simple configuration tool to adjust all oftheir parameters, if these parameters are published in the repository. New modules there-fore are not required to provide an own graphical configuration window to adjust theirparameters.

2.4 Physical Abstraction Layer

ORCS currently uses the Open Dynamics Engine (ODE)[9] as primary physics library tosimulate the rigid body dynamics. However ORCS was designed to work with a largenumber of physics engines, such es the promising Impulse-based dynamic simulation engine(IBDS)[2]. For this ORCS uses an abstraction layer to describe, configure and manipulate


2.5 Customization 2 SIMULATOR ARCHITECTURE OVERVIEW

physical objects. Because the actual physics library is hidden behind this abstraction layer,it can be exchanged with little effort. Other modules, depending only on the physicalabstraction layer, remain unaffected by such a change. This enables the ORCS library tomake use of new upcoming physics simulation technologies and to pursue very differentstrategies to accurately model motors and sensors of the robots.

Object collisions in the simulation are resolved within the native physics or collision detec-tion library, thus behind the physics abstraction layer. After each simulation step collisioninformation is projected to a physics independent format and can be used by any modulewithout dependencies to a specific physics or collision library.

2.5 Customization

Users can adjust most aspects of the simulator, ranging from the physics parameters ofobjects to graphical representations and the look and feel of the simulator. Such settingscan be stored and loaded at runtime or at startup, so each user can have an own workingenvironment with an own set of preferred physical parameters, scenarios and settings of theuser interface.

This can also be used when evolving neurocontrollers with evolutionary approaches to avoidan overfitting of controllers to the simulation environment. Different scenarios (e.g. differentfloor surfaces and objects) as well as different motor model parameters or physics parameters(e.g. simulation accuracy) can be loaded during the evaluation to provide a wide spectrumof physical parameters. This leads to the evolution of more robust controllers, which workwell in different scenarios and therefore do not rely too close on the (simulated) physicalproperties.


3 SIMULATION MODEL FOR THE A-SERIES

3 Simulation Model for the A-Series

The A-Series robot (figure 2) is based on the ROBOTIS construction kit Bioloid [7], whichcomes with a set of actuators of type Dynamixel AX-12+ [8] and different types of plasticparts to construct a humanoid robot. The A-Series robot uses only the contained actua-tors and some of the connecting parts. Some parts, like the legs, feet, battery cases andPDA holder, were replaced by more suitable parts, designed by the HUB-group using rapidprototyping methods.

Figure 2: The A-Series humanoid.

To enable the development of behaviors that can be transfered on the physical machine, aplausible, physically correct simulation needs to be created. For this purpose all requiredmeasurements were performed in collaboration with the HUB-group. Not only the massesof all individual objects but also the weight of all screws and cables that are required onthe physical agent were taken into consideration. Complex objects consisting of severalsmaller parts, that are modeled individually and connected by fixed joints, result in morecalculation work during simulation. Therefore most of these object assemblies are currentlyapproximated by box objects of according size and weight. The movement range of thejoints as well as the collision properties of the agent are not influenced by these assumptionsand approximate the properties of the physical robot. To take the screws and cables intoconsideration, the total weight of all screws and cables was measured and distributed to thebody parts according to their appearance on the hardware.

Besides considering the size and weight properties of all robot parts, also the friction prop-erties of the different employed materials were determined. By performing standard ex-periments for calculating the friction properties of two materials, the dynamic friction ofthe A-Series humanoid materials against each other, as well as towards three different floormaterials (PVC, one type of carpet and wood) were determined. Each simulation object


3.1 Actuators 3 SIMULATION MODEL FOR THE A-SERIES

allows the assignment of a material (section 6.3). During collision detection a lookup tablehelps to determine the appropriate friction coefficient for the two colliding objects.

Although creating realistic object representations in simulation is highly important for arealistic simulation, the development and definition of adequate models for all sensors andactuators is at least equally relevant. The next sections will present some details on thework towards defining these models.

3.1 Actuators

The A-Series humanoid is based on the Bioloid robot kit, and the only actuators includedinto the model are Dynamixel motors of the type AX-12+ [8]. The Dynamixel motor canmodulate the desired movement torque and the desired position to be reached individuallyor in combination. The A-Series humanoid therefore allows for a position guided motorcontrol (where the torque is set to its maximum) and for a torque guided motor control(where the position is set its maximum) of the Dynamixel motors. In both control modes,the motor has a limited working range of [-150, 150] degrees from starting position and istherefore incapable of performing unlimited movements in one direction.

Position Control

The definition and usage of the motor model follows the concept of online-manipulationthat is used throughout ORCS. All parameters that influence the motor model and therebythe behavior of the robot, are included into the variable repository (section 2.3) and areaccessible during execution time. The availability of all important parameters enabled aquick experimental search for parameter sets that show a close correlation with the physicalrobots’ behavior.

The basis of the motor model is a standard PID-controller (equation 1). The desired angularvelocity for the next simulation step v(t) is based on the current error between the desiredd(t) and the actual position p(t): (e(t) = d(t) − p(t)). In combination with the threemodifiable PID parameters Kp, Ti and Td the new desired velocity is calculated1.

v(t) = Kp ∗ (e(t) +h

Ti∗

t∑i=t−n

e(i) +Td

h∗ (e(t)− e(t− 1)) (1)

In addition to the PID-control, a simple linear model for static and dynamic motor frictionis included (equation 2). A fixed static friction value fs is added for angular motor velocitiessmaller than a threshold bs. The dynamic motor friction component is composed of a termfor modulating the current velocity v of the motor with the friction parameter αd and anoffset od.

f =

{fs + αd ∗ v + od, |v| ≤ bsαd ∗ v + od, |v| > bs

(2)

Besides the parameters for manipulating the equations of the motor model, the currentlyused physics engine ODE also offers a range of parameters to manipulate the physicalbehavior of the simulation. These include the maximum velocity that contacts are allowed

1h represents the time step width.


3.2 Sensors 3 SIMULATION MODEL FOR THE A-SERIES

to generate, parameters to influence the allowed remaining error of the joint correction andfurther parameters to influence collision handling, contact properties and joint behavior.

By using data from one of the physical A-Series robots, the motor model was adapted toclosely match the performance of the physical motors. Section 4 provides more details onthe process of comparing the physical robot and the simulation and shows the behavior ofthe currently used motor model.

Torque Control

Matthias Kubisch (HUB-Group) studied the properties of the torque controlled DynamixelAX-12+ and defined a motor model which approximates the physical motor [5]. The definedmodel includes a dynamic friction model – called LuGre-Model – as well as models for gearbox distortion, gear backlash and rotor inertia.

To create the model and to show its applicability, an experiment with a driven pendulumwas designed. An aluminum pole (weight 500g) was attached to a Dynamixel and couldbe moved by controlling the motor. Several types of movements (periodical, quick torquechanges, slow torque changes) were recorded and used to create and optimize the motormodel. The original motor simulation and the comparisons with the physical motor weredone in Breve [4], a 3D simulation environment based on ODE. The ORCS implementationof the motor model shows an equivalent behavior as in the Breve environment (see figure3).

3.2 Sensors

The current A-Series humanoid version provides sensors for acceleration as well as for thejoint angle of the Dynamixel motors.

The implemented angle sensor in ORCS calculates the motor angle with an adjustableamount of noise. As the angle sensors at the physical robot provide a quite accuratemeasurement, this noise can be set very low.

The acceleration sensor employed for the A-Series is the ADXL213 build by Analog Devices[3]. The sensor measures the acceleration along two perpendicular axes up to a maximalacceleration of 1.2g. In case of reaching a higher acceleration along one of the sensor axes,the sensor returns the last valid measurement for both axes instead of the real acceleration.The measurements of both axes remain unchanged as long as one of the sensor measurementsis out of range.

The implemented sensor model calculates the accelerations along the sensor axes, performsthe above described cut-off for values over 1.2g and adds an adjustable amount of gaussiannoise to the results. Additionally a simple adjustable low pass filter (equation 3) is in-cluded to reduce the effect of artifacts that can originate from the time-discrete simulation.The strength of the filter, as well as the noise can be manipulated during execution time.Therefore an on-line adaptation of the sensor model is possible.

fs(t) =∑t

i=t−n s(i)n

(3)


3.2 Sensors 3 SIMULATION MODEL FOR THE A-SERIES

4 80 156 232 308 384 460 536 612 688 764 840 916 992 1068 1144122012961372144815241600 16761752182819041980-1

-0,8-0,6-0,4

-0,20

0,20,4

0,60,8

1AX-12+Simulation

time

mot

or a

ngle

5 105580 155 230 305 380 455 530 605 680 755 830 905 980 113012051280135514301505158016551730180518801955-1

-0,8

-0,6

-0,4

-0,2

0

0,2

0,4

0,6

0,8

1AX-12+Simulation

time

mot

or a

ngle

4 80 156 232 308 384 460 536 612 688 764 840 916 992 106811441220129613721448 1524160016761752182819041980-1

-0,8

-0,6

-0,4

-0,2

0

0,2

0,4

0,6

0,8

1AX-12+Simulation

time

mot

or a

ngle

4 80 156 232 308 384 460 536 612 688 764 840 916 992 1068 114412201296137214481524160016761752182819041980-1

-0,8

-0,6

-0,4

-0,20

0,2

0,4

0,60,8

1Ax-12+Simulation

time

mot

or a

ngle

4 80 156 232 308 384 460 536 612 688 764 840 916 992 1068 1144122012961372144815241600 16761752182819041980-1

-0,8-0,6-0,4

-0,20

0,20,4

0,60,8

1AX-12+Simulation

time

mot

or a

ngle

5 105580 155 230 305 380 455 530 605 680 755 830 905 980 113012051280135514301505158016551730180518801955-1

-0,8

-0,6

-0,4

-0,2

0

0,2

0,4

0,6

0,8

1AX-12+Simulation

time

mot

or a

ngle

4 80 156 232 308 384 460 536 612 688 764 840 916 992 106811441220129613721448 1524160016761752182819041980-1

-0,8

-0,6

-0,4

-0,2

0

0,2

0,4

0,6

0,8

1AX-12+Simulation

time

mot

or a

ngle

4 80 156 232 308 384 460 536 612 688 764 840 916 992 1068 1144122012961372144815241600 16761752182819041980-1

-0,8-0,6-0,4

-0,20

0,20,4

0,60,8

1AX-12+Simulation

time

mot

or a

ngle

5 105580 155 230 305 380 455 530 605 680 755 830 905 980 113012051280135514301505158016551730180518801955-1

-0,8

-0,6

-0,4

-0,2

0

0,2

0,4

0,6

0,8

1AX-12+Simulation

time

mot

or a

ngle

4 80 156 232 308 384 460 536 612 688 764 840 916 992 106811441220129613721448 1524160016761752182819041980-1

-0,8

-0,6

-0,4

-0,2

0

0,2

0,4

0,6

0,8

1AX-12+Simulation

time

mot

or a

ngle

Figure 3: Plots showing the angle sensors of the motors resulting from the pendulumexperiments performed during the motor model optimization. The blue line shows theangular sensor information captured from a Dynamixel AX-12+ and the red line representsthe angular information of the simulated motor.


4 COMPARISON WITH PHYSICAL ROBOT

4 Comparison with Physical Robot

The technical information available from data sheets was not sufficient to define appropriatemotor settings. Therefore, as a first approximation, an experimental approach was chosento match the physical motors’ behavior as close as possible. For this purpose a set ofdifferent movement sequences of the arms, legs, torso and the whole robot was created withthe MotionEditor software (section 6.4.2). The movements were used to test the possibleextreme positions of the motors, the movement velocities, as well as the motor strengths.The generated sequences were executed on one of the physical robots to capture data setsfor the comparison with the simulation. For the data sets the angle sensors of the motors,the motor control settings and the acceleration sensors were captured.

The MotionEditor software can control the physical robot and the simulation, even at thesame time (for an example see figure 4). Thereby it is possible to play the same movementsequences in the simulation and on the physical robot, and to compare the behavior undersimilar conditions.

Because a purely visual comparison of the simulated behavior and the physical agents’behavior is not sufficient to generate a valid motor model, the sensor and motor data of thephysical robot was used to optimize the motor model. ORCS offers the possibility to visuallycompare the captured data of the physical agent with the sensor and motor sequences ofthe currently running simulation (ValuePlotter, see section 6.2.3). Using this ability toquickly compare these data sequences to each other, motor parameters could be found thatapproximate the physical robot data closely. The best matching model so far was includedinto the simulation and constitutes the default motor setting on simulation startup. If a userdevelops more appropriate motor settings later, the Simulation Properties tool (section6.2.3) can be used to save and load those settings.

Although many experiments have been performed, only a few will be discussed exemplaryin this section.

Simple Motions. One experiment is a slow movement of the waist motor to bend forwardand a slow inverse movement to return to an upright position again (see figures 4, 5). Dueto the properties of the Dynamixel AX-12+, the physical humanoid is not capable of gettingback in the starting position again.

Another interesting experiment, which also incorporated gravitational questions, was a freeleg movement. The upper body of the robot was fixed in a position above the ground toallow for free mobility of the legs. The movement sequence caused the leg to be movedforward and up as fast as possible and back down to the backwards extreme position (figure6). Due to the constraint strength of the motor, a movement curve as shown in figure 7results from this control. The plot showing the simulated motor and sensor data showsa close correlation to the real data. During capturing the data from the physical robot,some deviation in the movement curve and the strength of the motor could be observed.Therefore a too close match of the data sequences should not be expected.

Complex Motions. After adapting the motor model using the simple movement se-quences, more complex movements were tested. Among these are arm movements includingmultiple motors with different velocities (figure 8), a stand up sequence (figure 9) and a



Figure 4: Sequence of pictures showing the slow waist movement. The physical robot andthe simulation are simultaneously controlled by the MotionEditor. Hereby the waist motorwas moved slowly to its maximum position, which make the robot bending forward. Aftertwo seconds, the desired motor position is slowly changed back to the start position. Dueto the limited strength of the Dynamixel AX-12+ motors, the robot can not accomplish thefull movement back to an upright position.



(a) Comparing the desired motor positions and the measured actual motor positions.

(b) Comparing the acceleration data measured by the acceleration sensors on the arms.

(c) Comparing the acceleration data measured by the acceleration sensors on the shoulders.

Figure 5: Series of different plots of the simulated and the physical motor and sensormeasurements during the slow waist movement shown in figure 4



Figure 6: Sequence of pictures showing the performed leg movement. The agent is fixedin place with its back. The floor is low enough to avoid any contact with the robot. Thehip motor controlling the forward motion was set to the maximum forward position whichresults in an upward movement of the leg. After two seconds, the desired value for the hipmotor was inverted and set to the maximum backwards direction.



(a) Comparing the desired motor positions and the measured actual motor positions.

(b) Comparing the acceleration data measured by the acceleration sensor on the hip.

(c) Comparing the acceleration data measured by the acceleration sensor on the foot.

Figure 7: Series of different plots that show the values of the simulated and the physicalmotors and sensors during the leg movement shown in figure 6.



(a) Comparing the desired motor positions and the measured actual motor positions of the ellbow.

(b) Comparing the desired motor positions and the measured actual motor positions of the shoulder.

(c) Comparing the acceleration data measured by the acceleration sensor on the shoulder.

Figure 8: Series of different plots that show the values of the simulated and the physicalmotors and sensors during a complex arm movement.



Figure 9: Sequence of pictures showing a stand up motion. Both the physical and thesimulated agent succeed in ending in a standing position.



Figure 10: Sequence of pictures showing a sit up movement. Both the physical and thesimulated agent start lying on the back and end in a sitting position.

sequence to move from a lying position to a sitting position (figure 10). Interesting tomention is, that the arm movement sequence and sitting up sequence were constructed insimulation and were ported to the physical machine afterwards. The fact that it is possibleto create movement sequences in simulation proofs that the motor model is a good firstapproximation toward enabling evolution of portable controllers in simulation.


5 SIMULATOR INTERFACES

5 Simulator Interfaces

Users can integrate the ORCS simulator into their research environment through differentinterfaces. These interfaces use simple XML syntax to customize the simulator or UDPcommunication to control it from other applications. In addition to the interfaces describedin the remainder of this chapter, users can extend the open source simulation library itselfto implement any interaction mechanism they require to integrate the simulator to theirexisting software.

5.1 ORCS Standard Protocol

The ORCS standard protocol (OSP) is a UDP based network interface that can be used byexternal applications to exchange information with the simulator and to control simulatedagents. If the simulator is started with OSP support, a UDP based server is available,that manages incoming connection requests. Any number of external applications – such asevolution environments, monitoring tools or separate controller applications – can connectto the simulator through this server. As part of the connection request clients send theOSP version number they support. The server can use this information to create a separateclient handler for each connection request, taking into account the required protocol versionof the client. This simplifies the extension of the provided protocol, as new functionalitycan be added at any time in new protocol versions, hereby staying compatible with previousprotocol versions.

The current protocol version OSP 1.1 provides commands for the following interactions:

• Reset the entire communication. All client connections are canceled and the UDPserver restarts. This is useful if any of the clients crashed during a blocking operation.

• Obtain information about available agents. A set of commands allows a client to getthe name, type, control state and motor/sensor information of each agent. Motorsand sensors of an agent are identified by their name, which allows the user to matchmotors and sensors independently of their order and context.

• Read and modify all global variables of the variable repository (section 2.3). Sim-ulation properties like friction, motor and sensor parameters, material properties,simulation step size and the like can directly be controlled. External applications canuse this to modify settings while the simulator is running, e.g. to optimize the simu-lation parameters or to induce noise to the settings to prevent controllers to adapt toa specific simulation setting.

• Register for the observation of global variables in the variable repository. After eachsimulation step a client gets the current values of the observed variables. Clients canuse such information for instance in fitness functions or to observe the simulation.

• Register for the observation of global events (section 2.2). Clients can receive a listof all events that occurred during the last simulation step. Events are used in thesimulator – among normal object coordination – to signal collisions between simulatedobjects.


5.2 ORCS Direct Control Protocol 5 SIMULATOR INTERFACES

• Register as controller for an agent. The simulator guarantees that each agent iscontrolled only by one or no client.

• Demand the execution of a simulation step. This function has only an effect if theclient is controlling at least one agent in the simulation. The simulator collects theserequests until all such clients requested the next simulation step. This synchronizationguarantees that all clients could sent the desired motor settings for their controlledagents before the next simulation step is executed. After the execution of a simu-lation step a confirmation is sent to all these clients, including the current sensormeasurements of each controlled agent.

• Demand a simulation reset. This restarts the simulation with the initial object posi-tions and settings. A reset is executed synchronized, thus all agent controlling clientshave to request the reset before it is performed.

The full documentation of the protocol is available at the ORCS web site [1].

5.2 ORCS Direct Control Protocol

The ORCS direct control protocol (ODCP) is a low-weight UDP based network interface.For each simulated agent an ODCP server can be started to provide a UDP socket whichallows the direct control of the agent by external clients.

The current ODCP 1.0 protocol version is simple to implement as it supports only fourcommands:

• Connect to the ODCP server of an agent. This notifies the simulator that a client iscurrently using the interface.

• Disconnect from the ODCP server. Notifies the simulator that a client is not interestedin using the interface any more.

• Send motor settings. These settings are applied to the simulator during the nextsimulation step and kept until another command or controller changes these motorsettings.

• Request motor sensor data. The simulator sends a UDP message containing all anglesensors of the motors.

The ODCP protocol is not – in difference to the OSP 1.1 protocol – synchronized. Motorsettings are applied when a UDP packet was received, independently of the current simu-lation step. Therefore motor settings can be applied very irregularly, messages can get lostor be received too late. There is also no protection against multiple controllers, so an agentmight be controlled by two UDP clients which leads to unpredictable behavior.

The advantage of the protocol is its simplicity and its ability to be used in parallel toa real-time simulation loop or a ODCP based execution loop. It can be used to inducedisturbances to robot parts or to temporarily control agents not otherwise controlled byan ODP 1.1 client. The ODCP protocol is also used to interface with the MotionEditor(section 6.4.2).


5.3 XML Environment Descriptions 5 SIMULATOR INTERFACES

5.3 XML Environment Descriptions

The current A-SeriesSimulation application starts with a single A-Series robot in anempty environment. Users can describe their own environments with an XML syntax andload these environment files at startup (section 6.3). Currently users can create environ-ments from boxes, spheres and other A-Series robots. Such environment files can be usedto set up scenarios for the language games and to change these scenarios during evolution.


6 ORCS USER MANUAL

6 ORCS User Manual

6.1 Installation

Windows

Windows is supported with pre-build binaries. To install the simulator the A-SeriesSimula-tor.zip file has to be downloaded and unpacked.

The zip file contains the following files and directories:

• the simulator executable for interactive usage. (A-SeriesSimulator)

• the simulator executable for usage with the evolution system.(A-SeriesClientSimulator)

• the directory orcsconf which contains configuration files and resources of the simu-lator.

• an application to demonstrate the usage of the communication protocol.(ORSC OSP SliderControl)

• the MotionEditor to create and execute motion networks.

• a folder containing files for the described demonstration scenarios, like motion networkfiles, parameter settings and environment files.

• a number of batch files that start the simulator with specific scenarios, described inthis manual.

The executables require the Qt shared libraries to be on the library path. If not installed,they have to be obtained from the homepage of Qt [11]. Alternatively the libraries can bedownloaded from the ORCS home page [1] and be placed in the executable folder manually.

Linux (Binary)

For some Linux distributions (Ubuntu 8.04 up) binary packages can be downloaded on theweb page. These zip files contain the same content as the window version except of theMotionEditor (section 6.4.2), which is only available for Windows.

The executables require the Qt libraries to be on the library path. On Ubuntu and mostDebian based systems, these libraries can be installed as superuser via apt-get:

sudo apt-get install libqt4-core libqt4-dev libqt4-gui libqthreads-12

On the supported Linux distributions the simulator should run out of the box.


6.1 Installation 6 ORCS USER MANUAL

Linux (Build from Source)

For all other Linux systems the simulator has to be built from source code.

The source code can be obtained at the ORCS home page. In addition to the simulatorsource code the physics libraries ODE and/or IBDS are required (currently only ODE inversion 0.9 is fully supported). The physics library has to be compiled first. Then theheader files and the static physics library have to be put on the build path. Also commonbuild tools, like qmake (contained in the libqt4-dev package), make, java and g++ arerequired.

The required packages can be installed using apt-get:

sudo apt-get install sun-java6-jre g++ automake autoconf libqt4-corelibqt4-dev libqt4-gui libqthreads-12

The ODE physics library (Version 0.9) can be obtained from the ODE homepage (http://www.ode.org). It should be unpacked and compiled with double-precision support:

cd ODE_BUILD_DIRsh autogen.sh./configure --enable-double-precision --enable-releasemakesudo make install

Then a new directory has to be created at the same level as the ORCS source direc-tory named ode-0.9, with a subdirectory named lib. The compiled static ODE librarylibode.a has to be copied into the lib folder:

cd BUILD_DIRmkdir ode-0.9cd ode-0.9mkdir libcp ODE_BUILD_DIR/ode-0.9/ode/src/libode.a ./lib

When the prerequisites are fulfilled, the ORCS prepare tool qmakeprepare.jar should bestarted in the ORCS source directory. The prepare tool modifies the project files to buildthe ORCS sources with ODE support. In a later release other physics libraries might beenabled with the prepare tool before compilation. Enter qmake and make to build allprograms of the library.

cd BUILD_DIR/orcsjava -jar qmakeprepare.jarqmakemake

This builds the library including the application KeyFrameSniffer and the simulatorsA-SeriesSimulator and A-SeriesClientSimulator.


6.2 Graphical User Interface 6 ORCS USER MANUAL

6.2 Graphical User Interface

The ORCS simulator can be started with a graphical user interface (GUI). It providesan OpenGL visualization of the simulation environment and a set of additional tools andfeatures to manipulate different aspects of the simulation. This section will introduce themost important features and their usage. More information on using ORCS and its GUIcan be found in the ORCS help (F1 or Help menu).

All values representing entities are following the SI - International System of Units. Further-more angles are specified in degrees. This simplifies working with the simulation settings.Internally radians and quaternions are used for the physical simulation.

The main window (figure 11) offers a view on the simulation environment where the key-board and mouse can be used to change the viewing position. In addition further tools areavailable which can be opened using the menu or the corresponding keyboard command.

Figure 11: ORCS main window showing an A-Series robot. The menu above the visualiza-tion offers tools to manipulate and control the simulation.

6.2.1 Camera Control and Keyboard Features

ORCS offers a hybrid control of the viewpoint that uses keyboard commands as well asmouse manipulations. Table 1 shows an overview of all mouse and keyboard commands andtheir impact on the visualization. As the user can define multiple cameras that show thesame scene from different positions (see section 6.2.4), each camera can have an individualcamera control that is active as soon as the respective camera visualization window has thefocus.



Keyboard / Mouse Actionright mouse button Zooming in/out and translational left/right movement.left mouse button Rotational left/right movement and rotational up/down

movement. Changes of the orientation result in a rotationaround the current viewpoint.

middle mouse button Translational up/down movement.a or left arrow Translational left movement.d or right arrow Translational right movement.w or arrow up Zooming in.s or arrow up Zooming out.e or page up Translational up movement.q or page down Translational down movement.Shift Accelerates all viewpoint movements.Ctrl + Shift + v Save the current viewpoint. This will override the previously

stored viewpoint.Ctrl + v Restore the saved viewpoint.Ctrl + x Show/hide textures.Ctrl + t Show/hide time information, which shows the passed simu-

lation time and real-time since the last reset.Ctrl + a Show/hide coordinate axes.Ctrl + g Show/hide a grid with a edge length of 1m.

Table 1: List of all mouse and keyboard commands to control the viewpoint of a camera orthe visualization.

6.2.2 Control Menu

The Control menu offers several possibilities to manipulate the simulation. ORCS simula-tors using the Orcs Standard Protocol (OSP) (section 5.1) do not provide the control menu,as the simulation control is synchronously triggered by ORCS OSP clients (section 5.1).

Motor Control Manager. To enable an easy manipulation of the simulated agents,slider controls for all motors of each agent are available on demand. The Motor ControlManager shows a list of all known agents, which can be manipulated by sliders (figure 12).For each chosen agent, a separate window will be opened, which offers a slider for eachmotor of the agent.

The name of the agent is added to the list below the two buttons. Double clicking on aname will hide/show the according motor control window. A control window can be deletedwith the Remove Controls button.

To allow for repeating movements and to manipulate several motors simultaneously, a saveand restore mechanism is included into each slider panel. The keys [0− 9] can be used toswitch between previously saved motor settings.

Using Ctrl+[0− 9] the currently chosen motor values are stored internally and related withthe according number key. By pressing the according key, the previously saved combinationof desired motor positions is restored.



Figure 12: The Motor Control Manager (upper right) holds a list of available agents andallows the creation of control panels, which provide a list of sliders to manipulate the motorsof the agent. At the bottom an example of a slider control is shown.



Realtime. Runs the simulation in real-time. If the used machine is capable of runningthe simulator faster than read-time, this will reduce the speed and adapt it to real-time. Ifthe machine is too slow to reach real-time, enabling Real-time will result in running thesimulation at maximum speed.

Pause. Pauses or resumes the simulation.

Next Step. If the simulation is paused, this action triggers a single simulation step. Thisallows the user to step through the simulation.

Reset Simulation. Resets the simulation to the initial conditions. During startup ORCSsaves all physically necessary information of each object and joint. By calling reset, thephysics will be deleted and newly created using the previously saved information. By usingthe Object Properties tool (section 6.2.3), the initial information for a particular valuecan be changed.

6.2.3 Tools Menu

The simulator provides a large set of parameters that can be modified or observed by theuser. There are two types: simulation properties and object properties.

Simulation Properties. The Simulation Properties tool (see figure 13) allows directaccess to all available parameters of the variable repository (section 2.3), such as the settingsof the physics engine, the simulation delay and step size, or parameters of the graphicaluser interface. The global settings for the motors and sensors and all object parametersare available in this tool as well. The latter parameters are additionally accessible by theObject Properties tool, where plotting and logging mechanisms are available.

By using regular expressions in the upper text field the user can search for parameters. Allmatching parameters will be shown in the drop down box below and can be selected withthe mouse. To create visualizations for the current selection of parameters, the Add buttonor the return key can be used. As long as the checkbox of a parameter is checked, thevalue of that parameter is automatically updated when it was changed somewhere else inthe simulator.

A modification of the parameter value is possible by typing a new value into the accordingtext field and pressing the return key. To apply several changes simultaneously the Applybutton can be used instead of repeatedly pressing the return key.

A selected set of parameters can be saved to a file. Such parameter files can be loaded atstartup with -ival or -val (section 6.4.1), or can be loaded at runtime with the SimulationProperties Tool. In the latter case loaded values are not applied directly to prevent undesiredchanges. Instead they have to be applied manually with the Apply button.



Figure 13: The Simulation Properties tool. All parameters matching the current regularexpression in the upper text field will be shown in the lower part of the panel. A text fieldoffers the possibility of observing and modifying the value of the individual parameters. Ifthe user changed the value of a parameter, the according text field will be highlighted (seefigure) until the modification was applied. The buttons below provide additional featureslike saving and loading of value collections.



Object Properties. The Object Properties tool enables an observation and manipula-tion of object properties (figure 14). These properties describe states, settings or attributesof all simulated objects, like motors, sensors, body parts or environment objects. The selec-tion mechanism is similar to the Simulation Properties tool, so the user can search fordesired properties using regular expressions. In contrast to the Simulation Propertiestool, several tabs can be created (using the +/- buttons at the top), where independentsets of object parameters can be managed. The possibility of creating separate collectionsis important, because apart from observing and manipulating the property values, thesecollections can also be logged to files or plotted in graphs.

Figure 14: The Object Properties tool. Several tabs can be created to compose differentparameter sets that can be observed and manipulated. As in the Simulation Propertiestool, values modified by the user are highlighted until they were applied. The buttons at thebottom provide additional features like a ValuePlotter and a Logger. Below the buttons,the Logger panel is visible. On starting the Object Properties tool, the Logger is hiddenand opens only after pressing the Log button.

Parameters in the Object Properties tool can be changed manually as in the SimulationProperties tool. Most parameters however will be overwritten at a reset with the default



settings found at startup of the simulation. To keep a setting even at a simulation reset,the SetInit button has to be pressed to transfer the value to the initial settings.

Value Plotter. By pushing the Plot button a plotter opens and starts to record and plotthe history of all currently selected parameters (figure 15). This is useful for observationsof the motors and sensors of the robot during complex motions.

If the selection of values in the Object Properties panel changes while a plotter windowis open, the Plot button needs to be pressed again to update the selection of plotted pa-rameters. As the plotter visualizes the development of parameters over time only numericalvalues (double, int, vector3d) can be plotted.

Figure 15: The Value Plotter shows the time development of the currently selected pa-rameters of the Object Properties tool. The tabs on the right side offer feature as: colorselection, hide/show single parameters and loading offline data.

The Plotter Window has several tabs to manipulate the Value Plotter:

• General. The tools in the General tab allow to vertically scale and move the plottedvalues and to change the history size of the plotter.

• Values. The Values tab shows a list of all currently plotted parameters and enablesto change the name, color and visibility of each parameter.

• Offline. To compare the observed parameters to a previously recorded set of pa-rameters (e.g. recorded with the logger), a logged property sequence can be loadedin the Offline tab of the Plotter. The loaded sequences can be scaled and movedindependently of the values plotted on-line to find a match between the data for directcomparisons.

• Special. If the number of plotted values becomes large and the legend becomesdisturbing, the checkbox in the Special tab can be used to hide the legend.

Logger. During simulation the progression of the currently selected properties can belogged to a file for later analysis (see figure 14).



Using the three text fields that become visible on pressing the Log button, the file name forthe log, a comment character and a separator character can be set to simplify the usage ofthe data in external applications, like OpenOffice or Matlab. The logger adds the namesof the logged properties as comments to the start of the file, using the comment characteras prefix. Logging starts by pressing the Log Selected Variables button. Hereby thevalues of all selected properties are written to the selected file after each simulation step,using the separator character to separate the values. The logger can be stopped with theLog Selected Variables button.

6.2.4 View Menu

Camera Manager. The Camera Manager (figure 16) is a tool for creating and managingdifferent views on the simulation. The manager allows to choose a reference object in thesimulation, to which the new camera view will be attached. The initially chosen positionand orientation values are offset values from the reference objects position and orientation.If the reference object is moved, the camera view is moved accordingly and the view on thescene changes.

Figure 16: The Camera Manager consists of a property selection area and a list of existing,active views. Some examples of different views of the same scene are shown at the right.

The Controllable checkbox indicates, whether a camera control is available for a newlycreated view. If keyboard and mouse control is enabled, then movements are always relativeto the reference objects position and orientation.


6.3 XML Environment Description V 1.0 6 ORCS USER MANUAL

6.3 XML Environment Description V 1.0

For experiments requiring the agent to interact with its environment or other robots, an easyway to define such an environment is needed. Especially for the language games multipleobjects of different shape, size and color have to be defined and modified easily.

To avoid the necessity of any deeper knowledge about the simulator and its architecture, aXML-description for simple environment objects is provided. A set of environment objectscan be loaded at startup from an XML file. The XML file can be specified with the commandline option -env 〈environmentfile〉.xml. All objects defined in the according file will beadded to the simulation.

Besides the possibility of loading environment objects into the simulation, it is as wellpossible to add further A-Series robots.

The XML example below (exampleEnvironment.xml) shows a full environment descriptionthat adds two environment objects and places an additional agent into the simulation. Theresult of loading this XML-description is shown in figure 17.

<ORCS version="1.0"><Environment><object type="box"

name="FirstBox"position="(0.025,0.075,0.25)"orientation="(45,45,45)"dimension="(0.1,0.1,0.2)"color="(0,100,100,200)"texture="AccelBoard"/>

<object type="sphere"name="SecondSphere"position="(-0.255,0.5,0.2)"color="red"dynamic="true"mass="0.05"material="ABS"dimension="0.05"/>

</Environment>

<Agent><ASeries name="A-Series" position="(0.3,0.02,-0.3)" /></Agent>

</ORCS>

The <Environment> node of the XML definition allows for the definition of sphere- andbox-objects of different size and color. Each <object> tag marks one object definition,where several attributes are required to describe the properties of the environment objects.A list of all parameters, their meaning and their usage can be found in table 2 (objectparameters) and in table 3 (agent parameters).



Parameter Mandatory Usage

type The type defines the shape of the environment object and can havethe value box or sphere.

name Optional

All objects and joints in the simulation carry names. The name can beused during simulation to manipulate the properties of an object usingthe Object properties tool (section 6.2.3), or to change parametersbefore starting the simulation by loading a value-file (section 6.4.1).If no name parameter is defined, default names (box and sphere) willbe used.

position The position defines the location of the object within the simulationenvironment in global coordinates.

orientation Optional The orientation parameter sets the orientation of the environmentobject in degrees around the three coordinate axes (Euler angles).

dimension

The dimension parameter defines the type specific shape and sizeproperties. For sphere objects the dimension is a double value, rep-resenting the radius of the sphere. For box objects the dimensiondefines the width, height and depth.

material OptionalTo manipulate the collision properties of the environment object ma-terial information can be added (see section 6.3 for more information).If no material is chosen, the material Default is used.

color Optional

The color parameter allows to modify the color of the environmentobject. There are some predefined color values that can be chosen bynaming the color (e.g. blue, darkYellow). If non-standard colors needto be used an RGBA-representation of the desired color can be used.If no color is specified, white will be used as default.

texture OptionalThe texture defines the name of the texture image to be used for thevisualization (see section 6.3 for more information). If no texture ischosen, Default will be used.

dynamic Optional

As default all environment objects that are added to the simulationare static. Gravitation will not act on them and during collisions,their position will not be changed. To add an environment object,which can be manipulated, the parameter dynamic needs to be set totrue. All dynamic environment objects need to additionally define amass.

mass if dynamic For static objects no mass information is required. For dynamic ob-jects the mass describes their weight.

Table 2: List of attributes and their valid values for writing XML descriptions of environ-ment objects. Some attributes are mandatory, denoted in the second column. Knowing thename of the newly defined object, these parameters can also be modified during executiontime by using the Object Properties tool (section 6.2.3).

Parameter Mandatory Usage

name Optional Each new agent can have a name. All parameters describing the prop-erties of this particular agent will use this name as prefix.

position The global start position of the agent. If set to (0,0,0) the agent willstart standing at the coordinate center of the simulation.

Table 3: List of attributes and their valid values for writing XML-descriptions of A-Serieshumanoids.



Figure 17: Simulation resulting from loading the XML example exampleEnvironment.xmlpresented in the tutorial. The file defines a box object, a dynamic sphere object and anadditional A-Series humanoid.

Texture and Material Descriptions

As already mentioned in section 3 each simulated physical object in ORCS can be assignedwith a material. This material information influences the way collisions are resolved for thisobject. During collision handling, the materials of the two colliding objects are determinedand a lookup table is consulted to find the according friction coefficient. To change thecollision behavior, the user simply needs to change the material of the object.

Currently the following materials are supported: ABS (Acrylonitrile butadiene styrene –plastics material used for rapid prototyping), PVC, Carpet, Wood, Latex and two differentplastics from the Bioloid construction kit [7] named PlasticsWhite and PlasticsGray.There is a Default material that behaves as Latex on Carpet for every partner material.If the material is not set by the user the object will use this default material.

If an invalid material name or None is used to define the material, a second friction andelasticity calculation mechanism is used. Each object has two additional parameters toset the friction value (DynamicFriction) and the elasticity value (Elasticity). Duringcollision handling, at first the material information is checked. If the material informationof one of the objects is invalid, the above named parameters of both objects will be usedto determine the friction coefficient and the elasticity of the contact. For this purpose, thefriction values are summed up, whereas the elasticity constants are multiplied with eachother to generate the contact properties.

The OpenGL-based visualization can use textures for every object. To influence the texturethat will be applied to the object, two mechanisms are applied. Each object possesses aparameter called texture, where the name of an image file (without file extension) can benamed. If the program could find this image during startup, the texture will be appliedto the object. If no image with the according name could be found, the object will be



drawn without texture. If the texture parameter was not changed by the user, it is set toDefault. Setting the texture parameter to default will result in using the name of theobjects’ material to determine the texture.

There are a few textures (Sky, PVC and AccelBoard) that are included into the program.To use own images for the materials or to add new textures to the program, these files needto be placed into the folder orcsconf/textures relative to the place, where the programis started. The supported image formats are JPEG, PNG, TIFF, GIF, BMP and PPM. Even ifthe simulator already provides textures with the same name, the new images – if correctlyplaced – will be used instead.


6.4 Working with the Simulator 6 ORCS USER MANUAL

6.4 Working with the Simulator

6.4.1 A-SeriesSimulator Basic Usage

To use the simulator directly in an interactive mode, users can use the A-SeriesSimulatorapplication by starting the executable. The simulator can be adapted by using commandline arguments:

-env 〈environmentfile〉 loads an environment file (section 6.3).

-port 〈port〉 sets the port of the ODCP UDP server (section 5.2) for the direct control ofthe A-Series robot.

-gui / -nogui starts the application with graphical user interface enabled (default) or dis-abled (section 6.2).

-ival 〈valuefile〉 loads a parameter file before the initialization of the simulator takes place.This can be used to influence the initialization process. Users can use this commandline argument multiple times to load more than one value file. The command lineargument order is the order, in which the files are loaded.

-val 〈valuefile〉 loads a parameter file after the initialization of the simulation. This canbe used to change the settings of the completely initialized simulation, e.g. to set thecamera position, set motor model parameters or to change aspects of the graphicaluser interface.

-play 〈agent〉〈keyframerecord〉 installs the KeyFramePlayer (section 6.4.2) for the A-Series robot in the simulator and controls the robot named 〈agent〉 with previouslyrecorded keyframe motions loaded from the specified 〈keyframerecord〉 file. Such afile can be created with the KeyFrameSniffer tool together with the MotionEditoror manually with a text editor.

Once the application is started the simulation automatically starts running. The timedisplay in the upper left corner shows the simulation time and the real time. This indicateswhether the computer is fast enough to run the simulator in real time. By default thesimulator executes simulation steps with a user defined delay in between. A large delayleads to a simulation in slow motion, while a very low delay allows the simulation to runwith the maximum possible performance the computer is capable of.

The delay can be adjusted by manipulating the repository variable /Simulation/Delay inthe Simulation Properties tool (section 6.2.3).

The simulation can be paused and reset in the control menu. When paused, the controlmenu also allows for the execution of single simulation steps.

Manual Robot Control

The A-Series robot can be controlled manually with the Motor Control Manager (section6.2.2) found in the control menu. This tool provides slider panels to adjust the desiredangles of all motors of the A-Series robots.



Simulation and Object Properties

The simulation can be interactively modified using the property tools (section 6.2.3 and6.2.3) found in the Tools menu. The Simulation Properties tool enables loading andsaving of parameter sets, which is useful if motor or physics parameters have been experi-mentally determined. Such parameter sets can be saved to a file to reload the settings later.Modifications, observations and logging of simulated object properties can be done in theObject Properties tool. Desired settings of parameters can be transferred to the initialsettings of an object by pressing the Set Init button of a property. Such settings will berestored whenever the simulation is reset.

Camera Views

For language games or for presentation purposes additional views on the simulated scenariocan be added. The Camera Manager (section 6.2.4) in the View menu controls the creation ofsuch views. Independent cameras can be added at global scope or be attached to simulatedobjects. This way each robot can for instance provide an own view from the perspectiveof its head camera or the user can create a camera that automatically follows a robot in acertain distance.

6.4.2 Interaction with the MotionEditor

MotionEditor. Using the MotionEditor [10] (figure 18) it is possible to design keyframe-based motions for the A-Series humanoid robots. A keyframe defines the desired angles ofall motors. The composition of a sequence of these keyframes – connected by transitions –forms a motion net. To achieve a smooth motion, the MotionEditor interpolates betweentwo successive keyframes by slowly changing the settings from one keyframe to the next onewithin a given duration.

One is able to connect the MotionEditor directly via USB to the physical A-Series robotto execute motion nets or to deploy them to the flash memory of the robot. Furthermorea connection to a simulated robot in the ORCS simulator A-SeriesSimulation can beestablished. Even a simultaneous connection to a physical and a simulated robot is possible.Connections can be established via the Communication menu.

The MotionEditor works purely graphical. Keyframes are shown as small rectangles withtheir keyframe number. Transitions are shown as arrows connecting keyframes. Keyframesare generated using the right mouse button on the empty drawing area and by choosingNew keyframe in the context menu. To create a transition, action New transition in thecontext menu of a keyframe can be used. A click on another keyframe creates the transitiontowards the latter one. Using the context menu it is further possible to delete elements orcopy and paste them. The execution of a motion net will always start with keyframe 1. Fortesting purposes some sample motion nets (*.mef) have been included in the ORCS package(the program and the example motion files are available in the folder /tools/MotionEditorof the enclosed zip-file).

The main window is split into three areas. The largest is the editor area at the left. At theright one can find the motor settings area on top and the player area at the bottom. Inthe editor area one can edit the keyframe net. Selecting a keyframe or a transition shows



Figure 18: The MotionEditor. The graphical user interface is divided into three section: akeyframe editor area, a player area and an area to set the motor settings of keyframes.

the corresponding settings in the settings area (motor positions for keyframes and torquevalues for transitions). The player area is used to run the current motion net (at the robot,the simulator or both, regarding which devices are connected).

Selecting one or more keyframes (the Shift key enables multiple selections), motor valuesare shown at the right in the settings area. The sliders can be used to manipulate thesesettings. If a (physical or simulated) robot is connected, the current keyframe is immediatelytransmitted to the robot.

Selecting a transition shows the corresponding settings in the settings area. One is able tochange the transition time and the torque values2. Motion nets can have multiple transitionsstarting in one keyframe. The decision, which transition is chosen while playing is madeusing a global selector, which is an ASCII character. If the global selector is equal to theselector of a transition, this transition is chosen. The selector for a transition is set via thecontext menu of the transition.

A loaded or created motion net can be executed (played) using the integrated player. Click-ing the Start button starts playing from keyframe 1. The player can be controlled bychanging the global selector. This is accomplished by simply pressing the correspondingkey (e.g. pressing key R results in the global selector R).

2Currently ORCS v1.0 does not fully support torque modulations



KeyFrameSniffer and KeyFramePlayer. Instead of connecting to the ORCS simula-tion, the MotionEditor can be connected to the application KeyFrameSniffer. This toolprovides the same interface as the A-SeriesSimulation application, but does not containa simulator. The purpose of this program is to log received motion commands sent bythe MotionEditor while running a motion network. The keyframe settings are recordedas non-interactive keyframe sequences to a file to be replayed in the simulator without aconnection to the MotionEditor. The command-line based tool can be started with:

KeyFrameSniffer <fileName>

The filename defines the name of the file in which the logged data will be saved. TheKeyFrameSniffer runs without graphical user interface in the background and stops auto-matically when the MotionEditor is disconnected with action Disconnect Simulator inthe Communication menu.

Recorded keyframe sequences can be used to execute the exact movement sequences in thesimulator without a connection to the MotionEditor. For this the A-SeriesSimulationprovides a build-in KeyFramePlayer, which can handle the motion files. The simulator canbe started with a KeyFramePlayer as follows:

A-SeriesClientSimulator -play <agent> <keyframerecord>

The player requires the name of the agent to be controlled (e.g. Humanoid) and thefile <keyframerecord> with the motions to apply to the agent. Multiple agents can becontrolled by individual keyframe sequences by using the -play parameter several timeswith different agent names.

The purpose and reason for the KeyFrameSniffer and the KeyFramePlayer is the require-ment of the MotionEditor to have a real-time connection, as the transition durationsare based on real-time. Due to the fact that achieving constant and reliable real-timein a simulation program is difficult, this requirement might pose problems when tryingto compare very accurately the movements of the physical robot and the simulation. TheKeyFramePlayer therefore records a timestamp for each motion command received from theMotionEditor. This time information is used during playback to synchronize the motioncommands with the simulation time. This reliefs the simulation from the need to run inreal-time, as the motion commands now are applied based on the simulation time, not thereal-time. This is essential for comparable motions, especially on slower computers, wherereal-time can not be achieved.

6.4.3 A-SeriesClientSimulator Synchronous Control

The simulator version A-SeriesClientSimulator was designed to run on a computer clus-ter for the evolution of neuro-controllers. Therefore by default this simulator starts withoutgraphical user interface (GUI). To enable the GUI the application has to be started withcommand line parameter -gui.

In difference to the A-SeriesSimulator this simulator does not immediately start with thesimulation in its own simulation loop. Instead it waits for the connection of clients using



the OSP 1.1 protocol (section 5.1). The simulation is started only if at least one clientconnects and triggers the simulation execution steps via UDP commands.

To start the simulator with a simple example scenario (shown in figure 19), the followingline can be used:

A-SeriesClientSimulator -env languageGame.xml -val languageGame.val -gui

Figure 19: Result of opening the simulation with the XML-file languageGame.xml and thevalue-file languageGame.val.

Currently there are two applications that can be used to connect to the simulator and tocontrol the simulation: the ORCS OSP SliderControl and the ORCS NetworkEvaluator.

Application ORCS OSP SliderControl.

The ORCS OSP SliderControl application demonstrates the usage of the OSP 1.1 proto-col. The source code is provided with the deliverable and can be used to build customapplications to interact with the agents in the simulator.

To use the ORCS OSP SliderControl Java is required. The application can be started withthe following command:

java -jar ORSC_OSP_SliderControl

In the application window (figure 20) the ORCS OSP SliderControl can be connected to thesimulator. In the area below all detected robots are shown with detailed information abouttheir names and their motor/sensor interface. To control a robot it has to be selected inthe list and the Control Agent button has to be pressed. By pressing it again the controlof the agent is returned to the simulator. If an attempt is made to control an agent that



Figure 20: The application window of the ORCS OSP SliderControl. The program allowsto select an agent to be controlled by sliders. Additionally informations about all existingagents are provided.

is already controlled by another client, a notification is shown and the agent control is notcommitted to the ORCS OSP SliderControl.

Once control is successfully taken by the ORCS OSP SliderControl, each motor of the con-trolled agent can be adjusted with sliders. However, since the simulation execution is clienttriggered, nothing will happen before all clients, that control at least one agent, requesta simulation step. To do so the Run button has to be pressed. If more than one client isconnected to the simulator, then all these clients have to start sending their step requests.

To reset the simulator all agent controlling clients have to send a reset request.

This synchronization enables the control of multiple robots in a step triggered way withoutthe need to run all clients with exactly the same speed.

Application ORCS NetworkEvaluator

The ORCS NetworkEvaluator executes neural networks as controllers for a simulated agentin an A-SeriesClientSimulator and evaluates the network performance with a fitnessfunction. Such an application is used on the cluster computer to evaluate neural networkcontrollers during evolution.

Usage:

java -jar SEED_NetworkEvaluator.jar -net <network> <agentName> [<fitness>]

Example:

java -jar SEED_NetworkEcaluator.jar -net a-series_oscillator.smb humanoid

The provided network controller is a simple oscillator which is controlling some of the motorssimultaneously to demonstrate the network interface.


7 PLANNED EXTENSIONS

7 Planned Extensions

7.1 Motor Model

Position Control. The current motor model is based on a number of experiments inwhich different movement sequences have been executed to record the relevant sensor data.But all the data was captured from a single A-Series robot. Because it is known, that acertain deviation – due to small differences in the construction and the motor state – exists,it is important, that the same data is captured for several robots of the A-Series and themotor settings are adapted accordingly.

As the model for the position controlled motor proved to be realistic, the next steps willbe an extension of this model and a further adaptation of the parameter settings. Becauseof the deviation between the different robots as well as already for a single robot, it mightbe a good choice to search for several motor settings, which each optimize the performancewith respect to a different goal (e.g. for each robot one set).

To use these different sets during evolution of neural controllers, each controller will betested with each of the settings and is evaluated for the overall performance.

Because the amount of data will get rather large and the number of parameters mightincrease because of the inclusion of further components to the model, the optimizationprocess might be automated (e.g. using artificial evolution).

Torque Control. The preliminary work done by M. Kubisch [5] resulted in a very complexmodel of a torque driven Dynamixel AX-12+ motor with a promising accuracy. Adaptingthis model to the requirements of the A-Series humanoid robot will be a valuable extensionof the already existing robot model.

7.2 Randomization

To evolve controllers that are robust with respect to small changes (of the robot model, theenvironmental conditions or the physical properties), it is important to enable an automatic,user-defined randomization of different simulation aspects. During the evolutionary process,each evaluation experiment is tried with respect to the given fitness function. To avoidthe specialization on a fixed environmental and physical setting, these parameters can bechanged in between generations to test the new created individuals with respect to newenvironmental conditions. Furthermore each individual can be tested in several setupsinstead of only one to additionally ensure a high robustness of the controller.

There are two concepts of randomization that will be supported. The first was alreadydiscussed in the previous subsection. It should be possible to change distinct parametersettings (either choosing randomly between different sets or sequentially using one afterthe other). The second concept is a parameter randomization. It should be possible todefine a set of additional parameters (deviation, minimum and maximum value) and rulesaccording to which the value of a parameter is changed randomly. This concept can be usedto modify the starting position and orientation of the whole agent or the starting positionof the motors.


8 ACKNOWLEDGMENTS

8 Acknowledgments

Thanks to Christian Benckendorf, Matthias Kubisch, Thomas Lobig and Manfred Hild forsupporting the definition of the A-Series model and for providing a quick support in caseof hardware issues.


REFERENCES REFERENCES

References

[1] ORCS Home Page at the Neurocybernetics Group University of Osnabruck.http://www.cogsci.uni-osnabrueck.de/ neurokybernetik/tools/orcs, 2008.

[2] Jan Bender. Impulsbasierte Dynamiksimulation von Mehrkorpersystemen in dervirtuellen Realitat. PhD thesis, University Karlsruhe, 2007.

[3] Analog Devices. ADXL213, 04 2004.

[4] Jon Klein. BREVE: a 3D Environment for the Simulation of Decentralized Systems andArtificial Life. In Proceedings of Artificial Life VIII, the 8th International Conferenceon the Simulation and Synthesis of Living Systems, 2002.

[5] Matthias Kubisch. Simulation nichtlinearer Motoreigenschaften. Student ResearchProject, Humboldt Univeritat zu Berlin, 2008.

[6] Christian Rempis. Short-Term Memory Structures in Additive Recurrent Neural Net-works. Master’s thesis, University of Applied Sciences Bonn-Rhein-Sieg, 2007.

[7] ROBOTIS. Robot Kit ”Bioloid”. URL http://www.robotis.com/zbxe/bioloid_en.

[8] ROBOTIS. Dynamixel AX-12 User’s Manual, 06 2006.

[9] Russel Smith. OPEN DYNAMICS ENGINE v0.5 USER GUIDE, 2006.

[10] Christian Thiele. Integrierte Entwicklungsumgebung zur Bewegungssteuerung hu-manoider Roboter. Student Research Project, 2007.

[11] Trolltech. Qt Reference Documentation (Open Source Edition), 2005.


http://www.robotis.com/zbxe/bioloid_en

ALEAR - uni-osnabrueck.deneurokyberne...The simulator should provide interfaces to other scienti c...

Documents

Transcript of ALEAR - uni-osnabrueck.deneurokyberne...The simulator should provide interfaces to other scienti c...