RoboCup Advanced 3D Monitor

Carla Penedo, Joao Pavao, Pedro Nunes, and Luis Custodio

Instituto de Sistemas e RoboticaInstituto Superior Tecnico

Av. Rovisco Pais 1, 1049-001 Lisboa, Portugal

{ccfp,jpp,pmgn}@rnl.ist.utl.pt, lmmc@isr.ist.utl.pt

Abstract. RoboCup Advanced 3D Monitor is a three-dimensional ap-plication for visualizing and debugging games of the RoboCup SoccerSimulation League. This paper discusses the issues pertaining the imple-mentation of this monitor using OpenGL, a standard API for renderinghigh-performance 3D graphics. Our application provides a true 3D soc-cer game experience maintaining a healthy balance of realistic anima-tion features and high-speed rendering achieved by the implementationof specific computer graphics techniques. Besides its main usefulness asa visualization tool, this monitor may be used as a supporting tool forthe development of other robotics techniques. To illustrate this, two ofsuch techniques are discussed here: sensor fusion and Markov localizationmethods.

Keywords: three-dimensional monitor, simulation league, levels of de-tail, markov localization, sensor fusion.

1 Introduction

The RoboCup soccer server simulates a 2D virtual field in which two agent teamsplay a soccer match. Although the environment is two-dimensional, the soccerserver has proved to be an adequate platform for the development of realistic3D visualization tools.

In this paper, the RoboCup Advanced 3D Monitor (RA3DM), a three-dimen-sional monitor for the RoboCup Simulation League is introduced. RA3DM aimsto turn the visualization of simulated soccer games more realistic and entertain-ing. As we were developing RA3DM we realized that the RoboCup soccer serverprovides an interesting testbed for prototyping algorithms which could be usedafterwards on real robots. A Markov localization algorithm and a sensor fusionmethod were successfully implemented in this environment and the latter hasalready been adapted to real robots of the middle-size league. Our monitor wasextremely useful for developing/testing/debugging the implemented algorithms.

RA3DM is implemented on OpenGL, a low-level graphics library which isdesigned to be used with C and C++ programming languages. One of the mainadvantages that OpenGL presents is its independence from operating and win-dowing systems. This feature enabled the development of versions for the Linux,MacOS X and Windows platforms with only a little extra effort.

The remainder of this paper is organized as follows. In the next section theglobal architecture of the distributed system is briefly described. In Section 3we present the main features of the implementation of RA3DM. Section 4 ex-plains how RA3DM was used for the application of sensor fusion and Markovlocalization algorithms in the RoboCup Simulation League. Finally, in Section 5conclusions are drawn and future work is discussed.

2 System Overview

The RoboCup simulation system is controlled by a central process — the soccerserver — that runs the whole simulation, sending and receiving all the relevantinformation to and from all the client programs connected to it [1]. An extraconnection was added to enable us to visualize the internal state of a soccerplayer. The RA3DM connects to a special port on the soccer player program tobe monitored via a dedicated UDP socket. The global architecture employed inour implementation is illustrated in Fig. 1.

The soccer server program sends information to all its connected monitorsevery 0.1 s. However, the majority of hardware available today (both graphiccards and main processors) is capable of delivering full-screen image updaterates (or frame rates) well above 10 Hz (i.e., 10 frames per second). This factled us to an architecture that maximizes drawing performance by allocating themaximum possible CPU time updating the scene. To accomplish this we usetwo permanently running threads [2]. One of the threads is responsible for alltransactions with the soccer server such as sending commands issued by the userand receiving data concerning the state of the simulation. The other thread willupdate the displayed 3D scene as fast as the underlying hardware allows. Thisthread is also responsible for managing all user interaction through the use ofmenus, mouse and keyboard.

3 Implementation

3.1 Player

RA3DM uses a skeletal animation system which allows for three main animationsto be performed by the player: walk, run and kick (see Fig. 2). A skeleton isdefined by a hierarchy of bones connected by joints with a position and an

RA3DM

Soccer Server

Soccer Player

Display Data

Agent's Internal State

Perceptions

Actions

UDP Socket

Fig. 1. Global architecture of the distributed system.

orientation, arranged in a tree structure. The model, usually a single mesh ofvertices or polygons, is attached to the skeleton through a process known asskinning. When the latter is animated, the mesh is deformed accordingly, andtherefore, the model is continually morphed to match the movements of theskeleton.

The animation process is simplified by keyframe interpolation techniques.A keyframe animation is created by interpolating frames (snapshots in time)between several successive keyframes that define specific pivot points in theaction. Each keyframe contains the values of the rotations of each joint of theskeleton. The mesh is stored only once, along with the skeleton and the keyframesof the animation, resulting in a very compact representation. Each animation iscreated according to an uniform and constant speed that can be modulateddirectly in RA3DM to obtain different running or walking velocities.

3.2 Levels of Detail

The computation and storage requirements for complex scenes, typical in somecomputer graphics applications, far exceeds the capacity of modern hardware.In order to accelerate the rendering of such scenes, approximations of decreasingcomplexity (levels of detail) are produced similarly to the initial model. Thesimpler versions contain fewer details that can not be noticed when the objectis farther away [3]. Most of the best techniques of polygonal simplification arebased on Hughes Hoppe’s progressive mesh work [4]. One possible simplificationis achieved by continuously applying an edge collapse operator that merges twoedge vertices into one, thus removing that edge.

Vertex Remove This operation takes two vertices u and v (the edge uv)and “moves” or “collapses” one of them onto the other [5]. Figure 3 illustratesa polygon before and after the application of the edge collapse operator. Thefollowing steps explain how this operation is implemented:

1. Remove any triangles that have both u and v as vertices (i.e., remove trian-gles on the edge uv).

2. Update the remaining triangles that use u as a vertex to use v instead.3. Remove vertex u.

The removal process is repeated until the target polygon count is reached.

Selection of the Next Vertex to Collapse When selecting a vertex to col-lapse, the basic idea is to preserve (as far as possible) the global appearance of anobject, trying to cause the smallest visual change to it. Despite the considerablenumber of algorithms that determine the “minimal cost” vertex to collapse ateach step, they are, in general, too elaborate to implement. A simple approachfor this selection process may be to just consider the cost of collapsing an edgedefined as its length multiplied by a curvature term (the latter having half the

Edge Collapsev vu

Fig. 2. Player model and its bone struc-ture.

Fig. 3. Collapsing vertex u onto v.

weight of the former in this calculation). The curvature term for collapsing anedge uv is therefore determined by comparing dot products of face normals inorder to find the triangle adjacent to u that faces furthest away from the othertriangles that are along uv. Equation (1) expresses this idea, where Tu is the setof triangles that contain u, Tuv refers the set of triangles that contain both uand v, and fN and nN are the face normals of triangles f and n.

cost(u, v) = ‖u− v‖ × maxf∈Tu

{minn∈Tuv

{1− fN · nN

2

}}(1)

The algorithm described throughout this section can be summed up as fol-lows: while the current polygon count is greater than the desired target number,select a candidate edge to collapse (according to its associated cost) and applythe edge collapse operator to it.

Player Simplification In practice, the polygon simplification algorithm pro-duces very reasonable results. Our soccer player originally contained 347 verticesand 639 triangles which, after the application of this algorithm, were reduced to68 vertices and 130 triangles. These two versions correspond to the highest andthe lowest level of detail used in RA3DM. The former is used when the camerais closer to the player while the other is selected if it is far away. In betweenwe consider intermediate levels obtained by the continuous variation of the totalnumber of vertices used to draw the player. In this way, it is possible to avoidan undesired effect, named popping, that can be seen as an abrupt change in thedetail present in the object’s shape caused by the considerable difference in thenumber of vertices.

3.3 Cameras

There are three distinct types of views into the field available to the end user:static, user controlled and automatic cameras. A view similar to the display ofthe official 2D soccer monitor (i.e., aerial view, pointing towards the center ofthe soccer field) is available, as well as a completely custom camera that the usercontrols using the mouse and the keyboard. Next we describe two of the mostinteresting automatic cameras:

TV camera has a fixed position slightly raised on one side of the field andconstantly follows the ball with a small delay delivering image sequencessimilar to those captured by a human camera operator. This camera zoomsin and out as the subject being shot approaches and moves away from thecamera;

Event-driven cinematic camera consists of a series of cameras placed insome strategic points of the 3D scene. Some of these cameras have a fixedposition whereas others are capable of some dynamic motion. At any givenpoint in time during the simulation of a soccer match, each one of these pre-defined cameras will be assigned a subjective value representing the qualityand usefulness of the image sequence provided by it. The measurement ofthis value and its assignment to the cameras is taken care of by a directoragent through the use of some heuristics. This agent also decides how topoint the cameras to the scene and how much zoom to use. After this eval-uation, an editor agent in charge of producing the sequence of images seenon screen will decide (based on the values produced by the director) whichcamera to choose and for how long. Different director and editor agents canbe defined. By changing the way the director rates the footage and the waythe editor chooses among them will effectively change the resulting cine-matic style, thus giving a totally different experience to the user [6]. Forexample, if two players are fighting for the ball in the midfield, the directormay choose to zoom in one of the closest cameras to make a close-up andgive that camera a high priority value. The editor will then be able to usethat footage, always taking into account the minimum and maximum timeintervals allowed between cuts.

4 Applications

In this section we describe the implementation of sensor fusion and Markovlocalization algorithms in the framework of the RoboCup Simulation League.Our 3D monitor was a valuable resource allowing the visualization of the player’sinternal state and the gathering of experimental results. RA3DM includes a lowquality viewing mode that was extensively used in order to facilitate the dataobservations.

4.1 Sensor Fusion

Sharing information among robots increases the effective instantaneous percep-tion of the environment, allowing accurate modeling. Two known sensor fusionapproaches were tested in the RoboCup Simulation League: the Stroupe and theDurrant-Whyte methods [7].

To perform sensor fusion the soccer server’s noise model is approximated bya two-dimensional Gaussian distribution N(µ,CL), where µ is a vector repre-senting the calculated position of the object and CL is a diagonal matrix thatdenotes the variance along both axis (see Fig. 4). The variance along the axis

θ

σmajσmin

d

x = 0

(x, y)

Fig. 4. Distribution parameter definitions: mean (x, y), angle of major axis (θ), majorand minor standard deviations (σmaj , σmin) and distance to mean (d).

that points from the robot towards the observed object (σmaj) is calculatedbased on the quantization made by the soccer server. Thus, σmin represents thevariance along the perpendicular axis and is based on the maximum error inangle that an observation can have.

In order to use sensor fusion we must exchange sensor information betweenteam members. This exchange provides a basis through which individual sensorscan cooperate, resolve conflicts or disagreements, or complement each other’sview of the environment. In this case, an agent communicates information (µ,CL and θ) about seen objects to other agents.

Our goal was to compare the efficiency of the two mentioned sensor fusionmethods. The first approach simply merges the Gaussian distributions of theobservations made by the robot with the ones made by the others. The secondmethod takes into account the last known position of the object and tests ifthe readings obtained from several sensors are close enough to make the fusion.When this test fails, no fusion is made and the sensor reading with less varianceis chosen. The conditions in which this test fails and succeeds are presented inFig. 5.

F (p|z1, z2)

f1(p|z1) f2(p|z2)

F (p|z1, z2)

f1(p|z1) f2(p|z2)

Fig. 5. Two Bayesian observers with joint posterior likelihood indicating agreementand disagreement.

The ball in Fig. 7 (a) with the black and white texture represents the realball position communicated by the soccer server. The farthest ball from thereal position represents the position where the goalkeeper, on the other side ofthe field, sees the ball. The position communicated by a team member to thegoalkeeper is represented by the other dark ball. Finally, the white ball representsthe fusion between heard and seen ball obtained with the Stroupe method. As

can be noticed, the newly calculated position represents a better estimate thanthe observed value.

4.2 Markov Localization

For mobile robots, localization is the process of updating the pose of a robot,given information about its environment and the history of its sensor readings.An implementation of the Markov localization method was applied to self-localizea player in the RoboCup Simulation League [8].

At any point in time, Markov localization maintains a position probabil-ity density (belief) over the entire configuration space of the robot based onan incoming stream of sensor data (observations) and an outcome of actions.This probability framework employs multi-modal distributions for the robot be-lief enabling the representation of ambiguous situations by considering multiplehypotheses in parallel.

This particular implementation of Markov localization uses a fine-grainedgeometric discretization to represent the position of the robot. A position prob-ability grid is three-dimensional, as each possible location l is defined by a tuple〈x, y, θ〉 representing the robot position and orientation. The principle of theposition probability grid approach ascribes to each cell of the grid the proba-bility of the robot being located in that cell, denoted by Bel(Lt = l). Figure 6illustrates the structure of a position probability grid and the correspondencebetween grid and field positions in the RoboCup simulation system. Each layerof the grid assigns all possible poses of the robot with the same orientation.

Grid Environmentθ

x

y

Bel(Lt = l)

Fig. 6. Transformation of grid coordinates into field coordinates.

When used as a debugger, RA3DM enables us to inspect the internal state ofa player (more precisely, its position probability grid) in real time. Figure 7 (b)shows a screen capture of RA3DM while it was monitoring the goalkeeper player.The cells with a belief value above a certain threshold are drawn in a shade ofblue and the one with the highest belief at each time step is distinguished witha red color.

Experimental results show that, generally, the Markov localization methodkeeps the robot position error bound within very reasonable limits. For example,we obtained a medium error of 2.44 m for a grid of cells of 2.63× 1.70 meters.

(a) (b)

Fig. 7. (a) Ball localization using Stroupe’s sensor fusion method. (b) Goalkeeper selflocalization using Markov estimation.

5 Conclusions and Future Work

Watching simulated soccer games on a realistic 3D monitor such as RA3DM isfar more motivating than on the traditional 2D monitor.

We showed that the RA3DM is a powerful tool that can be used as an aidfor testing and debugging of prototype applications that may later be employedon real robots. Another possible application that we are considering consists ofusing RA3DM to simulate humanoid stereo vision. This way, the soccer playingagents could receive images supplied by RA3DM instead of the visual perceptionsin the form of strings provided by the soccer server.

There are some features we would like to add in the future, such as soundeffects, the recording of parts of a game to instant replay them later on and theaddition of a commentary system.

References

1. Noda, I., et al.: RoboCup Soccer Server: Users Manual for Soccer Server Version7.07 and later. (2001)

2. Barney, B.M.: POSIX threads programming (2001) Lawrence Livermore NationalLaboratory.

3. Krus, M., Bourdot, P., Guisnel, F., Thibault, G.: Levels of detail & polygonalsimplification. ACM Crossroads 3.4 (1997) 13–19

4. Hoppe, H., DeRose, T., Duchamp, T., McDonald, J., Stuetzle, W.: Mesh optimiza-tion. Computer Graphics 27 (1993) 19–26

5. Melax, S.: A simple, fast, and effective polygon reduction algorithm. Game Devel-oper Magazine (1998) 44–49

6. Hawkins, B.: Creating an event-driven cinematic camera. Game Developer (2002)7. Nunes, P., Marcelino, P., Lima, P., Ribeiro, M.I.: Improving object localization

through sensor fusion applied to soccer robots. In: Proc. of the Robotica 2003Portuguese Scientific Meeting, Lisbon (2003) 51–58

8. Penedo, C., Pavao, J., Lima, P., Ribeiro, M.I.: Markov localization in the RoboCupSimulation League. In: Proc. of the Robotica 2003 Portuguese Scientific Meeting,Lisbon (2003) 13–20