Car Setup Optimization Competition @ EvoStar 2010

Car Setup Optimization @ EVOStar-2010

Car Setup Optimization Competition

@ EVOStar 2010

Luigi Cardamone, Daniele Loiacono,

Markus Kemmerling, Mike Preuss

& TU Dortmund


TU Dortmund

What is the competition about?

The goal is to submit an optimization algorithm for a challenging problem

The problem: find the best car setup that achieves the highest performance on a previously unknown track

The submitted algorithms will be compared on three different tracks

Optimizing Car Setup in TORCS


TU Dortmund

Optimizing Car Setup

As in real racing competition, finding a good car racing setup is a very challenging problem.

The best set of parametersdepends on:

The track

The driver’s style (in this case AI controller)

The final measure

Challenges:

Limited amount of time available for the optimization

Evaluation times must be determined by the algorithm

Noisy fitness function


TU Dortmund

Which parameters?

A car presents many parameters that can be optimized:

Gear ratios

Rear/Front wing angle

Brakes

Rear differential

Rear/Front anti-roll bars

Wheels

• Ride

• Toe

• Camber

Suspensions

• Spring

• Bell crank…


TU Dortmund

Optimization Framework

The competition framework creates a physical separation between the optimizer and the TORCS engine:

Freedom to choose the programming language of the optimizer

Restrict the access only to a set of parameters defined by the designer

Optimization Algorithm

TORCS

Client Server

Parameter sequence

Fitness evaluation

UDP


TU Dortmund

The optimization process

The parameters involved in the optimization are normalized in the range [0,1]

The optimizer iteratively asks to the TORCS server to evaluate a given sequence of parameters

The optimizer must also specify the duration of the each evaluation (specified in game tics)

Each time an evaluation request arrives, the server:

1. Plug in the parameters in the car setup

2. A programmed policy (Berniw2) drives the car for the specified amount of time

3. Finally, the evaluation results are returned as: lap time, distance raced, top speed and damages

Submissions


TU Dortmund

The competitors

Five new entries have been submitted to the competition:

Jorge Muñoz - Universidad Carlos III de Madrid, Spain -MOEA

Moisés Martínez Muñoz, Emilio Martín Gallardo, Yago SaezAchaerandio – Universidad Carlos III de Madrid, Spain –(12+10)-EA

Pablo José García Evans, Yago Saez Achaerandio –Universidad Carlos III de Madrid, Spain – PSO

Wolfgang Walz, TU Dortmund, Germany - PSO

Antonio J. Fernández, Carlos Cotta, Alberto Fuentes –University of Malaga, Spain

Entries (not ranked) of the organizers:

Basic Genetic Algorithm (Luigi Cardamone et al., Politecnico diMilano)

CMA-ES with well chosen parameters (Markus Kemmerling, TU Dortmund)

The Results


TU Dortmund

Scoring process

Scoring process involved three tracks (unknown to the competitors):

Dirt 3 (a dirty track)

Poli-Track (unknown)

CG Track 2 (rather easy and fast)

For each track, 10 runs of each optimization technique have been performed

Each run is stopped after that 1.000.000 of game tics expires (approximately 2 minutes of computation)

The champion of each run was evaluated scoring the distance raced in the first 10.000 game tics of a race


TU Dortmund

2010 Tracks Pictures

Poli-trackDirt-3

CG track 2


TU Dortmund

CG track 2

0

2000

4000

6000

8000

10000

12000

avg

var


TU Dortmund

Poli-track

0100020003000400050006000700080009000

avg

var


TU Dortmund

Dirt-3

0

1000

2000

3000

4000

5000

6000

7000

avg

var


TU Dortmund

And now some Points (as in Formula-1):

Competitor CG track 2 Poli-track Dirt-3 Overall

Munoz-MOEA 10

Garcia/Saez-PSO 6

Walz-PSO 8

Fernandez/Cotta/Fuentes 4

Munoz/Martin/Saez-EA 5


TU Dortmund



Munoz-MOEA 10 6

Garcia/Saez-PSO 6 10

Walz-PSO 8 5

Fernandez/Cotta/Fuentes 4 4

Munoz/Martin/Saez-EA 5 8


TU Dortmund



Munoz-MOEA 10 6 8

Garcia/Saez-PSO 6 10 5

Walz-PSO 8 5 6

Fernandez/Cotta/Fuentes 4 4 10

Munoz/Martin/Saez-EA 5 8 4


TU Dortmund



Munoz-MOEA 10 6 8 24

Garcia/Saez-PSO 6 10 5 21

Walz-PSO 8 5 6 19

Fernandez/Cotta/Fuentes 4 4 10 18

Munoz/Martin/Saez-EA 5 8 4 17

And the winner is:

Jorge Muñoz - MOEAUniversidad Carlos III de Madrid, Spain


TU Dortmund

……and what about the organizers?


Munoz-MOEA 9831.83 7654.01 6128.29 23614.13

Garcia/Saez-PSO 8386.77 7979.86 5021.41 21388.04

Walz-PSO 8408.35 7304.54 5336.88 21049.77

Fernandez/Cotta/Fuentes 7553.21 5931.47 6263.40 19748.08

Munoz/Martin/Saez-EA 8167.60 7718.36 4629.33 20515.29

Cardamone-SimpleGA 9563.08 7273.06 5932.09 22768.23

Kemmerling-CMA_ES 10410.13 8392.49 6415.87 25218.49


TU Dortmund

Conclusions

Good news:

We had 5 new entries! Wow!

The center of car optimization in Europe is in Spain :-)

And:

Every track was won by a different algorithm

The winner algorithm is multi-objective and uses all 4 available return values: top speed, distanced raced, damage and lap time

3rd to 5th place very tight…

But:

It is obviously not that easy to beat the SimpleGA

Organizer entries show that there is still some potential

Car Setup Optimization Competition @ EvoStar 2010

Technology

Transcript of Car Setup Optimization Competition @ EvoStar 2010