Designing and Evolving an Unreal Tournament 2004

Expert Bot

A.M. Mora, F. Aisa, R. Caballero, P. García-Sánchez, J.J. Merelo, P.A. Castillo, R. Lara-Cabrera

International Work-Conference on Artificial Neural Networks

INDEX

• Unreal (game, environment)

• Pogamut tool

• Unreal Expert Bot (objectives, features)

• Evolutionary Algorithms

• Unreal Expert Bot Evolution (description,

approaches, results)

• E-BOT vs GE-BOT (results, demo)

• Conclusions

Environment descriptionUNREAL

Unreal is a first person shooter (FPS)first person shooter (FPS).

Famous due to the excelent AI of the enemiesexcelent AI of the enemies (bots), which makes it an amazing multiplayer game. Unreal Tournament series is widely extended.

It offers an editor (UnrealEd) editor (UnrealEd) which lets us change almost anything in the game even the behaviour of the bots. It uses the language UnrealScriptUnrealScript.

General descriptionPOGAMUT

A java middleware java middleware for Unreal Tournament series games and Defcon games.

The architecture is as follows:

It is possible to interact with the game from a java program, getting higher independence higher independence (avoiding Unrealscript restrictions)and increasing the possibilitiesincreasing the possibilities (java libraries).

On the contrary, the structures, classes, functions andstructures, classes, functions and

workflows defined in the Unreal engine, cannot be accessedworkflows defined in the Unreal engine, cannot be accessed, nor used.

ObjectivesUNREAL EXPERT BOT

• Create an autonomous agent for playing Unreal Tournament deathmatch championship.

• Considering the constraints of this competition:- 1 vs 1 matches- Small arenas- Weapons are not respawned- Some forbidden items (U-Damage, for instance)- 15 minutes per match instead of a number of frags (kills)

• Human-like behaviour is desired.

• Modelling Expert player knowledge (and tricks).- High control in timing (items respawn time)- Deep knowledge about weapons and their advantages and

disadvantages- Deep knowledge about items

FeaturesUNREAL EXPERT BOT

• Defined by means of a Finite State Machine based AIState Machine based AI with two state levels.

• Translated into a set of rules which determine its behaviour.

• Database which models the bot’s memory, since it is uploaded with data about locations of items and weapons in the map.

Bot performanceUNREAL EXPERT BOT

• Expert Bot (E-Bot) outperformed the standard bots in the game (considering the number of frags), even in the maximum difficulty level.

• This difficulty level is quite hard for a medium level player.

• E-Bot is hard to beat for humans, even for the expert.

• Medium level players usually lose against it.

Evolutionary AlgorithmsEXPERT BOT EVOLUTION

by Johann Dréo

i -> initial populationf -> evaluation function (fitness)? -> stop conditionSe -> selectionCr -> crossoverMu -> mutationRe -> replacement

Evolutionary Process in Unreal gameEXPERT BOT EVOLUTION

GE-BOTExpert Bot based in a

Genetic Algorithm

Evolutionaryprocess

population

FITNESS EVALUATION

• Analyze Expert bot’s FSM• Identify parameters • Optimize them

Expert Bot’s

AI

ApproachesEXPERT BOT EVOLUTION

• Generic FitnessJust considers frags/deads and damage produced/received

• Generational scheme • 4-elitism

• Complex Fitness- considers frags/deads - damage produced/received- time using the best or more

versatile weapons: Lightning Gun and Shock Rifle

- getting the best items: Shield and Super Shield

• Stationary scheme

• Chromosome 143

• Uniform Crossover• Random mutation• 4 Random individuals

• Chromosome 26

Approach 1: Chromosome 143 - Generic Fitness

EXPERT BOT EVOLUTION

• Generic FitnessJust considers frags/deads and damage produced/received

• Generational scheme • 4-elitism

• Chromosome 143

• Uniform Crossover• Random mutation• 4 Random individuals

Approach 1. ResultsEXPERT BOT EVOLUTION

• 30 generations• 30 individuals• 1 evaluation (left)• 3 evaluations (right) in order to avoid the noisy nature of the fitness function• 15 minutes per

evaluation

• 10 days per run (left)• One month (right)• Lightly improvement

tendency• Too many oscillations,

i.e. noise

• 143 genes are too much

EXPERT BOT EVOLUTION

• Generic FitnessJust considers frags/deads and damage produced/received

• Generational scheme • 4-elitism

• Uniform Crossover• Random mutation• 4 Random individuals

• Chromosome 26

Approach 2: Chromosome 26 - Generic Fitness

Approach 2. ResultsEXPERT BOT EVOLUTION

• 50 generations• 30 individuals• 5 minutes per

evaluation

• Results of 2 different runs

• 5 days per run• Again lightly

improvement tendency• Too much noise

• Too much diversity

EXPERT BOT EVOLUTION

• Complex Fitness- considers frags/deads - damage produced/received- time using the best or more

versatile weapons: Lightning Gun and Shock Rifle

- getting the best items: Shield and Super Shield

• Stationary scheme• Uniform Crossover• Random mutation• 4 Random individuals

• Chromosome 26

Approach 3: Chromosome 26 - Complex Fitness

Approach 3. ResultsEXPERT BOT EVOLUTION

• 40 generations• 30 individuals• 5 minutes per

evaluation• Stationary scheme to

increase the exploitation factor

• Results of 2 different runs

• 5 days per run• Quite good fitness

tendency• Noise still remains,

but in a lower factor

Numerical resultsE-BOT vs GE-BOT

• Expert Bot (E-Bot) and the best Genetic Expert Bots (GE-BOT) have been fighting in four battles (in two maps).

• The average results of these matches are:

• The approach with 143 genes per chromosome is defeated

• GE-Bot with 26 genes outperforms E-Bot.

• The approach with the complex fitness function gets the best results. Due to its lower noisy factor, and the higher exploitation component.

DEMOE-BOT vs GE-BOT

http://www.youtube.com/watch?v=ktcXHZ-nAfw

CONCLUSIONS

• We have designed a human-like Expert Bot (E-Bot) which outperforms the standard Unreal Tournament 2K4 bots in the hardest difficulty.

• It is also a hard rival against human players.

• We have tested three different approaches for improving this bot by means of Genetic Algorithms.

• Too long chromosomes population performs worse than small length one.

• These algorithms are affected by a high noisy factor regarding the generic (and easier) fitness function.

• We have defined a complex fitness function which performs better, with a softer noisy effect.

• The bots obtained after evolution outperform the E-Bot.

ENDTHE

Questions?!?!

Contact: amorag@geneura.ugr.es

Source Code: https://github.com/franaisa/ExpertAgent