Dynamic Task Allocation in a turn

based strategy game

Gilles SchtickzelleSeptember 2012

ULB

Problem Statement• Creating an intelligent player for a turn-based

strategy game.• Working Framework:

• Many possible challenges to meet:o Resource managemento Adversarial planningo Spatial reasoningo …

A game of FreeCol• Colonization of America• Establish settlements, grow and develop them• Victory: Declare independence & Beat the Royal

Expeditionary Force

Colony Management• Assigning tasks to units for optimal resources production

Division of labor in insect societies

• Ants and wasps colonies have efficient distributed task allocation mechanisms through stygmergy.

Bonabeau, E., Theraulaz, G., & Deneubourg, J.-L. (1996).

Response Threshold

• Ants have probabilistic response to stimuli:

• Varying threshold θ induces specializationo Reduces switching costso Increases individual efficiency

From insects to games• FreeCol

Colony

• Units

• Resources

• Expert units

• Ants/Wasps Colony

• Insects

• Tasks

• Specialization

Resources Dynamics• Surplus: Extra workers.

Shortage: Lose worker.

• Freedom. 50% required to win. Gives bonus or penalty to workers.

• required to make hammers

• Used to produce buildings or artileries

• required to make tools

• Used to produce buildings or artilleries

Allocation Mechanism• One stimulus Sr for each resource r =

• One set of dynamic thresholds θri per unit i

Stimuli and Thresholds

• Simple computation rules for each stimulus

• One set of dynamic thresholds θru per unit u

• Genetic Algorithm to find appropriate scale factors βr

𝜃𝑟𝑢=¿

Simple Scenario

AI goals

1. Reach the year 1776 with enough bells to be able to declare independence.

2. Have the best defense possible to resist the attack of the royal expeditionary force.

3. Allocate workers to1. minimize famine2. Keep the production modifier as high as possible

Results (Basic player) 

Freedom % Size Famine Military

EXPERT 100% 14 0 24

MEAN (100 games) 91.57 ± 2.39 15.12 ± 0.85 0.18 ± 0.09 15.99 ± 0.62

1 34 67 1001331661992322652983313643974300

0.51

1.52

2.5

(B)Evolution of the production modifier (AI vs Expert)

PRODUCTION MODIFIER (AI)PRODUCTION MODIFIER (EXPERT)

1 34 67 1001331661992322652983313643974300

10

20

30

(C)Evolution of the number of artilleries (AI vs Expert)

ARTILLERIES (AI)ARTILLERIES (EXPERT)

Planning approach• Suboptimal allocation: building too early

• Two planning methods:

o Layered response threshold.

o Rule-based planning.

1 29 57 85 11314116919722525328130933736539342144905

1015202530

(C)Evolution of the number of artilleries (AI vs Expert)

ARTILLERIES (AI)ARTILLERIES (EXPERT)

Planning approach• Layered response threshold :

o Use two sets of scale factors: • Optimized for growth• Optimized for production

• Rule-based planning :

𝑃𝐺𝑅𝑂𝑊𝑇𝐻=𝑆𝐺𝑅𝑂𝑊𝑇𝐻

2

𝑆𝐺𝑅𝑂𝑊𝑇𝐻2 +Θ𝐺𝑅𝑂𝑊𝑇𝐻

2

𝑆𝐻𝐴𝑀𝑀𝐸𝑅𝑆={10 𝑖𝑓 (𝑆𝑡𝑜𝑟𝑎𝑔𝑒𝐻𝐴𝑀𝑀𝐸𝑅𝑆≤𝐵𝑢𝑖𝑙𝑑𝐻𝐴𝑀𝑀𝐸𝑅𝑆𝑜𝑟 𝑆𝑡𝑜𝑟𝑎𝑔𝑒𝐿𝑈𝑀𝐵𝐸𝑅 ≥15

𝑜𝑟 𝑆𝑖𝑧𝑒𝑐>6 )0 𝑖𝑓 𝑛𝑜𝑡

Planning Results (1)Layered AI Rule-based AI

1 23 45 67 89 11113315517719922124326528730933135337539741944102468

1012141618

Evolution of the colony size

SIZE (MULTI LAYER AI) SIZE (EXPERT)

1 23 45 67 89 1111331551771992212432652873093313533753974194410

5

10

15

20

25

30

Evolution of the number of artilleries

ARTILLERIES (MULTI LAYER AI) ARTILLERIES (EXPERT)

1 23 45 67 89 11113315517719922124326528730933135337539741944102468

1012141618

Evolution of the colony size

SIZE (RULE BASED) SIZE (EXPERT)

1 23 45 67 89 1111331551771992212432652873093313533753974194410

5

10

15

20

25

30

Evolution of the number of artilleries

ARTILLERIES (RULE BASED) ARTILLERIES (EXPERT)

Planning Results (2) 

Military SoL % Size Famine

EXPERT 24 100 14 0

MEAN (BASIC) 15.99 ± 0.62 91.57 ± 2.39 15.12 ± 0.85 0.18 ± 0.09

MEAN (LAYERED) 17.49 ± 0.70 99.17 ± 0.65 14.53 ± 0.84 0.19 ± 0.10

MEAN (RULE BASED) 18.80 ± 0.54 98.22 ± 1.28 17.49 ± 1.03 0.16 ± 0.09

Statistics for 100 games with the simple scenario.

Modified Threshold rule

• Unit u produces resource r

• Unit u does not produces resource r

𝜃𝑢𝑟={0.1 𝑖𝑓 (𝑢𝑖𝑠 𝑒𝑥𝑝𝑒𝑟𝑡 𝑓𝑜𝑟 𝑟 )max ( 1, 𝜃𝑢𝑟−1 ) 𝑖𝑓 𝑛𝑜𝑡

𝜃𝑢𝑟={0.1 𝑖𝑓 (𝑢𝑖𝑠 𝑒𝑥𝑝𝑒𝑟𝑡 𝑓𝑜𝑟 𝑟 )min (10 , 𝜃𝑢𝑟 +1 ) 𝑖𝑓 𝑛𝑜𝑡

1 21 41 61 81 1011211411611812012212412612813013210

2

4

6

8

10

12

0

2

4

6

8

10

12

Evolution of thresholds and jobs for unit 5 (non expert)

FOOD BELLS NONE HAMMERS LUMBER TOOLS ORE jobs for unit 5

Thre

shol

ds

Jobs:1=FOOD2=BELLS3=NONE

4=HAMMERS5=LUMBER6=TOOLS

7=ORE

“State of the art” player

• Modified Threshold update rule + rule-based planning

1 24 47 70 93 116139162185208231254277300323346369392415438012345678

Evolution of unit's job over time (single layer AI player)

unit1 unit2 unit3 unit4 unit5 unit6 unit7 unit8 unit9 unit10

Jobs:1 = FOOD2 = BELLS3 = NONE

4 = HAMMERS5 = LUMBER6 = TOOLS

7 = ORE

1 24 47 70 93 116139162185208231254277300323346369392415438012345678

Evolution of unit's job over time (State-of-the-art AI player)

unit1 unit2 unit3 unit4 unit5 unit6 unit7 unit8 unit9 unit10

Jobs1 = FOOD2 = BELLS3 = NONE

4 = HAMMERS5 = LUMBER

6 = ORE7 = TOOLS

AI players comparison

Basic Layered rule-based State of the art0

5

10

15

20

25

30

0

5

10

15

20

25

30

AI players compared (number of military units)

Expert = 24

AI goal completion

1. Reach the year 1776 with enough bells to be able to declare independence.

2. Have the best defense possible to resist the attack of the royal expeditionary force.

3. Allocate workers to1. minimize famine2. Keep the production modifier has high as possible

 Freedom % Size Famine Military

EXPERT 100% 14 0 24

MEAN (100 games) 100% ± 0.92 20.21 ± 0.94 0.04 ± 0.07 20.99 ± 0.55

Conclusions Human-level performances can emerge from

simple rules, without cheating. Easy to implement (compared to traditional rule-based only

AI). Easy to tune down performances (if playing against

non-expert). Hybrid system (with planning instructions)

improves on basic RTM− Tendency to chaos with large number of stimuli− Difficult to extend to other game aspects (combat,

spatial reasoning, diplomacy,…).