CS510 AI and Games Final Report on Dec. 09 2008 Juncao Li.

CS510 AI and Games

Final Report on Dec. 09 2008

Juncao Li

[email protected]

Computer Science, Portland State University 2

Agenda

• About Advanced Protection• My Design and Work• Results and Evaluations• Conclusion and Discussion

[email protected]


Advanced Protection (AP)

• Author: Soren Johnson• Turn-based strategy game

– Player makes strategy each turn– AI plays against player’s strategy

• Static AI V.S. adaptive AI – Player learns the strategy of static AIs– Adaptive AI adapts based on users performance

[email protected]


AP (Cont.)

[email protected]


AP (Cont.)Figure A-1: Terrain Types

Terrain Color Farming Bonus Defense Bonus Impassable?Water

B l u e 0 0 Y

W e t l a n d s L i g h t G r e e n 2 0 N

F o r e s t D a r k G r e e n 1 0 N

H i l l s B r o w n 0 1 N

M o u n t a i n s G r a y - 1 2 N

Figure A-2: Human Units

Unit Cost SalvageAttack

StrengthDefense Strength

Farming Rate

Special Ability

D r o n e $ 2 0 $ 2 0 0 1 1 -

M i n e $ 5 0 $ 2 5 0 0 0 ( 1 )

F a r m e r $ 1 0 0 $ 8 0 0 1 1 -

I n f a n t r y $ 1 0 0 $ 9 0 3 6 0 -

S e t t l e r $ 1 5 0 $ 1 2 0 2 4 3 -

J a m m e r $ 1 5 0 $ 9 0 0 2 0 ( 2 )

A r m o r $ 2 0 0 $ 1 5 0 8 1 0 0 -

A r t i l l e r y $ 4 0 0 $ 3 0 0 3 2 0 ( 3 )

[email protected]


AP (Cont.)Figure A-3: Chaos's Minions

Unit CostAttack

StrengthDefense Strength

Movement Rate

Hearing Radius

Special Ability

Scout$ 1 0 0 1 1 - ( 4 )

S c a v e n g e r $ 1 0 0 0 1 1 - ( 5 )

C o m m o n B a r b a r i a n $ 5 0 6 3 1 2 -

A m p h i b i o u s B a r b a r i a n $ 6 0 6 3 1 2 ( 6 )

M o b i l e B a r b a r i a n $ 7 0 5 3 2 3 -

J u m p B a r b a r i a n $ 7 0 6 2 1 2 ( 7 )

K a m i k a z e e B a r b a r i a n $ 7 0 0 1 0 2 2 ( 8 )

A r m o r e d B a r b a r i a n $ 2 0 0 1 5 1 0 1 3 ( 9 )

[email protected]


AP (Cont.)

• Features– Each minion has a brain (automaton) encoded by

a 128-bit string– Four behaviors depending on the input:

• Move forward, turn right, turn left and do actions

– 250 brains for a minion to choose• 20 hardcoded, 230 from genetic algorithm• During the play, brains are rated based on their

performance against the player

– Dynamically choose best-fit brains• Each player has their own easy and hard minion brains • Players does not loose or win too much

[email protected]


Minion InputsFigure 3: Scout Inputs

Input Meaning000

N o i n p u t

0 0 1 F a c i n g i m p a s s a b l e t e r r a i n

0 1 0 S e n s e s j a m m i n g r a d i u s

0 1 1 S e n s e s a r t i l l e r y r a d i u s

1 0 0 F a c i n g d r o n e o r f a r m e r

1 0 1 F a c i n g i n f a n t r y , s e t t l e r , j a m m e r , o r a r t i l l e r y

1 1 0 F a c i n g a r m o r

1 1 1 F a c i n g m i n e

Figure 4: Scavenger Inputs

Input Meaning000

F a c i n g h u m a n


0 1 0 S e n s e g o l d

0 1 1 F a c i n g g o l d

1 0 0 N o f a r m i n g b o n u s

1 0 1 + 1 f a r m i n g b o n u s

1 1 0 + 2 f a r m i n g b o n u s

1 1 1 + 3 f a r m i n g b o n u s

Figure 5: Barbarian Inputs

Input Meaning000

N o i n p u t


0 1 0 S e n s e s s c o u t b r o a d c a s t

0 1 1 F a c i n g s c o u t b r o a d c a s t

1 0 0 F a c i n g d r o n e o r f a r m e r

1 0 1 F a c i n g i n f a n t r y o r s e t t l e r

1 1 0 F a c i n g a r m o r

1 1 1 F a c i n g j a m m e r o r a r t i l l e r y

[email protected]


Why Brain Matters

[email protected]


Why Brain Matters (Cont.)

[email protected]


Agenda


[email protected]


My Goal & Why

• Design AIs that play as players– Try to learn the player’s strategy– Try to perform best on all 250 brains

• Because Chaos selects brains based on player’s strategy• Win (defined later) brains as much as it could

• Design benchmark for the AIs on each side– How well each AI performs

• Learning about players helps game designs– Player will not like a game that they can never win!– Player will not like a game that they can easily win!

[email protected]


Finite State Machine (FSM)

• Code the game strategy in a FSM• Inputs or what matters

– Treasury: Human and Chaos– Terrain and current human nodes on the map

• Output– Where to place the units– What units to place– Based on current state

• Allocate more farmers for earning money• Allocate more infantry for attacking Chaos

[email protected]


FSM (Cont.)

[email protected]


FSM (Cont.)

• My FSM is static• Hard to make it dynamic and adaptive

– Too many possibilities (24*24 size map plus human ……)

• Works well as a benchmark– How well the Chaos AI performs

• Train Neural Network (NN)– Hopefully, teach the NN to recognize the map

[email protected]


Use Neural Network (NN)

• Input nodes: MAP_SIZE+2 – Map size: 24*24 = 576– Human+Chaos treasury: 1+1

• Output nodes: MAP_SIZE– Human units placement

• One hidden layer: MAP_SIZE+2• Normalize the inputs/outputs range: [0.0, 1.0]• Use other’s NN code:

– AI Game Engine: well developed, but not efficient– Tim Jones book: simple and efficient NN

[email protected]


Train the NN

• Firstly, use the static FSM to train the NN– Random generated maps– Random initial Human/Chaos treasury – FSM generates outputs as the training case

• Secondly, improve the NN– Randomly generate strategy– If it performs better than current NN– Train NN with the strategy

[email protected]


Understand the NN Outputs

• NN doesn’t always tell the exact answer• So we have to “guess”

Human_T MatchHuman(float type){float threashold = 0.2;type = type*HUMAN_TYPES;if(type<=threashold || type>HUMAN_TYPES+threashold)

return NO_HUMAN;

if(fabs(type-float(DRONE))<threashold)return DRONE;

else if(fabs(type-float(MINE))<threashold)return MINE;

else if(fabs(type-float(FARMER))<threashold)return FARMER;

else if(fabs(type-float(INFANTRY))<threashold)return INFANTRY;

else if(fabs(type-float(SETTLER))<threashold)return SETTLER;

else if(fabs(type-float(ARMOR))<threashold)return ARMOR;

else if(fabs(type-float(JAMMER))<threashold)return JAMMER;

else if(fabs(type-float(ARTILLERY))<threashold)return ARTILLERY;

return NO_HUMAN;}

CNNPlayer::DoTurns(){ ……//normalize the outputfloat low = actual[0];float high = actual[0];float avg = 0;for(i=0; i<OUTPUT_NEURONS; i++){

if(low>actual[i])low=actual[i];

if(high<actual[i])high=actual[i];

avg+=actual[i]/OUTPUT_NEURONS;}

for(i=0; i<OUTPUT_NEURONS; i++){if(actual[i]<=0)

actual[i] = 0;else{

actual[i] = (actual[i]-low)/(high-low);//actual[i] = actual[i]/avg;

}}……}

[email protected]


NN Outputs (4 hour & 1M inter.)

[email protected]


Recall the FSM

[email protected]


A Random NN Give You This!

[email protected]


Agenda


[email protected]


Evaluations

• Test based on each turn– Only for each turn, the Chaos’ brain is certain– My AI performs well (win) if and only if:

• Given random initial treasury and map• It has advantage against most Chaos’ brains (money

earned)

• Good performance on each turn leads to final win

[email protected]


Statistics of My FSM

Initial Treasury

We want this!

Not well dealing with money between this range

[email protected]


Statistics of My NN AI

Initial Treasury

We want this!

[email protected]


Agenda


[email protected]


Conclusion

• Studied AP– Adaptive turn-based strategy game

• Static FSM– Benchmark– Train NN

• NN– Developed a demo – Trained by FSM– Need more time to train it

[email protected]


Discussion

• Static FSM– Efficient to code– Pros: do exactly what you want– Cons: do exactly what you want

• NN– Like dealing with a child

• Patient• Be careful what you feed to it• Try to understand a child’s language• More trained, usually better performance

[email protected]


Discussion (Cont.)

• Coding & debugging issues– Error in coding NN is not easy to be detected

• Wrong training data• Inconsistent between training inputs and using inputs

– Be careful about important parts• Training data• Training process

– Set breakpoint to check– Code review– Train it by the same example, see if it adapts as

expected

[email protected]


Questions ?

CS510 AI and Games Final Report on Dec. 09 2008 Juncao Li.

Documents

Transcript of CS510 AI and Games Final Report on Dec. 09 2008 Juncao Li.