Agent Architecture Considerations for Real-Time Planning in Games Jeff Orkin Monolith Productions.

Agent Architecture Considerations for Real-Time Planning in Games

Jeff Orkin

Monolith Productions

F.E.A.R.

Agenda

MotivationProblemsSolutionsWas it worth it?

Agenda

Motivation – Why plan?Problems – Performance!Solutions – Agent architectureWas it worth it?

Why plan in real-time?

Goal-Oriented Behavior

Goal-Oriented Behavior

Problems:Managing dependenciesSharing behaviors

Problem: Dependencies

AttackFromCover

Gotot Attack


AttackFromCover

Gotot AttackDraw t


AttackFromCover

Gotot AttackDraw t

Gotot Flipt


AttackFromCover

Gotot AttackDrawt

Gotot

Flipt

Lightt

Lightt

Reloadt


tt tt tttt

tttttt tt

Problem: Sharing

No One Lives Forever 2 TRON 2.0

P.D.D.L.

Planning Domain Definition Language Goals

– Desired state

Actions– Preconditions– Effects

P.D.D.L.

Goal:(define (problem get-paid)

(:domain briefcase-world)(:init (place home) (place office)

(object p) (object d) (object b)(at B home) (at P home) (at D home) (in

P))(:goal (and (at B office) (at D office) (at P home))))

P.D.D.L.

Action:(:action put-in

:parameters (?x - physob ?l - location):precondition (not (= ?x B)):effect (when (and (at ?x ?l) (at B ?l))

(in ?x)) )

Other actions: take-out, move

P.D.D.L.

Modular– Goals– Actions

Decoupled Modules Related by symbols– World State– Preconditions– Effects

Applied PDDL’s structure to C++ toolkit in game code.


Management

How to Plan in Real-Time and Keep Your Job

Jeff Orkin


The Plan

Goal: KillEnemy

Plan:

Goto (couch)

UseObject (couch)

Goto (coverNode)

AttackFromCover

AI Performance Guideline: 1.0ms / frame

1 .0 m s

AI Performance: Off the Chart!

1 .0m s

30 0m s


Preconditions: Visibility

1 .0m s

30 0m s


Preconditions: Visibility Pathfinding

1 .0m s

30 0m s


Preconditions: Visibility Pathfinding Tactical Position

Validity

1 .0m s

30 0m s

Solution

Re-consider Agent ArchitectureDistributed processingCaching

Solution

Re-consider Agent ArchitectureDistributed processingCaching

Inspiration:MIT Media Lab’s C4Robotics

Solution

SensorsWorkingMemory

Planner

BlackboardNav igation

Animation /Movement

Targeting

Weapons

World

World

What is a soldier?

What is a soldier?

Sensors: See

What is a soldier?

Sensors: See Hear

What is a soldier?

Sensors: See Hear Feel pain

What is a soldier?

Subsystems: Navigate / Move

What is a soldier?

Subsystems: Navigate / Move Attention Selection

(Targeting)

What is a soldier?

Subsystems: Navigate / Move Attention Selection

(Targeting) Weapons

Distributed Processing with Sensors

Pseudo-parallel Amortize precondition processing across

many frames Allow incremental processing


More than Vision, Hearing, Touch Tactical analysis Internal desires


Update:– Every frame – Periodic Polling – Event-driven

Limit total number of expensive sensor updates per frame.

Sensor Example: CoverNode Sensor

Update 3 times / second

E

A

Sensor Example: Cover Node Sensor

Update 3 times / second

E

A

Sensor Example: PassTarget Sensor

Incremental update

AE

Sensor Example: PassTarget Sensor

Incremental update

A

E

Caching

Sensor

Sensor

Sensor

WorkingMemory

Targeting

Planning

Caching: WorkingMemoryFacts

Facts are really beliefs Common representation of all knowledge:

– Character– Object– Node– Disturbance– Task– PathInfo– Desire


WorkingMemoryFact {

Attribute<Vector3D> PositionAttribute<Vector3D> DirectionAttribute<StimulusType> StimulusAttribute<HANDLE> ObjectAttribute<DesireType> Desire

// 16 attributes total...}


Attribute<Type>{

Type Valuefloat fConfidence

}


SeeEnemy Sensor: Stimulus Confidence

C = 0.0

C = 0.0



C = 1.0

C = 1.0



C = 1.0

C = 0.63



C = 1.0

C = 0.0


CoverNode Sensor: Position Confidence

E

A

C=0.2

C=0.1C=0.4 C=0.9

C=0.8

The Planning Process


P

The Plan

Goal: KillEnemy

Plan:

Goto (couch)

UseObject (couch)

Goto (coverNode)

AttackFromCover

The Plan

Plan:

Goto (couch)

UseObject (couch)

Goto (coverNode)

AttackFromCover

Goal: KillEnemy


Goal: KillEnemy

Satisfaction: ???

Planning Process: Symbols

Simple representation of World State: Array of symbols Fixed number of symbols Symbol = Key, Value pair Union of 4 byte values (int, float, HANDLE, etc)


Simple representation of agent’s World State: Array of symbols Fixed number of symbols Symbol = Key, Value pair Union of 4 byte values (int, float, HANDLE, etc)

Examples: kWeaponArmed = TRUE kUsedObject = hSomeObject


Goal: KillEnemy

Satisfaction: kTargetIsDead = TRUE


Agent-centric representation

Character XStim Conf = 0.6

Character YStim Conf = 1.0

Character ZStim Conf = 0.0

Working Memory

Targeting

Blackboard

T arget =Character Y


Agent-centric representation– kTargetIsDead– Subsystem handles target selection.– Select from Character Facts in WorkingMemory.– Less symbols to evaluate during plan formulation.

The Plan

Plan:

Goto (couch)

UseObject (couch)

Goto (coverNode)

AttackFromCover

Goal: KillEnemy

Planning Process: Actions

AttackUse

Object

DrawW eapon Look At

Goto

Reload

Idle


Action Preconditions & Effects: Represented with same array of symbols Value may be constant or variable


Attack

Attackfrom

Vehicle

Attackfrom

Cov er

E: kT argetIsDead = T RUE


Attack

Attackfrom

Vehicle

Attackfrom

Cov er

E: kT argetIsDead = T RUE

Cost = 5Cost = 3

Cost = 1

Planning Process: Context Preconditions

Context preconditions Filter function of arbitrary code

– Am I riding a vehicle?– Does a path exist?– Is object visible?– Is valid cover available?

Symbolic preconditions are only used when we want the planner to chain a preceding action.


A: AttackFromCover

bool ValidateContextPreconditions()

{

// Is target visible?

...

// Does a valid node

// exist?

}

Character X

Working Memory

Character Z

Character Y

CoverNode576

CoverNode34

CoverNode29


Query QQ.Type = kFact_NodeQ.Node = kNode_Cover

Fact* pFact = FindFactMax( Q, kAtt_Position )

Finds the WMFact with highest confidence in some attribute.

If returns NULL, no valid cover exists.

The Plan

Plan:

Goto (couch)

UseObject (couch)

Goto (coverNode)

AttackFromCover

Goal: KillEnemy

Planning Process: Variables

G: KillEnemyS: kTargetIsDead = TRUE

A: AttackFromCoverE: kTargetIsDead = TRUE


G: KillEnemyS: kTargetIsDead = TRUE


P: kWeaponArmed = TRUE

kWeaponLoaded = TRUE

kAtObject = ???



P: kWeaponArmed = TRUE

kWeaponLoaded = TRUE

kAtObject = FindFactMax( Q, kAtt_Position )


A: GotoE: kAtObject = Parent.kAtObject


A: AttackFromCoverP: kAtObject =

A: GotoE: kAtObject =

FindFactMax( Q, kAtt_Position )

Parent.kAtObject


KillEnemyAttackFromCover

Goto

The Plan

Plan:

Goto (couch)

UseObject (couch)

Goto (coverNode)

AttackFromCover

Goal: KillEnemy


A: Goto

E: kAtObject = Parent.kAtObject


A: GotoE: kAtObject =

P: kUsedObject =

Parent.kAtObject

GetDependency( Parent.kAtObject )

The Plan

Plan:

Goto (couch)

UseObject (couch)

Goto (coverNode)

AttackFromCover

Goal: KillEnemy

Was It Worth It?

Was It Worth It?

Did we handle? Managing dependencies Sharing behaviors

Additional benefit: Replanning

Replanning

PA

Replanning

P A

Parting Thoughts

Warcraft (1994)

Parting Thoughts

Star Wars (1987)

Parting Thoughts

F.E.A.R. (2005)

Parting Thoughts

2D 3D

Pre-rendered Real-time rendered

graphics graphics

Parting Thoughts

Scripts, FSMs Planning

Pre-rendered Real-time simulated

reasoning & behavior reasoning & behavior

What new types of gameplay are possible?!

Questions?

Jeff Orkin


http://www.jorkin.com

Agent Architecture Considerations for Real-Time Planning in Games Jeff Orkin Monolith Productions.

Documents

Transcript of Agent Architecture Considerations for Real-Time Planning in Games Jeff Orkin Monolith Productions.