Chapter 5.3Artificial Intelligence:Agents, Architecture, and Techniques

Artificial IntelligenceIntelligence embodied in a man-made deviceHuman level AI still unobtainable

Game Artificial Intelligence:What is considered Game AI?Is it any NPC behavior?A single if statement?Scripted behavior?Pathfinding?Animation selection?Automatically generated environment?Best shot at a definition of game AI?

Possible Game AIDefinitionInclusive view of game AI:

Game AI is anything that contributes to the perceived intelligence of an entity, regardless of whats under the hood.

Goals of anAI Game ProgrammerDifferent than academic or defense industry

1. AI must be intelligent, yet purposely flawed2. AI must have no unintended weaknesses3. AI must perform within the constraints4. AI must be configurable by game designers or players5. AI must not keep the game from shipping

Specialization ofGame AI DeveloperNo one-size fits all solution to game AIResults in dramatic specialization Strategy GamesBattlefield analysisLong term planning and strategyFirst-Person Shooter GamesOne-on-one tactical analysisIntelligent movement at footstep levelReal-Time Strategy games the most demanding, with as many as three full-time AI game programmers

Game AgentsMay act as anOpponentAllyNeutral character

Continually loops through the Sense-Think-Act cycleOptional learning or remembering step

Sense-Think-Act Cycle:SensingAgent can have access to perfect information of the game worldMay be expensive/difficult to tease out useful infoGame World InformationComplete terrain layoutLocation and state of every game objectLocation and state of playerBut isnt this cheating???

Sensing:Enforcing LimitationsHuman limitations?Limitations such asNot knowing about unexplored areasNot seeing through wallsNot knowing location or state of playerCan only know about things seen, heard, or told aboutMust create a sensing model

Sensing:Human Vision Model for AgentsGet a list of all objects or agents; for each:1. Is it within the viewing distance of the agent?How far can the agent see?What does the code look like?2. Is it within the viewing angle of the agent?What is the agents viewing angle?What does the code look like?3. Is it unobscured by the environment?Most expensive test, so it is purposely lastWhat does the code look like?

Sensing:Vision ModelIsnt vision more than just detecting the existence of objects?

What about recognizing interesting terrain features?What would be interesting to an agent?

Sensing:Human Hearing ModelHumans can hear soundsCan recognize soundsKnows what emits each soundCan sense volumeIndicates distance of soundCan sense pitchSounds muffled through walls have more bassCan sense locationWhere sound is coming from

Sensing:Modeling HearingHow do you model hearing efficiently?Do you model how sounds reflect off every surface?How should an agent know about sounds?

Sensing:Modeling Hearing EfficientlyEvent-based approachWhen sound is emitted, it alerts interested agentsUse distance and zones to determine how far sound can travel

Sensing:CommunicationAgents might talk amongst themselves!Guards might alert other guardsAgents witness player location and spread the wordModel sensed knowledge through communicationEvent-driven when agents within vicinity of each other

Sensing:Reaction TimesAgents shouldnt see, hear, communicate instantaneouslyPlayers notice!Build in artificial reaction timesVision: to secondHearing: to secondCommunication: > 2 seconds

Sense-Think-Act Cycle: ThinkingSensed information gatheredMust process sensed informationTwo primary methodsProcess using pre-coded expert knowledgeUse search to find an optimal solution

Thinking:Expert KnowledgeMany different systemsFinite-state machinesProduction systemsDecision treesLogical inferenceEncoding expert knowledge is appealing because its relatively easyCan ask just the right questionsAs simple as if-then statementsProblems with expert knowledgeNot very scalable

Thinking:SearchEmploys search algorithm to find an optimal or near-optimal solutionA* pathfinding common use of search

Thinking:Machine LearningIf imparting expert knowledge and search are both not reasonable/possible, then machine learning might workExamples:Reinforcement learningNeural networksDecision tree learningNot often used by game developersWhy?

Thinking:Flip-Flopping DecisionsMust prevent flip-flopping of decisionsReaction times might help keep it from happening every frameMust make a decision and stick with itUntil situation changes enoughUntil enough time has passed

Sense-Think-Act Cycle:ActingSensing and thinking steps invisible to playerActing is how player witnesses intelligenceNumerous agent actions, for example:Change locationsPick up objectPlay animationPlay sound effectConverse with playerFire weapon

Acting:Showing IntelligenceAdeptness and subtlety of actions impact perceived level of intelligenceEnormous burden on asset generationAgent can only express intelligence in terms of vocabulary of actionsCurrent games have huge sets of animations/assetsMust use scalable solutions to make selections

Extra Step in Cycle:Learning and RememberingOptional 4th stepNot necessary in many gamesAgents dont live long enoughGame design might not desire it

LearningRemembering outcomes and generalizing to future situationsSimplest approach: gather statisticsIf 80% of time player attacks from leftThen expect this likely eventAdapts to player behavior

RememberingRemember hard factsObserved states, objects, or playersFor exampleWhere was the player last seen?What weapon did the player have?Where did I last see a health pack?Memories should fadeHelps keep memory requirements lowerSimulates poor, imprecise, selective human memory

Rememberingwithin the WorldAll memory doesnt need to be stored in the agent can be stored in the worldFor example:Agents get slaughtered in a certain areaArea might begin to smell of deathAgents path planning will avoid the areaSimulates group memory

Making Agents StupidSometimes very easy to trounce playerMake agents faster, stronger, more accurateSometimes necessary to dumb down agents, for example:Make shooting less accurateMake longer reaction timesEngage player only one at a timeChange locations to make self more vulnerable

Agent CheatingPlayers dont like agent cheatingWhen agent given unfair advantage in speed, strength, or knowledgeSometimes necessaryFor highest difficultly levelsFor CPU computation reasonsFor development time reasonsDont let the player catch you cheating!Consider letting the player know upfront

Finite-State Machine (FSM)Abstract model of computationFormally:Set of statesA starting stateAn input vocabularyA transition function that maps inputs and the current state to a next state

Finite-State Machine:In Game DevelopmentDeviate from formal definition1. States define behaviors (containing code)Wander, Attack, Flee2. Transition function divided among statesKeeps relation clear3. Blur between Moore and Mealy machinesMoore (within state), Mealy (transitions)4. Leverage randomness5. Extra state informationFor example, health

Most common game AI software patternNatural correspondence between states and behaviorsEasy to diagramEasy to programEasy to debugCompletely general to any problemProblemsExplosion of statesOften created with ad hoc structure

Finite-State Machine:UML Diagram

Finite-State Machine:ApproachesThree approachesHardcoded (switch statement)ScriptedHybrid Approach

Finite-State Machine: Hardcoded FSMvoid RunLogic( int * state ) { switch( state ) { case 0: //Wander Wander(); if( SeeEnemy() ) { *state = 1; } break; case 1: //Attack Attack(); if( LowOnHealth() ) { *state = 2; } if( NoEnemy() ) { *state = 0; } break;

case 2: //Flee Flee(); if( NoEnemy() ) { *state = 0; } break; }}

Finite-State Machine: Problems with switch FSM1. Code is ad hocLanguage doesnt enforce structure2. Transitions result from pollingInefficient event-driven sometimes better3. Cant determine 1st time state is entered4. Cant be edited or specified by game designers or players

Finite-State Machine:Scripted with alternative languageAgentFSM{ State( STATE_Wander ) OnUpdate Execute( Wander ) if( SeeEnemy ) SetState( STATE_Attack ) OnEvent( AttackedByEnemy ) SetState( Attack ) State( STATE_Attack ) OnEnter Execute( PrepareWeapon ) OnUpdate Execute( Attack ) if( LowOnHealth ) SetState( STATE_Flee ) if( NoEnemy ) SetState( STATE_Wander ) OnExit Execute( StoreWeapon ) State( STATE_Flee ) OnUpdate Execute( Flee ) if( NoEnemy ) SetState( STATE_Wander )}

Finite-State Machine:Scripting Advantages1. Structure enforced2. Events can be handed as well as polling3. OnEnter and OnExit concept exists4. Can be authored by game designersEasier learning curve than straight C/C++

Finite-State Machine:Scripting DisadvantagesNot trivial to implementSeveral months of developmentCustom compilerWith good compile-time error feedbackBytecode interpreterWith good debugging hooks and supportScripting languages often disliked by usersCan never approach polish and robustness of commercial compilers/debuggers

Finite-State Machine:Hybrid ApproachUse a class and C-style macros to approximate a scripting languageAllows FSM to be written completely in C++ leveraging existing compiler/debuggerCapture important features/extensionsOnEnter, OnExitTimersHandle eventsConsistent regulated structureAbility to log historyModular, flexible, stack-basedMultiple FSMs, Concurrent FSMsCant be edited by designers or players

Finite-State Machine:ExtensionsMany possible extensions to basic FSMOnEnter, OnExitTimersGlobal state, substatesStack-Based (states or entire FSMs)Multiple concurrent FSMsMessaging

Common Game AI TechniquesWhirlwind tour of common techniques

Common AI Techniques:A* PathfindingDirected search algorithm used for finding an optimal path through the game worldA* is regarded as the bestGuaranteed to find a path if one existsWill find the optimal pathVery efficient and fast

Common AI Techniques:Command HierarchyStrategy for dealing with decisions at different levelsFrom the general down to the foot soldierModeled after military hierarchiesGeneral directs high-level strategyFoot soldier concentrates on combat

Common AI Techniques:Dead ReckoningMethod for predicting objects future position based on current position, velocity and accelerationWorks well since movement is generally close to a straight line over short time periodsCan also give guidance to how far object could have moved

Common AI Techniques:Emergent BehaviorBehavior that wasnt explicitly programmedEmerges from the interaction of simpler behaviors or rules

Common AI Techniques:FlockingExample of emergent behaviorSimulates flocking birds, schooling fishDeveloped by Craig Reynolds1987 SIGGRAPH paperThree classic rules1. Separation avoid local flockmates2. Alignment steer toward average heading3. Cohesion steer toward average position

Common AI Techniques:FormationsGroup movement techniqueMimics military formationsSimilar to flocking, but actually distinctEach unit guided toward formation positionFlocking doesnt dictate goal positions

Common AI Techniques:Influence MappingMethod for viewing/abstracting distribution of power within game worldTypically 2D grid superimposed on landUnit influence is summed into each grid cellUnit influences neighboring cells with falloffFacilitates decisionsCan identify the front of the battle Can identify unguarded areas

Common AI Techniques:Level-of-Detail AIOptimization technique like graphical LODOnly perform AI computations if player will noticeFor exampleOnly compute detailed paths for visible agentsOff-screen agents dont think as often

Common AI Techniques:Manager Task AssignmentManager organizes cooperation between agentsManager may be invisible in gameAvoids complicated negotiation and communication between agentsManager identifies important tasks and assigns them to agents

Common AI Techniques:Obstacle AvoidancePaths generated from pathfinding algorithm consider only static terrain, not moving obstaclesGiven a path, agent must still avoid moving obstaclesRequires trajectory predictionRequires various steering behaviors

Common AI Techniques:ScriptingScripting specifies game data or logic outside of the games source languageScripting influence spectrumLevel 0: Everything hardcodedLevel 1: Data in files specify stats/locationsLevel 2: Scripted cut-scenes (non-interactive)Level 3: Lightweight logic, like trigger systemLevel 4: Heavy logic in scriptsLevel 5: Everything coded in scripts

Common AI Techniques:Scripting Pros and ConsProsScripts changed without recompiling gameDesigners empoweredPlayers can tinker with scriptsConsMore difficult to debugNonprogrammers required to programTime commitment for tools

Common AI Techniques:State MachineMost common game AI software patternSet of states and transitions, with only one state active at a timeEasy to program, debug, understand

Common AI Techniques:Stack-Based State MachineAlso referred to as push-down automataRemembers past statesAllows for diversions, later returning to previous behaviors

Common AI Techniques:Subsumption ArchitecturePopularized by the work of Rodney Brooks Separates behaviors into concurrently running finite-state machinesLower layersRudimentary behaviors (like obstacle avoidance)Higher layersGoal determination and goal seekingLower layers have prioritySystem stays robust

Common AI Techniques:Terrain AnalysisAnalyzes world terrain to identify strategic locationsIdentifyResourcesChoke pointsAmbush pointsSniper pointsCover points

Common AI Techniques:Trigger SystemHighly specialized scripting systemUses if/then rulesIf condition, then responseSimple for designers/players to understand and createMore robust than general scriptingTool development simpler than general scripting

Promising AI TechniquesShow potential for futureGenerally not used for gamesMay not be well knownMay be hard to understandMay have limited useMay require too much development timeMay require too many resources

Promising AI Techniques:Bayesian NetworksPerforms humanlike reasoning when faced with uncertaintyPotential for modeling what an AI should know about the playerAlternative to cheatingRTS ExampleAI can infer existence or nonexistence of player build units

Promising AI Techniques:Blackboard ArchitectureComplex problem is posted on a shared communication spaceAgents propose solutionsSolutions scored and selectedContinues until problem is solvedAlternatively, use concept to facilitate communication and cooperation

Promising AI Techniques:Decision Tree LearningConstructs a decision tree based on observed measurements from game worldBest known game use: Black & WhiteCreature would learn and form opinionsLearned what to eat in the world based on feedback from the player and world

Promising AI Techniques:Filtered RandomnessFilters randomness so that it appears random to players over short termRemoves undesirable eventsLike coin coming up heads 8 times in a rowStatistical randomness is largely preserved without gross peculiaritiesExample:In an FPS, opponents should randomly spawn from different locations (and never spawn from the same location more than 2 times in a row).

Promising AI Techniques:Fuzzy LogicExtension of classical logicIn classical crisp set theory, an object either does or doesnt belong to a setIn fuzzy set theory, an object can have continuous varying degrees of membership in fuzzy sets

Promising AI Techniques:Genetic AlgorithmsTechnique for search and optimization that uses evolutionary principlesGood at finding a solution in complex or poorly understood search spacesTypically done offline before game shipsExample:Game may have many settings for the AI, but interaction between settings makes it hard to find an optimal combination

Promising AI Techniques:N-Gram Statistical PredictionTechnique to predict next value in a sequenceIn the sequence 18181810181, it would predict 8 as being the next valueExampleIn street fighting game, player just did Low Kick followed by Low PunchPredict their next move and expect it

Promising AI Techniques:Neural NetworksComplex non-linear functions that relate one or more inputs to an outputMust be trained with numerous examplesTraining is computationally expensive making them unsuited for in-game learningTraining can take place before game shipsOnce fixed, extremely cheap to compute

Promising AI Techniques:PerceptronsSingle layer neural networkSimpler and easier to work with than multi-layer neural networkPerceptrons get stimulated enough to either fire or not fireSimple yes/no output

Promising AI Techniques:Perceptrons (2)Game example: Black & WhiteCreature used perceptron for hungerThree inputs: low energy, tasty food, and unhappinessIf creature ate and received positive or negative reinforcement, then perceptron weights were modifiedResults in learning

Promising AI Techniques:PlanningPlanning is a search to find a series of actions that change the current world state into a desired world stateIncreasingly desirable as game worlds become more rich and complexRequiresGood planning algorithmGood world representationAppropriate set of actions

Promising AI Techniques:Player ModelingBuild a profile of the players behaviorContinuously refine during gameplayAccumulate statistics and eventsPlayer model then used to adapt the AIMake the game easierMake the game harder

Promising AI Techniques:Production SystemsFormal rule-based systemDatabase of rulesDatabase of factsInference engine to decide which rules trigger resolves conflicts between rulesExampleSoar used experiment with Quake 2 botsUpwards of 800 rules for competent opponent

Promising AI Techniques:Reinforcement LearningMachine learning techniqueDiscovers solutions through trial and errorMust reward and punish at appropriate timesCan solve difficult or complex problems like physical control problemsUseful when AIs effects are uncertain or delayed

Promising AI Techniques:Reputation SystemModels players reputation within the game worldAgents learn new facts by watching player or from gossip from other agentsBased on what an agent knowsMight be friendly toward playerMight be hostile toward playerAffords new gameplay opportunitiesPlay nice OR make sure there are no witnesses

Promising AI Techniques:Smart TerrainPut intelligence into inanimate objectsAgent asks object how to use itAgents can use objects for which they werent originally programmed forAllows for expansion packs or user created objects, like in The SimsEnlightened by Affordance TheoryObjects by their very design afford a very specific type of interaction

Promising AI Techniques:Speech RecognitionPlayers can speak into microphone to control some aspect of gameplayLimited recognition means only simple commands possibleProblems with different accents, different genders, different ages (child vs adult)

Promising AI Techniques:Text-to-SpeechTurns ordinary text into synthesized speechCheaper than hiring voice actorsQuality of speech is still a problemNot particularly natural soundingIntonation problemsAlgorithms not good at voice actingLarge disc capacities make recording human voices not that big a problemNo need to resort to worse sounding solution

Promising AI Techniques:Weakness Modification LearningGeneral strategy to keep the AI from losing to the player in the same way every timeTwo main steps1. Record a key gameplay state that precedes a failure2. Recognize that state in the future and change something about the AI behaviorAI might not win more often or act more intelligently, but wont lose in the same way every timeKeeps history from repeating itself