GI/acm-RegionalgruppeBremen/Oldenburg12. Februar 2002

Ubbo Visser

Soccer: Benchmark for ArtificialIntelligence Methods and Robotics

Overview

Introduction to RoboCupArtificial IntelligenceRobotic Soccer and Artificial IntelligenceRemarks

Introduction to RoboCup

Hypothesis

“By the year 2050, develop ateam of fully autonomoushumanoid robots that can winagainst the human world soccerchampions.”

Hiroaki Kitano, 1999President of the RoboCupFederation

Introduction to RoboCup

The RoboCup Federation

GeneralInternational research &educationFoster AI and intelligentroboticsProviding standard problemIntegrate a wide range oftechnologiesSoccer as primary domain

TechnologiesDesign principles ofautonomous agentsMulti-agent collaborationStrategy acquisitionReal-time reasoningRoboticsSensor fusion

Introduction to RoboCup

RoboCup: Areas of interest

* Multi-Agent/Robot Systems * Robotics, Science Education* Sensor/Motor Control * Adversarial Planning* Self-localization and Navigation * Planning, Reasoning, and Modeling* Vision and Image-Processing * Learning and Adaptive Systems* Cooperation and Collaboration * Simulation and Visualization* Realtime and Concurrent Programming * Embedded and Mobile Hardware* Non-conventional actuation systems * Artificial muscles* Next generation sensors for robotics * Mobile Robots and Humanoids* Search and rescue robots * Adjustable Autonomy* Disaster rescue information systems * System integration* Computer and Robotic Entertainment * Speech Synthesis* Natural Language Generation * Distributed Sensor Fusion* Omnidirectional Vision * Smart Materials* Fuel Cell Batteries * Software Engineering* Dynamic Resource Allocation * Heterogeneous Agents

Introduction to RoboCup

The RoboCup Federation

RoboCupSoccerSimulation LeagueSmall Robot League (F-180)Middle Size Robot League (F-2000)Sony Legged Robot League (Sponsoredby Sony)Humanoid League (From 2002)

RoboCupRescueRescue Simulation LeagueRescue Robot League

RoboCupJuniorSoccerRescueDance

Introduction to RoboCup

Simulation League

Soccer SimulatorTool for multi-agent systemsEnables two teams of 11 simulated autonomous roboticplayers to play soccerTwo coachesPhysics (e.g. stamina, recovery,weight, speed, wind, etc.)Sensors (e.g. see, hear)Time: 6000 cycles, eachcycle has 100ms (�10 min)

Introduction to RoboCup

Small Robot League (F-180)

Five robots, each having a foot print of at most 180 cm²Field: green carpet within a wooden enclosureSize: ping-pong table (274 cm x 152,5 cm)Ball: golf ballTime: 20 minutes, divided in two equal halvesNo camera on board

Introduction to RoboCup

Full video Cornell Big Red vs. Lucky Star Singapore

Middle Size Robot League (F-2000)Four robots, each having a foot print of at most 2025 cm²Field: 9x5m, greenBall: FIFA size 5Time: 20 minutes, divided in two equal halvesOn-board camera & other sensors (e.g. ultrasound,laser,…)

Introduction to RoboCup

Full video CS-Freiburg vs. Trackies, 2001

Sony Legged Robot LeagueFour robotsField: 3,5x2m, green, landmarksTime: twenty minutes, divided in two equal halvesRobot has 20 degrees of freedom7 touch sensors, on-board camera

Introduction to RoboCup

Full video

Humanoid LeagueWalking using two legs, no wheel, approximate body, consists of twolegs, two arms, one body, and one headHeightmax 40, 80, 120cmFields (6*Hmax, 9*Hmax)Solo games (e.g. penalty shootout), games40! humanoids available (e.g. PINO, Asimo, P3)

Introduction to RoboCup

Problems in LeaguesSimulation League

Little co-operation, physics not real, communication via server, mostly reactive agents, mostlyoffline-learning

F-180No camera on-board, no co-operation, no communication, central control, autonomy, mechanicalproblems, no learning

F-2000Image processing in real time (no 25 fps), no learning, no co-operation, extreme mechanicalproblems

Sony Legged League(No) communication, movements, no co-operation, image processing in real time

Humanoid LeagueAll of the above, balance

Introduction to RoboCup

Questions to introduction?

Areas of Artificial Intelligence (AI)

Expert systemsNatural language

systems

Vision

Robotics

Automatic proofsystemsMachine

learning

Automaticprogramming

Applications incognitive

psychology

Logics

LinguisticsPsychology

Optics

Pattern recognition

Mechanicalengineering

Logics

Programverification

Systemprogramming

Linguistics

AppliedSciences

Artificial Intelligence

Methods of AI

Heuristicsearch

Neural networks

Problem

Solving;

Reasoning

AI languages and

systems

Biology

Neurology

Physiology

Logics

Theory ofprogramminglanguages

Parallel systems

Combinatorics

Graph theoryKnowledge

representation

Epistemology

Psychology

Artificial Intelligence

What is Artificial Intelligence (AI)?

“[The automation of] activities that weassociate with human thinking, activitiessuch as decision-making, problem solving,learning…'' (Bellman, 1978)

“The study of mental faculties through theuse of computational models.''(Charniak+McDermott, 1985)

“The study of how to make computers dothings at which, at the moment, people arebetter.''(Rich+Knight, 1991)

“The branch of computer science that isconcerned with the automation of intelligentbehavior.''(Luger+Stubblefield, 1993)

• Systems that think like humans• Systems that act like humans

• Systems that think rationally• Systems that act rationally

Source: Russell/Norvig, 1995

„The study of the computations that make it possible to perceive, reason, and act.“(Winston, 1992)

Artificial Intelligence

Standard problems:Chess vs. RoboCup

Chess RoboCupEnvironment Static DynamicState change Turn taking Real timeInformation accessibility Complete IncompleteSensor readings Symbolic Non-symbolicControl Central Distributed

Artificial Intelligence

Artificial Intelligence & RoboCup:Research topics

SearchPlanningLearningSpatial reasoning

Temporal reasoningDecision makingCommunicationPerception

Artificial Intelligence

Search & Planning

Used methodsA*-search for positioning,search over plansAdaptive path planning(SSL), Uni AucklandHybrid navigation planning(MSL, AGILO), chooses thebest planning algorithm

ShortcomingsSearch spaces too big!bd memory, bd time(example)States for planningalgorithms not easy todescribeLocal minima

Robotic Soccer and AI

Learning

Used methodsReinforcement learning forsoccer keep away (SL),AT&T, USAReinforcement learning foreverything (SL), UniKarlsruhe, GermanyLearning strategy ofopponent (SL), Uni Bremen,Germany, Uni Osaka, JapanK-nearest neighbor inpursuit-evasion situations(MSL), Uni Auckland, NZ

ShortcomingsLearning takes a long time(training phase)Online learning due to realtime processing almost notfeasible (only smallproblems)

Robotic Soccer and AI

Spatial and temporal reasoning

Used methodsObject tracking during thegame, Uni Bremen,GermanyReasoning with partonomies(Schlieder et al. 2001)

ShortcomingsModelling effort highReasoner not available

Robotic Soccer and AI

(Dynamic) Decision making

Used methodsState charts and utilityfunctions for rational agents,(SL), Uni Koblenz, GermanyRobot navigationconsidering observationalcost (Sony), Uni Tokyo,JapanHorn clauses logic fordecisions (SL), Uni Koblenz,Germany

ShortcomingsDecision making dependson value of informationDynamic Decision Networksretain forward searchthrough concrete states(typical for search alg.)Horn clauses expressivity

Robotic Soccer and AI

Communication

Used methodsRadio communication (SSL)Acoustics (Sony), UNSW,AustraliaCo-operation based behaviordesign (SSL), Nanyang Uni,SingaporeFormal languages (SL), Coachlanguage, various teamsCo-operation to complete worldmodel (MSL), CS Freiburg,Germany

ShortcomingsSony League: acusticcommunicationOverlapping frequencies„Meaning“ of packets

Robotic Soccer and AI

Perception

Used methodsCooperative sensing (globalview), CS-Freiburg, GermanyFast image processing (realtime) with the concept ofperspective view, SharifUniversity, IranImage processing tools, UniBremen, GermanyMulti-sensor navigation,SocRob, PortugalOmni directional vision forgoalkeeper, Padua,Italy

Shortcomings„Ghosts“ (also communication)

Image processing

Robotic Soccer and AI

Remarks

Where can AI methods help?PlanningSearchingLearningDecision makingSpatial reasoningTemporal reasoningComputational logicMotor control(Multi-agent systems)

Where can‘t they help?MechanicsElectronicsPerceiving (sensor fusion)Image processing

and:“Emotional reasoning”

Questions?

Simulation league

Small size league (2000 game)

Middle size league

Sony legged league

2001 finals

Example 1: Identification of situations

Behavior of goalkeeper in SLTime series based decision treeinductionCoach collects data for time seriesQualitative abstraction of keybehaviorsApproximation of time series bypartially linear functionsExtracting rules from decision-tree

Robotic Soccer and AI

ResultsFirst results shortly after 500 cyclesMethod recognizes values nearly asimplemented in our goalieQuality increases with longer timeseriesScenario: FC Portugal vs. VirtualWerderAnalyzing our own goalieDecision tree is generated every 500cycles

if DistGoalGoalie < 3.65 and DistBallGoal < 19.36 and DistBallGoalie < 13.73then 1 (0.972222)

(...)

if DistGoalGoalie > 3.65 and DistBallGoalie < 13.73then 1 (0.972222)

(...)

if DistGoalGoalie > 3.65 and DistBallGoalie > 13.73 and TeamMemInPenArea < 1 and DistBallGoal > 19.44then 0 (0.75)

class 1Goalie leaves goal probability

class 0 Goalie doesn't move

Example 1: Identification of situations (2)

Robotic Soccer and AI

back

Goal: Track objects and their spatial and temporal relationsover time and interpret them during the game

Spatial Relation Intervals Player/Ball location

distance

SE

E

NE

N

NW

W

SW

S

S

meets

med

far

close

210 220 240

player meetsor close to

the ball

playerapproaching

the ball

ball fromviewpoint ofplayer indirection toopposite goal

N

EW

S

time/cycles

Example 2: Identification of situations

Robotic Soccer and AI

Fig.1: Two players fighting for the ball

Object Motion Intervals Playerspeed

direction

SE

E

NE

N

NW

W

SW

S

S

210 220 240

Movement in SE/E direction,i.e. to the opposite goal

none

med

fast

slow

time/cycles

Example 2: Identification of situations (2)

Robotic Soccer and AI

Object Motion Intervals Ball

time/cycles

speed

direction

SE

E

NE

N

NW

W

SW

S

S

none

med

fast

slow

Movement in SE/Edirection, i.e. to theopposite goal

210 220 240

Example 2: Identification of situations (3)

Robotic Soccer and AI

se4=meets(p2,b)se2=approaching(p2,b)

se3=departing(b,p1)

Building complex situations from simple events:Player 1 passes the ball to player 2 ...

Temporal relations between time intervals:meets(se1,se2) AND equal(se2, se3) AND meets(se3, se4)

se1=meets(p1,b)

time

Simple event approaching: 1 spatial, 2 motion intervals:

Constraint:

[ ][ ][ ]111 ,1

,

,1

ppp

bbb

vdirpmotion

vdirbmotion

distlocpspatial

=

=

=

( ) 11 0:,1 pp dirlocvbpgapproachin =∧>

Fig.1: Two players fighting for the ball

Example 2: Identification of situations (4)

Robotic Soccer and AI

Fig.1: Two players fighting for the ball

(a) Cycle 216 (b) Cycle 221 (c) Cycle 225

(d) Cycle 237 (e) Cycle 257 (f) Cycle 261

(g) Cycle 265 (h) Cycle 270 (i) Cycle 273

ResultsScenario:

60 cycles in a 4 vs 4 environmentStarting at fig. 1 in the high lighted section

Player team 1 and team 2 are in relation. This is for the ball (fig. 1).Ball is from both players (fig. 2a).A second player from team 1 approaches the ball (fig. 2b).The player reaches the ball ( ) (fig. 2c).The player and the ball move in direction of opposite goal ( or ) (fig. 2d-f).Players of team 2 are moving backwards. Building a defense position (fig. 2).Defenders of team 2 approaching the player with the ball until one of them is meeting him(again for the ball) (fig. 2e-g).Player of team 2 is still in relation to the ball whereas the player of team 1 is

to the ball but no more it, i.e. has lost it (fig. 2h).Ball is from player of team 2, i.e. he passes the ball (fig. 2i).

Interpretation of spatio-temporal relations:meets fighting

departing

meetsmeets close

fightingmeets

close meetsdeparting

Example 2: Identification of situations (5)

Robotic Soccer and AI

back

Back to communicationBack to perception

Robotic Soccer and AI Robotic Soccer and AI

Hmax = 40, 80, 120cm

Robotic Soccer and AI

Penalty shootout Heuristics

Example 8-puzzleBranching factor = 3, typicalsolution after 20 stepsExhaustive search: 320 statesWith repeated states only 9! =362880

h1(n) = total number of misplaced tilesh2(n) = total Manhattan distance, i.e.,number of squares from desired locationof each tile

5 4

16

7 3 2

8

1 2

48

7 6 5

3

start state

h1(n)=7

h2(n) = 18

goal state

back

Robotic Soccer and AI

RMIT SocBot Play system recognition using artificial neuralnetworks

Online coach (total view)Bounding boxGrid of 64 cells (binary inputs)If players per cell � 1 then cellpositive, negative otherwiseLeads to input vector for ANN16 play systems known (e.g.Catenaccio)612 training sets, 68 test (�10%)

Play system recognition using artificial neuralnetworks (2)

Results:Virtual Werder vs. CMUnited 99Virtual Werder vs. Mainz Rolling Brains

Performance against CMU better withdefensive strategyPerformance against MRB better withoffensive strategy

Total: performance better if opponentsplay system is known!

Mainz RB CMU-99

Def

.5-4

-1

0.5:0.9 0.1:9

Off.

3-4-

3

3.1:0.7 0:14Virt

ualW

erde

r

back

AIBO sensors

Source: UNSW technical report

AIBO viewCCD Camera176 x 144 pixelOne of 8 colors per pixel256 x 256 x 256 YUV colorspace (� 16 MB!)Only 6 bits used (� 512KB!)Image Run-lenghtencoded (RLE)

High resolution digitized image Medium resolution digitized image

High resolution color image Medium resolution color imagePictures: UNSW technical report back

What are partonomies?

PartonomiesSP-PART-OF

Spatial reasoning

?

TaxonomiesIS-A

Terminological reasoning

?Description logic

DL theorem prover

Introduction

Intentional behaviorWhat activity is theuser/player currentlyengaged in?What is she/he intendingto do next?

Example: location-basedservices

Inferring the intentions ofthe user from the user´slocation

Queuing: Simple behavior

Motion patterns

Change of locationPath s(t)Speed v(t)Acceleration a(t)?

Location-based ServicesInferring intentions mustbe based on motionpatterns in this case Skiing: Complex

behavior

Spatial partonomies

Geographic SpaceHierarchical decompositioninto administrative orfunctional unitsPartonomies encode spatial-part-of relations (DAG, ...)

Research resultsAI: spatial reasoninge.g. spatio-temporal changeCognition: mental maps

exhibit

room

wing

museum

Qualitative abstraction

Qualitative parametersposition: {inside,outside}distance: {any}duration: [second]

Spatial reasoningRelational algebras(Tarski)Computational propertiesare defined bycomposition tables, ...

inside outside

inside

outside

inside outside

any any

ABC

Relational composition

Qualitative abstraction

Qualitative parametersposition: {inside,outside}distance: {any}duration: [second]

Spatial reasoningRelational algebras(Tarski)Computational propertiesare defined bycomposition tables, ...

inside outside

inside

outside

inside outside

any any

ABC

Relational composition

Strategy diagnosis

German Research CouncilProject

„Automatic diagnosis ofstrategies of opponentrobots in a co-operativeenvironment“

Dynamic partonomiesAnalyze motion in thecontext of partonomies!Partonomies defined bysoccer players

back

Reinforcement learning

4 vs. 3 KeepawayRL as framework forsequential decisionproblems

RL advantagesIncludes stochasticDelayed rewardsArchitecture for big statespaces and many actionsRandom delays betweenactions possible

Players skills:HoldBall()PassBall(f)GoToBall()GetOpen

Source: Stone & Sutton (2002)

Reinforcement learning

back Source: Stone & Sutton (2002)

Defendersalways GoToBall()

13 variables, e.g.Distances between objects

Dist(F1,F2), dist(F3,C)

MinimaMinimum(dist(F1,F2))

AnglesAng(F1,F2,D1)

Q-learning for forwardsResultsErgebnisse

Beispiele:RandomHand-codedLearned

Path planning

Source: Baltes & Hildreth (2001)

Adaptive planning algorithmfor car-like mobile robotsIdea:

Keep old plan as long aspossibleCreate a new plan byadapting the old plan to thenew situation

Representation

Path planning

back Source: Baltes & Hildreth (2001)

Repair strategiesPositional adjustment

Start & end position

Shape adjustmentLength and curvature of segment

Type adjustmentSign of curvature

Segment structure adjustmentInsertion, breaks, etc. of segments

Plan justification adjustmentRemoves unnecessary plansegments

EvaluationSignificantly faster