Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.
-
date post
20-Dec-2015 -
Category
Documents
-
view
215 -
download
1
Transcript of Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.
![Page 1: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/1.jpg)
Research Related to Real-Time Strategy Games
Robert Holte
November 8, 2002
![Page 2: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/2.jpg)
No Games Knowledge, but…
• Single-agent search = sequential decisions with additive costs– path-finding occurs in RTS games– are there other occurrences ?– transfer from 1-agent to 2-agent or N-agent games ?– is a 2-agent game with a deterministic opponent a
single-agent search problem ?
• Path-finding research group (P. Yap, Yngvi, …)
![Page 3: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/3.jpg)
and …
• Combinatorial optimization = resource allocation with complex constraints– combinatorial auctions– “boolean programming”– hill-climbing, ESG, a bit of CPLEX
![Page 4: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/4.jpg)
and …
• Machine Learning– opponent-modelling (poker)– user-modelling (adaptive games, personalization)– learning to make good moves (M. Mueller)
• Interactive Systems– mostly-automatic evaluation methodology
• Abstraction in Poker
![Page 5: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/5.jpg)
Using Abstraction to Speed Up Search
![Page 6: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/6.jpg)
Search = path-finding in a graph
GIVEN:– A graph– Two nodes in the graph (“start”, “goal”)
FIND:– a shortest path from start to goal
![Page 7: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/7.jpg)
Labyrinth at the gardens of Versailles(could be a computer game!)
![Page 8: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/8.jpg)
“Implicit” Graphs
Instead of an adjacency list/matrix,Define a graph by a successor function, succ,
and one or more “seed” nodes.Nodes of the graph = transitive closure of succ applied to the seed nodes
An edge exists from n1 to n2 iff n2 succ(n1)
A graph can be exponentially larger than its description.
![Page 9: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/9.jpg)
Towers of Hanoi puzzlefor D disks:
– graph has 3D nodes– description is O(D2)
![Page 10: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/10.jpg)
3-disk State Space
![Page 11: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/11.jpg)
Speed up Search by Using Extra Information
• Solution “skeleton” = chain of subgraphs leading from start to goal– Refinement: search in the first subgraph until you reach
the second, then search in the second until you reach the third, etc.
– very fast but path found not guaranteed to be optimal
• Heuristic = h(n) = estimate of the distance from node n to the goal node.– if it never overestimates, it can eliminate (prune) parts of
the graph and still guarantee finding an optimal path
![Page 12: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/12.jpg)
My Research
Aim:to generate heuristics or skeletal solutions automatically
Method:1. Create an abstraction of the given graph (simplified
version of your problem that “approximately” preserves “important” features of the problem).
2. Find a solution path in the abstract graph.3. Use the abstract solution as a skeletal solution, or use
its length as a heuristic.
![Page 13: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/13.jpg)
Example: Towers of Hanoistart
goal
![Page 14: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/14.jpg)
Abstraction:“Ignore the smallest
disk”start
goal
![Page 15: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/15.jpg)
Abstract Solution
start
goal
![Page 16: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/16.jpg)
Key Techniques
For Automatically Generating Abstractions– explicit graphs: STAR abstraction
– implicit graphs: domain abstraction
– select among abstractions using Korf & Reid’s method for predicting search time
Search Methods– refinement: AltO
– heuristic: Hierarchical A*, pattern databases
– anytime combination of refinement and heuristic search
![Page 17: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/17.jpg)
STAR abstraction
Abstract state = connected subgraph
(all nodes within distance R of a given node)
• Fully automatic (but need to amortize cost)
• Fine control over granularity (R)
• Can be applied recursively (abstraction hierarchy)
• Guarantees refinement will succeed• Produces a consistent heuristic
![Page 18: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/18.jpg)
AltO Refinement Technique
Conduct abstract search in the opposite direction to the base-level search;
Use the whole abstract search graph for refinement.
• 12% faster than standard refinement and produces better solutions (20% shorter)
• Robust – not sensitive to the abstract solution found, and not very sensitive to the radius of abstraction
• 10-40 times faster than A* (same abstraction) and solutions usually within 30% of optimal
![Page 19: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/19.jpg)
Pattern Database Successes• Joe Culberson & Jonathan Schaeffer (1994).
– 15-puzzle– PDB + Manhattan Distance (MD) reduced MD-
alone search tree 1000-fold
• Rich Korf (1997)– PDBs enabled optimal solutions of Rubik’s Cube
• Stefan Edelkamp (2001)– PDBs competitive with best planners
• Istvan Hernadvolgyi– PDBs reduced macro-table solution lengths from 90
to 50
![Page 20: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/20.jpg)
Example: 8-puzzle
1 2
3 4 5
6 7 8
Domain = blank 1 2 3 4 5 6 7 8
181,440 states
![Page 21: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/21.jpg)
“Patterns”created by domain mapping
1 2
3 4 5
6 7 8
This mapping produces9 patterns
Domain = blank 1 2 3 4 5 6 7 8Abstract = blank
corresponding patternoriginal state
![Page 22: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/22.jpg)
Pattern Database
Pattern
Distance to goal 0 1 1 2 2 2
Pattern
Distance to goal 3 3 4
![Page 23: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/23.jpg)
Calculating h(s)
Given a state in the original problem
Compute the corresponding pattern
and look up the abstract distance-to-goal
8 1 4
3 5
6 7 2
2
Heuristics defined by PDBs are consistent, not just admissible.
![Page 24: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/24.jpg)
Domain Abstraction
1 2
3 4 5
6 7 8 6 7 8
30,240 patternsDomain = blank 1 2 3 4 5 6 7 8Abstract = blank 6 7 8
![Page 25: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/25.jpg)
Automatic Creation of Domain Abstractions
• Easy to enumerate all possible domain abstractions
• They form a lattice, e.g.
is “more abstract” than the domain abstraction above
Domain = blank 1 2 3 4 5 6 7 8Abstract = blank
Domain = blank 1 2 3 4 5 6 7 8Abstract = blank
![Page 26: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/26.jpg)
8-puzzle: A* vs. PDB size#
node
s ex
pand
ed (
A*)
pattern database size (# of abstract states)
![Page 27: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/27.jpg)
Hierarchical A*Unlike pattern databases, each h(n) is computed on
demand.
If h(n) is not already known compute it by searching at the abstract level.
The search at an abstract level can be informed by a heuristic defined by a higher abstract level.
* many abstract searches for one base-level search* naïve implementation is 10 times slower than
blind base-level search.
![Page 28: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/28.jpg)
Speed versus Optimality
• Refinement– very fast (when refinability guaranteed)– suboptimal paths
• A*– much slower– optimal paths
• See also: bidirectional search, -admissible A*
![Page 29: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/29.jpg)
Hybrid of HA* and Refinement (1)
Although the algorithms seem very different there are actually only two differences.
(1) The relative weight given to g and h.HA*: g + h
Refinement: h (use g to break ties)
Generalization: *g + (1- )*h ( 0 1 )
![Page 30: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/30.jpg)
(2) What to do if a node n is encountered for which h(n) is not known from the first abstract search ?
HA*: compute h(n) by doing another abstract search
Refinement: ignore n
Generalization: choose randomly between the two strategies with probability P ( 0 P 1 )
Hybrid of HA* and Refinement (2)
![Page 31: Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.](https://reader033.fdocuments.us/reader033/viewer/2022051619/56649d4c5503460f94a29ecc/html5/thumbnails/31.jpg)
2-d Space of Algorithms
0 0.5 1.0
= weight given to g
1.0
P
0.5
0.0
A*
Refinement
GraphTraverser
OptimalRefinement