Post on 05-Jun-2020
Artificial Intelligence
Search
Marc ToussaintUniversity of StuttgartWinter 2016/17
(slides based on Stuart Russell’s AI course)
Motivation:Search algorithms are core tool for decision making, especially when the domain is toocomplex to use alternatives like Dynamic Programming. In recent years, with the increase incomputational power search methods became the method of choice for complex domains, likethe game of Go, or certain POMDPs.Learning about search tree algorithms is an important background for several reasons:
• The concept of decision trees, which represent the space of possible future decisionsand state transitions, is generally important for thinking about decision problems.
• In probabilistic domains, tree search algorithms are a special case of Monte-Carlomethods to estimate some expectation, typically the so-called Q-function. The respectiveMonte-Carlo Tree Search algorithms are the state-of-the-art in many domains.
• Tree search is also the background for backtracking in CSPs as well as forward andbackward search in logic domains.
We will cover the basic tree search methods (breadth, depth, iterative deepening) andeventually A∗
Search – – 1/48
Outline
• Problem formulation & examples
• Basic search algorithms
Search – – 2/48
Problem Formulation & Examples
Search – Problem Formulation & Examples – 3/48
Example: RomaniaOn holiday in Romania; currently in Arad.Flight leaves tomorrow from BucharestFormulate goal:
be in Bucharest, Sgoal = {Bucharest}Formulate problem:
states: various cities, S = {Arad, Timisoara, . . . }actions: drive between cities, A = {edges between states}
Find solution:sequence of cities, e.g., Arad, Sibiu, Fagaras, Bucharestminimize costs with cost function, (s, a) 7→ c
Search – Problem Formulation & Examples – 4/48
Example: Romania
Search – Problem Formulation & Examples – 5/48
Deterministic, fully observable search problemdef.
A deterministic, fully observable search problem is defined by fouritems:
initial state s0 ∈ S e.g., s0 = Aradsuccessor function succ : S×A→ S
e.g., succ(Arad,Arad-Zerind) = Zerindgoal states Sgoal ⊆ S
e.g., s = Buchareststep cost function cost(s, a, s′), assumed to be ≥ 0
e.g., traveled distance, number of actions executed, etc.the path cost is the sum of step costs
A solution is a sequence of actions leading from s0 to a goalAn optimal solution is a solution with minimal path costs
Search – Problem Formulation & Examples – 6/48
Example: vacuum world state space graph
states??:
integer dirt and robot locations (ignore dirt amounts etc.)actions??: Left, Right, Suck, NoOpgoal test??: no dirtpath cost??: 1 per action (0 for NoOp)
Search – Problem Formulation & Examples – 7/48
Example: vacuum world state space graph
states??: integer dirt and robot locations (ignore dirt amounts etc.)actions??:
Left, Right, Suck, NoOpgoal test??: no dirtpath cost??: 1 per action (0 for NoOp)
Search – Problem Formulation & Examples – 7/48
Example: vacuum world state space graph
states??: integer dirt and robot locations (ignore dirt amounts etc.)actions??: Left, Right, Suck, NoOpgoal test??:
no dirtpath cost??: 1 per action (0 for NoOp)
Search – Problem Formulation & Examples – 7/48
Example: vacuum world state space graph
states??: integer dirt and robot locations (ignore dirt amounts etc.)actions??: Left, Right, Suck, NoOpgoal test??: no dirtpath cost??:
1 per action (0 for NoOp)
Search – Problem Formulation & Examples – 7/48
Example: vacuum world state space graph
states??: integer dirt and robot locations (ignore dirt amounts etc.)actions??: Left, Right, Suck, NoOpgoal test??: no dirtpath cost??: 1 per action (0 for NoOp)
Search – Problem Formulation & Examples – 7/48
Example: The 8-puzzle
states??:
integer locations of tiles (ignore intermediate positions)actions??: move blank left, right, up, down (ignore unjamming etc.)goal test??: = goal state (given)path cost??: 1 per move[Note: optimal solution of n-Puzzle family is NP-hard]
Search – Problem Formulation & Examples – 8/48
Example: The 8-puzzle
states??: integer locations of tiles (ignore intermediate positions)actions??:
move blank left, right, up, down (ignore unjamming etc.)goal test??: = goal state (given)path cost??: 1 per move[Note: optimal solution of n-Puzzle family is NP-hard]
Search – Problem Formulation & Examples – 8/48
Example: The 8-puzzle
states??: integer locations of tiles (ignore intermediate positions)actions??: move blank left, right, up, down (ignore unjamming etc.)goal test??:
= goal state (given)path cost??: 1 per move[Note: optimal solution of n-Puzzle family is NP-hard]
Search – Problem Formulation & Examples – 8/48
Example: The 8-puzzle
states??: integer locations of tiles (ignore intermediate positions)actions??: move blank left, right, up, down (ignore unjamming etc.)goal test??: = goal state (given)path cost??:
1 per move[Note: optimal solution of n-Puzzle family is NP-hard]
Search – Problem Formulation & Examples – 8/48
Example: The 8-puzzle
states??: integer locations of tiles (ignore intermediate positions)actions??: move blank left, right, up, down (ignore unjamming etc.)goal test??: = goal state (given)path cost??: 1 per move[Note: optimal solution of n-Puzzle family is NP-hard]
Search – Problem Formulation & Examples – 8/48
Basic Tree Search Algorithms
Search – Basic Tree Search Algorithms – 9/48
Tree search example
Search – Basic Tree Search Algorithms – 10/48
Tree search example
Search – Basic Tree Search Algorithms – 11/48
Tree search example
Search – Basic Tree Search Algorithms – 12/48
Implementation: states vs. nodes
• A state is a (representation of) a physical configuration
• A node is a data structure constituting part of a search treeincludes parent, children, depth, path cost g(x)
(States do not have parents, children, depth, or path cost!)
• The Expand function creates new nodes, filling in the various fieldsand using the SuccessorFn of the problem to create thecorresponding states.
Search – Basic Tree Search Algorithms – 13/48
Implementation: general tree searchfunction Tree-Search( problem, fringe) returns a solution, or failure
fringe← Insert(Make-Node(Initial-State[problem]), fringe)loop do
if fringe is empty then return failurenode←Remove-Front(fringe)if Goal-Test(problem,State(node)) then return nodefringe← InsertAll(Expand(node, problem), fringe)
function Expand( node, problem) returns a set of nodessuccessors← the empty setfor each action, result in Successor-Fn(problem,State[node]) do
s← a new NodeParent-Node[s]← node; Action[s]← action; State[s]← resultPath-Cost[s]←Path-Cost[node] + Step-Cost(State[node], action, result)Depth[s]←Depth[node] + 1add s to successors
return successors
Search – Basic Tree Search Algorithms – 14/48
Search strategies
• A strategy is defined by picking the order of node expansion
• Strategies are evaluated along the following dimensions:completeness—does it always find a solution if one exists?time complexity—number of nodes generated/expandedspace complexity—maximum number of nodes in memoryoptimality—does it always find a least-cost solution?
• Time and space complexity are measured in terms ofb = maximum branching factor of the search treed = depth of the least-cost solutionm = maximum depth of the state space (may be∞)
Search – Basic Tree Search Algorithms – 15/48
Uninformed search strategies
• Uninformed strategies use only the information available in the problemdefinition
– Breadth-first search– Uniform-cost search– Depth-first search– Depth-limited search– Iterative deepening search
Search – Basic Tree Search Algorithms – 16/48
Breadth-first search
• Expand shallowest unexpanded node
• Implementation:fringe is a FIFO queue, i.e., new successors go at end
Search – Basic Tree Search Algorithms – 17/48
Breadth-first search
• Expand shallowest unexpanded node
• Implementation:fringe is a FIFO queue, i.e., new successors go at end
Search – Basic Tree Search Algorithms – 17/48
Breadth-first search
• Expand shallowest unexpanded node
• Implementation:fringe is a FIFO queue, i.e., new successors go at end
Search – Basic Tree Search Algorithms – 17/48
Breadth-first search
• Expand shallowest unexpanded node
• Implementation:fringe is a FIFO queue, i.e., new successors go at end
Search – Basic Tree Search Algorithms – 17/48
Properties of breadth-first searchComplete??
Yes (if b is finite)Time?? 1 + b+ b2 + b3 + . . .+ bd + b(bd − 1) = O(bd+1), i.e., exp. in dSpace?? O(bd+1) (keeps every node in memory)Optimal?? Yes, if cost-per-step=1; not optimal otherwise
Space is the big problem; can easily generate nodes at 100MB/secso 24hrs = 8640GB.
Search – Basic Tree Search Algorithms – 18/48
Properties of breadth-first searchComplete?? Yes (if b is finite)Time??
1 + b+ b2 + b3 + . . .+ bd + b(bd − 1) = O(bd+1), i.e., exp. in dSpace?? O(bd+1) (keeps every node in memory)Optimal?? Yes, if cost-per-step=1; not optimal otherwise
Space is the big problem; can easily generate nodes at 100MB/secso 24hrs = 8640GB.
Search – Basic Tree Search Algorithms – 18/48
Properties of breadth-first searchComplete?? Yes (if b is finite)Time?? 1 + b+ b2 + b3 + . . .+ bd + b(bd − 1) = O(bd+1), i.e., exp. in dSpace??
O(bd+1) (keeps every node in memory)Optimal?? Yes, if cost-per-step=1; not optimal otherwise
Space is the big problem; can easily generate nodes at 100MB/secso 24hrs = 8640GB.
Search – Basic Tree Search Algorithms – 18/48
Properties of breadth-first searchComplete?? Yes (if b is finite)Time?? 1 + b+ b2 + b3 + . . .+ bd + b(bd − 1) = O(bd+1), i.e., exp. in dSpace?? O(bd+1) (keeps every node in memory)Optimal??
Yes, if cost-per-step=1; not optimal otherwise
Space is the big problem; can easily generate nodes at 100MB/secso 24hrs = 8640GB.
Search – Basic Tree Search Algorithms – 18/48
Properties of breadth-first searchComplete?? Yes (if b is finite)Time?? 1 + b+ b2 + b3 + . . .+ bd + b(bd − 1) = O(bd+1), i.e., exp. in dSpace?? O(bd+1) (keeps every node in memory)Optimal?? Yes, if cost-per-step=1; not optimal otherwise
Space is the big problem; can easily generate nodes at 100MB/secso 24hrs = 8640GB.
Search – Basic Tree Search Algorithms – 18/48
Uniform-cost search
• “Cost-aware BFS”: Expand least-cost unexpanded node
• Implementation:fringe = queue ordered by path cost, lowest first
• Equivalent to breadth-first if step costs all equal
Complete?? Yes, if step cost ≥ εTime?? # of nodes with g ≤ cost-of-optimal-solution, O(bdC
∗/εe)
where C∗ is the cost of the optimal solutionSpace?? # of nodes with g ≤ cost-of-optimal-solution, O(bdC
∗/εe)
Optimal?? Yes: nodes expanded in increasing order of g(n)
Search – Basic Tree Search Algorithms – 19/48
Uniform-cost search
• “Cost-aware BFS”: Expand least-cost unexpanded node
• Implementation:fringe = queue ordered by path cost, lowest first
• Equivalent to breadth-first if step costs all equal
Complete?? Yes, if step cost ≥ εTime?? # of nodes with g ≤ cost-of-optimal-solution, O(bdC
∗/εe)
where C∗ is the cost of the optimal solutionSpace?? # of nodes with g ≤ cost-of-optimal-solution, O(bdC
∗/εe)
Optimal?? Yes: nodes expanded in increasing order of g(n)
Search – Basic Tree Search Algorithms – 19/48
Depth-first search
• Expand deepest unexpanded node
• Implementation:fringe = LIFO queue, i.e., put successors at front
Search – Basic Tree Search Algorithms – 20/48
Depth-first search
• Expand deepest unexpanded node
• Implementation:fringe = LIFO queue, i.e., put successors at front
Search – Basic Tree Search Algorithms – 20/48
Depth-first search
• Expand deepest unexpanded node
• Implementation:fringe = LIFO queue, i.e., put successors at front
Search – Basic Tree Search Algorithms – 20/48
Depth-first search
• Expand deepest unexpanded node
• Implementation:fringe = LIFO queue, i.e., put successors at front
Search – Basic Tree Search Algorithms – 20/48
Depth-first search
• Expand deepest unexpanded node
• Implementation:fringe = LIFO queue, i.e., put successors at front
Search – Basic Tree Search Algorithms – 20/48
Depth-first search
• Expand deepest unexpanded node
• Implementation:fringe = LIFO queue, i.e., put successors at front
Search – Basic Tree Search Algorithms – 20/48
Depth-first search
• Expand deepest unexpanded node
• Implementation:fringe = LIFO queue, i.e., put successors at front
Search – Basic Tree Search Algorithms – 20/48
Depth-first search
• Expand deepest unexpanded node
• Implementation:fringe = LIFO queue, i.e., put successors at front
Search – Basic Tree Search Algorithms – 20/48
Depth-first search
• Expand deepest unexpanded node
• Implementation:fringe = LIFO queue, i.e., put successors at front
Search – Basic Tree Search Algorithms – 20/48
Properties of depth-first searchComplete??
No: fails in infinite-depth spaces, spaces with loopsModify to avoid repeated states along path⇒ complete in finite
spacesTime?? O(bm): terrible if m is much larger than d
but if solutions are dense, may be much faster than breadth-firstSpace?? O(bm), i.e., linear space!Optimal?? No
Search – Basic Tree Search Algorithms – 21/48
Properties of depth-first searchComplete?? No: fails in infinite-depth spaces, spaces with loops
Modify to avoid repeated states along path⇒ complete in finitespacesTime??
O(bm): terrible if m is much larger than dbut if solutions are dense, may be much faster than breadth-first
Space?? O(bm), i.e., linear space!Optimal?? No
Search – Basic Tree Search Algorithms – 21/48
Properties of depth-first searchComplete?? No: fails in infinite-depth spaces, spaces with loops
Modify to avoid repeated states along path⇒ complete in finitespacesTime?? O(bm): terrible if m is much larger than d
but if solutions are dense, may be much faster than breadth-firstSpace??
O(bm), i.e., linear space!Optimal?? No
Search – Basic Tree Search Algorithms – 21/48
Properties of depth-first searchComplete?? No: fails in infinite-depth spaces, spaces with loops
Modify to avoid repeated states along path⇒ complete in finitespacesTime?? O(bm): terrible if m is much larger than d
but if solutions are dense, may be much faster than breadth-firstSpace?? O(bm), i.e., linear space!Optimal??
No
Search – Basic Tree Search Algorithms – 21/48
Properties of depth-first searchComplete?? No: fails in infinite-depth spaces, spaces with loops
Modify to avoid repeated states along path⇒ complete in finitespacesTime?? O(bm): terrible if m is much larger than d
but if solutions are dense, may be much faster than breadth-firstSpace?? O(bm), i.e., linear space!Optimal?? No
Search – Basic Tree Search Algorithms – 21/48
Depth-limited search
• depth-first search with depth limit l,i.e., nodes at depth l have no successors
• Recursive implementation using the stack as LIFO:
function Depth-Limited-Search( problem, limit) returns soln/fail/cutoffRecursive-DLS(Make-Node(Initial-State[problem]), problem, limit)
function Recursive-DLS(node, problem, limit) returns soln/fail/cutoffcutoff-occurred?← falseif Goal-Test(problem,State[node]) then return nodeelse if Depth[node] = limit then return cutoffelse for each successor in Expand(node, problem) do
result←Recursive-DLS(successor, problem, limit)if result = cutoff then cutoff-occurred?← trueelse if result 6= failure then return result
if cutoff-occurred? then return cutoff else return failure
Search – Basic Tree Search Algorithms – 22/48
Iterative deepening searchfunction Iterative-Deepening-Search( problem) returns a solution
inputs: problem, a problem
for depth← 0 to∞ doresult←Depth-Limited-Search( problem, depth)if result 6= cutoff then return result
end
Search – Basic Tree Search Algorithms – 23/48
Iterative deepening search
Search – Basic Tree Search Algorithms – 24/48
Iterative deepening search
Search – Basic Tree Search Algorithms – 24/48
Iterative deepening search
Search – Basic Tree Search Algorithms – 24/48
Iterative deepening search
Search – Basic Tree Search Algorithms – 24/48
Properties of iterative deepening searchComplete??
YesTime?? (d+ 1)b0 + db1 + (d− 1)b2 + . . .+ bd = O(bd)
Space?? O(bd)
Optimal?? Yes, if step cost = 1Can be modified to explore uniform-cost tree
Numerical comparison for b = 10 and d = 5, solution at far left leaf:
N(IDS) = 50 + 400 + 3 000 + 20 000 + 100 000 = 123 450
N(BFS) = 10 + 100 + 1 000 + 10 000 + 100 000 + 999 990 = 1 111 100
IDS does better because other nodes at depth d are not expandedBFS can be modified to apply goal test when a node is generated
Search – Basic Tree Search Algorithms – 25/48
Properties of iterative deepening searchComplete?? YesTime??
(d+ 1)b0 + db1 + (d− 1)b2 + . . .+ bd = O(bd)
Space?? O(bd)
Optimal?? Yes, if step cost = 1Can be modified to explore uniform-cost tree
Numerical comparison for b = 10 and d = 5, solution at far left leaf:
N(IDS) = 50 + 400 + 3 000 + 20 000 + 100 000 = 123 450
N(BFS) = 10 + 100 + 1 000 + 10 000 + 100 000 + 999 990 = 1 111 100
IDS does better because other nodes at depth d are not expandedBFS can be modified to apply goal test when a node is generated
Search – Basic Tree Search Algorithms – 25/48
Properties of iterative deepening searchComplete?? YesTime?? (d+ 1)b0 + db1 + (d− 1)b2 + . . .+ bd = O(bd)
Space??
O(bd)
Optimal?? Yes, if step cost = 1Can be modified to explore uniform-cost tree
Numerical comparison for b = 10 and d = 5, solution at far left leaf:
N(IDS) = 50 + 400 + 3 000 + 20 000 + 100 000 = 123 450
N(BFS) = 10 + 100 + 1 000 + 10 000 + 100 000 + 999 990 = 1 111 100
IDS does better because other nodes at depth d are not expandedBFS can be modified to apply goal test when a node is generated
Search – Basic Tree Search Algorithms – 25/48
Properties of iterative deepening searchComplete?? YesTime?? (d+ 1)b0 + db1 + (d− 1)b2 + . . .+ bd = O(bd)
Space?? O(bd)
Optimal??
Yes, if step cost = 1Can be modified to explore uniform-cost tree
Numerical comparison for b = 10 and d = 5, solution at far left leaf:
N(IDS) = 50 + 400 + 3 000 + 20 000 + 100 000 = 123 450
N(BFS) = 10 + 100 + 1 000 + 10 000 + 100 000 + 999 990 = 1 111 100
IDS does better because other nodes at depth d are not expandedBFS can be modified to apply goal test when a node is generated
Search – Basic Tree Search Algorithms – 25/48
Properties of iterative deepening searchComplete?? YesTime?? (d+ 1)b0 + db1 + (d− 1)b2 + . . .+ bd = O(bd)
Space?? O(bd)
Optimal?? Yes, if step cost = 1Can be modified to explore uniform-cost tree
Numerical comparison for b = 10 and d = 5, solution at far left leaf:
N(IDS) = 50 + 400 + 3 000 + 20 000 + 100 000 = 123 450
N(BFS) = 10 + 100 + 1 000 + 10 000 + 100 000 + 999 990 = 1 111 100
IDS does better because other nodes at depth d are not expandedBFS can be modified to apply goal test when a node is generated
Search – Basic Tree Search Algorithms – 25/48
Summary of algorithmsCriterion Breadth- Uniform- Depth- Depth- Iterative
First Cost First Limited Deepening
Complete? Yes∗ Yes∗ No Yes, if l ≥ d Yes
Time bd+1 bdC∗/εe bm bl bd
Space bd+1 bdC∗/εe bm bl bd
Optimal? Yes∗ Yes No No Yes∗
Search – Basic Tree Search Algorithms – 26/48
Loops: Repeated statesFailure to detect repeated states can turn a linear problem into anexponential one!
Search – Basic Tree Search Algorithms – 27/48
Graph searchfunction Graph-Search( problem, fringe) returns a solution, or failure
closed← an empty setfringe← Insert(Make-Node(Initial-State[problem]), fringe)loop do
if fringe is empty then return failurenode←Remove-Front(fringe)if Goal-Test(problem,State[node]) then return nodeif State[node] is not in closed then
add State[node] to closedfringe← InsertAll(Expand(node, problem), fringe)
end
But: storing all visited nodes leads again to exponential spacecomplexity (as for BFS)
Search – Basic Tree Search Algorithms – 28/48
SummaryIn BFS (or uniform-cost search), the fringe propagates layer-wise,containing nodes of similar distance-from-start (cost-so-far), leading tooptimal paths but exponential space complexity O(bd+1)
In DFS, the fringe is like a deep light beam sweeping over the tree, withspace complexity O(bm). Iteratively deepening it also leads to optimalpaths.Graph search can be exponentially more efficient than tree search, butstoring the visited nodes may lead to exponential space complexity asBFS.
Search – Basic Tree Search Algorithms – 29/48
Greedy and A∗Search
Search – Greedy and A∗Search – 30/48
Best-first searchIdea: use an arbitrary priority function f(n) for each node
– actually f(n) is neg-priority: nodes with lower f(n) have higherpriority
f(n) should reflect which nodes could be on an optimal path– could is optimistic – the lower f(n) the more optimistic you are
that n is on an optimal path⇒ Expand the unexpanded node with highes priorityImplementation:fringe is a queue sorted in decreasing order of prioritySpecial cases:
greedy searchA∗ search
Search – Greedy and A∗Search – 31/48
Uniform-Cost Search as special case
• Define g(n) = cost-so-far to reach n
• Then Uniform-Cost Search is Prioritized Search with f = g
Search – Greedy and A∗Search – 32/48
Romania with step costs in km
Search – Greedy and A∗Search – 33/48
Greedy searchWe set the priority function equal to a heuristic f(n) = h(n)
h(n) = estimate of cost from n to the closest goalE.g., hSLD(n) = straight-line distance from n to BucharestGreedy search expands the node that appears to be closest to goal
Search – Greedy and A∗Search – 34/48
Greedy search example
Search – Greedy and A∗Search – 35/48
Greedy search example
Search – Greedy and A∗Search – 35/48
Greedy search example
Search – Greedy and A∗Search – 35/48
Greedy search example
Search – Greedy and A∗Search – 35/48
Properties of greedy searchComplete??
No–can get stuck in loops, e.g.,Iasi→ Neamt→ Iasi→ Neamt→
Complete in finite space with repeated-state checkingTime?? O(bm), but a good heuristic can give dramatic improvementSpace?? O(bm)—keeps all nodes in memoryOptimal?? No
Greedy search does not care about the ’past’ (the cost-so-far).
Search – Greedy and A∗Search – 36/48
Properties of greedy searchComplete?? No–can get stuck in loops, e.g.,
Iasi→ Neamt→ Iasi→ Neamt→Complete in finite space with repeated-state checkingTime??
O(bm), but a good heuristic can give dramatic improvementSpace?? O(bm)—keeps all nodes in memoryOptimal?? No
Greedy search does not care about the ’past’ (the cost-so-far).
Search – Greedy and A∗Search – 36/48
Properties of greedy searchComplete?? No–can get stuck in loops, e.g.,
Iasi→ Neamt→ Iasi→ Neamt→Complete in finite space with repeated-state checkingTime?? O(bm), but a good heuristic can give dramatic improvementSpace??
O(bm)—keeps all nodes in memoryOptimal?? No
Greedy search does not care about the ’past’ (the cost-so-far).
Search – Greedy and A∗Search – 36/48
Properties of greedy searchComplete?? No–can get stuck in loops, e.g.,
Iasi→ Neamt→ Iasi→ Neamt→Complete in finite space with repeated-state checkingTime?? O(bm), but a good heuristic can give dramatic improvementSpace?? O(bm)—keeps all nodes in memoryOptimal??
No
Greedy search does not care about the ’past’ (the cost-so-far).
Search – Greedy and A∗Search – 36/48
Properties of greedy searchComplete?? No–can get stuck in loops, e.g.,
Iasi→ Neamt→ Iasi→ Neamt→Complete in finite space with repeated-state checkingTime?? O(bm), but a good heuristic can give dramatic improvementSpace?? O(bm)—keeps all nodes in memoryOptimal?? No
Greedy search does not care about the ’past’ (the cost-so-far).
Search – Greedy and A∗Search – 36/48
A∗ searchIdea: combine information from the past and the future
neg-priority = cost-so-far + estimated cost-to-goEvaluation function f(n) = g(n) + h(n)
g(n) = cost-so-far to reach nh(n) = estimated cost-to-go from n
f(n) = estimated total cost of path through n to goalA∗ search uses an admissible (=optimistic) heuristic
i.e., h(n) ≤ h∗(n) where h∗(n) is the true cost-to-go from n.(Also require h(n) ≥ 0, so h(G) = 0 for any goal G.)
E.g., hSLD(n) never overestimates the actual road distanceTheorem: A∗ search is optimal (=finds the optimal path)
Search – Greedy and A∗Search – 37/48
A∗ search example
Search – Greedy and A∗Search – 38/48
A∗ search example
Search – Greedy and A∗Search – 38/48
A∗ search example
Search – Greedy and A∗Search – 38/48
A∗ search example
Search – Greedy and A∗Search – 38/48
A∗ search example
Search – Greedy and A∗Search – 38/48
A∗ search example
Search – Greedy and A∗Search – 38/48
Proof of optimality of A∗
Suppose some suboptimal goal G2 has been generated and is in the fringe (buthas not yet been selected to be tested for goal condition!). We want to proof:Any node on a shortest path to an optimal goal G will be expanded before G2.
Let n be an unexpanded node on a shortest path to G.
f(G2) = g(G2) since h(G2) = 0
> g(G) since G2 is suboptimal
≥ f(n) since h is admissible
Since f(n) < f(G2), A∗ will expand n before G2. This is true for any n on theshortest path. In particular, at some time G is added to the fringe, and sincef(G) = g(G) < f(G2) = g(G2) it will select G before G2 for goal testing.
Search – Greedy and A∗Search – 39/48
Properties of A∗
Complete??
Yes, unless there are infinitely many nodes with f ≤ f(G)Time?? Exponential in [relative error in h × length of soln.]Space?? Exponential. Keeps all nodes in memoryOptimal?? YesA∗ expands all nodes with f(n) < C∗
A∗ expands some nodes with f(n) = C∗
A∗ expands no nodes with f(n) > C∗
Search – Greedy and A∗Search – 40/48
Properties of A∗
Complete?? Yes, unless there are infinitely many nodes with f ≤ f(G)Time??
Exponential in [relative error in h × length of soln.]Space?? Exponential. Keeps all nodes in memoryOptimal?? YesA∗ expands all nodes with f(n) < C∗
A∗ expands some nodes with f(n) = C∗
A∗ expands no nodes with f(n) > C∗
Search – Greedy and A∗Search – 40/48
Properties of A∗
Complete?? Yes, unless there are infinitely many nodes with f ≤ f(G)Time?? Exponential in [relative error in h × length of soln.]Space??
Exponential. Keeps all nodes in memoryOptimal?? YesA∗ expands all nodes with f(n) < C∗
A∗ expands some nodes with f(n) = C∗
A∗ expands no nodes with f(n) > C∗
Search – Greedy and A∗Search – 40/48
Properties of A∗
Complete?? Yes, unless there are infinitely many nodes with f ≤ f(G)Time?? Exponential in [relative error in h × length of soln.]Space?? Exponential. Keeps all nodes in memoryOptimal??
YesA∗ expands all nodes with f(n) < C∗
A∗ expands some nodes with f(n) = C∗
A∗ expands no nodes with f(n) > C∗
Search – Greedy and A∗Search – 40/48
Properties of A∗
Complete?? Yes, unless there are infinitely many nodes with f ≤ f(G)Time?? Exponential in [relative error in h × length of soln.]Space?? Exponential. Keeps all nodes in memoryOptimal?? YesA∗ expands all nodes with f(n) < C∗
A∗ expands some nodes with f(n) = C∗
A∗ expands no nodes with f(n) > C∗
Search – Greedy and A∗Search – 40/48
Optimality of A∗ (more useful)Lemma: A∗ expands nodes in order of increasing f value∗
Gradually adds “f -contours” of nodes (cf. breadth-first adds layers)Contour i has all nodes with f = fi, where fi < fi+1
Search – Greedy and A∗Search – 41/48
Proof of lemma: Consistency
A heuristic is consistent if
h(n) ≤ c(n, a, n′) + h(n′)
If h is consistent, we have
f(n′) = g(n′) + h(n′)
= g(n) + c(n, a, n′) + h(n′)
≥ g(n) + h(n)
= f(n)
I.e., f(n) is nondecreasing along any path.
Search – Greedy and A∗Search – 42/48
Admissible heuristicsE.g., for the 8-puzzle:h1(n) = number of misplaced tilesh2(n) = total Manhattan distance
(i.e., no. of squares from desired location of each tile)
h1(S) =??h2(S) =??
Search – Greedy and A∗Search – 43/48
Admissible heuristicsE.g., for the 8-puzzle:h1(n) = number of misplaced tilesh2(n) = total Manhattan distance
(i.e., no. of squares from desired location of each tile)
h1(S) =?? 6h2(S) =?? 4+0+3+3+1+0+2+1 = 14
Search – Greedy and A∗Search – 43/48
DominanceIf h2(n) ≥ h1(n) for all n (both admissible)then h2 dominates h1 and is better for searchTypical search costs:d = 14 IDS = 3,473,941 nodes
A∗(h1) = 539 nodes
A∗(h2) = 113 nodes
d = 24 IDS ≈ 54,000,000,000 nodes
A∗(h1) = 39,135 nodes
A∗(h2) = 1,641 nodesGiven any admissible heuristics ha, hb,
h(n) = max(ha(n), hb(n))
is also admissible and dominates ha, hb
Search – Greedy and A∗Search – 44/48
Relaxed problemsAdmissible heuristics can be derived from the exactsolution cost of a relaxed version of the problemIf the rules of the 8-puzzle are relaxed so that a tile can moveanywhere, then h1(n) gives the shortest solutionIf the rules are relaxed so that a tile can move to any adjacent square,then h2(n) gives the shortest solutionKey point: the optimal solution cost of a relaxed problemis no greater than the optimal solution cost of the real problem
Search – Greedy and A∗Search – 45/48
Memory-bounded A∗
As with BFS, A∗ has exponential space complexityIterative-deepening A∗, works for integer path costs, but problematic forreal-valued(Simplified) Memory-bounded A∗ (SMA∗):
– Expand as usual until a memory bound is reach– Then, whenever adding a node, remove the worst node n′ from
the tree– worst means: the n′ with highest f(n′)– To not loose information, backup the measured step-cost
cost(n, a, n′)
to improve the heuristic h(n) of its parentSMA∗ is complete and optimal if the depth of the optimal path is withinthe memory bound
Search – Greedy and A∗Search – 46/48
SummaryCombine information from the past and the futureA heuristic function h(n) represents information about the future
– it estimates cost-to-go optimisticallyGood heuristics can dramatically reduce search costGreedy best-first search expands lowest h
– incomplete and not always optimalA∗ search expands lowest f = g + h
– neg-priority = cost-so-far + estimated cost-to-go– complete and optimal– also optimally efficient (up to tie-breaks, for forward search)
Admissible heuristics can be derived from exact solution of relaxedproblemsMemory-bounded startegies exist
Search – Greedy and A∗Search – 47/48
OutlookTree search with partial observations
– rather discuss this in a fully probabilistic setting laterTree search for games
– minimax extension to tree search– discuss state-of-the-art probabilistic Monte-Carlo tree search
methods later
Search – Greedy and A∗Search – 48/48