SEARCH Heng Ji Feb 02/05, 2016. Search We will consider the problem of designing goal- based agents...

SEARCH

Heng Ji

[email protected] 02/05, 2016

Search• We will consider the problem of designing goal-based agents in fully observable, deterministic, discrete, known environments

• Example:Start state

Goal state

Search• We will consider the problem of designing goal-based agents in fully observable, deterministic, discrete, known environments • The solution is a fixed sequence of actions• Search is the process of looking for the sequence of

actions that reaches the goal• Once the agent begins executing the search solution, it

can ignore its percepts (open-loop system)

Search problem components• Initial state• Actions• Transition model

• What is the result of performing a given action in a given state?

• Goal state• Path cost

• Assume that it is a sum of nonnegative step costs

• The optimal solution is the sequence of actions that gives the lowest path cost for reaching the goal

Initialstate

Goal state

Example: Romania

• Initial state• Arad

• Actions• Go from one city to another

• Transition model• If you go from city A to

city B, you end up in city B• Goal state

• Bucharest• Path cost

• Sum of edge costs

• On vacation in Romania; currently in Arad• Flight leaves tomorrow from Bucharest

State space• The initial state, actions, and transition model define the state

space of the problem• The set of all states reachable from initial state by any sequence of

actions• Can be represented as a directed graph where the nodes are states

and links between nodes are actions• What is the state space for the Romania problem?

Search• Given:

• Initial state• Actions• Transition model• Goal state• Path cost

• How do we find the optimal solution?• How about building the state space and then using Dijkstra’s

shortest path algorithm?• The state space may be huge!• Complexity of Dijkstra’s is O(E + V log V), where V is the size of the state

space

Tree Search• Let’s begin at the start node and expand it by making a list of all

possible successor states• Maintain a fringe or a list of unexpanded states• At each step, pick a state from the fringe to expand • Keep going until you reach the goal state• Try to expand as few states as possible

Search tree• “What if” tree of possible actions and

outcomes• The root node corresponds to the

starting state• The children of a node correspond to the

successor states of that node’s state• A path through the tree corresponds to a

sequence of actions• A solution is a path ending in the goal state

• Nodes vs. states• A state is a representation of a physical

configuration, while a node is a data structure that is part of the search tree

…………

Starting state

Successor state

Action

Goal state

Tree Search Algorithm Outline• Initialize the fringe using the starting state• While the fringe is not empty

• Choose a fringe node to expand according to search strategy

• If the node contains the goal state, return solution• Else expand the node and add its children to the fringe

Tree search example

Tree search exampleFringe

Search strategies• A search strategy is defined by picking the order of node

expansion• Strategies are evaluated along the following dimensions:

• Completeness: does it always find a solution if one exists?• Optimality: does it always find a least-cost solution?• Time complexity: number of nodes generated• Space complexity: maximum number of nodes in memory

• Time and space complexity are measured in terms of • b: maximum branching factor of the search tree• d: depth of the least-cost solution• m: maximum length of any path in the state space (may be infinite)

Uninformed search strategies• Uninformed search strategies use only the information

available in the problem definition

• Breadth-first search• Uniform-cost search• Depth-first search• Iterative deepening search

Review: Search problem formulation• Initial state• Actions• Transition model• Goal state• Path cost

• What is the optimal solution?• What is the state space?

Review: Tree search• Initialize the fringe using the starting state• While the fringe is not empty

• Choose a fringe node to expand according to search strategy

• If the node contains the goal state, return solution• Else expand the node and add its children to the fringe

• To handle repeated states:• Keep an explored set; add each node to the explored set

every time you expand it• Every time you add a node to the fringe, check whether it

already exists in the fringe with a higher path cost, and if yes, replace that node with the new one

Search strategies• A search strategy is defined by picking the order of node

expansion• Strategies are evaluated along the following dimensions:

• Completeness: does it always find a solution if one exists?• Optimality: does it always find a least-cost solution?• Time complexity: number of nodes generated• Space complexity: maximum number of nodes in memory

• Time and space complexity are measured in terms of • b: maximum branching factor of the search tree• d: depth of the optimal solution• m: maximum length of any path in the state space (may be infinite)

Uninformed search strategies• Uninformed search strategies use only the information

available in the problem definition

• Breadth-first search• Uniform-cost search• Depth-first search• Iterative deepening search

Search Algorithm Properties

Breadth-First Search

Breadth-first search• Expand shallowest unexpanded node• Implementation:

• fringe is a FIFO queue, i.e., new successors go at end

A

D F

B C

E G

Properties of breadth-first search• Complete?

Yes (if branching factor b is finite)• Optimal?

Yes – if cost = 1 per step• Time?

Number of nodes in a b-ary tree of depth d: O(bd)(d is the depth of the optimal solution)

• Space? O(bd)

• Space is the bigger problem (more than time)

Uniform-cost search• Expand least-cost unexpanded node• Implementation: fringe is a queue ordered by path cost (priority

queue)• Equivalent to breadth-first if step costs all equal

• Complete? Yes, if step cost is greater than some positive constant ε (we don’t want

infinite sequences of steps that have a finite total cost)• Optimal?

Yes – nodes expanded in increasing order of path cost• Time?

Number of nodes with path cost ≤ cost of optimal solution (C*), O(bC*/ ε)This can be greater than O(bd): the search can explore long paths consisting

of small steps before exploring shorter paths consisting of larger steps • Space?

O(bC*/ ε)

Depth-First Search

Depth-first search• Expand deepest unexpanded node• Implementation:

• fringe = LIFO queue, i.e., put successors at front

A

D F

B C

E G

Properties of depth-first search• Complete?

Fails in infinite-depth spaces, spaces with loopsModify to avoid repeated states along path

complete in finite spaces

• Optimal?No – returns the first solution it finds

• Time? Could be the time to reach a solution at maximum depth m: O(bm)Terrible if m is much larger than dBut if there are lots of solutions, may be much faster than BFS

• Space? O(bm), i.e., linear space!

Iterative deepening search• Use DFS as a subroutine

1. Check the root2. Do a DFS searching for a path of length 13. If there is no path of length 1, do a DFS searching for a path of

length 24. If there is no path of length 2, do a DFS searching for a path of

length 3…

Iterative deepening search

Properties of iterative deepening search

• Complete?Yes

• Optimal?Yes, if step cost = 1

• Time?(d+1)b0 + d b1 + (d-1)b2 + … + bd = O(bd)

• Space?O(bd)

Informed search• So far, have assumed that no nongoal state looks better than another

• Unrealistic• Even without knowing the road structure, some locations

seem closer to the goal than others• Some states of the 8s puzzle seem closer to the goal than

others• Makes sense to expand closer-seeming nodes first

Informed search• Idea: give the algorithm “hints” about the desirability of

different states • Use an evaluation function to rank nodes and select the most

promising one for expansion

• Greedy best-first search• A* search

Heuristic function• Heuristic function h(n) estimates the cost of reaching goal

from node n• Example:

Start state

Goal state

Heuristics• Key notion: heuristic function h(n) gives an estimate of the distance from n to the goal• h(n)=0 for goal nodes

• E.g. straight-line distance for traveling problem

A

BC

FD E

3 4 4

39

2

2

start stategoal state

• Say: h(A) = 9, h(B) = 8, h(C) = 9, h(D) = 6, h(E) = 3, h(F) = 0

• We’re adding something new to the problem!

• Can use heuristic to decide which nodes to expand first

Heuristic for the Romania problem

Greedy best-first search• Expand the node that has the lowest value of the heuristic

function h(n)

Greedy best-first search example

Properties of greedy best-first search• Complete?

No – can get stuck in loops

startgoal


No – can get stuck in loops• Optimal?

No


No – can get stuck in loops• Optimal?

No• Time?

Worst case: O(bm)Best case: O(bd) – If h(n) is 100% accurate

• Space?Worst case: O(bm)

A bad example for greedy

A

B

FD E

3 4 4

7 6

start stategoal state

• Say: h(A) = 9, h(B) = 5, h(D) = 6, h(E) = 3, h(F) = 0

• Problem: greedy evaluates the promise of a node only by how far is left to go, does not take cost occurred already into account

How can we fix the greedy problem?

A* search• Idea: avoid expanding paths that are already expensive

• The evaluation function f(n) is the estimated total cost of the path through node n to the goal:

f(n) = g(n) + h(n)

g(n): cost so far to reach n (path cost)h(n): estimated cost from n to goal (heuristic)

Romania with step costs in km

A* search exampleOpen List:

Arad

We start with our initial state Arad. We make a node and add it to the open list. Since it’s the only thing on the open list, we expand the node.

Think of the open list as a priority queue (or heap) that sorts the nodes inside of it according to their g()+h() score.


Sibiu

Timisoara

Zerind

We add the three nodes we found to the open list.

We sort them according to the g()+h() calculation.


Rimricu Vicea

Fagaras

Timisoara

Zerind

Arad

Oradea

We’ve been to Arad before. Don’t list it again on the open list.

When we expand Sibiu, we run into Arad again. But we’ve already expanded this node once; so, we don’t add it to the open list again.


Rimricu Vicea

Fagaras

Timisoara

Zerind

Oradea

We see that Rimricu Vicea is at the top of the open list; so, it’s the next node we will expand.


Fagaras

Pitesti

Timisoara

Zerind

Craiova

Sibiu

Oradea

When we expand Rimricu Vicea, we run into Sibiu again.

But we’ve already expanded this node once; so, we don’t add it to the open list again.


Fagaras

Pitesti

Timisoara

Zerind

Craiova

Oradea

Fagaras will be the next node we should expand – it’s at the top of the sorted open list.


Pitesti

Timisoara

Zerind

Bucharest

Craiova

Sibiu

Oradea

When we expand Fagaras, we find Sibiu again. We don’t add it to the open list.

We also find Bucharest, but we’re not done. The algorithm doesn’t end until we “expand” the goal node – it has to be at the top of the open list.


Pitesti

Timisoara

Zerind

Bucharest

Craiova

Oradea

It looks like Pitesti is the next node we should expand.


Bucharest

Timisoara

Zerind

Craiova

Rimricu Vicea

Oradea

We just found a better value for Bucharest; so, it got moved higher in the list. We also found a worse value for Craiova – we just ignore this.

And of course, we ran into Rimricu Vicea again. Since it’s already been expanded once, we don’t re-add it to the Open List.


Bucharest

Timisoara

Zerind

Craiova

Oradea

Now it looks like Bucharest is at the top of the open list…


Bucharest

Timisoara

Zerind

Craiova

Oradea

Now we “expand” the node for Bucharest.

We’re done! (And we know the path that we’ve found is optimal.)

Admissible heuristics• A heuristic h(n) is admissible if for every node n, h(n) ≤ h*(n), where h*(n) is the true cost to reach the goal state from n

• An admissible heuristic never overestimates the cost to reach the goal, i.e., it is optimistic

• Example: straight line distance never overestimates the actual road distance

• Theorem: If h(n) is admissible, A* is optimal

Optimality of A*

• Suppose A* terminates its search at n* • It has found a path whose actual cost f(n*) = g(n*) is lower than the estimated cost f(n) of any path going through any fringe node

• Since f(n) is an optimistic estimate, there is no way n can have a successor goal state n’ with g(n’) < C*

n

n*f(n*) = C*

f(n) > C*

n' g(n') f(n) > C*

Optimality of A*• A* is optimally efficient – no other tree-based algorithm

that uses the same heuristic can expand fewer nodes and still be guaranteed to find the optimal solution• Any algorithm that does not expand all nodes with f(n) < C* risks

missing the optimal solution

Properties of A*• Complete?

Yes – unless there are infinitely many nodes with f(n) ≤ C*• Optimal?

Yes• Time?

Number of nodes for which f(n) ≤ C* (exponential)• Space?

Exponential

Uninformed search strategiesAlgorithm Complete? Optimal? Time

complexitySpace

complexity

BFS

UCS

DFS

IDS

b: maximum branching factor of the search treed: depth of the optimal solutionm: maximum length of any path in the state spaceC*: cost of optimal solution

Yes

Yes

No

Yes

If all step costs are equal


Yes

No

O(bd)

O(bm)

O(bd)

O(bd)

O(bm)

O(bd)

Number of nodes with g(n) ≤ C*

All search strategies

Algorithm Complete? Optimal? Time complexity

Space complexity

BFS

UCS

DFS

IDS

Greedy

A*

Yes

Yes

No

Yes



Yes

No

O(bd)

Number of nodes with g(n) ≤ C*

O(bm)

O(bd)

O(bd)

O(bm)

O(bd)

No NoWorst case: O(bm)

Yes Yes

Best case: O(bd)

Number of nodes with g(n)+h(n) ≤ C*

Exercise• Trace from Lugoj to Bucharest

Search where the path doesn’t matter• So far, looked at problems where the path was the solution• Traveling on a graph• Eights puzzle

• However, in many problems, we just want to find a goal state• Doesn’t matter how we get there

Queens puzzle

Q

Q

Q

Q

Q

Q

Q

Q

• Place eight queens on a chessboard so that no two attack each other

Backtracking• Backtracking is a technique used to solve problems with a large search space, by systematically trying and eliminating possibilities.

• A standard example of backtracking would be going through a maze. • At some point in a maze, you might have two options

of which direction to go:

Junction

Portion A

Port

ion

B

Backtracking

Junction

Portion B

Port

ion

A

One strategy would be to try going through Portion A of the maze. If you get stuck before

you find your way out, then you "backtrack" to the junction.

At this point in time you

know that Portion A will NOT lead you out of the maze, so you then start

searching in Portion B

Backtracking• Clearly, at a single junction you could have even more than 2 choices.

• The backtracking strategy says to try each choice, one after the other, • if you ever get stuck,

"backtrack" to the junction and try the next choice.

• If you try all choices and never found a way out, then there IS no solution to the maze.

Junction

BC

A

N-queens problem• Find a configuration of n queens not attacking each other

• What is the maximum number of queens that can be placed on an n x n chessboard such that no two attack one another?

• The answer is n queens, which gives eight queens for the usual 8x8 board

Backtracking – Eight Queens Problem• Find an arrangement of 8 queens on a single chess board such that no two queens are attacking one another.

• In chess, queens can move all the way down any row, column or diagonal (so long as no pieces are in the way).

• Due to the first two restrictions, it's clear that each row and column of the board will have exactly one queen.

12 Unique Solutions8-Queens Problem

8-Queens Problem• There are 92 distinct solutions• There are 12 unique solutions discounting symmetrical answers (rotatations/reflections)

• How many solutions for 4-Queens? N-Queens?

Wikipedia.org

Problems N < 4

3

2

1

N < 4

Cannot use N Queens

Backtracking in Decision Trees•Idea: Start at the root of the decision tree and move downwards, that is, make a sequence of decisions, until you either reach a solution or you enter a situation from where no solution can be reached by any further sequence of decisions.•In the latter case, backtrack to the parent of the current node and take a different path downwards from there. If all paths from this node have already been explored, backtrack to its parent.•Continue this procedure until you find a solution or establish that no solution exists (there are no more paths to try out).

Solution: Backtracking• place a queen in the top row, then note the column and diagonals it

occupies.

• then place a queen in the next row down, taking care not to place it in the same column or diagonal. Keep track of the occupied columns and diagonals and move on to the next row.

• If no position is open in the next row, we back track to the previous row and move the queen over to the next available spot in its row and the process starts over again.

Backtracking in Decision Trees

QQ

Q

Q

Q

Q

Q

Q

Q

Q

Q

QQ

Q

Q

QQ

Q

place 1st queen

place 2nd queen

place 3rd queen

place 4th queen

empty board

• The backtracking strategy is as follows:

1) Place a queen on the first available square in row 1.

2) Move onto the next row, placing a queen on the first available square there (that doesn't conflict with the previously placed queens).

3) Continue in this fashion until either:

a) you have solved the problem, or

b) you get stuck. • When you get stuck, remove the

queens that got you there, until you get to a row where there is another valid square to try.

Backtracking – Eight Queens Problem

Animated Example:http://www.hbmeyer.de/backtrack/achtdamen/eight.htm#up

QQ

QQ

Q Q

Continue…

http://www.hbmeyer.de/backtrack/achtdamen/eight.htm#up

http://www.hbmeyer.de/backtrack/achtdamen/eight.htm#up

Exercise – now it’s your turn!

Backtracking – Eight Queens Problem

• The neat thing about coding up backtracking, is that it can be done recursively, without having to do all the bookkeeping at once.• Instead, the stack or recursive calls does most of the bookkeeping • (ie, keeping track of which queens we've placed, and which

combinations we've tried so far, etc.)

void solveItRec(int perm[], int location, struct onesquare usedList[]) {

if (location == SIZE) { printSol(perm); } for (int i=0; i<SIZE; i++) {

if (usedList[i] == false) {

if (!conflict(perm, location, i)) { perm[location] = i; usedList[i] = true; solveItRec(perm, location+1, usedList); usedList[i] = false; } } } }

perm[] - stores a valid permutation of queens from index 0 to location-1. location – the column we are placing the next queenusedList[] – keeps track of the rows in which the queens have already been placed.

Found a solution to the problem, so print it!

Loop through possible rows to place this queen.

Only try this row if it hasn’t been used

Check if this position conflicts with any previous queens on the diagonal

1) mark the queen in this row2) mark the row as used3) solve the next column location

recursively4) un-mark the row as used, so

we can get ALL possible valid solutions.

Backtracking – 8 queens problem - Analysis• Another possible brute-force algorithm is generate the permutations

of the numbers 1 through 8 (of which there are 8! = 40,320), • and uses the elements of each permutation as indices to place a queen

on each row. • Then it rejects those boards with diagonal attacking positions.

• The backtracking algorithm, is a slight improvement on the permutation method, • constructs the search tree by considering one row of the board at a time,

eliminating most non-solution board positions at a very early stage in their construction.

• Because it rejects row and diagonal attacks even on incomplete boards, it examines only 15,720 possible queen placements.

• A further improvement which examines only 5,508 possible queen placements is to combine the permutation based method with the early pruning method: • The permutations are generated depth-first, and the search space is

pruned if the partial permutation produces a diagonal attack

100

Solving Sudoku – Later Steps1 1 2 1 2 4

1 2 4 8 1 2 4 8 9

uh oh!

101

Sudoku – A Dead End• We have reached a dead end in our search

• With the current set up none of the nine digits work in the top right corner

1 2 4 8 9

CS314 102

Backing Up• When the search reaches a dead end in backs up to the previous cell it was trying to fill and goes onto to the next digit

• We would back up to the cell with a 9 and that turns out to be a dead end as well so we back up again• so the algorithm needs to remember

what digit to try next• Now in the cell with the 8. We try and 9 and move forward again.

1 2 4 8 9

1 2 4 9

Characteristics of Brute Forceand Backtracking

• Brute force algorithms are slow• The don't employ a lot of logic

• For example we know a 6 can't go in the last 3 columns of the first row, but the brute force algorithm will plow ahead any way

• But, brute force algorithms are fairly easy to implement as a first pass solution• many backtracking algorithms are brute force algorithms

Key Insights• After trying placing a digit in a cell we want to solve the

new sudoku board• Isn't that a smaller (or simpler version) of the same problem

we started with?!?!?!?• After placing a number in a cell the we need to

remember the next number to try in case things don't work out.

• We need to know if things worked out (found a solution) or they didn't, and if they didn't try the next number

• If we try all numbers and none of them work in our cell we need to report back that things didn't work

Minesweeper

CS314 Recursive Backtracking 105

Minesweeper Reveal Algorithm• Minesweeper• click a cell

• if bomb game over• if cell that has 1 or more bombs on border

then reveal the number of bombs that border cell• if a cell that has 0 bombs on border

then reveal that cell as a blank and click on the 8 surrounding cells

CS314 Recursive Backtracking 107Another Backtracking ProblemA Simple Maze

Search maze until wayout is found. If no wayout possible report that.

Mazes and Backtracking• A final example of something that can be solved using backtracking is a maze. • From your start point, you will iterate through each

possible starting move. • From there, you recursively move forward. • If you ever get stuck, the recursion takes you back to

where you were, and you try the next possible move.

• In dealing with a maze, to make sure you don't try too many possibilities, • one should mark which locations in the maze have

been visited already so that no location in the maze gets visited twice.

• (If a place has already been visited, there is no point in trying to reach the end of the maze from there again.

109

The Local View

North

EastWest

Behind me, to the South is a door leading South

Which way doI go to getout?

Modified Backtracking Algorithm for Maze

• If the current square is outside, return TRUE to indicate that a solution has been found.If the current square is marked, return FALSE to indicate that this path has been tried.Mark the current square.for (each of the four compass directions) { if ( this direction is not blocked by a wall )

{ Move one step in the indicated direction from the current square.Try to solve the maze from there by making a recursive call.If this call shows the maze to be solvable, return TRUE to indicate

that fact.}

}Unmark the current square.Return FALSE to indicate that none of the four directions led to a solution.


Backtracking in ActionThe crucial part of the algorithm is the for loop that takes us through the alternatives from the current square. Here we have moved to the North.

for (dir = North; dir <= West; dir++){ if (!WallExists(pt, dir))

{if (SolveMaze(AdjacentPoint(pt, dir)))return(TRUE);

}


Backtracking in Action

Here we have moved North again, but there isa wall to the North .East is alsoblocked, so we try South. That call discovers thatthe square is marked, so it just returns.

113

So the next move we can make is West.

Where is this leading?


This path reaches a dead end.

Time to backtrack!

Remember theprogram stack!


The recursive calls end and return until we find ourselves back here.


And now we trySouth


Path Eventually Found

118

Other Backtracking Problems• Knapsack problem / Exhaustive Search

• Filling a knapsack. Given a choice of items with various weights and a limited carrying capacity find the optimal load out. 50 lb. knapsack. items are 1 40 lb, 1 32 lb. 2 22 lbs, 1 15 lb, 1 5 lb. A greedy algorithm would choose the 40 lb item first. Then the 5 lb. Load out = 45lb. Exhaustive search 22 + 22 + 5 = 49.

Assignment 1

SEARCH Heng Ji Feb 02/05, 2016. Search We will consider the problem of designing goal- based agents...

Documents

Transcript of SEARCH Heng Ji Feb 02/05, 2016. Search We will consider the problem of designing goal- based agents...