CS 430 Lecture 10 - University of...

Wednesday, February 1 CS 430 Artificial Intelligence - Lecture 8 1

Lecture 10

Reminder: Homework 2 due today Homework 3 posted to course webpage Questions?


Outline

Chapter 3 - Solving Problems by Searching Informed (Heuristic) Search Strategies

Heuristic Functions

Chapter 4 - Beyond Classical Search Local Search Algorithms and Optimization

Problems Hill-Climbing Search Simulated Annealing Genetic Algorithms


Heuristic Functions

Average solution is 22 steps. b is about 3 Uninformed tree search is 322 ~= 3.1 x 1010

Uninformed graph search is ~170,000 For 15-puzzle, graph search is ~1013


Heuristic Functions

To use A* search, need an admissible h(n), two common ones:

h1 = number of misplaced tiles. Clearly admissible

since every misplaced tile must be moved at least once.

h2 = sum of the distances of tiles from their goal

positions. Use Manhattan distance, the sum of the vertical and horizontal distances, since tiles cannot move diagonally. Admissible, since any move can only move a tile one step closer to its goal position.


Heuristic Functions

For a problem with solution at d = 22 IDS generates more nodes than will fit in memory

A*(h1) generates ~18,000 nodes

A*(h2) generates ~1200 nodes

Even for a problem with solution at d = 6 IDS generates ~680 nodes




Heuristic Functions

To compare heuristic functions, look at the effective branching factor, b*. For a solution with N total nodes and solution at depth d, b* is the branching factor that a uniform tree of depth d would have in order to contain N+1 nodes. I.e., N+1 = 1 + b* + (b*)2 + … + (b*)d

Better heuristics generate b* values close to 1. If h

2(n) >= h

1(n) for any node n, then can say h

2

dominates h1. Means that h

2 will never expand

more nodes than h1.


Generating Admissible Heuristics

One way to generate an admissible heuristic is to look at solutions to relaxed problems, problems with fewer restrictions on the actions.

The state space of a relaxed problem adds edges to the original state space. The optimal solution to the original problem will be a solution to the relaxed problem, but their may be better solutions, thus the cost of an optimal solution to a relaxed problem is an admissible heuristic for the original problem.


Example: 8-puzzle

Original problem: A tile can move from square A to square B if A is horizontally or vertically adjacent to B and B is blank.

Remove one or both restrictions to get relaxed problems:

(a) A tile can move from square A to square B if A is adjacent to B.

(b) A tile can move from square A to square B if B is blank.

(c) A tile can move from square A to square B.


Example: 8-puzzle

From (a), can derive h2 (Manhattan distance)

From (c), can derive h1 (misplaced tiles)

Heuristic derived from (b) is a homework problem...


Generating Admissible Heuristics

Admissible heuristics can be generated from pattern databases storing exact solutions to subproblems of the original problem. E.g., all solutions to configurations of the 8-puzzle involving 1-2-3-4-blank.


Learning Heuristics from Experience

An agent could construct a function h(n) by solving many instances and comparing the value of a feature relevant to predicting the state's value and the actual cost.

E.g., a feature might be "number of misplaced tiles", called x(n), and after 100 randomly generated puzzles, could determine that when x(n) = 5 that average solution cost is ~14. Then can use x(n) to predict h(n).


Local Search

Chapter 3 covers problems that have observable, deterministic, and known environments with solutions that are a sequence of actions.

Chapter 4 relaxes these assumptions. First consider problems where the path to the goal does not matter. The goal state is itself the solution to the problem. E.g. 8-queens problem. Do not care about the order that the queens are placed, just that they are placed.


Local Search

Strategies for solving such problems are called local search algorithms. Generally, they operate on a single current node and move only to the neighbors of that node in an effort to improve a solution.

Although not systematic, local search has two advantages

Use very little memory – usually a constant amount Often can find reasonable solutions in large or

continuous state spaces


State-Space Landscape

Goal is to find global minimum (minimize cost) or global maximum (maximize objective function).


Hill-Climbing Search

Hill-climbing search (HC) is the most basic local search technique. At each step, move towards the increasing value. Stop when there are no larger successors.

No search tree is maintained, so a single node is used and its state and value is updated as better states are discovered.

Does not look ahead past the immediate neighbors of the current state.



Steepest ascent version:

Receives: problemReturns: state that is local maximum

1. Set current to MakeNode(problem.InitialState)2. Loop

2.1 Set neighbor to highest-value successor of current2.2 If neighbor.Value <= current.Value then return current.State2.3 Set current to neighbor


Example: 8-queens Problem

Use a complete-state formulation – states are potential solutions. For 8-queens, states where there is one queen in each column.

Successors are all possible states generated by moving one queen to another space in the same column. 8 x 7 = 56 successor states.

Heuristic cost function h = number of pairs of queens that are attacking each other directly or indirectly. Global minimum is 0 when have an actual solution.


Example: 8-queens Problem

(a) h = 17, numbers are h value of successor states best moves are to h = 12.

(b) h = 1, but all successors higher, local minimum



HC sometimes called greedy local search, since it grabs a good neighbor without thinking about where to go next.

Works well in some cases. E.g. can get from (a) to (b) in 5 steps. Unfortunately often gets stuck. For random 8-queens problems, 86% get stuck and 14% are solved, but only 3-4 steps on average to find out – not bad for state space with 88 ~= 17 million states



Ridges – sequence of local maxima.

Gets stuck due to: Local maxima – goes towards peak, but stops

before getting to global maximum. E.g., state in (b) Plateaux – flat areas where there is no uphill, just

wander around. Could be a shoulder or "mesa".



Given HC algorithm stops when reaches a plateau. Can try sideways moves – moves to states with same value – in hopes that a plateau is really a shoulder. Must be careful not to allow infinite loop if on a real plateau.

Common to limit number of consecutive sideways moves. E.g., 100 consecutive sideways moves for 8-queens. Solved goes up to 94%, but average number of steps is 21 for solutions and 64 for failures.



HC variants choose next state in different ways Stochastic hill-climbing: choose at random from

uphill moves; probability of selection correlated with steepness. Usually converges slower than steepest ascent, but can find better solutions.

First-choice hill-climbing: generate successors at random until get one that is better than current state. Good when states have many (e.g., thousands) of successors.



Random-restart hill-climbing can be used to provide completeness. Conduct a series of HC searches from randomly generated initial states. If search has probability p of success then expected number of restarts is 1/p.

For 8-queens, without sideways moves p ~= 0.14, so expect 7 iterations (6 unsuccessful, 1 successful) with an average of 22 steps. With sideways moves p ~= .94, so expect 1.06 iterations giving ~25 steps.


Simulated Annealing

Basic hill-climbing is incomplete, since it never goes downhill and can get stuck on a local maxima. A purely random walk – choosing a successor uniformly at random – is complete, but extremely inefficient.

Simulated annealing (SA) combines ideas of hill-climbing and random walk. Annealing is a physical process used to harden (temper) metals and glass by heating to a high temperature then cool gradually.


Simulated Annealing

In computing, SA is usually done to minimize cost, so look at gradient descent trying to find the global minimum. Idea is to "escape" a local minima by "shaking" it out.

Start out by shaking hard (i.e., high temperature) causing large and/or frequent jumps at the beginning, but gradually reducing the intensity of shaking (i.e., lower the temperature) causing decreased size and frequency of jumps.


Simulated Annealing

Receives: problem, schedule // a time -> temp mapReturns: solution state

1. Initialize current to MakeNode (problem.InitialState)2. For t from 1 to infinity do

2.1 T = schedule[t] // get current temp. from sched.2.2 If T = 0 then return current.State2.3 next = random successor of current2.4 ΔE = next.Value – current.Value2.5 If ΔE > 0 then current = next2.6 Else current = next only with probability eΔE/T


Simulated Annealing

eΔE/T decreases exponentially with the "badness" of the move (ΔE) and decreases as T (temperature) decreases.

If the schedule lowers T slowly enough, the global minimum will be found with probability approaching 1.


Genetic Algorithms

In a Genetic algorithm (GA) successor states are generated by combining two parent states rather than modifying a single state.

Combination depends on Fitness function that ranks states Probability of selection as parents Probability of crossover – how much of parent

continues into next generation Probability of mutation


Example: 8-Queens

k randomly generated states is a population

Individual represented as strings, e.g. position of each queen in 8 columns

Fitness function = # of non-attacking pairs of queens. Selection probability proportional to score.


Example: 8-Queens

States of first pair of parents combined into first new child state. Shaded part is lost while unshaded part is retained in next generation.

In the beginning, children can be very different from parents => large steps; later population is similar, so children are similar => small steps

CS 430 Lecture 10 - University of...

Documents

Transcript of CS 430 Lecture 10 - University of...