Warm Up A* Exercise

In groups, draw the tree that A* builds to find a path from Oradea to Bucharest marking g, h, and f for each node. Use straight-line distance as h(n)

In what order are the nodes expanded? What is the path found?

Local Search and Constraint Reasoning

CS 510 Lecture 3October 6, 2009

Reminder

• Project proposal : due in two weeks

• Thanks for the pre-proposal, should be graded, some have comments

• Homework 1 due next week

• Next week: guest lecture by Janith Weerasinghe, no supplemental readings

Overview

• Local Search (Hill climbing, simulated annealing, genetic algorithms)

• Constraint Reasoning (Constraint Satisfaction problems (CSP), constraint optimization (COP), Distributed COP

• Paper discussion

Local Search

• In many optimization problems, the path to the goal is irrelevant; the goal state itself is the solution

• State space = set of "complete" configurations

• Find configuration satisfying constraints, e.g., n-queens

• In such cases, we can use local search algorithms

• keep a single "current" state, try to improve it

Example: n-queens

• Put n queens on an n × n board with no two queens on the same row, column, or diagonal

Hill-Climbing Search

• "Like climbing Everest in thick fog with amnesia"

Hill-climbing search

• Problem: depending on initial state, can get stuck in local maxima

Hill-climbing search 8-queens problem

• h = number of pairs of queens that are attacking each other, either directly or indirectly

• h = 17 for the above state

Hill-climbing search 8-queens problem

• A local minimum with h=1

Annealing in Metallurgy

• Remove crystal dislocations in metal

• Heat the material - cause atoms to become unstuck from position

• Then slowly cool, atoms wander try to find configurations with lower energy

Simulate this process to escape local minima

• Replace solution with nearby solution

• How nearby depends on “temperature”

• Heat - nearly random solution

• Slowly cool - gradually improve

Simulated annealing search

• Idea: escape local maxima by allowing some "bad" moves but gradually decrease their frequency

Properties of Simulated annealing search

• One can prove: If T decreases slowly enough, then simulated annealing search will find a global optimum with probability approaching 1

• Widely used in VLSI layout, airline scheduling, etc

Local beam search

• Keep track of k states rather than just one

• Start with k randomly generated states

• At each iteration, all the successors of all k states are generated

• If any one is a goal state, stop; else select the k best successors from the complete list and repeat.

Genetic Algorithms

• A successor state is generated by combining two parent states

• Start with k randomly generated states (population)

• A state is represented as a string over a finite alphabet (often a string of 0s and 1s)

• Evaluation function (fitness function). Higher values for better states.

• Produce the next generation of states by selection, crossover, and mutation

Genetic algorithms

• Fitness function: number of non-attacking pairs of queens (min = 0, max = 8 × 7/2 = 28)

• 24/(24+23+20+11) = 31%

• 23/(24+23+20+11) = 29% etc

Genetic algorithms

Constraint Reasoning

Constraint Satisfaction Problem (CSP)

• Variables: V1,V2,V3...with domains D1, D2, D3

• Constraints: Set of allowed value pairs

• V1 “not equal” V2 = {(red,blue),(green, blue,(green, red)}

• Solution: Assign values to variables that satisfy all constraints

• V1=red, V2=blue, V3=green

V1

V2 V3

= =

D1={red, green}

D2={blue,red} D3={green}

3SAT as CSP

• xi are variables

• values are true and false

• Constraints expressed in conjunctive normal form

• ex 1: (x1 v x2 v x3) ^ (~x1 v ~x2 v ~x3) ^ (~x1 v x2 v ~x3)

• ex 2: (x1 v ~x2) ^ (~x1 v ~x2) ^ (~x1 v ~x2)

Class Exercise

• Formulate the 4 queens problem as a CSP

• variables

• values

• constraints

What if no solution? Or multiple solutions?

• If no solution, minimize broken constraints?

• If multiple solutions, some solutions may be preferred

• Gives rise to the Constraint Optimization Problem

Constraint Optimization

• Constraints have degrees of violation (or degrees of satisfaction)

• Goal is to minimize violation (or maximize satisfaction)

• Significantly generalizes CSPs

Constraint Optimization Problem (COP)

• Given:

• Variables (x1,x2,...,xn)

• Finite, discrete domains (D1,D2,...,Dn)

• Foreach xi, xj:

• Valued constraint function

• fij: Di x Dj N

Constraint Optimization(COP)

• Goal: Find complete solution: an assignment A that minimizes F(A) where

• F(A) = Sum(fij(di,dj), xi di, xj dj in A

• Example:

di dj f(di,dj)

3

1

1

2

3

333

3

1 11

1 11

2F(A)=12 F(A)=6 F(A)=5

Example: Map Coloring

• Variables WA, NT, Q, NSW, V, SA, T

• Domains Di = {red,green,blue}

• Constraints: adjacent regions must have different colors

e.g., WA ≠ NT, or (WA,NT) in {(red,green),(red,blue),(green,red), (green,blue),(blue,red),(blue,green)}

Examples: Map Coloring

• Solutions are complete and consistent assignments, e.g., WA = red, NT = green,Q = red,NSW = green,V = red,SA = blue,T = green

Constraint Graph

• Binary CSP: each constraint relates two variables

• Constraint graph: nodes are variables, arcs are constraints

CSP as search

• Initial State: Empty assignment {}

• Successor Function: Assign value to an unassigned variable

• Goal Test: Complete assignment that does not violate any constraints

• Path Cost: Constant for each step

Backtracking Search Basically DFS

• Map coloring (no adjacent regions the same color)

Improving efficiency

• General-purpose heuristics can give huge gains in speed:

• Which variable should be assigned next?

• In what order should its values be tried?

• Can we detect inevitable failure early?

Most Constrained Variable

• Most constrained variable:

choose the variable with the fewest legal values

• a.k.a. minimum remaining values (MRV) heuristic

Most Constraining Variable

• Tie-breaker among most constrained variables

• Most constraining variable:

• Choose the variable with the most constraints on remaining variables

Least constraining variable assignment

• Given a variable, choose the least constraining value:

• the one that rules out the fewest values in the remaining variables

• Combining these heuristics makes 1000 queens possible (as opposed to about 25)

Forward Checking

• Idea:

• Keep track of remaining legal values for unassigned variables

• Terminate search when any variable has no legal values

Forward Checking

• Idea:

• Keep track of remaining legal values for unassigned variables

• Terminate search when any variable has no legal values

Constraint Propagation

• Forward checking propagates information from assigned to unassigned variables, but doesn't provide early detection for all failures:

• NT and SA cannot both be blue!

• Constraint propagation repeatedly enforces constraints locally

Arc Consistency

• Simplest form of propagation makes each arc consistent

• X Y is consistent iff

• for every value x of X there is some allowed y

Arc Consistency

• Simplest form of propagation makes each arc consistent

• X Y is consistent iff

• for every value x of X there is some allowed y

• If X loses a value, neighbors of X must be rechecked

Arc Consistency• Simplest form of propagation makes each arc consistent

• X Y is consistent iff

• for every value x of X there is some allowed y

• If X loses a value, neighbors of X must be rechecked

• Detects failures quicker than forward checking

Local Search for CSPs

• Hill-climbing, simulated annealing typically work with "complete" states, i.e., all variables assigned

• To apply to CSPs:

• allow states with unsatisfied constraints

• operators reassign variable values

• Variable selection: randomly select any conflicted variable

• Value selection by min-conflicts heuristic:

• choose value that violates the fewest constraints

• i.e., hill-climb with h(n) = total number of violated constraints

Example: 4 Queens• States: 4 queens in 4 columns (44 = 256 states)

• Actions: move queen in column

• Goal test: no attacks

• Evaluation: h(n) = number of attacks

• Given random initial state, can solve n-queens in almost constant time for arbitrary n with high probability (e.g., n = 10,000,000)

44

Distributed Optimization Problem

“How do a set of agents optimize over a set of alternatives that have varying degrees of global quality?”

Examples

• allocating resources

• constructing schedules

• planning activities

Difficulties

• No global control/knowledge

• Localized communication

• Quality guarantees required

• Limited Time

Interesting Research Questions

• How to evaluate performance?

• Cycles, CBR, CCC, etc?

• How much distribution counts? OptAPO?

• Local approximations of optimization

• How to apply DCR to new domains?

• More flexible DCOP? multiply constrained, resource constraints?

• Privacy?45

46

• Common model for automated coordination

• Assign values to variables, subject to constraints

• Problems

• Constraints may be personal or proprietary

• Current algorithms not designed for privacy

• NP-hard!

Privacy in Constraint Optimization

Supply Chain

47

EucalyptusGrower

Transport Company

Transport Company

Corn Farmer

Shipping Company

Gas

Alcohol

Bundled

leavesBundled

corn

DistributorPackedleaves

Bundledcorn

Tanked alcohol

Packedleaves

RetailerWholesaler

Need to coordinate when to send and how much

Algs: Asynchronous Backtracking (ABT)

• Not DCOP, constraint satisfaction

• Agents form communication/priority chain

• Pass down tentative assignments (ok? messages)

• If value inconsistent with higher agents, change

• Pass up violations (nogood messages)

• nogoods causes higher agents to change value

• All happens concurrently

48

Synchronous Branch and Bound (SynchBB)

• Simulate branch and bound in distributed environment

• Messages sent in predefined, sequential order

• First agent sends a partial solution (one variable)

• Next agent adds variable, evaluates

• If < bound (best complete solution cost), send on

• If > bound, try new values

• If all values tried, backtrack

49

Synchronous Iterative Deepening (SynchID)

• Simulates iterative deepening

• First agent picks value, sends it and cost limit (initially 0) to next agent

• Next agent tries to find value under limit

• If succeed, pass on values, limit to next agent

• If fail, backtrack up and increase the limit

50

Adopt (Asynchronous Optimization)

• Agents organized in tree, not chain, constraints between ancestors/descendants not siblings

• When agents receive messages

• choose value with min cost

• send VALUE message to descendants

• send COST message to parents

• send THRESHOLD message to child

51

DPOP (Dynamic Pseudotree)

• Same tree as Adopt, but sequential

• Dynamic programming/variable elimination approach

• Each agent sends only one message up (with constraint information) and then one message down (with value information)

52

Discussion: PageRank and Eigenvalues

• Create matrix whose values aij are the number of backlinks into page i from page j divided by number of forward links of page j.

• a14 = 1 link from page 4/2 total links from page 4 = 1/2

• To get page rank find eigenvector Ax=lambda(x) where lambda =1 (eigenvalue)

53

Discussion: PageRank and Eigenvalues

• To get page rank find eigenvector Ax=lambda(x) where lambda =1 (eigenvalue)

• A [12 4 9 6] = [12 4 9 6]

• All multiples of eigenvectors are eigenvectors (normalize to fraction)

• [12/31 4/31 9/31 6/31]

54

Page Rank

• Have to add in E to fix link problems

• Vector [0.15 0.15 0.15 0.15] for democratic E

• or [x 0 0 0] for specific E

• rank now c(A + E x 1)R’ = R’ still looking for an eigenvector just more complex one

55

Iterating Rank• R0 = [0.15 0.15 0.15 0.15]

• R1 = df R0

• R1 = [0.22 0.05 0.19 0.12]

• d = 0.3 - 0.32

• R1 = R1 + [0.003 0.003 0.003 0.003]

• delta = ||[0.073 -0.097 0.043 0.027]||||v|| = sqrt(v12 + v22 + ... + vn2)

Is PageRank Admissible?

• How does the type of search talked about here connect with the other search algorithms we’ve studied?

Customized PageRank

• Where does google customize pagerank today?

• What are the pitfalls of this?

58