Easy Optimization Problems,

Relaxation,Local Processing

for a single variable

The advantages of multileveling

Linear running time Provided a “good coarsening” is performed:1.The number of variables is reduced in such

a way that preserves the essence (skeleton) of the graph => an easier problem

2.Enables processing in different scales, moves which are not likely to happen systematically in a “flat” approach

A better GLOBAL solver

General 1D Arrangement Problems

A graph G(v,

aij

General 1D Arrangement Problems

E(x)= i j ai j | xi -xj |p

xi = vi /2 + k:k)<i) vk

i j

xi xj

From the graph

To the arrangement

aij

aij

The complexity of pointwise relaxation for P=2

Go over all variables in lexicographic order,

put xi at the weighted average location of its graph neighbors

Problem: Does not preserve the volume demands!

Reinforce volume demands at the end of each sweep If the reinforcement is done after every variable, the

complexity will be quadratic and not linear ! Sorting of xi is O(nlogn), however, usually logn<C,

where C is the constant of the linear complexity If the ‘sort’ is too slow, use bucketing instead

Different types of relaxation

Variable by variable relaxation – strict minimization

Changing a small subset of variables simultaneously – Window strict minimization relaxation

Stochastic relaxation – may increase the energy – should be followed by strict minimization

Variable by variable strict unconstrained minimization

Discrete (combinatorial) case : Ising model

Exc#1: 2D Ising spins exercise

Minimize

Periodic boundary condition Initialize randomly: with probability .51. Go over the grid in lexicographic order, for each spin

choose 1 or -1 whichever minimizes the energy (choose with probability ½ when the two possibilities have the

same energy) until no changes are observed.

2. Repeat 3 times for each of the 4 possibilities of (h1,h2).

3. Is the global minimum achievable?

4. What local minima do you observe?

i

iiji

ji shss,

1is

Variable by variable strict unconstrained minimization

Discrete (combinatorial) case : Ising model Quadratic case : P=2 General functional : P=1 , P>2

Exc#2: Pointwise relaxation for P=1

Minimize

Pick a variable , fix all at

Minimize

Find the optimal location for

ij

jiij xxax ||)(

j

jiiji xxax |~|)(

ix ijjx jx~

ix

Pointwise relaxation for P=6

Minimize

Pick a variable , fix all at

Minimize

Find the roots (zeros) of

ij

jiij xxax 6)()(

j

)x(xax jiiji6~)(

ix ijjx jx~

0)~(6)(0)(

)( 5

jij

ijii

ixxaxF

xd

xd

)( ixF

Variable by variable strict unconstrained minimization

Discrete (combinatorial) case : Ising model Quadratic case : P=2 General functional : P=1 , P>2Newton’s Method

Newton’s Method (Newton-Raphson)

Geometry Taylor expansion

Starting close enough to the root results with a very fast convergence

What is “close enough”? May even diverge or oscillate

Verify local reduction in E

Variable by variable strict unconstrained minimization

Discrete (combinatorial) case : Ising model Quadratic case : P=2 General functional : P=1 , P>2Newton’s MethodVerify local reduction in ENumerical derivatives

Numerical derivatives

Newton’s Method :

Calculate numerically

)('

)(1

n

nnn xF

xFxx

)(' nxF

x

nxnn

xFxFxF

)()(

)('

Variable by variable strict unconstrained minimization

Discrete (combinatorial) case : Ising model Quadratic case : P=2 General functional : P=1 , P>2Newton’s MethodVerify local reduction in ENumerical derivativesSteepest descent

Steepest descent Level sets CxE )(

Steepest descent Level sets E(x,y)=x2+y2

Steepest descent Level sets E(x,y)=x2+y2

c1

c2

c1 < c2

Steepest descent Level sets The gradient at a point is perpendicular to

the level set and is directed towards the maximal rate of increase in the energy

Vector field of gradients

CxE )(

Steepest descentThe vector field of the gradient of E(x,y)=x2+y2

At every point it is perpendicular to the level set

c1

c2

c1 < c2

Steepest descent Level sets The gradient at a point is perpendicular to

the level set and is directed towards the maximal rate of increase in the energy

Vector field of gradients

=> Choose the opposite direction of the

gradient as the direction for maximal

decrease of the energy

How much should you go in this direction?

CxE )(

Variable by variable strict unconstrained minimization

Discrete (combinatorial) case : Ising model Quadratic case : P=2 General functional : P=1 , P>2Newton’s MethodVerify local reduction in ENumerical derivativesSteepest descentLine search

Line search Starting at some Minimize along Exact minimization: solve Guess an and use backtracking Quadratic approximation: Choose ,

then , draw a parabola through the 3 points and find its minimum

Verify local reduction in EIf not - choose the available minimum

0x

)()( 00 xEx 0/))(( ddE

2

Exc#3: Steepest descent exercise

For

at

Find the steepest descent directionCompare its analytical and numerical calculationsChoose 2 small steps in this directionDraw a parabola through the 3 pointsFind the minimum of the parabolaVerify reduction in the energyFind a step that increases the energy

)3

cos(150)10

sin(1002),( 22 yxyxyxE

)15,12(),( yx

An example of a single node relaxation for the placement

problem

The placement problemGiven a hypergraph:

1. A list of nodes each with its length and pins’ location

2. A list of lists of subsets of nodes - hyperedges

The hypergraph for a microchip

The placement problemGiven a hypergraph:

1. A list of nodes each with its length and pins’ location

2. A list of lists of subsets of nodes - hyperedges Minimize the sum of all wires approximated by

the half Bounding Box of each hyperedge

Bounding Box

The bounding box

Pins’ locations

The placement problemGiven a hypergraph:

1. A list of nodes each with its length and pins’ location

2. A list of lists of subsets of nodes - hyperedges Minimize the sum of all wires approximated by

the half Bounding Box of each hyperedge Approximate the hypergraph by a graph and

the Bounding Box by a quadratic functional

From hypergraph to graphAdd a virtual node at the center of mass of

the nodes belonging to an hyperedge

From hypergraph to graphAdd a virtual node at x0 , the center of mass of the nodes

The resulting graph is

x1

x3

x2

x4

x0=(x1 + x2 + x3 + x4) /4

E(x)=i(xi - x0)2 , i=1,…,4

By eliminating x0 : E(x)=ij(xi - xj)2/4 , i,j=1,…,4

From hypergraph to graph

E(x)=ij(xi - xj)2/4 , i,j=1,…,4

x1

x3

x2

x4

A hyperedge with n nodes contributes n(n-1)/2 quadratic terms with weight 1/n to E(x)

Creates a clique of connections

The placement problemGiven a hypergraph:

1. A list of nodes each with its length and pins’ location

2. A list of lists of subsets of nodes - hyperedges Minimize the sum of all wires approximated by

the half Bounding Box of each hyperedge Approximate the hypergraph by a graph and

the Bounding Box by a quadratic function

The placement has 2 phases: global and detailed Use the original definition towards the detailed

Approximations for the placement problem

Given a hypergraph => translate it to a graph The nodes are now connected at their center

of mass (not at the pins) with straight lines (not rectilinear connections)

The used energy function is quadratic (not the bounding box)

Towards the end of the global placement and definitely at the (discrete) detailed placement use the original definition of the problem not the approximations

Data structure

For each node in the graph keep

1. A list of all the graph’s neighbors: for each neighbor keep a pair of index and weight

2. …

3. …

4. Its current placement

5. The unique square in the grid the node belongs to

For each square in the grid keep

1. A list of all the nodes which are mostly within Defines the current physical neighborhood

The augmented E(xi,yi) to be minimized

For node i, fix all other nodes at their current

and minimize the augmented functional

----------- ---------------------------------- --------------------

where j are the nodes in the 3x3 window of squares around the square which includes i

j

jijij

ijii j),(iyyxxayxE overlap])~()~[(),( 22

)~,~( jj yx

The augmented E(xi,yi) to be minimized

For node i, fix all other nodes at their current

and minimize the augmented functional

----------- ---------------------------------- --------------------

where j are the nodes in the 3x3 window of squares around the square which includes i

How can the “steepest descent direction” be found?

j

jijij

ijii j),(iyyxxayxE overlap])~()~[(),( 22

)~,~( jj yx

The “steepest descent”

The augmented E(xi,yi) to be minimized

For node i, fix all other nodes at their current

and minimize the augmented functional

----------- ---------------------------------- --------------------

where j are the nodes in the 3x3 window of squares around the square which includes i

How can the “steepest descent direction” be found? Use numerical “discrete derivatives” For simplicity calculate each direction separately! This is a numerical discrete line search minimization

j

jijij

ijii j),(iyyxxayxE overlap])~()~[(),( 22

)~,~( jj yx

Move node to the rightix

i

Changes in the energy and overlap

X-direction “discrete derivatives”

For node , fix all other nodes at their current Current overlap

------- ---------------- -------------------------

Calculate

Calculate

Calculate optimal change by quadratic approximation Check that the energy has indeed decreased Update the data structure at the end of the sweep

j

jij

iji xxaxE 2)~()( ),( jii jx

~

)(ixixE

)2(ixixE

Problems

Slowly converging Minimization of one variable may conflict

and influence other variables

=> instead of a single variable at a time,

use a small subset

Window minimization