Post on 19-Dec-2015
Easy Optimization Problems,
Relaxation,Local Processing
for a single variable
The advantages of multileveling
Linear running time Provided a “good coarsening” is performed:1.The number of variables is reduced in such
a way that preserves the essence (skeleton) of the graph => an easier problem
2.Enables processing in different scales, moves which are not likely to happen systematically in a “flat” approach
A better GLOBAL solver
General 1D Arrangement Problems
A graph G(v,
aij
General 1D Arrangement Problems
E(x)= i j ai j | xi -xj |p
xi = vi /2 + k:k)<i) vk
i j
xi xj
From the graph
To the arrangement
aij
aij
The complexity of pointwise relaxation for P=2
Go over all variables in lexicographic order,
put xi at the weighted average location of its graph neighbors
Problem: Does not preserve the volume demands!
Reinforce volume demands at the end of each sweep If the reinforcement is done after every variable, the
complexity will be quadratic and not linear ! Sorting of xi is O(nlogn), however, usually logn<C,
where C is the constant of the linear complexity If the ‘sort’ is too slow, use bucketing instead
Different types of relaxation
Variable by variable relaxation – strict minimization
Changing a small subset of variables simultaneously – Window strict minimization relaxation
Stochastic relaxation – may increase the energy – should be followed by strict minimization
Variable by variable strict unconstrained minimization
Discrete (combinatorial) case : Ising model
Exc#1: 2D Ising spins exercise
Minimize
Periodic boundary condition Initialize randomly: with probability .51. Go over the grid in lexicographic order, for each spin
choose 1 or -1 whichever minimizes the energy (choose with probability ½ when the two possibilities have the
same energy) until no changes are observed.
2. Repeat 3 times for each of the 4 possibilities of (h1,h2).
3. Is the global minimum achievable?
4. What local minima do you observe?
i
iiji
ji shss,
1is
Variable by variable strict unconstrained minimization
Discrete (combinatorial) case : Ising model Quadratic case : P=2 General functional : P=1 , P>2
Exc#2: Pointwise relaxation for P=1
Minimize
Pick a variable , fix all at
Minimize
Find the optimal location for
ij
jiij xxax ||)(
j
jiiji xxax |~|)(
ix ijjx jx~
ix
Pointwise relaxation for P=6
Minimize
Pick a variable , fix all at
Minimize
Find the roots (zeros) of
ij
jiij xxax 6)()(
j
)x(xax jiiji6~)(
ix ijjx jx~
0)~(6)(0)(
)( 5
jij
ijii
ixxaxF
xd
xd
)( ixF
Variable by variable strict unconstrained minimization
Discrete (combinatorial) case : Ising model Quadratic case : P=2 General functional : P=1 , P>2Newton’s Method
Newton’s Method (Newton-Raphson)
Geometry Taylor expansion
Starting close enough to the root results with a very fast convergence
What is “close enough”? May even diverge or oscillate
Verify local reduction in E
Variable by variable strict unconstrained minimization
Discrete (combinatorial) case : Ising model Quadratic case : P=2 General functional : P=1 , P>2Newton’s MethodVerify local reduction in ENumerical derivatives
Numerical derivatives
Newton’s Method :
Calculate numerically
)('
)(1
n
nnn xF
xFxx
)(' nxF
x
nxnn
xFxFxF
)()(
)('
Variable by variable strict unconstrained minimization
Discrete (combinatorial) case : Ising model Quadratic case : P=2 General functional : P=1 , P>2Newton’s MethodVerify local reduction in ENumerical derivativesSteepest descent
Steepest descent Level sets CxE )(
Steepest descent Level sets E(x,y)=x2+y2
Steepest descent Level sets E(x,y)=x2+y2
c1
c2
c1 < c2
Steepest descent Level sets The gradient at a point is perpendicular to
the level set and is directed towards the maximal rate of increase in the energy
Vector field of gradients
CxE )(
Steepest descentThe vector field of the gradient of E(x,y)=x2+y2
At every point it is perpendicular to the level set
c1
c2
c1 < c2
Steepest descent Level sets The gradient at a point is perpendicular to
the level set and is directed towards the maximal rate of increase in the energy
Vector field of gradients
=> Choose the opposite direction of the
gradient as the direction for maximal
decrease of the energy
How much should you go in this direction?
CxE )(
Variable by variable strict unconstrained minimization
Discrete (combinatorial) case : Ising model Quadratic case : P=2 General functional : P=1 , P>2Newton’s MethodVerify local reduction in ENumerical derivativesSteepest descentLine search
Line search Starting at some Minimize along Exact minimization: solve Guess an and use backtracking Quadratic approximation: Choose ,
then , draw a parabola through the 3 points and find its minimum
Verify local reduction in EIf not - choose the available minimum
0x
)()( 00 xEx 0/))(( ddE
2
Exc#3: Steepest descent exercise
For
at
Find the steepest descent directionCompare its analytical and numerical calculationsChoose 2 small steps in this directionDraw a parabola through the 3 pointsFind the minimum of the parabolaVerify reduction in the energyFind a step that increases the energy
)3
cos(150)10
sin(1002),( 22 yxyxyxE
)15,12(),( yx
An example of a single node relaxation for the placement
problem
The placement problemGiven a hypergraph:
1. A list of nodes each with its length and pins’ location
2. A list of lists of subsets of nodes - hyperedges
The hypergraph for a microchip
The placement problemGiven a hypergraph:
1. A list of nodes each with its length and pins’ location
2. A list of lists of subsets of nodes - hyperedges Minimize the sum of all wires approximated by
the half Bounding Box of each hyperedge
Bounding Box
The bounding box
Pins’ locations
The placement problemGiven a hypergraph:
1. A list of nodes each with its length and pins’ location
2. A list of lists of subsets of nodes - hyperedges Minimize the sum of all wires approximated by
the half Bounding Box of each hyperedge Approximate the hypergraph by a graph and
the Bounding Box by a quadratic functional
From hypergraph to graphAdd a virtual node at the center of mass of
the nodes belonging to an hyperedge
From hypergraph to graphAdd a virtual node at x0 , the center of mass of the nodes
The resulting graph is
x1
x3
x2
x4
x0=(x1 + x2 + x3 + x4) /4
E(x)=i(xi - x0)2 , i=1,…,4
By eliminating x0 : E(x)=ij(xi - xj)2/4 , i,j=1,…,4
From hypergraph to graph
E(x)=ij(xi - xj)2/4 , i,j=1,…,4
x1
x3
x2
x4
A hyperedge with n nodes contributes n(n-1)/2 quadratic terms with weight 1/n to E(x)
Creates a clique of connections
The placement problemGiven a hypergraph:
1. A list of nodes each with its length and pins’ location
2. A list of lists of subsets of nodes - hyperedges Minimize the sum of all wires approximated by
the half Bounding Box of each hyperedge Approximate the hypergraph by a graph and
the Bounding Box by a quadratic function
The placement has 2 phases: global and detailed Use the original definition towards the detailed
Approximations for the placement problem
Given a hypergraph => translate it to a graph The nodes are now connected at their center
of mass (not at the pins) with straight lines (not rectilinear connections)
The used energy function is quadratic (not the bounding box)
Towards the end of the global placement and definitely at the (discrete) detailed placement use the original definition of the problem not the approximations
Data structure
For each node in the graph keep
1. A list of all the graph’s neighbors: for each neighbor keep a pair of index and weight
2. …
3. …
4. Its current placement
5. The unique square in the grid the node belongs to
For each square in the grid keep
1. A list of all the nodes which are mostly within Defines the current physical neighborhood
The augmented E(xi,yi) to be minimized
For node i, fix all other nodes at their current
and minimize the augmented functional
----------- ---------------------------------- --------------------
where j are the nodes in the 3x3 window of squares around the square which includes i
j
jijij
ijii j),(iyyxxayxE overlap])~()~[(),( 22
)~,~( jj yx
The augmented E(xi,yi) to be minimized
For node i, fix all other nodes at their current
and minimize the augmented functional
----------- ---------------------------------- --------------------
where j are the nodes in the 3x3 window of squares around the square which includes i
How can the “steepest descent direction” be found?
j
jijij
ijii j),(iyyxxayxE overlap])~()~[(),( 22
)~,~( jj yx
The “steepest descent”
The augmented E(xi,yi) to be minimized
For node i, fix all other nodes at their current
and minimize the augmented functional
----------- ---------------------------------- --------------------
where j are the nodes in the 3x3 window of squares around the square which includes i
How can the “steepest descent direction” be found? Use numerical “discrete derivatives” For simplicity calculate each direction separately! This is a numerical discrete line search minimization
j
jijij
ijii j),(iyyxxayxE overlap])~()~[(),( 22
)~,~( jj yx
Move node to the rightix
i
Changes in the energy and overlap
X-direction “discrete derivatives”
For node , fix all other nodes at their current Current overlap
------- ---------------- -------------------------
Calculate
Calculate
Calculate optimal change by quadratic approximation Check that the energy has indeed decreased Update the data structure at the end of the sweep
j
jij
iji xxaxE 2)~()( ),( jii jx
~
)(ixixE
)2(ixixE
Problems
Slowly converging Minimization of one variable may conflict
and influence other variables
=> instead of a single variable at a time,
use a small subset
Window minimization