Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.
-
date post
19-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.
![Page 1: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/1.jpg)
Easy Optimization Problems,
Relaxation,Local Processing
for a small subset of variables
![Page 2: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/2.jpg)
Changes in the energy and overlap
![Page 3: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/3.jpg)
X-direction line search and “discrete derivatives”
For node , fix all other nodes at their current Current overlap
------- ---------------- -------------------------
Calculate => choose sign Calculate sign => quadratic approx.
To effectively calculate the derivative which means:
Calculate and average:
j
jij
iji xxaxE 2)~()( ),( jii jx
~
)(,)(ii xixi xExE
)2(ixixE
in change unit per in change of rate the ii xxE )(
)(,)(ii xixi xExE
2
)/()]()([/)]()([iiii xixixixi xExExExE
Linesearch
Discretederivative
![Page 4: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/4.jpg)
Different types of relaxation
Variable by variable relaxation – strict minimization
Changing a small subset of variables simultaneously – Window strict minimization relaxation
Stochastic relaxation – may increase the energy – should be followed by strict minimization
![Page 5: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/5.jpg)
Window strict unconstrained minimization
Discrete (combinatorial) case :
Permutations of small subsets P=2, placement
![Page 6: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/6.jpg)
1D data base
The nodes: 1 2 3 4 5 6 7 8 9A permutation 5 39 6 2 7 1 4 8 (1)= 7 , (2)=5 , (3)=2 …
To find a consecutive subset of nodes in the current permutation, we need the inverse of -1
-1(1)= 5 , -1(2)= 3 , -1(3)= 9 …
In 2D we have to insert a grid and store the list of nodes within each square
![Page 7: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/7.jpg)
Window strict unconstrained minimization
Discrete (combinatorial) case :
Permutations of small subsets P=2, placement
Problem: very small number of variables! Quadratic case : P=2
![Page 8: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/8.jpg)
Window relaxation for P=2 unconstrained version
Minimize
Pick a window of variables , fix all variables at Find a correctionto so as to
minimize
Quadratic functional in many variables – easy to solve!
ij
jiij xxax 2)()(
2
,,
2 ~~~~)( )xx(a)xx(a jiijjiij iWjWiWji
ji
bAi
systemlinear the solve0)(
)(
Wi x~
i Wixi ,~
![Page 9: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/9.jpg)
Updating the window variables For each i in the window W
insert the correction: xi new
=xi+i
Sort the xi news and rearrange the window
accordingly To improve the result obtained by the
inner changes apply node-by-node relaxation on W and on its boundary
At the and compare the “old” energy with the “new” energy and accept / reject
Revision process: try a “big” change, improve it by local minimization, choose
![Page 10: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/10.jpg)
Window relaxation for P=2 constrained version
To prevent nodes from collapsing on each other To express the aim of having an
approximate permutation of add 2 constraints:
Wiiix }~{
2,1,~)~(
mvxvx i
m
Wiii
mi
Wii
Wiix }~{
![Page 11: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/11.jpg)
Exc#4: Permutation’s invariants
1) Prove that are
invariant under permutation.
2) Is it also true for m=3?
2,1,~1
mvx i
n
i
mi
![Page 12: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/12.jpg)
Window relaxation for P=2 constrained version
To prevent nodes from collapsing on each other To express the aim of having an
approximate permutation of add 2 constraints:
Minimization with equality constraints Lagrange multipliers
Wiiix }~{
2,1,~)~(
mvxvx i
m
Wiii
mi
Wii
Wiix }~{
0~2
01
Wiiii
Wiii
xvm
vm
![Page 13: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/13.jpg)
Lagrange multipliers
Goal: Transform a constrained optimization problem with n variables and m equality constraints to an unconstrained optimization problem with n+m variables. The new m variables are called the Lagrange multipliers
Geometry explanation
![Page 14: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/14.jpg)
![Page 15: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/15.jpg)
![Page 16: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/16.jpg)
2 constraints in 3D
![Page 17: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/17.jpg)
The optimal ellipsoid is tangent to the constraints curve
![Page 18: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/18.jpg)
Lagrange multipliers
Goal: Transform a constrained optimization problem with n variables and m equality constraints to an unconstrained optimization problem with n+m variables. The new m variables are called the Lagrange multipliers
Geometry explanation Construct an augmented functional –
the Lagrangian
![Page 19: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/19.jpg)
The LagrangianGiven E(x) subject to m equality constraints:
hk(x)=0 , k=1,…,m , construct the Lagrangian
L(x,) = E(x) + kkhk(x) and solve the system
of n+m equations
The value of is meaningful
mkxL
nix
xL
k
i
,...,1,0),(
,...,1,0),(
for
for
The constraints!
![Page 20: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/20.jpg)
![Page 21: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/21.jpg)
The Lagrangian: an example Minimize E(x,y)=x+y Subject to h(x,y)=x2+y2-2 The Lagrangian: L(x,y,=E(x,y)+(x2+y2-2
020
0210
0210
0
22
yxE
yy
E
xx
E
hEhEL
The constraint!
The co-linearityof the gradients
![Page 22: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/22.jpg)
Window relaxation for 1D ordering constrained/unconstrained version
Minimize
Pick a window of variables , fix all variables at Find a correctionto Update the window’s variables, restore volume
constraints and revise around the window
Switch to the next window chosen with overlap Use a (small) sequence of variable size windows For example use windows with 5,10,15,20,25 nodes
ij
Pjiij xxax ||)(
Wi x~
i Wixi ,~
![Page 23: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/23.jpg)
Easy to solve problems Quadratic functional and linear constraintsSolve a linear system of equationsQuadratization of the functional: P=1, P>2
![Page 24: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/24.jpg)
Quadratization for P=1 and P>2
Minimize ;
Given a current approximation
Minimize
Minimize
ij
jiij xxax ||)(
ij
ji
ji
ij
xxxx
ax 2)(
|~~|)(ˆ
ix~
ij
jiij xxax 6)()(
ij
ji
ji
ij
xxxx
ax 2
4)(
)~~()(ˆ
![Page 25: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/25.jpg)
Easy to solve problems Quadratic functional and linear constraintsSolve a linear system of equationsQuadratization of the functional: P=1, P>2Linearization of the constraints: P=2
![Page 26: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/26.jpg)
Window relaxation for P=2 constrained version
To prevent nodes from collapsing on each other To express the aim of having an
approximate permutation of add 2 constraints:
The terms were neglected assuming they are small enough compared with other terms in the equation
Wiiix }~{
2,1,~)~(
mvxvx i
m
Wiii
mi
Wii
Wiix }~{
0~2
01
Wiiii
Wiii
xvm
vm
2
![Page 27: Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d3a5503460f94a14a11/html5/thumbnails/27.jpg)
Easy to solve problems Quadratic functional and linear constraintsSolve a linear system of equationsQuadratization of the functional: P=1, P>2Linearization of the constraints: P=2
Inequality constraints: active set method