Neural Networks for Solving Quadratic Assignment Problems
-
Upload
anirut-trakultritrung -
Category
Documents
-
view
40 -
download
1
description
Transcript of Neural Networks for Solving Quadratic Assignment Problems
![Page 1: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/1.jpg)
![Page 2: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/2.jpg)
Agenda Hopfield Neural Network Gaussian Machine Quadratic Assignment Problems How to Solve Problem Computation Results Conclusion
![Page 3: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/3.jpg)
Artificial Neural Network (ANN) An mathematic model Inspired by the biological nervous systems Acquires knowledge through learning ANN’s knowledge is stored within inter-neuron
connection strengths (Synaptic weights)
![Page 4: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/4.jpg)
Use continuous activation function Fully connected recurrent network Notion of energy Each state has an energy Computes recursively until a stable state
reached Converges to stable states
Hopfield Network Model:
Hopfield Neural Network
1
( ) ( ) ( )N
ii i j i i
j
du t u t T x t Idt =
= - + +å
0
( )1( ) ( ( )) 1 tanh2
ii i
u tx t f u ta
æ öæ ö= = +ç ÷ç ÷ç ÷è øè ø
Dynamic
Equation:
Output:
![Page 5: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/5.jpg)
Hopfield Neural Network
Activation Function
a0 parameter determines behavior of the gain function Higher ~ gentle Lower ~ steep
0
( )1( ) ( ( )) 1 tanh2
ii i
u tx t f u ta
æ öæ ö= = +ç ÷ç ÷ç ÷è øè ø
ui
xi =f(ui)
0
1
![Page 6: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/6.jpg)
Hopfield Network: Energy Function Quantification of current energy of the network Energy surface determines stable states Stable states are local minima
Each update converges to stable state Symmetric connections
2
1
0N
i
i
dxdEdt dt=
æ ö= - £ç ÷è ø
å
1 1 1
1( )2
N N N
i j i j i ii j i
E x T x x I x= = =
= - -åå å
, 0i j ji iiT T T= =
Energy
function:
Lyapunov
Condition:
![Page 7: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/7.jpg)
Gaussian Machine:An Improvement of Hopfield NNGaussian Machine Objective: Allow the system to escape from local minima Added Gaussian noises that its power vary in time Vary the activation function gain in time
,1
( ) ( ) ( ) ( )N
ii i j j i i
j
du t u t T x t I tdt =
= - + + +å
1( ) ( ( )) 1 tanh2 ( )i ix t f u tæ öæ ö
= = +ç ÷ç ÷ç ÷è øè ø
Dynamic
Equation:
Output:
![Page 8: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/8.jpg)
Gaussian Machine:An Improvement of Hopfield NNGaussian Machine Added Gaussian noises that its power vary in time
( ) (0, ( ))i it N th s=
Dynamic
Equation:Temperature
0 0.5 1 1.5 2 2.5 3-1.5
-1
-0.5
0
0.5
1
1.5
t
( )i th
,1
( ) ( ) ( ) ( )N
ii i j j i i
j
du t u t T x t I tdt =
= - + + +å
![Page 9: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/9.jpg)
Gaussian Machine:An Improvement of Hopfield NNGaussian Machine Vary the activation function gain in time
Output:
ui
xi =f(ui)1
1( ) ( ( )) 1 tanh2 ( )i ix t f u tæ öæ ö
= = +ç ÷ç ÷ç ÷è øè ø
![Page 10: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/10.jpg)
Quadratic Assignment Problems-QAP
12 23 13AB BC ACCost f d f d f d= + +
Assign N Facilities to N locations Minimum sum of product of
“flow between facilities” and
“distance between locations”
N! Possible Solutions
Computationally hard problem, grows exponentially
N = 12, 479001600 solutions
N = 20, 2432902008176640000 solutions
How to find a solution from this ocean ?
Quadratic Assignment Problems-QAP
![Page 11: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/11.jpg)
Problem Representation: ~ the distance between location k and l and ~ the traffic flow between facility i and j and
A solution: Permutation Matrix (NxN Matrix)
Quadratic Assignment Problems-QAP
[ ]klD d=
[ ]ijF f=
0 ,ii ij jif f f= =
[ ]ikP x=
0 ,kk kl lkd d d= =
1 if is assigned to 0 otherwise ik
i kx ì
= íî
![Page 12: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/12.jpg)
Objective: Assign N Facilities to N locations Minimize the total cost of assignment
The constraints
Quadratic Assignment Problems-QAP
1 1 1 1
1min2
N N N N
ij kl ik jlx i j k lC f d x x
= = = =
= åååå
1
1
1 for 1,...,
1 for 1,...,
[0,1] ,
N
ikiN
ikk
ik
x i N
x k N
x i k
=
=
= =
= =
Î "
å
å
: Only one location k is assigned in each facility i
: Only one facility i is assigned in each location k
: Output level boundary
Quadratic function QAP
![Page 13: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/13.jpg)
QAP Example
![Page 14: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/14.jpg)
Neural Network as QAP solver
Find a representation for the problem
Define a problem energy function
Derive T and I matrixes from the energy function
Construct the network using T and I matrixes
![Page 15: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/15.jpg)
Representation of QAP Permutation matrix represents an assignment Rows ~ Facilities Columns ~ Locations
12 23 34 45 51DA BC CB BE EDCost f d f d f d f d f d= + + + +
[ ]ikP x= =1 if is assigned to 0 otherwise ik
i kx ì
= íî
![Page 16: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/16.jpg)
Representation of QAP Use a neuron to represent each entry of the matrix ��If the entry is 1, neuron is on ( ≈ 1) ��If the entry is 0, neuron is off ( ≈ 0)
N-facilities problem represented using N2 neurons
ikx
ikx
![Page 17: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/17.jpg)
Hopfield Neural Network for QAP Arrange the neuron in a matrix form Neurons addressed with double indices
,1 1 1 1 1 1
1( )2
N N N N N N
ik jl ik jl ik iki j k l i k
E x T x x I x= = = = = =
= - -åååå åå
,1 1
( ) ( ) ( )N N
ikik ik jl jl ik
j l
du t u t T x t Idt = =
= - + +åå
0
( )1( ) ( ( )) 1 tanh2
ikik ik
u tx t f u ta
æ öæ ö= = +ç ÷ç ÷ç ÷è øè ø
Dynamic
Equation:
Output:
Energy
function:
![Page 18: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/18.jpg)
Gaussian Machine for QAP
Same notation as Hopfield network
,1 1 1 1 1 1
1( )2
N N N N N N
ik jl ik jl ik iki j k l i k
E x T x x I x= = = = = =
= - -åååå åå
Dynamic
Equation:
Output:
Energy
function:
( )1( ) ( ( )) 1 tanh2 ( )
ikik ik
u tx t f u tæ öæ ö
= = +ç ÷ç ÷ç ÷è øè ø
,1 1
( ) ( ) ( ) ( )N N
ikik ik jl jl ik ik
j l
du t u t T x t I tdt = =
= - + + +åå
![Page 19: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/19.jpg)
Neural Network as QAP solver
Find a representation for the problem
Define a problem energy function
Derive T and I matrixes from the energy function
Construct the network using T and I matrixes
![Page 20: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/20.jpg)
Energy Function for QAP Its minima must correspond to the valid solutions Shorter paths and flow must have lower energy So, break it down into
penalty cost( )E x E E= +
![Page 21: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/21.jpg)
Energy Function for QAP Constraint Satisfaction:
Cost:
2 2
penalty1 1 1 1 1 1
1 1 (1 )2 2 2
N N N N N N
ik ik ik iki k k i k i
A A CE x x x x= = = = = =
æ ö æ ö= - + - + -ç ÷ ç ÷
è ø è øå å å å åå
cost, 1 , 12
N N
ij kl ik jli j k l
BE f d x x= =
= åå
Only one “1” in each row Only one “1” in each column Output level close to “1”
![Page 22: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/22.jpg)
Neural Network as QAP solver
Find a representation for the problem
Define a problem energy function
Derive T and I matrixes from the energy function
Construct the network using T and I matrixes
![Page 23: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/23.jpg)
Mapping QAP onto Neural Network
Quadratic terms for T values Linear terms for I values
,, 1 , 1 , 1
1( )2
N N N
ik jl ik jl ik iki j k l i k
E x T x x I x= = =
= - -åå åQuadratic term Linear term
![Page 24: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/24.jpg)
Mapping QAP onto Neural Network QAP Energy Function
Network Energy Function
2 2
1 1 1 1 1 1
, 1 , 1
( ) 1 1 (1 )2 2 2
2
N N N N N N
ik ik ik iki k k i k i
N N
ij kl ik jli j k l
A A CE x x x x x
B f d x x
= = = = = =
= =
æ ö æ ö= - + - + -ç ÷ ç ÷è ø è ø
+
å å å å åå
åå
,, 1 , 1 , 1
1( )2
N N N
ik jl ik jl ik iki j k l i k
E x T x x I x= = =
= - -åå å
![Page 25: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/25.jpg)
, , 1 , , 1 , , , 1 , 1 , 1
1 1
( )2 2 2 2
2
N N N N N
ik jl ik jl ik jl ij kl ik jli k l k i j i j k l i j k l
N N
ik ik ikk i
A A C BE x x x x x x x f d x x
CA x A x x
= = = = =
= =
= + - +
- - -
å å å åå
å å
Mapping QAP onto Neural Network QAP Energy Function
Network Energy Function
,, 1 , 1 , 1
1( )2
N N N
ik jl ik jl ik iki j k l i k
E x T x x I x= = =
= - -åå å
![Page 26: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/26.jpg)
Mapping QAP onto Neural Network
Network Energy Function
( )E x T x x I x= - -
Linear term
![Page 27: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/27.jpg)
Mapping QAP onto Neural Network
Network Energy Function
Derived T and I matrices
( )E x T x x I x= - -
Linear term
, , , , ,
22
ik jl i j k l i j k l ij kl
ik
T A A C Bf dCI A
d d d d= - - + -
= +1 ,0 ,ik
i ji j
d=ì
= í ¹î
Kronecker Delta
![Page 28: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/28.jpg)
Neural Network as QAP solver
Find a representation for the problem
Define a problem energy function
Derive T and I matrixes from the energy function
Construct the network using T and I matrixes
![Page 29: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/29.jpg)
Agenda Hopfield Neural Network Gaussian Machine Quadratic Assignment Problems How to Solve Problem Computation Results Conclusion
![Page 30: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/30.jpg)
Computation Results
![Page 31: Neural Networks for Solving Quadratic Assignment Problems](https://reader036.fdocuments.us/reader036/viewer/2022082213/55cf995c550346d0339cfa21/html5/thumbnails/31.jpg)
Conclusion