Neural Programs: Towards Adaptive Control in Cyber ...
Transcript of Neural Programs: Towards Adaptive Control in Cyber ...
![Page 1: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/1.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Neural Programs:Towards Adaptive Control in Cyber-Physical Systems
Konstantin Selyunin1, Denise Ratasich1
Ezio Bartocci1, M.A. Islam2
Scott A. Smolka2, Radu Grosu1
1Vienna University of TechnologyCyber-Physical Systems Group E182-1
Institut of Computer Engineering
2Stony Brook University, NY, USA
![Page 2: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/2.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Motivation
• Programs are not robust
• Case studies: neural circuit simulation & parallel parking
• Parameter synthesis: plateaus are bad for optimizations0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16
Time, s
0.030
0.028
0.026
0.024
0.022
0.020
0.018
0.016
Pote
ntia
l, V
AVAAVB
![Page 3: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/3.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Motivation II
This presentation:
• How to incorporate “smooth” decisions in CPS to makesystems more robust using neural circuits and GBN
• Technique to learn parameters of a model
• Application to two case studies and the relation between them
![Page 4: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/4.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Background
• Bayesian Networks• express probabilistic dependencies between variables• are represented as DAGs• allow compact representation using CPDs
• Gaussian Distributions• Univariate and Multivariate Gaussian distributions
• Step function vs. sigmoid
D I
G
L
S
![Page 5: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/5.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Background
• Bayesian Networks• express probabilistic dependencies between variables• are represented as DAGs• allow compact representation using CPDs
• Gaussian Distributions• Univariate and Multivariate Gaussian distributions
• Step function vs. sigmoid
D I
G
L
S
![Page 6: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/6.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Background
• Bayesian Networks• express probabilistic dependencies between variables• are represented as DAGs• allow compact representation using CPDs
• Gaussian Distributions• Univariate and Multivariate Gaussian distributions
• Step function vs. sigmoid
D I
G
L
S
![Page 7: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/7.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Background
• Passing random variables through conditions
if( q > 0.15) { q;}
if( p <= 3.0) { p;}
q ~ N(0 , 0.5)
p ~ N(4 , 0.7)
![Page 8: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/8.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Towards the nif statement
Our setting:
• Program operates random variables (RVs)
• RVs are mutually dependent Gaussians
Questions:
• How to incorporate uncertainty of making a decision andmake decisions “smooth”?
• How to avoid cutting distributions when passing a variablethrough a condition or a loop?
We propose to use nifs instead of traditional if statements.
![Page 9: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/9.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Towards the nif statement
Our setting:
• Program operates random variables (RVs)
• RVs are mutually dependent Gaussians
Questions:
• How to incorporate uncertainty of making a decision andmake decisions “smooth”?
• How to avoid cutting distributions when passing a variablethrough a condition or a loop?
We propose to use nifs instead of traditional if statements.
![Page 10: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/10.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Towards the nif statement
Our setting:
• Program operates random variables (RVs)
• RVs are mutually dependent Gaussians
Questions:
• How to incorporate uncertainty of making a decision andmake decisions “smooth”?
• How to avoid cutting distributions when passing a variablethrough a condition or a loop?
We propose to use nifs instead of traditional if statements.
![Page 11: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/11.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Neural if
The nif statement: nif( x # y, σ2 )
• Inequality relation {≥, >,<,≤}• Variance (represents our confidence of making a decision)
Example:
nif( x >= a, σ2) S1 else S2
![Page 12: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/12.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
nif( x # a, σ2 ) : Evaluation
1. Compute the difference between x, a
diff(x,a) =
x - a− ε if # is >,
x - a if # is ≥,a - x− ε if # is <,
a - x if # is ≤ .
2. Compute quantiles of the probability density function
q1(0,4)
q1(0,0.4) q2(0,0.4)q1(0, ) q2(0, )
q2(0,4)
0.0 0.0-5 0 5 -5 0 5
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
di�(x,a)
di�(1,0)
x
* **
3. Check if a random sample is within the interval
![Page 13: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/13.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
nif( x # a, σ2 ) : Evaluation
1. Compute the difference between x, a
diff(x,a) =
x - a− ε if # is >,
x - a if # is ≥,a - x− ε if # is <,
a - x if # is ≤ .
2. Compute quantiles of the probability density function
q1(0,4)
q1(0,0.4) q2(0,0.4)q1(0, ) q2(0, )
q2(0,4)
0.0 0.0-5 0 5 -5 0 5
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
di�(x,a)
di�(1,0)
x
* **
3. Check if a random sample is within the interval
![Page 14: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/14.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
nif( x # a, σ2 ) : Evaluation
1. Compute the difference between x, a
diff(x,a) =
x - a− ε if # is >,
x - a if # is ≥,a - x− ε if # is <,
a - x if # is ≤ .
2. Compute quantiles of the probability density function
q1(0,4)
q1(0,0.4) q2(0,0.4)q1(0, ) q2(0, )
q2(0,4)
0.0 0.0-5 0 5 -5 0 5
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
di�(x,a)
di�(1,0)
x
* **
3. Check if a random sample is within the interval
![Page 15: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/15.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
nif: Example
if( x > 0.15) { x;}
nif( x > 0.15, 0.1) { x;}
x ~ N(0 , 0.1)
x
No. o
f sam
ples
0.0 0.2-0.2-0.4 0.4
x
x0.0 0.2-0.2-0.4 0.4
0.0 0.2-0.2-0.4 0.4No
. of s
ampl
esNo
. of s
ampl
es
![Page 16: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/16.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Limit case σ2 → 0
• For the case with “no uncertainty” (σ2 → 0) the PDF isexpressed as the Dirac function:
• δ(x) = +∞ if x = 0 else 0•∫∞−∞ δ(x) dx = 1
• σ2 → 0 : the nif statement is equivalent to the if condition
nif( x >= a, σ2) S1 else S2
![Page 17: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/17.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Limit case σ2 → 0
• For the case with “no uncertainty” (σ2 → 0) the PDF isexpressed as the Dirac function:
• δ(x) = +∞ if x = 0 else 0•∫∞−∞ δ(x) dx = 1
• σ2 → 0 : the nif statement is equivalent to the if condition
nif( x >= a, σ2) S1 else S2
![Page 18: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/18.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
nwhile
Extension of a traditional while statement that incorporatesuncertainty
nwhile( x # a, σ2){P1}
Evaluation:
1. Compute diff(x,a), obtain quantiles q1 and q2
2. Check if a random sample is within the interval
3. If sample within the interval, execute P1 and go to 1, else exit
![Page 19: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/19.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
nwhile
Extension of a traditional while statement that incorporatesuncertainty
nwhile( x # a, σ2){P1}
Evaluation:
1. Compute diff(x,a), obtain quantiles q1 and q2
2. Check if a random sample is within the interval
3. If sample within the interval, execute P1 and go to 1, else exit
![Page 20: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/20.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Case study 1: C.elegans
C.elegans
• a 1-mm round worm
• each adult individual has exactly 302 neurons
• extensively studied in evolutional- and neurobiology
Tap withdrawal response
• apply stimulus to mechanosensory (input) neurons
• observe the behavior: forward / backward movement
Goal
• express the behavior using neural program
![Page 21: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/21.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Case study 1: C.elegans
C.elegans
• a 1-mm round worm
• each adult individual has exactly 302 neurons
• extensively studied in evolutional- and neurobiology
Tap withdrawal response
• apply stimulus to mechanosensory (input) neurons
• observe the behavior: forward / backward movement
Goal
• express the behavior using neural program
![Page 22: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/22.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Case study 1: C.elegans
C.elegans
• a 1-mm round worm
• each adult individual has exactly 302 neurons
• extensively studied in evolutional- and neurobiology
Tap withdrawal response
• apply stimulus to mechanosensory (input) neurons
• observe the behavior: forward / backward movement
Goal
• express the behavior using neural program
![Page 23: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/23.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Neural connections 101
V(i)
V(i) ggap
wgapV(j)
V(j)
V(i) V(j)
V(i)
E(ij)
gsyn(V(i))
wsyn
V(j)
Synaptic connection
• chemical nature
• either active or not
• synaptic weight wsyn
• use nif to model eachsynaptic connection
Gap junction connection
• instantaneous resistive connection
• linear combination of inputs
• gap junction weight wgap
![Page 24: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/24.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
C.elegans Tap withdrawal circuit
REV
FWD
2 2
AVM
PLM
Why not as simple as this?
56
REV
28
FWD
AVB
AVA
2 2
PVC
AVD
AVM
PLM
Why gap junc>ons AVM-‐AVD and PLM-‐PVC?
Why the synapses and neurons AVD, PVC?
What is a gap junc>on from the point of view of control?
56
REV
10 28
FWD
4
AVB
AVA
2
1
2
PVC
8
1 AVD
AVM
PLM
What is the role of inhibitory junc>ons such as PLM-‐AVA?
56
REV
10 28
FWD
4
6 2
AVB
10 17
21
AVA 4
2
13
1
2
PVC
8
1 AVD
1 3
AVM
PLM
Why is the en>re circuit so complicated?
Why both synapses and gap junc>ons between PVC-‐AVA?
PLM
REV FWD
PVD ALM AVM
AVA
DVA
AVB
PVC
21
2
1
12
55 51
1
2827
9
10
12
227
28108
1
44
14
27270
AVD
![Page 25: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/25.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
C.elegans Tap withdrawal circuit
REV
FWD
2 2
AVM
PLM
Why not as simple as this?
56
REV
28
FWD
AVB
AVA
2 2
PVC
AVD
AVM
PLM
Why gap junc>ons AVM-‐AVD and PLM-‐PVC?
Why the synapses and neurons AVD, PVC?
What is a gap junc>on from the point of view of control?
56
REV
10 28
FWD
4
AVB
AVA
2
1
2
PVC
8
1 AVD
AVM
PLM
What is the role of inhibitory junc>ons such as PLM-‐AVA?
56
REV
10 28
FWD
4
6 2
AVB
10 17
21
AVA 4
2
13
1
2
PVC
8
1 AVD
1 3
AVM
PLM
Why is the en>re circuit so complicated?
Why both synapses and gap junc>ons between PVC-‐AVA?
PLM
REV FWD
PVD ALM AVM
AVA
DVA
AVB
PVC
21
2
1
12
55 51
1
2827
9
10
12
227
28108
1
44
14
27270
AVD
![Page 26: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/26.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
C.elegans Tap withdrawal circuit
REV
FWD
2 2
AVM
PLM
Why not as simple as this?
56
REV
28
FWD
AVB
AVA
2 2
PVC
AVD
AVM
PLM
Why gap junc>ons AVM-‐AVD and PLM-‐PVC?
Why the synapses and neurons AVD, PVC?
What is a gap junc>on from the point of view of control?
56
REV
10 28
FWD
4
AVB
AVA
2
1
2
PVC
8
1 AVD
AVM
PLM
What is the role of inhibitory junc>ons such as PLM-‐AVA?
56
REV
10 28
FWD
4
6 2
AVB
10 17
21
AVA 4
2
13
1
2
PVC
8
1 AVD
1 3
AVM
PLM
Why is the en>re circuit so complicated?
Why both synapses and gap junc>ons between PVC-‐AVA?
PLM
REV FWD
PVD ALM AVM
AVA
DVA
AVB
PVC
21
2
1
12
55 51
1
2827
9
10
12
227
28108
1
44
14
27270
AVD
![Page 27: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/27.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
C.elegans Tap withdrawal circuit
REV
FWD
2 2
AVM
PLM
Why not as simple as this?
56
REV
28
FWD
AVB
AVA
2 2
PVC
AVD
AVM
PLM
Why gap junc>ons AVM-‐AVD and PLM-‐PVC?
Why the synapses and neurons AVD, PVC?
What is a gap junc>on from the point of view of control?
56
REV
10 28
FWD
4
AVB
AVA
2
1
2
PVC
8
1 AVD
AVM
PLM
What is the role of inhibitory junc>ons such as PLM-‐AVA?
56
REV
10 28
FWD
4
6 2
AVB
10 17
21
AVA 4
2
13
1
2
PVC
8
1 AVD
1 3
AVM
PLM
Why is the en>re circuit so complicated?
Why both synapses and gap junc>ons between PVC-‐AVA?
PLM
REV FWD
PVD ALM AVM
AVA
DVA
AVB
PVC
21
2
1
12
55 51
1
2827
9
10
12
227
28108
1
44
14
27270
AVD
![Page 28: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/28.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
C.elegans Tap withdrawal circuit
REV
FWD
2 2
AVM
PLM
Why not as simple as this?
56
REV
28
FWD
AVB
AVA
2 2
PVC
AVD
AVM
PLM
Why gap junc>ons AVM-‐AVD and PLM-‐PVC?
Why the synapses and neurons AVD, PVC?
What is a gap junc>on from the point of view of control?
56
REV
10 28
FWD
4
AVB
AVA
2
1
2
PVC
8
1 AVD
AVM
PLM
What is the role of inhibitory junc>ons such as PLM-‐AVA?
56
REV
10 28
FWD
4
6 2
AVB
10 17
21
AVA 4
2
13
1
2
PVC
8
1 AVD
1 3
AVM
PLM
Why is the en>re circuit so complicated?
Why both synapses and gap junc>ons between PVC-‐AVA?
PLM
REV FWD
PVD ALM AVM
AVA
DVA
AVB
PVC
21
2
1
12
55 51
1
2827
9
10
12
227
28108
1
44
14
27270
AVD
![Page 29: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/29.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Tap Withdrawal Simulations as Neural Program
Biological Model Neural Program
dV (i)
dt=
VLeak − V (i)
R(i)m C
(i)m
+
∑Nj=1(I
(ij)syn + I
(ij)gap) + I
(i)stim
C(i)m
(1)
I(ij)gap = w
(ij)gap g
(ij)gap (Vj − Vi ) (2)
I(ij)syn = w
(ij)syn g
(ij)syn (E (ij) − V (j)) (3)
g(ij)syn (V (j)) =
gsyn
1 + eK
(V (j)−Veqj
Vrange
) (4)
1: nwhile ( t ≤ tdur , 0)
2: compute I(ij)gap using equation 2
3: nwhile ( k ≤ w(ij)syn , 0)
4: nif (V (j) ≤ Veq , K/Vrange)
5: g(ij)syn ← g
(ij)syn + gsyn
6: compute I(ij)syn using equation 3
7: compute dV (i) using equation 18: V (i) ← V (i) + dV (i)
9: t ← t + dt
![Page 30: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/30.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
C.elegans Tap withdrawal simulations
ODE Neural Program: One execution Neural Program: Average
![Page 31: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/31.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Case study 2: Parallel parking
Given:
• P3AT-SH Pioneer rover
• Carma Devkit
• ROS on Ubuntu 12.04
• Pi with Gertboard
Goal:Write a parallel parking controller as a neural program
Raspberry PI withaccelerometer and gyroscope
Carma Board runningUbuntu & ROS
Sonars
Bumbers
![Page 32: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/32.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Program skeleton
nwhile(currentDistance < targetLocation1 , sigma1){
moving ();
currentDistance = getPose ();
}
updateTargetLocations ();
nwhile(currentAngle < targetLocation2 , sigma2){
turning ();
currentAngle = getAngle ();
}
updateTargetLocations ();
nwhile(currentDistance < targetLocation3 , sigma3){
moving ();
currentDistance = getPose ();
}
...
Question: how to find the unknown parameters and how uncertainare we about each of them?
![Page 33: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/33.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Program skeleton
nwhile(currentDistance < targetLocation1 , sigma1){
moving ();
currentDistance = getPose ();
}
updateTargetLocations ();
nwhile(currentAngle < targetLocation2 , sigma2){
turning ();
currentAngle = getAngle ();
}
updateTargetLocations ();
nwhile(currentDistance < targetLocation3 , sigma3){
moving ();
currentDistance = getPose ();
}
...
Question: how to find the unknown parameters and how uncertainare we about each of them?
![Page 34: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/34.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Learning
Parking example:
• Sequence of moves and turns
• Each action depends on the previous one
• The dependence is probabilistic
• RVs are normally distributed (assumption)
Gaussian Bayesian Network:
l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76
![Page 35: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/35.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Learning
Parking example:
• Sequence of moves and turns
• Each action depends on the previous one
• The dependence is probabilistic
• RVs are normally distributed (assumption)
Gaussian Bayesian Network:
l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76
![Page 36: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/36.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Learning: Good traces
l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76
0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.61
0.5
0
0.5
Task: learn the parameters of the GBN from the good traces
1. Convert the GBN to the MGD[HG95]
2. Update the precision matrix T of the MGD[Nea03]
3. Extract σ2s and bijs from T
![Page 37: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/37.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Learning: Good traces
l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76
0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.61
0.5
0
0.5
Task: learn the parameters of the GBN from the good traces
1. Convert the GBN to the MGD[HG95]
2. Update the precision matrix T of the MGD[Nea03]
3. Extract σ2s and bijs from T
![Page 38: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/38.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Learning: Update step
• Iterative learning procedure• Incrementally update mean µ and covariance matrix β of the
prior• Mean update:
x =
∑Mh=1 x
(h)
M
µ∗ =vµ+ Mx
v + M
• Covariance matrix update:
s =M∑h=1
(x(h) − x
)(x(h) − x
)Tβ∗ = β + s +
vM
v + M
(x(h) − x
)(x(h) − x
)T(T∗)−1 ∼ β∗
![Page 39: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/39.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Learning: Update step
• Iterative learning procedure• Incrementally update mean µ and covariance matrix β of the
prior• Mean update:
x =
∑Mh=1 x
(h)
M
µ∗ =vµ+ Mx
v + M
• Covariance matrix update:
s =M∑h=1
(x(h) − x
)(x(h) − x
)Tβ∗ = β + s +
vM
v + M
(x(h) − x
)(x(h) − x
)T(T∗)−1 ∼ β∗
![Page 40: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/40.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Learning: Data
“Good trajectories”:
• *.bag files (collection of messagesthat are broadcasted in ROS)
• extract coordinates in the 2-Dspace and angle
• find important points
• obtain samples in the form:l1, α1, l2, α2, l3, α3, l4
.
.
.
.
.
.
.
.
.
1.204 -0.911 -1.221
1.207 -0.920 -1.221
1.209 -0.927 -1.221
1.211 -0.930 -1.221
1.211 -0.931 -1.221
1.211 -0.931 -1.221
1.211 -0.931 -1.221
1.211 -0.931 -1.215
.
.
.
.
.
.
.
.
.
![Page 41: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/41.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Learning: Data
“Good trajectories”:
• *.bag files (collection of messagesthat are broadcasted in ROS)
• extract coordinates in the 2-Dspace and angle
• find important points
• obtain samples in the form:l1, α1, l2, α2, l3, α3, l4
.
.
.
.
.
.
.
.
.
1.204 -0.911 -1.221
1.207 -0.920 -1.221
1.209 -0.927 -1.221
1.211 -0.930 -1.221
1.211 -0.931 -1.221
1.211 -0.931 -1.221
1.211 -0.931 -1.221
1.211 -0.931 -1.215
.
.
.
.
.
.
.
.
.
![Page 42: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/42.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Learning: Data
“Good trajectories”:
• *.bag files (collection of messagesthat are broadcasted in ROS)
• extract coordinates in the 2-Dspace and angle
• find important points
• obtain samples in the form:l1, α1, l2, α2, l3, α3, l4
.
.
.
.
.
.
.
.
.
1.204 -0.911 -1.221
1.207 -0.920 -1.221
1.209 -0.927 -1.221
1.211 -0.930 -1.221
1.211 -0.931 -1.221
1.211 -0.931 -1.221
1.211 -0.931 -1.221
1.211 -0.931 -1.215
.
.
.
.
.
.
.
.
.
![Page 43: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/43.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Learning: Data
“Good trajectories”:
• *.bag files (collection of messagesthat are broadcasted in ROS)
• extract coordinates in the 2-Dspace and angle
• find important points
• obtain samples in the form:l1, α1, l2, α2, l3, α3, l4
.
.
.
.
.
.
.
.
.
1.204 -0.911 -1.221
1.207 -0.920 -1.221
1.209 -0.927 -1.221
1.211 -0.930 -1.221
1.211 -0.931 -1.221
1.211 -0.931 -1.221
1.211 -0.931 -1.221
1.211 -0.931 -1.215
.
.
.
.
.
.
.
.
.
![Page 44: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/44.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Parking system architecture
Engine
nwhile ( . ) moving();nwhile ( . ) turning();...
initial motioncommands
velocitycommands
vl, vr, a, ω
x, y, θ
actualposition
resampledcommands
GBN(distributions)
Rover Interface
SensorFusion
![Page 45: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/45.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Conclusion and Future Work
Recap:
• use of smooth Probit distribution in conditional and loopstatements
• use of Gaussian Bayesian Network to capture dependenciesbetween Probit distributions
• Case studies: robust parking controller and tap withdrawalsimulation
Future work:
• Apply these techniques to monitoring
![Page 46: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/46.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
References I
David Heckerman and Dan Geiger.Learning bayesian networks: A unification for discrete andgaussian domains.In UAI, pages 274–284, 1995.
Richard E. Neapolitan.Learning Bayesian Networks.Prentice-Hall, Inc., Upper Saddle River, NJ, USA, 2003.
![Page 47: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/47.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
![Page 48: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/48.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Learning Parameters in a Neural Program
Neural Program with Uncertain Parameters
Set of traces
Samplingfrom GBN
with learnedparameters
Learningprocedure
Learning phase
Execution phase
LearnedParameters
![Page 49: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/49.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Integration into ROS
• Rover Interface
• Sensor Fusion
• GBN and Engine
![Page 50: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/50.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Denotational Semantics
E ::= xi | c | bop(E1,E2) | uop(E1)
S ::= skip | xi := E | S1;S2 | nif( xi # c , σ2 ) S1 else S2 |nwhile( xi # c , σ2){ S1 }
![Page 51: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/51.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Denotational Semantics
JskipK(x) = x
Jxi := E K(x) = x[JEK(x) 7→ xi ]
J S1;S2 K(x) = JS2K(JS1K(x))
Jnif( xi # c , σ2) S1 else S2K(x) =
Jcheck(xi , a, σ2,#)K(x)JS1K(x) +
J¬check(xi , a, σ2,#)K(x)JS2K(x)
Jnwhile( xi # c, σ2){ S1 }K(x) =
xJ¬check(xi , a, σ2,# )K(x) +
Jcheck(xi , a, σ2,# )K(x)Jnwhile( xi # c , σ2){ S1 }K(JS1Kx)
![Page 52: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/52.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Learning: Conversion step
1. Define an ordering starting from initial node
t1 =1
σ21
bi =
bi1
...
bi,i−1
µ =
µ1
...
µn
2. Use iterative algorithm from[Heckerman and Geiger, 1995
]:
T1 = (t1);
for(i = 2; i ≤ n; i + +)
Ti =
Ti−1 + tibibTi −tibi
−tibTi ti
;
T = Tn;
l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76
![Page 53: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/53.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Learning: Conversion step
1. Define an ordering starting from initial node
t1 =1
σ21
bi =
bi1
...
bi,i−1
µ =
µ1
...
µn
2. Use iterative algorithm from
[Heckerman and Geiger, 1995
]:
T1 = (t1);
for(i = 2; i ≤ n; i + +)
Ti =
Ti−1 + tibibTi −tibi
−tibTi ti
;
T = Tn;
l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76
![Page 54: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/54.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Learning: Conversion step
• l1, α1:
T1 = (t1) =1
σ21
;
T2 =
1σ2
1+
b221
σ22−b21
σ22
−b21
σ22
1σ2
2
b2 =
(b21
)
l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76
![Page 55: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/55.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Learning: Conversion step
• l2:
T3 =
1σ2
1+
b221
σ22
−b21
σ22
0
−b21
σ22
1σ2
2+
b232
σ23−b32
σ23
0 −b32
σ23
1σ2
3
b3 =
0
b21
l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76
![Page 56: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/56.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Learning: Conversion step
• α2:
T4 =
1σ2
1+
b221
σ22
−b21
σ22
0 0
−b21
σ22
1σ2
2+
b232
σ23
−b32
σ23
0
0 −b32
σ23
1σ2
3+
b243
σ24−b43
σ24
0 0 −b43
σ24
1σ2
4
b4 =
0
0
b43
l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76
![Page 57: Neural Programs: Towards Adaptive Control in Cyber ...](https://reader030.fdocuments.us/reader030/viewer/2022012021/61689d41d394e9041f7126e4/html5/thumbnails/57.jpg)
Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion
Learning: Conversion step
• l3:
T5 =
1σ2
1+
b221
σ22
−b21
σ22
0 0 0
−b21
σ22
1σ2
2+
b232
σ23
−b32
σ23
0 0
0 −b32
σ23
1σ2
3+
b243
σ24
−b43
σ24
0
0 0 −b43
σ24
1σ2
4+
b254
σ25−b54
σ25
0 0 0 −b54
σ25
1σ2
5
b5 =
0
0
0
b54
We can generalize T for arbitrary number of moves
l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76