Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based...
Transcript of Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based...
![Page 1: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/1.jpg)
Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection
Md. Shahanur Alam, B. Rasitha Fernando, Yassine Jaoudi, Chris Yakopcic, Raqibul Hasan, Tarek M. Taha, and Guru Subramanyam
Dept. Of Electrical and Computer Engineering, University of Dayton, Dayton, OH, USA
![Page 2: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/2.jpg)
M. S. Alam et. al. 2
• Introduction
• Anomaly Detection Methods and Applications
• Motivation and Challenges
• Proposed Anomaly Detection System
• Results of Intrusion and Anomaly Detection System
• Summary
• Future work
Outline
![Page 3: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/3.jpg)
M. S. Alam et. al. 3
Introduction
• Network Intrusion
• Intrusion Detection system
• SNORT
• What if new unknown packet comes?
E.g. ‘Zero Day’
Neural Network
SNO
RTRouter Positive
Negative
Positive + Zero Day
Block diagram of the neural network-based intrusion detection system
NormalAnomaly
![Page 4: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/4.jpg)
M. S. Alam et. al. 4
Introduction (Contd.)
• Memristive system could be a solution
Neural Network Vs Power Consumption
IoTs and Edge Devices
𝑀(𝑞) =𝑑𝜙
𝑑𝑞
≈200W
Memristor
![Page 5: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/5.jpg)
M. S. Alam et. al.
What are the anomalies?
𝐷2𝐷1
𝐷3
𝑁1
𝑁2
𝑋
𝑌
Illustration of anomalies in two-dimensional data set
• Abnormalities/outliers
Anomaly detection Methods:
• Unsupervised (AE, GAN, RNN, LSTM etc)
• Supervised (DNN, CNN)
• Hybrid model (AE+SVM)
• One-Class Neural Network
Applications:
• Cyber-Intrusion Detection
• Malware Detection
• Internet of Things (IoTs) Big Data Anomaly Detection
• Fraud Detection
• Medical Anomaly Detection
• Industrial Damage Detection
Anomaly Detection Methods and Applications
5
![Page 6: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/6.jpg)
M. S. Alam et. al.
Motivation and Challenges
Motivation:
• Neural Network implementation for IoTs and edge devices
• Detection of anomalies in real-time
Challenges:
• Boundary between normal and malicious is not explicitly defined
• Continual learning and the catastrophic forgetting
6
![Page 7: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/7.jpg)
M. S. Alam et. al.
Dataset Preprocessing
7
• NSL-KDD network dataset KDD Cup’99 dataset
• Training data has125,973 packets, 23 different data types
• 43 attributes, consists numerical and alphanumeric data
• Preprocessed and sorted out the packets
• Network is pretrained with 90% of Normal
• Tested with 10% normal and 10% of total malicious data
0,tcp,ftp_data,SF,491,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,2,0,0,
0,0,1,0,0,150,25,0.17,0.03,0.17,0,0,0,0.05,0,normal,20
0,tcp,ftp_data,SF,334,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,2,2,0,0,
0,0,1,0,0,2,20,1,0,1,0.20,0,0,0,0, warezclient,15
0,0.5,0.188,0.629,3.55𝑒−7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.003
91,0.00391,0,0,0,0,1,0,0,0.588,0.098,0.17,0.03,0.17,0,0,0,0.05
,0,0,0.9523
0,0.5,0.188,0.629,2.42𝑒−7,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0.003
91,0.0039,0,0,0,0,1,0,0,0.0078,0.078,1,0,1,0.2,0,0,0,0,1,0.714
Normal Packet
Malicious Packet
Preprocessed Malicious Packet
Preprocessed Normal Packet
![Page 8: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/8.jpg)
M. S. Alam et. al.
Positive
Normal Data
Malicious Data
AE-2:Real-Time Training
Known
Unknown
AE-1: Pretrained Section
Router
SNORT
1234
Positive
Negative
Enterprise Network
Positive=Normal + ‘zero day’ packets
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
System Prototype Model Autoencoder (AE) Neural Network
Intrusion And Anomaly Detection System with AE neural Network
Proposed Anomaly Detection System
8
• AE learns to regenerate the input data at output• AE can reduce the dimension of input data
. . .
. . .
x1
x2
xi
x41
. . .
. . .
h1,3
h1,4
h1,j
h1,90
h1,1
h1,2
. . .
. . .
x'1
x'2
x'i
x'41
. . .
. . .
h3,3
h3,4
h3,j
h3,90
h3,1
h3,2
. . .
. . .
h2,1
h2,k
h2,10
w'1(j,i)w2(j,k)
w'2(k,j)w1(i,j)
41→90→10→90→41
Encoder Decoder
Bottle Neck
![Page 9: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/9.jpg)
M. S. Alam et. al.
𝑔 𝑥 = ቐ1, 𝑥 > 20.25𝑥 + 0.5, 𝑥 ≤ 20, 𝑥 < 2
(3)
𝑓 𝑥 =1
1+𝑒−𝑥(2)
𝐷𝑃𝑗 = σ𝑖=1𝑁+1 𝑥𝑖 × 𝜎𝑖𝑗
+ − 𝜎𝑖𝑗− (1)
DOT Product:
(b)
. . . xN+1
x1
x2
. . .
xN=
yM
A1
AB
β
yj
+ -
+ -
+A−A
Memristor
C
Synapse
RRf
R
A2
A3
AM
y3y2y1
. . .
x3
Memristor Crossbar Circuits
(c)
Sigmoid Approximation:
Memristive Neural Network and Crossbar Circuit
9Ideal and approximate Sigmoid Function
(a) Single Neuron
Synapse
![Page 10: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/10.jpg)
M. S. Alam et. al. 10
Training of the Network
• apply 𝑥𝑖
• crossbar computes the dot product 𝐷𝑃𝑗
• output signal 𝑦𝑗
• error : 𝛿𝑗 = 𝑥𝑖 − 𝑦𝑗 𝑓′ 𝐷𝑃𝑗
• backpropagate the error 𝛿𝑗 = σ𝑘 𝛿𝑘 𝑤𝑘,𝑗𝑓′ 𝐷𝑃𝑗 in each hidden layer
• update the weights according 𝛿𝑗 as Δ𝑤𝑗 = 𝜂𝛿𝑗𝑥
• calculate 𝐷𝑚= 1
𝑁σ(𝑋𝑖 − 𝑌𝑗)
2 and 𝐷𝑆𝐷 =σ(𝐷−𝐷𝑚)
2
𝑁
![Page 11: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/11.jpg)
M. S. Alam et. al.
𝒆 = 𝒆𝟎 +σ(𝑿′𝒊 − 𝒀𝒊)
𝟐 D=
σ(𝑿′𝒊 − 𝒀𝒊)𝟐
∆= 𝑫 − 𝑫𝒎
For
∆> 𝑫𝑺𝑫, 𝑳 = 𝟏&
∆< 𝑫𝑺𝑫, 𝑳 = 𝟎
𝑳 = 𝟏/0
?
AE-1
Forward
YData (𝑿’)
𝒆′ = 𝒆𝟎 +σ(𝑿′𝒊 − 𝒀′𝒊)
𝟐∆𝟏= 𝑫′ − 𝑫′𝒎
For
∆𝟏> 𝑫𝑺𝑫𝟏, 𝒖𝒏𝒌𝒏𝒐𝒘𝒏 &
∆𝟏< 𝑫𝑺𝑫𝟏, 𝒌𝒏𝒐𝒘𝒏
AE-2
Forward
Y’
Flowchart of Real-time Anomaly detection System
Anomaly Detection System
System Flowchart of Anomaly Detection System
11
Positive
Normal Data
Malicious Data
AE-2:Real-Time Training
Known
Unknown
AE-1: Pretrained Section
Router
SNORT
1234
Positive
Negative
Enterprise Network
Positive=Normal + ‘zero day’ packets
. .
..
. .
. .
..
. .
. .
..
. .
. .
..
. .
. .
..
. .
. .
..
. .
. .
..
. .
. .
..
. .
. .
..
. .
. .
..
. .
Unknown
?
Update Weight of AE-2
![Page 12: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/12.jpg)
M. S. Alam et. al.
Pretraining of Autoencoder-1 (AE-1)
12
Input feature and regenerated feature of a sample through (AE-1)
a. b.
Training Error (MSE) in software and memristor X-bar
![Page 13: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/13.jpg)
M. S. Alam et. al.
a. b.
13
Intrusion detection Accuracy (AE-1)
Intrusion Detection Accuracy
Pretraining Epochs Global Accuracy 𝑵𝑴𝑵 𝑵𝑵𝑴 𝑵𝑭 Case
50 95.22% 56 546 602 Software
50 92.91% 65 868 933 Memristor
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 =𝑁𝑠−𝑁𝐹
𝑁𝑠× 100%
False Detection (Malicious + Normal)
![Page 14: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/14.jpg)
M. S. Alam et. al.14
Intrusion Detection Accuracy (contd.)
a. b.
Malicious Packet Vs Epochs Malicious Packet Detection Accuracy Vs Epochs
![Page 15: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/15.jpg)
M. S. Alam et. al. 2/23
Anomaly Detection in real-time
𝑇1 = 𝑥11 , 𝑥2
1, 𝑥12 , 𝑥2
2, 𝑥13 , 𝑥2
3 , …𝑇2 = 𝑥1
1 , 𝑥21 , 𝑥3
1 , 𝑥12 , 𝑥2
2 , 𝑥32 , …
𝑇3 = 𝑥11 , 𝑥2
1 , 𝑥31 , 𝑥4
1 , 𝑥12 , 𝑥2
2 , 𝑥32 , 𝑥4
2, …𝑇4 = 𝑥1
1, 𝑥21, 𝑥3
1, 𝑥41, 𝑥5
1, 𝑥12, 𝑥2
2, 𝑥32, 𝑥4
2, 𝑥52, …
Real-Time Anomaly Detection:
Positive
Normal Data
Malicious Data
AE-2:Real-Time Training
Known
Unknown
AE-1: Pretrained Section
Router
SNORT
1234
Positive
Negative
Enterprise Network
Positive=Normal + ‘zero day’ packets
. .
..
. .
. .
..
. .
. .
..
. .
. .
..
. .
. .
..
. .
. .
..
. .
. .
..
. .
. .
..
. .
. .
..
. .
. .
..
. .
𝑥1 = 𝑛𝑜𝑟𝑚𝑎𝑙, 𝑥2 = 𝑛𝑒𝑝𝑡𝑢𝑛𝑒, 𝑥3 = 𝑠𝑎𝑡𝑎𝑛, 𝑥4=𝑖𝑝𝑠𝑤𝑒𝑒𝑝, 𝑥5 = 𝑏𝑎𝑐𝑘
Anomaly Detection System
Real-time learning and anomaly detection
![Page 16: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/16.jpg)
M. S. Alam et. al. 16
Power, Area and Timing Analysis
Parameter Training Data Recognition Data
Area (mm2) 0.00271 0.00271
Power (mW) 20.6 7.56
Time (µs)/sample 4.02 0.384
Energy (nJ)/One Sample 82 2.90
• 𝑅𝑜𝑓𝑓 = 1 × 107Ω, 𝑅𝑜𝑛 = 5 × 104 Ω
• Wire Resistance =5 Ω, 𝑉𝑚𝑒𝑚 = 1.3𝑣𝑜𝑙𝑡
• Transistor Feature Size : F= 45nm
• Op-amp power = 3 × 10−6 𝑤𝑎𝑡𝑡
• Transistor Size= 50𝐹2
• Memristor area= 1 × 104 𝑛𝑚2
![Page 17: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/17.jpg)
M. S. Alam et. al. 17
Summary
• Introduced the problem and proposed a possible solution
• Presented the Autoencoder with memristor X-bar and the functionalities
• Overall accuracy 92.91% with malicious packet detection accuracy 98.89%
• Presented real-time anomaly detection system
• Explained the power and energy requirement
![Page 18: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/18.jpg)
M. S. Alam et. al. 18
Current and future work
• Multiple autoencoders for intrusion and malware detection
• Incremental learning algorithm & unseen class detection
![Page 19: Memristor Based Autoencoder for Unsupervised Real-Time … · 2020-02-04 · Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur](https://reader030.fdocuments.us/reader030/viewer/2022040921/5e992b8a6f191f567b270686/html5/thumbnails/19.jpg)
M. S. Alam et. al. 19
THANK YOU
Questions?