2012 06-21-inauguration-labo-additive-manufacturing Sirris - Umberto Baraldi
Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for...
Transcript of Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for...
![Page 1: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/1.jpg)
Reservoir Computing Methods for
Prognostics and Health Management (PHM)
Piero Baraldi
Energy Department
Politecnico di Milano
Italy
![Page 2: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/2.jpg)
2 Piero Baraldi
Industry 4.0 2
20172012 Time
Da
ta
Available data
• Digitalization2.8 Trillion GD (ZD)
generated in 2016
• Analytics
AnalyticsData
![Page 3: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/3.jpg)
3 Piero Baraldi
Industry 4.0 3
20172012 Time
Da
ta
Available data
• Digitalization2.8 Trillion GD (ZD)
generated in 2016
• Analytics
AnalyticsData Predictive
Maintenance
![Page 4: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/4.jpg)
4 Piero Baraldi
4Predictive Maintenance 4
Monitored
Signals
Ambient &
Operating
Conditions
Prognostic
ModelIndustry 4.0
Remaining Useful Life
(RUL)
tp
𝑢1
𝑢2
𝑢𝑁
Analytics
Present time
Failure time
![Page 5: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/5.jpg)
5 Piero Baraldi
5Predictive Maintenance 5
• Future demand
• Logistics options
• …
Monitored
Signals
Ambient &
Operating
Conditions
Prognostic
Model
Optimal
Maintenance DecisionsSafety
improvement,
Cost Saving,
New Business
• Maximum Availability
• Business continuity
• Warehouse savings
• Zero-defect production
• Zero-waste production
Industry 4.0
Remaining Useful Life
(RUL)
tpPresent
timeFailure
time
𝑢1
𝑢2
𝑢𝑁
Analytics
![Page 6: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/6.jpg)
6 Piero Baraldi
In this Presentaton
• Prognostics
• Recurrent Neural Network (RNN)
• Reservoir Computing
• Echo State Network
• Application to the Prediction of Turbofan Engine RUL
6
![Page 7: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/7.jpg)
7
Prognostics: What is the Problem?
Aircraft Turbofan Engine
N Monitored Signals
• Signal 1
• Signal 2
• ………..
• Signal N
Time
Te
mp
era
ture
![Page 8: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/8.jpg)
8
Prognostics: What is the Problem?
Aircraft Turbofan Engine
N Monitored Signals
• Signal 1
• Signal 2
• ………..
• Signal N
Aircraft Engine
RUL Prediction
Time Time
RU
L
Te
mp
era
ture
Prognostic
Model
![Page 9: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/9.jpg)
9 Piero Baraldi
Prognostics: the Challenge 9
• 2 identical components
• Same measurements at
time 𝑡𝑝: 𝑢 𝑡𝑝 ≡ 𝑢(𝑡𝑝)
• System evolution depends on present and past signal values
(the memory of the history)
Monitored
Signal
𝑡𝑝
𝑢
𝑡𝑓0 𝑡𝑝
𝑡𝑝 𝑡𝑝 𝑡𝑓
𝑢
Monitored
Signal
![Page 10: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/10.jpg)
10
• Connection:
• Computational unit (neurons):
Prognostics: Methods
Feedforward Neural Network
𝑤11𝑢1
𝑢1𝑤11
𝑢1𝑤11
𝑢2𝑤21
𝑢3𝑤31
𝑓
𝑖=1
3
𝑢𝑖 𝑤𝑖1
𝑅𝑈𝐿 𝑡𝑝
![Page 11: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/11.jpg)
11
Prognostics: Methods
• connections only "from left
to right", no connection
cycle
• no memory
Feedforward Neural Network
𝑅𝑈𝐿 𝑡𝑝
![Page 12: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/12.jpg)
12
• connections only "from left
to right", no connection
cycle
• no memory
Prognostics: Methods
Feedforward Neural Network
• at least one connection
cycle
• activation can
"reverberate", persist even
with no input
• system with memory
Recurrent Neural Network
𝑅𝑈𝐿 𝑡𝑝𝑅𝑈𝐿 𝑡𝑝
![Page 13: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/13.jpg)
13 Piero Baraldi
In this presentaton
• Prognostics
• Recurrent Neural Network (RNN)
• Reservoir Computing
• Echo State Network
• Application to the Prediction of Turbofan Engine RUL
13
![Page 14: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/14.jpg)
14 Piero Baraldi
Recurrent NN: General Idea 14
𝑅𝑈𝐿 𝑡𝑝𝒖 1: 𝑡𝑝
𝑢1
𝑢𝑁Time
trajectory
𝑡 = 1
𝑡 = 𝑡𝑝
PROGNOSTIC MODEL
𝒖 1: 𝑡𝑝
![Page 15: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/15.jpg)
15 Piero Baraldi
Recurrent NN: General Idea 15
𝑥2
𝑥1
𝑥𝑀
Linear
Regression
Non Linear
Expansion
𝑅𝑈𝐿 𝑡𝑝 = 𝑾𝒐𝒖𝒕𝒙(𝑡𝑝)𝒖 1: 𝑡𝑝
𝑢1
𝑢𝑁Time
trajectory
𝑡 = 1
𝑡 = 𝑡𝑝
𝑟𝑢𝑙
𝑀 ≫ 𝑁
𝒙 𝑡𝑝 = 𝑓 𝒖 1: 𝑡𝑝
𝒙 𝑡𝑝 = 𝑓 𝒙(𝑡𝑝 − 1), 𝒖 𝑡𝑝
Recursive
definition
𝒖 1: 𝑡𝑝
𝒙 𝑡𝑝
𝑅𝑈𝐿 𝑡𝑝
![Page 16: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/16.jpg)
16 Piero Baraldi
Recurrent NN 16
𝒖 1: 𝑡𝑝 𝑥1(𝑡𝑝) = 𝑓
𝑖=1
𝑁
𝑤𝑖1𝑖𝑛 𝑢𝑖(𝑡𝑝) +
𝑖=1
𝑀
𝑤𝑖1 𝑥𝑖 (𝑡𝑝 − 1)
𝑾𝒊𝒏
𝑾Non Linear Expansion
𝑢3(𝑡𝑝)
𝑢2(𝑡𝑝)
𝑢1(𝑡𝑝)
![Page 17: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/17.jpg)
17 Piero Baraldi
Recurrent NN 17
𝒖 1: 𝑡𝑝 𝒙(𝑡𝑝) = 𝑓 𝑾𝒊𝒏𝒖(𝑡𝑝) +𝑾𝒙(𝑡𝑝 − 1) 𝑅𝑈𝐿 𝑡𝑝 = 𝑊𝑜𝑢𝑡𝒙(𝑡𝑝)
𝑾Linear RegresionNon Linear Expansion
𝑢3(𝑡𝑝)
𝑢2(𝑡𝑝)
𝑢1(𝑡𝑝)
𝑾𝒊𝒏
𝑾𝒐𝒖𝒕
𝑅𝑈𝐿 𝑡𝑝
![Page 18: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/18.jpg)
18
RNN: Training
𝑊𝑖𝑛,𝑊,𝑊𝑜𝑢𝑡
TRAINING SET
𝒖 1 , 𝑅𝑈𝐿𝐺𝑇 1 = 𝑡𝑓 − 1
…𝒖 5 , 𝑅𝑈𝐿𝐺𝑇 5 = 𝑡𝑓 − 5
…
𝒖 𝑡𝑓 − 1 , 𝑅𝑈𝐿𝐺𝑇 𝑡𝑓 − 1 = 1
Run-to-failure
degradation trajectory
t
𝒖
5 𝑡𝑓
𝒖(5)
𝑅𝑈𝐿𝐺𝑇 5 = 𝑡𝑓 − 5
![Page 19: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/19.jpg)
19
RNN: Training19
𝑊𝑖𝑛,𝑊,𝑊𝑜𝑢𝑡
Training Objective: minimize the error function
𝐸 𝑅𝑈𝐿, 𝑅𝑈𝐿𝐺𝑇 =RMSE=
𝑡=1
𝑡𝑓−11
𝑡𝑓 − 1𝑅𝑈𝐿(𝑡) − 𝑅𝑈𝐿𝐺𝑇(𝑡) 2
TRAINING SET
𝒖 1 , 𝑅𝑈𝐿𝐺𝑇 1 = 𝑡𝑓 − 1
𝒖 2 , 𝑅𝑈𝐿𝐺𝑇 2 = 𝑡𝑓 − 2
…
𝒖 𝑡𝑓 − 1 , 𝑅𝑈𝐿𝐺𝑇 𝑡𝑓 − 1 = 1
![Page 20: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/20.jpg)
20
RNN: Training20
𝑊𝑖𝑛,𝑊,𝑊𝑜𝑢𝑡
Training Objective: minimize the error function
𝐸 𝑅𝑈𝐿, 𝑅𝑈𝐿𝐺𝑇 =RMSE=
𝑡=1
𝑡𝑓−11
𝑡𝑓 − 1𝑅𝑈𝐿(𝑡) − 𝑅𝑈𝐿𝐺𝑇(𝑡) 2
TRAINING SET
𝒖 1 , 𝑅𝑈𝐿𝐺𝑇 1 = 𝑡𝑓 − 1
𝒖 2 , 𝑅𝑈𝐿𝐺𝑇 2 = 𝑡𝑓 − 2
…
𝒖 𝑡𝑓 − 1 , 𝑅𝑈𝐿𝐺𝑇 𝑡𝑓 − 1 = 1
Training Methods:
• Gradient-descent-based methods
• Reservoir Computing
![Page 21: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/21.jpg)
21
Gradient-descent-based methods for RNN
RNN are difficult to train using gradient-descent-based methods:
• Bifurcations
• Many updating cycles Too long training times
• Hard to obtain long range memory
21
𝑾𝒊𝒏 𝑾𝒐𝒖𝒕
𝑾
-
𝑅𝑈𝐿𝐺𝑇(𝑡)
𝑅𝑈𝐿(𝑡)
Error(t)
𝒖(𝑡)
![Page 22: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/22.jpg)
22 Piero Baraldi
In this presentaton
• Prognostics
• Recurrent Neural Network (RNN)
• Reservoir Computing
• Echo State Network
• Application to the Prediction of Turbofan Engine RUL
22
![Page 23: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/23.jpg)
23 Piero Baraldi
Reservoir Computing (RC): Terminology
What is it? Purpose
ReservoirNon-linear temporal
expansion function
Expand the input history 𝒖 1: 𝑡𝑝 into a rich-enough
reservoir space 𝒙(𝑡𝑝)
readout Linear function
Combine the neuron signals 𝒙(𝑡𝑝) into the desired output
signal target 𝑅𝑈𝐿 𝑡𝑝
![Page 24: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/24.jpg)
24 Piero Baraldi
Reservoir Computing (RC): Basic Idea 24
What is it? Purpose
ReservoirNon-linear temporal
expansion function
Expand the input hystory 𝒖 1: 𝑡𝑝 into a rich-enough
reservoir space 𝒙(𝑡𝑝)
readout Linear function
Combine the neuron signals 𝒙(𝑡𝑝) into the desired output
signal target 𝑅𝑈𝐿 𝑡𝑝
Reservoir and readout
serve different
purposes
They can be
separately
trained
![Page 25: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/25.jpg)
25 Piero Baraldi
Reservoir Methods
• Echo State Networks
• Liquid State Machines
• Evolino
• Backpropagation-Decorrelation
• Temporal Recurrent Networks
• …
25
![Page 26: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/26.jpg)
26 Piero Baraldi
In this presentaton
• Prognostics
• Recurrent Neural Network (RNN)
• Reservoir Computing
• Echo State Network
• Application to the Prediction of Turbofan Engine RUL
26
![Page 27: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/27.jpg)
27
Generate the reservoir
• Purpose: obtain a rich enough reservoir space 𝒙(𝑛)
• Recipe:
➢ Big reservoir (𝑀 up to 104) rich enough reservoir space
➢ Sparsely connected 𝑊 is sparse (no more than 20% of possible
connections)
➢ Randomly connected weights of the connections are randomly
generated from a uniform distribution symmetric around the zero value
...
...
𝑾
![Page 28: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/28.jpg)
28
Readout
• Purpose: learn the weights 𝑊𝑜𝑢𝑡 which minimize:
𝐸 𝑅𝑈𝐿, 𝑅𝑈𝐿𝐺𝑇 =RMSE=
𝑡=1
𝑡𝑓−11
𝑡𝑓 − 1𝑊𝑜𝑢𝑡𝒙(𝑡) − 𝑅𝑈𝐿𝐺𝑇(𝑡) 2
𝑾𝒊𝒏
𝑾𝒐𝒖𝒕
𝑾
(𝑡𝑝)
𝑢1(𝑡)
𝑢2(𝑡)
𝑢𝑁(𝑡)
𝑅𝑈𝐿 𝑡 = 𝑊𝑜𝑢𝑡𝑥(𝑡)
Readout𝑊𝑜𝑢𝑡𝒙 𝑡 = 𝑅𝑈𝐿𝐺𝑇(𝑡)
Linear regression
![Page 29: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/29.jpg)
29
Traditional RNN ESN
𝑾𝒊𝒏 𝑾𝒐𝒖𝒕
𝑾
-
Training: Traditional RNN VS ESN 29
error
𝑾𝒊𝒏
𝑾𝒐𝒖𝒕
𝑾
-
𝑅𝑈𝐿𝐺𝑇
𝑅𝑈𝐿
error
𝑅𝑈𝐿𝐺𝑇
𝑅𝑈𝐿
random
𝒖𝒖
![Page 30: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/30.jpg)
30
The Echo State Property
• The effect of 𝒙 𝑡 and 𝒖 𝑡 on a future state 𝒙 𝑡 + 𝑘 should vanish
gradually as time passes (i.e., 𝑘 → ∞ ) and not persist or even get
amplified.
• For most practical purposes:
𝜌 𝑊 : spectral radius of 𝑊 = largest absolute eigenvalue of W < 1
Echo State Property is satisfied
![Page 31: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/31.jpg)
31
In this presentaton
• Prognostics:
• Recurrent Neural Networks (RNN)
• Reservoir Computing
• Echo State Network
• Application to the prediction of Turbofan Engine RULs
31
![Page 32: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/32.jpg)
32
Prognostics: What is the Problem?
Aircraft Turbofan Engine
N Monitored Signals
• Signal 1
• Signal 2
• ………..
• Signal N
Aircraft Engine
RUL Prediction
TimeR
UL
Prognostic
Model
Te
mp
era
ture
Time
![Page 33: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/33.jpg)
33
• 260 run-to-failure trajectories
• 21 measured signals + 3 signals representative of the operating
conditions
• 6 different operating conditions
Data
Preprocessing**
The C-MAPPS dataset*
* A. Saxena, K. Goebel, D. Simon, N. Eklund, Damage propagation modeling for aircraft engine run-to-failure simulation,
PHM2008
**M. Rigamonti, P. Baraldi, E. Zio, I. Roychoudhury, K. Goebel, S. Poll, Echo State Network for Remaining Useful Life Prediction
of a Turbofan Engine, PHM 2016, Bilbao
![Page 34: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/34.jpg)
34
ESN Architecture Optimization
Network Architecture Optimization: Parameters
1) Network Dimensions
2) Spectral Radius
3) Connectivity
4) Input Scaling
5) Output Scaling
RUL(t)
![Page 35: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/35.jpg)
35
Network Architecture Optimization: Parameters
ESN Architecture Optimization
RUL(t)
1) Network Dimensions
2) Spectral Radius
3) Connectivity
4) Input Scaling
5) Output Scaling
![Page 36: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/36.jpg)
36
RUL(t)
Network Architecture Optimization: Parameters
1) Network Dimensions
2) Spectral Radius
3) Connectivity
4) Input Scaling/Shifting
5) Output Scaling/Shifting
6) Output Feedback
ESN Architecture Optimization
![Page 37: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/37.jpg)
37
RUL(t)
Network Architecture Optimization: Parameters
1) Network Dimensions
2) Spectral Radius
3) Connectivity
4) Input Scaling
5) Output Scaling
ESN Architecture Optimization
![Page 38: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/38.jpg)
38
RUL(t)
Network Architecture Optimization: Parameters
1) Network Dimensions
2) Spectral Radius
3) Connectivity
4) Input Scaling
5) Input Shifting
ESN Architecture Optimization
![Page 39: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/39.jpg)
39
RUL(t)
Network Architecture Optimization: Parameters
1) Network Dimensions
2) Spectral Radius
3) Connectivity
4) Input Scaling
5) Input Shifting
ESN Architecture Optimization
Sigmoidal Activation Function
![Page 40: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/40.jpg)
40
ESN Architecture Optimization
• Optimization Algorithm
• Population-based:
• Evolutionary-based
CHROMOSOME
Network
DimensionsConnectivity
Spectral
Radius
Input
Scaling
Inout
Shifting
Initialization Mutation Crossover Selection
Objective function: 𝑅𝐴 =σ 𝑅𝑈𝐿𝐺𝑇−𝑅𝑈𝐿
𝑅𝑈𝐿𝐺𝑇
• Experience + trial & errors difficult, good performance not guaranteed
• Differential evolution
![Page 41: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/41.jpg)
41
Optimal Architecture
Network Architecture Optimization: Parameters
1) Network Dimensions
2) Spectral Radius
3) Connectivity
4) Input Scaling
5) Input Shifting
RUL(t)
Network
DimensionsConnectivity
Spectral
Radius
Input
Scaling
Output
Scaling
385 0.17 0.67 0.45 -0.05
![Page 42: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/42.jpg)
42
ESN for Prognostics: Results (I)
0 20 40 60 80 100 1200
20
40
60
80
100
120
Time (Cycle)
RU
L (
Cyc
le)
RUL Prediction for Tansient 157
True RUL
ESN
FS
ELM
ESN = Echo State Network
FS = Fuzzy Similarity-based Prognosti Method
ELM = Extreme Learning Machine
![Page 43: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/43.jpg)
43
43
Cumulative Relative Accuracy Alpha-Lambda
𝜶 = 𝟎. 𝟐
Steadiness
Extreme
Learning
Machine
0.42 ± 0.03 0.31 ± 0.04 15.3 ± 2.2
Fuzzy
Similarity-based
Method
0.48 ± 0.04 0.34 ± 0.04 12.7 ± 0.7
Echo State
Network 0.37 ± 0.03 0.38 ± 0.04 12.4 ± 1.2
➢ Results – Prognostic Metrics (70 test trajectories)
RUL
RULLURRA
GT
ˆ,)var( :)( tttt TSI
ESN for Prognostics: Results (II)
![Page 44: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/44.jpg)
44
Conclusions
Recurrent Neural Network
Training: Reservoir Computing
Echo State Network
• Accurate RUL prediction
• Short Training Time
• Able to catch the system dynamics
Time
RU
L
Dynamic problem
![Page 45: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/45.jpg)
45
Acknowledgments
• Dr. Sameer Al-Dahidi
• Francesco Cannarile
• Dr. Michele Compare
• Dr. Francesco Di Maio
• Dr. Marco Rigamonti
• Mingjing Xu
• Zhe Yang
• Prof. Enrico Zio
45
![Page 46: Reservoir Computing Methods for Prognostics and Health ... · Reservoir Computing Methods for Prognostics and Health Management (PHM) Piero Baraldi Energy Department Politecnico di](https://reader033.fdocuments.us/reader033/viewer/2022053014/5f115c7b4c9d1b16411fb089/html5/thumbnails/46.jpg)
46
Thank
You!