Wind turbine fault prediction using soft label...
Transcript of Wind turbine fault prediction using soft label...
![Page 1: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/1.jpg)
Wind Turbine Fault Prediction
Using Soft Label SVM
Rui Zhao Md Ridwan Al Iqbal Kristin P. Bennett Qiang Ji
Contact: [email protected]
![Page 2: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/2.jpg)
Introduction: Motivation
Increasing growth of wind energy consumption
High cost of maintenance and repair
Significant lost of when turbine is forced out
[1]
![Page 3: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/3.jpg)
Diavik Diamond
Mine, Canada [1]
![Page 4: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/4.jpg)
Introduction: Background
Forced outage: turbine shutdown due to
unexpected internal fault of the system
Three main categories of prognostics strategies [3]
Physical modeling based approach
Signal analysis based approach
Machine learning based approach
Different components
of a wind turbine [2]
![Page 5: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/5.jpg)
Introduction: Challenges
The developing process of fault is often
unknown – No exact label information
The signature of fault is often unknown –
No single indicative feature
Different turbines may have different
symptoms and causes of fault – Significant
heterogeneity
![Page 6: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/6.jpg)
Problem Statement: Goal
Notation:
𝑥𝑡 ∈ 𝑅𝑑 is 𝑑-dimensional feature vector at time 𝑡
𝐱𝑛𝑙 = {𝑥𝑛, … , 𝑥𝑛+𝑙−1} is a subsequence of time series
𝑦𝑛𝑙 is binary hidden label of subsequence
-1: Normal
+1: Pre-fault
Goal: learn a mapping 𝑓: 𝑦𝑛𝑙 = sign(𝑓 𝐱𝑛
𝑙 )
Major challenge: Classification without exact labels
![Page 7: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/7.jpg)
Problem Statement:
Assumptions
Exploit the uncertainty of label information
Empirical observations: The closer the time to the
forced outage, the more likely the turbine is at pre-
fault status.
Assumptions:
Non-decreasing probability of being in pre-fault
status as time approaches the forced outage
event.
𝑃 𝑦𝑛𝑙 = 1 𝐱𝑛
𝑙 ≥ 𝑃 𝑦𝑚𝑙 = 1 𝐱𝑚
𝑙 , ∀ 𝑛 > 𝑚
For testing purpose, assume
𝑦𝑛𝑙 =−1, 𝑛 < 𝑛−
1, 𝑛 > 𝑛+
![Page 8: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/8.jpg)
Methods: SVM
Support Vector Machines (SVM) [4] as base framework
Parameterization: 𝑦𝑛 = sign 𝑓(𝐱𝑛) ≡ sign(𝐰𝑇𝜙 𝐱𝑛 + 𝑏)
Training data: 𝑦𝑛, 𝐱𝑛 𝑛=1𝑁
Model parameters: 𝐰, 𝑏
Primal problem:
min𝐰,𝑏
1
2𝐰2+ 𝛾
𝑛=1
𝑁
𝜉𝑛
subject to 𝑦𝑛 𝐰𝑇𝜙 𝐱𝑛 + 𝑏 ≥ 1 − 𝜉𝑛,
𝜉𝑛≥ 0, 𝑛 = 1,… ,𝑁
Requires fully supervision supplied with label 𝑦𝑛
(SVM-P)
![Page 9: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/9.jpg)
Methods: Our Approach
Soft Label Support Vector Machines (SLSVM)
Same parameterization and model parameters as SVM
𝑦𝑛 = sign 𝑓(𝐱𝑛) ≡ sign(𝐰𝑇𝜙 𝐱𝑛 + 𝑏)
Training data: 𝑢𝑛+, 𝑢𝑛−, 𝐱𝑛 𝑛=1
𝑁
𝑃 𝑦𝑛 = 1 𝐱𝑛 = 𝑢𝑛+
𝑃 𝑦𝑛 = −1 𝐱𝑛 = 𝑢𝑛−
𝑢𝑛+ + 𝑢𝑛
− = 1, 𝑛 = 1,… ,𝑁
Primal problem:
min𝐰,𝑏
1
2𝐰2+ 𝛾
𝑛=1
𝑁
(𝑢𝑛+𝜉𝑛+ + 𝑢𝑛
−𝜉𝑛−)
subject to 𝐰𝑇𝜙 𝐱𝑛 + 𝑏 ≥ 1 − 𝜉𝑛+,
−𝐰𝑇𝜙 𝐱𝑛 − 𝑏 ≥ 1 − 𝜉𝑛−
𝜉𝑛+≥ 0, 𝜉𝑛
− ≥ 0, 𝑛 = 1,… , 𝑁
(SLSVM-P)
![Page 10: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/10.jpg)
Methods: Comparison
Illustration of slack variables
SVM SLSVM
𝑦 = −1
𝑦 = 0
𝑦 = 1𝜉 = 0
𝜉 < 1 𝜉 < 1
𝜉 > 1
𝜉 = 0
𝑦 = −1
𝑦 = 0
𝑦 = 1
𝜉+ < 1𝜉− > 1
𝜉+ > 1𝜉− < 1
𝜉+ > 1𝜉− < 1
𝜉+ = 0𝜉− > 1
𝜉+ > 1𝜉− = 0
![Page 11: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/11.jpg)
Methods: SLSVM
Lagrangian Dual problem
max𝜶𝒏+,𝜶𝒏−
𝑛=1
𝑁
(𝛼𝑛+ + 𝛼𝑛
−) −1
2
𝑚=1
𝑁
𝑛=1
𝑁
𝛼𝑚+ − 𝛼𝑚
− 𝛼𝑛+ − 𝛼𝑛
− 𝑘(𝐱𝑚, 𝐱𝑛)
subject to 0 ≤ 𝛼𝑛+ ≤ 𝛾𝑢𝑛
+, 0 ≤ 𝛼𝑛− ≤ 𝛾𝑢𝑛
−
𝑛=1𝑁 (𝛼𝑛
+ − 𝛼𝑛−) = 0, 𝑛 = 1, … , 𝑁
Where 𝑘 𝐱𝑚, 𝐱𝑛 = 𝜙 𝐱𝑚𝑇𝜙(𝐱𝑛) is the kernel function
(SLSVM-D)
![Page 12: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/12.jpg)
Methods: Solution
(SLSVM-D) can be solved by quadratic
programming [5]
From KKT condition, we can recover 𝐰, 𝑏
Given new data 𝐱, we can evaluate its score
𝑓 𝐱 = 𝐰𝑇𝜙 𝐱 + 𝑏 =
𝑛=1
𝑁
𝛼𝑛+ − 𝛼𝑛
− 𝑘 𝐱, 𝐱𝑛 + 𝑏
![Page 13: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/13.jpg)
Methods: Generalized
Formulation
Generalized primal problem
min𝐰,𝑏
1
2𝐰𝑝+ 𝛾
𝑐=1
2
𝑛=1
𝑁
𝑢𝑛𝑐𝐸[𝑓 𝐱𝑛 , 𝑦𝑐]
𝑝 > 0 : order of regularization
𝑢𝑛𝑐 = 𝑃 𝑦𝑛 = 𝑦𝑐 𝐱𝑛 , 𝑦𝑐 = −1𝑐 : soft label
𝐸(𝑓 𝐱𝑛 , 𝑦𝑐) : loss function, e.g.
Hinge loss: 𝐸 = max (0,1 − 𝑦𝑐𝑓(𝐱𝑛))
Squared hinge loss: 𝐸 = max 0,1 − 𝑦𝑐𝑓 𝐱𝑛2
Squared loss: 𝐸 = 1 − 𝑦𝑐𝑓 𝐱𝑛2
Optimization: ADMM [6] for primal problem and quadratic
programming for dual problem
(SLSVM-PG)
![Page 14: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/14.jpg)
Experiment – Data
Source: 55 channels of sensor data collected
from 125 wind turbines (GE 1.6 MW)
Granularity: data are sampled at sub-second level and averaged over 10 minutes period.
Other facts:
Time span: June 2013 to May 2014
Number of forced outages: 38
Total down time: 2350 hours
Fault related component: electrical subsystem
![Page 15: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/15.jpg)
Experiment – Process
sensor
series
SLSVM
trainingClassification
Data pre-
processing
Feature
extraction
AUC,
feature
rank
Empirical
probability
assignment
Output: Input:
Locate
forced
outages
1. Data cleaning
2. Channel pruning
![Page 16: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/16.jpg)
Experiment – Process
1. Customized normalization
2. Spatial feature: covariance
3. Temporal feature: autoregressive model coefficients
sensor
series
SLSVM
trainingClassification
Data pre-
processingFeature
extraction
AUC,
feature
rank
Empirical
probability
assignment
Output: Input:
Locate
forced
outages
![Page 17: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/17.jpg)
Experiment – Process
1. Logged 38 forced outages events
2. Truncate up to 12 days prior to event
sensor
series
SLSVM
trainingClassification
Data pre-
processing
Feature
extraction
AUC,
feature
rank
Empirical
probability
assignment
Output: Input:
Locate
forced
outages
![Page 18: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/18.jpg)
Experiment – Process
1. Linear
2. Sigmoid
3. Exponential
sensor
series
SLSVM
trainingClassification
Data pre-
processing
Feature
extraction
AUC,
feature
rank
Empirical
probability
assignment
Output: Input:
Locate
forced
outages
![Page 19: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/19.jpg)
Experiment – Process
3 different loss functions 𝐸2 different values of regularization order 𝑝Linear kernel is used
sensor
series
SLSVM
trainingClassification
Data pre-
processing
Feature
extraction
AUC,
feature
rank
Empirical
probability
assignment
Output: Input:
Locate
forced
outages
![Page 20: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/20.jpg)
Experiment – Process
Based on assumed
groundtruth labels
sensor
series
SLSVM
trainingClassification
Data pre-
processing
Feature
extraction
AUC,
feature
rank
Empirical
probability
assignment
Output: Input:
Locate
forced
outages
![Page 21: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/21.jpg)
Experiment – Soft Labels
![Page 22: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/22.jpg)
Experiment – Process
sensor
series
SLSVM
trainingClassification
Data pre-
processing
Feature
extraction
AUC,
feature
rank
Empirical
probability
assignment
Output: Input:
Locate
forced
outages
![Page 23: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/23.jpg)
Results – Classification
Prediction quality
All soft label approach outperforms hard label approach
(standard SVM) with exponential soft label performs the best
![Page 24: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/24.jpg)
Results – Classification
Prediction quality
All soft label approach outperforms hard label approach
(standard SVM) with exponential soft label performs the best
Extension for feature selection is considered
![Page 25: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/25.jpg)
Results – Classification
Prediction quality
All soft label approach outperforms hard label approach
(standard SVM) with exponential soft label performs the best
Extension for feature selection is considered
Simple clustering method performs the worst
![Page 26: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/26.jpg)
Results – Classification
Prediction horizon
Varies the prediction horizon and subsequence length
18 hours ahead achieves the highest average AUC value 0.91
![Page 27: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/27.jpg)
Results – Feature selection
L1-norm regularized formulation for feature selection
![Page 28: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/28.jpg)
Summary
Formalize a time-series classification problem for wind
turbine prognostics
Proposed a classification framework SLSVM that can
handle uncertainty of label information
Extend SLSVM with feature selection capability to
provide insight for prognostics
Demonstrate the effectives of SLSVM for fault
prediction on real turbine operation data
![Page 29: Wind turbine fault prediction using soft label SVMcvrl/zhaor/files/Presentation_ICPR2016_2.pdfSupport Vector Machines (SVM) [4] as base framework ... Results –Classification Prediction](https://reader033.fdocuments.us/reader033/viewer/2022042313/5edcbda0ad6a402d66678998/html5/thumbnails/29.jpg)
Thank you
References
[1] Global Energy Council, Global Wind Energy Report, 2016
[2] M. Schlechtingena, I. F. Santosb and S. Achichec, Wind turbine condition monitoring based on SCADA data using normal behavior models, Applied Soft Computing, 2013
[3] Z. Hameed, Y. Hong, Y. Cho, S. Ahn, and C. Song, Condition monitoring and fault detection of wind turbines and related algorithms: A review, Renewable and Sustainable Energy Reviews, 2009
[4] B. E. Boser, I. M. Guyon, and V. N. Vapnik, A training algorithm for optimal margin classifier, ACM annual workshop on Computational Learning theory, 1992
[5]S. Boyd and L. Vandenberghe, Convex Programming, Cambridge University Press, 2004
[6] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers, Foundations and Trends in Machine Learning, 2011
Contact: [email protected]
Q&A