Anomalies...- Key performance indicators (KPI) data determines which base stations are down at a...
Transcript of Anomalies...- Key performance indicators (KPI) data determines which base stations are down at a...
KPI raw data(5,855,201 rows)
CDR raw data(9,232,275 rows)
Twitter API(44,989 tweets)
Keras: Open-source neural network library.The deep learning engineto train the model.
Bert: Pre-training language model.The embedding layer.
VaderSentimentSentiment analysisalgorithm for socialmedia content.The sentiment analysistool.
PostgreSQL:Open-source relationaldatabase managementsystem.Storing data in psql table.
Grafana: Open sourcemetric analytics &visualization suite.Visualize the anomaly results andtwitter sentiment analysis result.
Docker: A tool forbuilding and runningdistributed applications.Application deployment anenvironment version control.
Bert Embedding + LSTM87.1%
Keras Embedding + LSTM51.33%
Twitter APILive streaming data
VaderSentiment
Bert Embedding
Customer Issue Content(CSV file, 3 labels, 42,267 rows)
KPI STD data
LSTM
Reconstruction errorcalculation
Dimension Reduction AutoEncoder200 39 36 24 36 39
Data Anomalies
Output Shape39 Dimensions
Output Shape200 Dimensions
Input Shape39 Dimensions
Anomalies
As telecommunication companies seek to improve network coverage andmaintain customer satisfaction, they need to constantly monitorcustomer complaints and the status of their networks.This project, in collaboration with Tupl Inc. seeks to reduce the latencybetween the time a network coverage area experiences issues and whenthe network operator notices these issues. Deep learning models processseveral types of telecom data to predict which data are anomalies. - Call detail records (CDR) data from customer complaints determines if customers have recurrent cellular issues. - Key performance indicators (KPI) data determines which base stations are down at a given time. - Twitter tweets identify customers’ dissatisfaction on social media that isn’t directly reported to the company.
Sentiment Trends
Tweets locationwith sentiment value
SentimentDistribution
Issue labelDistribution
Detail of tweets
Anomaly station on topology map
Twitter Sentiment Analysis
Anomaly & Standard Deviation Heatmap
Anomalies
LSTM Layer(256 units)
DenseLayer(100 units, relu)
DenseLayer(3 units, softmax)
dropout = 0.1dropout = 0.1
CDR raw data(41 features)
2/3G data
4G VoLTE data
KPI-Like Data(5 features)
KPI-Like Data(8 features)
x1
x2
x3
x4
x5
x6
x7
x1
x2
x3
x4
x5
x6
x7
a1a1
a2
a3
a4
a5
a2
a3
a4
a5
a1
a2
a3
Reconstructionerror
2/3G Anomaly Result
4G Anomaly Result
Input Layer1432
Hidden Layer256
Output Layer1432716 716
Find Anomalies
Otsu ThresholdEncoder Decoder
Training Data
Input Shape(30, 768)
ClassificationResult
Telcom related tweet
Tweet with sentiment value
Comparing the Upload Traffic Volume, we plotted the input data in blue vs.the prediction data from our model in orange. We calculated the reconstruction error between the two datasets anddetermined the anomaly threshold for this feature to identify the anomalies.
Index of Site ID
IMSF
aultC
odeR
ate
PredictionIMSFaultCodeRate