Negative Selection for Algorithm for Anomaly Detection
-
Upload
xavier-llora -
Category
Technology
-
view
3.990 -
download
3
description
Transcript of Negative Selection for Algorithm for Anomaly Detection
![Page 1: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/1.jpg)
Professor, Department of Computer Science
![Page 2: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/2.jpg)
D. Dasgupta 2
Role of Biological Immune System (BIS)
• Its primary role is to distinguish the host (body cells) from external entities (pathogens).
• When an entity is recognized as non-self (or dangerous) - activates several defense mechanisms leading to its destruction (or neutralization).
• Subsequent exposure to similar entity results in rapid immune response (Secondary Response).
• Overall behavior of the immune system is an emergent property of many local interactions.
![Page 3: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/3.jpg)
D. Dasgupta 3
An abstract view of BIS:
![Page 4: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/4.jpg)
D. Dasgupta 4
Multi-Level Detection
![Page 5: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/5.jpg)
D. Dasgupta 5
From the computational point of view, the immune system is a
• Distributed information processing system• Novel pattern recognizer: Self/non-self
(Danger) Discrimination • Multi-level Self regulated Defense System• Having unique mechanisms for
– Decentralized control– Signaling and Message-passing– Co-stimulation– Learning and memory– Diversity
![Page 6: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/6.jpg)
D. Dasgupta 6
Computational Models & Algorithms
• Immune Network Models ( Jerne’74)• Negative Selection Algorithms (Forrest’94)
• Immune Gene Libraries (Hightower’90)• Associative Memory (Gilbert’94, Smith’96)• Artificial Immune Systems (Hunt’95, Timmis’97)• Immune Agent Architecture (Mori’98, Dasgupta’99)
• Artificial Germinal Centers (Dasgupta’ 02)• Other Models (Farmer’86, Bersini’90,Varela,’91,
etc.)
![Page 7: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/7.jpg)
D. Dasgupta 7
Artificial Immune Systems (AIS)
- Function optimizationSearch, optimizationClonal selection(Clonalg, aiNet)
- SecAgent architectures- Decentralized robot control
Distributed processing
Cell Mobility (ImmAg)
- Classification- Clustering- Data analysis- Stream data-mining
Learning (supervised and unsupervised)
Immune Networks(AINE,RLAIS,AIRS,FuzzyAIS)
- Computer security- Fault detection
Anomaly or change detection
Self/non-self recognition (NSA)
Typical ApplicationsComputational Problem
ImmunologicalAspect
![Page 8: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/8.jpg)
Negative Selection Algorithm (NSA)( Forrest ‘94)
An algorithm for change detection based on the principles of self-nonself discrimination (by T Cell receptors) in the immune system. The receptors can detect antigens.
Partition of the Universe of Antigens
SNS:self and nonself (a and b)
![Page 9: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/9.jpg)
D. Dasgupta 9
Illustration of NS Algorithm:
Match10111000
Don’t Match10111101
r=2Selfstrings (S)
Generaterandom strings
(R0)Match Detector
Set (R)
Reject
No
Yes
For binary representation:• There exists efficient BNS algorithm that runs on linear
time with the size of self (D’haeseleer’96).– Efficient algorithm to count number of holes.
– Theoretical analysis based on Information Theory.
![Page 10: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/10.jpg)
D. Dasgupta 10
Defining the Negative Selection Algorithm (NSA) :
• Define Self as a normal pattern of activity or stable behavior of a system/process – A collection of logically split segments (equal-size) of pattern
sequence. – Represent the collection as a multiset S of strings of length l
over a finite alphabet.• Generate a set R of detectors, each of which fails to
match any string in S.• Monitor new observations (of S) for changes by
continually testing the detectors matching against representatives of S. If any detector ever matches, a change ( or deviation) must have occurred in system behavior.
![Page 11: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/11.jpg)
NS Greedy Algorithm: (D’haeseleer’96)
It can generate a diverse set of detectors to provide better coverage in the non-self space. Particularly, instead of generating detectors randomly (in the second phase), the greedy algorithm chooses detectors that are far apart, in order to avoid possible overlapping of detectors and to provide enough coverage in the non-self space.
![Page 12: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/12.jpg)
D. Dasgupta 12
Partial Matching Rule(r-contiguous symbols)
X: ABCBBCDEABCE
Y: BCCBCCAEABAE
Choose a threshold (r):
match (X, Y) = T r ≤ 3
PM ≅ m-r [(l - r) (m-1) / m + 1]
m = size of alphabet
l = num of symbols in string
e.g.: strings of length l=30, matching length r=8010101001001110010001111110100111010101101110010100010011110
![Page 13: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/13.jpg)
D. Dasgupta 13
Anomaly Detection in Time Series• Dasgupta & Forrest (1996) on time series data, based on the previously
discussed negative-selection algorithm.
![Page 14: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/14.jpg)
Anomaly Detection ProcessAnomaly Detection Process
![Page 15: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/15.jpg)
D. Dasgupta 15
Analyzing the Expressiveness of Binary Matching Rules
• 2-dimensional Euclidean problem space
• NS with binary rules is applied
• The generated detectors are mapped back to the problem space
• Self set: a section of Mackey-Glass data set
![Page 16: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/16.jpg)
D. Dasgupta 16
Problem Space Representation
![Page 17: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/17.jpg)
D. Dasgupta 17
Generated Coverings
r-contiguous r-chunk Hamming
Bin
ary
Gra
y
1001010011010110
10010100**0101**
1 0 0 1 0 1 0 01 1 0 1 0 1 1 01+0+1+1+1+1+0+1=6
r = 9 r = 8 r = 12
![Page 18: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/18.jpg)
D. Dasgupta 18
Shape of Binary Matching Rules
r-contiguous r-chunk Hamming
Bin
ary
Gra
y
11000010101000111000000010000000
1001000010001000****00001000****
r = 4 r = 8 r = 8
110000101010001110000000100000001 1111 111 111 = 11
![Page 19: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/19.jpg)
D. Dasgupta 19
Coverings Generated by Different Values of r
r = 6 r = 7 r = 8 r = 9
![Page 20: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/20.jpg)
D. Dasgupta 20
Limitations of BMRs in NSABinary matching rules are not able to capture the semantics of some complex self/non-self spaces.It is not easy to extract meaningful domain knowledge.Scalability issues: In some cases, large number of detectors are needed to guarantee a good level of detection.It is difficult to integrate the NS algorithm with other immune algorithms.Crisp boundary of self and non-self may be hard to define
![Page 21: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/21.jpg)
D. Dasgupta 21
Advances in NSA:Developments in NSA
Hybrid ImmuneLearning Algorithm
New representation
New detector gene-ration algorithms
Non-crisp self/non-selfdistinction
Hyper-rectanglesCrisp If-Then rules
Fuzzy If-Then rules
Hyper-spheres
NSDR:- Seq Niching- Det. Crowding
NSFDR RNSRRNS
Multi-shaped
![Page 22: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/22.jpg)
D. Dasgupta 22
Real-Valued Self/Non-self Space Use of a multi-dimensional real
representation of the space:– Appropriate for diverse
applications
– Some geometrical properties of Rn that may speed up the negative selection
– It is easier to map the detectors back to the problem space
– Other AIS approaches use this kind of representation
Self
Non_Self
Self
X1
X
![Page 23: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/23.jpg)
D. Dasgupta 23
Evolving Fault detectors• Goal: to evolve 'good' fault indicators (detectors) in
the non-self (abnormal) space.• 'good' detector means:
– It must not cover self.
– It has to be as general as possible: the larger the volume, the better.
– Collectively provide maximum coverage of the non-self space with minimum overlap
• Some detectors serve as specialized (signature for known fault conditions) and others are for probable (or possible) faulty conditions.
![Page 24: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/24.jpg)
D. Dasgupta 24
RNS Algorithm: Flow Diagram
![Page 25: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/25.jpg)
D. Dasgupta 25
NS Rule Evolution: Different Levels of Deviation
• Define different levels of variability on the self set.
• Evolve detectors for the different levels.
Level 1
Level 2
Normal
Normal
Normal
![Page 26: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/26.jpg)
D. Dasgupta 26
A Heuristic Algorithm for Generating Hyper-spherical
Detectors (RNS)Self Data
Generate randompopulation of
detectors
Optimize detectordistribution
![Page 27: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/27.jpg)
D. Dasgupta 27
Generation of Detector using Genetic Algorithm
Self Data
Generate Initial
population
Choose two parents
and cross them
Replace closestparent if fitness
is better.
![Page 28: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/28.jpg)
D. Dasgupta 28
Multi-shaped detectors
![Page 29: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/29.jpg)
D. Dasgupta 29
Anomaly Detection Function
µself : Rn Range
Self
Non_Self
Self
X1
X0
Crisp
Non-crispdiscrete
Normal
Abnormal
Abnormal
Abnormal
Normal
Normal
Fuzzy
![Page 30: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/30.jpg)
D. Dasgupta 30
Immunity Based Fault Detection
Concept Illustration
Self
Non_Self
Self
F1
F3
F2
F4
![Page 31: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/31.jpg)
![Page 32: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/32.jpg)
D. Dasgupta 32
![Page 33: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/33.jpg)
0.36%0.43%95.6 %3Wing 3
0.47%1.04%91.8 %7Tail 1
0.26%0.76 %94.7 %9Tail 3
0.33%0.15 %97.8 %10Left Engine
False Alarm (std)
False Alarm (mean)Detection Rate (mean)
Activated Detectors
Fault Type
![Page 34: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/34.jpg)
D. Dasgupta 34
0.320.45False Alarm (Std)
0.98%0.87%False Alarm (mean)
1.671.43Detection rate (Std)
92%89%Detection rate (mean)
10882# of activated detectors
WingTailType of Fault
Testing of two different faults (Tail and wing failure)
![Page 35: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/35.jpg)
D. Dasgupta 35
Variable size Fault Detectors
![Page 36: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/36.jpg)
D. Dasgupta 36
Combining Negative Selection (NS) and Classification Techniques for Anomaly
Detection (Gonzalez’02)• The idea is to combine conventional classification
algorithms and Artificial Immune Systems techniques to perform anomaly detection.
– In many anomaly detection applications, only positive (normal) samples are available at the training stage.
– Conventional classification algorithms need positive and negative samples.
– The proposed approach uses the positive (normal) samples to generate negative samples that are used as training data for a neural network.
![Page 37: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/37.jpg)
D. Dasgupta 37
Generating Classifier dataset
![Page 38: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/38.jpg)
D. Dasgupta 38
Advantages of Negative Selection• From an information theory point of view,
characterization of the normal space is equivalent to the characterization of the abnormal space.
• Distributed detection: Different set of detectors can be distributed at different location
• Other possibilities– Generalized and specialized detectors
– Dynamic detector sets
– Detectors with specific features
– Artificial Fault signatures
– Data samples for classification techniques
![Page 39: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/39.jpg)
D. Dasgupta 39
MMultilevelultilevelIImmunemmune
LLearningearningAAlgorithmlgorithm
![Page 40: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/40.jpg)
D. Dasgupta 40MILA Algorithm Overview
![Page 41: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/41.jpg)
D. Dasgupta 41
MILA Algorithm Implementation: Basic Strategies
Shape-space model: e.g., Ag or Ab is represented as m = < m1, m2
…, mL>
Euclidean distance: calculate the degree of Ag-Ab interaction.
Partial matching rule:Ag
Abmatch
![Page 42: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/42.jpg)
D. Dasgupta 42
Algorithm Implementation: Basic Strategies
APC recognition: default
Th recognition: low-level
Ts recognition: suppression
B recognition: high-level
Cloning and mutation
o targeted (not blind) cloningo positive selection (higher affinity) and
negative selection (self tolerant)
![Page 43: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/43.jpg)
D. Dasgupta 43
Low Level Th recognition
a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, … ad
Peptide length = k = 4
![Page 44: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/44.jpg)
D. Dasgupta 44
High Level B recognition go back
aLa2a1
L, mL3, m21, m1
Ag
B
1 3 L
![Page 45: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/45.jpg)
D. Dasgupta 45
Mysterious Cell---- Ts cell
Ts exactly exists in body and suppresses immune response! Ts has specificity for special antigen.
Mechanism remains unknown
For the problem of anomaly detection, Tsdetector is regarded as a special self-detecting agent.
Initialization phase: Ts detector will be selected if it still matches the self-antigen under more stringent threshold.
Recognition phase: the response will be terminated when Ts detector matches a special antigen resembling self-data pattern.
![Page 46: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/46.jpg)
D. Dasgupta 46
Dynamic detector sets
1
2 34
5
Normal Sample Testing Sample
Dynamic Detector set
ROC 1
ROC 2ROC 3
ROC 4
ROC 5
![Page 47: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/47.jpg)
D. Dasgupta 47
New Features of MILA
Combines several immunological metaphors instead of implementing in a piecemeal manner. Uses multiple strategies for detection to make the system either very sensitive to any changes or robust to noise.Detector generation is problem dependent: different threshold parameters are available tuning the system performance
![Page 48: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/48.jpg)
D. Dasgupta 48
Detector set in MILA is dynamic whereas detector set in Negative Selection Algorithm remains constant once it is generated in training phase.
The cloning, mutation and selection after detect phase in MILA is actually a process of on-line learning and optimization. The process of cloning in MILA is a targeted (not blind) cloning. Only those detectors that are activated in recognition phase can be cloned.This strategy ensures that both the speed and accuracy of detection becomes successively higher after each detecting.
New Features of MILA (Cont..)
![Page 49: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/49.jpg)
D. Dasgupta 49
Summary
• AIS emerged in 1990s as a new paradigm in AI, and has earned its position on the map of soft computing
• Being used in many applications – anomaly detection, pattern recognition, data mining, computer security, adaptive control, fault detection
• The long-term usefulness of AIS methods still depend on – Uniqueness– Effectiveness
We need unified AIS architecture and/or algorithm
![Page 50: Negative Selection for Algorithm for Anomaly Detection](https://reader031.fdocuments.us/reader031/viewer/2022020217/549fde43ac795938768b4af5/html5/thumbnails/50.jpg)
For Information onArtificial Immune System
Related Events and
Bibliography
Visit the website
http://www.cs.memphis.edu/~dasgupta/AIS/
The University of Memphis